[HttpClient] Upgrading from version 5.4.3 to version 5.4.5 #46823

TomLorenzi · 2022-07-01T12:43:17Z

Symfony version(s) affected

>5.4.5

Description

I just upraded my symfony/http-client package with composer using composer require symfony/http-client:5.4.5
(I first upgraded it to 5.4.9 and then I found my performance issue where coming from 5.4.3 to 5.4.5)
Since, I got huge performance issue doing some request, but it didn't seems to apply to every request.

The request I found my performance issues where GET request (can't be sure if it doesn't slow down POST request as well).

How to reproduce

Those request where really simple and here is a code sample :

$client = HttpClient::create();
$options = [
    "auth_bearer" => "bearer token",
    "body" => [
        "timeout" => 3600
    ]
];
$response = $this->client->request('GET', 'http://my.url/request', $options);

And when doing $response->getStatusCode() it was taking a huge amount of time compared to before my package upgrade

Possible Solution

Seems the difference between 5.4.3 and 5.4.5 it some default values but I can't tell how it can result in this

Additional Context

Absolutly client sided as the same request send with another tool like Postman didn't get any problems

The text was updated successfully, but these errors were encountered:

ghost · 2022-07-01T18:45:19Z

It's a very generic description and it is unlikely to be reproduced by anyone. It could be the server that was temporarily unhealthy or you connection could suffer. Go shopping and ride back to the home and try again.

Btw. Why do you update from 5.4.3 to 5.4.5 and not to the 5.4.9? These all are bigfix release so you should upgrade up to the lastes bugfix version for 5.4.* constraint.

TomLorenzi · 2022-07-02T16:45:33Z

(I first upgraded it to 5.4.9 and then I found my performance issue where coming from 5.4.3 to 5.4.5)

Was in my ticket, so I first did try to update to 5.4.9 and my problem appeared, then I tried to upgrade version one by one to find and the specific version since that bug appered.

I'm sure that's not about server or connection performance I had the problem on local dev, preprod server, and production server. Reported by me and other people from different locations.

xabbuh · 2022-07-03T10:42:36Z

Can you profile such a request with for example Blackfire with both versions? Then we could compare the profiles and maybe spot a potential issue.

TomLorenzi · 2022-07-04T10:52:46Z

Seems that i can't use Blackfire for free ? Unfortunatly 😢

Btw i edited my main post, as I'm sure the error is client sided and not from server as with a request sent from Postman or whatever to the same url I didn't get any performance issues

stof · 2022-07-04T11:17:32Z

yeah, they recently dropped their free plan.

xabbuh · 2022-07-12T15:12:37Z

Can you create a small example application that allows to reproduce your issue?

xabbuh · 2022-07-29T09:50:02Z

@TomLorenzi Do you have something that we could look into?

xabbuh · 2022-08-01T11:22:15Z

I am going to close here for now due to the lack of feedback. Please let us know when you have more information and we can consider to reopen.

nicolas-grekas · 2022-08-01T17:07:38Z

@TomLorenzi if you still have the issue, I would be interested in reading the output of calling $response->getInfo('debug') before and after the issue has been introduced. Note that this call should be done after reading the status code of the response.

TomLorenzi · 2022-08-01T18:21:39Z

Sorry for the delay, I didn't get the time to make a small demo project to show it.

Next week i'll get the info with debug and i'll post it here @nicolas-grekas

TomLorenzi · 2022-08-08T15:14:39Z

@nicolas-grekas Sorry for the delay but it seems I have some news

First, here is a configuration with 5.4.3 and then 5.4.9 (in my "name": "symfony/http-client" block) :

"dist": {
    "type": "zip",
    "url": "https://api.github.com/repos/symfony/http-client/zipball/a5a467b62dc91eb253db51a91a2c1977f611f60c",
    "reference": "a5a467b62dc91eb253db51a91a2c1977f611f60c",
    "shasum": ""
},

But this is slow :

"dist": {
    "type": "zip",
    "url": "https://api.github.com/repos/symfony/http-client/zipball/dc0b15e42b762c040761c1eb9ce86a55d47cf672",
    "reference": "dc0b15e42b762c040761c1eb9ce86a55d47cf672",
    "shasum": ""
},

Here is 5.4.3

* Found bundle for host databaseapi.local: 0x564760fbd6f0 [serially]
* Can not multiplex, even if we wanted to!
* Re-using existing connection! (#0) with host databaseapi.local
* Connected to databaseapi.local (127.0.0.1) port 80 (#0)
> GET /api/dam/format/list/ HTTP/1.1
Host: databaseapi.local
Accept: */*
Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiJ9.eyJpYXQiOjE2NTk5NzA5MzksImV4cCI6MTY2MDA1NTMzOSwicm9sZXMiOlsiUk9MRV9BRE1JTiIsIlJPTEVfVVNFUiJdLCJsb2dpb ▶
User-Agent: Symfony HttpClient/Curl
Accept-Encoding: gzip
Content-Length: 16
Content-Type: application/x-www-form-urlencoded
* upload completely sent off: 16 out of 16 bytes
* Mark bundle as not supporting multiuse
< HTTP/1.1 301 Moved Permanently
< Date: Mon, 08 Aug 2022 15:02:41 GMT
< Server: Apache/2.4.41 (Ubuntu)
< Vary: Authorization
< Location: http://databaseapi.local/api/dam/format/list
< X-Robots-Tag: noindex
< Content-Length: 422
< Content-Type: application/json
< 
* Ignoring the response-body
* Connection #0 to host databaseapi.local left intact
* Issue another request to this URL: 'http://databaseapi.local/api/dam/format/list'
* Switch from POST to GET
* Found bundle for host databaseapi.local: 0x564760fbd6f0 [serially]
* Can not multiplex, even if we wanted to!
* Re-using existing connection! (#0) with host databaseapi.local
* Connected to databaseapi.local (127.0.0.1) port 80 (#0)
> GET /api/dam/format/list HTTP/1.1
Host: databaseapi.local
Accept: */*
Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiJ9.eyJpYXQiOjE2NTk5NzA5MzksImV4cCI6MTY2MDA1NTMzOSwicm9sZXMiOlsiUk9MRV9BRE1JTiIsIlJPTEVfVVNFUiJdLCJsb2dpb ▶
User-Agent: Symfony HttpClient/Curl
Accept-Encoding: gzip
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Date: Mon, 08 Aug 2022 15:02:41 GMT
< Server: Apache/2.4.41 (Ubuntu)
< Vary: Authorization
< Cache-Control: no-cache, private
< X-Robots-Tag: noindex
< Content-Length: 387
< Content-Type: application/json
< 
* Connection #0 to host databaseapi.local left intact

With 5.4.9 being slow :

* Found bundle for host databaseapi.local: 0x564760fb3dc0 [serially]
* Can not multiplex, even if we wanted to!
* Re-using existing connection! (#0) with host databaseapi.local
* Connected to databaseapi.local (127.0.0.1) port 80 (#0)
> GET /api/dam/format/list/ HTTP/1.1
Host: databaseapi.local
Accept: */*
Content-Length: 16
Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiJ9.eyJpYXQiOjE2NTk5NzA5MzksImV4cCI6MTY2MDA1NTMzOSwicm9sZXMiOlsiUk9MRV9BRE1JTiIsIlJPTEVfVVNFUiJdLCJsb2dpb ▶
User-Agent: Symfony HttpClient/Curl
Accept-Encoding: gzip
Content-Type: application/x-www-form-urlencoded
* upload completely sent off: 16 out of 16 bytes
* Mark bundle as not supporting multiuse
< HTTP/1.1 301 Moved Permanently
< Date: Mon, 08 Aug 2022 15:06:18 GMT
< Server: Apache/2.4.41 (Ubuntu)
< Vary: Authorization
< Location: http://databaseapi.local/api/dam/format/list
< X-Robots-Tag: noindex
< Content-Length: 422
< Content-Type: application/json
< 
* Ignoring the response-body
* Connection #0 to host databaseapi.local left intact
* Issue another request to this URL: 'http://databaseapi.local/api/dam/format/list'
* Switch from POST to GET
* Found bundle for host databaseapi.local: 0x564760fb3dc0 [serially]
* Can not multiplex, even if we wanted to!
* Re-using existing connection! (#0) with host databaseapi.local
* Connected to databaseapi.local (127.0.0.1) port 80 (#0)
> GET /api/dam/format/list HTTP/1.1
Host: databaseapi.local
Accept: */*
Content-Length: 16
Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiJ9.eyJpYXQiOjE2NTk5NzA5MzksImV4cCI6MTY2MDA1NTMzOSwicm9sZXMiOlsiUk9MRV9BRE1JTiIsIlJPTEVfVVNFUiJdLCJsb2dpb ▶
User-Agent: Symfony HttpClient/Curl
Accept-Encoding: gzip
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Date: Mon, 08 Aug 2022 15:06:18 GMT
< Server: Apache/2.4.41 (Ubuntu)
< Vary: Authorization
< Cache-Control: no-cache, private
< X-Robots-Tag: noindex
< Content-Length: 387
< Connection: close
< Content-Type: application/json
< 
* Closing connection 0

Sorry this may not be the easiest thing to read...
This is by the way a local example but resulted the same on multiple servers and on local machines

TomLorenzi · 2022-12-15T09:50:34Z

May i bump this ?

nicolas-grekas · 2022-12-16T16:09:31Z

The diff between both is Content-Length: 16 sent after the redirection in the second dump.
We rely on curl removing this header usually.
Could this be a bug in curl? Which version are you using? (php --ri curl)
If not the latest, can you try with a more recent one?

TomLorenzi · 2022-12-21T11:44:24Z

Bug happened with cURL Information => 7.68.0 / 7.64.0 / 7.52.1

nicolas-grekas · 2022-12-22T10:24:04Z

Does it happen with more recent versions?

nicolas-grekas · 2023-01-12T16:00:08Z

Should be fixed by #48966. If you have the opportunity, please give it a try and report back.

…as-grekas) This PR was merged into the 5.4 branch. Discussion ---------- [HttpClient] Let curl handle content-length headers | Q | A | ------------- | --- | Branch? | 5.4 | Bug fix? | yes | New feature? | no | Deprecations? | no | Tickets | Fix #47973, #46499, #46823 | License | MIT | Doc PR | - I'm not sure why (maybe some older and buggy curl versions?) but it looks like curl might send content-length headers to proxies. This didn't happen before #45261 so I'm trying to use curl in the same was as we did by the time, aka letting it handle content-length header when possible. This is something that I already attempted in #45814 but was partly reverted in #45998. The attached patch is free from such issue. Commits ------- 5b27dc2 [HttpClient] Let curl handle content-length headers

TomLorenzi added the Bug label Jul 1, 2022

carsonbot added the Status: Needs Review label Jul 1, 2022

xabbuh added the HttpClient label Jul 3, 2022

xabbuh added the Status: Waiting feedback label Jul 3, 2022

OskarStark changed the title ~~[http-client] upgrading from version 5.4.3 to version 5.4.5~~ [HttpClient] Upgrading from version 5.4.3 to version 5.4.5 Jul 14, 2022

xabbuh mentioned this issue Jul 14, 2022

[HttpClient] TCP ports not closed #46937

Closed

xabbuh closed this as completed Aug 1, 2022

nicolas-grekas reopened this Aug 1, 2022

nicolas-grekas mentioned this issue Jan 12, 2023

[HttpClient] Let curl handle content-length headers #48966

Merged

nicolas-grekas closed this as completed Jan 13, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[HttpClient] Upgrading from version 5.4.3 to version 5.4.5 #46823

[HttpClient] Upgrading from version 5.4.3 to version 5.4.5 #46823

TomLorenzi commented Jul 1, 2022 •

edited

Loading

ghost commented Jul 1, 2022

TomLorenzi commented Jul 2, 2022 •

edited

Loading

xabbuh commented Jul 3, 2022

TomLorenzi commented Jul 4, 2022 •

edited

Loading

stof commented Jul 4, 2022

xabbuh commented Jul 12, 2022

xabbuh commented Jul 29, 2022

xabbuh commented Aug 1, 2022

nicolas-grekas commented Aug 1, 2022

TomLorenzi commented Aug 1, 2022

TomLorenzi commented Aug 8, 2022 •

edited

Loading

TomLorenzi commented Dec 15, 2022

nicolas-grekas commented Dec 16, 2022

TomLorenzi commented Dec 21, 2022

nicolas-grekas commented Dec 22, 2022

nicolas-grekas commented Jan 12, 2023

[HttpClient] Upgrading from version 5.4.3 to version 5.4.5 #46823

[HttpClient] Upgrading from version 5.4.3 to version 5.4.5 #46823

Comments

TomLorenzi commented Jul 1, 2022 • edited Loading

Symfony version(s) affected

Description

How to reproduce

Possible Solution

Additional Context

ghost commented Jul 1, 2022

TomLorenzi commented Jul 2, 2022 • edited Loading

xabbuh commented Jul 3, 2022

TomLorenzi commented Jul 4, 2022 • edited Loading

stof commented Jul 4, 2022

xabbuh commented Jul 12, 2022

xabbuh commented Jul 29, 2022

xabbuh commented Aug 1, 2022

nicolas-grekas commented Aug 1, 2022

TomLorenzi commented Aug 1, 2022

TomLorenzi commented Aug 8, 2022 • edited Loading

TomLorenzi commented Dec 15, 2022

nicolas-grekas commented Dec 16, 2022

TomLorenzi commented Dec 21, 2022

nicolas-grekas commented Dec 22, 2022

nicolas-grekas commented Jan 12, 2023

TomLorenzi commented Jul 1, 2022 •

edited

Loading

TomLorenzi commented Jul 2, 2022 •

edited

Loading

TomLorenzi commented Jul 4, 2022 •

edited

Loading

TomLorenzi commented Aug 8, 2022 •

edited

Loading