Thanks to visit codestin.com
Credit goes to github.com

Skip to content

[HttpClient] Gzipped data isn't inflated #34238

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
loevgaard opened this issue Nov 5, 2019 · 11 comments
Closed

[HttpClient] Gzipped data isn't inflated #34238

loevgaard opened this issue Nov 5, 2019 · 11 comments
Labels
Bug HttpClient Status: Needs Review

Comments

@loevgaard
Copy link
Contributor

Symfony version(s) affected: 4.3

Description
When I request json data from Microsofts Business Central, i.e. https://api.businesscentral.dynamics.com I get gzipped data back, but the Symfony HTTP client doesn't inflate it.

I can do something like this:

$response = $this->client->request('GET', $url);
$data = gzinflate($response->getContent());

and that works. My question is whether it is intended that I should do this myself or not? The headers in the response has this in them:

[content-encoding] => Array (
    [0] => gzip
)
@nicolas-grekas
Copy link
Member

can you dump($response->getInfo('debug')) after the call to getContent?

@stof
Copy link
Member

stof commented Nov 5, 2019

Are you using the curl client or the native client ?

@loevgaard
Copy link
Contributor Author

@stof the client injected into my command is Symfony\Component\HttpClient\ScopingHttpClient, but the client injected into that is Symfony\Component\HttpClient\CurlHttpClient.

Here is the dump, @nicolas-grekas:

"""
*   Trying 40.113.99.50:443...\n
* TCP_NODELAY set\n
* Connected to api.businesscentral.dynamics.com (40.113.99.50) port 443 (#0)\n
* ALPN, offering h2\n
* ALPN, offering http/1.1\n
* successfully set certificate verify locations:\n
*   CAfile: /usr/local/etc/[email protected]/cert.pem\n
  CApath: /usr/local/etc/[email protected]/certs\n
* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384\n
* ALPN, server accepted to use h2\n
* Server certificate:\n
*  subject: CN=api.businesscentral.dynamics.com\n
*  start date: May 23 12:16:17 2018 GMT\n
*  expire date: May 23 12:16:17 2020 GMT\n
*  subjectAltName: host "api.businesscentral.dynamics.com" matched cert's "api.businesscentral.dynamics.com"\n
*  issuer: C=US; ST=Washington; L=Redmond; O=Microsoft Corporation; OU=Microsoft IT; CN=Microsoft IT TLS CA 1\n
*  SSL certificate verify ok.\n
* Using HTTP2, server supports multi-use\n
* Connection state changed (HTTP/2 confirmed)\n
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0\n
* Using Stream ID: 1 (easy handle 0x7fbdda00f600)\n
> GET /v1.0/Ehandel9000.onmicrosoft.com/sandbox/ODataV4/Company('EHANDEL%209000%20ApS')/workflowSalesDocuments HTTP/2\r\n
Host: api.businesscentral.dynamics.com\r\n
User-Agent: Symfony HttpClient/Curl\r\n
Accept-Encoding: deflate, gzip, br\r\n
Authorization: Basic REMOVED\r\n
\r\n
* Connection state changed (MAX_CONCURRENT_STREAMS == 100)!\n
< HTTP/2 200 \r\n
< content-length: 2944\r\n
< content-type: application/json; odata.metadata=minimal; odata.streaming=true\r\n
< content-encoding: gzip\r\n
< server: Microsoft-HTTPAPI/2.0\r\n
< access-control-allow-headers: Origin, X-Requested-With, Authorization\r\n
< access-control-allow-origin: *\r\n
< access-control-allow-credentials: true\r\n
< ms-correlation-x: 2d8f9cdf-142a-4c1c-85e6-62b795c67bb9\r\n
< odata-version: 4.0\r\n
< request-id: c5c96b38-cc08-4200-b5a3-0a4559f9b232\r\n
< strict-transport-security: max-age=31536000; includeSubDomains\r\n
< x-content-type-options: nosniff\r\n
< access-control-expose-headers: Date\r\n
< access-control-expose-headers: Content-Length\r\n
< access-control-expose-headers: Server\r\n
< access-control-expose-headers: OData-Version\r\n
< access-control-expose-headers: ms-correlation-x\r\n
< date: Wed, 06 Nov 2019 08:21:35 GMT\r\n
< \r\n
* Connection #0 to host api.businesscentral.dynamics.com left intact\n
"""

Thank you very much for the quick reply :)

@nicolas-grekas
Copy link
Member

OK thanks.
The reason, which is just a supposition for now but I could bet on it, is that you explicitly send the request with the "accept-encoding" header. When you do this, you disable transparent compression and you opt-in for explicit compression, which means it's now your job to effectively decompress.
Don't send the header if that's not what you want.

@loevgaard
Copy link
Contributor Author

I am not explicitly setting that header in my request

@loevgaard
Copy link
Contributor Author

Here is my framework config:

framework:
    http_client:
        scoped_clients:
            business_central.client:
                base_uri: '%env(APP_BUSINESS_CENTRAL_BASE_URI)%'
                auth_basic: '%env(APP_BUSINESS_CENTRAL_USERNAME)%:%env(APP_BUSINESS_CENTRAL_PASSWORD)%'

and here is all my code right now:

$response = $this->client->request('GET', "ODataV4/Company('EHANDEL%209000%20ApS')/workflowSalesDocuments");
$data = gzinflate($response->getContent());
dump($response->getInfo('debug'));

@nicolas-grekas
Copy link
Member

Strange, then I have no idea - could it be an issue with your build of curl?
Do you have the same issue when requesting https://repo.packagist.org/packages.json?
On my side it decompresses correctly this URL.

@loevgaard
Copy link
Contributor Author

I can retrieve that url with doing anything manually.

@nicolas-grekas
Copy link
Member

The issue: when the client sends Accept-Encoding: deflate, gzip, the server gzips, then deflates 🤦‍♂️
#34251 works around this faulty behavior by sending only "gzip" when doing transparent HTTP compression.
Until the patch is released, you can work around the double compression by explicitly sending accept-encoding: gzip, then using gzdecode($response->getContent()).

@loevgaard
Copy link
Contributor Author

Thank you very much, @nicolas-grekas!

@loevgaard
Copy link
Contributor Author

Just updated to symfony/http-client 4.3.7 and it just works. Thank you :)

@nicolas-grekas nicolas-grekas mentioned this issue Dec 26, 2019
Merged
@laurenzonneveld laurenzonneveld mentioned this issue Jul 21, 2022
Closed
@ezimuel ezimuel mentioned this issue Aug 3, 2022
Merged
@ramon-nijman ramon-nijman mentioned this issue Jun 1, 2023
Open
@ttskch ttskch mentioned this issue Oct 3, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Bug HttpClient Status: Needs Review
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@javiereguiluz @nicolas-grekas @stof @loevgaard @carsonbot