Description
Symfony version(s) affected
5.3.10
Description
Recently came up with very interesting bug or I would say a side affect of symfony http client using in a long running process:
I have a long running process which consumes async envelopes and does http requests to constant host (process might run up to couple of hours). At some point of time the host which we are requesting changed its IP address and therefore my consumer process started to fail to send http requests. After a brief investigation I have found out that symfony's http client has an dns cache in its base clients (CurlHttpClient, NativeHttpClient) and it only resets on class destruction.
I do understand why this dns cache is implemented on http client, but looks like it might be a bottleneck for a long running process.
I could help with a possible fix and PR preparation, but need more insights on this issues
How to reproduce
My environment:
-
Php version:
7.4.21
-
Curl version:
7.64.0
-
Symfony version:
5.3.10
-
http_client.yaml configuration:
framework:
http_client:
default_options:
retry_failed:
max_retries: 5
-
under the hood using
CurlHttpClient
-
all the services injecting http client by using
Symfony\Contracts\HttpClient\HttpClientInterface
typehint -
have a long running process which does http requests to same host using symfony http client
-
DNS changes host ip
-
the same long running process's requests start to fail (due to using old ip)
Possible Solution
- one of the solutions would be to reduce consumer lifetime so that it would restart resulting in refreshing app state, but from my perspective its just a workaround and not a future proof solution.
- another solution would be to add a
kernel.reset
tag forCurlHttpClient
and make sure that with every handled envelope theServicesResetter
would be called. I even tried to do it locally, but without luck.. What I tried:- On
vendor/symfony/framework-bundle/Resources/config/http_client.php
http_client
definition added->tag('kernel.reset', ['method' => 'reset'])
with a hope that it would be included toServicesResetter
, but as I'm having a retry configuration (see on reproduce tab) thehttp_client
service at some point is removed from container and replaced with decoratedhttp_client.retryable
(see screenshot in additional context) service and thishttp_client.retryable
gets kernel.rest tag, but unfortunately it does not work as it doesnt implement ResetInterface. If I remove retry configuration, then it works fine.
- On