[HttpClient] Add a ConditionalHttpClient #30592

XuruDragon · 2019-03-18T10:22:23Z

Q	A
Branch?	master
Bug fix?	no
New feature?	yes
BC breaks?	no
Deprecations?	no
Tests pass?	yes
Fixed tickets	n/a
License	MIT
Doc PR

Adding a new client ConfitionalHttpClient to the new HttpClient component.

This class is intended to allow different options to be sent based on a regexp executed on the url of the request. In simple words : Auto-configure the default options based on the requested URL.
This class implement the HttpClientInterface interface
The first regexp that math the url win.
Its first parameter expects an object implementing the HttpClientInterface interface
The second parameter is an array, of default options to use when the regexp provided as key matches the requested URL, to be given to the HttpClientInterface request method.

Here is an example of how to use this class :

$regexpOptions = [
   '#^.*length-broken$#' => [ 'headers' => [
         'ConditionalHttpClient' => 'CurlHttpClient',
         'ConditionalHttpClient2' => 'url:length-broken',
   ]],
   '#^.*$#' => ['headers' => ['ConditionalHttpClient' => 'CurlHttpClient']],
]

$client = new ConditionalHttpClient(new CurlHttpClient(), $regexpOptions );

//Will result in a merge of the second $regexpOptions index value and
//the options given to $client->request()
$client->request('POST', 'http://anydomain/', [
    'headers' => [ 'Content-Type' => 'application/json'],
]);

//Only the first $regexpOptions index value will be use as we do not give
//extra options to the request call
$client->request('GET', 'http://anydomain/length-broken');

src/Symfony/Component/HttpClient/ConditionalHttpClient.php

src/Symfony/Component/HttpClient/Tests/CurlConditionalHttpClientTest.php

javiereguiluz · 2019-03-18T15:59:45Z

@XuruDragon I appreciate your contribution to Symfony ... but to be completely honest, I don't like the idea behind this proposal.

If your app is complex enough to have lots of different sets of requests, you can already configure multiple clients to use the most appropriate one. I can't see a common-enough use case for this feature.

@XuruDragon sorry if I'm being too honest 😅 .. and of course, thanks again for helping Symfony with your contributions!

XuruDragon · 2019-03-18T16:26:31Z

@XuruDragon I appreciate your contribution to Symfony ... but to be completely honest, I don't like the idea behind this proposal.

If your app is complex enough to have lots of different sets of requests, you can already configure multiple clients to use the most appropriate one. I can't see a common-enough use case for this feature.

@XuruDragon sorry if I'm being too honest .. and of course, thanks again for helping Symfony with your contributions!

Hi @javiereguiluz
There is no problem. Opinions are like tastes and colors, everyone has them and not necessarily identical. We can talk about this feature, this way we can improve it :)

I understand what you means.
Here is not really about configure multiple clients, but rather use the same client with different default options according to the url called.
After all, this is experimental, like the whole component.
But no one has a feature like this neither Guzzle or PHP or whatever you want.
It could really be usefull I think.

Imagine that you have to send a request to a server that, on the same domain name / url, has an API on "/api" and a website on "/land" that you want to explore (I do not know why, but suppose that is the case). But this API is in JSON and the website is not ...
Here, with this feature, you will only use one client (in fact, 2, the ConditionalHttpClient + the client given to the previous one).
But for all the URLs starting with "/api", the 'Content-Type: json' header was automatically added.

I do not know if I expressed myself well?

kunicmarko20 · 2019-03-18T16:32:45Z

Hi @XuruDragon,

Did you have a case where you needed something like this? I never had one and I really never had a case where I needed to access some external API and their Website.

Is just imagining an edge case like this enough to add this Client?

XuruDragon · 2019-03-18T16:40:43Z

Hi @XuruDragon,

Did you have a case where you needed something like this? I never had one and I really never had a case where I needed to access some external API and their Website.

Is just imagining an edge case like this enough to add this Client? Feels more like something that would end user do on its own.

Hi @kunicmarko20,
This was just an example, not very good Indeed, wasn't really inspired...
But think about maybe you want to communicate with 10 differents API, each one have a different authentication system (Oauth, JWT, headers, etc Whatever you want), and a lot of custom headers per API.
You just have to authenticate yourself against them, fill a default Options array with your custom headers, authentification credentials, etc... keyed this array with regexp to match url of each api.
Here you are ! You have Only one client to call all your APIs without manipulate X clients etc...

nicolas-grekas · 2019-03-18T16:43:53Z

You have Only one client to call all your API without manipulate Xclient etc...

💯 this - this also spans to security strengthening: when configuring a client with default credentials, nothing guarantees that the consumer will use the client with the expected target host/URL right now. This means that if you configure defaults with some bearer for foo.com but the consumer hits bar.com instead, the bearer is sent to bar.com.

javiereguiluz · 2019-03-18T16:49:26Z

But if you have a bearer for foo.com and not for bar.com, you make requests to bar.com with the default HTTP client and requests to foo.com with a client defined for it and with the bearer token defined as a config option.

What I mean is that we already have a solution for this problem: different clients with different config options. Here we'd introduce a slightly different mechanism: different options with the same client depending on the URL 😕

nicolas-grekas · 2019-03-18T16:52:24Z

@javiereguiluz that's putting a lot of responsibility on the shoulder of users - and how is dealing with one instance (like this PR allows) supposed to be simpler than juggling with two of them in userland?

Here, this client acts as a router: based on the URL, it decides which credentials/etc should be used.
That's actually much simpler to use than dealing with many clients. I would even consider removing the possibility to define several clients personally, in favor of defining those URL-based conditions.

XuruDragon · 2019-03-18T16:59:11Z

With a very basic comparaison :
I think we can see the client as a car, we will not change our car every time we change our destination, IMO it's the same for the client.

javiereguiluz · 2019-03-18T17:02:39Z

@XuruDragon but this is not how existing Symfony features work.

When you have different cache configs, you define 2 or more cache pools ... you don't define 1 pool with different configs based on a regexp applied to the cache key.

When you have different log configs, you define 2 or more log handlers ... you don't define 1 handler with different configs based on a regexp applied to the lgo channel or message.

The same happens everywhere: Messenger, Serializer, Lock, etc. We always define N "clients" for them, not 1 client with a regexp-based discriminator.

nicolas-grekas · 2019-03-18T17:07:56Z

@javiereguiluz here is the difference: log/cache/message buses all need several transports. HTTP doesn't: there is only one single WWW. Its routing layer is universal.

Having one single HTTP client with a router on top is much more aligned to the web, and also much more DX. (ie there's one client and management of credentials is centralized, consumers don't need to care).

oliverde8 · 2019-03-19T17:10:41Z

This has also the advantage for concurrent calls. I might be mistaken but there is a multi curl resource per client. Which means with 2 clients we can't get proper advantage of it.

Currently with Guzzle for example if we need to call 2 independent api's we will end up creating 2 api's until we decide to optimise the 2 calls. We had such an issue quite recently and ended up having to use a single client and it's not that pretty.

I think this idea is great and can be made even greater.

nicolas-grekas · 2019-03-19T17:12:38Z

@oliverde8 💯 ! I forgot to mention it and you're perfectly right. A single client maximizes the reuse of existing connections and maximizes concurrency monitoring.

XuruDragon · 2019-03-19T17:16:23Z

Thanks @oliverde8.
Do you have some suggestions To make this pr greater ?

oliverde8 · 2019-03-19T17:27:28Z

Just thinking, having a single client for multiple api's have it's issues; mainly for authorization.

You need to get all the authorization tokens even if you might not need them all. For fast expiring tokens this can be bothersome.
Another issue is when using a single client for extended periods of time is the authorization tokens expiring. So having a single client means the whole client needs to be updated; this is more of an issue for long-running console commands of course.

Why not pass list of objects, RequestEnricherInterfaces with 2 methods supports, and alter. It would work much like Voters and the other Symfony components? That way the RequestEnricher would only execute if needed and could renew the token if needed.

Not sure if it's a good idea, or over complicated

ro0NL · 2019-03-19T17:29:29Z

Isnt it as easy to handle this type of business logic in a custom decorator? Would be much more explicit isnt it, and eventually more flexible.

Is the generic feature really a must? Do we aim for some framework config integration? Which might define N clients 🤔

edit: what about supporting a array|callable $defaultOptions in the existing clients?

src/Symfony/Component/HttpClient/ConditionalHttpClient.php

fabpot · 2019-03-23T14:56:58Z

Now that #30654 is merged, this one should probably be rebased (and adapter).

@XuruDragon

This PR was merged into the 4.3-dev branch. Discussion ---------- [HttpClient] Add a ScopingHttpClient | Q | A | ------------- | --- | Branch? | master | Bug fix? | no | New feature? | yes | BC breaks? | no | Deprecations? | no | Tests pass? | yes | Fixed tickets | - | License | MIT | Doc PR | - This PR is a follow up of #30592 by @XuruDragon, with two main differences: - I think `ScopingHttpClient` might be a better name for what is called a `ConditionalHttpClient` there, - the `FrameworkBundle` part is removed so that it can be submitted separately later on. With a `ScopingHttpClient`, you can add some default options conditionally based on the requested URL and a regexp that it should match. This allows building clients that add e.g. credentials based on the requested scheme/host/path. When the requested URL is a relative one, a default index can be provided - whose corresponding default options (the `base_uri` one especially) will be used to turn it into an absolute URL. Regexps are anchored on their left side. E.g. this defines a client that will send some github token when a request is made to the corresponding API, and will not send those credentials if any other host is requested, while also turning relative URLs to github ones: ```php $client = HttpClient::create(); $githubClient = new ScopingClient($client, [ 'http://api\.github\.com/' => [ 'base_uri' => 'http://api.github.com/', 'headers' => ['Authorization: token '.$githubToken], ], ], 'http://api\.github\.com/'); ``` Of course, it's possible to define several regexps as keys so that one can create a client that is authenticated against several hosts/paths. Commits ------- 1ee0a11 [HttpClient] Add a ScopingHttpClient

…ingClient

nicolas-grekas · 2019-03-24T21:11:32Z

I continued the framework-bundle part in #30674
Thank you @XuruDragons
I invite everyone to review the new PR now.

carsonbot added Status: Needs Review Feature labels Mar 18, 2019

XuruDragon force-pushed the conditional-http-client branch 2 times, most recently from 8223b4d to eaf6bb7 Compare March 18, 2019 14:59

nicolas-grekas requested changes Mar 18, 2019

View reviewed changes

carsonbot added Status: Needs Work and removed Status: Needs Review labels Mar 18, 2019

XuruDragon force-pushed the conditional-http-client branch 5 times, most recently from 4b3e4e2 to a1b5933 Compare March 19, 2019 16:52

nicolas-grekas reviewed Mar 19, 2019

View reviewed changes

src/Symfony/Component/HttpClient/ConditionalHttpClient.php Outdated Show resolved Hide resolved

XuruDragon force-pushed the conditional-http-client branch 2 times, most recently from 6bc396e to 65d08a7 Compare March 20, 2019 12:28

XuruDragon force-pushed the conditional-http-client branch 3 times, most recently from 7432f9b to 1fed2f6 Compare March 20, 2019 16:28

nicolas-grekas added this to the next milestone Mar 21, 2019

XuruDragon force-pushed the conditional-http-client branch 11 times, most recently from 9b66a31 to 7a1a8a0 Compare March 21, 2019 16:44

nicolas-grekas mentioned this pull request Mar 23, 2019

[HttpClient] Add a ScopingHttpClient #30654

Merged

[FrameworkBundle] configure HttpClient services using scopes and Scop…

1d16850

…ingClient

nicolas-grekas force-pushed the conditional-http-client branch from 7a1a8a0 to 1d16850 Compare March 23, 2019 22:28

nicolas-grekas closed this Mar 24, 2019

nicolas-grekas modified the milestones: next, 4.3 Apr 30, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[HttpClient] Add a ConditionalHttpClient #30592

[HttpClient] Add a ConditionalHttpClient #30592

XuruDragon commented Mar 18, 2019 •

edited

Loading

javiereguiluz commented Mar 18, 2019

XuruDragon commented Mar 18, 2019 •

edited

Loading

kunicmarko20 commented Mar 18, 2019 •

edited

Loading

XuruDragon commented Mar 18, 2019 •

edited

Loading

nicolas-grekas commented Mar 18, 2019

javiereguiluz commented Mar 18, 2019

nicolas-grekas commented Mar 18, 2019 •

edited

Loading

XuruDragon commented Mar 18, 2019 •

edited

Loading

javiereguiluz commented Mar 18, 2019

nicolas-grekas commented Mar 18, 2019 •

edited

Loading

oliverde8 commented Mar 19, 2019

nicolas-grekas commented Mar 19, 2019

XuruDragon commented Mar 19, 2019

oliverde8 commented Mar 19, 2019 •

edited

Loading

ro0NL commented Mar 19, 2019 •

edited

Loading

fabpot commented Mar 23, 2019

nicolas-grekas commented Mar 24, 2019

[HttpClient] Add a ConditionalHttpClient #30592

[HttpClient] Add a ConditionalHttpClient #30592

Conversation

XuruDragon commented Mar 18, 2019 • edited Loading

javiereguiluz commented Mar 18, 2019

XuruDragon commented Mar 18, 2019 • edited Loading

kunicmarko20 commented Mar 18, 2019 • edited Loading

XuruDragon commented Mar 18, 2019 • edited Loading

nicolas-grekas commented Mar 18, 2019

javiereguiluz commented Mar 18, 2019

nicolas-grekas commented Mar 18, 2019 • edited Loading

XuruDragon commented Mar 18, 2019 • edited Loading

javiereguiluz commented Mar 18, 2019

nicolas-grekas commented Mar 18, 2019 • edited Loading

oliverde8 commented Mar 19, 2019

nicolas-grekas commented Mar 19, 2019

XuruDragon commented Mar 19, 2019

oliverde8 commented Mar 19, 2019 • edited Loading

ro0NL commented Mar 19, 2019 • edited Loading

fabpot commented Mar 23, 2019

nicolas-grekas commented Mar 24, 2019

XuruDragon commented Mar 18, 2019 •

edited

Loading

XuruDragon commented Mar 18, 2019 •

edited

Loading

kunicmarko20 commented Mar 18, 2019 •

edited

Loading

XuruDragon commented Mar 18, 2019 •

edited

Loading

nicolas-grekas commented Mar 18, 2019 •

edited

Loading

XuruDragon commented Mar 18, 2019 •

edited

Loading

nicolas-grekas commented Mar 18, 2019 •

edited

Loading

oliverde8 commented Mar 19, 2019 •

edited

Loading

ro0NL commented Mar 19, 2019 •

edited

Loading