Thanks to visit codestin.com
Credit goes to github.com

Skip to content

[Cache] handle redis cluster creation by dsn #28300

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 2 commits into from

Conversation

alekitto
Copy link
Contributor

Q A
Branch? master
Bug fix? no
New feature? yes
BC breaks? no
Deprecations? no
Tests pass? yes
License MIT
Doc PR TBD

This PR adds support for \RedisCluster connection creation via DSN passing redis+cluster as protocol (ex: redis+cluster://redis-server-1:6379,redis-server-2:6379).

This allow to have the same configuration for different environments and change only the env var containing the connection string.

@alekitto alekitto force-pushed the redis_cluster branch 4 times, most recently from 0e0598d to 3fc3261 Compare August 29, 2018 06:33
@nicolas-grekas nicolas-grekas added this to the next milestone Aug 30, 2018
Copy link
Member

@nicolas-grekas nicolas-grekas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, thanks!
Here are some 1st review comments.
I'm not sold on the "redis+cluster" scheme. The reason is that when using Predis, there is another way to configure a cluster proposed in #28175
It would be awesome to be able to have a similar way of declaring a cluster when using either RedisCluster from the extension or from Predis.
Maybe let's merge #28175 as is, rebase this PR on top and refactor it to handle a common URL syntax?

} elseif ($this->redis instanceof \RedisCluster || $this->redis instanceof RedisClusterProxy) {
$keys = array();

foreach ($this->redis->_masters() as $nodeParams) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Coincidently, I did also fix doClear in #28269, with a slightly different implementation, see
https://github.com/symfony/symfony/pull/28269/files#diff-a87375907f78e0dcbee340d314e3d052L236
For this reason, this should be reverted from this PR.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, i've reverted it.

@@ -45,7 +46,8 @@ public function __construct($redis, array $options = array())
!$redis instanceof \RedisArray &&
!$redis instanceof \RedisCluster &&
!$redis instanceof \Predis\Client &&
!$redis instanceof RedisProxy
!$redis instanceof RedisProxy &&
!$redis instanceof RedisClusterProxy
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the Lock component also needs a similar patch

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 Done

@alekitto
Copy link
Contributor Author

I've rebased this PR on master and fixed some conflicts.
For the url syntax question, what about cluster=1 or cluster=true option in query string instead of redis+cluster scheme?

@jralph
Copy link

jralph commented Sep 10, 2018

Would cluster= need to be a string? With redis you generally have 2 cluster options, handle the cluster from the client side, or let redis handle the cluster. So maybe a cluster=client and a cluster=server options. If cluster is server, then RedisCluser or Predis can be used, if it is client, then I believe only Predis would work.

@nicolas-grekas
Copy link
Member

Yep, there are two modes to select from, both legit choices; cluster=client also works with the extension: that's what the RedisArray class is for.

@nicolas-grekas
Copy link
Member

Something like that?

redis://localhost:1234?server_version=2.8&cluster=array&server[host1:port1]&server[host2:port2]=weight2

would work also for memcached:
memcached://localhost:1234?server_version=2.8&cluster=array&server[host1:port1]&server[host2:port2]=weight2

@alekitto
Copy link
Contributor Author

I’ve implemented the cluster=server part, while I think that the array option should be done in a separate PR since other options could be added/discussed (the “previous” array ring option for example)

@jralph
Copy link

jralph commented Sep 24, 2018

With regards to using predis for clustering, there is a downside I only recently found. Currently, the connection factory predis provides doesn't support creating a redis cluster connection.

For clustering to work with predis, the connection dsn's must be passed in as an array instead of using the factory, or the cluster=redis option is just plain ignored.

@alekitto alekitto force-pushed the redis_cluster branch 2 times, most recently from ff2366c to 1d52b4c Compare September 25, 2018 00:11

// Predis cluster only supports an array of hosts as first argument, otherwise
// options array is ignored.
$redis = new $class($host, array('cluster' => 'redis'));
Copy link

@jralph jralph Sep 25, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe cluster=redis should be based on the cluster dsn parameter. If the dsn parameter is server, then here it should be redis, if the dsn parameter is client, then here it should be predis.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Like I said in a previous comment, the client part has been not implemented as could involve some other options to be handled, so I think it should be done and discussed in another PR

nicolas-grekas added a commit that referenced this pull request Oct 2, 2018
… to Redis clusters (alekitto)

This PR was merged into the 4.2-dev branch.

Discussion
----------

[Cache] add RedisClusterProxy to create lazy connections to Redis clusters

| Q             | A
| ------------- | ---
| Branch?       | master
| Bug fix?      | no
| New feature?  | yes
| BC breaks?    | no
| Deprecations? | no
| Tests pass?   | yes
| Fixed tickets | -
| License       | MIT
| Doc PR        | -

Code provided by @alekitto in #28300
I'm working on an alternative to this PR, but these bits are required.

Commits
-------

239a022 [Cache] add RedisClusterProxy to create lazy connections to Redis clusters
@nicolas-grekas
Copy link
Member

Closing in favor of #28713
Please review/test this PR and see if this works for you.
It'd be awesome if you could contribute some doc for it :)

fabpot added a commit that referenced this pull request Oct 10, 2018
… via DSN (nicolas-grekas)

This PR was merged into the 4.2-dev branch.

Discussion
----------

[Cache] added support for connecting to Redis clusters via DSN

| Q             | A
| ------------- | ---
| Branch?       | master
| Bug fix?      | no
| New feature?  | yes
| BC breaks?    | no
| Deprecations? | no
| Tests pass?   | yes
| Fixed tickets | -
| License       | MIT
| Doc PR        | -

Replaces #28300 and #28175

This PR allows configuring a cluster of Redis servers using all available options of either the phpredis extension or the Predis package:
- the `redis_cluster=0/1` boolean option configures whether the client should use the Redis cluster protocol;
- several hosts can be provided using a syntax very similar to #28598, enabling consistent hashing distribution of keys;
- `failover=error/distribute/slaves` can be set to direct reads at slave servers;
- extra options are passed as is to the driver (e.g. `profile=2.8`)
- Predis per-server settings are also possible, using e.g. `host[localhost][alias]=foo` in the query string, or `host[localhost]=alias%3Dfoo` (ie PHP query arrays or urlencoded key/value pairs)

Commits
-------

a42e877 [Cache] added support for connecting to Redis clusters via DSN
@nicolas-grekas nicolas-grekas modified the milestones: next, 4.2 Nov 1, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants