Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Messenger with multiple AMQP nodes #35098

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
melvinkoopmans opened this issue Dec 24, 2019 · 29 comments
Closed

Messenger with multiple AMQP nodes #35098

melvinkoopmans opened this issue Dec 24, 2019 · 29 comments
Labels
Feature Messenger

Comments

@melvinkoopmans
Copy link

Description
I have a RabbitMQ cluster with multiple nodes. In order to have redundancy the application needs to be able to connect to all of the nodes. As of now, Symfony Messenger does not allow you to specify more than one node. Right now we "solve" the problem by using HAProxy. It would be better if Symfony allowed you to specify all of the nodes in the DSN.

Proposal
The default AMQP extension of PHP does not allow you to specify multiple nodes (as far as I'm concerned). The library php-amqplib allows you to do this with its AMQPStreamConnection class. Like this:

$connection = AMQPStreamConnection::create_connection([
    ['host' => HOST1, 'port' => PORT, 'user' => USER, 'password' => PASS, 'vhost' => VHOST],
    ['host' => HOST2, 'port' => PORT, 'user' => USER, 'password' => PASS, 'vhost' => VHOST]
],
$options);

I propose something similar as how Symfony uses Redis Sentinel. To use Redis Sentinel you need the Predis library and you can specify your DSN as follows:

redis:?host[redis1:26379]&host[redis2:26379]&host[redis3:26379]&redis_sentinel=mymaster

Then for AMQP you would specify the DSN as follows:

amqp:?host[node1:5672]&host[node2:5672]&host[node3:5672]
@melvinkoopmans
Copy link
Author

@xabbuh I see this feature request is tagged under Mailer, but it corresponds to Messenger.

@xabbuh xabbuh added Messenger and removed Mailer labels Dec 28, 2019
@jderusse
Copy link
Member

jderusse commented Apr 6, 2020

I've started working on it, but, finally think Symfony should not provide such configuration, because it's only supported by the php adapter, and an issue in amqp-extension suggest to use a haproxy in front of the cluster instead. Which is probably the better option.

@melvinkoopmans
Copy link
Author

@jderusse That's what we've been using in the meantime. I've experienced no issues with HAproxy so far.

Isn't the Redis component doing the same thing with respect to configuration options? Since Redis Sentinel is only available in Symfony if you have nrk/predis installed.

@DamienHarper
Copy link
Contributor

@jderusse I also use HAProxy in front of my rabbit cluster (4 nodes) and I experience quite a lot of random "Library error: connection closed unexpectedly".

@Nyholm Nyholm mentioned this issue May 28, 2020
Closed
3 tasks
@carsonbot
Copy link

Thank you for this suggestion.
There has not been a lot of activity here for a while. Would you still like to see this feature?

@Ases
Copy link

Ases commented Feb 24, 2021

Thank you for this suggestion.
There has not been a lot of activity here for a while. Would you still like to see this feature?

Yes please.

@carsonbot carsonbot removed the Stalled label Feb 24, 2021
@AdrienBr
Copy link

Hi,
Also interested here !

Thank you

@rruiz-medtrainer
Copy link

Hi,
Also interested here !

Thank you

@xabbuh
Copy link
Member

xabbuh commented Jul 30, 2021 •
edited
Loading

Is anyone of you willing to give this a try?

@Wulfu
Copy link

Wulfu commented Aug 12, 2021 •
edited
Loading

Interested AF

@jkobus
Copy link
Contributor

jkobus commented Oct 13, 2021

Failover transport syntax is worth looking at in the context of this feature.

@fancyweb fancyweb mentioned this issue Feb 11, 2022
Closed
@carsonbot
Copy link

Thank you for this suggestion.
There has not been a lot of activity here for a while. Would you still like to see this feature?

@TheHett
Copy link

TheHett commented Apr 14, 2022

Yep, now it is impossible to work with a cluster without a haproxy

@carsonbot carsonbot removed the Stalled label Apr 14, 2022
@bluebrown
Copy link

bluebrown commented Aug 17, 2022 •
edited
Loading

Load balancing with HAProxy or similar does not make sense, I guess. The same way it would not make sense to simply load balance across a redis cluster.

That's because you establish a connection with a node. And the client library probably expects to reach the same node when sending data. Otherwise, you send some data to a node with which you have not established a connection yet and authenticated and so on. Hence, we see errors like mentioned by @DamienHarper:

"Library error: connection closed unexpectedly".

@carsonbot
Copy link

Thank you for this suggestion.
There has not been a lot of activity here for a while. Would you still like to see this feature?

@bluebrown
Copy link

Thank you for this suggestion. There has not been a lot of activity here for a while. Would you still like to see this feature?

Yes please.

@carsonbot carsonbot removed the Stalled label Mar 2, 2023
@carsonbot
Copy link

Thank you for this suggestion.
There has not been a lot of activity here for a while. Would you still like to see this feature?

@Wulfu
Copy link

Wulfu commented Sep 5, 2023

Still yes

@carsonbot carsonbot removed the Stalled label Sep 5, 2023
@carsonbot
Copy link

Thank you for this suggestion.
There has not been a lot of activity here for a while. Would you still like to see this feature?

@freswa
Copy link

freswa commented Mar 6, 2024

Yes

@carsonbot carsonbot removed the Stalled label Mar 6, 2024
@xabbuh
Copy link
Member

xabbuh commented Mar 6, 2024

Well, someone needs to provide the PR for it. The feature won't appear magically on its own.

@carsonbot
Copy link

Thank you for this suggestion.
There has not been a lot of activity here for a while. Would you still like to see this feature?
Every feature is developed by the community.
Perhaps someone would like to try?
You can read how to contribute to get started.

@carsonbot
Copy link

Could I get an answer? If I do not hear anything I will assume this issue is resolved or abandoned. Please get back to me <3

@freswa
Copy link

freswa commented Sep 21, 2024

@carsonbot Yes, this is still a missing feature.

@carsonbot carsonbot removed the Stalled label Sep 21, 2024
@xabbuh
Copy link
Member

xabbuh commented Sep 21, 2024

@freswa Would you like to contribute it?

@freswa
Copy link

freswa commented Sep 21, 2024 •
edited
Loading

@xabbuh Not within weeks, but when work settles a bit, why not?

@carsonbot
Copy link

Thank you for this suggestion.
There has not been a lot of activity here for a while. Would you still like to see this feature?
Every feature is developed by the community.
Perhaps someone would like to try?
You can read how to contribute to get started.

@buffcode
Copy link
Contributor

buffcode commented Apr 2, 2025

Still interested and useful

@carsonbot carsonbot removed the Stalled label Apr 2, 2025
@xabbuh
Copy link
Member

xabbuh commented Apr 3, 2025

I am closing here as apparently in over four years nobody was interested enough to contribute this feature.

@xabbuh xabbuh closed this as completed Apr 3, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Feature Messenger
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests