-
-
Notifications
You must be signed in to change notification settings - Fork 9.6k
[Messenger] Messages with AmqpStamp($routingKey) and direct RabbitMQ exchange are not retry-able #32994
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Comments
Hello, can anybody please give some kind of feedback? Are there any deadlines/processes set? Can I help in any way? This is starting to limit us as we will need this functionality to work correctly very soon. |
Faced the same issue, is critical for us as well. Would be very appreciate for update when this could be fixed. |
Same issue here :) |
Excellent bug report!
I would welcome a PR for this :). |
Thanks for the detailed report. To 1. Edit: Since the Connection does not know about the Envelope, the AmqpReceiver should just create an AmqpStamp based on the |
Same issue here :) |
Someone just needs to create a PR based on @Tobion’s thoughts :) |
See #34134 for a fix |
This is caused because you use the same exchange for the failed messages as for the normal ones. Instead of making the failed transport direct, you can just change the exchange name:
Then you don't have the problem with the routing key when publishing to the failed transport. You don't want to change the routing key of those messages so they can easily be retried and keep the same routing key. |
…nd properties (Tobion) This PR was merged into the 4.3 branch. Discussion ---------- [Messenger] fix retry of messages losing the routing key and properties | Q | A | ------------- | --- | Branch? | 4.3 | Bug fix? | yes | New feature? | no <!-- please update src/**/CHANGELOG.md files --> | Deprecations? | no <!-- please update UPGRADE-*.md and src/**/CHANGELOG.md files --> | Tickets | Fix #32994 <!-- prefix each issue number with "Fix #", if any --> | License | MIT | Doc PR | Messages sent for retry in rabbitmq lost the routing key and properties like the priority. Now we read those original properties and sent the retry message with the same properties (unless those properties have already been set manually before). Commits ------- 75c674d [Messenger] fix retry of messages losing the routing key and properties
Symfony version(s) affected: 4.3.3
RabbitMQ version(s) affected: 3.7.17
Description
Symfony Messenger by default creates fanout exchange
Messages
and all queues are receiving any messages published to the bus. In our project this is not acceptable. We thought that Messenger routing would be enough to place particular messages into corresponding queues. It was not.To route some particular message into (and only to) configured queue, you need to configure Messenger to create a direct exchange, bind queues with routing key and finally to dispatch message with
AmqpStamp()
with said routing key:With this setup we defined two queues bound to direct default exchange with routing key. We can inspect setup via management:
We can inspect binding of the queue in detail, we see it is really bound to exchange with our routing-key:
Now we dispatch message with routing-key like this:
Problem is, this setup will not correctly handle message retrying with selected retryPolicy. I step-debugged code today and found that for correct enqueue to delayed retry queue there must by routing-key present in form of
AmqpStamp
in Envelope object.This Stamp is however not serialized into message because it is implementing
NonSendableStampInterface
. I don't know exact reasons behind this, but changing interface toStampInterface
will solve this problem!However a new problem will now emerge - shoveling the message into failed transport after message reaches configured retryCount in retryPolicy (default is 3).
After that, message should be enqueued to "failed" transport if configured. In our setup (see above) it will fail with error
This is solvable pretty easily, again by defining exchange type for failed transport to direct
However our message will still not be enqueued to failed transport after reaching retryCount. Thats because our previous fix (changing interface of
AmqpStamp
) is now doing bad things when enqueuing failed message - message have routing-key attached to the Envelope which is not wanted!To solve this second problem we must strip this Stamp from Envelope in
SendFailedMessageToFailureTransportListener
. Code should look like this:Now everything is working as expected!
To sumarize
fanout
exchangedirect
default exchangeAmqpStamp($routingKey)
with message when dispatchingAmpqStamp
fromNonSendableStampInterface
toStampInterface
direct
exchange for failed transport as wellAmqpStamp
from envelope in listener from message to be correctly enqueued to failed transportHope this bug report is explaining our problems in understable way.
Wish all the best.
The text was updated successfully, but these errors were encountered: