Thanks to visit codestin.com
Credit goes to github.com

Skip to content

[Messenger] Add TransportNamesStamp to change the transport while dispatching a message #39306

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Aug 5, 2022

Conversation

abarkine
Copy link
Contributor

@abarkine abarkine commented Dec 4, 2020

Q A
Bug fix? no
New feature? yes
Deprecations? no
Tickets Fix #38200, Fix #39306
License MIT
Doc PR symfony/symfony-docs#17113

This feature will allow developers to choose transport layer while dispatching a message. Normally, we define transport layer in configuration. As a result, developer doesn't have any power to modify it after the deployment. This PR aims to allow overriding already defined transport layer for a message and rerouting it to different transport per dispatch request.

If newly added stamp is not used, default configuration will be used for choosing transport.

Since I've added a new stamp, there shouldn't be any BC change.

@carsonbot
Copy link

Hey!

I see that this is your first PR. That is great! Welcome!

Symfony has a contribution guide which I suggest you to read.

In short:

  • Always add tests
  • Keep backward compatibility (see https://symfony.com/bc).
  • Bug fixes must be submitted against the lowest maintained branch where they apply (see https://symfony.com/releases)
  • Features and deprecations must be submitted against the 5.x branch.

Review the GitHub status checks of your pull request and try to solve the reported issues. If some tests are failing, try to see if they are failing because of this change.

When two Symfony core team members approve this change, it will be merged and you will become an official Symfony contributor!
If this PR is merged in a lower version branch, it will be merged up to all maintained branches within a few days.

I am going to sit back now and wait for the reviews.

Cheers!

Carsonbot

@jderusse jderusse added this to the 5.3 milestone Dec 4, 2020
Copy link
Member

@jderusse jderusse left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for this contribution.

What if people want keeping the original senders?.

I wonder if we couldn't use something similar to the BusNameStamp:

  • use a middleware that initialize the envelope with a SenderNamesStamp
  • send the envelope to the senders contained in the last SenderNamesStamp

@abarkine
Copy link
Contributor Author

abarkine commented Dec 5, 2020

BusNameStamp and using middleware to insert Stamp

Wouldn't it add an overhead? In current way, Envelope will not include OverrideStamp if not necessary. What do you think?

@diimpp
Copy link
Member

diimpp commented Dec 5, 2020

On naming note, public stamp name shouldn't be derived from this moment internal implementation, so something like SendersOverride is not very suitable in my opinion.

As library user, who is not privy to all the internals, first naming choice would be TransportRouteStamp, but I do see that the routing is the framework-bundle config concept, rather than messenger concept, so maybe TransportSendersStamp

@abarkine
Copy link
Contributor Author

abarkine commented Dec 7, 2020

On naming note, public stamp name shouldn't be derived from this moment internal implementation, so something like SendersOverride is not very suitable in my opinion.

As library user, who is not privy to all the internals, first naming choice would be TransportRouteStamp, but I do see that the routing is the framework-bundle config concept, rather than messenger concept, so maybe TransportSendersStamp

@diimpp , thanks for your input, I will change the name, what do you think about the implementation?

@diimpp
Copy link
Member

diimpp commented Dec 7, 2020

@asilelik looks alright to me, but I'm not a messenger specialist.

One more thing, I don't why there are multiple senders routing in the messenger and what is the use case, but maybe you will want to allow to have both behaviors to override/extend senders config with bool flag at Stamp class.

@abarkine
Copy link
Contributor Author

abarkine commented Dec 7, 2020

@asilelik looks alright to me, but I'm not a messenger specialist.

One more thing, I don't why there are multiple senders routing in the messenger and what is the use case, but maybe you will want to allow to have both behaviors to override/extends senders config with bool flag at Stamp class.

Do you mean to send envelope to both configured transport and overridden one?

@diimpp
Copy link
Member

diimpp commented Dec 7, 2020

@asilelik looks alright to me, but I'm not a messenger specialist.
One more thing, I don't why there are multiple senders routing in the messenger and what is the use case, but maybe you will want to allow to have both behaviors to override/extends senders config with bool flag at Stamp class.

Do you mean to send envelope to both configured transport and overridden one?

For example if somebody already have their transports setup, but want conditionally to specify audit transport for certain messages.

Then they will want to keep their original transports list and just add another one, e.g. new TransportSendersStamp(['audit'], $extendSenders = true)

My personal use case is quite different, I need to conditionally send messeges to sync or async transports, so I will need to override original transports routing if preset, e.g. new TransportSendersStamp(['async'], $overrideSenders = true)

@abarkine
Copy link
Contributor Author

abarkine commented Dec 7, 2020

@asilelik looks alright to me, but I'm not a messenger specialist.
One more thing, I don't why there are multiple senders routing in the messenger and what is the use case, but maybe you will want to allow to have both behaviors to override/extends senders config with bool flag at Stamp class.

Do you mean to send envelope to both configured transport and overridden one?

For example if somebody already have their transports setup, but want conditionally to specify audit transport for certain messages.

Then they will want to keep their original transports list and just add another one, e.g. new TransportSendersStamp(['audit'], $extendSenders = true)

My personal use case is quite different, I need to conditionally send messeges to sync or async transports, so I will need to override original transports routing if preset, e.g. new TransportSendersStamp(['async'], $overrideSenders = true)

Good point. It will make SendersLocator class a bit complex but I guess it will add a great value to the Stamp.

@nicolas-grekas nicolas-grekas modified the milestones: 5.3, 5.x Dec 8, 2020
@abarkine abarkine requested a review from jderusse January 4, 2021 09:31
@abarkine
Copy link
Contributor Author

@jderusse , can you take a second look at this PR and comment on suggestions? If you are fine with current implementation, I will create the tests to finalize PR.

@r6rob
Copy link

r6rob commented Aug 2, 2021

My use case is very similar to @diimpp. Building a job queue and would love to specify whether a job will be handled as sync or async programatically.

This would be very helpful, eagerly awaiting PR.

@fabpot fabpot modified the milestones: 5.4, 6.1 Nov 16, 2021
@chalasr
Copy link
Member

chalasr commented Mar 29, 2022

@asilelik Thank you for the PR. Please go ahead with the missing tests, this looks good to me.

@abarkine
Copy link
Contributor Author

@chalasr I've added the test. Apart from git rebase error, how can I fix other failures? They don't seem like related errors to my PR.

@chalasr
Copy link
Member

chalasr commented Mar 30, 2022

Yes, the failing test is unrelated.

@abarkine
Copy link
Contributor Author

@chalasr what is the preferred way for getting rid of merge commit? Can I force push or is there any other suggestion?

@chalasr
Copy link
Member

chalasr commented Mar 31, 2022

I missed that Jérémy's comment, thanks.

What about conflicting when both are sent?

Given the originally proposed name, I suppose one may want to configure the sender the regular way and use the stamp to change the sender under certain circumstances only.
Maybe the stamp could take a flag controlling whether to replace existing senders or not?

@fabpot fabpot modified the milestones: 6.1, 6.2 May 20, 2022
@tg-mvooges
Copy link

tg-mvooges commented Jul 18, 2022

if we only look at the stamp, it's less clear to me that adding one ViaSenderStamp is going to erase all senders set via the framework config

This maps to @jderusse's comment:

What if people want keeping the original senders?.

What about conflicting when both are sent? Aka throw if a sender is configured while the stamp is also used? That'd make the situation impossible.

I've not read the full thread (just the last few messages) but I would not for for a "when both are set". I would expect an either/or situation: You either use the default symfony-sender-decider-logic, OR the specified ones. If you would override the default configed target queue via a Stamp I think that could be made obvious with a clear name.

To add a little insight on how we would like to use this: We have a few microservices which listen to the queue via the normal config. The failed messages all get send to a central located microservice which has logic to place them back on their respective queues.
But that could be any queue, the central micro doesnt perse know everything. It has to now, which is inconvinient

@TCM-dev
Copy link

TCM-dev commented Aug 4, 2022

I'm also of the opinion that this should either use the default configured ones OR the specified ones (which I believe makes sense with the stamp name OverrideSendersStamp)

I personnaly need this because in my application, I need to have the same message/handler to have a different priority depending on some context. (I can provide real world example if needed)

Unless I'm mistaken, the only way I could do this right now would require me to duplicate the message/handlers with another name and configure them with another transport which is not ideal.

Adding one stamp to decide if a message has an higher priority than another on the fly makes perfect sense to me and I was persuaded that it was actually a feature in messenger.

Are the unsuccessful checks the only things preventing this to be merged or is this still open to discussion?

Copy link
Member

@fabpot fabpot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've rebased this PR and made some naming changes to finish it.

@fabpot fabpot force-pushed the feature/define-transport-dynamically branch 2 times, most recently from adfbd37 to 1a5f229 Compare August 4, 2022 15:15
@tg-mvooges
Copy link

I've rebased this PR and made some naming changes to finish it.

I'm not 100% sure, but does this now ADD senders instead of overwriting? That might be slightly limiting. Not for my case, but supose you have a message default send to queueA and you want to CHANGE that to queueB. If you can only ADD, that might be problematic. Using an Override of some kind might be a more broad solution. Or some other method to reset the current senders.

Again, doesnt bother me, I'm looking forward for this feature either way ^_^

@fabpot fabpot force-pushed the feature/define-transport-dynamically branch from 1a5f229 to 239ea41 Compare August 4, 2022 15:25
@fabpot
Copy link
Member

fabpot commented Aug 4, 2022

@tg-mvooges That's the case. See https://github.com/symfony/symfony/pull/39306/files#diff-357beebf345f58f777f74eb1127331b8107de09438ff78435093085a531c52beR52, if the stamp is set, we return and bypass the default configuration. Did I miss something?

@fabpot fabpot changed the title [Messenger] Add functionality for choosing transport while dispatching message [Messenger] Add SenderNamesStamp to change the transport while dispatching a message Aug 4, 2022
@TCM-dev
Copy link

TCM-dev commented Aug 4, 2022

@tg-mvooges That's the case. See https://github.com/symfony/symfony/pull/39306/files#diff-357beebf345f58f777f74eb1127331b8107de09438ff78435093085a531c52beR52, if the stamp is set, we return and bypass the default configuration. Did I miss something?

Looks perfect to me and more consistent with other stamp names.
I'm just wondering if "sender" is the correct word, the documentation talks about "transport", this issue talk about being able to change "transport" and yet we name it sender.
I may be wrong but wouldn't it be even better if it was named TransportsStamp/TransportNamesStamp ?

@fabpot fabpot force-pushed the feature/define-transport-dynamically branch from 239ea41 to bffb292 Compare August 4, 2022 18:42
@fabpot
Copy link
Member

fabpot commented Aug 4, 2022

@tg-mvooges That's the case. See https://github.com/symfony/symfony/pull/39306/files#diff-357beebf345f58f777f74eb1127331b8107de09438ff78435093085a531c52beR52, if the stamp is set, we return and bypass the default configuration. Did I miss something?

Looks perfect to me and more consistent with other stamp names. I'm just wondering if "sender" is the correct word, the documentation talks about "transport", this issue talk about being able to change "transport" and yet we name it sender. I may be wrong but wouldn't it be even better if it was named TransportsStamp/TransportNamesStamp ?

Even if Sender is the right word, I think you're right that using Transport is easier to understand. Updated accordingly.

@fabpot fabpot force-pushed the feature/define-transport-dynamically branch from bffb292 to d9f1254 Compare August 4, 2022 18:44
@fabpot fabpot changed the title [Messenger] Add SenderNamesStamp to change the transport while dispatching a message [Messenger] Add TransportNamesStamp to change the transport while dispatching a message Aug 5, 2022
@fabpot fabpot force-pushed the feature/define-transport-dynamically branch from d9f1254 to c47334e Compare August 5, 2022 06:03
@fabpot fabpot force-pushed the feature/define-transport-dynamically branch from c47334e to f526af0 Compare August 5, 2022 06:45
@fabpot
Copy link
Member

fabpot commented Aug 5, 2022

Thank you @asilelik.

@fabpot fabpot closed this in 1a4a5d5 Aug 5, 2022
@fabpot fabpot merged commit 1a4a5d5 into symfony:6.2 Aug 5, 2022
fabpot added a commit that referenced this pull request Aug 7, 2022
…ally (fabpot)

This PR was merged into the 6.2 branch.

Discussion
----------

[Mailer] Add a way to change the Bus transport dynamically

| Q             | A
| ------------- | ---
| Branch?       | 6.2
| Bug fix?      | no
| New feature?  | yes <!-- please update src/**/CHANGELOG.md files -->
| Deprecations? | no <!-- please update UPGRADE-*.md and src/**/CHANGELOG.md files -->
| Tickets       | Fix #36348 <!-- prefix each issue number with "Fix #", no need to create an issue if none exist, explain below instead -->
| License       | MIT
| Doc PR        |

This PR relies on #39306. Once merged, this PR will be rebased.
This allows overriding the bus transport "dynamically" via the `X-Bus-Transport` header.

Commits
-------

ffe3600 [Mailer] Add a way to change the Bus transport dynamically
@tg-mvooges
Copy link

@tg-mvooges That's the case. See https://github.com/symfony/symfony/pull/39306/files#diff-357beebf345f58f777f74eb1127331b8107de09438ff78435093085a531c52beR52, if the stamp is set, we return and bypass the default configuration. Did I miss something?

Ah, with the focus on that snippet, yes that seems perfect! I'm fairly new to this message concept, I'm still getting used to this 'Stamps' logic and how it works. Great feature coming up, looking forward to be able to use it, good job!

@fabpot fabpot mentioned this pull request Oct 24, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Messenger] Defining transport dynamically
10 participants