[adapters] NATS: Fix "consumer already exists" error for named consumers #5525
Conversation
There was a problem hiding this comment.
Pull request overview
This PR fixes a bug in the NATS adapter where pipelines with named consumers would fail with "consumer already exists" errors during rapid restarts or replays before the previous consumer expires.
Changes:
- Append a UUID suffix to named NATS consumers to prevent name collisions
- Add error context when consumer creation fails
- Add test case verifying named consumers work correctly during restart/replay cycles
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| crates/adapters/src/transport/nats/input.rs | Implements UUID suffix for named consumers and improves error messaging |
| crates/adapters/src/transport/nats/input/test.rs | Adds test infrastructure and test case for named consumer restart/replay scenarios |
| .create_consumer_strict_on_stream(consumer_config, stream_name) | ||
| .await?) | ||
| // Add a unique suffix to named consumers. | ||
| // If consumer is unamed, NATS automatically generates a random name. |
There was a problem hiding this comment.
Corrected spelling of 'unamed' to 'unnamed'.
| // If consumer is unamed, NATS automatically generates a random name. | |
| // If consumer is unnamed, NATS automatically generates a random name. |
There was a problem hiding this comment.
this looks like a useful suggestion
mihaibudiu
left a comment
There was a problem hiding this comment.
We may need to make a clone this PR in order to merge it... our CI still does not like external contributions.
| .create_consumer_strict_on_stream(consumer_config, stream_name) | ||
| .await?) | ||
| // Add a unique suffix to named consumers. | ||
| // If consumer is unamed, NATS automatically generates a random name. |
There was a problem hiding this comment.
this looks like a useful suggestion
|
@kfollesdal Please add Signed-off-by to these commits since we use the DCO. Thank you! I'm continuing to review this. |
|
Besides the need for DCO, this seems easy to approve. Thanks! |
|
@blp Will do. I’m off my computer now, but I’ll take care of it when I’m back at it. |
Add test to check that we do not get error/bug "consumer exist", that we got when pipeline with a named nats consumer got restarted/replayed. Signed-off-by: kfollesdal <[email protected]>
This is a fix for the bug "consumer exist" that we get when we had a named NATS consumer in a pipline that got restarted/replayed. Signed-off-by: kfollesdal <[email protected]>
Signed-off-by: kfollesdal <[email protected]>
ef211ea to
0436564
Compare
|
Thanks @blp, I have now signed off the commits. |
|
This is going to merge via #5526, so I closing it in favor of that one (sorry about our awkward process). |
Description:
Fixes a bug where pipelines with named NATS consumers would fail with "consumer already exists" errors during rapid restarts or replays, before the previous consumer expires (inactive_threshold).
Changes:
Checklist
Breaking Changes?
Mark if you think the answer is yes for any of these components:
Describe Incompatible Changes