-
Notifications
You must be signed in to change notification settings - Fork 10
fix: handle graceful closes of the amqp channels #186
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix: handle graceful closes of the amqp channels #186
Conversation
Signed-off-by: Valery Piashchynski <[email protected]>
WalkthroughThe changes in the Changes
Poem
Warning There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure. 🔧 golangci-lint (1.64.8)Error: you are using a configuration file for golangci-lint v2 with golangci-lint v1: please use golangci-lint v2 📜 Recent review detailsConfiguration used: CodeRabbit UI 📒 Files selected for processing (2)
💤 Files with no reviewable changes (1)
⏰ Context from checks skipped due to timeout of 90000ms (3)
🔇 Additional comments (1)
✨ Finishing Touches
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
CodeRabbit Configuration File (
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This PR updates the AMQP driver's redialer to better handle graceful closures of various AMQP channels by checking for both error and closed states. The changes include reading a tuple (err, closed) from the notify channels, refining the clean-up logic with IsClosed() checks, and updating log messages to better reflect the connection state.
Files not reviewed (1)
- go.work.sum: Language not supported
Comments suppressed due to low confidence (1)
amqpjobs/redial.go:86
- [nitpick] There is an inconsistency in handling graceful close notifications: connection and consume channels use 'continue' while publish and statistic channels use 'return'. Please confirm if this behavior difference is intentional and consider adding inline comments to clarify the design.
if closed && err == nil { d.log.Debug("[notify close publish channel]: channel is closed")
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actionable comments posted: 2
🧹 Nitpick comments (4)
amqpjobs/redial.go (4)
80-81: Incorrect log message for consume channelThe log message indicates the driver is stopped, but it should indicate that the redial channel is full (similar to other handlers).
- d.log.Debug("[notify close consume channel]: channel is not closed, but driver is stopped") + d.log.Debug("[notify close consume channel]: redial channel is full (or redial in progress)")
118-119: Minor formatting issue in log messageThere's an extra closing bracket at the end of the log message.
- d.log.Debug("[notify close statistic channel]: channel is not closed, but driver is stopped]") + d.log.Debug("[notify close statistic channel]: channel is not closed, but driver is stopped")
130-131: Inconsistent log message for statistics channelThe statistic channel log message says "redialer stopped" while other handlers use "redial channel is full". For consistency, consider using the same message pattern.
- d.log.Debug("[notify close statistic channel]: redialer stopped") + d.log.Debug("[notify close statistic channel]: redial channel is full (or redial in progress)")
34-132: Suggestion: Extract common patterns into helper methodsThe four channel notification handlers follow very similar patterns with repeated code. Consider extracting this pattern into a helper method to improve maintainability and reduce duplication.
+ func (d *Driver) handleChannelClose(channelType string, err *amqp.Error, closed bool, notifySourceName string) bool { + // exit on a graceful close + if closed && err == nil { + d.log.Debug(fmt.Sprintf("[notify close %s channel]: channel is closed", notifySourceName)) + return true // continue the loop + } + + // stopped + if atomic.LoadUint64(&d.stopped) == 1 { + d.log.Debug(fmt.Sprintf("[notify close %s channel]: channel is not closed, but driver is stopped", notifySourceName)) + return false // return to exit goroutine + } + + select { + case d.redialCh <- &redialMsg{ + t: channelType, + err: err, + }: + d.log.Debug(fmt.Sprintf("[notify close %s channel]: redial message was sent", notifySourceName)) + return false + default: + d.log.Debug(fmt.Sprintf("[notify close %s channel]: redial channel is full (or redial in progress)", notifySourceName)) + return false + } + }Usage would look like:
case err, closed := <-d.notifyCloseConnCh: - // exit on a graceful close - if closed && err == nil { - d.log.Debug("[notify close connection channel]: channel is closed") - continue - } - - // stopped - if atomic.LoadUint64(&d.stopped) == 1 { - d.log.Debug("[notify close connection channel]: channel is not closed, but driver is stopped") - return - } - - select { - case d.redialCh <- &redialMsg{ - t: ConnCloseType, - err: err, - }: - d.log.Debug("[notify close connection channel]: redial message was sent") - return - default: - d.log.Debug("[notify close connection channel]: redial channel is full (or redial in progress)") - return - } + if d.handleChannelClose(ConnCloseType, err, closed, "connection") { + continue + } + return
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
⛔ Files ignored due to path filters (1)
go.work.sumis excluded by!**/*.sum
📒 Files selected for processing (1)
amqpjobs/redial.go(5 hunks)
⏰ Context from checks skipped due to timeout of 90000ms (3)
- GitHub Check: RabbitMQ plugin (Go stable, PHP 8.4, OS ubuntu-latest)
- GitHub Check: RabbitMQ plugin durability (Go stable, PHP 8.4, OS ubuntu-latest)
- GitHub Check: Analyze (go)
🔇 Additional comments (8)
amqpjobs/redial.go (8)
34-39: Good addition for handling graceful closesThe change to receive both the error and a closed boolean from the channel is a good improvement. This properly distinguishes between graceful closes (closed && err == nil) and actual errors that require redial.
42-45: Clearer logging for driver stopped stateThe updated debug log message is more descriptive, making it clear that while the channel isn't closed, the driver is stopped. This helps with troubleshooting connection states.
52-56: Enhanced logging for redial operationsThe log messages now clearly indicate whether a redial message was sent or if the channel is full. This improvement helps track the redial process flow.
59-64: Consistent pattern for graceful close handlingThe consume channel handler correctly follows the same pattern established for connection channels, improving consistency in error handling across different channel types.
172-177: Good safety check for consumeChanAdding the nil check and IsClosed() verification before attempting to close the channel prevents potential errors when operating on already closed or nil channels. This is a good defensive programming practice.
179-184: Good safety check for connectionSimilar to the channel check, this safety check for the connection prevents attempts to close already closed connections, which is a good practice.
193-194: Improved reset method with better logging and connection checksThe reset method now includes better debug logging and properly checks if the connection is not closed before attempting to close it, which is a good safety measure.
Also applies to: 213-220
235-236: Improved comment clarityThe updated comment more clearly describes what the redial function does - it trashes broken publishing channels and closes the connection.
Co-authored-by: Copilot <[email protected]>
Signed-off-by: Valery Piashchynski <[email protected]>
Signed-off-by: Valery Piashchynski <[email protected]>
Signed-off-by: Valery Piashchynski <[email protected]>
Reason for This PR
ref: https://github.com/orgs/roadrunner-server/discussions/2012
closes: roadrunner-server/roadrunner#2091
Description of Changes
License Acceptance
By submitting this pull request, I confirm that my contribution is made under
the terms of the MIT license.
PR Checklist
[Author TODO: Meet these criteria.][Reviewer TODO: Verify that these criteria are met. Request changes if not]git commit -s).CHANGELOG.md.Summary by CodeRabbit
Bug Fixes
Refactor
Chores