rename watcher to monitor and add redis heartbeat module#1
Conversation
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 2 potential issues.
Bugbot Autofix is OFF. To automatically fix reported issues with Cloud Agents, enable Autofix in the Cursor dashboard.
| "#{node_id} missed the last #{state.missed_heartbeats_allowed} heartbeats. Re-enqueing jobs from backup." | ||
| ) | ||
|
|
||
| Enum.uniq(Exq.Redis.JobQueue.list_queues(state.redis, state.namespace) ++ state.queues) |
There was a problem hiding this comment.
Monitor crashes on queue lookup failures
Medium Severity
re_enqueue_backup/2 calls Exq.Redis.JobQueue.list_queues/2, which uses the bang variant and raises on Redis errors. A transient SMEMBERS failure now terminates Exq.Heartbeat.Monitor instead of handling the error path, so orphan recovery can stop during failover and the process may enter restart loops.
|
|
||
| def smembers(redis, set) do | ||
| q(redis, ["SMEMBERS", set]) | ||
| end |
There was a problem hiding this comment.
Unused Redis helper increases maintenance burden
Low Severity
Connection.smembers/2 is newly exported but has no in-repo callers. This introduces dead code and a parallel API surface next to smembers!/2, which makes Redis access patterns less clear and increases long-term maintenance overhead.
Note
Medium Risk
Touches heartbeat/failover behavior and Redis interactions that can affect job reprocessing; mistakes could cause missed or duplicate work, but changes are localized and covered by a new test.
Overview
Adds a new
Exq.Heartbeat.Monitorprocess (replacingExq.Heartbeat.Watcherin supervisor wiring) that periodically finds nodes with stale heartbeats and re-enqueues their backup jobs across known queues, then unregisters the node.Extracts heartbeat Redis interactions into
Exq.Redis.Heartbeatand updatesExq.Heartbeat.Serverto use it for heartbeat registration.Exq.Redis.Connectiongainssmembers/2and slightly hardens failover reconnect logic, and a new test covers re-enqueueing behavior when a heartbeat stops.Written by Cursor Bugbot for commit 04d8531. This will update automatically on new commits. Configure here.