-
-
Notifications
You must be signed in to change notification settings - Fork 9.7k
Description
Symfony version(s) affected
5.4, 6.3
Description
After a message is claimed nextClaim field is reseted:
symfony/src/Symfony/Component/Messenger/Bridge/Redis/Transport/Connection.php
Lines 285 to 303 in 1f7bc10
| if (\count($claimableIds) > 0) { | |
| try { | |
| $this->getRedis()->xclaim( | |
| $this->stream, | |
| $this->group, | |
| $this->consumer, | |
| $this->redeliverTimeout, | |
| $claimableIds, | |
| ['JUSTID'] | |
| ); | |
| $this->couldHavePendingMessages = true; | |
| } catch (\RedisException $e) { | |
| throw new TransportException($e->getMessage(), 0, $e); | |
| } | |
| } | |
| $this->nextClaim = microtime(true) + $this->claimInterval; | |
| } |
which does not allow to claim next messages:
symfony/src/Symfony/Component/Messenger/Bridge/Redis/Transport/Connection.php
Lines 337 to 339 in 1f7bc10
| if (!$this->couldHavePendingMessages && $this->nextClaim <= microtime(true)) { | |
| $this->claimOldPendingMessages(); | |
| } |
This logic might be intended and works in many cases but it doesn't work well for batch workers with large batch size or large claim_interval setting. It can potentially cause messages to pile up in worst cases or to be processed far too late.
How to reproduce
Possible to reproduce this by SIGKILL'ing a batch worker with messages stored. I hope that my explanation is good enough and a reproducer is not needed.
Possible Solution
Should be possible to fix this by moving nextClaim update to a different place. The field should not be updated if there is a pending message with the same consumer (already does) or a message from another consumer was claimed (this bug).
Additional Context
No response