Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Commit a0cefaa

Browse files
nicolas-grekasnicolas-grekas
committed
feature #34217 [Messenger] use events consistently in worker (Tobion)
This PR was merged into the 4.4 branch. Discussion ---------- [Messenger] use events consistently in worker | Q | A | ------------- | --- | Branch? | 4.4 | Bug fix? | no | New feature? | yes | Deprecations? | no <!-- please update UPGRADE-*.md and src/**/CHANGELOG.md files --> | Tickets | Fix #32560, #32614, #33843 | License | MIT | Doc PR | The worker had the three ways to handle events 1. $onHandledCallback in `run(array $options = [], callable $onHandledCallback = null)` 2. events dispatched using the event dispatcher 3. hardcoded things inside the worker This PR refactores the messenger worker to only use event dispatching. So instead of a hardcoded `$onHandledCallback` and worker decorators, we use event listeners and we don't need a `WorkerInterface` at all. The behavior of all the options like `--memory-limit` etc remains the same. I introduced two new events - `WorkerStartedEvent` - `WorkerRunningEvent` Together with the existing `WorkerStoppedEvent` it's very symmetrical and solves the referenced issues. Commits ------- 201f159 [Messenger] use events consistently in worker
2 parents 72dd176 + 201f159 commit a0cefaa

38 files changed

+746
-714
lines changed

‎UPGRADE-4.4.md

Lines changed: 7 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -168,13 +168,17 @@ Lock
168168
Messenger
169169
---------
170170

171-
* Deprecated passing a `ContainerInterface` instance as first argument of the `ConsumeMessagesCommand` constructor,
172-
pass a `RoutableMessageBus` instance instead.
173171
* [BC BREAK] Removed `SendersLocatorInterface::getSenderByAlias` added in 4.3.
174172
* [BC BREAK] Removed `$retryStrategies` argument from `Worker::__construct`.
175-
* [BC BREAK] Removed `$retryStrategyLocator` argument from `ConsumeMessagesCommand::__construct`.
173+
* [BC BREAK] Changed arguments of `ConsumeMessagesCommand::__construct`.
176174
* [BC BREAK] Removed `$senderClassOrAlias` argument from `RedeliveryStamp::__construct`.
177175
* [BC BREAK] Removed `UnknownSenderException`.
176+
* [BC BREAK] Removed `WorkerInterface`.
177+
* [BC BREAK] Removed `$onHandledCallback` of `Worker::run(array $options = [], callable $onHandledCallback = null)`.
178+
* [BC BREAK] Removed `StopWhenMemoryUsageIsExceededWorker` in favor of `StopWorkerOnMemoryLimitListener`.
179+
* [BC BREAK] Removed `StopWhenMessageCountIsExceededWorker` in favor of `StopWorkerOnMessageLimitListener`.
180+
* [BC BREAK] Removed `StopWhenTimeLimitIsReachedWorker` in favor of `StopWorkerOnTimeLimitListener`.
181+
* [BC BREAK] Removed `StopWhenRestartSignalIsReceived` in favor of `StopWorkerOnRestartSignalListener`.
178182
* Marked the `MessengerDataCollector` class as `@final`.
179183

180184
Mime

‎src/Symfony/Bundle/FrameworkBundle/Resources/config/console.xml

Lines changed: 1 addition & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -88,12 +88,9 @@
8888
<service id="console.command.messenger_consume_messages" class="Symfony\Component\Messenger\Command\ConsumeMessagesCommand">
8989
<argument /> <!-- Routable message bus -->
9090
<argument type="service" id="messenger.receiver_locator" />
91+
<argument type="service" id="event_dispatcher" />
9192
<argument type="service" id="logger" on-invalid="null" />
9293
<argument type="collection" /> <!-- Receiver names -->
93-
<argument type="service" id="event_dispatcher" />
94-
<call method="setCachePoolForRestartSignal">
95-
<argument type="service" id="cache.messenger.restart_workers_signal" />
96-
</call>
9794

9895
<tag name="console.command" command="messenger:consume" />
9996
<tag name="console.command" command="messenger:consume-messages" />

‎src/Symfony/Bundle/FrameworkBundle/Resources/config/messenger.xml

Lines changed: 16 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -98,6 +98,7 @@
9898
<argument /> <!-- max delay ms -->
9999
</service>
100100

101+
<!-- worker event listeners -->
101102
<service id="messenger.retry.send_failed_message_for_retry_listener" class="Symfony\Component\Messenger\EventListener\SendFailedMessageForRetryListener">
102103
<tag name="kernel.event_subscriber" />
103104
<tag name="monolog.logger" channel="messenger" />
@@ -106,14 +107,28 @@
106107
<argument type="service" id="logger" on-invalid="ignore" />
107108
</service>
108109

109-
<!-- failed handling -->
110110
<service id="messenger.failure.send_failed_message_to_failure_transport_listener" class="Symfony\Component\Messenger\EventListener\SendFailedMessageToFailureTransportListener">
111111
<tag name="kernel.event_subscriber" />
112112
<tag name="monolog.logger" channel="messenger" />
113113
<argument /> <!-- Failure transport -->
114114
<argument type="service" id="logger" on-invalid="ignore" />
115115
</service>
116116

117+
<service id="messenger.listener.dispatch_pcntl_signal_listener" class="Symfony\Component\Messenger\EventListener\DispatchPcntlSignalListener">
118+
<tag name="kernel.event_subscriber" />
119+
</service>
120+
121+
<service id="messenger.listener.stop_worker_on_restart_signal_listener" class="Symfony\Component\Messenger\EventListener\StopWorkerOnRestartSignalListener">
122+
<tag name="kernel.event_subscriber" />
123+
<tag name="monolog.logger" channel="messenger" />
124+
<argument type="service" id="cache.messenger.restart_workers_signal" />
125+
<argument type="service" id="logger" on-invalid="ignore" />
126+
</service>
127+
128+
<service id="messenger.listener.stop_worker_on_sigterm_signal_listener" class="Symfony\Component\Messenger\EventListener\StopWorkerOnSigtermSignalListener">
129+
<tag name="kernel.event_subscriber" />
130+
</service>
131+
117132
<!-- routable message bus -->
118133
<service id="messenger.routable_message_bus" class="Symfony\Component\Messenger\RoutableMessageBus">
119134
<argument /> <!-- Message bus locator -->

‎src/Symfony/Component/Messenger/CHANGELOG.md

Lines changed: 8 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -4,18 +4,23 @@ CHANGELOG
44
4.4.0
55
-----
66

7-
* Deprecated passing a `ContainerInterface` instance as first argument of the `ConsumeMessagesCommand` constructor,
8-
pass a `RoutableMessageBus` instance instead.
97
* Added support for auto trimming of Redis streams.
108
* `InMemoryTransport` handle acknowledged and rejected messages.
119
* Made all dispatched worker event classes final.
1210
* Added support for `from_transport` attribute on `messenger.message_handler` tag.
1311
* Added support for passing `dbindex` as a query parameter to the redis transport DSN.
12+
* Added `WorkerStartedEvent` and `WorkerRunningEvent`
1413
* [BC BREAK] Removed `SendersLocatorInterface::getSenderByAlias` added in 4.3.
1514
* [BC BREAK] Removed `$retryStrategies` argument from `Worker::__construct`.
16-
* [BC BREAK] Removed `$retryStrategyLocator` argument from `ConsumeMessagesCommand::__construct`.
15+
* [BC BREAK] Changed arguments of `ConsumeMessagesCommand::__construct`.
1716
* [BC BREAK] Removed `$senderClassOrAlias` argument from `RedeliveryStamp::__construct`.
1817
* [BC BREAK] Removed `UnknownSenderException`.
18+
* [BC BREAK] Removed `WorkerInterface`.
19+
* [BC BREAK] Removed `$onHandledCallback` of `Worker::run(array $options = [], callable $onHandledCallback = null)`.
20+
* [BC BREAK] Removed `StopWhenMemoryUsageIsExceededWorker` in favor of `StopWorkerOnMemoryLimitListener`.
21+
* [BC BREAK] Removed `StopWhenMessageCountIsExceededWorker` in favor of `StopWorkerOnMessageLimitListener`.
22+
* [BC BREAK] Removed `StopWhenTimeLimitIsReachedWorker` in favor of `StopWorkerOnTimeLimitListener`.
23+
* [BC BREAK] Removed `StopWhenRestartSignalIsReceived` in favor of `StopWorkerOnRestartSignalListener`.
1924
* The component is not marked as `@experimental` anymore.
2025
* Marked the `MessengerDataCollector` class as `@final`.
2126

‎src/Symfony/Component/Messenger/Command/ConsumeMessagesCommand.php

Lines changed: 12 additions & 30 deletions
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,6 @@
1111

1212
namespace Symfony\Component\Messenger\Command;
1313

14-
use Psr\Cache\CacheItemPoolInterface;
1514
use Psr\Container\ContainerInterface;
1615
use Psr\Log\LoggerInterface;
1716
use Symfony\Component\Console\Command\Command;
@@ -23,13 +22,12 @@
2322
use Symfony\Component\Console\Output\OutputInterface;
2423
use Symfony\Component\Console\Question\ChoiceQuestion;
2524
use Symfony\Component\Console\Style\SymfonyStyle;
25+
use Symfony\Component\EventDispatcher\EventDispatcherInterface;
26+
use Symfony\Component\Messenger\EventListener\StopWorkerOnMemoryLimitListener;
27+
use Symfony\Component\Messenger\EventListener\StopWorkerOnMessageLimitListener;
28+
use Symfony\Component\Messenger\EventListener\StopWorkerOnTimeLimitListener;
2629
use Symfony\Component\Messenger\RoutableMessageBus;
2730
use Symfony\Component\Messenger\Worker;
28-
use Symfony\Component\Messenger\Worker\StopWhenMemoryUsageIsExceededWorker;
29-
use Symfony\Component\Messenger\Worker\StopWhenMessageCountIsExceededWorker;
30-
use Symfony\Component\Messenger\Worker\StopWhenRestartSignalIsReceived;
31-
use Symfony\Component\Messenger\Worker\StopWhenTimeLimitIsReachedWorker;
32-
use Symfony\Contracts\EventDispatcher\EventDispatcherInterface;
3331

3432
/**
3533
* @author Samuel Roze <[email protected]>
@@ -43,13 +41,11 @@ class ConsumeMessagesCommand extends Command
4341
private $logger;
4442
private $receiverNames;
4543
private $eventDispatcher;
46-
/** @var CacheItemPoolInterface|null */
47-
private $restartSignalCachePool;
4844

4945
/**
5046
* @param RoutableMessageBus $routableBus
5147
*/
52-
public function __construct($routableBus, ContainerInterface $receiverLocator, LoggerInterface $logger = null, array $receiverNames = [], /* EventDispatcherInterface */ $eventDispatcher = null)
48+
public function __construct($routableBus, ContainerInterface $receiverLocator, EventDispatcherInterface $eventDispatcher, LoggerInterface $logger = null, array $receiverNames = [])
5349
{
5450
if ($routableBus instanceof ContainerInterface) {
5551
@trigger_error(sprintf('Passing a "%s" instance as first argument to "%s()" is deprecated since Symfony 4.4, pass a "%s" instance instead.', ContainerInterface::class, __METHOD__, RoutableMessageBus::class), E_USER_DEPRECATED);
@@ -58,12 +54,6 @@ public function __construct($routableBus, ContainerInterface $receiverLocator, L
5854
throw new \TypeError(sprintf('The first argument must be an instance of "%s".', RoutableMessageBus::class));
5955
}
6056

61-
if (null !== $eventDispatcher && !$eventDispatcher instanceof EventDispatcherInterface) {
62-
@trigger_error(sprintf('The 5th argument of the class "%s" should be a "%s"', __CLASS__, EventDispatcherInterface::class), E_USER_DEPRECATED);
63-
64-
$eventDispatcher = null;
65-
}
66-
6757
$this->routableBus = $routableBus;
6858
$this->receiverLocator = $receiverLocator;
6959
$this->logger = $logger;
@@ -73,11 +63,6 @@ public function __construct($routableBus, ContainerInterface $receiverLocator, L
7363
parent::__construct();
7464
}
7565

76-
public function setCachePoolForRestartSignal(CacheItemPoolInterface $restartSignalCachePool)
77-
{
78-
$this->restartSignalCachePool = $restartSignalCachePool;
79-
}
80-
8166
/**
8267
* {@inheritdoc}
8368
*/
@@ -177,29 +162,23 @@ protected function execute(InputInterface $input, OutputInterface $output)
177162
$receivers[$receiverName] = $this->receiverLocator->get($receiverName);
178163
}
179164

180-
$bus = $input->getOption('bus') ? $this->routableBus->getMessageBus($input->getOption('bus')) : $this->routableBus;
181-
182-
$worker = new Worker($receivers, $bus, $this->eventDispatcher, $this->logger);
183165
$stopsWhen = [];
184166
if ($limit = $input->getOption('limit')) {
185167
$stopsWhen[] = "processed {$limit} messages";
186-
$worker = new StopWhenMessageCountIsExceededWorker($worker, $limit, $this->logger);
168+
$this->eventDispatcher->addSubscriber(new StopWorkerOnMessageLimitListener($limit, $this->logger));
187169
}
188170

189171
if ($memoryLimit = $input->getOption('memory-limit')) {
190172
$stopsWhen[] = "exceeded {$memoryLimit} of memory";
191-
$worker = new StopWhenMemoryUsageIsExceededWorker($worker, $this->convertToBytes($memoryLimit), $this->logger);
173+
$this->eventDispatcher->addSubscriber(new StopWorkerOnMemoryLimitListener($this->convertToBytes($memoryLimit), $this->logger));
192174
}
193175

194176
if ($timeLimit = $input->getOption('time-limit')) {
195177
$stopsWhen[] = "been running for {$timeLimit}s";
196-
$worker = new StopWhenTimeLimitIsReachedWorker($worker, $timeLimit, $this->logger);
178+
$this->eventDispatcher->addSubscriber(new StopWorkerOnTimeLimitListener($timeLimit, $this->logger));
197179
}
198180

199-
if (null !== $this->restartSignalCachePool) {
200-
$stopsWhen[] = 'received a stop signal via the messenger:stop-workers command';
201-
$worker = new StopWhenRestartSignalIsReceived($worker, $this->restartSignalCachePool, $this->logger);
202-
}
181+
$stopsWhen[] = 'received a stop signal via the messenger:stop-workers command';
203182

204183
$io = new SymfonyStyle($input, $output instanceof ConsoleOutputInterface ? $output->getErrorOutput() : $output);
205184
$io->success(sprintf('Consuming messages from transport%s "%s".', \count($receivers) > 0 ? 's' : '', implode(', ', $receiverNames)));
@@ -216,6 +195,9 @@ protected function execute(InputInterface $input, OutputInterface $output)
216195
$io->comment('Re-run the command with a -vv option to see logs about consumed messages.');
217196
}
218197

198+
$bus = $input->getOption('bus') ? $this->routableBus->getMessageBus($input->getOption('bus')) : $this->routableBus;
199+
200+
$worker = new Worker($receivers, $bus, $this->eventDispatcher, $this->logger);
219201
$worker->run([
220202
'sleep' => $input->getOption('sleep') * 1000000,
221203
]);

‎src/Symfony/Component/Messenger/Command/FailedMessagesRetryCommand.php

Lines changed: 7 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -20,8 +20,8 @@
2020
use Symfony\Component\Console\Output\OutputInterface;
2121
use Symfony\Component\Console\Style\SymfonyStyle;
2222
use Symfony\Component\EventDispatcher\EventDispatcherInterface;
23-
use Symfony\Component\Messenger\Envelope;
2423
use Symfony\Component\Messenger\Event\WorkerMessageReceivedEvent;
24+
use Symfony\Component\Messenger\EventListener\StopWorkerOnMessageLimitListener;
2525
use Symfony\Component\Messenger\Exception\LogicException;
2626
use Symfony\Component\Messenger\MessageBusInterface;
2727
use Symfony\Component\Messenger\Transport\Receiver\ListableReceiverInterface;
@@ -87,6 +87,8 @@ protected function configure(): void
8787
*/
8888
protected function execute(InputInterface $input, OutputInterface $output)
8989
{
90+
$this->eventDispatcher->addSubscriber(new StopWorkerOnMessageLimitListener(1));
91+
9092
$io = new SymfonyStyle($input, $output instanceof ConsoleOutputInterface ? $output->getErrorOutput() : $output);
9193
$io->comment('Quit this command with CONTROL-C.');
9294
if (!$output->isVeryVerbose()) {
@@ -158,7 +160,9 @@ private function runInteractive(SymfonyStyle $io, bool $shouldForce)
158160

159161
private function runWorker(ReceiverInterface $receiver, SymfonyStyle $io, bool $shouldForce): int
160162
{
161-
$listener = function (WorkerMessageReceivedEvent $messageReceivedEvent) use ($io, $receiver, $shouldForce) {
163+
$count = 0;
164+
$listener = function (WorkerMessageReceivedEvent $messageReceivedEvent) use ($io, $receiver, $shouldForce, &$count) {
165+
++$count;
162166
$envelope = $messageReceivedEvent->getEnvelope();
163167

164168
$this->displaySingleMessage($envelope, $io);
@@ -181,14 +185,8 @@ private function runWorker(ReceiverInterface $receiver, SymfonyStyle $io, bool $
181185
$this->logger
182186
);
183187

184-
$count = 0;
185188
try {
186-
$worker->run([], function (?Envelope $envelope) use ($worker, &$count) {
187-
++$count;
188-
if (null === $envelope) {
189-
$worker->stop();
190-
}
191-
});
189+
$worker->run();
192190
} finally {
193191
$this->eventDispatcher->removeListener(WorkerMessageReceivedEvent::class, $listener);
194192
}

‎src/Symfony/Component/Messenger/Command/StopWorkersCommand.php

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,7 @@
1717
use Symfony\Component\Console\Output\ConsoleOutputInterface;
1818
use Symfony\Component\Console\Output\OutputInterface;
1919
use Symfony\Component\Console\Style\SymfonyStyle;
20-
use Symfony\Component\Messenger\Worker\StopWhenRestartSignalIsReceived;
20+
use Symfony\Component\Messenger\EventListener\StopWorkerOnRestartSignalListener;
2121

2222
/**
2323
* @author Ryan Weaver <[email protected]>
@@ -63,7 +63,7 @@ protected function execute(InputInterface $input, OutputInterface $output)
6363
{
6464
$io = new SymfonyStyle($input, $output instanceof ConsoleOutputInterface ? $output->getErrorOutput() : $output);
6565

66-
$cacheItem = $this->restartSignalCachePool->getItem(StopWhenRestartSignalIsReceived::RESTART_REQUESTED_TIMESTAMP_KEY);
66+
$cacheItem = $this->restartSignalCachePool->getItem(StopWorkerOnRestartSignalListener::RESTART_REQUESTED_TIMESTAMP_KEY);
6767
$cacheItem->set(microtime(true));
6868
$this->restartSignalCachePool->save($cacheItem);
6969

‎src/Symfony/Component/Messenger/DependencyInjection/MessengerPass.php

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -276,7 +276,7 @@ private function registerReceivers(ContainerBuilder $container, array $busIds)
276276
$consumeCommandDefinition->replaceArgument(0, new Reference('messenger.routable_message_bus'));
277277
}
278278

279-
$consumeCommandDefinition->replaceArgument(3, array_values($receiverNames));
279+
$consumeCommandDefinition->replaceArgument(4, array_values($receiverNames));
280280
}
281281

282282
if ($container->hasDefinition('console.command.messenger_setup_transports')) {

‎src/Symfony/Component/Messenger/Event/WorkerRunningEvent.php

Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,44 @@
1+
<?php
2+
3+
/*
4+
* This file is part of the Symfony package.
5+
*
6+
* (c) Fabien Potencier <[email protected]>
7+
*
8+
* For the full copyright and license information, please view the LICENSE
9+
* file that was distributed with this source code.
10+
*/
11+
12+
namespace Symfony\Component\Messenger\Event;
13+
14+
use Symfony\Component\Messenger\Worker;
15+
16+
/**
17+
* Dispatched after the worker processed a message or didn't receive a message at all.
18+
*
19+
* @author Tobias Schultze <http://tobion.de>
20+
*/
21+
final class WorkerRunningEvent
22+
{
23+
private $worker;
24+
private $isWorkerIdle;
25+
26+
public function __construct(Worker $worker, bool $isWorkerIdle)
27+
{
28+
$this->worker = $worker;
29+
$this->isWorkerIdle = $isWorkerIdle;
30+
}
31+
32+
public function getWorker(): Worker
33+
{
34+
return $this->worker;
35+
}
36+
37+
/**
38+
* Returns true when no message has been received by the worker.
39+
*/
40+
public function isWorkerIdle(): bool
41+
{
42+
return $this->isWorkerIdle;
43+
}
44+
}

‎src/Symfony/Component/Messenger/Event/WorkerStartedEvent.php

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
<?php
2+
3+
/*
4+
* This file is part of the Symfony package.
5+
*
6+
* (c) Fabien Potencier <[email protected]>
7+
*
8+
* For the full copyright and license information, please view the LICENSE
9+
* file that was distributed with this source code.
10+
*/
11+
12+
namespace Symfony\Component\Messenger\Event;
13+
14+
use Symfony\Component\Messenger\Worker;
15+
16+
/**
17+
* Dispatched when a worker has been started.
18+
*
19+
* @author Tobias Schultze <http://tobion.de>
20+
*/
21+
final class WorkerStartedEvent
22+
{
23+
private $worker;
24+
25+
public function __construct(Worker $worker)
26+
{
27+
$this->worker = $worker;
28+
}
29+
30+
public function getWorker(): Worker
31+
{
32+
return $this->worker;
33+
}
34+
}

‎src/Symfony/Component/Messenger/Event/WorkerStoppedEvent.php

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -11,11 +11,24 @@
1111

1212
namespace Symfony\Component\Messenger\Event;
1313

14+
use Symfony\Component\Messenger\Worker;
15+
1416
/**
1517
* Dispatched when a worker has been stopped.
1618
*
1719
* @author Robin Chalas <[email protected]>
1820
*/
1921
final class WorkerStoppedEvent
2022
{
23+
private $worker;
24+
25+
public function __construct(Worker $worker)
26+
{
27+
$this->worker = $worker;
28+
}
29+
30+
public function getWorker(): Worker
31+
{
32+
return $this->worker;
33+
}
2134
}

0 commit comments

Comments
 (0)