Thanks to visit codestin.com
Credit goes to github.com

Skip to content

[RFC][Mailer][FrameworkBundle] Add support for rate limited mailer transports #59985

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 4 commits into
base: 7.4
Choose a base branch
from
Open

[RFC][Mailer][FrameworkBundle] Add support for rate limited mailer transports #59985

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
85ae910
First draft
fritzmg Mar 15, 2025
2db50eb
Fix retryDelay
fritzmg Mar 16, 2025
f311ac4
update the CHANGELOG.md files
fritzmg Mar 16, 2025
ff86149
do not use named arguments
fritzmg Mar 16, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions src/Symfony/Bundle/FrameworkBundle/CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ CHANGELOG
* Add support for signal plain name in the `messenger.stop_worker_on_signals` configuration
* Deprecate the `framework.validation.cache` option
* Add `--method` option to the `debug:router` command
* Add support for rate limited mailer transports

7.2
---
Expand Down
16 changes: 15 additions & 1 deletion src/Symfony/Bundle/FrameworkBundle/DependencyInjection/Configuration.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2249,7 +2249,21 @@ private function addMailerSection(ArrayNodeDefinition $rootNode, callable $enabl
->scalarNode('dsn')->defaultNull()->end()
->arrayNode('transports')
->useAttributeAsKey('name')
->prototype('scalar')->end()
->arrayPrototype()
->beforeNormalization()
->ifString()
->then(function (string $dsn) {
return ['dsn' => $dsn];
})
->end()
->children()
->scalarNode('dsn')->end()
->scalarNode('rate_limiter')
->defaultNull()
->info('Rate limiter name to use when sending messages.')
->end()
->end()
->end()
->end()
->arrayNode('envelope')
->info('Mailer Envelope configuration')
Expand Down
21 changes: 20 additions & 1 deletion src/Symfony/Bundle/FrameworkBundle/DependencyInjection/FrameworkExtension.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2781,7 +2781,26 @@ private function registerMailerConfiguration(array $config, ContainerBuilder $co
$config['dsn'] = 'smtp://null';
}
$transports = $config['dsn'] ? ['main' => $config['dsn']] : $config['transports'];
$container->getDefinition('mailer.transports')->setArgument(0, $transports);
$container->getDefinition('mailer.transports')->setArgument(0, array_combine(array_keys($config['transports']), array_column($config['transports'], 'dsn')));

$transportRateLimiterReferences = [];

foreach ($transports as $name => $transport) {
if ($transport['rate_limiter']) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

isset or array_key_exists?

Copy link
Contributor Author

@fritzmg fritzmg Apr 8, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not necessary as the key will always exist due to the processed configuration. See also:

symfony/src/Symfony/Bundle/FrameworkBundle/DependencyInjection/FrameworkExtension.php

Line 2457 in 3ec30b6

if ($transport['rate_limiter']) {

if (!interface_exists(LimiterInterface::class)) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This if could be moved outside the for loop.
The verification can be done just after if $transportRateLimiterReferences is not empty and the interface is missing.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes - however, the same would apply to this existing code:

symfony/src/Symfony/Bundle/FrameworkBundle/DependencyInjection/FrameworkExtension.php

Lines 2458 to 2460 in 3ec30b6

if (!interface_exists(LimiterInterface::class)) {
throw new LogicException('Rate limiter cannot be used within Messenger as the RateLimiter component is not installed. Try running "composer require symfony/rate-limiter".');
}

Do you want me to change it in the unrelated section as well?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

shouldn't this rather check isConfigEnabled() for the rate_limiter config ? the fact that the component is installed does not guarantee that the config is enabled for it (smart defaults are based on the availability of the component, but projects can still configure things explicitly)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am using the same logic as here:

symfony/src/Symfony/Bundle/FrameworkBundle/DependencyInjection/FrameworkExtension.php

Lines 2421 to 2464 in 3ec30b6

$senderAliases = [];
$transportRetryReferences = [];
$transportRateLimiterReferences = [];
foreach ($config['transports'] as $name => $transport) {
$serializerId = $transport['serializer'] ?? 'messenger.default_serializer';
$tags = [
'alias' => $name,
'is_failure_transport' => \in_array($name, $failureTransports, true),
];
if (str_starts_with($transport['dsn'], 'sync://')) {
$tags['is_consumable'] = false;
}
$transportDefinition = (new Definition(TransportInterface::class))
->setFactory([new Reference('messenger.transport_factory'), 'createTransport'])
->setArguments([$transport['dsn'], $transport['options'] + ['transport_name' => $name], new Reference($serializerId)])
->addTag('messenger.receiver', $tags)
;
$container->setDefinition($transportId = 'messenger.transport.'.$name, $transportDefinition);
$senderAliases[$name] = $transportId;
if (null !== $transport['retry_strategy']['service']) {
$transportRetryReferences[$name] = new Reference($transport['retry_strategy']['service']);
} else {
$retryServiceId = \sprintf('messenger.retry.multiplier_retry_strategy.%s', $name);
$retryDefinition = new ChildDefinition('messenger.retry.abstract_multiplier_retry_strategy');
$retryDefinition
->replaceArgument(0, $transport['retry_strategy']['max_retries'])
->replaceArgument(1, $transport['retry_strategy']['delay'])
->replaceArgument(2, $transport['retry_strategy']['multiplier'])
->replaceArgument(3, $transport['retry_strategy']['max_delay'])
->replaceArgument(4, $transport['retry_strategy']['jitter']);
$container->setDefinition($retryServiceId, $retryDefinition);
$transportRetryReferences[$name] = new Reference($retryServiceId);
}
if ($transport['rate_limiter']) {
if (!interface_exists(LimiterInterface::class)) {
throw new LogicException('Rate limiter cannot be used within Messenger as the RateLimiter component is not installed. Try running "composer require symfony/rate-limiter".');
}
$transportRateLimiterReferences[$name] = new Reference('limiter.'.$transport['rate_limiter']);
}
}

Should this not employ the same logic as the rate_limiter config for messenger for consistency?

throw new LogicException('Rate limiter cannot be used within Mailer as the RateLimiter component is not installed. Try running "composer require symfony/rate-limiter".');
}

$transportRateLimiterReferences[$name] = new Reference('limiter.'.$transport['rate_limiter']);
}
}

if (!$transportRateLimiterReferences) {
$container->removeDefinition('mailer.rate_limiter_locator');
} else {
$container->getDefinition('mailer.rate_limiter_locator')
->replaceArgument(0, $transportRateLimiterReferences);
}

$mailer = $container->getDefinition('mailer.mailer');
if (false === $messageBus = $config['message_bus']) {
Expand Down
6 changes: 6 additions & 0 deletions src/Symfony/Bundle/FrameworkBundle/Resources/config/mailer.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@

namespace Symfony\Component\DependencyInjection\Loader\Configurator;

use Symfony\Component\DependencyInjection\ServiceLocator;
use Symfony\Component\Mailer\Command\MailerTestCommand;
use Symfony\Component\Mailer\EventListener\DkimSignedMessageListener;
use Symfony\Component\Mailer\EventListener\EnvelopeListener;
Expand Down Expand Up @@ -49,6 +50,7 @@
->set('mailer.transport_factory', Transport::class)
->args([
tagged_iterator('mailer.transport_factory'),
service('mailer.rate_limiter_locator')->nullOnInvalid(),
])

->set('mailer.default_transport', TransportInterface::class)
Expand Down Expand Up @@ -128,5 +130,9 @@
service('mailer.transports'),
])
->tag('console.command')

->set('mailer.rate_limiter_locator', ServiceLocator::class)
->args([[]])
->tag('container.service_locator')
;
};
1 change: 1 addition & 0 deletions src/Symfony/Component/Mailer/CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ CHANGELOG
* Add DSN param `source_ip` to allow binding to a (specific) IPv4 or IPv6 address.
* Add DSN param `require_tls` to enforce use of TLS/STARTTLS
* Add `DkimSignedMessageListener`, `SmimeEncryptedMessageListener`, and `SmimeSignedMessageListener`
* Add support for rate limited transports

7.2
---
Expand Down
10 changes: 9 additions & 1 deletion src/Symfony/Component/Mailer/Messenger/MessageHandler.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,8 @@

use Symfony\Component\Mailer\SentMessage;
use Symfony\Component\Mailer\Transport\TransportInterface;
use Symfony\Component\Messenger\Exception\RecoverableMessageHandlingException;
use Symfony\Component\RateLimiter\Exception\RateLimitExceededException;

/**
* @author Fabien Potencier <[email protected]>
Expand All @@ -26,6 +28,12 @@ public function __construct(

public function __invoke(SendEmailMessage $message): ?SentMessage
{
return $this->transport->send($message->getMessage(), $message->getEnvelope());
try {
return $this->transport->send($message->getMessage(), $message->getEnvelope());
} catch (RateLimitExceededException $e) {
$retryDelay = ($e->getRetryAfter()->getTimestamp() - time()) * 1000;

throw new RecoverableMessageHandlingException('Rate limit for mailer transport exceeded.', 0, $e, $retryDelay);
}
}
}
19 changes: 17 additions & 2 deletions src/Symfony/Component/Mailer/Transport.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,8 @@

namespace Symfony\Component\Mailer;

use Psr\Container\ContainerInterface;
use Psr\Container\NotFoundExceptionInterface;
use Psr\EventDispatcher\EventDispatcherInterface;
use Psr\Log\LoggerInterface;
use Symfony\Component\Mailer\Bridge\AhaSend\Transport\AhaSendTransportFactory;
Expand All @@ -34,6 +36,7 @@
use Symfony\Component\Mailer\Bridge\Sweego\Transport\SweegoTransportFactory;
use Symfony\Component\Mailer\Exception\InvalidArgumentException;
use Symfony\Component\Mailer\Exception\UnsupportedSchemeException;
use Symfony\Component\Mailer\Transport\AbstractTransport;
use Symfony\Component\Mailer\Transport\Dsn;
use Symfony\Component\Mailer\Transport\FailoverTransport;
use Symfony\Component\Mailer\Transport\NativeTransportFactory;
Expand All @@ -44,6 +47,7 @@
use Symfony\Component\Mailer\Transport\TransportFactoryInterface;
use Symfony\Component\Mailer\Transport\TransportInterface;
use Symfony\Component\Mailer\Transport\Transports;
use Symfony\Component\RateLimiter\RateLimiterFactoryInterface;
use Symfony\Contracts\HttpClient\HttpClientInterface;

/**
Expand Down Expand Up @@ -93,26 +97,37 @@ public static function fromDsns(#[\SensitiveParameter] array $dsns, ?EventDispat
*/
public function __construct(
private iterable $factories,
private ?ContainerInterface $rateLimiterLocator = null,
) {
}

public function fromStrings(#[\SensitiveParameter] array $dsns): Transports
{
$transports = [];
foreach ($dsns as $name => $dsn) {
$transports[$name] = $this->fromString($dsn);
try {
$rateLimiter = $this->rateLimiterLocator?->get($name);
} catch (NotFoundExceptionInterface) {
$rateLimiter = null;
}

$transports[$name] = $this->fromString($dsn, $rateLimiter);
}

return new Transports($transports);
}

public function fromString(#[\SensitiveParameter] string $dsn): TransportInterface
public function fromString(#[\SensitiveParameter] string $dsn, ?RateLimiterFactoryInterface $rateLimiter = null): TransportInterface
{
[$transport, $offset] = $this->parseDsn($dsn);
if ($offset !== \strlen($dsn)) {
throw new InvalidArgumentException('The mailer DSN has some garbage at the end.');
}

if ($rateLimiter && $transport instanceof AbstractTransport) {
$transport->setRateLimiterFactory($rateLimiter);
}

return $transport;
}

Expand Down
15 changes: 15 additions & 0 deletions src/Symfony/Component/Mailer/Transport/AbstractTransport.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,13 +24,15 @@
use Symfony\Component\Mime\Address;
use Symfony\Component\Mime\BodyRendererInterface;
use Symfony\Component\Mime\RawMessage;
use Symfony\Component\RateLimiter\RateLimiterFactoryInterface;

/**
* @author Fabien Potencier <[email protected]>
*/
abstract class AbstractTransport implements TransportInterface
{
private LoggerInterface $logger;
private ?RateLimiterFactoryInterface $rateLimiterFactory = null;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

New argument should be placed after to avoid BC breaks.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm, how is this related to BC? This is just a private member variable - and not a constructor variable with constructor property promotion.

private float $rate = 0;
private float $lastSent = 0;

Expand Down Expand Up @@ -62,10 +64,14 @@ public function send(RawMessage $message, ?Envelope $envelope = null): ?SentMess
{
$message = clone $message;
$envelope = null !== $envelope ? clone $envelope : Envelope::create($message);
$rateLimiter = $this->rateLimiterFactory?->create();

try {
if (!$this->dispatcher) {
$sentMessage = new SentMessage($message, $envelope);

$rateLimiter?->consume(1)->ensureAccepted();

$this->doSend($sentMessage);

return $sentMessage;
Expand All @@ -87,6 +93,8 @@ public function send(RawMessage $message, ?Envelope $envelope = null): ?SentMess
$sentMessage = new SentMessage($message, $envelope);

try {
$rateLimiter?->consume(1)->ensureAccepted();

$this->doSend($sentMessage);
} catch (\Throwable $error) {
$this->dispatcher->dispatch(new FailedMessageEvent($message, $error));
Expand All @@ -103,6 +111,13 @@ public function send(RawMessage $message, ?Envelope $envelope = null): ?SentMess
}
}

public function setRateLimiterFactory(RateLimiterFactoryInterface $rateLimiterFactory): static
{
$this->rateLimiterFactory = $rateLimiterFactory;

return $this;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we should return $this here. It makes the API less clear whether this is a method mutating the instance or returning a new one. Our setters generally don't return $this in Symfony.

Copy link
Contributor Author

@fritzmg fritzmg Apr 8, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The setters in Symfony\Component\Mailer\Transport\AbstractTransport all return $this - so I think we should also return $this here, otherwise it would be inconsistent, would it not?

}

abstract protected function doSend(SentMessage $message): void;

/**
Expand Down
Loading