Thanks to visit codestin.com
Credit goes to github.com

Skip to content

[Messenger] Reduce lock time when using MySQL for transport #60207

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 5 commits into
base: 7.4
Choose a base branch
from
Open

[Messenger] Reduce lock time when using MySQL for transport #60207

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
bc1fbeb
[Messenger][Doctrine][Transport]
JanPaulBeumer Apr 13, 2025
ef84582
Update src/Symfony/Component/Messenger/Bridge/Doctrine/Transport/Conn…
JanPaulBeumer Apr 20, 2025
110629a
Update src/Symfony/Component/Messenger/Bridge/Doctrine/Transport/Conn…
JanPaulBeumer Apr 20, 2025
4259f68
Update src/Symfony/Component/Messenger/Bridge/Doctrine/Transport/Conn…
JanPaulBeumer Apr 20, 2025
d5db0bf
[Messenger][Doctrine][Transport]
JanPaulBeumer Apr 20, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
110 changes: 75 additions & 35 deletions src/Symfony/Component/Messenger/Bridge/Doctrine/Tests/Transport/ConnectionTest.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -668,46 +668,41 @@ public function testGeneratedSql(AbstractPlatform $platform, string $expectedSql
$connection->get();
}

public static function providePlatformSql(): iterable
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it'd be nice to reduce the diff on this file, doable?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tbh no. i changed the algorithm and the queries. i tried to, but that was the best i could do here on the unit test

/**
* @dataProvider providePlatformMySQLSql
*/
public function testGeneratedMySQLSqlOnGet(AbstractPlatform $platform, string $expectedSql, string $expectedUpdateSql)
{
yield 'MySQL' => [
class_exists(MySQLPlatform::class) ? new MySQLPlatform() : new MySQL57Platform(),
'SELECT m.* FROM messenger_messages m WHERE (m.queue_name = ?) AND (m.delivered_at is null OR m.delivered_at < ?) AND (m.available_at <= ?) ORDER BY available_at ASC LIMIT 1 FOR UPDATE',
];

if (class_exists(MySQL80Platform::class) && !method_exists(QueryBuilder::class, 'forUpdate')) {
yield 'MySQL8 & DBAL<3.8' => [
new MySQL80Platform(),
'SELECT m.* FROM messenger_messages m WHERE (m.queue_name = ?) AND (m.delivered_at is null OR m.delivered_at < ?) AND (m.available_at <= ?) ORDER BY available_at ASC LIMIT 1 FOR UPDATE',
];
}

if (class_exists(MySQL80Platform::class) && method_exists(QueryBuilder::class, 'forUpdate')) {
yield 'MySQL8 & DBAL>=3.8' => [
new MySQL80Platform(),
'SELECT m.* FROM messenger_messages m WHERE (m.queue_name = ?) AND (m.delivered_at is null OR m.delivered_at < ?) AND (m.available_at <= ?) ORDER BY available_at ASC LIMIT 1 FOR UPDATE SKIP LOCKED',
];
}
$driverConnection = $this->createMock(DBALConnection::class);
$driverConnection->method('getDatabasePlatform')->willReturn($platform);
$driverConnection->method('createQueryBuilder')->willReturnCallback(fn () => new QueryBuilder($driverConnection));

yield 'MariaDB' => [
new MariaDBPlatform(),
'SELECT m.* FROM messenger_messages m WHERE (m.queue_name = ?) AND (m.delivered_at is null OR m.delivered_at < ?) AND (m.available_at <= ?) ORDER BY available_at ASC LIMIT 1 FOR UPDATE',
];
$result = $this->createMock(Result::class);
$result->method('fetchFirstColumn')->willReturn(['id' => 3]);

if (interface_exists(DBALException::class)) {
// DBAL 4+
$mariaDbPlatformClass = 'Doctrine\DBAL\Platforms\MariaDB1060Platform';
} else {
$mariaDbPlatformClass = 'Doctrine\DBAL\Platforms\MariaDb1060Platform';
}
$driverConnection
->expects($this->once())
->method('executeQuery')
->with($this->callback(function ($sql) use ($expectedSql) {
return trim($expectedSql) === trim($sql);
}))
->willReturn($result)
;
$driverConnection
->expects($this->once())
->method('executeStatement')
->with($this->callback(function ($sql) use ($expectedUpdateSql) {
return trim($expectedUpdateSql) === trim($sql);
}))
->willReturn(0)
;

if (class_exists($mariaDbPlatformClass)) {
yield 'MariaDB106' => [
new $mariaDbPlatformClass(),
'SELECT m.* FROM messenger_messages m WHERE (m.queue_name = ?) AND (m.delivered_at is null OR m.delivered_at < ?) AND (m.available_at <= ?) ORDER BY available_at ASC LIMIT 1 FOR UPDATE SKIP LOCKED',
];
}
$connection = new Connection([], $driverConnection);
$connection->get();
}

public static function providePlatformSql(): iterable
{
if (class_exists(MySQL57Platform::class)) {
yield 'Postgres & DBAL<4' => [
new PostgreSQLPlatform(),
Expand Down Expand Up @@ -773,6 +768,51 @@ class_exists(SQLServerPlatform::class) && !class_exists(SQLServer2012Platform::c
}
}

public static function providePlatformMySQLSql(): iterable
{
yield 'MySQL' => [
class_exists(MySQLPlatform::class) ? new MySQLPlatform() : new MySQL57Platform(),
'SELECT id FROM messenger_messages m WHERE (m.queue_name = ?) AND (m.delivered_at is null OR m.delivered_at < ?) AND (m.available_at <= ?) ORDER BY available_at ASC LIMIT 100',
'UPDATE messenger_messages SET delivered_at = :now WHERE (id = :id) AND (queue_name = :queue_name) AND (delivered_at is null OR delivered_at < :redeliver_imit)',
];

if (class_exists(MySQL80Platform::class) && !method_exists(QueryBuilder::class, 'forUpdate')) {
yield 'MySQL8 & DBAL<3.8' => [
new MySQL80Platform(),
'SELECT id FROM messenger_messages m WHERE (m.queue_name = ?) AND (m.delivered_at is null OR m.delivered_at < ?) AND (m.available_at <= ?) ORDER BY available_at ASC LIMIT 100',
'UPDATE messenger_messages SET delivered_at = :now WHERE (id = :id) AND (queue_name = :queue_name) AND (delivered_at is null OR delivered_at < :redeliver_imit)',
];
}

if (class_exists(MySQL80Platform::class) && method_exists(QueryBuilder::class, 'forUpdate')) {
yield 'MySQL8 & DBAL>=3.8' => [
new MySQL80Platform(),
'SELECT id FROM messenger_messages m WHERE (m.queue_name = ?) AND (m.delivered_at is null OR m.delivered_at < ?) AND (m.available_at <= ?) ORDER BY available_at ASC LIMIT 100',
'UPDATE messenger_messages SET delivered_at = :now WHERE (id = :id) AND (queue_name = :queue_name) AND (delivered_at is null OR delivered_at < :redeliver_imit)',
];
}
yield 'MariaDB' => [
new MariaDBPlatform(),
'SELECT id FROM messenger_messages m WHERE (m.queue_name = ?) AND (m.delivered_at is null OR m.delivered_at < ?) AND (m.available_at <= ?) ORDER BY available_at ASC LIMIT 100',
'UPDATE messenger_messages SET delivered_at = :now WHERE (id = :id) AND (queue_name = :queue_name) AND (delivered_at is null OR delivered_at < :redeliver_imit)',
];

if (interface_exists(DBALException::class)) {
// DBAL 4+
$mariaDbPlatformClass = 'Doctrine\DBAL\Platforms\MariaDB1060Platform';
} else {
$mariaDbPlatformClass = 'Doctrine\DBAL\Platforms\MariaDb1060Platform';
}

if (class_exists($mariaDbPlatformClass)) {
yield 'MariaDB106' => [
new $mariaDbPlatformClass(),
'SELECT id FROM messenger_messages m WHERE (m.queue_name = ?) AND (m.delivered_at is null OR m.delivered_at < ?) AND (m.available_at <= ?) ORDER BY available_at ASC LIMIT 100',
'UPDATE messenger_messages SET delivered_at = :now WHERE (id = :id) AND (queue_name = :queue_name) AND (delivered_at is null OR delivered_at < :redeliver_imit)',
];
}
}

public function testConfigureSchema()
{
$driverConnection = $this->getDBALConnectionMock();
Expand Down
122 changes: 116 additions & 6 deletions src/Symfony/Component/Messenger/Bridge/Doctrine/Transport/Connection.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@

namespace Symfony\Component\Messenger\Bridge\Doctrine\Transport;

use Doctrine\DBAL\ArrayParameterType;
use Doctrine\DBAL\Connection as DBALConnection;
use Doctrine\DBAL\Driver\Exception as DriverException;
use Doctrine\DBAL\Exception as DBALException;
Expand Down Expand Up @@ -158,15 +159,17 @@ public function send(string $body, array $headers, int $delay = 0): string

public function get(): ?array
{
if ($this->doMysqlCleanup && $this->driverConnection->getDatabasePlatform() instanceof AbstractMySQLPlatform) {
if ($this->driverConnection->getDatabasePlatform() instanceof AbstractMySQLPlatform) {
if ($this->doMysqlCleanup) {
$this->deleteDeliveredMessageForMySQLPlatform();
}
try {
$this->driverConnection->delete($this->configuration['table_name'], ['delivered_at' => '9999-12-31 23:59:59']);
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this made an exclusive lock on more than a row or record level. this is problematic since it blocks all other processes

$this->doMysqlCleanup = false;
} catch (DriverException $e) {
// Ignore the exception
} catch (TableNotFoundException $e) {
return $this->getMessageForMySQLPlatform();
} catch (TableNotFoundException) {
if ($this->autoSetup) {
$this->setup();

return $this->getMessageForMySQLPlatform();
}
}
}
Expand Down Expand Up @@ -627,4 +630,111 @@ private function fallBackToForUpdate(QueryBuilder $query, string $sql): string
return $sql;
}
}

private function getMessageForMySQLPlatform(): ?array
{
$possibleIdsToClaim = $this->createAvailableMessagesQueryBuilder()
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fetch only ids till a message is claimed to not load all the payloads unnecessary (they can be huge)

->select('id')
->orderBy('available_at', 'ASC')
->setMaxResults(100)->fetchFirstColumn();

if (!$possibleIdsToClaim) {
$this->queueEmptiedAt = microtime(true) * 1000;

return null;
}
$this->queueEmptiedAt = null;

$claimedId = null;
foreach ($possibleIdsToClaim as $id) {
if (null === $claimedId = $this->claimMessage($id)) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if (null === $claimedId = $this->claimMessage($id)) {
if (null !== $claimedId = $this->claimMessage($id)) {

break;
}
}

if (!null === $claimedId) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if (!null === $claimedId) {
if (null === $claimedId) {

Typo i gues

// all open messages already have been claimed by other workers.
return null;
}

$messageData = $this->createQueryBuilder()
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

load the data only for the claimed message

->andWhere('m.id = :id')
->setParameter(':id', $claimedId)
->fetchAssociative();

return $this->decodeEnvelopeHeaders($messageData);
}

/**
* @throws DBALException
*/
private function deleteDeliveredMessageForMySQLPlatform(): void
{
try {
$ids = $this->selectMessageIdsToDelete();
$this->driverConnection->createQueryBuilder()
->delete($this->configuration['table_name'])
->where('id IN (:ids)')
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

row/record level exclusive lock to not interfere with the selecting part

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

would it work to replace this by a subquery instead of doing a roundtrip to get the ids?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

id rather not use subqueries. they are very often a cause of performance flaws

->setParameter('ids', $ids, ArrayParameterType::INTEGER)
->executeQuery()
;
$this->doMysqlCleanup = false;
} catch (DriverException $e) {
// Ignore the exception
} catch (TableNotFoundException $e) {
if ($this->autoSetup) {
$this->setup();
}
}
}

/**
* @return array<int, mixed>
*
* @throws DBALException
*/
private function selectMessageIdsToDelete(): array
{
return $this->driverConnection->createQueryBuilder()
->select('m.id')
->from($this->configuration['table_name'], 'm')
->where('m.queue_name = ?')
->andWhere('m.delivered_at = ?')
->setParameters([
$this->configuration['queue_name'],
'9999-12-31 23:59:59',
], [
Types::STRING,
Types::STRING,
])
->setMaxResults(5_000)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what if there are more?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no problem. each time one message gets claimed 5k are being deleted.

->fetchFirstColumn();
}

private function claimMessage(mixed $id): mixed
{
$now = new \DateTimeImmutable('UTC');
$redeliverLimit = $now->modify(\sprintf('-%d seconds', $this->configuration['redeliver_timeout']));

$claimed = 0 < $this->driverConnection->createQueryBuilder()
->update($this->configuration['table_name'])
->set('delivered_at', ':now')
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this will invoke an exclusive lock on row/record level to ensure we wont have race conditions and multiple workers handling the same message.

either we can update the message, that means the message id has not been updated (delivered_at)
or we wont find the message to update and go on with the next message id we can try

->andWhere('id = :id')
->andWhere('queue_name = :queue_name')
->andWhere('delivered_at is null OR delivered_at < :redeliver_imit')
->setParameters([
'now' => $now,
'id' => $id,
'queue_name' => $this->configuration['queue_name'],
'redeliver_imit' => $redeliverLimit,
], [
'now' => Types::DATETIME_IMMUTABLE,
'id' => Types::INTEGER,
'queue_name' => Types::STRING,
'redeliver_imit' => Types::DATETIME_IMMUTABLE,
])
->executeStatement();

return $claimed ? $id : null;
}
}
Loading