Thanks to visit codestin.com
Credit goes to github.com

Skip to content

[Messenger] Issues after migrating to AWS RabbitMQ #48241

@xorik

Description

@xorik

Symfony version(s) affected

6.0

Description

We are switching from normal RabbitMQ server, to AWS-managed cluster.

After that periodically we have this error in log:

(AMQPException): Library error: a SSL error occurred

It happens on line $exchange->publish(...) in \Symfony\Component\Messenger\Bridge\Amqp\Transport\Connection::publishOnExchange

The problem is that after some time, the connection becomes "dead", but method Connection::clearWhenDisconnected doesn't detect is as disconnected.

It happened after 350 seconds, because they have some idle timeout.

I've provided a small script, that reproduces the issue: https://gist.github.com/xorik/24065aab0db094c99e746b277466fcba

Here's output:

Channel is connected!
Published!
Now wait for 360 seconds....
Channel is connected!
Exception (AMQPException): Library error: a SSL error occurred
#0 /tmp/1.php(37): AMQPExchange->publish('howdy-do', 'hello')
#1 {main}

I can give you credentials to a test server, if you want (PM https://twitter.com/xorik_dev/ or give me your contact).

I've tried to make persistent connection, adding heartbeat, but it won't help.

This is probably connected with #36538, but suggested fix doesn't help.

How to reproduce

  • Connect to a AWS RabbitMQ
  • publish a message
  • wait 350+ seconds
  • publish a message

Possible Solution

  • Actually the connection isn't 100% dead, so reconnect is not required and this code actually fixes the issue:
try {
    $exchange->publish($message, $routing_key);
} catch (\AMQPException) {
    $exchange->publish($message, $routing_key);
}
  • Alternative solution is perform some kind of ping or heartbeat, before publishing a message (improve method clearWhenDisconnected)

Additional Context

This is clearly not a Symfony issue, but I think Messenger should handle this issue, since this AWS RabbitMQ is quite popular, and there is no easy way to workaround it by developers, except creating a new transport and copy+pasting or extending bunch of Symfony classes.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions