This is not specific to SQS bridge, but the same implementation for all adapters.

What else can we do?

• The retry mechanism needs to decode the message to be able to retry it.
• if we stop deleting the message, it will be re-consumed infinitely

Since decoding is not exactly reverse of encoding with arrays (int vs string keys), legitimate messages end up being dropped

I don't get your point. If the serializer/unserializer messed up with a payload leading to a corrupted message, then the payload will never be unserializable. You can drop the message because there is no case you'll be able to restore your corrupted message.

In my opinion, there is only one case where legitimate messages can be dropped:
The publisher uses a different version of code than the consumer, and the encoded message refers to a class that does not exist in the consumer, in that way, the consumer won't be able YET to unserialize the payload.

But IMHO this is a generic issue with multi-servers applications that does not only impact messenger (you have the same issue with the session, or shared cache, or API calls, or database schema...). This problem should be solved in the way you release your code by either:

• upgrading all servers in the same time (with downtime)
• or using a migration path to provide code compatible with both versions at the same time

I have the same problem with AMQP.

Normally, the failed message would be routed to the dead-letter queue but because of the unhandled exception this doesn't happen. I'd love to be able to route problem messages to a dead-letter queue. Perhaps we could wrap the decoding exception with a try..catch and dispatch a MessageHandlingFailed event so that these edge-cases can be handled by users as they see fit?

It would also be beneficial to provide the failure transports somehow, right now you can't really get them in a service/listener...

Also, at least in my case, what fails is decoding the message body -- the headers (read: stamps) are decoded without problem so having access to the failure transport + encoded message is good enough to route it to a dead letter queue.

To illustrate the use case: we have a message with an object ID + status and a consumer which will update the object with the new status. Now, if there's a mistake in the consumer code, these messages are gone forever, however, if they were rerouted to a queue we could collect them and fix the objects.

Beyond anything else, this show some incoherences in messenger. A validation failure, as tested by the validation middleware, would send the message to the deadletter queue, even retrying it. A validation failure based on something else, such as assertions or the type system itself, would lead to a deserialization error and the message would be dropped.
I would expect any issue not caused by the transport to be handled the same way. Serialization is not a transport concern. Invalid serialized data for the application is still valid data for the transport. Serialization error should be treated as application errors and sent to a deadletter queue without retry.

Hey, thanks for your report!
There has not been a lot of activity here for a while. Is this bug still relevant? Have you managed to find a workaround?

Still a real issue.

I will work on a fix. Which version should I target? 5.4 and 6.*??

I have the same problem.

I'm using an AMQP transport and custom serializer to read messages from other symfony apps. So if I decode a message that I am not ready to consume, e.g. a bug or a new message version, MessageDecodingFailedException removes the message and is not sent to failure transport.

maybe is there any way to send the message to failure transport?

@alex-dev Can you work on this feature? You can read #39622 for some help

Yes. Just need time at my job.

Hey, thanks for your report!
There has not been a lot of activity here for a while. Is this bug still relevant? Have you managed to find a workaround?

Considering there is a PR....

@lyrixx I read what you did and from what I understand it would still be possible to lose messages upon other unserialization issues. For example, a field has been renamed and thus does not exist anymore.

Maybe we could have a completly new Envelop that would wrap the original raw envelop that was queued. This envelop would be marked with the Stamp MessageDecodingFailedStamp::class.

Do you have any other idea to fix that issue? Do you think the proposed solution would work?

Proposed solution dealt with every issue except unserializing the envelope itself. Which is what PhpSerializer was doing anyway. Handling envelope unserialization itself is not easily doable as each transport can transit envelope in its own way (SQS uses MessageAttributes and payload, redis serialize it all together). It is also less likely to cause an issue unlike message unserialization.
It also move handling unserialization failure to the worker so it can be passed through the proper events.
It does leverage MessageDecodingFailedStamp and genealize its use.

Thanks for the feedback. If we can do something simpler that handles every cases it would be great though.

What do you think about what I proposed? Simply wrapping the original raw data, a bit like you did but we don't need to deal with PHP Serialization format.

I think you should check the implementation itself. Only place I deal with PHP serialization itself, is to standardize features between serializers.
Main issue with just wrapping the raw data, is you lose all stamps. Which can cause issues depending on how the component evolves. It's better to ensure everything works the same for all implementation of SerializerInterface

@alex-dev I checked out your branch but it does not work for my case where I have a class not found for the Message of the Envelop. I will dig more. Maybe related to PHP 8.2.

@alex-dev It works actually, I was serializing this thing the wrong way 👍

Thanks!

If you updated my code to work with newer Symfony, could you open a PR yourself? I closed mine because I could not justify the time to keep with Symfony changes and fix conflicts without guarantee someone would merge.

@alex-dev @lyrixx I have a PoC in preparation that proposes to add a new configuration option in Messenger:

framework:
  messenger:
     failure_message_decoding_transport: failed

What do you think about such a solution?

For the implementation: I am going to add new event MessageDecodingFailedEvent which will enable extensibility and provide the new behavior mentioned above.

What would be the other options? throw or crash? Why would any sane person select that?
I worry about addiing configurations but for essentially behaviour correction that end up digging pretty deep. If you implement the whole thing, and just throw instead of dispatching after having built the full, failing, envelope.... It would be simple enough that I think it could work if wee can imagine a use case for it.

The goal is to keep BC while letting developers opting in this new behavior. So if the option is not set it would crash just like now.

This feels like https://xkcd.com/1172. Not sure I'd bother. but if you want to go the extra mile. Go for it I guess.

I'm facing the same issue: when a MessageDecodingFailedException is thrown while decoding a message, the message is not passed to the retry mechanism. Instead, the consumer crashes and the message is lost. @B-Galati were you able to submit a PR with your PoC?

@julienfalque I abandoned the subject, not on priority anymore sorry :/

I implemented a new transport with my idea but it's hacky.
Here is the code of the custom AmqpReceiver:

    private function getEnvelope(string $queueName): iterable
    {
        try {
            $amqpEnvelope = $this->connection->get($queueName);
        } catch (\AMQPException $exception) {
            throw new TransportException($exception->getMessage(), 0, $exception);
        }

        if (null === $amqpEnvelope) {
            return;
        }

        $body = $amqpEnvelope->getBody();

        try {
            $envelope = $this->serializer->decode([
                'body'    => $body,
                'headers' => $amqpEnvelope->getHeaders(),
            ]);
        } catch (\Throwable $exception) {
            $this->logger->warning(
                'Rejected message because decoding failed. It will be sent to the default failure transport.',
                ['exception' => $exception]
            );

            $message = new \stdClass();
            $message->rawBody = $body;
            $message->rawHeaders = $amqpEnvelope->getHeaders();
            $envelope = new Envelope($message, [
                new MessageDecodingFailedStamp(),
                ErrorDetailsStamp::create($exception),
                new SentToFailureTransportStamp($queueName),
                new DelayStamp(0),
                new RedeliveryStamp(0),
            ]);

            $this->defaultFailureSender->send($envelope);

            // invalid message of some type
            $this->rejectAmqpEnvelope($amqpEnvelope, $queueName);

            return;
        }

        yield $envelope->with(new AmqpReceivedStamp($amqpEnvelope, $queueName));
    }

In Symfony it could be implemented with a Listener I guess. That would be cleaner.