[Mailer] Add advanced logger #37570

Warxcell · 2020-07-14T07:38:01Z

Description
Add advanced debug logger. Just like in Doctrine you can see every single SQL sent to server, it would be useful to be able to see same for mailer. Currently the only option to read it to override the MessageHandler (when used in conjunction with Messenger)

Example

fabpot · 2020-07-14T08:16:15Z

I think that you are looking for this: https://symfony.com/doc/current/mailer.html#debugging-emails

Warxcell · 2020-07-14T08:18:18Z

Yeah I know that - my idea is to integrate it in the Framework Bundle.

Currently this is what I did for debugging:

<?php

declare(strict_types=1);

namespace App\Mail;

use Psr\Log\LoggerInterface;
use Symfony\Component\Mailer\Exception\TransportException;
use Symfony\Component\Mailer\Messenger\SendEmailMessage;
use Symfony\Component\Mailer\Transport\TransportInterface;
use Symfony\Component\Messenger\Handler\MessageHandlerInterface;

class MessageHandler implements MessageHandlerInterface
{
    private TransportInterface $transport;
    private LoggerInterface $logger;

    public function __construct(TransportInterface $transport, LoggerInterface $mailerLogger)
    {
        $this->transport = $transport;
        $this->logger = $mailerLogger;
    }

    public function __invoke(SendEmailMessage $message)
    {
        try {
            $this->transport->send($message->getMessage(), $message->getEnvelope());
        } catch (TransportException $exception) {
            $this->logger->error($exception, ['debug' => $exception->getDebug()]);

            throw $exception;
        }
    }
}

And that overrides: mailer.messenger.message_handler

Warxcell · 2020-07-14T10:20:47Z

And even that's not enough - sometimes the Debug info is missing at all. I had to refactor it like that to get missing debug info.

<?php

declare(strict_types=1);

namespace App\Mail;

use Psr\Log\LoggerInterface;
use Symfony\Component\Mailer\Exception\TransportException;
use Symfony\Component\Mailer\Messenger\SendEmailMessage;
use Symfony\Component\Mailer\Transport\Smtp\EsmtpTransport;
use Symfony\Component\Mailer\Transport\TransportInterface;
use Symfony\Component\Messenger\Handler\MessageHandlerInterface;

class MessageHandler implements MessageHandlerInterface
{
    private TransportInterface $transport;
    private LoggerInterface $logger;

    public function __construct(TransportInterface $transport, LoggerInterface $mailerLogger)
    {
        $this->transport = $transport;
        $this->logger = $mailerLogger;
    }

    public function __invoke(SendEmailMessage $message)
    {
        try {
             $this->transport->send($message->getMessage(), $message->getEnvelope());
        } catch (TransportException $exception) {
            $context = [];
            if ($this->transport instanceof EsmtpTransport) {
                $context['debug'] = $this->transport->getStream()->getDebug();
            } else {
                $context['debug'] = $exception->getDebug();
            }
            $this->logger->error($exception, $context);

            throw $exception;
        }
    }
}

The problem is that I get

Expected response code "250" but got code "235", with message "235 2.7.0 Authentication successful". in /app/vendor/symfony/mailer/Transport/Smtp/SmtpTransport.php:297 
Stack trace:
   #0 /app/vendor/symfony/mailer/Transport/Smtp/SmtpTransport.php(180): Symfony\Component\Mailer\Transport\Smtp\SmtpTransport->assertResponseCode('235 2.7.0 Authe...', Array) 
   #1 /app/vendor/symfony/mailer/Transport/Smtp/EsmtpTransport.php(169): Symfony\Component\Mailer\Transport\Smtp\SmtpTransport->executeCommand('RSET\r\n', Array) 
   #2 /app/vendor/symfony/mailer/Transport/Smtp/EsmtpTransport.php(130): Symfony\Component\Mailer\Transport\Smtp\EsmtpTransport->handleAuth(Array)

Randomly.

Warxcell · 2020-07-14T13:37:25Z

In the end - it was because timeout.
I didn't found easy way to configure this, so I had to make this.

<?php

declare(strict_types=1);

namespace App\Mail;

use Psr\Log\LoggerInterface;
use Symfony\Component\Mailer\Exception\TransportException;
use Symfony\Component\Mailer\Messenger\SendEmailMessage;
use Symfony\Component\Mailer\Transport\Smtp\EsmtpTransport;
use Symfony\Component\Mailer\Transport\Smtp\Stream\SocketStream;
use Symfony\Component\Mailer\Transport\TransportInterface;
use Symfony\Component\Messenger\Handler\MessageHandlerInterface;

class MessageHandler implements MessageHandlerInterface
{
    private TransportInterface $transport;
    private LoggerInterface $logger;

    public function __construct(TransportInterface $transport, LoggerInterface $mailerLogger)
    {
        $this->transport = $transport;
        $this->logger = $mailerLogger;
    }

    public function __invoke(SendEmailMessage $message)
    {
        try {
            if ($this->transport instanceof EsmtpTransport) {
                $stream = $this->transport->getStream();

                if ($stream instanceof SocketStream) {
                    $stream->setTimeout(30);
                }
            }
            $this->transport->send($message->getMessage(), $message->getEnvelope());
        } catch (TransportException $exception) {
            $context = [];
            if ($this->transport instanceof EsmtpTransport) {
                $context['debug'] = $this->transport->getStream()->getDebug();
            } else {
                $context['debug'] = $exception->getDebug();
            }
            $this->logger->error($exception, $context);

            throw $exception;
        }
    }
}

It's good to have these configurable.

C-Duv · 2021-01-03T20:55:47Z

Having the number of bytes sent over the wire could be useful for logging/debugging (from Transport or Mailer?).

carsonbot · 2021-07-04T13:02:36Z

Thank you for this suggestion.
There has not been a lot of activity here for a while. Would you still like to see this feature?

carsonbot · 2021-07-18T13:04:57Z

Just a quick reminder to make a comment on this. If I don't hear anything I'll close this.

C-Duv · 2021-07-18T15:56:15Z

Hi bot, il do would like this feature (number of bytes sent over the wire) 😀.

amenk · 2021-11-21T17:39:09Z

What is also very useful and needed is to log the message ID of the remote system. See also swiftmailer/swiftmailer#913

carsonbot · 2022-05-22T13:00:58Z

Thank you for this suggestion.
There has not been a lot of activity here for a while. Would you still like to see this feature?

carsonbot · 2022-06-05T13:05:57Z

Could I get a reply or should I close this?

amenk · 2022-06-05T13:12:54Z

Still interested, but maybe already solved? I find it very important to log the message ID.

fabpot · 2022-07-27T13:43:19Z

#47080 adds a new FailureMessageEvent that you can listen to, making logging a bit easier.
I would not add this in core as the debug output can be very verbose.

carsonbot · 2023-03-02T11:48:40Z

Thank you for this suggestion.
There has not been a lot of activity here for a while. Would you still like to see this feature?

amenk · 2023-03-02T11:59:06Z

Logging the message ID would still be great - without the need for verbose debug output :-)

oojacoboo · 2023-07-27T01:18:00Z

I'm assuming this would stand-in for Swift_Mailer's loggers.

https://swiftmailer.symfony.com/docs/plugins.html#logger-plugin

Does symfony/mailer not have any logging and only explicit exceptions? What if you want to log success emails as well for auditing records? We can write something for this internally if that's the goal here. Would be nice to have some clarity here though. Moving from Swift_Mailer is requiring quite a bit of dancing to replicate the functionality.

It's also worth mentioning that the Symfony docs aren't super helpful if you're not using the framework.

ghost · 2023-08-04T10:02:45Z

I'm assuming this would stand-in for Swift_Mailer's loggers.

https://swiftmailer.symfony.com/docs/plugins.html#logger-plugin

Does symfony/mailer not have any logging and only explicit exceptions? What if you want to log success emails as well for auditing records? We can write something for this internally if that's the goal here. Would be nice to have some clarity here though. Moving from Swift_Mailer is requiring quite a bit of dancing to replicate the functionality.

It's also worth mentioning that the Symfony docs aren't super helpful if you're not using the framework.

Also stumbled across this issue today.
The Symfony mailer has completely removed very useful features like registering custom plugins.
Previously I could simply add a logger via registerPlugin and then capture the SMTP log and save and evaluate it for debug purposes.

$mailer->registerPlugin(new Swift_Plugins_LoggerPlugin($this->logger));

https://symfony.com/blog/the-end-of-swiftmailer#:~:text=Migrating%20from%20Swiftmailer%20to%20Symfony%20Mailer%20is%20a%20relatively%20easy%20task%20as%20the%20concepts%20are%20the%20same%20between%20the%20two%20projects.

I could not find a replacement for the DependencyContainer either.
It is possible this Hack is not needed anymore.

$instance = Swift_DependencyContainer::getInstance();

		$instance
			->register('mime.charstream')
			->asNewInstanceOf('Sys\\Bundle\\Portal\\Component\\Swift\\CharacterStream\\MbCharacterStream')
			->withDependencies( [ 'mime.characterreaderfactory', 'properties.charset' ] );

What is the new Way of requesting the DSN and MDN with the Symfony-Mailer?
This was also possible with the DSN-Plugin and the MDN was native function by the swiftmailer.

https://github.com/search?q=repo%3Aswiftmailer%2Fswiftmailer+setReadReceiptTo&type=code
https://swiftmailer.symfony.com/docs/messages.html#requesting-a-read-receipt

$mailer->registerPlugin(new DsnPlugin([new NotifyHandler('success,failure')]));

https://gist.github.com/Orgoth/22fb841c6887b1cfbfe58eba79d37642

https://datatracker.ietf.org/doc/html/rfc8098
https://datatracker.ietf.org/doc/html/rfc6533

@fabpot

swiftmailer/swiftmailer#338

oojacoboo · 2023-08-04T15:30:28Z

@Orgoth We ended up using our adapter service for handling this. It'd have been nice to have provided some more clear direction when Swift_Mailer was dropped, as to what the recommended implementation equivalent is for unsupported functionality.

stof · 2023-08-04T15:56:10Z

Well, symfony/mailer is not a drop-in replacement for Swiftmailer. It has a totally different architecture. Swiftmailer plugins indeed do not fit in this architecture. Symfony has different kind of extension points (and maybe new ones would be useful, but they should be requested as dedicated feature request with a use case, not just "I want to be able to use my Swiftmailer plugins" which is a no-go as that's not possible).

Note that the ReadReceiptTo feature is really just about adding the Disposition-Notification-To header in the email. There is no dedicated helper method in the Email class, but adding headers is supported.

stof · 2023-08-04T15:59:31Z

@oojacoboo logging can be implemented using events. We have SentMessageEvent for successful ones and FailureMessageEvent for failures.

ghost · 2023-08-04T16:25:28Z

@stof Thank you for the clarification.
I do not use Symfony it self in the current project.
Therefore the features of the Swiftmailer where very helpfull to use the mailer as standalone.
So to say the Swiftmailer was discontinued without an equivalent replacement of a standalone mailer with the same features.
I have already started to reimplement DSN and MDN today.
Which was not so easy, because for me it is not really clear which type of header I have to use and when, because there are now 6 different functions for it.

Other projects have solved the transition better by pointing out which functions can or should be used instead.
e.g.:
@ deprecated Use $mailer->getHeader()->addMailboxHeader($asd,$asd) instead.

But just setting it completely without a migration guide to pick up the developers is very unattractive.

I just need to explain this to my employer, why it is taking so long and what else needs to be done to get back to the same level as with SwiftMailer.

Edit:
I have already used Symfony in various projects and there are virtually no problems thanks to the direct integration of the mailer, only if you use projects such as the swiftmailer in a non-Symfony projects, then the transition is more than burdensome when the most important functions are simply rationalized away.

In particular the plugin system!

stof · 2023-08-04T17:50:49Z

@Orgoth symfony/mailer is a totally different project than swiftmailer. We don't have a 1:1 mapping of each method. that's why there is no such deprecation warnings on Swifmailer methods.
Btw, if you look at the implementation of the method in Swiftmailer (where I found the header name), you will also find the type of header (you can also find it in the RFC 6533 which defines the header value as being a mailbox list) as the various kinds of headers are not something invented by symfony/mailer but part of the spec: https://github.com/swiftmailer/swiftmailer/blob/47a449111b9ca73d5d5f2933d76b971537f17a80/lib/classes/Swift/Mime/SimpleMessage.php#L500-L505

And symfony/mailer can be used standalone. but we indeed miss the documentation about that (which is mostly about documenting the wiring done to instantiate a Mailer, as done by FrameworkBundle). The documentation of the new component focused on the usage in the framework first and nobody has written a doc about the standalone setup yet.

when the most important functions are simply rationalized away.

In particular the plugin system!

Since the creation of the component, you are literally the first one asking us for a replacement for the setReadReceiptTo method of Swiftmailer. So I would not call that a "most important function".

And we cannot just bring the Swiftmailer plugin system in symfony/mailer. That plugin system was closely related to the architecture of swiftmailer. The available extension points in symfony/mailer are following a different architecture. That's why a previous comment of mine said that if we are missing some extension points to solve some use cases, new feature requests should be opened as dedicated issues explaining that use case so that we can find the appropriate extension point for that.

ghost · 2023-08-07T15:09:36Z

@stof
It is what it is. ;)

For all others who come here and use the mailer as a standalone.

Regarding the advanced logging:

do not use the Class "Mailer"

The background is, with the SwiftMailer still a separate Mailer class was used, this is no longer necessary here.
https://swiftmailer.symfony.com/docs/introduction.html#basic-usage
And the Mailer class is not returning the sentMessage object which is needed for debugging.
https://github.com/symfony/mailer/blob/6.3/Mailer.php#L39

Just do it yourself.
For example:

$transfer = new EsmtpTransport($auth['host'],$auth['port'],null,null,$smtpLogger);
$transfer->setUsername($auth['user']);
$transfer->setPassword($auth['pass']);

$sentMessage  = $transfer->send($message);
$sentMessage->getDebug();
$transport->getStream()->getDebug();

This provides access to all necessary resources for debugging and logging.
A monolog logger can also be passed.
Have a look at the 5th parameter of the EsmtpTransport constructor.

https://github.com/symfony/symfony/blob/6.3/src/Symfony/Component/Mailer/SentMessage.php

https://github.com/symfony/symfony/blob/6.4/src/Symfony/Component/Mailer/Transport/Smtp/Stream/AbstractStream.php#L96

Important, once getDebug has been called to the stream, the parameter is emptied in the Object and is no longer available in a later call to the same function!
Here it would be desirable that a parameter can be passed with that the debug parameter is not cleaned. @stof @fabpot
e.g.:

function getDebug( bool $cleanUp = true );
$stream->getDebug(false);

With the Symfony/Mailer 6.1, it is also possible to implement the DSN-Functionality.
But be aware, it is not possible with the Mailer Version 5.4 without any changes, if you are still stuck with PHP 7.x.
Then you need to backport the changes from 6.1.
see: e3306c4#diff-aa51d5b53fc97b526677eb16df7f72f77b24cc2ea467d84f315515a3ea0af1af

If this is given, then the class "EsmtpTransport" can be extended and the function implemented

For example:
https://gist.github.com/Orgoth/ce9376c8321aefb01f1c0e4119dc5f31

MDN is much simpler:

$headers = $message->getHeaders();
$headers->addMailboxHeader('Disposition-Notification-To',$var);

stof · 2023-08-07T15:26:16Z

The Mailer class is intentionally not returning the SentMessage object because this high-level API is meant to be compatible with async processing of the sending (when instantiating it with a Messenger Bus).
The SentMessage is accessible by event listeners for the SentMessageEvent (which is of course only dispatched if you pass a PSR-14 event dispatcher to the constructor of your transport, not if you pass null)

ghost · 2023-08-07T15:29:40Z

The Mailer class is intentionally not returning the SentMessage object because this high-level API is meant to be compatible with async processing of the sending (when instantiating it with a Messenger Bus). The SentMessage is accessible by event listeners for the SentMessageEvent (which is of course only dispatched if you pass a PSR-14 event dispatcher to the constructor of your transport, not if you pass null)

Yes, this information is also only for users who have used SwiftMailer and the separate Mailer class.
For all others who use the Symfony framework with events etc. this information is irrelevant and can be ignored.

Here I would refer to @Warxcell's post.
#37570 (comment)

stof · 2023-08-07T15:39:51Z

@Orgoth even if you use the mailer standalone, I would still encourage to wire an event dispatcher (note that this does not require using the Symfony one as any PSR-14 implementation is supported). Several features (the Twig integration for TemplatedEmail for instead) rely on those events as those features are implemented as listeners.

ghost · 2023-08-07T15:55:17Z

@stof The Mailer is used in an old project where are not Events, Listeners and so on. PSR-14 is nonexistence there.
It is much simpler to do it direct in my current case and it is marked as todo for later. :)

Update the Project to PSR-14

But since this project is nearly 20 years old and i am only the small sending part, it will take some time. :)
The Project is still not compatible with php 8.0. :(

In all other cases, where the effort is manageable, PSR-14 should be used as recommended by you, in order to use the full potential.
Here I am completely with you.

oojacoboo · 2023-08-12T05:21:29Z

@oojacoboo logging can be implemented using events. We have SentMessageEvent for successful ones and FailureMessageEvent for failures.

@stof I noticed this. But since we're not using Symfony framework or the event system, implementing support for that seemed overkill and not worth it. Maybe the event system is decent, I spent some time looking into it. But, for now we're sticking with our current event system that dispatches events to another node for processing (can support message queues). Subscribing to events is an interesting concept, one I haven't used extensively. But, it seems like it'd invite a whole mess of interwoven dependencies and reliance. Whereas dispatching events to another node that can process them off in a more predictable fashion has far less side-effects. It also comes with the benefit of knowing/debugging exactly what those side effects are for each event. I don't know - maybe I'm missing the ah ha.

carsonbot · 2024-02-13T13:03:43Z

Thank you for this suggestion.
There has not been a lot of activity here for a while. Would you still like to see this feature?

carsonbot · 2024-02-27T13:05:59Z

Just a quick reminder to make a comment on this. If I don't hear anything I'll close this.

carsonbot · 2024-03-12T13:10:59Z

Hey,

I didn't hear anything so I'm going to close it. Feel free to comment if this is still relevant, I can always reopen!

…geEvent` (wkania) This PR was merged into the 6.4 branch. Discussion ---------- [Mailer] Mention the `SentMessageEvent` and `FailedMessageEvent` The symfony/symfony#47080 PR was created to solve the problem of [debugging](symfony/symfony#37570) and to get [information](symfony/symfony#42108) about email after it was sent. The section about [debugging](https://symfony.com/doc/current/mailer.html#debugging-emails) does not mention those events. Added some links for better navigation. For example what debug returns: ``` < 220 ESMTP SYMFONY.COM > EHLO [127.0.0.1] < 250-smtp.symfony.com < 250-PIPELINING < 250-SIZE 157286400 < 250-AUTH PLAIN LOGIN PLAIN LOGIN PLAIN LOGIN < 250-AUTH=PLAIN LOGIN PLAIN LOGIN PLAIN LOGIN < 250-ENHANCEDSTATUSCODES < 250-8BITMIME < 250 SMTPUTF8 > AUTH LOGIN < 334 VXNlcm5hbWU6 > ZXhhbXBsZUBzeW1mb255LmNvbQ== < 334 UGFzc3dvcmQ6 > U3ltcGhvbnkg.aXMgQXdlc29tZQ== < 235 2.7.0 Authentication successful > MAIL FROM:<[email protected]> < 250 2.1.0 Ok > RCPT TO:<[email protected]> < 250 2.1.5 Ok > DATA < 354 End data with <CR><LF>.<CR><LF> > . < 250 OK. ID: a20fb6ebbc54d22b ``` P.S. I see that the checks now can find repeated words: ``` mailer.rst ✘ 1765: The word "the" is used more times in a row. -> ``FailedMessageEvent`` allows acting on the the initial message in case of a failure and some ``` Commits ------- 8e3c1db [Mailer] Mention the SentMessageEvent and FailedMessageEvent in the debug section

xabbuh added Feature Mailer labels Jul 14, 2020

carsonbot added the Stalled label Jul 4, 2021

carsonbot removed the Stalled label Jul 18, 2021

carsonbot added the Stalled label May 22, 2022

carsonbot removed the Stalled label Jun 5, 2022

carsonbot added the Stalled label Mar 2, 2023

carsonbot removed the Stalled label Mar 2, 2023

ghost mentioned this issue Aug 11, 2023

MAILER - Support delivery status notifications #51354

Open

carsonbot added the Stalled label Feb 13, 2024

carsonbot closed this as completed Mar 12, 2024

wkania mentioned this issue Sep 1, 2024

[Mailer] Mention the SentMessageEvent and FailedMessageEvent symfony/symfony-docs#20186

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Mailer] Add advanced logger #37570

[Mailer] Add advanced logger #37570

Warxcell commented Jul 14, 2020 •

edited

Loading

fabpot commented Jul 14, 2020

Warxcell commented Jul 14, 2020 •

edited

Loading

Warxcell commented Jul 14, 2020 •

edited

Loading

Warxcell commented Jul 14, 2020 •

edited

Loading

C-Duv commented Jan 3, 2021

carsonbot commented Jul 4, 2021

carsonbot commented Jul 18, 2021

C-Duv commented Jul 18, 2021

amenk commented Nov 21, 2021

carsonbot commented May 22, 2022

carsonbot commented Jun 5, 2022

amenk commented Jun 5, 2022

fabpot commented Jul 27, 2022

carsonbot commented Mar 2, 2023

amenk commented Mar 2, 2023

oojacoboo commented Jul 27, 2023 •

edited

Loading

ghost commented Aug 4, 2023 •

edited by ghost

Loading

oojacoboo commented Aug 4, 2023

stof commented Aug 4, 2023

stof commented Aug 4, 2023

ghost commented Aug 4, 2023 •

edited by ghost

Loading

stof commented Aug 4, 2023

ghost commented Aug 7, 2023

stof commented Aug 7, 2023

ghost commented Aug 7, 2023 •

edited by ghost

Loading

stof commented Aug 7, 2023

ghost commented Aug 7, 2023 •

edited by ghost

Loading

oojacoboo commented Aug 12, 2023

carsonbot commented Feb 13, 2024

carsonbot commented Feb 27, 2024

carsonbot commented Mar 12, 2024

[Mailer] Add advanced logger #37570

[Mailer] Add advanced logger #37570

Comments

Warxcell commented Jul 14, 2020 • edited Loading

fabpot commented Jul 14, 2020

Warxcell commented Jul 14, 2020 • edited Loading

Warxcell commented Jul 14, 2020 • edited Loading

Warxcell commented Jul 14, 2020 • edited Loading

C-Duv commented Jan 3, 2021

carsonbot commented Jul 4, 2021

carsonbot commented Jul 18, 2021

C-Duv commented Jul 18, 2021

amenk commented Nov 21, 2021

carsonbot commented May 22, 2022

carsonbot commented Jun 5, 2022

amenk commented Jun 5, 2022

fabpot commented Jul 27, 2022

carsonbot commented Mar 2, 2023

amenk commented Mar 2, 2023

oojacoboo commented Jul 27, 2023 • edited Loading

ghost commented Aug 4, 2023 • edited by ghost Loading

oojacoboo commented Aug 4, 2023

stof commented Aug 4, 2023

stof commented Aug 4, 2023

ghost commented Aug 4, 2023 • edited by ghost Loading

stof commented Aug 4, 2023

ghost commented Aug 7, 2023

For all others who come here and use the mailer as a standalone.

Regarding the advanced logging:

stof commented Aug 7, 2023

ghost commented Aug 7, 2023 • edited by ghost Loading

stof commented Aug 7, 2023

ghost commented Aug 7, 2023 • edited by ghost Loading

oojacoboo commented Aug 12, 2023

carsonbot commented Feb 13, 2024

carsonbot commented Feb 27, 2024

carsonbot commented Mar 12, 2024

Warxcell commented Jul 14, 2020 •

edited

Loading

Warxcell commented Jul 14, 2020 •

edited

Loading

Warxcell commented Jul 14, 2020 •

edited

Loading

Warxcell commented Jul 14, 2020 •

edited

Loading

oojacoboo commented Jul 27, 2023 •

edited

Loading

ghost commented Aug 4, 2023 •

edited by ghost

Loading

ghost commented Aug 4, 2023 •

edited by ghost

Loading

ghost commented Aug 7, 2023 •

edited by ghost

Loading

ghost commented Aug 7, 2023 •

edited by ghost

Loading