Mailer return path is not set correctly #41322

peteruu · 2021-05-20T09:01:14Z

Symfony version(s) affected: 4.4.16

Description
When we create a email with defined ->returnPath('[email protected]') email is delivered but in email source we can see that return path was replaced by ->from('[email protected]') We tried to upgrade this component to v5.0.6 and also v5.0.9 both versions are working correctly. WE was using our server DSN and also tried this setting: MAILER_DSN=sendmail://default without change

How to reproduce
Just sent a email with setup "returnPath" and returnPath should be in your mailbox replaced with "from"

Possible Solution
probably something in suggested versions

xabbuh · 2021-05-21T14:20:12Z

To clarify: Do you mean that the issue does only exist in 4.4 and has been fixed in 5.1?

peteruu · 2021-05-21T16:04:18Z

exist in 4.4 fixed in 5.0.6

peteruu · 2021-06-17T07:44:40Z

Any help ?

carsonbot · 2021-12-18T13:04:07Z

Hey, thanks for your report!
There has not been a lot of activity here for a while. Is this bug still relevant? Have you managed to find a workaround?

peteruu · 2021-12-20T09:32:02Z

Hi no i did not find a workaround and it is still relevant. Some help would be really nice because as I said in new version it is fixed.

xabbuh · 2021-12-20T15:26:09Z

Would you be able to contribute a test case for the Mailer component that succeeds with 5.1 but fails on 4.4? That could make it easier for someone else to contribute a fix for this issue.

peteruu · 2021-12-21T09:21:03Z

Sorry but i don't have such a skills i only tried it with such a versions of mailer component and I know where it works and where it doesn't that is all, but I can not update to 5.1 because of other packages in my project

peteruu · 2022-02-24T11:44:57Z

Hi i find out it was last time working in 5.1.3.
In 5.1.4 it is not working and I think I find out the bug.
In component symfony/mailer there is file DelayedEnvelope.php and at the end is this function

private static function getSenderFromHeaders(Headers $headers): Address
    {
        if ($sender = $headers->get('Sender')) {
            return $sender->getAddress();
        }
        if ($from = $headers->get('From')) {
            return $from->getAddresses()[0];
        }
        if ($return = $headers->get('Return-Path')) {
            return $return->getAddress();
        }

        throw new LogicException('Unable to determine the sender of the message.');
    }

Correct one is this

     private static function getSenderFromHeaders(Headers $headers): Address
    {
        if ($return = $headers->get('Return-Path')) {
            return $return->getAddress();
        }
        if ($sender = $headers->get('Sender')) {
            return $sender->getAddress();
        }
        if ($from = $headers->get('From')) {
            return $from->getAddresses()[0];
        }


        throw new LogicException('Unable to determine the sender of the message.');
    }

as you can see it is only changed the order... Correct one is how it was in 5.1.3 I tried to change it in 5.1.4 (and also 5.4.3) manually and it is working. I don't know what it does exactly but changing order helped.
So if you can anybody merge it with at least 5.4.3 version it would be great.

mailer info was ignoring return-path setting more info here symfony#41322

fabpot · 2022-02-25T07:32:00Z

I don't get the issue. The method you are referring to does not change anything to the Return-Path, it uses its value for the sender. As the Return-Path is used for SPF, many providers are actually overriding the Return-Path automatically.

peteruu · 2022-02-25T11:49:33Z

hi issue is we wanted to use return path because of bounced emails. as i know that is the reason why this exists, ir am i wrong? but now it is always replaced by sender email. Dňa pi 25. 2. 2022, 8:32 Fabien Potencier ***@***.***> napísal(a):

…

I don't get the issue. The method you are referring to does not change anything to the Return-Path, it uses its value for the sender. As the Return-Path is used for SPF, many providers are actually overriding the Return-Path automatically. — Reply to this email directly, view it on GitHub <#41322 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ALEENRIIN6C3HHT7DOCQEZDU44V7XANCNFSM45GQL7WQ> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>. You are receiving this because you authored the thread.Message ID: ***@***.***>

fabpot · 2022-02-26T10:28:31Z

Are you sure that the Return-Path header is not overridden by your email provider?

peteruu · 2022-02-26T11:19:03Z

i think it is not. because when i change order in that function as i suggested i can see it in source of the email. so i think it is bug on symfony side not my provider Dňa so 26. 2. 2022, 11:28 Fabien Potencier ***@***.***> napísal(a):

…

Are you sure that the Return-Path header is not overridden by your email provider? — Reply to this email directly, view it on GitHub <#41322 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ALEENRKL2FRTBQJNOM6PCKTU5CTNVANCNFSM45GQL7WQ> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>. You are receiving this because you authored the thread.Message ID: ***@***.***>

sarahnekam · 2022-03-01T11:15:43Z

Hello,

we have discovered the same behavior as peteruu said.

When we set the return path via $mail->returnPath(), it is ignored. The return path in the email header is set to the from property.

We didn't set the from property because we want to receive bounce mails to our bounce mail address.

The fix from peteruu seems to be correct. Perhaps it would be helpful if there was a comment on this function describing the order of the properties (returnPath, sender, from) so that they remain the same in the future.

Is there a plan to include the fix in a release?

cscrewsandcaptains · 2022-03-01T13:17:29Z

Hi,

After switching from swiftmailer to symfony mailer, the problem appeared. We have not changed anything else.

To clarify: The function that peteruu suggests to change is indirectly called to set the parameter -f in the sendmail command (which corresponds to the sender in the sendmail documentation).

It may be that sendmail then replaces the email header "Return-Path" with this address (this seems correct to me).

Current workaround: Set "Sender", leads to: "Sender" and "From" being displayed in the email programme :(

If this function is used elsewhere, it may be necessary to create a second function that corresponds to the function of peteruu.

peteruu · 2022-03-30T08:59:50Z

@fabpot so as you can see also other people face the same issue. Is there any idea how to make it work?

cscrewsandcaptains · 2022-03-30T09:42:11Z

peteruu I think your fix is not quite correct. Please refer to my last comment on your pull request (#45545).

fabpot · 2022-03-30T10:05:24Z

In Swiftmailer, the order is what we had initially in Symfony Mailer: "return path" then "sender" then "from".

As we never had any complaints about this in Swiftmailer, I suppose that reverting #45545 might be the "best" solution.

fabpot · 2022-03-30T10:05:57Z

I would love to have references from other mailer libraries (PHP or any other language) to check what they are doing.

tpetry · 2022-04-11T09:53:21Z

I've experienced the bug also by switching from Swiftmailer to Symfony Mailer. The pull request #45545 did change the behaviour, and Swiftmailer was more correct.

The problem is some mail servers will ignore the Return-Path header in the message. They will always use the envelope address, the email address used in the smtp connection in the MAIL FROM command. As #45545 did change the ordering making the From header more important than the Return-Path header, the From address is now used for these mail servers. Because of this change the other people and me are getting incorrect Return-Path headers in the mail message for some mail servers.

The correct ordering would either be Sender > Return-Path > From or Return-Path > Sender > From. In my opinion Sender > Return-Path > From is the correct ordering, but simply reverting #45545 is also ok to fix the current bug for us:

From is used as a fallback when nothing is defined: ✓
Return-Path is used when defined for the mail servers overwriting the header: ✓
Sender is used when defined because the developer specifically stated to use a specific sender address in the smtp protocol: ✓

I found these references on how to order the addresses correct:

Wikipedia states that the MAIL FROM address is either Return-Path or Sender [Reference]
Pear Mail used Return-Path > From[Reference]
PHPMailer uses Sender > From [Reference]
nodemailer does not apply any logic, the first header in an array with a From, Reply-To or Sender is used [Reference]
emailjs uses Return-Path > From [Reference]

…ddress (fixes symfony#41322)

…ess than From address (tpetry) This PR was merged into the 4.4 branch. Discussion ---------- [Mailer] Return-Path has higher priority for envelope address than From address | Q | A | ------------- | --- | Branch? | 4.4 | Bug fix? | yes | New feature? | no | Deprecations? | no | Tickets | Fix #41322 | License | MIT | Doc PR | - Commits ------- 4945059 fix: return-path has higher priority for envelope address than from address (fixes #41322)

peteruu · 2022-04-12T07:23:12Z

Can you please put it also into 5.4 branch. There is the same problem.

tpetry · 2022-04-12T11:49:37Z

Won't the fix be ported to newer versions somehow in the process @fabpot? I was asked to provide the bugfix to 4.4 instead of 6.x which I am using. I expected the bug to trickle-up in the release process, is this valid?

stof · 2022-04-12T12:05:03Z

yeah, the fix will be merged up by the core team in the next merge up

* 4.4: [Workflow] Catch error when trying to get an uninitialized marking Add missing license header Use reference date in reverse transform Fixes #40997 Fix env resolution in lock configuration Fix Symfony not working on SMB share #45990 [Cache] make LockRegistry use static properties instead of static variables fix: return-path has higher priority for envelope address than from address (fixes #41322) [HttpClient] Fix sending content-length when streaming the body [Console] Header with column max width is now well wrap with separator [DependencyInjection] Add TaggedIteratorArgument unit tests

* 5.4: (21 commits) Add missing license header [Workflow] Catch error when trying to get an uninitialized marking Add missing license header Allow usage of Provider domains if possible Use reference date in reverse transform Fixes #40997 Fix env resolution in lock configuration Fix Symfony not working on SMB share #45990 [Messenger] DoctrineTransportFactory works with notify and decorated PostgreSQL driver [Cache] make LockRegistry use static properties instead of static variables fix: return-path has higher priority for envelope address than from address (fixes #41322) [HttpClient] Fix sending content-length when streaming the body [Console] Header with column max width is now well wrap with separator Fix use_cookies framework session configuration [FrameworkBundle] [Command] Fix `debug:router --no-interaction` error … [Intl] Update the ICU data to 71.1 - 5.4 [Intl] Update the ICU data to 71.1 - 4.4 Add tests to messenger connection get for OraclePlatform [RateLimiter] Adding default empty value [DependencyInjection] Add TaggedIteratorArgument unit tests [Process] Fix Process::getEnv() when setEnv() hasn't been called before ...

* 6.0: (22 commits) Add missing license header Add missing license header [Workflow] Catch error when trying to get an uninitialized marking Add missing license header Allow usage of Provider domains if possible Use reference date in reverse transform Fixes #40997 Fix env resolution in lock configuration Fix Symfony not working on SMB share #45990 [Messenger] DoctrineTransportFactory works with notify and decorated PostgreSQL driver [Cache] make LockRegistry use static properties instead of static variables fix: return-path has higher priority for envelope address than from address (fixes #41322) [HttpClient] Fix sending content-length when streaming the body [Console] Header with column max width is now well wrap with separator Fix use_cookies framework session configuration [FrameworkBundle] [Command] Fix `debug:router --no-interaction` error … [Intl] Update the ICU data to 71.1 - 5.4 [Intl] Update the ICU data to 71.1 - 4.4 Add tests to messenger connection get for OraclePlatform [RateLimiter] Adding default empty value [DependencyInjection] Add TaggedIteratorArgument unit tests ...

imclean557 · 2022-09-27T22:20:27Z

@tpetry, just a note that PHPMailer dropped support for the Return-path header in 2014.

PHPMailer uses Sender > From [Reference]

I don't think that is PHPMailer. Adding the return-path header at the point of sending contravenes the RFCs:

Because of this, PHPMailer no longer sets the header:

czhDavid · 2023-01-13T16:47:10Z

Just to maybe save someone some headache. This broke our application as Sparkpost is using Return-path for bounced emails which are different then the header.

* 4.4: [Workflow] Catch error when trying to get an uninitialized marking Add missing license header Use reference date in reverse transform Fixes symfony#40997 Fix env resolution in lock configuration Fix Symfony not working on SMB share symfony#45990 [Cache] make LockRegistry use static properties instead of static variables fix: return-path has higher priority for envelope address than from address (fixes symfony#41322) [HttpClient] Fix sending content-length when streaming the body [Console] Header with column max width is now well wrap with separator [DependencyInjection] Add TaggedIteratorArgument unit tests

* 5.4: (21 commits) Add missing license header [Workflow] Catch error when trying to get an uninitialized marking Add missing license header Allow usage of Provider domains if possible Use reference date in reverse transform Fixes symfony#40997 Fix env resolution in lock configuration Fix Symfony not working on SMB share symfony#45990 [Messenger] DoctrineTransportFactory works with notify and decorated PostgreSQL driver [Cache] make LockRegistry use static properties instead of static variables fix: return-path has higher priority for envelope address than from address (fixes symfony#41322) [HttpClient] Fix sending content-length when streaming the body [Console] Header with column max width is now well wrap with separator Fix use_cookies framework session configuration [FrameworkBundle] [Command] Fix `debug:router --no-interaction` error … [Intl] Update the ICU data to 71.1 - 5.4 [Intl] Update the ICU data to 71.1 - 4.4 Add tests to messenger connection get for OraclePlatform [RateLimiter] Adding default empty value [DependencyInjection] Add TaggedIteratorArgument unit tests [Process] Fix Process::getEnv() when setEnv() hasn't been called before ...

* 6.0: (22 commits) Add missing license header Add missing license header [Workflow] Catch error when trying to get an uninitialized marking Add missing license header Allow usage of Provider domains if possible Use reference date in reverse transform Fixes symfony#40997 Fix env resolution in lock configuration Fix Symfony not working on SMB share symfony#45990 [Messenger] DoctrineTransportFactory works with notify and decorated PostgreSQL driver [Cache] make LockRegistry use static properties instead of static variables fix: return-path has higher priority for envelope address than from address (fixes symfony#41322) [HttpClient] Fix sending content-length when streaming the body [Console] Header with column max width is now well wrap with separator Fix use_cookies framework session configuration [FrameworkBundle] [Command] Fix `debug:router --no-interaction` error … [Intl] Update the ICU data to 71.1 - 5.4 [Intl] Update the ICU data to 71.1 - 4.4 Add tests to messenger connection get for OraclePlatform [RateLimiter] Adding default empty value [DependencyInjection] Add TaggedIteratorArgument unit tests ...

peteruu added the Bug label May 20, 2021

carsonbot added the Status: Needs Review label May 20, 2021

xabbuh added Mailer Status: Waiting feedback labels May 21, 2021

carsonbot added the Stalled label Dec 18, 2021

carsonbot removed the Stalled label Dec 20, 2021

peteruu mentioned this issue Feb 24, 2022

Fixing return-path symfony/mailer#7

Closed

peteruu added a commit to peteruu/symfony that referenced this issue Feb 24, 2022

fixed bug with return-path

af005e8

mailer info was ignoring return-path setting more info here symfony#41322

peteruu mentioned this issue Feb 24, 2022

fixed bug with return-path #45545

Closed

tpetry added a commit to mailspice/symfony that referenced this issue Apr 11, 2022

fix: return-path has higher priority for envelope address than from a…

c5724db

…ddress (fixes symfony#41322)

tpetry added a commit to mailspice/symfony that referenced this issue Apr 11, 2022

fix: return-path has higher priority for envelope address than from a…

b2e75df

…ddress (fixes symfony#41322)

tpetry mentioned this issue Apr 11, 2022

[Mailer] Return-Path has higher priority for envelope address than From address #45992

Merged

tpetry added a commit to mailspice/symfony that referenced this issue Apr 11, 2022

fix: return-path has higher priority for envelope address than from a…

4c98940

…ddress (fixes symfony#41322)

tpetry added a commit to mailspice/symfony that referenced this issue Apr 11, 2022

fix: return-path has higher priority for envelope address than from a…

7dd5c60

…ddress (fixes symfony#41322)

tpetry added a commit to mailspice/symfony that referenced this issue Apr 12, 2022

fix: return-path has higher priority for envelope address than from a…

4945059

…ddress (fixes symfony#41322)

nicolas-grekas closed this as completed Apr 12, 2022

joeschi mentioned this issue Sep 22, 2022

Return_path not working anymore kartolo/direct_mail#251

Closed

Patta mentioned this issue Mar 8, 2023

ReturnPath not working anymore in2code-de/powermail#668

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Mailer return path is not set correctly #41322

Mailer return path is not set correctly #41322

peteruu commented May 20, 2021

xabbuh commented May 21, 2021

peteruu commented May 21, 2021

peteruu commented Jun 17, 2021

carsonbot commented Dec 18, 2021

peteruu commented Dec 20, 2021

xabbuh commented Dec 20, 2021

peteruu commented Dec 21, 2021

peteruu commented Feb 24, 2022

fabpot commented Feb 25, 2022

peteruu commented Feb 25, 2022 via email

fabpot commented Feb 26, 2022

peteruu commented Feb 26, 2022 via email

sarahnekam commented Mar 1, 2022

cscrewsandcaptains commented Mar 1, 2022

peteruu commented Mar 30, 2022

cscrewsandcaptains commented Mar 30, 2022

fabpot commented Mar 30, 2022

fabpot commented Mar 30, 2022

tpetry commented Apr 11, 2022

peteruu commented Apr 12, 2022

tpetry commented Apr 12, 2022

stof commented Apr 12, 2022

imclean557 commented Sep 27, 2022

czhDavid commented Jan 13, 2023

Mailer return path is not set correctly #41322

Mailer return path is not set correctly #41322

Comments

peteruu commented May 20, 2021

xabbuh commented May 21, 2021

peteruu commented May 21, 2021

peteruu commented Jun 17, 2021

carsonbot commented Dec 18, 2021

peteruu commented Dec 20, 2021

xabbuh commented Dec 20, 2021

peteruu commented Dec 21, 2021

peteruu commented Feb 24, 2022

fabpot commented Feb 25, 2022

peteruu commented Feb 25, 2022 via email

fabpot commented Feb 26, 2022

peteruu commented Feb 26, 2022 via email

sarahnekam commented Mar 1, 2022

cscrewsandcaptains commented Mar 1, 2022

peteruu commented Mar 30, 2022

cscrewsandcaptains commented Mar 30, 2022

fabpot commented Mar 30, 2022

fabpot commented Mar 30, 2022

tpetry commented Apr 11, 2022

peteruu commented Apr 12, 2022

tpetry commented Apr 12, 2022

stof commented Apr 12, 2022

imclean557 commented Sep 27, 2022

czhDavid commented Jan 13, 2023