Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Conversation

@norrs
Copy link
Contributor

@norrs norrs commented Aug 7, 2023

Description

Add tips section to the documentation, and include how I solved ensuring SMTP clients in postfix was bound to the right IP-addresses to match the entries in my SPF-record and reverse DNS (PTR) configuration on those specific IP-addresses.

Tricky part is that when you first involving the smtp_bind_address it also uses this as source IP address for forwarding to content_filter amavis. Since amavis is running on same node, enforce it to bind to localhost.

Interesting topics to maybe look into later:

Maybe you have even more advanced setup and want to look into specific IP-addresses used for particular domains (ie used for bulk-sending) and get some inspiration from https://serverfault.com/a/650326 .

I were happy with only binding to the correct IP-addresses for now as I don't really have any bulk-needs in this setup.

Type of change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Improvement (non-breaking change that does improve existing functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • This change requires a documentation update

Checklist:

  • [? ] My code follows the style guidelines of this project
  • I have performed a self-review of my own code
  • [-] I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation (README.md or the documentation under docs/)
  • [-] If necessary I have added tests that prove my fix is effective or that my feature works
  • [-] New and existing unit tests pass locally with my changes

@georglauterbach georglauterbach added kind/improvement Improve an existing feature, configuration file or the documentation area/documentation labels Aug 8, 2023
@norrs
Copy link
Contributor Author

norrs commented Aug 8, 2023

@polarathene What I'm wondering about is if it should be filed under Examples -> Use cases`` instead of Configuration -> Tips?

Copy link
Member

@polarathene polarathene left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll apply these via the web UI, then pull the changes on your end to apply future feedback.

@polarathene
Copy link
Member

What I'm wondering about is if it should be filed under Examples -> Use cases`` instead of Configuration -> Tips?

Yes, I think this is more niche. It would be better added to our Example docs for now. The config section of our docs is already quite big.

Please move the file and choose a smaller title like "Binding to a specific network". It's possible for DMS to have more than one network attached without using host network mode too, so the advice can still be helpful there.


Thanks for taking the time to contribute to DMS! It's very much appreciated 😁

Copy link
Member

@polarathene polarathene left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, last few changes to address should be good to go now 👍

@polarathene

This comment was marked as resolved.

@norrs
Copy link
Contributor Author

norrs commented Aug 14, 2023

This is rather a very specific and special setup, because I host several things at one node with multiple IP addresses.

Be free to to squash your commits and amend while insert your-self with Co-authored-by: NAME <[email protected]> in the bottom of the commit message @polarathene .

Postfix is configured in DMS to deliver incoming mail and use a content_filter to forward mails with SMTP protocol to the smtp-amavis for scanning. This again is delivered back to postfix again on :10025 via localhost for further processing.

That said smtp_bind_address + smtp_bind_address6 could possibly be set in postfix-master.cf to target the smtp service specifically, instead of Amavis (and anything else that may be affected at some point/configuration), if it's only a relevant setting for the smtp service (outbound mail)?:

This is basically what I'm doing with the custom postfix-master.cf for smtp traffic targeting amavis. (which is used by the content_filter.) See docker exec mailserver postconf -P .

The default DMS configuration for postfix-master.cf could probably contain this default binding to localhost (smtp-amavis/unix/smtp_bind_address = 127.0.0.1), as long as the setup doesn't change where amavis runs on another node (for scaling, ie requires multiple instances of amavis) and postfix requires to communicate with it by non-loopback. However, those who need that and uses DMS, probably know how to update these configuration by using the possibilities of overriding the main and master configs.

@polarathene
Copy link
Member

I'll wrap up this PR and merge it, although your feedback on advice below would be good to hear first :)


This is rather a very specific and special setup, because I host several things at one node with multiple IP addresses.

Is there any reason you use host mode networking for DMS btw? If you want to target a specific interface, I believe you can specify that beside the port mapping in compose.yaml and it will bind only to that IP on the host. Presumably that would also restrict the interface used if the others are not needed in DMS.

If you could test that and let me know that'd be appreciated, we could add it in the docs page too.

However, those who need that and uses DMS, probably know how to update these configuration by using the possibilities of overriding the main and master configs.

Yes, and DMS isn't really maintained for internal services to be scaled out individually (although we do provide a means to disable/enable many of them specifically).

If smtp_bind_address was not already inherited, the smtp-amavis service could just set inet_interfaces=loopback-only I believe. Looking back on master.cf, on the right-most column is the process that will run, while on the left-most is the service definition (named unix socket or inet port); it'd probably be valid to set smtp_bind_address on the smtp inet service with smtp-amavis/unix/smtp_bind_address = your_ip_here, assuming no other service needs the change (every other service in master.cf would inherit the main.cf default smtp_bind_address (empty) then AFAIK):

smtp/inet/smtp_bind_address = 198.51.100.10
smtp/inet/smtp_bind_address6 = 2001:DB8::10

No smtp-amavis (postfix-master.cf) or postfix-main.cf changes would be required if the above works.

As smtp-amavis is for internal use in DMS (and planned to eventually be replaced by rspamd AFAIK), I'm happy to upstream the smtp_bind_address=127.0.0.1 + smtp_bind_address6=::1 settings to the amavis config in postfix-amavis.cf.

@norrs
Copy link
Contributor Author

norrs commented Aug 14, 2023

I'll wrap up this PR and merge it, although your feedback on advice below would be good to hear first :)

This is rather a very specific and special setup, because I host several things at one node with multiple IP addresses.

Is there any reason you use host mode networking for DMS btw? If you want to target a specific interface, I believe you can specify that beside the port mapping in compose.yaml and it will bind only to that IP on the host. Presumably that would also restrict the interface used if the others are not needed in DMS.

If you could test that and let me know that'd be appreciated, we could add it in the docs page too.

I only ended up with host networking due to gmail rejecting emails with spf failures, and then I looked into why it was rejecting. I didn't notice right away that outbound smtp connections randomly bound to the "first" random IP address on the interface.

I started a new job this week, but I can probably test during the weekend if time allows it by temporarying adding a server and configure a domain for testing in https://github.com/norrs/ansible/tree/main/playbooks/mailserver .

As smtp-amavis is for internal use in DMS (and planned to eventually be replaced by rspamd AFAIK), I'm happy to upstream the smtp_bind_address=127.0.0.1 + smtp_bind_address6=::1 settings to the amavis config in postfix-amavis.cf.

I believe those changes should work without issues, even in non host-mode in docker networking. Unsure what happens if you try to configure IPv6 binding and your system doesn't have IPv6 enabled tho..

Also IPv6 in docker probably needs to be configured like mentioned here ( https://docs.docker.com/config/daemon/ipv6/ ), which I actually haven't done myself .. 😨

Which I guess is also why I went direct to networking host-mode in the first place 🙃

Comment on lines 12 to 13
bind outgoing SMTP connections to specific IP-addresses to ensure MX records are aligned with
PTR-records in DNS when sending emails to avoid getting blocked by SPF for example.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe the phrase about SPF should be more clear:

bind outgoing SMTP connections to specific IP-addresses to ensure MX records are aligned with PTR and SPF records in DNS when sending (and receiving emails).

You expect at least the resolving IP-addresses from the MX records in DNS to have reverse DNS (PTR records) matching the MX records.
And those IP-addresses for sending emails need to be specified in the SPF records in DNS. (should be correct for bounces etc as well, so affects receiving as well?)

Agree @polarathene ?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think the PTR records have to match MX records since that's only related to where mail should be delivered/received for that domain.

IIRC, when this type of check is done, it's not necessarily strict, it's usually acceptable to resolve the sending mail-server shares the same IP as the PTR record and the address it points to are the same.

Example setup:

  • DMS server + config:
    • PTR 1.2.3.4 => example.com
    • A example.com => 1.2.3.4
    • A mail.example.com => 1.2.3.4
    • compose.yaml with hostname mail.example.com + setup email add [email protected]
  • DNS support for mail account [email protected] (mail accounts don't need to share the same domain or server as DMS):
    • A example.org => 5.6.7.8 (not important to DMS)
    • MX example.org => mail.example.com (Mail sent to example.org will be handled by DMS, not relevant to example below)
    • TXT (SPF) example.org =>"v=spf1 mx ip4:1.2.3.4 a:mail.example.com -all" (only the mx should be necessary, but could alternatively specify an IP address authorized to send mail on behalf of example.org, that IP address could implicitly be resolved for an A record lookup too if the MX of example.org is not appropriate)

MTA connection example:
1.When connecting to another MTA to send mail to (with sender [email protected]), DMS will greet the MTA with HELO / EHLO of mail.example.com.
2. The MTA we are sending the mail to (eg: belonging to a recipient, or is a relay service) may do a PTR lookup for the IP that mail.example.com points to (1.2.3.4), which the PTR record resolves to example.com, which in turn resolves back to 1.2.3.4, same as DMS at mail.example.com, pass ✔️
3. SPF record can also be checked, verifying that example.org authorizes mail.example.com (or in this case anything sending from that IP).

I may be mistaken if I'm recalling the above incorrectly. But the PTR check is tied to DMS hostname (that Postfix gets configured for), like with the TLS certificate, the trust is in the mail-server here, the sender address is not relevant (separately checked via DNS lookup for records like SPF that authorize DMS server to send).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So perhaps it should be advising in the docs here to match the IP that hostname (compose.yaml) would resolve to via DNS?

@polarathene
Copy link
Member

I started a new job this week, but I can probably test during the weekend if time allows it by temporarying adding a server and configure a domain for testing

Congrats on the new role! 🎉

No rush. If you can find the time to test and get back to me (in this PR is fine, even if it's closed), that'd be appreciated 👍

Also IPv6 in docker probably needs to be configured like mentioned here ( https://docs.docker.com/config/daemon/ipv6/ ), which I actually haven't done myself .. 😨

Which I guess is also why I went direct to networking host-mode in the first place 🙃

I recently rewrote our IPv6 docs, until v13 is released you can only see them in the edge versioned docs: https://docker-mailserver.github.io/docker-mailserver/edge/config/advanced/ipv6/

I also contributed to the updated upstream Docker IPv6 docs you linked, as prior they were not very good. I'd advise giving our IPv6 docs a good look.

If you have a more advanced IPv6 setup where ULA is not appropriate for you, you'll find some extra tips linked to the IPv6 docs rewrite PR, as I didn't have the time to figure all that out and document it properly. IPv6 ULA should meet most users needs very well though (just specify the IPv6 address on host to bind the containers internal ULA address to).

@github-actions github-actions bot added the meta/stale This issue / PR has become stale and will be closed if there is no further activity label Sep 4, 2023
@docker-mailserver docker-mailserver deleted a comment from github-actions bot Sep 4, 2023
@polarathene polarathene added stale-bot/ignore Indicates that this issue / PR shall not be closed by our stale-checking CI and removed meta/stale This issue / PR has become stale and will be closed if there is no further activity labels Sep 4, 2023
@norrs
Copy link
Contributor Author

norrs commented Sep 9, 2023

For the record:

@polarathene I have not forgotten about this, but I havent had any free time the last weeks. Maybe I find some wriggle room next week or early october.

As I plan to test #3465 (comment) this in regards of networking: bridge mode

@georglauterbach georglauterbach added the meta/feature freeze On hold due to upcoming release process label Nov 8, 2023
@georglauterbach georglauterbach removed the meta/feature freeze On hold due to upcoming release process label Nov 26, 2023
@georglauterbach
Copy link
Member

georglauterbach commented Dec 3, 2023

@norrs can you please resume working on this? I am trying to "clean" up old PRs because I know from experience that PRs like this tend to fade and rot 🙈


I removed the label for the stale-bot; this PR has approximately 3 more weeks until it is labelled as stale.

@georglauterbach georglauterbach removed the stale-bot/ignore Indicates that this issue / PR shall not be closed by our stale-checking CI label Dec 3, 2023
@georglauterbach
Copy link
Member

I'll do a review pass over this again sometime this week. If it's worthwhile to me then, I'll wrap this PR up and merge, otherwise will reject.

👍🏼 Please close if you come to the conclusion that this may not be worthwhile :)

Copy link
Member

@polarathene polarathene left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll apply these revisions.

Comment on lines 11 to 13
If your host system is running multiple IPv4 and IPv6 IP-addresses, you probably have an interest to
bind outgoing SMTP connections to specific IP-addresses to ensure MX records are aligned with
PTR-records in DNS when sending emails to avoid getting blocked by SPF for example.
Copy link
Member

@polarathene polarathene Jan 19, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
If your host system is running multiple IPv4 and IPv6 IP-addresses, you probably have an interest to
bind outgoing SMTP connections to specific IP-addresses to ensure MX records are aligned with
PTR-records in DNS when sending emails to avoid getting blocked by SPF for example.
If your Docker host is running multiple IPv4 and IPv6 IP-addresses, it may be beneficial to bind outgoing SMTP connections to specific IP-address / interface. When a mail is sent outbound from DMS, it greets the MTA it is connecting to with a EHLO (DMS FQDN) which might be verified against the IP resolved, and that a `PTR` record for that IP resolves an address back to the same IP. If DMS connections are inconsistent with the IP used here, these DNS checks are likely to fail.

reject_unknown_sender is a setting DMS has which other mail servers may also check for, where a foreign sender address is checked for having a valid DNS MX or A record. The linked issue is an SPF failure related to that.

There are two related restrictions Postfix can configure related to the suggestion that other MTAs may also check for.

reject_unknown_reverse_client_hostname:
Reject the request when the client IP address has no address -> name mapping.
This is a weaker restriction than the reject_unknown_client_hostname feature, which requires not only that the address -> name and name -> address mappings exist, but also that the two mappings reproduce the client IP address.

Related references within DMS contributions:


SPF only comes into play when it authorizes a mail server via mx / ip4 / ip6 / a etc, as demonstrated in my prior review comment. Likewise those DNS records related changes would need to have consistent IP.

@polarathene polarathene modified the milestones: v14.0.0, v13.3.0 Jan 19, 2024
@polarathene polarathene changed the title Docs: Tips about server with multiple IP-addresses docs: Guidance for binding outbound SMTP with multiple interfaces available Jan 19, 2024
@polarathene polarathene added service/postfix service/security/amavis area/networking and removed stale-bot/ignore Indicates that this issue / PR shall not be closed by our stale-checking CI labels Jan 19, 2024
Copy link
Member

@polarathene polarathene left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the contribution! ❤️

@github-actions
Copy link
Contributor

Documentation preview for this PR is ready! 🎉

Built with commit: ff7bcee

@polarathene polarathene merged commit deb0d2d into docker-mailserver:master Jan 19, 2024
@norrs
Copy link
Contributor Author

norrs commented Jun 19, 2024

You sir, are awesome

A huge thanks @polarathene , I never managed to follow this up in a timely manner and I apology for this. In the end, the suggested changes and extra work you put into it made it even better 💜

Next challenge, able to solve this with a non host networking mode as mentioned in #3465 (comment) ? 🤔
Maybe I brew something together one day 😅

@polarathene
Copy link
Member

Next challenge, able to solve this with a non host networking mode as mentioned in #3465 (comment) ? 🤔

Just letting you know we received a contribution recently that resolves that concern: #4330

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/documentation area/networking kind/improvement Improve an existing feature, configuration file or the documentation service/postfix service/security/amavis

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants