bug report: Rspamd crashing in loop from setup config dkim #4579
Description
📝 Preliminary Checks
- I tried searching for an existing issue and followed the debugging docs advice, but still need assistance.
👀 What Happened?
When running setup config dkim using Rspamd, I noticed sometimes it was failing like this:
$ setup config dkim selector dkim
2025-09-23 00:22:27+00:00 INFO rspamd-dkim: Creating DKIM keys of type 'rsa' and length '2048' with selector 'dkim' for domain 'example.test'
2025-09-23 00:22:27+00:00 INFO rspamd-dkim: Successfully created DKIM keys
2025-09-23 00:22:27+00:00 INFO rspamd-dkim: Supplying a default configuration (to '/tmp/docker-mailserver/rspamd/override.d/dkim_signing.conf')
rspamd: ERROR (not running)
rspamd: ERROR (abnormal termination)
2025-09-23 00:22:28+00:00 ERROR rspamd-dkim: Unexpected error occured :: script = /usr/local/bin/rspamd-dkim | function = _setup_default_signing_conf | command = supervisorctl restart rspamd | line = 260 | exit code = 7Similarly I could sometimes reproduce this with just:
$ supervisorctl restart rspamd
rspamd: ERROR (not running)
rspamd: ERROR (abnormal termination)
$ echo $?
7If you then checked if the process was running via ps -aux or supervisorctl status rspamd it would often appear to be running, but the PID was constantly changing (along with the uptime not incrementing), so this was stuck in a loop crashing...
There is no log files for rspamd being captured from the looks of it 🤔 (EDIT: Nevermind, see below update, they were configured for a different location)
Here we can see the crash loop occurring:
$ cat /var/log/supervisor/supervisord.log
2025-09-23 00:11:44,401 WARN process 'rspamd' already running
2025-09-23 00:18:08,747 WARN stopped: rspamd (terminated by SIGTERM)
2025-09-23 00:18:16,472 WARN stopped: rspamd (terminated by SIGTERM)
2025-09-23 00:22:12,557 WARN stopped: rspamd (terminated by SIGTERM)
2025-09-23 00:22:28,265 WARN process 'rspamd' already running
2025-09-23 00:22:44,651 WARN process 'rspamd' already running
2025-09-23 00:23:06,038 WARN process 'rspamd' already running
2025-09-23 00:23:19,869 WARN process 'rspamd' already running
2025-09-23 00:23:39,164 WARN killing 'rspamd' (3152) with SIGKILL
2025-09-23 00:23:39,198 WARN stopped: rspamd (terminated by SIGKILL)
2025-09-23 00:23:40,172 WARN exited: rspamd (exit status 1; not expected)
2025-09-23 00:23:41,192 WARN exited: rspamd (exit status 1; not expected)
2025-09-23 00:23:42,110 WARN stopped: rspamd (terminated by SIGTERM)
2025-09-23 00:23:42,830 WARN exited: rspamd (exit status 1; not expected)
2025-09-23 00:23:44,124 WARN exited: rspamd (exit status 1; not expected)
2025-09-23 00:23:44,877 WARN exited: rspamd (exit status 1; not expected)
2025-09-23 00:23:46,683 WARN exited: rspamd (exit status 1; not expected)
2025-09-23 00:23:48,595 WARN exited: rspamd (exit status 1; not expected)
2025-09-23 00:23:50,469 WARN exited: rspamd (exit status 1; not expected)
2025-09-23 00:23:52,203 WARN exited: rspamd (exit status 1; not expected)
2025-09-23 00:23:54,076 WARN exited: rspamd (exit status 1; not expected)
2025-09-23 00:23:56,166 WARN exited: rspamd (exit status 1; not expected)
2025-09-23 00:23:57,974 WARN exited: rspamd (exit status 1; not expected)
2025-09-23 00:23:59,758 WARN exited: rspamd (exit status 1; not expected)
2025-09-23 00:24:01,509 WARN exited: rspamd (exit status 1; not expected)
2025-09-23 00:24:03,205 WARN exited: rspamd (exit status 1; not expected)
2025-09-23 00:24:04,908 WARN exited: rspamd (exit status 1; not expected)
2025-09-23 00:24:06,633 WARN exited: rspamd (exit status 1; not expected)$ supervisorctl status rspamd
rspamd RUNNING pid 4074, uptime 0:00:01
$ supervisorctl status rspamd
rspamd RUNNING pid 4081, uptime 0:00:00
$ supervisorctl status rspamd
rspamd RUNNING pid 4093, uptime 0:00:00
$ supervisorctl status rspamd
rspamd RUNNING pid 4100, uptime 0:00:00
$ supervisorctl status rspamd
rspamd RUNNING pid 4105, uptime 0:00:00Based on the observed logs shared below, and changing PID of the service, it appears to be a race condition where the main process crashed before it could properly cleanup it's child processes. The container would need to be restarted or the children processes killed for this rspamd service to start properly again.
👟 Reproduction Steps
$ docker compose up -d --force-recreate
$ docker compose exec dms bash
# Repeat this command to trigger (usually just a potential service restart failure, crash loop is rarer):
$ rm -rf /tmp/docker-mailserver/rspamd/* && setup config dkim🐋 DMS Version
15.1.0
💻 Operating System and Architecture
Ubuntu 24.04 VPS
⚙️ Container configuration files
name: example
services:
dms:
image: ghcr.io/docker-mailserver/docker-mailserver:${DMS_RELEASE:-15.1.0}
hostname: mail.example.test
# Optional, used to verify limits weren't regressing anything:
ulimits:
nofile:
soft: 1024
hard: 65536
environment:
ENABLE_RSPAMD: 1
ENABLE_OPENDKIM: 0
# NOTE: These ENV below are just to simplify the reproduction and aren't relevant to reproduction:
ENABLE_AMAVIS: 0
ENABLE_UPDATE_CHECK: 0
# Instead of `volumes` to provide these config files, I'm using the Docker Compose `configs` feature
# so it's all embedded into the single `compose.yaml`:
configs:
- source: dms-accounts
target: /tmp/docker-mailserver/postfix-accounts.cf
# Optional - Persist rspamd config (should reproduce without, but useful if needing a fresh container):
volumes:
- ./rspamd/:/tmp/docker-mailserver/rspamd/
# DMS (Postfix) will run security checks to verify the sender/recipient domains,
# Add an alias to the container here to leverage Dockers internal DNS, only since
# this reproduction example doesn't have proper DNS records setup:
networks:
default:
aliases:
- example.test
# The Docker Compose `configs` feature inlines file content into `compose.yaml`
# NOTE: `$` will be inferred as an ENV on the host to replace with a value if found,
# `$$` is required as an escape to opt-out of that feature when an actual `$` is expected in the file content.
configs:
dms-accounts:
content: |
[email protected]|{SHA512-CRYPT}$$6$$sbgFRCmQ.KWS5ryb$$EsWrlYosiadgdUOxCBHY0DQ3qFbeudDhNMqHs6jZt.8gmxUwiLVy738knqkHD4zj4amkb296HFqQ3yDq4UXt8.📜 Relevant log output
When running the process directly, we can see why it's crashing:
$ docker compose exec bash
$ /usr/bin/rspamd --no-fork --user=_rspamd --group=_rspamd
# ...
2025-09-23 00:30:09 #4515(main) <4ce3b8>; main; main: rspamd 3.12.1 is starting, build id: release
2025-09-23 00:30:09 #4515(main) <4ce3b8>; main; main: cpu features: avx2, avx, sse2, sse3, ssse3, sse4.1, sse4.2, rdrand
2025-09-23 00:30:09 #4515(main) <4ce3b8>; main; main: cryptobox configuration: curve25519(libsodium), chacha20(avx2), poly1305(libsodium), siphash(libsodium), blake2(libsodium), base64(avx2)
2025-09-23 00:30:09 #4515(main) <4ce3b8>; main; main: libottery prf: AES-128
2025-09-23 00:30:09 #4515(main) <4ce3b8>; main; main: simdutf implementation: haswell(Intel/AMD AVX2)
2025-09-23 00:30:09 #4515(main) <4ce3b8>; main; main: skip writing pid in no-fork mode
2025-09-23 00:30:09 #4515(main) <4ce3b8>; main; main: event loop initialised with backend: epoll
2025-09-23 00:30:09 #4515(main) <4ce3b8>; main; spawn_workers: worker of type fuzzy(localhost:11335) is disabled in the config, skip spawning
2025-09-23 00:30:09 #4515(main) rspamd_inet_address_listen: bind 127.0.0.1:11332 failed: 98, 'Address already in use'
2025-09-23 00:30:09 #4515(main) <4ce3b8>; main; spawn_workers: cannot listen on normal socket 127.0.0.1:11332: Address already in use
2025-09-23 00:30:09 #4515(main) <4ce3b8>; main; spawn_workers: cannot create listen socket for rspamd_proxy at 127.0.0.1:11332
2025-09-23 00:30:09 #4515(main) <4ce3b8>; main; rspamd_hard_terminate: shutting down Rspamd due to fatal errorPort 11332 is the rspamd service for Postfix to integrate with: https://docs.rspamd.com/tutorials/scanning_outbound#mta-with-milter-support-eg-postfix-or-sendmail
The process tree looks like this when the container is started and has DKIM config present (NOTE: It takes about 20 seconds on this VPS deployment from the start of the rspamd process before the child processes are spawned):
$ grep rspamd <<< $(ps -auxf)
root 631 74.9 3.2 649292 261344 ? S 01:36 0:20 \_ rspamd: main process
_rspamd 823 1.4 3.0 649548 244268 ? S 01:36 0:00 | \_ rspamd: rspamd_proxy process (127.0.0.1:11332)
_rspamd 824 0.9 3.0 649548 244140 ? S 01:36 0:00 | \_ rspamd: rspamd_proxy process (127.0.0.1:11332)
_rspamd 825 8.4 2.6 650392 211564 ? S 01:36 0:00 | \_ rspamd: controller process (0.0.0.0:11334)
_rspamd 826 20.7 1.8 648064 147224 ? S 01:36 0:01 | \_ rspamd: hs_helper processUPDATE: Oh my bad the service logs are written to /var/log/mail/rspamd.log, while other services are logging to /var/log/supervisor/<service>.log
The related service config at /etc/supervisor/conf.d/dms-services.conf:
[program:rspamd]
startsecs=0
stopwaitsecs=55
autostart=false
autorestart=true
stdout_logfile=/var/log/mail/%(program_name)s.log
stderr_logfile=/var/log/mail/%(program_name)s.log
command=/usr/bin/rspamd --no-fork --user=_rspamd --group=_rspamd