OS: Debian Bookworm 12.11
AlternC version : 3.5.0, but also applies to 3.5.2 (configuration template is the same)

Symptom

Lot of dovecot errors in mail.log:

Error: Mailbox INBOX: mmap(size=755661208) failed with file /var/mail/alternc_index/[email protected]/.INBOX/dovecot.index.cache: Cannot allocate memory

The file was 721 MB.

Cause

AlternC forces vsz_limit = 512M for the imap service in /etc/alternc/templates/dovecot/conf.d/95_alternc.conf.

Dovecot 2.3.11 introduces a new parameter mail_cache_max_size to limit de cache size, which defaults to 1 GB (I could not found any info about prior behavior).

As such, index cache can grow larger than the vsz_limit, and dovecot fails to use the cache, has stated in the error message.

Solutions

One possible solution is to set the global parameter mail_cache_max_size to 512M to match vsz_limit.
An alternative possibility is to increase the vsz_limit, but it might drag too much memory pressure on environments with large imap users.