[Mimedefang] bayes database lock error [SEC=OFFICIAL]

[Kris Deugau]

Ryan Lindsay via MIMEDefang wrote:
> We run 4 mail servers with the same version of sendmail and mimedefang 
> on them.
> 
> Running on RHEL 8.10
> 
> sendmail-milter-8.15.2-34.el8.x86_64
> 
> mimedefang-3.4.1-1.el8.x86_64
> 
> We basically use the servers to add the boilerplate water mark to our 
> email.
> 
> On one of the servers, I’ve noticed this error re-occurs fairly regularly
> 
> Feb  4 12:14:37 prod-vmail02.aipo.gov.au 
> mimedefang-multiplexor[1978037]: 5141ER2q1989140: Worker 0 stderr: 
> bayes: cannot open bayes databases /var/spool/MD-Quarantine/.sp
> 
> Feb  4 12:14:37 prod-vmail02.aipo.gov.au 
> mimedefang-multiplexor[1978037]: 5141ER2q1989140: Worker 0 stderr: 
> amassassin/bayes_* R/W: lock failed: File exists
> 
> I’m a little stumped by this as I’m not sure what it’s telling me.

It's been a while since I used file-based global/shared Bayes on any 
appreciable scale, but IIRC this is a "normal expected" error.  It's not 
really anything do to with MIMEDefang, it comes from SpamAssassin's 
Bayes module.

In the default setup with MIMEDefang, SpamAssassin locks and unlocks a 
global file-based Bayes DB on each request, but under load will trigger 
this error pretty regularly as each request updates various fields and 
records in the Bayes DB.  It will trigger more often if you have left it 
using autolearn or running auto-expiry of Bayes tokens as this will 
happen opportunistically whenever the DB crosses one of the relevant 
thresholds.

It's only a "problem" if you expect to use autolearn, expect to rely on 
the default expiry handling for Bayes tokens, or absolutely can't have 
ignored errors in your logs.

> [root at prod-vmail02 .spamassassin]# ls -lah
> 
> total 4.2G
> 
> drwx------. 2 defang defang   57 Feb  4 12:25 .
> drwxr-x---. 3 defang defang   26 May 16  2023 ..
> -rw-------. 1 defang defang   33 Feb  4 12:25 bayes.lock
> -rw-------. 1 defang defang 320M Feb  4 10:39 bayes_seen
> -rw-rw-rw-. 1 defang defang 5.0G Feb  4 12:25 bayes_toks

Yikes.  That's pretty huge for bayes_toks.  For comparison, the SA 
filter cluster I maintain has settled in at just over 100M of MySQL 
table, after years of fine-tuning Bayes and starting with a token limit 
of IIRC ~~20x the default, nudged down over time to ~10x the default.

What are your bayes_* settings in your SA configuration?

Are you running sa-learn --force-expire from cron or similar, or just 
letting SA's default Bayes expiry handling do its thing?

If you're not intentionally or knowingly using SpamAssassin for 
anything, you should either uninstall it (so MD doesn't detect and use 
it) or at least disable it in MD (since SA may be a dependency and rpm 
will complain if you try to remove it).

If you want to keep SA, I'd advise moving your Bayes DB into at least 
SQL, or better, Redis.  This will also allow you to share the Bayes DB 
between hosts.

-kgd