[Mimedefang] Help with header checking

[Bill Cole]

On 26 Nov 2020, at 11:55, Andrea Venturoli wrote:
[...]
>> Anyway... you need to capture the message (or at least the headers)
>> so we can analyze what's going on.
>
> Here's a sample:
>> Return-Path: <pagina at poeconomico.casa>
>> Received: from soth.netfence.it ([unix socket])
>> 	 by mailserver.netfence.it (Cyrus 3.0.14) with LMTPA;
>> 	 Wed, 25 Nov 2020 03:45:44 +0100

That Received header is added by Cyrus during delivery, so obviously 
it's not present when MD sees the message.

>> X-Cyrus-Session-Id: 
>> mailserver.netfence.it-557-1606272344-1-3657946293514545252
>> X-Sieve: CMU Sieve 3.0
>> Received: from poeconomico.casa (vds74451.mgn-host.ru 
>> [89.191.230.250] (may be forged))
>> 	by soth.netfence.it (8.16.1/8.16.1) with ESMTPS id 0AP2jef2000844
>> 	(version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 
>> verify=NO)
>> 	for <andrea at netfence.it>; Wed, 25 Nov 2020 03:45:43 +0100 (CET)
>> 	(envelope-from pagina at poeconomico.casa)

That Received header is added by Sendmail *AFTER* all milters have done 
their end-of-data work. It is not present when MD sees the message.


>> Authentication-Results: soth.netfence.it;
>> 	dkim=pass (2048-bit key) header.d=poeconomico.casa 
>> header.i=pagina at poeconomico.casa header.b=cGnTmyJh
>> X-Authentication-Warning: soth.netfence.it: Host vds74451.mgn-host.ru 
>> [89.191.230.250] (may be forged) claimed to be poeconomico.casa
>> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; s=key1; 
>> d=poeconomico.casa;
>>  h=Message-ID:From:To:Subject:Date:MIME-Version:Content-Type;
>>  i=pagina at poeconomico.casa;
>>  bh=xbJLlOE1CWUnav77hJisuzISPwtefQrfatVm8E+8Sow=;
>>  b=cGnTmyJh1B9VDyiBCFcRI2pVOQqJ+fw65kJL6vCU15L3GTJXXNxpgd0HHyeFDlXYj/1o+HHX3mkt
>>    m1YEVxiN/83OcZzQGMRhFLk6rVtoTMARuN/uO1fYAaxcCLqpsM5YLyU6NPIwsYsCkZx0pz4vCtMo
>>    Scl4h3E9zx52tto+NClcudYfpP+NW8QkC1J3Wu3ZkwGcBE2HkxsX7TOkR0OAk8ottDAu3OThcvCL
>>    SCuDoaaZxBxok24KZUJ663tjzPFMPih+Lna0Gx7bmYi//3mvI+7vkwQNMztima+51SQiI+UI77Ro
>>    H/M9ke7T0CNZfImI7dd+x4KluyNSe4dyH83DKQ==
>> Message-ID: <2a3970dc95e4cec62a2f9935fd496366a1ebc7 at poeconomico.casa>
>> From: accountant <pagina at poeconomico.casa>
>> To: xxxxxx at netfence.it
>> Subject: Ho trovata la tua email attraverso il servizio di 
>> appuntamenti "meetic.it".
>> Date: Wed, 25 Nov 2020 02:44:07 +0100
>> MIME-Version: 1.0
>> Content-Type: multipart/related; 
>> boundary="2bd19889d80c22e13d3871e175a182d1cd7a"
>> X-Scanned-By: MIMEDefang 2.83
>
> As you can see:
> _ there's no X-Spam-Score header: either SpamAssassin didn't detect 
> this or it wasn't even launched;
> _ if it ran, it should have added 100 points alone since "*.casa" is 
> blacklisted; that alone should have been enough;

A common reason for SA not being called by MD is a size limit. The 
example mimedefang-filter script includes a limit that made more sense 
15 years ago than it does today, when spammers routinely send huge 
garbage. Somewhere in the filter() or filter_end() subroutine in your 
mimedefang-filter there's a conditional code structure that governs 
whether SA is called, and that is where to look for the failure. If your 
code has retained the check from the distribution example, it will look 
something like this:

     # Spam checks if SpamAssassin is installed
     if ($Features{"SpamAssassin"}) {
	if (-s "./INPUTMSG" < 100*1024) {
	    # Only scan messages smaller than 100kB.  Larger messages
	    # are extremely unlikely to be spam, and SpamAssassin is
	    # dreadfully slow on very large messages.
         my($hits, $req, $names, $report) = spam_assassin_check();


Note also that the comment is a bit outdated. We've done a LOT of 
improvement in SA's performance with large messages, mostly by 
eliminating the use of ".*" in rules except when absolutely necessary.

> _ also 89.191.230.250 range is in my personal DNSBL and again this 
> alone should have been enough.

If that DNSBL is being used directly from Sendmail, that's a different 
failure. If it's being used via SpamAssassin, it's also due to not 
calling SA from MD.


> N.B.
> Running spamassassin on the command line effectively gives the score I 
> expect, so I just *think* it's not called. What in the end I'm trying 
> to see is why.
> Calling spamassassin and the code I posted are two different things: 
> but I see the latter is also failing and I thought that might give 
> some hint.
> If there's a better way to see why spamassassin fails it would 
> probably enough (although curiosity... :).

Look for the call to "spam_assassin_check()" in mimedefang-filter and 
work backwards.

-- 
Bill Cole
bill at scconsult.com or billcole at apache.org
(AKA @grumpybozo and many *@billmail.scconsult.com addresses)
Not Currently Available For Hire