[Mimedefang] Spamassassin and MimeDefang custom rules
Jon Rowlan
jon.rowlan at sads.com
Wed Oct 15 05:57:34 EDT 2008
Hiya Paul, that's a name I recognise, how are you doing?
I will give this a go later when its quiet.
I knew there must be something I was missing.
I had repeated the same rule in any/all sa style .cf files I could find
with no joy.
I will let you know how it goes!
Cheers,
jON
-----Original Message-----
From: mimedefang-bounces at lists.roaringpenguin.com
[mailto:mimedefang-bounces at lists.roaringpenguin.com] On Behalf Of Paul
Murphy
Sent: 15 October 2008 10:30
To: MIMEDefang at lists.roaringpenguin.com
Subject: Re: [Mimedefang] Spamassassin and MimeDefang custom rules
Jon,
>/etc/mail/sa-mimedefang.cf
>In this config file I have the following custom test
>header BAD_TEST Subject =~ /xyzzy/i
>score BAD_TEST 6.00
>describe BAD_TEST Local Rule
This file is NOT parsed for rulesets - only for the main SA
configuration options.
>Is there some way to capture a log of SA activity so I can perhaps see
>where I am going wrong?
Change your MD configuration to leave the
/var/spool/MIMEDefang/mdefang-<msgid> directory in place (start with -d
option), restart MD, send a test message, then take out the -d, find the
directory (grep the mal log to get the message ID), become your filter
user, and then in that directory, run:
spamassassin -p /etc/mail/sa-mimedefang.cf -t -d < INPUTMSG
This will run SA with your settings, and in debug mode - it won't
produce exactly the same scores as MD due to several issues, but it will
show whether your local rules are being considered, and whether they are
being matched.
The output will be something like this: (I've added line numbers to
assist in the commentary)
1 [27281] dbg: logger: adding facilities: all
2 [27281] dbg: logger: logging level is DBG
3 [27281] dbg: generic: SpamAssassin version 3.1.7
4 [27281] dbg: config: score set 0 chosen.
5 [27281] dbg: util: running in taint mode? yes
6 [27281] dbg: util: taint mode: deleting unsafe environment
variables, resetting PATH
7 [27281] dbg: util: PATH included '/opt/kde3/bin', keeping
8 [27281] dbg: util: PATH included '/opt/gnome/bin', keeping
9 [27281] dbg: util: PATH included '/usr/games', keeping
10 [27281] dbg: util: PATH included '/home/defang/bin', keeping
11 [27281] dbg: util: PATH included '/usr/bin/X11', keeping
12 [27281] dbg: util: PATH included '/bin', keeping
13 [27281] dbg: util: PATH included '/usr/bin', keeping
14 [27281] dbg: util: PATH included '/usr/local/bin', keeping
15 [27281] dbg: util: PATH included '/packages/bin', which doesn't
exist, dropping
16 [27281] dbg: util: PATH included '/usr/sbin', keeping
17 [27281] dbg: util: PATH included '/sbin', keeping
18 [27281] dbg: util: PATH included '.', which is not absolute,
dropping
19 [27281] dbg: util: final PATH set to:
/opt/kde3/bin:/opt/gnome/bin:/usr/games:/home/defang/bin:/usr/bin/X11:/b
in:/usr/bin:/usr/local/bin:/usr/sbin:/sbin
20 [27281] dbg: message: ---- MIME PARSER START ----
21 [27281] dbg: message: main message type: text/plain
22 [27281] dbg: message: parsing normal part
23 [27281] dbg: message: added part, type: text/plain
24 [27281] dbg: message: ---- MIME PARSER END ----
25 [27281] dbg: dns: is Net::DNS::Resolver available? yes
26 [27281] dbg: dns: Net::DNS version: 0.46
27 [27281] dbg: config: using "/etc/mail/spamassassin" for site
rules pre files
28 [27281] dbg: config: read file /etc/mail/spamassassin/init.pre
29 [27281] dbg: config: read file /etc/mail/spamassassin/v310.pre
30 [27281] dbg: config: read file /etc/mail/spamassassin/v312.pre
31 [27281] dbg: config: using "/var/lib/spamassassin/3.001007" for
sys rules pre files
32 [27281] dbg: config: read file
/var/lib/spamassassin/3.001007/updates_spamassassin_org.pre
33 [27281] dbg: config: using "/var/lib/spamassassin/3.001007" for
default rules dir
34 [27281] dbg: config: read file
/var/lib/spamassassin/3.001007/updates_spamassassin_org.cf
35 [27281] dbg: config: using "/etc/mail/spamassassin" for site
rules dir
36 [27281] dbg: config: read file
/etc/mail/spamassassin/10_report_template.cf
37 [27281] dbg: config: read file
/etc/mail/spamassassin/70_sare_adult.cf
38 [27281] dbg: config: read file
/etc/mail/spamassassin/70_sare_bayes_poison_nxm.cf
39 [27281] dbg: config: read file
/etc/mail/spamassassin/70_sare_evilnum0.cf
40 [27281] dbg: config: read file
/etc/mail/spamassassin/70_sare_evilnum1.cf
41 [27281] dbg: config: read file
/etc/mail/spamassassin/70_sare_evilnum2.cf
42 [27281] dbg: config: read file
/etc/mail/spamassassin/70_sare_genlsubj.cf
43 [27281] dbg: config: read file
/etc/mail/spamassassin/70_sare_header.cf
44 [27281] dbg: config: read file
/etc/mail/spamassassin/70_sare_html.cf
45 [27281] dbg: config: read file
/etc/mail/spamassassin/70_sare_obfu.cf
46 [27281] dbg: config: read file
/etc/mail/spamassassin/70_sare_oem.cf
47 [27281] dbg: config: read file
/etc/mail/spamassassin/70_sare_random.cf
48 [27281] dbg: config: read file
/etc/mail/spamassassin/70_sare_spoof.cf
49 [27281] dbg: config: read file
/etc/mail/spamassassin/70_sare_stocks.cf
50 [27281] dbg: config: read file
/etc/mail/spamassassin/70_sare_unsub.cf
51 [27281] dbg: config: read file
/etc/mail/spamassassin/70_sare_uri0.cf
52 [27281] dbg: config: read file
/etc/mail/spamassassin/70_sare_whitelist.cf
53 [27281] dbg: config: read file
/etc/mail/spamassassin/72_sare_bml_post25x.cf
54 [27281] dbg: config: read file
/etc/mail/spamassassin/72_sare_redirect_post3.0.0.cf
55 [27281] dbg: config: read file
/etc/mail/spamassassin/99_local.cf
56 [27281] dbg: config: read file
/etc/mail/spamassassin/99_sare_fraud_post25x.cf
57 [27281] dbg: config: read file
/etc/mail/spamassassin/FuzzyOcr.cf
58 [27281] dbg: config: read file
/etc/mail/spamassassin/ImageInfo.cf
59 [27281] dbg: config: read file
/etc/mail/spamassassin/bogus-virus-warnings.cf
60 [27281] dbg: config: read file /etc/mail/spamassassin/local.cf
61 [27281] dbg: config: read file
/etc/mail/spamassassin/sa-mimedefang.cf
62 [27281] dbg: config: read file
/etc/mail/spamassassin/tripwire.cf
63 [27281] dbg: config: using "/home/defang/.spamassassin" for user
state dir
64 [27281] dbg: config: using "/etc/mail/sa-mimedefang.cf" for user
prefs file
65 [27281] dbg: config: read file /etc/mail/sa-mimedefang.cf
...
Update files processed:
96 [27281] dbg: config: using
"/var/lib/spamassassin/3.001007/updates_spamassassin_org/empty.pre" for
included file
97 [27281] dbg: plugin: fixed relative path:
/var/lib/spamassassin/3.001007/updates_spamassassin_org/25_uribl.cf
98 [27281] dbg: config: using
"/var/lib/spamassassin/3.001007/updates_spamassassin_org/25_uribl.cf"
for included file
99 [27281] dbg: config: read file
/var/lib/spamassassin/3.001007/updates_spamassassin_org/25_uribl.cf
100 [27281] dbg: plugin: fixed relative path:
/var/lib/spamassassin/3.001007/updates_spamassassin_org/20_anti_ratware.
cf
101 [27281] dbg: config: using
"/var/lib/spamassassin/3.001007/updates_spamassassin_org/20_anti_ratware
.cf" for included file
102 [27281] dbg: config: read file
/var/lib/spamassassin/3.001007/updates_spamassassin_org/20_anti_ratware.
cf
103 [27281] dbg: plugin: fixed relative path:
/var/lib/spamassassin/3.001007/updates_spamassassin_org/25_razor2.cf
...
Blacklist checks:
318 [27281] dbg: uridnsbl: domains to query: untiesticks.com
questiontrains.com
319 [27281] dbg: dns: checking RBL sa-other.bondedsender.org., set
bsp-untrusted
320 [27281] dbg: dns: checking RBL combined.njabl.org., set
njabl-lastexternal
321 [27281] dbg: dns: checking RBL combined.njabl.org., set njabl
...
Header and body regular expression checks:
333 [27281] dbg: check: running tests for priority: 0
334 [27281] dbg: rules: running header regexp tests; score so far=0
335 [27281] dbg: rules: ran header rule __HAS_MSGID ======> got hit:
"<"
336 [27281] dbg: rules: ran header rule __SANE_MSGID ======> got
hit: "<MUjhdgbjdmcacWS at UntieSticks.com>
337 [27281] dbg: rules: "
338 [27281] dbg: rules: ran header rule __MSGID_OK_HOST ======> got
hit: "@UntieSticks.com>"
339 [27281] dbg: rules: ran header rule __SUBJECT_ENCODED_QP ======>
got hit: "=?ISO-8859-1?Q?"
340 [27281] dbg: rules: ran header rule __SARE_HEAD_MIME_VALID
======> got hit: "1.0"
341 [27281] dbg: rules: ran header rule __CT ======> got hit: "m"
342 [27281] dbg: rules: ran header rule __TOCC_EXISTS ======> got
hit: "<"
343 [27281] dbg: rules: ran header rule __HAS_SUBJECT ======> got
hit: "Y"
344 [27281] dbg: rules: ran header rule __SARE_WHITELIST_FLAG
======> got hit: "R"
345 [27281] dbg: rules: ran header rule __CTYPE_MULTIPART_ALT
======> got hit: "multipart/alternative"
346 [27281] dbg: rules: ran header rule __CTYPE_HAS_BOUNDARY ======>
got hit: "boundary"
347 [27281] dbg: rules: ran header rule __MIME_VERSION ======> got
hit: "1"
348 [27281] dbg: rules: ran header rule BAD_ENC_HEADER ======> got
hit: "=?ISO-8859-1?Q?You could get up to 750 GBP =2D Instantly with a
1-minute "
...
Summarising SA tests:
485 [27281] dbg: rules: running header regexp tests; score so
far=32.795
486 [27281] dbg: rules: running body-text per-line regexp tests;
score so far=32.795
487 [27281] dbg: uri: running uri tests; score so far=32.795
488 [27281] dbg: rules: running raw-body-text per-line regexp tests;
score so far=32.795
489 [27281] dbg: rules: running full-text regexp tests; score so
far=32.795
490 [27281] dbg: check: running tests for priority: 900
491 [27281] dbg: rules: running meta tests; score so far=32.795
492 [27281] dbg: rules: running header regexp tests; score so
far=32.795
493 [27281] dbg: rules: running body-text per-line regexp tests;
score so far=32.795
494 [27281] dbg: uri: running uri tests; score so far=32.795
495 [27281] dbg: rules: running raw-body-text per-line regexp tests;
score so far=32.795
496 [27281] dbg: rules: running full-text regexp tests; score so
far=32.795
497 [27281] dbg: check: running tests for priority: 1000
498 [27281] dbg: rules: running meta tests; score so far=32.795
499 [27281] dbg: rules: running header regexp tests; score so
far=32.795
...
AWL update:
512 [27281] dbg: plugin:
Mail::SpamAssassin::Plugin::AutoLearnThreshold=HASH(0x9514bb0)
implements 'autolearn_discriminator'
513 [27281] dbg: learn: auto-learn: currently using scoreset 3,
recomputing score based on scoreset 1
514 [27281] dbg: learn: auto-learn: message score: 32.795, computed
score for autolearn: 25.953
515 [27281] dbg: learn: auto-learn? ham=1, spam=7,
body-points=23.698, head-points=6.255, learned-points=0
516 [27281] dbg: learn: auto-learn? yes, spam (25.953 > 7)
517 [27281] dbg: learn: initializing learner
518 [27281] dbg: learn: learning spam
...
Bayes learn:
556 [27281] dbg: bayes: database connection established
557 [27281] dbg: bayes: found bayes db version 3
558 [27281] dbg: bayes: Using userid: 2
559 [27281] dbg: bayes:
90e28644e964898dcc6015cc5f5f318aa876d4a1 at sa_generated already learnt
correctly, not learning twice
560 [27281] dbg: learn: initializing learner
...
And finally, a decision:
561 [27281] dbg: check: is spam? score=32.795 required=5
562 [27281] dbg: check:
tests=BAD_ENC_HEADER,EXCUSE_REMOVE,HTML_MESSAGE,NO_RECEIVED,NO_RELAYS,RA
ZOR2_CF_RANGE_51_100,RAZOR2_CF_RANGE_E4_51_100,RAZOR2_CF_RANGE_E8_51_100
,RAZOR2_CHECK,SUBJECT_EXCESS_QP,URIBL_BLACK,URIBL_OB_SURBL,URI_NOVOWEL
563 [27281] dbg: check:
subtests=__CT,__CTYPE_HAS_BOUNDARY,__CTYPE_MULTIPART_ALT,__HAS_MSGID,__H
AS_SUBJECT,__HTML_LINK_IMAGE,__LOCAL_PP_NONPPURL,__MIME_HTML,__MIME_QP,_
_MIME_VERSION,__MSGID_OK_HOST,__NONEMPTY_BODY,__SANE_MSGID,__SARE_BLACK_
FG_COLOR,__SARE_BODY_BLNK_5_100,__SARE_HAS_BG_COLOR,__SARE_HAS_FG_COLOR,
__SARE_HEAD_MIME_VALID,__SARE_HTML_HAS_A,__SARE_HTML_HAS_BR,__SARE_HTML_
HAS_FONT,__SARE_HTML_HAS_IMG,__SARE_HTML_HAS_P,__SARE_HTML_HAS_TITLE,__S
ARE_META_MURTY3,__SARE_URI_ANY,__SARE_WHITELIST_FLAG,__SARE_WHITE_BG_COL
OR,__SARE_WHITE_FG_COLOR,__SUBJECT_ENCODED_QP,__TAG_EXISTS_BODY,__TAG_EX
ISTS_CENTER,__TAG_EXISTS_HEAD,__TAG_EXISTS_HTML,__TAG_EXISTS_META,__TOCC
_EXISTS,__TVD_BODY,__TVD_MIME_ATT_TP,__UNUSABLE_MSGID
The key part is on line 64 - sa-mimedefang.cf is read as a user
preferences file, so it is not parsed for rulesets.
Add your rules to /etc/spamassassin/local.cf instead.
Best Wishes,
Paul.
_______________________________________________________________________
Argenta Discovery Ltd, 8-9 Spire Green Centre, Harlow, Essex, CM19 5TR
Registered in England No. 3671653
_______________________________________________________________________
_______________________________________________
NOTE: If there is a disclaimer or other legal boilerplate in the above
message, it is NULL AND VOID. You may ignore it.
Visit http://www.mimedefang.org and http://www.roaringpenguin.com
MIMEDefang mailing list MIMEDefang at lists.roaringpenguin.com
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
More information about the MIMEDefang
mailing list