[Mimedefang] stream_by_* placement question

[Steffen Kaiser]

Hello,

I'm implementing a stream_by_class function, that is to work like
stream_by_recipient and stream_by_domain, but with a more generic way to
categorize into which chunks the mail is broken up. So far, I'm using
Mimedefang v2.39 and it seems to work. (I named it stream_by_class below.)

However, I wonder about the following points:

S: I usually split off only few recipients, mainly postmaster/abuse/noc
and some internal lists.
Q: The current implementation of stream_by_domain completely discards the
current instance of the mail and generates a "resend" mail for each
domain. Can I keep some recipients left in the current instance?

I thought to perform:

resend_message(@{ $recip });
foreach my $r (@{ $recip }) {
   delete_recipient($r);
}

Where @{ $recip } contains the unmodified addresses of the split-off
recipients.

S: The current stream_by_domain's doc explicitly states that one should
call the function "early" in filter_begin.
Q: Are there side effects -- other than sending the _completely_ unmodifed
mail -- when to resend a mail later, say in filter() or filter_end()?

I thought to split the mail only, when necessary; example:

Only three out of hundreds recipients shall have attachments replaced by
URLs. So in filter() I find this condition and call stream_by_class() and,
if the current class is to replace the attachment, action_replace_by_URL()
is invoked.

I assume this question is answered by man for the "resend_message()"
function; meaning: It is possible, but no change is resent.

===

Some considerations about this setup:

a) When one keeps one class of recipients within the current instance of
the mail, there is one resent copy less then usual; hence, you use less
resources to process a mail, that will split up. That applies to run the
MIMEDefang process (decomposing the mail etc.pp.) one time.

b) If the mail is resend _late_ in the filter_begin/filter/filter_end
chain, the same mail is possibly processed multiple times, e.g. two times
the virus check is invoked.

b1) If there is virus in the mail, the filter can reject/drop the mail
once, before it had been resent at all. The total number of virus scans
per mail drops to "one", from once per resent copy.
b2) If there is no virus in the mail and the mail is resend _early_, the
mail is also checked multiple times, because once per resent copy and if I
keep one class within the current instance of MIMEDefang, hence, it is not
resent, the total number of virus scans per mail remains the same.

c) To be able to split the mail when the filter needs to alter the
message, decreases the amount to resend the mail, e.g. in my case:

if($lcType !~ /^text\// && $lcType ne 'message/delivery-status') {
# Now split the mail if necessary
	if(!defined($Class) && stream_by_class ($RecipientClasses) ) {
		return;
	}
	if($Class eq 'list') {

"$Class" is undef until stream_by_class is performed. There is no need to
split the mail, if there is no non-text attachment.

d) I also split up the "categorization" and "resent" process; the
categorization of the recipients is performed early, but the separation
process on demand. Second see above. The categorization is used to control
full-message bounces, when recipients of mixed categories are sent to,
e.g.:

        $RecipientClasses = categorise_recipients();

        if(exists($SendmailMacros{'auth_authen'})) {
                $isFHSender = $isInfSender = 1;
        } else {
# [snip]
        }

        unless($isFHSender) {
                if(defined($RecipientClasses->{'list'})) {
                        return action_bounce('You are not qualified for sending to a list.');
                }
        }

So a mail to a mixed set of recipients is completely bounced, if at least
one recipient class denies sending. This is what people expect here.

That Aliases and forwards may break this setup, is a known bug ;-), but is
resolved by "organizational rules".

====

I also wonder if it is possible to keep certain information of resend
messages, e.g. in a DB_File; for instance to bypass the virus scan and use
the SpamAssassin values computed in the primary instance of MIMEDefang
already.

E.g. using "$Sender|$MSgID" as key and a secondary instance looks up the
key for all mails that are send by the localhost (127.0.0.1).

The the primary instance could run all tests on the message, places the
values into the DB; and late in filter_end() one could actually split the
message in order to specifically alter the body according to the class of
the recipients.

====

I do attach a cut down filter, for reference. The skalog function simply
drop the arguments to syslog.

Bye,

-- 
Steffen Kaiser
-------------- next part --------------
A non-text attachment was scrubbed...
Name: mimedefang-filter_cut-down.gz
Type: application/octet-stream
Size: 3300 bytes
Desc: Cut down mimedefang-filter
URL: <https://lists.mimedefang.org/pipermail/mimedefang_lists.mimedefang.org/attachments/20040309/746f3596/attachment.obj>