[Mimedefang] Rewarding plaintext

[Kenneth Porter]

--On Thursday, October 16, 2003 9:16 AM -0400 "David F. Skoll"
<dfs at roaringpenguin.com> wrote:

> Another option would be to record a hash of all the messages received,
> and allow plaintext ones through immediately, but tempfail HTML ones
> "n" times, where n is a small number that depends on how much penalty you
> wish to apply.  (The hash lets you key the counter to the message.)
> 
> Of course, this means that you server will consume n times the usual
> bandwidth to process HTML messages, so you hurt yourself as much as (or more
> than) senders of HTML messages.

No, that goes the opposite direction to what I'm suggesting. The idea is to
put *any* extra processing in a slow queue. The only processing that need be
done up front is detecting the MIME type, to determine whether to shift to
low-priority mode.
 
> Yet another option would be to shunt HTML messages into a queue where they
> sit (consuming disk space, but not CPU time) for a specified time, after
> which they are delivered.  This would add a constant delay for all HTML
> messages.

The delay isn't important. If the queue runner is niced, it'll run only when
all plaintext has been passed, which is the actual objective. If the queue
runner never runs because the server is completely loaded by plaintext, then
we've still accomplished our objective, which is to ensure that all plaintext
always gets higher priority than any other type.

The disadvantage of using a queue is that we can't give an immediate refusal
for malware (including spam). So it may be better to reserve some of the MD
pool to run niced. This lets us issue a 5xx during submission, but it may take
awhile since SA and the AV are running niced and plaintext submissions are
getting all the CPU.

If a new message comes in with a non-plaintext part and all niced members of
the pool are busy (ie. the current process is in the plaintext class), the
message should be tempfailed, freeing the process to handle the next plaintext
connection. The hard part here is figuring out how to structure the pool
system to implement this policy.

One might want to remember recent tempfails by hashing the message header,
only to keep aggressive spammers at bay. Detecting the re-submission of the
same header eliminates the need to search further in the message for
non-plaintext MIME types, so we can tempfail with less processing.

On a related note, how might one use this to get sendmail to run separate
queues for local submissions (ie. outbounds)? Presumably MD doesn't care which
way the traffic is flowing, so it should impose the same penalties on
outbounds without doing anything special.