[Mimedefang] rate-limiting for outbound mails per sender

[David F. Skoll]

On Mon, 10 Feb 2014 07:08:38 +0100 (CET)
Steffen Kaiser <skmimedefang at smail.inf.fh-bonn-rhein-sieg.de> wrote:

> > what would be a good way to implement rate-limiting for outbound
> > mails per sender e.g. 50 recipients per smtp_auth sender per hour?

> I do this in filter_begin. There I detect the sender, assign a
> "SenderID" based on different rules and query a SQL database about
> the sum of recipients of this SenderID in the last hour & day. If the
> SMTP quota is not exceeded, the tuple (scalar(@Recipients), SenderID,
> NOW()) as added to the database.

Our CanIt product also implements rate-limiting, but we do it slightly
differently.  We keep a table like this (PostgreSQLisms, sorry...)

CREATE TABLE sender_rate_limit (
       sender     TEXT PRIMARY KEY,
       first_seen TIMESTAMP WITH TIME ZONE,
       count      INTEGER
);

Then when we see a RCPT from a sender (which can actually be an
envelope sender, the SMTP AUTH user, or the sending relay IP address)
we do the equivalent of:

BEGIN;
# Update existing bucket
UPDATE sender_rate_limit SET count = count + 1 WHERE sender = '$sender' AND first_seen >= now() - '1 hour'::interval;
# If the previous update did not update any rows, then:
   # The next delete deletes a potential row where first_seen is
   # more than one hour old
   DELETE FROM sender_rate_limit WHERE sender = '$sender';

   # And start a new bucket.
   INSERT INTO sender_rate_limit(sender, first_seen, count) VALUES('$sender', now(), 1);
COMMIT;

When it comes time to impose a rate limit, we can just do a:

SELECT count FROM sender_rate_limit WHERE sender = '$sender';

The lookups are very fast and don't require a SUM operation.  The
downside is that it doesn't keep a running hourly sum, but resets
(more-or-less) once an hour so a sender can actually exceed the rate
limit for a while if the timing window is just right.  In practice,
this has not been a problem.

Regards,

David.