[Mimedefang] Too Many Open Files (again?)

[Matlock , Justin]

This ended up being a semi-denial-of-service attack.

By semi, I mean it wasn't really an attack, but it did deny service. :)

Our ISP (Bellsouth) is having problems with their end of our DS3.  What
seems to be happening is a connection opens, and we get a small 10-16k burst
of data, then it just stops.  We get maybe 2-3 characters every 4-5 minutes
after that. 

What ended up happening, is these super-slow connections were opening tons
of sockets, but those sockets were never closing.  Sendmail never timed out,
because characters were being sent, just *very* slowly. 

This, of course, isn't noticeable for most normal things, like web browsing,
etc.  But it's definitely noticeable with email coming into our front-end
mail servers (which have MD on them).  We didn't even know we were having
network problems until we found this -- traceroutes/pings/etc were going at
their normal speeds.

I never thought to look at the current socket count -- I was thrown by the
'too many open files' error, and didn't even think about it.  Solaris'
wonderful error reporting (coming through syslog) also stated 'too many open
files' -- so we never even considered the sockets.

We discovered the problem when our web servers started crashing and locking
up, with the same error.

So; it wasn't a problem with MD, or Sendmail... Just the network connection
coming to our facility.  It would be nice if sendmail had some sort of
function like "if data doesn't flow at least this fast, abort connection",
though. 

It might be a good thing to consider for addition to a FAQ, though. heh

Thanks for your help!
Justin

P.S.  Bellsouth still hasn't been able to find the problem -- we've been
experiencing it since Monday night (Atlanta got hit with a massive
thunderstorm that night, so we think it's related).  It seems like it's a
router issue on their end having to do with TCP slow-start or maybe TCP
congestion avoidance.

-----Original Message-----
From: Joseph Brennan [mailto:brennan at columbia.edu] 
Sent: Wednesday, June 18, 2003 11:19 AM
To: mimedefang at lists.roaringpenguin.com
Subject: Re: [Mimedefang] Too Many Open Files (again?)



>> Jun 17 14:45:45 gaatsml01 mimedefang[6691]: [ID 838820 mail.warning]
>> h5HIjjd3007350: Could not create
>> /var/spool/MIMEDefang/mdefang-h5HIjjd3007350/HEADERS: Too many open 
>> files
>
> You probably need to increase the number of allowable open files per 
> process.  I believe the default on Solaris 9 is 256.  Try increasing 
> it with "ulimit" in the script that starts mimedefang.


But what kind of message would require more than 256 open files to process?
Doesn't this always suggest an error somewhere else that is leaving
filehandles open?  (I am using the word "always" to provoke replies!)

Joe Brennan














_______________________________________________
MIMEDefang mailing list
MIMEDefang at lists.roaringpenguin.com
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang