Re: [Libevent-users] avoiding file descriptor collisions

[William Ahern <william@xxxxxxxxxxxxxxxxxx>]

On Fri, Dec 09, 2011 at 07:21:44AM -0800, Myk Taylor wrote:
> Hi all,
> 
> I've run into another bit of trouble, but I don't see an easy way out 
> this time.  The http(s) client I'm writing has many connections going up 
> and down continuously, and I'm running into what seem to be file 
> descriptor collisions in evhttp.  I believe this is what is happening:
> 
> 1) connection A opens a socket and receives fd N for a request
> 2) The request fails on the server and the server closes the TCP 
> connection (sends a FIN)
> 3) the OS frees up fd N for the process, even though connection A hasn't 
> closed it explicitly yet
> 4) connection B opens a socket and receives fd N
> 5) connection A runs its callback and closes fd N
> 6) B attempts to use fd N and errors out
> 
> Does this sound possible?

No. Unfortunately, there's a bug in your application.

A descriptor is always valid until an explicit close. However, under POSIX
"the open() function shall return a file descriptor for the named file that
is the lowest file descriptor not currently open for that process."
socket(2) doesn't appear to have that requirement, but it behaves similarly
under unix. Therefore it's common that you'll see a descriptor number reused
immediately after being closed; necessarily if it was the lowest number not
still allocated.

A wild guess is that some code for connection Z (which predates A and B)
holds a stale descriptor value and is closing it. This is a common bug in
application code. Easiest way to avoid it is to never pass file descriptors
as values--except to low-level routines--but as references (say, as a member
of another object). When the descriptor is closed, set the descriptor value
to -1 so that there's no loaded gun lying around. It's the same concept as
setting a pointer to NULL after calling free. Technically it's not needed
but it pays divdends when trying to track down bugs.

An even wilder guess is that you have a pending callback which is firing
after some other callback which closed the descriptor, resulting in the
previous guess scenario. Juggling multiple concurrent callbacks for the same
resource can be difficult and is best avoided if possible.
***********************************************************************
To unsubscribe, send an e-mail to majordomo@xxxxxxxxxxxxx with
unsubscribe libevent-users    in the body.