Re: [Libevent-users] libevent2 and delete-after-close with kqueue

[Nick Mathewson <nickm@xxxxxxxxxxxxx>]

On Sat, Jun 7, 2014 at 3:41 PM, Adrian Chadd <adrian@xxxxxxxxxxxxxxx> wrote:
> Hi all,
>
> I'm trying to chase down a bug with libevent2-trunk.
>
> The TL;DR is that I have ktrace traces from FreeBSD showing that an FD
> delete event is being kqueued to a kqueue FD after the FD has been
> closed in that thread. It's responsible for ENOTCAPABLE showing up.
> Something like EBADF just drops through; ENOTCAPABLE currently causes
> the IO loop to terminate. This is with a multi-threaded server, but
> there's one event base per thread and one accept FD per event base.
>
> It _looks_ like the evhttp / bufferevent code is doing the right thing
> - ie, it's deleting the events (which should delete them from the
> kqueue list) before calling close(). But there's a bunch of places
> that I have to actually go and check. Unfortunately "EV_DELETE"
> entries are being left on the kqueue list and not removed. The only
> way to know that these need to be removed is to actually call the
> event deletion function from evutil.c when the socket is closed.
> Teaching evutil_closesocket() would do it, but it would require an
> eventbase. It also implies that a socket is only in one event base -
> it's quite possible servers have an FD in multiple event bases. Ugh.
>
> What do people think?

Ow, that's annoying.

One thing to consider is that there's a parallel issue with closing
fds with an epoll-based backend.  (Look at the documentation about the
difference between the epoll and the epoll-with-changelist backends.)
Possibilities that I can see include:

  * Have a kqueue-without-changelist backend that flushes all
EV_DELETEs immediately.

  * Have an API like event_finalize that removes any pending
EV_DELETEs as needed.

  * Have a "delete event and close fd" API.

Obviously, these possibilities are maybe not so good.  Let's see if we
can think of others.



cheers,
-- 
Nick
***********************************************************************
To unsubscribe, send an e-mail to majordomo@xxxxxxxxxxxxx with
unsubscribe libevent-users    in the body.