Re: [Libevent-users] WaitForMultipleObjects backend?

[Nick Mathewson <nickm@xxxxxxxxxxxxx>]

On Wed, Mar 13, 2013 at 9:01 PM, Patrick Pelletier
<ppelletier@xxxxxxxxxx> wrote:
> I was perusing whatsnew-2.0.txt and saw:
>
>   Libevent 2.1 may add a multithreaded WaitForMultipleEvents-based
>   backend for better performance with many inactive sockets and better
>   integration with Windows events.
>
> Although that hasn't been added yet, I was wondering if anyone had thought
> about it, and had any suggestions on how hard writing such a backend would
> be, or how best to approach it.
>
> I have some existing code which uses WaitForMultipleObjects to wait on a
> combination of sockets and "manual reset events" created with CreateEvent().
> I'd like to port this code to libevent, but I'm not sure how to handle the
> manual reset events.  It seems like the "correct" solution would be to write
> a new backend as suggested in whatsnew-2.0.txt, but I'm not sure how big of
> a project I'd be getting myself into if I set out to do that.  Any other
> approach I can think of feels like a horrible hack, though.  (Such as having
> a separate thread to do WaitForMultipleObjects on the manual reset events,
> and then write a byte to a socketpair that libevent is waiting on.)
>
> Any thoughts/suggestions?

Oh, this is going to be a *fun* one!

Here's a brain-dump.  I'm not an expert on Windows APIs, so there may
be errors or omissions.

**

Any WaitForMultipleObjects-based solution is going to run into the
fact that WSAEventSelect doesn't really do a sensible thing on write
events.  It considers a socket to have a write event trigger when the
socket that was not previously writeable becomes writeable.  So it's
not meaningful to ask for write events on a socket until after you
have gotten a WSAEWOULDBLOCK when attempting to write on that socket.
It's a little tricky to simulate EV_WRITE as-is with this.

**

One idea that I had for solving that problem was to give the EV_WRITE
event flag an alternative API, such that the user is required to say
call a function (maybe event_clear_write_event()) on an EV_WRITE event
upon receiving EV_WRITE on a socket.

This alternative API would be optional; you'd turn it on while setting
up your event_config for your event_base.

To implement this, you'd need EV_WRITE events to be 'sticky' : to have
them *start* active, and stay acti eeven after triggering, and only
become inactive once a client calls event_clear_write_event.  You
could do this in libevent 2.1 in terms of the new internal notion of
"active_later" events.

**

Another thing you could look at for starters is the branch that Chris
Davis tried to write to handle this code.  I don't think it wound up
working completely, but it had some good ideas.  I have it archived in
branch "hybrid-loop2" in my personal repository at
https://github.com/nmathewson/libevent/tree/hybrid-loop2

It probably has some good ideas!

**

Integration with IOCP could be a clever thing here. IOCP->Event and
Event->IOCP signals seem to go pretty fast on windows, whereas mucking
around with stuff like evutil_socketpair() is less so.

**

evutil_socketpair() fails on some hosts because of overzealous AV
software. Any solution that depends on it will sometimes have to fall
back to plain old select(). :(

**

You're going to run into the 64-event (or is it 63?) limitation of
WaitForMultipleEvents.  Your options are:

  * You can use WSAEventSelect to assign multiple sockets' events to a
single Event, and use WSAEnumNetworkEvents a lot on sockets that
aren't really ready. I am assured by one competent programmer that
this works better for him than it would sound.

 * You can use RegisterWaitForSingleObject and friends to do a
threadpool-backed implementation

 * You can roll your own threadpool.

 * You can max out at 64 events. That's a nonstarter IMO.

**

The interface for libevent backends is straightforward but
underdocumented. I would be more than happy to answer any questions
you have.  The shortest backend of general interest is poll.c; the
longest is epoll.c.

yrs,
-- 
Nick
***********************************************************************
To unsubscribe, send an e-mail to majordomo@xxxxxxxxxxxxx with
unsubscribe libevent-users    in the body.