[Libevent-users] Fwd: Windows / IOCP support for Libevent

[Nick Mathewson <nickm@xxxxxxxxxxxxx>]

Forwarding with permission.  Edited slightly for clarity.

---------- Forwarded message ----------
From: Nick Mathewson <nickm@xxxxxxxxxxxxx>
Date: Mon, Mar 5, 2012 at 10:59 AM
Subject: Re: Windows / IOCP support for Libevent

On Mon, Mar 5, 2012 at 10:41 AM, Tom Dial wrote:
> Great! I've got such a program, but it's rather involved.  Perhaps I'll make a simpler test harness and go for it that way.
>
> I haven't looked at the IOCP code in libevent yet but I will be interested to learn about the design.  Given that overlapped I/O is on Windows is more proactive in the sense that you are issuing I/Os and waiting for the result, rather than waiting for readiness and the doing the I/O, I thought it might be tough to design around both!
>

Right now, the only overlapped IO/IOCP backend is at the bufferevent
level: it's implemented in bufferevent_async.c, buffer_iocp.c, and
event_iocp.c.  It uses WSASend/WSARecv in about the way you'd expect.
There's also an IOCP-based listener implementation in listener.c.

To additionally replace select() with IOCP[*] is ... harder.  Some
people have claimed that you can do it by doing overlapped send and
recv operations with size 0. In testing, though, it appears that
doesn't work with send: An overlapped WSASend of size 0 appears to
complete immediately, whether or not the connection will allow more
data to be sent.

So to *actually* replace select() on Windows will be ... trickier! One
promising avenue: WSAEventSelect plus one of the WaitForMultipleEvents
variants (ideally, one using thread pools via
RegisterWaitForSingleObject) might work.  Chris Davis implemented a
version of this  a while back, using manual thread pools; there is a
copy in the "hybrid-loop2" branch of my Libevent repository on github.

But there's a problem there: the behavior of WSAEventSelect's FD_WRITE
does not match the behavior of select (or of poll, kqueue, epoll,
/dev/poll, or evport): FD_WRITE is (oddly) edge-triggered.  So we
would make FD_WRITE events remain active across libevent main loop
invocations, AND require the user to manually tell us when a write
operation had returned WSAEWOULDBLOCK.  Since this is an incompatible
API change, it couldn't be on by default: we'd only be able to use
this backend when the user specifically enabled it.  This isn't an
insurmountable requirement, but it is an annoying one.

So, that's about the state of affairs in making Libevent faster on
Windows these days.

Mind if I forward this mail to Libevent-users?  I think other people
would be interested too.

yrs,
--
Nick

[*] as Tom notes in a follow-up email, replacing select() with a
better backend on Windows is not in fact better than using overlapped
IO where available.  We ought to be able to get the most speed on
windows with overlapped IO+IOCP, since it has a pretty good
implementation, and avoids an extra round-trip of syscalls.  The point
of any improved select() replacement would be only to get more "struct
event"-based code working better and faster.  For bufferevent-based
code, we could sure do better.
***********************************************************************
To unsubscribe, send an e-mail to majordomo@xxxxxxxxxxxxx with
unsubscribe libevent-users    in the body.