[Libevent-users] Multi-threading question

[Programmist Setevik <programmist.setevik@xxxxxxxxx>]

I have a  "multiplexed multi-threaded" network IO program  that been working just fine for a long while. It is a typical hand-coded epoll-driven,  as opposed to libevent-driven, model that I am thinking about converting to libevent. Just a literal conversion  is what I have in mind.

I am really pleased with how my logic been working in this very flexible non-blocking and scale-able manner and I want to see if I can replicate it with libevent2, while making my life a bit easier by delegating this net IO minutiae to libevent.

Here's my question. Can I have:

1) "Main" thread  that does something like this, (pseudo code):

for ( all configured listen ports)  {

 FD = socket()

 bind()

 set non-block()

 listen

 add FD to a list

}

for (configurable # of worker threads) {

 ctx = malloc()

 ctx->ev_base = setup event_base

 add each FD , register it with ctx->ev_base  // mark the FD as "listener" FD so we know to acept() on EV_READ

 start worker thread, passing it the ctx // so that each thread has its own ev_base

}

...

do whatever, w/o ever touching any of the ev_bases and/or accesing any of the sockets/FD that are registered with

ev_bases

...

2) Worker threads, in their run logic

do the regular loop over their own ev_base, 

accepting new conns, registering them as needed with  their own ev_base,  for read/write/error callbacks and doing the regular net IO stuff

 

#####################

Each thread only accesses the ev_base that was passed to it, never seeing/accessing the ev_bases of its peers (other threads).

Will this work with libevent on Linux ? Don't care about supporting Win/Irix/Solaris/BSD etc. I understand, and my legacy code is a proof of, that in Linux you can have same FD added to multiple epoll sets, in different threads and OS will inherently lock it appropriately so that different thread don't step on ea-other - at least when using them for EV_READY leading to accept(). 

In other words, an FD that is registered with a number of epoll selectors in a number of threads, when an IO is ready, will only be reported to _one_ selector/thread. It is a race and only one worker thread will win it, but it still works fine.

Any advice ?  Will it work as intended (I don't see how it shouldn't because behind the scenes, on Linux, libevent will use the very same epoll. Any gotchas ? Again, Linux is all I care about.

I don't want to have a dedicated accept()or thread that then passes the accept()ed FDs to the workers (using the usual things, like a synchronized queue, or passing FD over a socket etc).