[Libevent-users] Using libevent in multi-threaded server

[Leif Thuresson <leif.thuresson@xxxxxxxx>]

Hi,
Seeking advice from experienced libevent users.

I'm working on a server doing SSL socket I/O and I want to hand off the
CPU hungry SSL read/write to threads in a threadpool to gain performance.
I want to have read events pending all the time but write events only
when there is something in queue for output.
To be able to switch read and write events on/off independently  I think 
the standard solution
is to create two separate event object for each file descriptor.
In the server one thread is running the event-loop and the event 
callback hand off the
execution of a secondary application I/O callbacks to a thread in the 
threadpool that
performs the actual I/O.
A problem with using two separate event objects is that I risk getting 
both a read and
a write application callback running at the same time in different 
threads which will crash the SSL I/O.

I have come up with a few alternative solutions but maybe someone has 
better ideas.

I.  Have a separate mutex lock for each file descriptor.
    Lots of locks but it will solve the runtime problem. It might still 
be tricky to close
    a session when the lock should be destroyed.

II. In the event callback do event_del() on the other event object for 
the current file
    descriptor before handing over the file descriptor to the thread 
pool for
    application callback execution.
    My concern about this solution is if it might be to late to stop 
the other event
    object from being activated. I can be sure that the other event 
object isn't
    already active and waiting in some sort of queue for event callback 
execution?
    Also it will create extra calls to event_del/event_add which might 
be bad for performance.

III. If I could use just one event object per file descriptor that would 
simplify things
     a lot, but as far as I can see the only way to change the 
eventmask is to recreate
     the event object using event_del() + event_assign() + event_add() 
but that sound like
     a bit to much brute force.

Your thoughts on this would be much appreciated.
Thanks,
/Leif


***********************************************************************
To unsubscribe, send an e-mail to majordomo@xxxxxxxxxxxxx with
unsubscribe libevent-users    in the body.