Re: [Libevent-users] bufferevent openssl deadlock

[Mark Ellzey <mthomas@xxxxxxxxxx>]

On Fri, Feb 24, 2012 at 07:39:26PM -0600, Mark Ellzey wrote:
> On Fri, Feb 24, 2012 at 07:14:58PM -0600, Amarin Phaosawasdi wrote:
> > Hello,
> > 
> > About a year ago, there was a thread about deadlocking in
> > bufferevent_openssl in multi-threaded mode (
> > http://archives.seul.org/libevent/users/Jan-2011/msg00019.html).
> > 
> > We've run into the same problem.
> > 
> 
> Here are some more details for everyone:
> 
> Thread 2 (Thread 0x7ffff65d9700 (LWP 13521)):
> #0  pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:162
> #1  0x00007ffff73a5020 in evthread_posix_cond_wait (_cond=0x674520, _lock=0x6744f0, tv=0x0) at evthread_pthread.c:156
> #2  0x00007ffff7b9a844 in event_del_internal (ev=0x68cf18) at event.c:2184
> #3  0x00007ffff7b9a6bc in event_del (ev=0x68cf18) at event.c:2152
> #4  0x00007ffff71a056a in consider_writing (bev_ssl=0x68ce80) at bufferevent_openssl.c:830
> #5  0x00007ffff71a0e28 in be_openssl_outbuf_cb (buf=0x68d160, cbinfo=0x7ffff65d8d40, arg=0x68ce80) at bufferevent_openssl.c:1066
> #6  0x00007ffff7b9df98 in evbuffer_run_callbacks (buffer=0x68d160, running_deferred=0) at buffer.c:486
> 
> Thread 1 (Thread 0x7ffff7fdf700 (LWP 13519)):
> #0  evthread_posix_lock (mode=0, _lock=0x68d230) at evthread_pthread.c:70
> #1  0x00007ffff7ba59fa in _bufferevent_incref_and_lock (bufev=0x68ce80) at bufferevent.c:582
> #2  0x00007ffff71a0717 in be_openssl_writeeventcb (fd=10, what=4, ptr=0x68ce80) at bufferevent_ope
> 
> (gdb) print * lock
> $4 = {
>   __data = {
>     __lock = 1,
>     __count = 1,
>     __owner = 13521,
> 
> 
> As you can see, the owner of the lock is pid 13521 (thread 2) using a cond_wait, but thread 1 is attempting to acquire the lock
> resulting in a deadlock.
> 
> 
> Nick mentioned to me, the real issue is the use of conditions to handle event_del(). So looking into that may be 
> a better start.
> 
> I have been on-and-off attempting to patch this, but with the openssl api is quite dreadful right now, it's a slow game. 
> 

Oh I forgot to add a general statement about how it happens:

- parent thread calls write
- child goes into consider_writing
***********************************************************************
To unsubscribe, send an e-mail to majordomo@xxxxxxxxxxxxx with
unsubscribe libevent-users    in the body.