Re: [Libevent-users] multithreading problem

[Björn K. <bjoernk2@xxxxxxxxxxxxxx>]

The posted code compiles on my machine without modifications. Thus I
wonder why you have to do some cleanup.
But the example code fails silently. Perhaps you don't run it with
enough rights to bind to port 443?

Most of the time I access the server with a browser and keep the
reload button pressed. Sometimes I use the apache ab tool.

I tried to understand how libevhtp works. If I get it right, all
connections are handled in the main thread and only the payload is
processed in a worker thread. For non-ssl connections this is fine but
for example for a ssl-only static web page server I think I do not
really profit from multithreading this way because all of the ssl
stuff is done in the main thread.

2013/1/8 Nir Soffer <nirsof@xxxxxxxxx>:
>
> On Jan 8, 2013, at 12:55 AM, Björn K. wrote:
>
> That piece of code should distribute the incoming connections over the
> threads (I want to change that code later to select the thread with
> the lowest number of handled connections). If I don't use this code
> and use instead
> bev = bufferevent_openssl_socket_new(thread_base[0], sock, client_ctx, ...
> the problem remains.
>
> Furthermore I don't think it's a problem of running out of memory. The
> segmentation fault even occurs when there are just a few connections
> to the server (once the third connection produced the crash).
> I tried to get a segmentation fault during a run in valgrind. One time
> I succeeded and got this:
>
> ==310== Invalid read of size 8
> ==310==    at 0x41327D: bufferevent_incref_and_lock_ (bufferevent.c:626)
> ==310==    by 0x41403F: bufferevent_enable (bufferevent.c:448)
> ==310==    by 0x4044A9: acceptcb (in /home/login/test/simple)
> ==310==    by 0x424325: listener_read_cb (listener.c:412)
> ==310==    by 0x41D889: event_process_active_single_queue (event.c:1471)
> ==310==    by 0x41E0E6: event_base_loop (event.c:1538)
> ==310==    by 0x4049E0: main (in /home/login/test/simple)
> ==310==  Address 0x62531a0 is 464 bytes inside a block of size 560 free'd
> ==310==    at 0x4C240FD: free (vg_replace_malloc.c:366)
> ==310==    by 0x413E2A: bufferevent_decref_and_unlock_ (bufferevent.c:703)
> ==310==    by 0x41DAD4: event_process_active_single_queue (event.c:1476)
> ==310==    by 0x41E0E6: event_base_loop (event.c:1538)
> ==310==    by 0x404693: worker_function (in /home/login/test/simple)
> ==310==    by 0x54258C9: start_thread (pthread_create.c:300)
> ==310==
>
> (And many similar messages. What they have in common is that some
> memory is freed in the worker thread and the main thread tries to
> access it.)
>
> But I don't know why memory of the bufferevent can be freed in the
> worker thread before the buffer event is enabled.
>
>
> I think the key to solve these issues is change the design - do not use
> libevent apis from multiple threads except event_active().
>
> The simplest solution would be to use libevhtp
> <https://github.com/ellzey/libevhtp/> which uses smart event based thread
> pool and looks very clean.
>
> If you want more control, the simplest solution is to get a single process
> working, then fork few worker processes in the main process, all sharing the
> same listener file descriptor, bound in the main process. Then let the
> kernel do the load balancing for you. Successful projects like nginx and
> unicorn <https://github.com/blog/517-unicorn> work like this.
>
> When I try the code you submitted (after some cleanup to make it compile and
> run), it does not respond when accessing with a browser. How are you stress
> testing this code?
>
>
> 2013/1/7 Mark Ellzey <mthomas@xxxxxxxxxx>:
>
> On Sun, Jan 06, 2013 at 09:39:48PM +0100, Bj?rn K. wrote:
>
>
>
> at a second glance:
>
>
> next_thread = (next_thread + 1) % NUM_THREADS; <- are you sure this is
>
> working as intended?
>
> ***********************************************************************
>
> To unsubscribe, send an e-mail to majordomo@xxxxxxxxxxxxx with
>
> unsubscribe libevent-users    in the body.
>
> ***********************************************************************
> To unsubscribe, send an e-mail to majordomo@xxxxxxxxxxxxx with
> unsubscribe libevent-users    in the body.
>
>
***********************************************************************
To unsubscribe, send an e-mail to majordomo@xxxxxxxxxxxxx with
unsubscribe libevent-users    in the body.