Re: [Libevent-users] multithreading problem

On Tue, Jan 8, 2013 at 12:30 PM, Björn K. <bjoernk2@xxxxxxxxxxxxxx> wrote:

The posted code compiles on my machine without modifications. Thus I
wonder why you have to do some cleanup.

Maybe because I run with older system (10.6).

But the example code fails silently. Perhaps you don't run it with
enough rights to bind to port 443?

I fixed the places that fail silently and now it runs - but does not respond to request from a browser on port 443.

Most of the time I access the server with a browser and keep the
reload button pressed. Sometimes I use the apache ab tool.

I tried to understand how libevhtp works. If I get it right, all
connections are handled in the main thread and only the payload is
processed in a worker thread. For non-ssl connections this is fine but
for example for a ssl-only static web page server I think I do not
really profit from multithreading this way because all of the ssl
stuff is done in the main thread.

2013/1/8 Nir Soffer <nirsof@xxxxxxxxx>:

>
> On Jan 8, 2013, at 12:55 AM, Björn K. wrote:
>
> That piece of code should distribute the incoming connections over the
> threads (I want to change that code later to select the thread with
> the lowest number of handled connections). If I don't use this code
> and use instead
> bev = bufferevent_openssl_socket_new(thread_base[0], sock, client_ctx, ...
> the problem remains.
>
> Furthermore I don't think it's a problem of running out of memory. The
> segmentation fault even occurs when there are just a few connections
> to the server (once the third connection produced the crash).
> I tried to get a segmentation fault during a run in valgrind. One time
> I succeeded and got this:
>
> ==310== Invalid read of size 8
> ==310== at 0x41327D: bufferevent_incref_and_lock_ (bufferevent.c:626)
> ==310== by 0x41403F: bufferevent_enable (bufferevent.c:448)
> ==310== by 0x4044A9: acceptcb (in /home/login/test/simple)
> ==310== by 0x424325: listener_read_cb (listener.c:412)
> ==310== by 0x41D889: event_process_active_single_queue (event.c:1471)
> ==310== by 0x41E0E6: event_base_loop (event.c:1538)
> ==310== by 0x4049E0: main (in /home/login/test/simple)
> ==310== Address 0x62531a0 is 464 bytes inside a block of size 560 free'd
> ==310== at 0x4C240FD: free (vg_replace_malloc.c:366)
> ==310== by 0x413E2A: bufferevent_decref_and_unlock_ (bufferevent.c:703)
> ==310== by 0x41DAD4: event_process_active_single_queue (event.c:1476)
> ==310== by 0x41E0E6: event_base_loop (event.c:1538)
> ==310== by 0x404693: worker_function (in /home/login/test/simple)
> ==310== by 0x54258C9: start_thread (pthread_create.c:300)
> ==310==
>
> (And many similar messages. What they have in common is that some
> memory is freed in the worker thread and the main thread tries to
> access it.)
>
> But I don't know why memory of the bufferevent can be freed in the
> worker thread before the buffer event is enabled.
>
>
> I think the key to solve these issues is change the design - do not use
> libevent apis from multiple threads except event_active().
>
> The simplest solution would be to use libevhtp
> <https://github.com/ellzey/libevhtp/> which uses smart event based thread
> pool and looks very clean.
>
> If you want more control, the simplest solution is to get a single process
> working, then fork few worker processes in the main process, all sharing the
> same listener file descriptor, bound in the main process. Then let the
> kernel do the load balancing for you. Successful projects like nginx and
> unicorn <https://github.com/blog/517-unicorn> work like this.
>
> When I try the code you submitted (after some cleanup to make it compile and
> run), it does not respond when accessing with a browser. How are you stress
> testing this code?
>
>
> 2013/1/7 Mark Ellzey <mthomas@xxxxxxxxxx>:
>
> On Sun, Jan 06, 2013 at 09:39:48PM +0100, Bj?rn K. wrote:
>
>
>
> at a second glance:
>
>
> next_thread = (next_thread + 1) % NUM_THREADS; <- are you sure this is
>
> working as intended?
>
> ***********************************************************************
>
> To unsubscribe, send an e-mail to majordomo@xxxxxxxxxxxxx with
>
> unsubscribe libevent-users in the body.
>
> ***********************************************************************
> To unsubscribe, send an e-mail to majordomo@xxxxxxxxxxxxx with
> unsubscribe libevent-users in the body.
>
>
***********************************************************************
To unsubscribe, send an e-mail to majordomo@xxxxxxxxxxxxx with
unsubscribe libevent-users in the body.