Re: [Libevent-users] Beginners' question: multi-threaded HTTP service

[Mark Ellzey <mthomas@xxxxxxxxxx>]

On Mon, Nov 19, 2012 at 03:31:17PM +0200, Nir Soffer wrote:
> 
> I don't think this is a good example for using libevent (maybe I do not
> understand what it does correctly).
> 
> That server creates one event loop for accepting connections, and then one
> event loop for each connection. Since the event loops are run from worker
> threads, your server is actually a standard threaded server, and you don't
> get the benefits of event based servers. For example, you can serve only
> limited amount of connections in the same time.
> 
> One event loop for doing all IO will probably be fine for your minimal and
> fast http server. Add worker threads for cpu bound tasks or working with
> blocking apis, and use queues to pass jobs to the workers and back to the
> io thread.
> 
> If one IO thread is not enough, you can use one event loop for each
> core. This is how ngix works:
> http://www.aosabook.org/en/nginx.html

This is actually how libevhtp's threading model works; each thread in
the pool maintains its own event base, a connection is passed to a
thread with the least amount of utilization before accept() is called.
In short, the main thread only deals with the socket that triggered an
event, the threads actually accept() and handle the connection.

The trick to having the least amount of lock contention is to use the
event base that the thread is tied to for other events. This way you
don't even have to enable threading in libevent.

Creating a threaded webserver in libevhtp is trivial:

#include <evhtp.h>

#define NUM_THREADS 4

int main(int argc, char ** argv) {
	struct event_base * evbase = event_base_new();
	evhtp_t           * htp    = evhtp_new(evbase, NULL); 

	evhtp_set_gencb(htp, mycallback, NULL);
	evhtp_use_threads(htp, NULL, NUM_THREADS, NULL);
	evhtp_bind_socket(htp, "127.0.0.1", 8080, 1024);
	event_base_loop(evbase, 0);

	return 0;
}

The second argument of evhtp_use_threads is a callback function that is 
executed for each new thread that spawns. The first argument in the
callback is your evthr_t structure that contains the thread-specific
event_base. You can use this to add events in a threadsafe manner.

Some people have told me the method it uses is "not optimal", but it is
able to handle over 10,000 connections and service 70-90,000 requests/s,
so I have yet to see a better solution that avoids libevent locking. I
actually like the idea that each event_base is sandboxed in their own
thread.
***********************************************************************
To unsubscribe, send an e-mail to majordomo@xxxxxxxxxxxxx with
unsubscribe libevent-users    in the body.