Re: [Libevent-users] Multithreading to handle HTTP keep-alive connections

[Azat Khuzhin <a3at.mail@xxxxxxxxx>]

On Sun, May 18, 2014 at 06:43:29PM +0200, Nick Giles wrote:
> Hello.
> 
> A while ago I asked this question on Stackoverflow ( http://stackoverflow.com/questions/21677154/libevent-multithreading-to-handle-http-keep-alive-connections
> ) but didn't have much luck with getting any replies, so I thought I'd ask
> on the mailing list.   Here's the question:
> 
> I am writing an HTTP reverse-proxy in C using Libevent and I would like to
> implement multithreading to make use of all available CPU cores. I had a
> look at this example: http://roncemer.com/software-development/multi-threaded-libevent-server-example/
> 
> In this example it appears that one thread is used for the full duration of
> a connection, but for HTTP 1.1 I don't think this will be the most effective
> solution as connections are kept alive by default after each request so that
> they can be reused later. I have noticed that even one browser panel can
> open several connections to one server and keep them open until the tab is
> closed which would immediately exhaust the thread pool. For an HTTP 1.1
> proxy there will be many open connections but only very few of them actively
> transferring data at a given moment.

Yes, connection pool is a standart technique for browser, to avoid
timeouts for connect's.
And personally I think that there is not much difference between
keep-alive and not for event-based mechanism, since it will work only
when there will be some data to transfer (read/write).

> 
> So I was thinking of an alternative, to have one event base for all incoming
> connections and have the event callback functions delegate to worker
> threads. This way we could have many open connections and make use of a
> thread only when data arrives on a connection, returning it back to the pool
> once the data has been dealt with.
> 

Sounds better, I think you can give it a try.

> My question is: is this a suitable implementation of threads with Libevent?

You could look into libevhtp, and in one of recent emails libevhtp2 was
announced - https://github.com/threatstack/libevhtp/tree/libevhtp2.

> 
> Specifically -- is there any need to have one event base per connection as
> in the example or is one for all connections sufficient?
> 

I think that this is not so _optimal_.
Since when you will have 100K connections than you will have 100K event
bases, which is waste of memory and file descriptors, and even more this
is not the case for which fd monitoring was created
(epoll/kqueue/select/...)
One event base per thread this is optimal I think.

> Also -- are there any other issues I should be aware of?
> 
> Currently the only problem I can see is with burstiness, when data is
> received in many small chunks triggering many read events per HTTP response
> which would lead to a lot of handing-off to worker threads. Would this be a
> problem? If it would be, then it could be somewhat negated using Libevent's
> watermarking, although I'm not sure how that works if a request arrives in
> two chunks and the second chunk is sufficiently small to leave the buffer
> size below the watermark. Would it then stay there until more data arrives?

About libevent and watermarks look here, section "Callbacks and
watermarks":
http://www.wangafu.net/~nickm/libevent-book/Ref6_bufferevent.html

> 
> Also, I would need to implement scheduling so that a chunk is only sent once
> the previous chunk has been fully sent.
> 
> The second problem I thought of is when the thread pool is exhausted, i.e.
> all threads are currently doing something, and another read event occurs --
> this would lead to the read event callback blocking. Does that matter? I
> thought of putting these into another queue, but surely that's exactly what
> happens internally in the event base. On the other hand, a second queue
> might be a good way to organise scheduling of the chunks without blocking
> worker threads.
> 
> Thanks!
> 
> Nick.
> 

-- 
Respectfully
Azat Khuzhin
***********************************************************************
To unsubscribe, send an e-mail to majordomo@xxxxxxxxxxxxx with
unsubscribe libevent-users    in the body.