Re: [Libevent-users] http server and infinite streams

[Scott Lamb <slamb@xxxxxxxxx>]

On Wed, May 11, 2011 at 10:12 AM, Cliff Frey <cliff@xxxxxxxxxx> wrote:
> I spent some more time thinking about this, and I have a few use cases that
> it might be worth keeping in mind.
>
> It would be nice if the API made it easy to hook up a received HTTP request
> (where we are the server) to an outgoing HTTP request (where we are the
> client) in order to make building an HTTP proxy very simple.  Not a huge
> deal if this isn't super simple though.
> It should be easy to hook up a zlib compressor/decompressor to the HTTP
> library for supporting compressed transfers (with Accept-Encoding).

Yay.

>
> I'm not sure who should be responsible for allocating the bufferevent
> objects.  If there was an easy way to connect two existing bufferevent
> objects to one another (like bufferevent_pair_new, but for existing ones),
> then it would be fine for the http library to provide the bufferevent
> objects, but it would probably be more flexible if the caller provided the
> bufferevent objects (but then I am unsure who should be responsible for
> cleanup of them).
> So, in any case, it boils down to something like
> Option 1 (from my previous email)
> // used instead of evhttp_set_gen_cb, cb is called as soon as the request
> // is received, before the content of the request has been received.
> void evhttp_set_gen_req_startcb(evhttp *, void (*cb)(struct evhttp_request
> *, struct bufferevent *bev, void *), void *arg);
> // this returns a bufferevent that can be used to send a reply to the client
> struct bufferevent *evhttp_send_reply_bev(struct evhttp_request *req, int
> code, const char *reason);
> Option 2
> // used instead of evhttp_set_gen_cb, cb is called as soon as the request
> // is received, before the content of the request has been received.
> // To received the contents of the message, a bufferevent * must be
> returned,
> // which the evhttp library will fill with the request data, respecting the
> // highwater mark.  Once BEV_EVENT_EOF has been received, the evhttp library
> // will not access the bufferevent * again, and it is up to the user to free
> // the bufferevent.
> struct bufferevent *evhttp_set_gen_req_startcb(evhttp *, void (*cb)(struct
> evhttp_request *, void *), void *arg);
> // evhttp will free bev after receiving BEV_EVENT_EOF

presumably BEV_EVENT_ERROR or BEV_EVENT_TIMEOUT as well

> void evhttp_send_reply_bev(struct evhttp_request *req, int code, const char
> *reason, struct bufferevent *bev);
>
> I think that I like option 2 better.
> Cliff

I like option 2 as well; as you say, it's very easy for proxy servers
to use. Somehow I'd avoided reading up too much on bufferevents
before; I just did so and see that they're perfect for this. One
caveat is that if not all aspects of bufferevents work with evhttp,
that should be documented.

>
> On Tue, May 10, 2011 at 11:36 PM, Cliff Frey <cliff@xxxxxxxxxx> wrote:
>>>
>>> So I like the idea of getting read-by-read notification of data as it
>>> arrives.  I'm not thrilled with the idea of changing default behavior
>>> wrt what counts as a read, though.  Either a flag or another kind of
>>> callback would make me much more comfortable about the change here.
>>
>> I will try and make a new version of the read-breakup change that adds a
>> flag of some sort.
>>>
>>> I think that the evhttp_send_reply_chunk_with_cb behavior might be a
>>> little screwy.  Suppose that you send one chunk with a cb, then send
>>> another chunk with cb == NULL.  The second time you call
>>> evhttp_send_reply_chunk_with_cb, it won't clear out the cb, so the
>>> first cb will get called for the second chunk.  That can't be
>>> as-intended, right?
>>>
>>> I *think* that some part of this could be done by exposing the
>>> bufferevent more, but like you I'm a little unclear on the details
>>> here.  Pausing and unpausing could (I think) be done with
>>> bufferevent_enable and disable on the user's part; throttling could be
>>> done with the rate-limiting functionality and the high-water/low-water
>>> functionality; and did you know that you can have more than one
>>> callback on an evbuffer?  You can get into lots of fun with that.
>>>
>>> But on consideration, this code is pretty darn nice.  I'm hoping other
>>> evhttp users will have a look at it too, but on the whole I am pretty
>>> happy about merging it into 2.1.
>>
>> On the whole I agree with you that my proposed API is a little bit screwy.
>>  It does get the job done in a fairly simple manner though... which is why I
>> did it.  In any case, here is an attempt at a cleaner API, but I wanted to
>> get feedback from the list on it.  This API very-much just builds on the
>> current API (although I imagine that in the actual change to http.c, much of
>> the current API would be implemented in terms of the new API...)
>> Receiving HTTP requests as an HTTP server:
>>
>> Current interface
>>   Useful for simple servers, impossible to receive infinite/streaming
>> requests
>>     set a maximum incoming request size (to avoid OOM)
>>     set a callback that gets a complete request:
>>       evhttp_set_gencb(evhttp *, void (*cb)(struct evhttp_request *, void
>> *), void *arg);
>> Proposed interface
>> Leave the current interface for simple servers, but add another callback
>> that supports receiving streaming requests.  If this other callback is
>> registered, then it takes precedence over the old one.
>> This is called as soon as a request is received, before the body of the
>> request has been received (if any)
>>
>> evhttp_set_gen_req_startcb(evhttp *, void (*cb)(struct evhttp_request *,
>> struct bufferevent *bev, void *), void *arg);
>>
>> If the request has content-length 0, then bev is NULL.  If 'cb' sets the
>> highwater mark of bev, then the http library will attempt to respect that,
>> and will stop reading from the underlying TCP connection if bev is full.  Is
>> is up to 'cb' to register callbacks on bev that will receive future events
>> (including EOF for the end of the request)
>>
>> Sending an HTTP response as an HTTP server:
>>
>> Current interface: provides two decent options, does not support flow
>> control when sending possibly-infinite streams to slow clients (i.e. very
>> frequent stock ticker symbol updates over a persistent connection to a very
>> very slow client)
>> either
>>   evhttp_send_reply(struct evhttp_request *req, int code, const char
>> *reason, struct evbuffer *databuf);
>> or
>>   evhttp_send_reply_start(struct evhttp_request *req, int code, const char
>> *reason);
>>   evhttp_send_reply_chunk(struct evhttp_request *req, struct evbuffer
>> *databuf);
>>   evhttp_send_reply_chunk_with_cb(struct evhttp_request *req, struct
>> evbuffer *databuf, void (*cb)(struct evhttp_request *, void *), void *arg);
>> Proposed interface: add a third option that allows for flow control
>>   struct bufferevent *evhttp_send_reply_bev(struct evhttp_request *req,
>> int code, const char *reason);
>> It is the caller's responsibility to fill the returned bufferevent with
>> the data to be sent to the client, and to eventually
>> bufferevent_flush(BEV_FINISHED) on it.  At this point, the caller should
>> never access the bufferevent or evhttp_request again (i.e. the http library
>> will free it when finished).  The evhttp library would only empty this
>> bufferevent when the underlying TCP connection's bufferevent was not too
>> full (probably with a configurable highwater mark defining full, i.e.
>> evhttp_connection_set_xmit_highwater or something)
>>
>> I believe that a similar set of additions could be made with the
>> http-client API, but I wanted to get feedback on this potential server API
>> first.  It would likely involve incompatible changes to the evhttp_request
>> structure, but people are not supposed to depend on that anyways.
>> Cliff
>



-- 
Scott Lamb <http://www.slamb.org/>
***********************************************************************
To unsubscribe, send an e-mail to majordomo@xxxxxxxxxxxxx with
unsubscribe libevent-users    in the body.