Re: [Libevent-users] http server and infinite streams

[Scott Lamb <slamb@xxxxxxxxx>]

On Wed, May 11, 2011 at 11:21 AM, Scott Lamb <slamb@xxxxxxxxx> wrote:
> On Wed, May 11, 2011 at 10:12 AM, Cliff Frey <cliff@xxxxxxxxxx> wrote:
>> I spent some more time thinking about this, and I have a few use cases that
>> it might be worth keeping in mind.
>>
>> It would be nice if the API made it easy to hook up a received HTTP request
>> (where we are the server) to an outgoing HTTP request (where we are the
>> client) in order to make building an HTTP proxy very simple.  Not a huge
>> deal if this isn't super simple though.
>> It should be easy to hook up a zlib compressor/decompressor to the HTTP
>> library for supporting compressed transfers (with Accept-Encoding).
>
> Yay.
>
>>
>> I'm not sure who should be responsible for allocating the bufferevent
>> objects.  If there was an easy way to connect two existing bufferevent
>> objects to one another (like bufferevent_pair_new, but for existing ones),
>> then it would be fine for the http library to provide the bufferevent
>> objects, but it would probably be more flexible if the caller provided the
>> bufferevent objects (but then I am unsure who should be responsible for
>> cleanup of them).
>> So, in any case, it boils down to something like
>> Option 1 (from my previous email)
>> // used instead of evhttp_set_gen_cb, cb is called as soon as the request
>> // is received, before the content of the request has been received.
>> void evhttp_set_gen_req_startcb(evhttp *, void (*cb)(struct evhttp_request
>> *, struct bufferevent *bev, void *), void *arg);
>> // this returns a bufferevent that can be used to send a reply to the client
>> struct bufferevent *evhttp_send_reply_bev(struct evhttp_request *req, int
>> code, const char *reason);
>> Option 2
>> // used instead of evhttp_set_gen_cb, cb is called as soon as the request
>> // is received, before the content of the request has been received.
>> // To received the contents of the message, a bufferevent * must be
>> returned,
>> // which the evhttp library will fill with the request data, respecting the
>> // highwater mark.  Once BEV_EVENT_EOF has been received, the evhttp library
>> // will not access the bufferevent * again, and it is up to the user to free
>> // the bufferevent.
>> struct bufferevent *evhttp_set_gen_req_startcb(evhttp *, void (*cb)(struct
>> evhttp_request *, void *), void *arg);
>> // evhttp will free bev after receiving BEV_EVENT_EOF
>
> presumably BEV_EVENT_ERROR or BEV_EVENT_TIMEOUT as well
>
>> void evhttp_send_reply_bev(struct evhttp_request *req, int code, const char
>> *reason, struct bufferevent *bev);
>>
>> I think that I like option 2 better.
>> Cliff
>
> I like option 2 as well; as you say, it's very easy for proxy servers
> to use. Somehow I'd avoided reading up too much on bufferevents
> before; I just did so and see that they're perfect for this. One
> caveat is that if not all aspects of bufferevents work with evhttp,
> that should be documented.

Also, sorry if this is obvious to someone who's actually used
bufferevents, but a question: how would the error behavior work?

* If the client side fails, it probably indicates that via
BEV_EVENT_ERROR|BEV_EVENT_READING? (Presumably even if the error at
the TCP layer was on a write call.)

* If the server side fails, BEV_EVENT_ERROR|BEV_EVENT_WRITING? If the
bufferevent in question came from a client request, how do both the
client evhttp request and the user code get error notification? (Maybe
the least surprising thing would be if the user could freely call
bufferevent_setcb without worrying about the client request? It
probably should be made clear if the client request still exists when
this callback is called.)

* If the user code wants to abort a request in progress (single or
linked), how would it do so?

>
>>
>> On Tue, May 10, 2011 at 11:36 PM, Cliff Frey <cliff@xxxxxxxxxx> wrote:
>>>>
>>>> So I like the idea of getting read-by-read notification of data as it
>>>> arrives.  I'm not thrilled with the idea of changing default behavior
>>>> wrt what counts as a read, though.  Either a flag or another kind of
>>>> callback would make me much more comfortable about the change here.
>>>
>>> I will try and make a new version of the read-breakup change that adds a
>>> flag of some sort.
>>>>
>>>> I think that the evhttp_send_reply_chunk_with_cb behavior might be a
>>>> little screwy.  Suppose that you send one chunk with a cb, then send
>>>> another chunk with cb == NULL.  The second time you call
>>>> evhttp_send_reply_chunk_with_cb, it won't clear out the cb, so the
>>>> first cb will get called for the second chunk.  That can't be
>>>> as-intended, right?
>>>>
>>>> I *think* that some part of this could be done by exposing the
>>>> bufferevent more, but like you I'm a little unclear on the details
>>>> here.  Pausing and unpausing could (I think) be done with
>>>> bufferevent_enable and disable on the user's part; throttling could be
>>>> done with the rate-limiting functionality and the high-water/low-water
>>>> functionality; and did you know that you can have more than one
>>>> callback on an evbuffer?  You can get into lots of fun with that.
>>>>
>>>> But on consideration, this code is pretty darn nice.  I'm hoping other
>>>> evhttp users will have a look at it too, but on the whole I am pretty
>>>> happy about merging it into 2.1.
>>>
>>> On the whole I agree with you that my proposed API is a little bit screwy.
>>>  It does get the job done in a fairly simple manner though... which is why I
>>> did it.  In any case, here is an attempt at a cleaner API, but I wanted to
>>> get feedback from the list on it.  This API very-much just builds on the
>>> current API (although I imagine that in the actual change to http.c, much of
>>> the current API would be implemented in terms of the new API...)
>>> Receiving HTTP requests as an HTTP server:
>>>
>>> Current interface
>>>   Useful for simple servers, impossible to receive infinite/streaming
>>> requests
>>>     set a maximum incoming request size (to avoid OOM)
>>>     set a callback that gets a complete request:
>>>       evhttp_set_gencb(evhttp *, void (*cb)(struct evhttp_request *, void
>>> *), void *arg);
>>> Proposed interface
>>> Leave the current interface for simple servers, but add another callback
>>> that supports receiving streaming requests.  If this other callback is
>>> registered, then it takes precedence over the old one.
>>> This is called as soon as a request is received, before the body of the
>>> request has been received (if any)
>>>
>>> evhttp_set_gen_req_startcb(evhttp *, void (*cb)(struct evhttp_request *,
>>> struct bufferevent *bev, void *), void *arg);
>>>
>>> If the request has content-length 0, then bev is NULL.  If 'cb' sets the
>>> highwater mark of bev, then the http library will attempt to respect that,
>>> and will stop reading from the underlying TCP connection if bev is full.  Is
>>> is up to 'cb' to register callbacks on bev that will receive future events
>>> (including EOF for the end of the request)
>>>
>>> Sending an HTTP response as an HTTP server:
>>>
>>> Current interface: provides two decent options, does not support flow
>>> control when sending possibly-infinite streams to slow clients (i.e. very
>>> frequent stock ticker symbol updates over a persistent connection to a very
>>> very slow client)
>>> either
>>>   evhttp_send_reply(struct evhttp_request *req, int code, const char
>>> *reason, struct evbuffer *databuf);
>>> or
>>>   evhttp_send_reply_start(struct evhttp_request *req, int code, const char
>>> *reason);
>>>   evhttp_send_reply_chunk(struct evhttp_request *req, struct evbuffer
>>> *databuf);
>>>   evhttp_send_reply_chunk_with_cb(struct evhttp_request *req, struct
>>> evbuffer *databuf, void (*cb)(struct evhttp_request *, void *), void *arg);
>>> Proposed interface: add a third option that allows for flow control
>>>   struct bufferevent *evhttp_send_reply_bev(struct evhttp_request *req,
>>> int code, const char *reason);
>>> It is the caller's responsibility to fill the returned bufferevent with
>>> the data to be sent to the client, and to eventually
>>> bufferevent_flush(BEV_FINISHED) on it.  At this point, the caller should
>>> never access the bufferevent or evhttp_request again (i.e. the http library
>>> will free it when finished).  The evhttp library would only empty this
>>> bufferevent when the underlying TCP connection's bufferevent was not too
>>> full (probably with a configurable highwater mark defining full, i.e.
>>> evhttp_connection_set_xmit_highwater or something)
>>>
>>> I believe that a similar set of additions could be made with the
>>> http-client API, but I wanted to get feedback on this potential server API
>>> first.  It would likely involve incompatible changes to the evhttp_request
>>> structure, but people are not supposed to depend on that anyways.
>>> Cliff
>>
>
>
>
> --
> Scott Lamb <http://www.slamb.org/>
>



-- 
Scott Lamb <http://www.slamb.org/>
***********************************************************************
To unsubscribe, send an e-mail to majordomo@xxxxxxxxxxxxx with
unsubscribe libevent-users    in the body.