Re: [Libevent-users] http server and infinite streams

[Cliff Frey <cliff@xxxxxxxxxx>]

> So I like the idea of getting read-by-read notification of data as it
> arrives.  I'm not thrilled with the idea of changing default behavior
> wrt what counts as a read, though.  Either a flag or another kind of
> callback would make me much more comfortable about the change here.

I will try and make a new version of the read-breakup change that adds a flag of some sort.

I think that the evhttp_send_reply_chunk_with_cb behavior might be a
little screwy.  Suppose that you send one chunk with a cb, then send
another chunk with cb == NULL.  The second time you call
evhttp_send_reply_chunk_with_cb, it won't clear out the cb, so the
first cb will get called for the second chunk.  That can't be
as-intended, right?

I *think* that some part of this could be done by exposing the
bufferevent more, but like you I'm a little unclear on the details
here.  Pausing and unpausing could (I think) be done with
bufferevent_enable and disable on the user's part; throttling could be
done with the rate-limiting functionality and the high-water/low-water
functionality; and did you know that you can have more than one
callback on an evbuffer?  You can get into lots of fun with that.

But on consideration, this code is pretty darn nice.  I'm hoping other
evhttp users will have a look at it too, but on the whole I am pretty
happy about merging it into 2.1.

On the whole I agree with you that my proposed API is a little bit screwy.  It does get the job done in a fairly simple manner though... which is why I did it.  In any case, here is an attempt at a cleaner API, but I wanted to get feedback from the list on it.  This API very-much just builds on the current API (although I imagine that in the actual change to http.c, much of the current API would be implemented in terms of the new API...)

Receiving HTTP requests as an HTTP server:

Current interface
  Useful for simple servers, impossible to receive infinite/streaming requests
    set a maximum incoming request size (to avoid OOM)
    set a callback that gets a complete request:

      evhttp_set_gencb(evhttp *, void (*cb)(struct evhttp_request *, void *), void *arg);

Proposed interface

Leave the current interface for simple servers, but add another callback that supports receiving streaming requests.  If this other callback is registered, then it takes precedence over the old one.

This is called as soon as a request is received, before the body of the request has been received (if any)

evhttp_set_gen_req_startcb(evhttp *, void (*cb)(struct evhttp_request *, struct bufferevent *bev, void *), void *arg);

If the request has content-length 0, then bev is NULL.  If 'cb' sets the highwater mark of bev, then the http library will attempt to respect that, and will stop reading from the underlying TCP connection if bev is full.  Is is up to 'cb' to register callbacks on bev that will receive future events (including EOF for the end of the request)

Sending an HTTP response as an HTTP server:

Current interface: provides two decent options, does not support flow control when sending possibly-infinite streams to slow clients (i.e. very frequent stock ticker symbol updates over a persistent connection to a very very slow client)

either

  evhttp_send_reply(struct evhttp_request *req, int code, const char *reason, struct evbuffer *databuf);

or

  evhttp_send_reply_start(struct evhttp_request *req, int code, const char *reason);

  evhttp_send_reply_chunk(struct evhttp_request *req, struct evbuffer *databuf);

  evhttp_send_reply_chunk_with_cb(struct evhttp_request *req, struct evbuffer *databuf, void (*cb)(struct evhttp_request *, void *), void *arg);

Proposed interface: add a third option that allows for flow control

  struct bufferevent *evhttp_send_reply_bev(struct evhttp_request *req, int code, const char *reason);

It is the caller's responsibility to fill the returned bufferevent with the data to be sent to the client, and to eventually bufferevent_flush(BEV_FINISHED) on it.  At this point, the caller should never access the bufferevent or evhttp_request again (i.e. the http library will free it when finished).  The evhttp library would only empty this bufferevent when the underlying TCP connection's bufferevent was not too full (probably with a configurable highwater mark defining full, i.e. evhttp_connection_set_xmit_highwater or something)

I believe that a similar set of additions could be made with the http-client API, but I wanted to get feedback on this potential server API first.  It would likely involve incompatible changes to the evhttp_request structure, but people are not supposed to depend on that anyways.

Cliff