Re: [Libevent-users] libevhtp beginnings API

[Mark Ellzey <mthomas@xxxxxxxxxx>]

> On Mon, May 16, 2011 at 10:30 PM, Mark Ellzey <mthomas@xxxxxxxxxx> wrote:
> Are you the copyright holder?  If so, why not stick to BSD to keep
> things simple.
> 
> Kevin

As an FYI, Libevent already does a good job of keeping track of all
non-BSD specific code in their LICENCE file.

But..

I have been reconsidering the idea of actually putting this into
Libevent itself, mostly because I have intentions to abstract a lot of
the duties a real http server would do: thread pooling, AIO spooling, 
RESTfull type configuration, and some other things which may piss people
off, even as an option. *waves to guy who told me how all my ideas were bad*

I am the owner, but the ry/http-parser is MIT. But odd thing is - it is a direct
copy from NGINX parser; 2 Clause BSD, but re-licensed as MIT. 

I asked him about the possibility of a license change but didn't seem
interested.

But it seems as if the newest versions of the NGINX parser is actually
much better than what what was used for the ry project at the time. I 
intend to go back and take the generalized FSM NGINX uses, implement
callbacks where needed, along with additional ones to boot. Then releasing 
back under BSD with a proper derived-from statement. 

This also includes a very nice and speedy uri argument parser(?=blah&whatever=1,etc..etc..)
 
Over the weekend I was able to get most everything in working order, including
full-control over streaming responses and a few new hooks.

Streaming/chunked replies are handled a little bit different than how a
chunked reply is done in Libevent right now, (as seen in regress_http).
Instead of having a wee-bit-confusing start-and-end set of callbacks
along with setting your own events in the user code, the process in
evhtp has been simplified a bit. Here is a quick example: 

static char * chunks[] = {
    "foo\n",
    "bar\n",
    "baz\n",
    NULL
};

static evhtp_res
_send_chunk(evhtp_request_t * req, void * arg) {
    int * idx = (int *)arg;

    if (chunks[*idx] == NULL) {
        return EVHTP_RES_DONE;
    }

    evhtp_request_make_chunk(req, chunks[*idx], strlen(chunks[*idx]));

    (*idx)++;

    return EVHTP_RES_OK;
}

static void
test_streaming(evhtp_request_t * req, void * arg) {
    int * index = calloc(sizeof(int), 1);

    evhtp_send_reply_stream(req, EVHTP_CODE_OK, _send_chunk, index);
}

int main(int argc, char ** argv) {
    .....
    evhtp_set_cb(htp, "/stream", test_streaming, NULL);
    event_base_loop(....);
    .....
}

Libevhtp will continually call your callback set by *_reply_stream until 
one of the three conditions have been met 

EVHTP_RES_ERROR: some type of error (haven't created an errcb mechanism
                 yet, so it just closes the connection) 
EVHTP_RES_OK:    Send this chunk and keep on calling the streaming
                 callback.
EVHTP_RES_DONE:  All chunks have been sent, and tells the backend to
                 send the final eof chunk.
EVHTP_RES_MORE:  This hasn't been implemented yet, but the idea is to
                 buffer this data until the OK status has finally been
                 returned.

It also most be noted that in the current state, the  developer
doesn't have to know an input of data is a chunk, this is all
done transparently. You just see a constant stream of data (if
chunked, just the bodies, not the headers or trailers). The "stream" is
only "streamed" if the EVHTP_HOOK_READ has been set, otherwise buffered.

I understand that having a way of telling the user application a set of
data is the start, part, or end of a chunk is something they need, but 
in the current state, this can't be done. A bi-product of the current
ry/http-parser api. My "rewrite" will fix this. 

Once again this API can be used exactly like the native Libevent API, but
optionally gives the developer to set various per-connection hooks and
attributes. 

Using test/bench_http versus the test app included in libevhtp I was
able to get some really nice boosts in performance. Using httperf with
200 concurrent connections and 400 requests each:

Libevent bench_http: Request rate: 37487.7 req/s (0.0 ms/req)
Libevhtp test:       Request rate: 54438.2 req/s (0.0 ms/req)

I have been carefully following RFC for both 1.0 and 1.1, but have still
given the ability to break some compatibility intentionally:

During the post-connection hook phase you can pass:
    evhtp_set_close_on(conn,
        EVHTP_CLOSE_ON_400 |
        EVHTP_CLOSE_ON_500 |
        EVHTP_CLOSE_ON_EXPECT_ERR); 

Where the EVHTP_CLOSE_ON* tells the informs the API to close on any parent
status codes no matter if a 1.0 request includes keep-alive, or 1.1 does
not inherently set connection:close header. The manual termination of the
connection is still done in an RFC compliant way.

All of these features can be found within my *develop* branch on github: 
https://github.com/ellzey/libevhtp/tree/develop


***********************************************************************
To unsubscribe, send an e-mail to majordomo@xxxxxxxxxxxxx with
unsubscribe libevent-users    in the body.