Re: [Libevent-users] evhttp: Abort after receiving a certain number of bytes

[Bas Verhoeven <libevent@xxxxxxxxxx>]

On 2/16/2011 9:37 AM, Bas Verhoeven wrote:
> On 2/15/2011 9:38 PM, Nick Mathewson wrote:
> > evhttp_connection_set_max_body_size() sounds like what you're asking for.
>
> That looks perfect indeed. Strange that I didn't find it earlier, thanks!

I have been doing some testing and it turns out that this function does 
not work properly. I have made a little utility to test this: 
https://gist.github.com/829248 which I tested with libevent 2.0.10-stable.

There are various issues when testing with this utility.

If you run the utility you will see that it times out after about 10 
seconds (even tho the timeout is set to 2 seconds), not sure what is 
causing this yet and why evhttp is ignoring the timeout I provided for it.

You will also notice that the evhttp does not abort after receiving the 
25KB of data, instead it keeps receiving data until it eventually hits a 
timeout and then notices it has been reading way too much data, at which 
time it aborts.

After adding some debugging messages in http.c (evhttp_read_body) I 
noticed that req->body_size is not being updated, which the max body 
size check seems to depend on:

  if (req->body_size > req->evcon->max_body_size) {
      /* failed body length test */
      event_debug(("Request body is too long"));
      evhttp_connection_fail(evcon,
                     EVCON_HTTP_INVALID_HEADER);
      return;
  }

(Not sure why it's failing with 'EVCON_HTTP_INVALID_HEADER' instead of 
'DATA_TOO_LONG' by the way)

Below are two of the debugging messages after running for a few seconds:

  evhttp_read_body - bytes read: 6447682, ntoread: 33554432
  evhttp_read_body - body size: 0, max body size: 25600

As you can see the body size is still 0 even tho the buffer's length is 
already 6447682 bytes, this of course includes the header too, but is 
still a lot more than the 'max body size', yet it still keeps reading.

Now that I've read the code some more I guess req->body_size does not 
have to be updated, but the 'max body size' check should simply also 
count the remaining bytes (e.g. ntoread).

I just tested and the following change seems to work fine:

  if (req->body_size + req->ntoread > req->evcon->max_body_size) {

I guess this also makes more sense because it's kind of silly to 
continue reading even tho you know that there are more bytes remaining 
than the limit allows. I don't know enough of libevent's internals to 
make sure that this won't break anything tho. I also only tested this 
with non-chunked data.

Also, is it me or is there no way to know what caused a request to fail? 
I seem to be getting a NULL request passed to the 'request failed' 
callback every time (unless it's an actual HTTP error), at which time I 
will have no idea what the error was or for which HTTP request that 
error was generated.

Cheers,
Bas
***********************************************************************
To unsubscribe, send an e-mail to majordomo@xxxxxxxxxxxxx with
unsubscribe libevent-users    in the body.