Re: [Libevent-users] Constraining size of output buffer

[Mina Naguib <mina.naguib@xxxxxxxxxxxxxxxx>]

On 2011-06-24, at 10:57 PM, Nick Mathewson wrote:

> Tentatively, I'm going to blame this one on some kind of buffering or
> stalling issue in netcat or in your terminal, not on libevent.  When I
> write a trivial libevent-based client program to hammer your server
> program, it works fine.  See  the attached file -- I used your server
> program as a template.

Thank you very much Nick.

I think you're right.  It actually seems very obvious now what's happening once I dug in on netcat's side:

$ yes | strace -tt nc localhost 6565 > /dev/null
14:14:43.804107 read(3, "MIAMI\nWELCOME TO MIAMI\nWELCOME T"..., 8192) = 8192
14:14:43.804877 write(3, "y\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\n"..., 8192

Netcat reads from the socket in 8-k chunks, but doesn't completely drain it - instead it alternates fairly between reading and writing.

So, it reaches a point where it read 8k data from the server (but there's still plenty more), the server at that point stopped draining its reads.  The netcat tries to write to the server, but the server had stopped reading from that client, and netcat never goes back to draining its incoming data buffer for the server to get out of self-survival mode.

Had the average request length been longer than the average response length this would've never happened, but it just happened that in this example "y\n" averaged 2 bytes and each response averaged  17 bytes, so the output buffer filled much quicker and entered this condition.

This can be confirmed by making the average request longer than the average response:
$ yes "12345678901234567890" | strace -tt nc localhost 6565 > /dev/null

.. and indeed, no blocking occurs.

Also explains perfectly why your evented client works.  If the write buffer is continuously drained, the read buffer never gets paused.


> 
> The underlying pattern you're using here is reasonable, though you'd
> probably want to polish it a little.  Instead of handling as much as
> possible from the inbuf whenever the outbuf starts out less-than-full,
> I'd suggest draining the inbuf and adding to the outbuf only until the
> outbuf becomes full.
> 
> Also, for a situation like this one, I'd suggest setting the
> low-watermark for write to something nonzero: In some cases like this
> you don't want to let your write buffer drop to 0 either.

Yup makes sense.  I had actually implemented it that way, but for the purpose of simplifying the server to demonstrate the problem I removed them.

Thank you very much for your time and this excellent library.


***********************************************************************
To unsubscribe, send an e-mail to majordomo@xxxxxxxxxxxxx with
unsubscribe libevent-users    in the body.