Is tor being slowed up to ~20ms per hop by TCP delayed ACK?

[Adam Richter <adam_richter2004@xxxxxxxxx>]

Hi everybody,

	TCP connections sometimes suffer from ~20 millisecond per hop
delays (set longer in some operating systems) due to the interaction
of two policies: (1) the "Nagle algorithm" delaying transmission of
small data packets with less than the maximum payload when a previous
packet of less than maximum payload is still awaiting acknowledgement,
and (2) "delayed ACK" delaying acknowledgement of small packets when
there is no data to send in the other direction.  These policies save
network bandwidth by coallescing small packets into packets of the
maximum transfer size at points in the stream where earlier reception
up to that point would be of little use to the receiver.  The problem
is that this can add unnecessary delays when the sender is not going
to generate more data in the next before the receivers "delayed ACK"
timer expires or when sending data of urgency that outweighs the risk
of network congestion.

	For typical tor usage, I think that the worst case would be
that each transaction experiences six of these delays when being
forwarded over three hops, round trip.  I believe a web browser will
usually fetch up to four URL's simultaneously.  So, if I go to a major
web site that has ~80 pictures on a page, then I might expect a worst
case delay from the Nagle algorithm of:

80 URL's / 4 streams x 6 delays x 20ms/delay = 2.4 seconds

	The situation may be a better than this, as the Nagle
algorithm only delays sending a partial packet if another partial
packet previously sent is awaiting acknowlegement, but it is pretty
easy for that situation to arise.

	A solution to this in most unix-like operating systems is that
setting the TCP_NODELAY option causes queued any pending TCP data to
be queued for immediate transmission.  In addition to supporting
TCP_NODELAY, Linux supports a variant called TCP_CORK.  Neither
TCP_CORK nor TCP_NODELAY appear in the sources for tor-0.20.30,
tor-0.2.1.2-alpha or libevent-1.3e.  Have I missed some other library
that tor is using to avoid unnecessary Nagle delays?

	I am thinking about modifying tor so that, _when it has
nothing else to do_, it will set and clear TCP_NODELAY on any sockets
that the outgoing data is flushed.  Outgoing TCP data would continue
to be coalesced when tor has more input pending from any connection.
The pushes would only be done if tor is about to go to sleep in
select() and has transmitted unpushed data pending.

	Am I overlooking something?  Has anyone else looked into this?

Adam Richter