Re: Tor speed

[Scott Bennett <bennett@xxxxxxxxxx>]

     On Fri, 13 Feb 2009 09:55:45 +0100 slush <slush@xxxxxxxx> wrote:
>I know that latency is the real reason of user experience. But my speed
>tests were averages for longer time and many requests. When I have faster
>connection, but it feels much slower.
>
>Tor:
>request > ..... noooothiiiing for minute > ........ page is loading in ten
>seconds with all images (and bandwidth meter shows me speed above 140 kB/s).
>
>JAP:
>request > .... few second for first response > slower, but continuous page
>loading for longer than minute > done (and maximum speed wasnt better than
>few kB/s)
>
>Second behaviour is much better, because you feel the responsivity and in
>some cases, you can read page until it is fully loaded.
>
>Im asking, why is Tor different in this. Is there any reason in tor design
>or it is simply not too optimized? Because in total, there IS enough
>bandwidth on nodes (overal speed is sufficient).
>
>Reason 1, latency)
>I understand, that JAP (with most servers in germany) will be more
>responsive than Tor, which have nodes around whole world and one request is
>pending across continents. But: There are 3 nodes, me and target server (4
>connections in line total). We can expect, that every node is on different
>continent and on bad line (~ 400 ms latency). My latencies (Im 3ms to Czech
>internet bone) to USA are around 140ms. So at total:
>
>me -- 400 + 140 + 400 -- node1 --- 400 + 140 + 400 --- node2 --- 940 ---
>node3 --- 940 --- target
>
>==== 3760 ms for one-way, it is ~7,5 second to response in the WORST case!!
>Dont tell me, that 940 ms is typical latency between nodes...
>
>Reason 2, node overall throughput)
>Slow responsibility can be done by slow nodes connection. But overall speed
>is not a problem (not for me), because there IS enough connectivity, but
>probably badly spread in time. Im running exit node (
>https://torstatus.blutmagie.de/router_detail.php?FP=9a8eb14286dea095815b702f6d6f7d1d6d051da7)
>and I can see, that my bandwidth is not used in the best way. Sometimes node
>is idle and sometimes it hit the speed limits. In idle time is probably

     What sort of speed limit do you refer to above?  The hardware limit?
The torrc limits?

>everything ok - node is working on every request without own speed
>limitation. But what in the case of short-time overloading? My tip is, that
>in this case, node is working for somebody faster than for else. I think so,
>because it can be the reason, why I often have ZERO throughput thru Tor for
>tens seconds. My guess is that some nodes simply dont give me a slot for my
>data, because somebody else is transfering something big, what "kill" the
>bandwidth for a moment. And because there are three nodes in the line, it is

     It doesn't really work quite like that.  Remember first that tor
packetizes everything into "cells", which are the tor protocol analog of IP
packets, so cells being sent out to other ORs are queued according to the
next OR.  Incoming cells arrive, go through any required encryption/decryption,
and get queued for the next place they are supposed to go.  tor attempts to
empty its output queues as fast as possible, but can only send just so much
across a connection before the next OR has to accept those arriving cells or
send back the equivalent of a "source quench" command.  A particular circuit
or a particular stream doesn't get prioritized relative to other circuits or
streams.
     I know nothing about what ISPs offer in your neck of the woods, but in
the U.S., nearly all Internet connections through ISPs are asymmetric and
have a reception capacity that is several times as great as their transmission
capacity.  If you have a similar situation, then the chances of your own use
of tor slowing down its performance when you are just web browsing are slim to
none because you will basically be using reception capacity that was not in
use before that.  When asymmetric links are used, tor tends to have very
similar average input and output rates.(*)

>the big probability I "hit" this problem on more nodes - and I will finally
>very slow responsibility.
>
>Hight throughput on the end of request (as I wrote on begin) is also an
>evidence - when I use so much bandwidth for a few seconds, somebody else
>must wait. Because there are tens of users on one node, it support my idea.

     Which node?  Yours?
>
>Is there anybody, who knows, if it is possible?

     It sounds like you could use a brief review of statistics and probability
to understand why your traffic levels might fluctuate as much you you say they
do.  Also, there are serious problems with the way peak data rate statistics
are handled by both tor relays and tor directory authorities that result in
poor overall quality of the values published in the directories that, in turn,
leads to less than optimal distribution by clients of their circuits over the
nodes available at any given time.  In any case, a tor client uses the
perceived distribution of capacities among the various nodes in selecting the
sequence of nodes to be used in building a circuit, so even if the reported
statistics were much better, there would be quasi-random fluctuations in
traffic levels all the time.  My relay currently runs between ~60 KB/s and
~360 KB/s most of the time, once it has been up and running long enough, but
does occasionally spike above 500 KB/s.
     Any given stream in tor is subject to an entire hierarchy of potential
delays, some of which are even stacked one on top of another.  For example,
at the bottom level, there are potential bottlenecks in your local connection
to your Internet link, in that link to your ISP, in the series of hops from
your ISP to the other OR's ISP, and from that ISP to the OR at the other end
of that TCP connection.  Over that bottlenecked route your tor client builds
a circuit hop to that OR, so if the circuit being built is sharing that TCP
connection with other circuits, then it is subject to any bottlenecs at the
level below.  Your client then builds another circuit hop to the next OR 
selected for the circuit.  If the OR at the end of the first hop already has
a TCP connection open to the next OR, extending the circuit to the second OR
can proceed fairly quickly, yet still requires another onion skin to be
encrypted by your client and decrypted by that second OR.  However, if the
entry OR does not already have an open connection to the second OR, it must
make one before you client can talk to the second (middleman) OR.  That can
add delay, too.  Repeat all that for the third (exit) OR.  At each step of
the way, the circuit being built may have to conpete with other circuits
served over one or more of the TCP connections between ORs that your circuit
is traversing.  Worse, once your circuit has been built you may have multiple
streams sharing the circuit, which then may share hops with other circuits.
     So the competitions and overheads occur at pretty much every level of
the system.  There is apparently a plan to stop using TCP connections between
clients and ORs and between different ORs, using UDP instead.  By reinventing
TCP within tor, the proponent(s) believe they can achieve better overall
performance, even though they lose the basic operating-system level of support
for reliable delivery in the process and I think they lose one layer of
encryption, as well.  They may be correct about that, but I am still wondering
why UDP has been chosen over SCTP, which looks to me as though it would
provide just about anything alternative that might be useful in improving
throughput, reliable delivery, and signalling, but that is a question that
no one has responded to so far.

(*) An exception, of course, exists when a relay is also a directory server,
in which case, its transmission rate may often exceed its actual reception
because the actual reception rate is limited by how fast that input can be
relayed to the next OR or exited to a destination, and that transmission
rate limit is being shared with the directory service transmissions.


                                  Scott Bennett, Comm. ASMELG, CFIAG
**********************************************************************
* Internet:       bennett at cs.niu.edu                              *
*--------------------------------------------------------------------*
* "A well regulated and disciplined militia, is at all times a good  *
* objection to the introduction of that bane of all free governments *
* -- a standing army."                                               *
*    -- Gov. John Hancock, New York Journal, 28 January 1790         *
**********************************************************************