We (myself and my student, cc'ed here) have some questions regarding Tor relay bandwidth. We created a small testbed (private Tor network)involving relays, clients and servers over virtual machines. The client uses the relays to create two hop circtuits to communicate to the server. The link bandwidths between the virtual machines is otherwise ~ 10 Gbps (tested via end-to-end measurement tools like iperf and via downloading large files of sizes ~2 GB through https downloads).
When not using Tor, the client achieve a download rate of approximately 8 Gbit/s while downloading a file of approximately 2 Gbytes. The same process, when repeated over Tor circuit (through relays of the private Tor network, that used virtual machines), provides a maxmium download rate of about 700 Mbit/s for the client (while the CPU utilization was over 80%). This is much less compared to the expected throughput of 8Gbps achieved when client communicates directly
to the server (without using Tor circuits).
We tinkered around with the Tor config and discovered that when changed the config fields of BandwidthBurst and BandwidthRate to 100 MBytes, we achieved end-to-end throughput of about 100 Mbit/s (while the CPU Utilization was less than 20%) (while the underlying network link capacity is 1 Gbit/s). The moment
the BandwidthBurst and BandwidthRate values are increased to 1 GB (which should effectively be 8 Gbit/s), the achieved throughput is about 650-700 Mbps (which is still 8 times slower than expected).
Further, removing all bandwidth restrictions, the inter VM bandwidth (for the relays in the private Tor network) is around 8 Gbit/s (measured both via Iperf and through downloading a file > 2 GBytes using encrypted HTTP traffic). While this is as per the expected behavior of TCP, to utilize all the available bandwidth, the same does not happen for Tor which continues to ramp upto a maximum of 700 Mbit/s
We are confused as to why Tor's link bandwidth utilization is not higher (Even on a private Tor network), even when using a high capacity VMs in the private Tor network -- each one having about 4 cores and 8 GBytes of RAM. I understand
after speaking from some Tor developers and maintainers that Tor does not
utilize all the available CPU cores. Our VMs however do not have any other process running either. We run the Tor processes over Debian servers with
little no other programs. More strangely, while download via HTTPS achieves
a very large fraction of the link capacity (>80%), Tor traffic happens to achieve
a throughput of around 700 Mbit/s (which is around 90% lower than the max
link bandwidth).
We tried the same with 2-hop Tor circuits, hoping to see some improvement. But
to no avail. We continue to observe the same poor bandwidth utilization.
It would be great if you shed some light as to where we may be going wrong.
Regards
Sambuddho