Re: How to Run High Capacity Tor Relays

[coderman <coderman@xxxxxxxxx>]

On Fri, Aug 27, 2010 at 3:10 AM, Moritz Bartl <tor@xxxxxxxxxxxxxx> wrote:
> Hi Mike,
>
> Thanks for putting this together. I wanted to do that for a while now, but I
> guess making you do it perfectly serves the cause as well. :)
>
>> As far as ethernet cards, the Intel e1000e *should* be theoretically
>> good, but they seem to fail at properly irq balancing across multiple
>> CPUs on recent kernels, which can cause you to bottleneck at 100% CPU
>> on one core.
>
> I think we have finally solved this. At the moment, CPU usage is good and is
> not limiting throughput (which could be higher though, not sure if the Tor
> network is saturated?).
>
> The key was to enable software Receive Packet Steering (RPS) introduced on
> kernel 2.6.35. I wrote a small howto: http://bit.ly/aqkDvR

is there a list of known good cards somewhere?

i know there were a few recent kernel upgrade tricks to get the most
out of the PRO/1000 PT cards on Linux 2.6.30+. perhaps use of sysstat,
iostat, kprobes, etc. to identify troublespots would be useful. i bet
there's lots of scattered lore on this among the relay ops...

last but not least, i've measured a theoretical improvement using
haproxy in tcp relay mode and transparent (reverse) mode to aggregate
the front end to client communication and tcp buffer management in a
first layer proxy, which then communicates via jumboframe or standard
local link to the Tor / SSL server. these TCP connections are much
more efficient to manage on the Tor / SSL host due to this offload
sharing.  i have no idea how well this would apply to a current and
well tuned Tor instance, but might be worth a test...