[Author Prev][Author Next][Thread Prev][Thread Next][Author Index][Thread Index]

[tor-relays] Raspberry Pi Relay Node Performance and future Plans on Documentation and more



Hi Gordon and Matthias,

I've split your discussion from the original thread "Running exit-node in Germany" and created a new one.

I fully agree with you that the Raspberry Pi is the perfect device to let others run a Tor Relay Node very easily. What follows is a long mail about my experiences and more thoughts about the Pi as relay.

On Thu, Aug 1, 2013 at 5:29 PM, Gordon Morehouse <gordon@xxxxxxxxxxxx> wrote:
Matthias Redies:
> Ok that is good to know. Right know I will probably run it on 1-1.5 Mbps
> and later on 3-4 Mbps. What is the maximum your raspberry is capable to
> do? Please let me know if you publish your tutorial.

I had it pushing about 1.5Mbps and crashing only about once a week
before I started having TCP connect floods and had to take it offline
until I could pay attention for a while.  I'm still tuning it.  It
crashed much, much more often before some basic tuning, though.

I'm running a relay node on a Raspberry Pi at my VDSL home connection for several weeks now (provider "Deutsche Telekom", connection is 50mbps down, 10 up) and I didn't run into such issues so far.

There're already several howtos out there which describe how to setup a relay node on the Raspberry Pi. I used this German one as starting point: http://blog.weezerle.de/2013/01/11/raspberry-pi-als-tor-server/

My current configuration has only the following two non-default options:

  DisableDebuggerAttachment 0
  AvoidDiskWrites 1

(Plus, I run it as relay only by disabling the socks proxy with "SocksPort 0".)

Here's my node:
https://atlas.torproject.org/#details/5AB2F49A1B79148E3A8F52808A00451DA00841B1
http://globe.rndm.de/#/relay/5AB2F49A1B79148E3A8F52808A00451DA00841B1

As you can see, it forwards on average between 2-4mbps (256-512 kB/sec) and there're peaks as high as 5.5mbps (~700 kB/s). The throughput varies depending on the day of week and time of day and therefore I believe the low average isn't the Raspberry's fault. I guess, if I set a higher advertised value, then I would see a more constant throughput closer to the peak value. Currently, it's set to 10mbps:

  RelayBandwidthRate 1024 KB
  RelayBandwidthBurst 1024 KB

(Note that the Pi is definitely not able to forward 10mbps of Tor traffic, but it would be great to max it out 24/7.)

The logged throughput is also consistent with what I saw with the console traffic monitor "nload" (suggested command line options for nicer units and less refreshes: nload -u K -U G -t 3000). I guess, I saw even higher peaks there. Monitoring everything with "arm" is also nice but far too CPU intensive.

More information about my setup:
- I have the B revision with 512 MB memory
- running stock Raspbian
- installed "tor" package 0.2.3.25-1 from the Raspbian repository
- CPU is overclocked using "raspi-config" to 950 MHz

Before I overclocked the Pi, I saw a similar average throughput.

When you search for "rasp" at Tor Atlas or Globe (e.g. http://globe.rndm.de/#/search/query=rasp) , you can see that there're already more than 40 devices running. My node performs quite well compared to others and I would argue that it's in the top 5 among all Raspberry Pi nodes so far.

To track down possible CPU problems, we should also compare the openssl benchmark results. The Raspberry Pi wiki has some reference values: http://elinux.org/RPi_Performance#Results_3

I've attached the summary output of "openssl speed" for my overclocked Pi (950 MHz) to this mail. As you can see, the numbers are higher than the reference one (I guess probably due to overclocking and different OpenSSL version).

I've also attached the log of notices of the last 4 days (unfortunately no longer logs available). Within four days there was only one "Your computer is too slow to handle this many circuit creation requests!" warning and one "Failed to hand off onionskin.". In general I would say, I haven't seen any serious issues so far.
 

And my plan is to publish my results to the entire list, because at $35,
Raspberry Pis can make *great* relays for slower home broadband, but
they need a little tender loving care first.  :)


I totally agree with you. Recently, there has been quite some buzz about the OnionPi howto (http://learn.adafruit.com/onion-pi/overview) but its main goal wasn't providing a long-running relay node. In my opinion, this should get addressed separately and made as simple as possible for novices. My guess is that many people want to contribute bandwidth, but they do not want to deal with Linux specifics (and definitely do not want to get into any trouble with the police). They need a plug-and-forget solution: attach the Pi to the router and leave it running in the closet for months. If it has to be restarted, Tor Weather can send them an email.

What do you think about documenting the setup of the relay in the Tor wiki?

https://trac.torproject.org/projects/tor/wiki

I'm not very familiar with the Tor Project yet, but to me it looks like the wiki will be the best place. Other people can contribute as well and it will be visible enough. It would be good if there's an always up-to-date wiki page which could become the reference for new users.

The idea of the Raspberry Pi as relay node could be even further expanded:
- Matthias already mentioned that there should be a specific image for it. I agree!
- New users would prefer a simple webinterface to configure the relay (basically a Vidalia as webinterface :-)
- Such a webinterface should also show statistics e.g., "Your node has forwarded 1 TB in the last 4 weeks." This will show people that their contribution made a difference and it will make them happy and more confident about it. Eventually, they'll convince others to run a relay as well.

I won't have the time to realize these ideas, but maybe others want to jump in. Further work and testing could be coordinated on this mailing list?

I hope to have something up in a week or two, I need to watch it for a
while and continue to tweak, and maybe develop a solution for the TCP
storms that can bring down a lot of consumer routers, before publishing
for all.

I run a regular Linux PC as router which forwards all the traffic of the Pi. The machine has more than enough CPU power and therefore I haven't seen any issues there. Since the two months that I run the relay node on the Pi, my Linux PC router has frozen twice (which didn't happen before). But I'm not sure that it can be related to the forwarding of the relay node traffic.

For a broader adaption it's definitely necessary to look into the behavior of the most popular routers e.g., the ones given out by the major providers. These things could be documented on the Tor Wiki as well.

Best regards,
Michael
OpenSSL 1.0.1e 11 Feb 2013
built on: Sun Mar 24 12:44:00 UTC 2013
options:bn(64,32) rc4(ptr,char) des(idx,cisc,16,long) aes(partial) blowfish(ptr)
compiler: gcc -fPIC -DOPENSSL_PIC -DZLIB -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -DL_ENDIAN -DTERMIO -g -O2 -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2 -Wl,-z,relro -Wa,--noexecstack -Wall -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DAES_ASM -DGHASH_ASM
The 'numbers' are in 1000s of bytes per second processed.
type             16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes
md2                  0.00         0.00         0.00         0.00         0.00
mdc2                 0.00         0.00         0.00         0.00         0.00
md4               2600.95k     9691.50k    30354.32k    64472.75k    96227.23k
md5               1909.94k     7068.47k    22301.54k    48309.51k    73045.79k
hmac(md5)         3494.14k    11837.23k    32889.41k    59205.69k    76712.62k
sha1              2125.37k     7237.09k    19115.72k    32521.76k    41017.92k
rmd160            1601.52k     5205.79k    13228.20k    21736.54k    26989.76k
rc4              47299.41k    54870.05k    57123.19k    57508.52k    57981.77k
des cbc           8557.03k     9046.09k     9171.39k     9199.22k     9189.29k
des ede3          3121.73k     3230.74k     3268.98k     3265.16k     3260.36k
idea cbc             0.00         0.00         0.00         0.00         0.00
seed cbc          9102.76k    11680.00k    12610.30k    12982.46k    13038.48k
rc2 cbc           8312.11k     8763.62k     8817.49k     8903.99k     8850.09k
rc5-32/12 cbc        0.00         0.00         0.00         0.00         0.00
blowfish cbc     16177.60k    17847.18k    18252.20k    18493.51k    18395.02k
cast cbc         13416.19k    15583.59k    16299.15k    16472.02k    16430.58k
aes-128 cbc      17918.61k    20484.20k    21281.82k    21499.00k    21562.22k
aes-192 cbc      15864.90k    17859.27k    18461.02k    18613.51k    18731.64k
aes-256 cbc      14200.77k    15928.39k    16346.46k    16412.43k    16577.08k
camellia-128 cbc     9985.35k    13278.78k    14391.91k    14761.70k    14805.67k
camellia-192 cbc     8591.10k    10699.11k    11420.34k    11568.38k    11744.62k
camellia-256 cbc     8608.10k    10741.85k    11419.23k    11601.68k    11618.99k
sha256            4872.42k    11548.39k    20366.52k    25060.60k    26904.83k
sha512            1484.95k     5915.80k     8530.88k    11663.67k    13030.74k
whirlpool          536.90k     1064.52k     1742.51k     2122.66k     2250.07k
aes-128 ige      15971.43k    18931.14k    19790.00k    20034.44k    19496.41k
aes-192 ige      14274.86k    16714.82k    17315.02k    17476.38k    17085.39k
aes-256 ige      13018.78k    14776.81k    15436.80k    15485.27k    15259.66k
ghash            21814.75k    23936.17k    24688.67k    24773.97k    24955.36k
                  sign    verify    sign/s verify/s
rsa  512 bits 0.001635s 0.000167s    611.5   5981.2
rsa 1024 bits 0.008259s 0.000472s    121.1   2117.3
rsa 2048 bits 0.053641s 0.001664s     18.6    601.0
rsa 4096 bits 0.390385s 0.006317s      2.6    158.3
                  sign    verify    sign/s verify/s
dsa  512 bits 0.001649s 0.001692s    606.6    590.9
dsa 1024 bits 0.004628s 0.005217s    216.1    191.7
dsa 2048 bits 0.016175s 0.018778s     61.8     53.3
                              sign    verify    sign/s verify/s
 160 bit ecdsa (secp160r1)   0.0013s   0.0047s    743.7    211.5
 192 bit ecdsa (nistp192)   0.0016s   0.0059s    619.8    168.3
 224 bit ecdsa (nistp224)   0.0020s   0.0077s    498.4    129.4
 256 bit ecdsa (nistp256)   0.0025s   0.0103s    404.9     96.9
 384 bit ecdsa (nistp384)   0.0049s   0.0231s    204.6     43.3
 521 bit ecdsa (nistp521)   0.0097s   0.0523s    103.6     19.1
 163 bit ecdsa (nistk163)   0.0036s   0.0122s    280.7     82.2
 233 bit ecdsa (nistk233)   0.0070s   0.0219s    143.5     45.6
 283 bit ecdsa (nistk283)   0.0104s   0.0403s     96.6     24.8
 409 bit ecdsa (nistk409)   0.0257s   0.0884s     38.9     11.3
 571 bit ecdsa (nistk571)   0.0619s   0.2076s     16.1      4.8
 163 bit ecdsa (nistb163)   0.0035s   0.0132s    289.1     75.6
 233 bit ecdsa (nistb233)   0.0068s   0.0241s    146.4     41.6
 283 bit ecdsa (nistb283)   0.0103s   0.0450s     96.7     22.2
 409 bit ecdsa (nistb409)   0.0258s   0.1017s     38.8      9.8
 571 bit ecdsa (nistb571)   0.0620s   0.2381s     16.1      4.2
                              op      op/s
 160 bit ecdh (secp160r1)   0.0036s    280.8
 192 bit ecdh (nistp192)   0.0047s    212.4
 224 bit ecdh (nistp224)   0.0062s    160.8
 256 bit ecdh (nistp256)   0.0081s    123.0
 384 bit ecdh (nistp384)   0.0189s     52.8
 521 bit ecdh (nistp521)   0.0418s     23.9
 163 bit ecdh (nistk163)   0.0059s    169.2
 233 bit ecdh (nistk233)   0.0107s     93.5
 283 bit ecdh (nistk283)   0.0199s     50.3
 409 bit ecdh (nistk409)   0.0441s     22.7
 571 bit ecdh (nistk571)   0.1030s      9.7
 163 bit ecdh (nistb163)   0.0064s    157.1
 233 bit ecdh (nistb233)   0.0118s     84.4
 283 bit ecdh (nistb283)   0.0221s     45.2
 409 bit ecdh (nistb409)   0.0501s     20.0
 571 bit ecdh (nistb571)   0.1185s      8.4
# zcat /var/log/tor/notices.log*.gz | sort | uniq
Jul 27 06:25:07.000 [notice] Tor 0.2.3.25 (git-3fed5eb096d2d187) opening new log file.
Jul 27 07:15:27.000 [notice] Heartbeat: Tor's uptime is 7 days 15:17 hours, with 638 circuits open. I've sent 407.08 GB and received 399.69 GB.
Jul 27 13:15:27.000 [notice] Heartbeat: Tor's uptime is 7 days 21:17 hours, with 1160 circuits open. I've sent 415.67 GB and received 408.17 GB.
Jul 27 19:15:27.000 [notice] Heartbeat: Tor's uptime is 8 days 3:17 hours, with 571 circuits open. I've sent 423.10 GB and received 415.48 GB.
Jul 28 01:15:27.000 [notice] Heartbeat: Tor's uptime is 8 days 9:17 hours, with 401 circuits open. I've sent 432.89 GB and received 425.16 GB.
Jul 28 06:25:09.000 [notice] Read configuration file "/etc/tor/torrc".
Jul 28 06:25:09.000 [notice] Read configuration file "/usr/share/tor/tor-service-defaults-torrc".
Jul 28 06:25:09.000 [notice] Received reload signal (hup). Reloading config and resetting internal state.
Jul 28 06:25:09.000 [notice] Tor 0.2.3.25 (git-3fed5eb096d2d187) opening new log file.
Jul 28 07:15:27.000 [notice] Heartbeat: Tor's uptime is 8 days 15:17 hours, with 428 circuits open. I've sent 439.80 GB and received 432.00 GB.
Jul 28 13:15:27.000 [notice] Heartbeat: Tor's uptime is 8 days 21:17 hours, with 794 circuits open. I've sent 445.19 GB and received 437.28 GB.
Jul 28 19:15:27.000 [notice] Heartbeat: Tor's uptime is 9 days 3:17 hours, with 1079 circuits open. I've sent 452.40 GB and received 444.36 GB.
Jul 29 01:15:27.000 [notice] Heartbeat: Tor's uptime is 9 days 9:17 hours, with 664 circuits open. I've sent 461.49 GB and received 453.30 GB.
Jul 29 06:25:08.000 [notice] Read configuration file "/etc/tor/torrc".
Jul 29 06:25:08.000 [notice] Read configuration file "/usr/share/tor/tor-service-defaults-torrc".
Jul 29 06:25:08.000 [notice] Received reload signal (hup). Reloading config and resetting internal state.
Jul 29 06:25:08.000 [notice] Tor 0.2.3.25 (git-3fed5eb096d2d187) opening new log file.
Jul 29 07:15:27.000 [notice] Heartbeat: Tor's uptime is 9 days 15:17 hours, with 779 circuits open. I've sent 468.98 GB and received 460.71 GB.
Jul 29 13:15:27.000 [notice] Heartbeat: Tor's uptime is 9 days 21:17 hours, with 1549 circuits open. I've sent 475.90 GB and received 467.46 GB.
Jul 29 19:15:27.000 [notice] Heartbeat: Tor's uptime is 10 days 3:17 hours, with 1344 circuits open. I've sent 484.08 GB and received 475.44 GB.
Jul 29 19:21:37.000 [warn] Failed to hand off onionskin. Closing. [33164 similar message(s) suppressed in last 21600 seconds]
Jul 29 19:21:37.000 [warn] Your computer is too slow to handle this many circuit creation requests! Please consider using the MaxAdvertisedBandwidth config option or choosing a more restricted exit policy. [68 similar message(s) suppressed in last 60 seconds]
Jul 30 01:15:27.000 [notice] Heartbeat: Tor's uptime is 10 days 9:17 hours, with 892 circuits open. I've sent 493.42 GB and received 484.64 GB.
Jul 30 06:25:09.000 [notice] Read configuration file "/etc/tor/torrc".
Jul 30 06:25:09.000 [notice] Read configuration file "/usr/share/tor/tor-service-defaults-torrc".
Jul 30 06:25:09.000 [notice] Received reload signal (hup). Reloading config and resetting internal state.
Jul 30 06:25:09.000 [notice] Tor 0.2.3.25 (git-3fed5eb096d2d187) opening new log file.
Jul 30 07:15:27.000 [notice] Heartbeat: Tor's uptime is 10 days 15:17 hours, with 1107 circuits open. I've sent 501.62 GB and received 492.74 GB.
Jul 30 13:15:27.000 [notice] Heartbeat: Tor's uptime is 10 days 21:17 hours, with 1225 circuits open. I've sent 510.17 GB and received 501.15 GB.
Jul 30 19:15:27.000 [notice] Heartbeat: Tor's uptime is 11 days 3:17 hours, with 1000 circuits open. I've sent 518.06 GB and received 508.89 GB.
Jul 31 01:15:27.000 [notice] Heartbeat: Tor's uptime is 11 days 9:17 hours, with 594 circuits open. I've sent 525.04 GB and received 515.74 GB.
Jul 31 06:25:07.000 [notice] Read configuration file "/etc/tor/torrc".
Jul 31 06:25:07.000 [notice] Read configuration file "/usr/share/tor/tor-service-defaults-torrc".
Jul 31 06:25:07.000 [notice] Received reload signal (hup). Reloading config and resetting internal state.
_______________________________________________
tor-relays mailing list
tor-relays@xxxxxxxxxxxxxxxxxxxx
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays