[tor-relays] Re: How is this attack causing 900 Mbps download and 100 Mbps upload?

[khankrum359--- via tor-relays <tor-relays@xxxxxxxxxxxxxxxxxxxx>]

Hi,
A follow-up to my previous message, because the situation has developed in an interesting way.
In my original comparison, KhanKrumCloud3 was the affected relay, while KhanKrumCloud2, a similar 2-core ARM Guard+HSDir relay hosted by a different provider, appeared completely healthy and served as a useful comparison.
KhanKrumCloud3 later recovered spontaneously. I deliberately did not restart Tor and made no configuration changes. The abnormal BEGIN_DIR activity and the strong traffic asymmetry disappeared suddenly, while Tor continued running with the same PID.
Now KhanKrumCloud2, the relay that was previously healthy, has developed very similar behaviour.
KhanKrumCloud2 is hosted at Hetzner. During a two-hour monitoring run I observed BEGIN_DIR rates frequently in the range of approximately 1,400-2,300 per second. At the same time inbound traffic was usually only around 4-8 MB/s, while outbound traffic reached approximately 70-115 MB/s.
The Tor main thread was saturated and Tor repeatedly logged warnings about too many circuit creation requests.
The behaviour was also clearly intermittent.
At approximately 18:44 UTC, KhanKrumCloud2 was still processing more than 2,000 BEGIN_DIR streams per second and writing more than 100 MB/s. Less than a minute later BEGIN_DIR dropped to almost zero and outbound traffic fell below 1 MB/s.
The relay then remained mostly quiet for about 75 minutes.
At the end of the monitoring period, the behaviour returned very suddenly:
19:59:50 UTC - BEGIN_DIR 0/s, outbound about 1 MB/s, 2,991 open circuits
20:00:55 UTC - BEGIN_DIR 1,226/s, outbound about 57 MB/s, 5,502 open circuits
20:02:05 UTC - BEGIN_DIR 2,345/s, outbound about 115 MB/s, 9,029 open circuits
Tor remained the same process during these transitions.
I then upgraded KhanKrumCloud2 from Tor 0.4.9.13 to Tor 0.4.9.14, updated the kernel and rebooted the server.
After reboot, KhanKrumCloud2 came back normally with all its previous flags:
Fast, Guard, HSDir, Running, Stable, V2Dir, Valid
Unfortunately, the abnormal workload was still present.
A new 60-second measurement after the upgrade and reboot showed 31,353 BEGIN_DIR streams, or 522.55 BEGIN_DIR/s.
At the same time Tor was using around 130% CPU. Inbound traffic was only around 1-2 MB/s, while outbound traffic was again pinned at the configured relay bandwidth limit.
I have now reduced the bandwidth limits on KhanKrumCloud2 substantially:
RelayBandwidthRate 35 Mbits
RelayBandwidthBurst 50 Mbits
MaxAdvertisedBandwidth 34 Mbits
I did this mainly to protect the server's traffic quota.
This is becoming a practical problem for cloud-hosted relay operators, not only an interesting traffic anomaly. Most cloud providers meter or charge outbound traffic. During the worst periods, KhanKrumCloud2 was sending approximately 100-115 MB/s while receiving only a few MB/s.
Sustained at that level, this would correspond to roughly 8-10 TB of outbound traffic per day.
For a hobby relay operator using a cloud provider, that is enough to exhaust a monthly traffic allowance very quickly or potentially generate unexpected charges.
If other cloud-hosted relays are experiencing the same behaviour, it could realistically cause operators to throttle or shut down otherwise healthy relays.
I still cannot say whether this is an intentional attack, so I would prefer to describe it as abnormal or pathological traffic rather than claim a cause that I cannot prove.
What is especially interesting to me is that KhanKrumCloud2, which was healthy in my previous comparison and is hosted by a different provider from KhanKrumCloud3, has now developed the same type of BEGIN_DIR/circuit-creation workload.
Also, upgrading KhanKrumCloud2 to Tor 0.4.9.14 and rebooting did not eliminate it.
I would really appreciate guidance from anyone familiar with Tor's relay and circuit handling on what additional data would be useful. I have the full two-hour CSV monitoring capture, Tor logs and metrics available, and I can run specific diagnostics if that would help identify what is happening.
Regards,
KhanKrum
_______________________________________________
tor-relays mailing list -- tor-relays@xxxxxxxxxxxxxxxxxxxx
To unsubscribe send an email to tor-relays-leave@xxxxxxxxxxxxxxxxxxxx