Velope on IRC suggested a better workaround. It turns out these
connections actually end up in state INVALID when the transproxy side
dies. I tested this with my repro case and confirmed that the --ctstate
rule is working by itself.
Additional iptables rules inline below. Preserving full original
text for historical record.
Mike Perry:
> Hello all,
>
> I've discovered that the Linux kernel appears to have a leak in how it
> applies transproxy rules to the TCP CLOSE_WAIT shutdown condition under
> certain circumstances. This applies to both the kernels in use by common
> Android devices (Cyanogenmod 10.x and 11-M4), as well as the Linux
> kernel in Ubuntu 13.04 (3.8.0-35-generic).
>
> The bug can be triggered either by a remote server closing a connection,
> or by restarting the local tor client.
>
> Basically, the bug happens when a transproxy connection shuts down
> completely before a client application properly closes the socket. This
> seems to cause the kernel to lose track of the fact that the client
> application connection was being transproxied, and when the client
> application finally does close its socket (or exits), the Linux kernel
> generates a FIN ACK that completely bypasses any transproxy rules you
> have installed. It sends this packet first as the UID of the app in
> question, and if that fails, it resends it as a blank UID (the kernel
> itself).
>
>
> Here's how to reproduce it and see for yourself:
>
> First run the attached iptables script, and launch a tor daemon with the
> attached torrc (edit the iptables script's TOR_UID=`id -u debian-tor`
> and NETWORK_USER_ID=1000 vars if your setup is different).
>
> Then, fire up tcpdump, like so:
> sudo tcpdump -n -i wlan0 host 74.125.28.104 and tcp port 80
>
> Replace '-i wlan0' with your network interface. If you use '-i any', you
> will also see transproxied packets (which are not normally leaked).
>
> Then, as your transproxied user, paste this python snippet into a python
> interpreter:
> import socket
> s = socket.create_connection(("74.125.28.104", 80))
>
> (That IP handles www.google.com).
>
> After the connection is made, you should see something like the
> following in 'netstat -natp':
> tcp 0 0 127.0.0.1:9040 192.168.1.23:42235 ESTABLISHED 1121/tor
> tcp 0 0 192.168.1.23:42235 74.125.28.104:80 ESTABLISHED 977/python
>
> At this point, either wait a couple minutes for Google to close that
> connection on you, or shut down your Tor daemon. In either case, you
> should see the first connection transition to TIME_WAIT, or FIN_WAIT2,
> or similar, lose track of its PID+UID, and then finally disappear
> entirely, but the python program will remain in CLOSE_WAIT indefinitely.
>
> Once the first connection is fully gone (this takes 60s from TIME_WAIT
> state with default TCP settings), issue this in your python shell:
> s.close()
>
> At this point, you will see a FIN ACK or RST ACK packet appear in your
> tcpdump window. That packet has leaked past the iptables firewall rules,
> and past the transproxy rules. It went straight to Google.
>
> I have noticed several Android apps (including Firefox, F-Droid, and
> many Google apps and Android services) that allow their sockets to sit
> in CLOSE_WAIT upon remote close while transproxied, and they all leak
> packets in this case, which happens frequently in normal usage. I am not
> sure if this is just a common programming error, an issue with how the
> Android networking APIs are designed, something specifically exacerbated
> by the transproxy, or some combination of these.
>
>
> For a workaround, I was able to prevent this issue with the addition
> of the following rules:
> iptables -I OUTPUT ! -o lo ! -d 127.0.0.1 ! -s 127.0.0.1 -p tcp -m tcp --tcp-flags ACK,FIN ACK,FIN -j DROP
> iptables -I OUTPUT ! -o lo ! -d 127.0.0.1 ! -s 127.0.0.1 -p tcp -m tcp --tcp-flags ACK,RST ACK,RST -j DROP
Here's a set of rules to try both --ctstate and --state invalid, as well
as log which ones get hit, for testing purposes. Note the use of -A in
this case, for readability wrt ordering. These rules should come before
any other rule in the OUTPUT chain section of the firewall script you
use:
#iptables -A OUTPUT -m conntrack --ctstate INVALID -j LOG --log-prefix "Transproxy ctstate leak blocked: " --log-uid
iptables -A OUTPUT -m conntrack --ctstate INVALID -j DROP
iptables -A OUTPUT -m state --state INVALID -j LOG --log-prefix "Transproxy state leak blocked: " --log-uid
iptables -A OUTPUT -m state --state INVALID -j DROP
iptables -A OUTPUT ! -o lo ! -d 127.0.0.1 ! -s 127.0.0.1 -p tcp -m tcp --tcp-flags ACK,FIN ACK,FIN -j LOG --log-prefix "Transproxy leak blocked: " --log-uid
iptables -A OUTPUT ! -o lo ! -d 127.0.0.1 ! -s 127.0.0.1 -p tcp -m tcp --tcp-flags ACK,RST ACK,RST -j LOG --log-prefix "Transproxy leak blocked: " --log-uid
iptables -A OUTPUT ! -o lo ! -d 127.0.0.1 ! -s 127.0.0.1 -p tcp -m tcp --tcp-flags ACK,FIN ACK,FIN -j DROP
iptables -A OUTPUT ! -o lo ! -d 127.0.0.1 ! -s 127.0.0.1 -p tcp -m tcp --tcp-flags ACK,RST ACK,RST -j DROP
It's likely only the first pair is needed, and you may want to comment
out the --ctstate LOG line as I did to limit noise for successfully
handled --ctstate INVALID DROP blocks.
I did test this with the above repro method, and --ctstate INVALID did
appear sufficient by itself, but reports of any --ctstate DROP rule
bypass happening will be tremendously useful (which will result in the
later LOG lines being hit, and sending output to 'dmesg').
> None of the transproxy documentation I could find mentions this issue,
> nor suggests any additional safety rules. This means every transproxied
> Tor user is unwittingly leaking packets, at least some of the time.
> Sorry to be the bearer of bad news.
>
> Please send workaround discussion to tor-talk, and kernel/TCP state
> machine discussion to tor-dev. I Cc'd both like a jerk, because I figure
> each group might have different sets of commentary, and both groups
> should be aware of this issue. Don't be a jerk like me, please. Use your
> best judgment to Cc one list or the other.
>
>
>
> --
> Mike Perry
> #!/bin/bash
>
> IPTABLES=/sbin/iptables
> TOR_UID=`id -u debian-tor`
> NETWORK_USER_ID=1000
>
> # Clear existing rules
> $IPTABLES -F INPUT
> $IPTABLES -F OUTPUT
> $IPTABLES -t nat -F
>
> ## Transproxy rules for Tor
> $IPTABLES -t nat -A OUTPUT ! -d 127.0.0.1 -m owner ! --uid-owner $TOR_UID -p tcp -j REDIRECT --to-ports 9040 || exit
> $IPTABLES -t nat -A OUTPUT -p udp -m owner ! --uid-owner $TOR_UID -m udp --dport 53 -j REDIRECT --to-ports 5300 || exit
>
> # Allow Tor and the network user
> $IPTABLES -A OUTPUT -m owner --uid-owner $TOR_UID -j ACCEPT || exit
> $IPTABLES -A OUTPUT -m owner --uid-owner $NETWORK_USER_ID -j ACCEPT
> $IPTABLES -A INPUT -j LOG --log-prefix "OUTPUT DROPPED: " --log-uid || exit
> $IPTABLES -A OUTPUT -j DROP || exit
>
>
> # Create INPUT firewall. Allow established connections and transproxy
> $IPTABLES -A INPUT -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT || exit
> $IPTABLES -A INPUT -i lo -j ACCEPT # Transproxy output comes from lo
> $IPTABLES -A INPUT -d 127.0.0.1 -m udp -p udp --dport 5300 -j ACCEPT || exit
> $IPTABLES -A INPUT -j LOG --log-prefix "INPUT DROPPED: " --log-uid || exit
> $IPTABLES -A INPUT -j DROP || exit
> RunAsDaemon 1
> DataDirectory /var/lib/tor
> Log info file /var/lib/tor/log
> User debian-tor
> DNSPort 5300
> TransPort 9040
--
Mike Perry
Attachment:
signature.asc
Description: Digital signature
-- tor-talk mailing list - tor-talk@xxxxxxxxxxxxxxxxxxxx To unsubscribe or change other settings go to https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-talk