At Roger's request, I examined the suitability of SCTP as a transport medium
for Tor.
Why do we care?
===============
Here's the background: right now, the Tor protocol transmits cells over TLS
over TCP. We multiplex all circuits going from a given OR (Call it "OR1")
and another ("OR2") over a single TLS-encrypted TCP stream.
This is problematic when TCP goes funny on us. Suppose the stream contains
cells on 3 circuits, like this:
Circ1, Circ1, Circ2, Circ3, Circ2, Circ1, Circ3, Circ1
Then suppose the TCP segment containing the first cell gets dropped and
needs to be retransmitted. There is no reason that the receiving OR
shouldn't be able to decrypt the subsequent cells, and keep circ2 and circ3
from blocking, delaying _only_ the cells on circ1 until the missing cell is
finally redelivered. But TCP implementations don't work that way; they
present a stream interface to applications, so that if segment N is dropped,
no subsequent segments are available to the application, even if they have
already been received.
How would it work?
==================
SCTP has the nice feature that is supports independent ordered streams within a
single SCTP "association". (It also has lots of stuff we don't need, like
multihoming and so forth.)
This would seem pretty handy for us, but the big problem is how to layer
encryption over SCTP. If we use one SCTP stream per circuit, then we can't
do our encryption on a per-stream level, or else an observer could easily
tell which cells belonged to which circuit. (RFC 3496 describes a way to do
this with TLS.) Instead, we need to encrypt the entire SCTP association.
More promising than RFC 3496's TLS-over-SCTP proposal is the SCTP-over-IPSec
proposal of RFC 3554. This does about what we would want: it encapsulates
the entire SCTP association with an IPSec Security Association (SA).
Why not do it right now?
========================
First off, there's not much support for SCTP. Not everybody _has_ SCTP, and
there may be firewall issues, and NAT may not do the right thing, and, and
...
And IPSec support is also a messy deal; very few people actually use it, and
if we need to tell people to reconfigure they networking stuff to get it to
work, then we're kinda sunk there too.
So it might work, but if we do it, we should do it as an alternative
transport, so that people don't need IPSec/SCTP to use it. It would make
the most sense for OR-to-OR links. But since we're hoping that everything
will get all P2P in future implementations, I'm not sure how much it'll help
for the especially-well-configured servers to get less blocking. If the bulk
of all servers still block circuits when TCP streams get messed up, then
most circuits will still block sometimes.
In fact, I'd guess that the servers most likely to deploy SCTP and IPSec are
likely to have *less* packet loss than average, all other things being equal,
since people who like to play with network software probably tend to have
better networks.
What could we do instead?
=========================
If we could tie into a user-space TCP implementation, or find a hacked TCP
implementation that let us read data as it's received (i.e., read around
gaps in the stream), that would work just fine too, and wouldn't require us
to change out protocol at all.
Another neat possibility is to look at iptables, ipf, and so on. We could
capture TCP segments as they arrive for the Tor in order to do readahead,
and then later read (and ignore) the stream from the kernel. The kernel
could handle all the important stuff (MTU discovery, acknowledgment,
retransmission, flow control, congestion avoidance, etc etc); all we'd have
to do is parse the segments enough to find out where stuff was along the TCP
stream.
(Of course, we'd need to figure out how to make OpenSSL do out-of-order
reading. But this is probably doable as well.)
Still, not easy. :)
HTH,
--
Nick Mathewson
(PGP key changed on 15Aug2004; see http://wangafu.net/key.txt)
Attachment:
pgp00000.pgp
Description: PGP signature