[Author Prev][Author Next][Thread Prev][Thread Next][Author Index][Thread Index]
Re: [tor-talk] Pluggable transports for other projects...
TheMindwareGroup <themindwaregroup@xxxxxxxxx> writes:
> Bypassing DPI filters is a constantly evolving art form and entire
> field of research all on its own, and just like encryption extremely
> difficult to do well (check out bit torrent and emule obfuscation for
> example, a nice effort but when scrutinized not particularily effective).
>
> It would be nice if pluggable transports stenographic tunnels where
> easily available for other projects to use also maybe with an easy
> socks proxy interface like Tor uses or an api interface (dont know how
> you have implemented it).
>
> I am working on my own project but most of my effort is on getting the
> core up and running, I could throw in some transformation but it isnt
> going to be anywhere near as good as what Fornat Transforming
> Encryption or stegotorus could provide. Its the same problem with most
> programs getting over DPI firewalls is always gonna be a big problem.
>
> If you could post a page about how it works and how to use it on you
> site, specifically how to interface with it, it would be most helpful
> for other developers.
>
> ~Shadowman
>
> ~TheMindwareGroup
> TheMindwareGroup@xxxxxxxxx PGP: 0xf4b6586f
Greetings Shadowman,
I admit that the documentation resources on how to integrate PTs on
your project are not particularly good. This is mainly because not
many projects had expressed interest in the past and we had more
urgent things to do. Recently a few projects have asked us for help on
integrating PTs into their system. This is a good sign but it also
means that we should improve our documentation. A related trac ticket
is https://trac.torproject.org/projects/tor/ticket/10629
For starters, I suggest you read
https://gitweb.torproject.org/torspec.git/blob/HEAD:/pt-spec.txt .
This will give you an idea of how Tor uses pluggable transports. That
document can be vastly improved and I plan to do it "soon". Till then,
patches are more than welcome (I especially need to add some sections
and structure to that document, since it doesn't have a nice flow.)
Now, let me give you a small description of how Tor manages its
pluggable transports. It will not be ultra robust, but it might help
you understand what's going on:
Client side:
0) The user is supposed to use the torrc configuration file to inform
Tor about bridges, their pluggable transports and their
IP:port. This happens using the Bridge and ClientTransportPlugin
torrc directives (see the man page, the pt-spec or the source
code).
1) Tor now knows which bridges the user wants to connect to and which
pluggable transports it needs to use. It gets the filepath of the
pluggable transports from the ClientTransportPlugin line and gets
ready to start them up.
Before starting the PT, Tor needs to prepare some managerial
information that the PT needs to know. For example, the directory
that the PT should use if it needs to store permanent state, which
transports the PT application should initialize (if it supports
more than one transport) etc. To do so, Tor sets up the TOR_PT_*
environment variables accordingly. You can find more info about
those in the pt-spec.txt.
After Tor prepares the environment variables, it fires up the PT
(using execve() or whatever).
2) The PT application now parses the environment variables and
initializes the PTs that Tor asked it to use. Each PT starts up a
SOCKS listener that Tor is supposed to connect to. After the
initialization has been done, the PT sends a 'CMETHOD' line to
stdout that informs Tor of the IP:port that the PT SOCKS listener
is listening on, the SOCKS version that it uses etc. You can read
more about CMETHOD lines in pt-spec.
We've developed helper libraries that manage this step for you,
check out pyptlib, goptlib and liballium [0]. For example, you
might want to skim over this:
http://pyptlib.readthedocs.org/en/latest/API.html#general-overview
3) Tor now knows which SOCKS listener it should connect to for each
PT. When Tor wants to connect to a bridge that uses a PT, it does a
SOCKS CONNECT request to the appropriate SOCKS listener and asks it
to CONNECT to the destination (the destination should of course be
the server-side PT IP:port).
In case the PT requires further parameters for each connection
(e.g. if the server-side PT protocol requires a shared-secret) the
SOCKS handshake is (ab)used to transfer those parameters to the
PT. Currently we are using the username/password fields of the
SOCKS handshake to transfer those parameters, but we are not very
happy about this. For more info see:
https://trac.torproject.org/projects/tor/ticket/10671
4) When the PT receives a CONNECT request to its SOCKS listener, it
connects to the destination and starts relaying traffic between Tor
and the destination. Specifically, everytime the PT receives
application-layer traffic from Tor, it obfuscates it appropriately,
and relays it to the destination.
That's how the client side works. Now let's look at the server (bridge) side.
Server side:
0) The bridge operator is supposed to use the torrc configuration file
to inform Tor about the PTs it wants to support in her bridge. This
happens using the ServerTransportPlugin torrc directive (again see
the man page, the pt-spec or the source code).
1) Tor, again, prepares the environment variables that need to be
passed to the PT and spawns it.
In the server-side there is even more information that Tor needs to
pass to the PT. Take a look at the server-only TOR_PT_* environment
variables in pt-spec.txt. For example, Tor needs to instruct the PT
to bind on a specific IP:port, so that the PT listener stays in the
same place even after Tor reboots (Tor saves the IP:port in its
state file).
2) The PT, again, parses the environment variables, fires up its
listeners and informs Tor of its IP:port using SMETHOD lines in
stdout.
The pyptlib/goptlib/liballium libraries help with this step too.
3) When the PT receives traffic from a client, it deobfuscates it and
passes it to Tor.
In the server side, the PT uses a special protocol to communicate
with Tor. It's called the "Extended ORPort" protocol and its
basically a round of metadata transfer before the actual
application-layer traffic are forwarded to Tor. This metadata
includes the actual IP address of the client and the name of the PT
used. We use this information to keep statistics about the clients
that each bridge sees.
The Extended ORPort protocol is specified here:
https://gitweb.torproject.org/torspec.git/blob/HEAD:/proposals/196-transport-control-ports.txt
https://gitweb.torproject.org/torspec.git/blob/HEAD:/proposals/217-ext-orport-auth.txt
Ignore the TransportControlPort part of proposal 196, as that's not
implemented. Also, don't ask me why this stuff are not in
pt-spec.txt because I don't know; we should fix this.
For the extended OR port, you may also want to look at
some existing source code:
https://gitweb.torproject.org/pluggable-transports/obfsproxy.git/blob/b88efc0ce3f145d5e767d5b938889e99b47021b7:/obfsproxy/network/extended_orport.py
https://gitweb.torproject.org/pluggable-transports/goptlib.git/blob/abeea884f554b4119ebd84974c612c0dca6ce941:/pt.go#l581
So... this is how PTs work with Tor. Parts of it work like this
because we wanted PTs to work transparently; so that the end user
doesn't need to care at all about how PTs work and why they work like
that.
If you don't need all of this, you might enjoy the simpler "external
mode" functionality of PTs, as is specified in pt-spec.txt.
I already wrote a bit too much and I'm not really sure what you are
actually interested in so I'll shut up and wait for any questions you
might have.
Also, feel free to drop by #tor-dev at the OFTC IRC network for a more
synchronous communication experience. We also have biweekly PT
meetings in IRC every second Friday. Tomorrow is one of those Fridays
and we are going to gather there at 17:00 UTC. Feel free to join.
(BTW, I've been meaning to write a "How to integrate PTs with your
project" document for a while. I'll probably reuse parts of this post
when I do so.)
Cheers!
[0]: https://trac.torproject.org/projects/tor/wiki/doc/PluggableTransports#OtherPTsoftware
--
tor-talk mailing list - tor-talk@xxxxxxxxxxxxxxxxxxxx
To unsubscribe or change other settings go to
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-talk