Re: [tor-talk] IDEA: Compress traffic at exit

[Roger Dingledine <arma@xxxxxxx>]

On Sat, Jan 21, 2012 at 02:39:53PM -0000, proper@xxxxxxxxxxx wrote:
> >    This is about compressing traffic at the exit, where it's passed from
> >    outside the network through other relays to the client where it
> >    gets decompressed.
> >
> >    The compression should happen on the fly.

Yep. We actually had this idea way back in 2003:
http://archives.seul.org/tor/dev/Mar-2003/msg00004.html

We even partially implemented it. The implementation got messy quicker
than expected, so we abandoned it.

It's possible that with the new libevent2 "filter layer" design it would
be a lot easier in the future.

We decided at the time that it was ok to abandon it on the theory that a)
most big things on the internet are compressed already, and b) if it's
not a big thing, then compressing it isn't really going to buy you a
whole lot. It would be great to see some numbers to support or refute
this theory.

> I see three different ways to compress traffic. 1), 2) and 2) They could
> be all implemented alone or all in combination.
> 
> 1) website compression
> Long time ago I tested services like traffic compressor (a free legitimate
> public proxy). It really enhanced the speed, especially websites with lots
> of high quality pictures.

Theory: Good webservers compress by default if the browser headers
indicate it's supported; and good browsers offer to receive compressed
content by default.

Again, it would be great to get some numbers to support/refute. It would
also be good to confirm that we don't screw any of that up with Torbutton.

> 2) caching proxy
> Just like existing caching proxys. The exit server would safe bandwidth if
> they wouldn't always request all websites fresh but use a local cache.
> Also option for both, Tor-user and exit server if they wish to use the
> feature.

This idea was also listed on the 2003 post. :)

For more details, there's even a paper written about the topic:
http://freehaven.net/anonbib/#shsm03

This idea comes with an additional advantage: it can improve anonymity,
since it breaks up the timing and volume characteristics of the flows
at each side.

Ultimately, the deal-breaker here is that the legal liability question
shifts dramatically when you change from being a transit provider ("bytes
go in, bytes go out") to having content stored "at" the relays. From a
technical perspective we might just put the cache in ram and thus there's
no stored content; but it's really not worth introducing the possibility
that some poor exit relay operator will have to sit down with the judge,
jury, and lawyers and try to teach them how computers work -- in terms
that fit a telephone metaphor since that's what most of the laws expect.

--Roger

_______________________________________________
tor-talk mailing list
tor-talk@xxxxxxxxxxxxxxxxxxxx
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-talk