Re: Directory mirror and BandwidthRate/BandwidthBurst

[Roger Dingledine <arma@xxxxxxx>]

On Sat, Jun 02, 2007 at 03:09:35PM +0200, Stephan Lichtenhagen wrote:
> I'm running the node 'communicator' and upgraded from 0.1.1.25 to 0.1.2.14 
> last week. Since then I've been wondering why my node didn't want to 
> deliver the directory listing anymore - even at times when the used 
> bandwidth was _way_ below the configured maximum bandwidth of 1MB/s 
> ("HTTP/1.0 503 Directory busy, try again later").
> 
> After some experimenting I think I stumbled upon the answer: it seems to me 
> that tor tries to deliver the whole directory listing (roughly 2.6MB) in 
> one second. At least that seems to be the needed bandwidth tor is 
> calculating for that transaction. Thus you need to have configured a 
> BandwidthRate or BandwidthBurst of at least 2.6 MB/s to be able to deliver 
> directory listings (even more as the network of tor nodes grows).

Note that you're talking about the v1 directory:
https://tor.eff.org/svn/trunk/doc/spec/dir-spec-v1.txt
which is only used by Tor versions 0.1.0.x and earlier.

As you say, it is huge. I believe most of the people who fetch it these
days are a) directory mirrors who fetch a fresh copy in case somebody
asks for it, or b) humans like you fetching it manually. So I have no
problems with phasing it out. See also Nick's recent mail declaring
its imminent obsolescence:
http://archives.seul.org/or/talk/May-2007/msg00229.html

> Is this working as intended? If it is, then please add a hint about this to 
> the documentation. But in my opinion it would be sufficient to deliver the 
> directory listing in 5 or 10 seconds, reducing the needed bandwidth 
> accordingly.

I'd rather save your bandwidth for more useful directory stuff. More
generally, this was one of the tradeoffs I wrestled with when I was
putting the 'write limiting' feature in -- I wanted to make sure we
never went over our write limit, but I also wanted to minimize directory
overhead in general, since every byte we spend for directory stuff is
a byte we can't spend passing Tor traffic.

> Btw: would it be possible to (g)zip the directory listing before delivering 
> it? That would cut the file size down about 50%. 

We already do. Ask for it with ".z" appended to the end of the url.

Thanks!
--Roger