Re: Plan for proposal 104 (was: New system for modifying Tor protocol)

[Roger Dingledine <arma@xxxxxxx>]

On Sun, Mar 11, 2007 at 11:10:07PM -0400, Nick Mathewson wrote:
> > http://tor.eff.org/svn/trunk/doc/spec/proposals/104-short-descriptors.txt
> > 
> > Solution 4 seems like a nice plan: in many cases, the external services
> > that use read-history and write-history are directory authorities
> > themselves, so they just use their local opinion.
> 
> To elaborate: solution 4 is the one where we move bandwidth and
> (later) other expensive, unused fields into a signed "extra info"
> document that goes to the directory authorities and that is wholly
> unlinked to the regular router descriptors.

Actually, solution 4 is the one where directory authorities concatenate
and serve the long descriptors at a special URL, similar to the v1
whole directory. (But yes, they are unlinked to the short descriptors.
I suspect we're in agreement here.)

> > I think we should go with the long/short descriptor plan, along
> > with solution 4. We don't want to just upload a bandwidth message,
> > because that involves new data structures for every new piece of
> > information we decide to upload. I suspect we'll realize once this
> > is deployed that there is other info we want to put in the long
> > descriptors.
> > 
> [...]
> > However, we may still need some basic reconciling algorithms between
> > authorities -- otherwise, if a router uploads to four authorities
> > and fails to reach the fifth, then that fifth will never have the new
> > descriptor. This will mean that the best strategy for external tools
> > is to fetch full concatenated-style long-descriptor lists from every
> > single authority, and merge them locally. So each authority should
> > periodically fetch the list from the others and take the new ones.
> 
> If each authority fetches from the others and takes new ones, that
> should be sufficient to ensure that tools only need to fetch from a
> single authority.  Tools fetching from all authorities instead of one
> would probably hurt bw worse than authorities keeping synchronized.

Right.

> > Is that it? Are there other pitfalls I'm missing? Are there better
> > options?
> 
> The remaining issue in my mind is: if later we decide that we need to
> keep "extra info" documents verifiably reconciled among authorities,
> will we then wish that we had done more now to make it possible to do
> so in the future.  In particular, upgrading authorities is easy, but
> upgrading all the servers on the network isn't.  What server-side work
> would we need to do to make extra-info documents reconcilable, and how
> much of it (if any) needs to happen now?

Notice that whenever we create a short descriptor we'll also be creating
a long descriptor. (Otherwise there's important info that the current
short descriptor has but the current long descriptor doesn't have, and
we'll be creating bad incentives for our tool writers again.) So we could
create them together, and put the same timestamp in each. If authorities
(or anybody else) ever sees two long descriptors with the same timestamp,
they know something funny is going on.

To add suspenders as well as belt, we could include a hash of the long
descriptor in the short descriptor -- basically "solution 3" in the
proposal. The caveat listed in the proposal isn't necessarily true if
we do solutions 3 and 4 together: you don't need to know which longdescs
to fetch if you just fetch all of them.

Yet another option, if we don't like the bandwidth waste of serving
v1-style longdesc directories, is to take advantage of the fact that
short and long descs are generated together: update a longdesc anytime
the networkstatus indicates there's a new shortdesc, and not otherwise.

--Roger