Proposal: Download consensus documents only when it will be trusted

[Peter Palfrader <peter@xxxxxxxxxxxxx>]

This came up in the More robust consensus voting with diverse authority
sets thread but is a different issue.

There are several open issues throughout the proposal and they are
marked with XXX in the text.


Filename: 134-robust-voting.txt
Title: Download consensus documents only when it will be trusted
Author: Peter Palfrader
Created: 2008-04-13
Status: Draft

Overview:

  Servers only provide consensus documents to clients when it is known that
  the client will trust it.


Motivation:

  When clients[1] want a new network status consensus they request it
  from a Tor server using the URL path /tor/status-vote/current/consensus.
  Then after downloading the client checks if this consensus can be
  trusted.  Whether the client trusts the consensus depends on the
  authorities that the client trusts and how many of those
  authorities signed the consensus document.

  If the client cannot trust the consensus document it is disregarded
  and a new download is tried at a later time.  Several hundred
  kilobytes of server bandwidth were wasted by this single client's
  request.

  With hundreds of thousands of clients this will have undesirable
  consequences when the list of authorities has changed so much that a
  large number of established clients no longer can trust any consensus
  document formed.

Objective:

  The objective of this proposal is to make clients not download
  consensuses they will not trust.

Proposal:

  The list of authorities that are trusted by a client are encoded in
  the URL they send to the directory server when requesting a consensus
  document.

  The directory server then only sends back the consensus when more than
  half of the authorities listed in the request have signed the
  consensus.  If it is known that the consensus will not be trusted
  a reasonable error code is sent back to the client.
  [XXX: What is a reasonable error code?
        "404 no pon^Wconsensus for you"?]

  This proposal does not require directory caches to keep more than one
  consensus document.  This proposal also does not require authorities
  to verify the signature on the consensus document of authorities they
  do not recognize.

  [XXX: Do caches keep signatures from authorities they do not recognize
        on the consensus document they download currently?  If not, they
        should.]
  [XXX: Is using the existence of a directory-signature block at the end
        of the consensus sufficient to tell if an authority has signed
        the consensus?  Can anybody just post signatures and we will
        distribute them, or will an authority that appends them to the
        consensus verify the signature, and that this signature's
        creator is listed in the consensus's dir-source?]

  The new URL scheme to download a consensus is
  /tor/status-vote/current/consensus/<F> where F is a list of
  fingerprints, sorted in ascending order, and concatenated using a +
  sign.

  Fingerprints are uppercase hexadecimal encodings of the authority
  identity key's digest.  Servers should also accept requests that
  use lower case or mixed case hexadecimal encodings.

  [XXX: Can we get away with a shortened hex digest?  40 characters per
        authority brings us up to 600 bytes just for fingerprint part of
        the GET URL, and some proxies might have issues with URLs that
        get much longer than that.  What's the worst that can happen
        when we only use the significant 2 or so bytes (4 hex chars)
        for this purpose?]

  A .z URL for compressed versions of the consensus will be provided
  similarly to existing resources and is the URL that usually should
  be used by clients.

Migration:

  The old location of the consensus should continue to work
  indefinitely.  Not only is it used by old clients, but it is a useful
  resource for automated tools that do not particularly care which
  authorities have signed the consensus.

  Authorities that are known to the client a priori by being shipped
  with the Tor code are assumed to handle this format.
  [XXX: or we add a new flag, "v3c" or whatever to the dirserver
        line.  It depends on how fast we can upgrade the
        authorities before we add this to client code.]

  When downloading a consensus document from caches clients only pick
  caches that are known to support this new format.
  [XXX OR: When downloading a consensus document from caches that
           do not support this new format they fall back to the old
           download location].

  Caches support the new format starting with Tor version [XXX].

Anonymity Implications:

  By supplying the list of authorities a client trusts to the directory
  server we leak information (like likely version of Tor client) to the
  directory server.  In the current system we also leak that we are
  very old - by re-downloading the consensus over and over again, but
  only when we are so old that we no longer can trust the consensus.



Footnotes:
 1. For the purpose of this proposal a client can be any Tor instance
    that downloads a consensus document.  This includes relays,
    directory caches as well as end users.