Re: Proposal 137: Keep controllers informed as Tor bootstraps

[Nick Mathewson <nickm@xxxxxxxxxxxxx>]

On Wed, Jun 11, 2008 at 01:38:24AM -0400, Roger Dingledine wrote:
 [...]

All notes are based on latest version from svn.

> 
>   The generic status event is:
> 
>     "650" SP StatusType SP StatusSeverity SP StatusAction
>                                         [SP StatusArguments] CRLF
> 
>   So in this case we send
>   650 STATUS_CLIENT NOTICE/WARN BOOTSTRAP \
>   PROGRESS=num TAG=string SUMMARY=string WARNING=string REASON=string
> 
>   "Progress" gives a number between 0 and 100 for how far through
>   the bootstrapping process we are. "Summary" is a string that can be
>   displayed to the user to describe the *next* task that Tor will tackle,
>   i.e., the task it is working on after sending the status event. "Tag"
>   is an optional string that controllers can use to recognize bootstrap
>   phases from Section 3, if they want to do something smarter than just
>   blindly displaying the summary string.

From a spec POV, you should put the optional aruments into square
brackets to indicate that they're optional.  There should also be the
standard line that controllers MUST ignore arguments they don't
recognize.

Are the strings quoted?  They should be.  But if this is so, I think
maybe REASON should be an identifier-like thing rather than a string:
using an identifier will discourage us from making cosmetic
compatibility-breaking improvements to reasons in the future.

 [...]
> 3. The bootstrap phases.
> 
>   This section describes the various phases currently reported by
>   Tor. Controllers should not assume that the percentages and tags listed
>   here will continue to match up, or even that the tags will stay in
>   the same order. Some phases might also be skipped (not reported) if the
>   associated bootstrap step is already complete.

Or if the associated bootstrap step is no longer done by some future
Tor version.  Step 50's future, for instance, seems to be threatened
by the pre-proposal-ideas in xxx-grand-scaling-plan.txt

Also, future Tor versions may use tags not listed now; controllers
MUST tolerate this.

Question: are bootstrapping events meant to be sent only during the
initial setup phase, or might they appear later if Tor needs to
bootstrap again?  In other words, should controllers expect to see
only a monotonically increasing series of phase numbers, or should
they be ready for a Tor to tell them, "I am no longer a bootstrapped
Tor; I'm back at phase X"?

>   Phase 0:
>   tag=starting summary="starting"
> 
>   Tor starts out in this phase. It doesn't actually send a status event
>   to say so.

What's it for, then?  

>   Phase 5:
>   tag=conn_dir summary="Connecting to directory mirror"
> 
>   Tor sends this event as soon as Tor has chosen a directory mirror ---
>   one of the authorities if bootstrapping for the first time or after
>   a long downtime, or one of the relays listed in its cached directory
>   information otherwise.

(This will not be so once/if the fallback-consensus feature is working
and emabled: Tors will try long-lived directory mirrors before falling
back to the authorities.)

>   Tor will stay at this phase until it has successfully established
>   a TCP connection with some directory mirror. Problems in this phase
>   generally happen because Tor doesn't have a network connection, or
>   because the local firewall is dropping SYN packets.
> 
>   Phase 10
>   tag=handshake_dir summary="Finishing handshake with directory mirror"
> 
>   This event occurs when Tor establishes a TCP connection with a relay
>   (or its https proxy if it's using one). Tor remains in this phase until
>   the TLS handshake with the relay is finished.

Do you mean to say "relay" here?  I know all directories are relays,
but it might be more reasonable to call the server in question a
"directory".
 
 [...]
>   Phase 25:
>   tag=loading_status summary="Loading networkstatus consensus"
> 
>   Once we've established a directory connection, we will start fetching
>   the networkstatus consensus document. This could take a while; this
>   phase is a good opportunity for using the "progress" keyword to indicate
>   partial progress.
> 
>   This phase could stall if the directory mirror we picked doesn't
>   have a copy of the networkstatus consensus so we have to ask another,
>   or it does give us a copy but we don't find it valid.

If we fall back to another directory, do we go back to phase 5?  It
would seem reasonable...

 [...]
>   The reason string is the same argument as the reason string for ORCONN
>   failure events; the controller can recognize the various reasons
>   and help the user accordingly. The warning string currently tries to
>   provide the equivalent of strerror() -- this isn't very useful if the
>   controller can recognize reason tags and be smarter, but for a very
>   simple controller it should be better than nothing.

When this is merged into the spec, you'll want to phrase this part
more specishly.  Something like:

   The reason strings are long-term-stable controller-facing tags to
   identify particular issues in a bootstrapping step.  The warning
   strings, on the other hand, are human-readable.  Controllers SHOULD
   NOT rely on the format of any warning string. 

We should also say what the scope of reason strings are.  Does a
reason always mean the same thing with tag X as it does with tag Y?

>   Currently Tor ignores the first nine bootstrap problem reports for
>   a given phase, reports the tenth to the controller, and then ignores
>   further problems at that phase. Hopefully this is a good balance between
>   tolerating occasional errors and reporting serious problems quickly. (We
>   will want to revisit this approach if there are many different 'reason'
>   values being reported among the first ten problem reports, since in
>   this case the controller will only hear one of them.)

This is ugly and fragile.  Instead of suppressing N failures and
reporting the N+1th, we should report all the failures, perhaps with a
count of how many have occurred so far or an indication of when we'll
next retry.  We should also offer advice to controller developers
about when they should report failures.  (Perhaps: "Wait for X
failures in any phase before deciding the phase is hopeless."  Or
perhaps: "After a failure, wait X seconds to see if Tor recovers and
the phase advances before deciding the phase is stalled.  Wait Y
seconds longer before deciding the phase is hopeless."  Or perhaps:
"If the retry time on any failure is 'now' or under 30 seconds from
now, Tor believes it can fix the problem itself.")

 [...]
>   Controllers should also have some mechanism to alert their user when
>   bootstrapping problems are reported. Perhaps we should gather a set of
>   help texts and the controller can send the user to the right anchor in a
>   "bootstrapping problems" help page?

This brings up an interesting issue.  There are lots of pieces of
translatable text that could reasonably used across different UIs.
We'd prefer not to stick them in Tor, since Tor doesn't get
internationalized.  But making each tool maintain their own lists is
also a little sad, since it results in a bit much duplication of
effort.  Thoughts?

-- 
Nick