The value of specifications (was Re: flash in a stand-alone player)

[Roger Dingledine <arma@xxxxxxx>]

On Fri, Jul 27, 2007 at 04:26:21AM -0700, Kyle Williams wrote:
> http://janusvm.com
> 
> Transparent Proxy....enough said.

Hi folks,

Unfortunately, there is more that needs to be said before any of these
approaches for bundling Tor can be convincingly secure.

I tend to focus my attention on Steve when trying to explain the need
for documentation, because he seems most likely to get lots of users,
but I shouldn't pick on just him -- right now roughly none of the various
Tor bundling projects tell me exactly what's going on in detail.

So what do I mean by "in detail"? Let me take an aside to describe the
Tor specifications and why they're useful.

Our aim when writing the Tor spec documents is to provide sufficient
detail that somebody can build a compatible Tor implementation. That is,
to let people build something that a) does the same thing (has the same
features and behavior) and b) has the same level of security that Tor
aims to provide.

Beyond just letting people build compatible versions, the specs also
serve a critical role in security analysis. There are many security
people who are good at reading design docs and ok at reading specs,
and they can now look at both and tell us if we screwed up the spec
somewhere. We get many researchers coming to us saying "hey, I think if
you do it the way your spec says, you'll have a problem". On the other
side, there are also many security people who are good at looking at an
implementation and ok at reading specs, and they can look at both and
say "your spec says you do this, but the code doesn't do that."

The key is that these classes of security people are often different --
very few people are skilled at both. If we just had a design doc and an
implementation, then people who look at the code would have to divine
what we *meant* to do, as well as what we actually do, if they want to
find/fix a bug. Similarly, people who look at the design document would
have to guess what we meant for the code to do, because the design doc
isn't clear on exactly how to do each thing.

So that's the background, and the reasoning for why Tor has spec
documents. I believe our specs are one of the main reasons that Tor
is the standard anonymity system in the research community now. They
do take a lot of energy to write and maintain, but they also help us
make decisions carefully about what security properties we have now,
and what a given change would imply. And it helps the rest of the world
have confidence that we've gotten things right, because we also tell
them what we *mean* to do.

What I ultimately want from each of these bundling projects is all three
pieces: a design doc ("here are the intuitions for what we're trying to
do, and the assumptions we're working under"), a spec ("here is enough
detail to build your own, and if you followed this document we would
be confident that you built an ok one"), and the implementation itself
("here is the code, here are the scripts, here are the config files"). The
design doc needs to explain what behaviors and security properties you
have in mind, and the spec should include details of how you provide them.
A few examples may include: if you're trying to be subtle, what traces do
you leave behind that your program was there? What application behavior
do you modify?

I understand that not all of these projects are at the stages where
writing a spec makes sense. While you're still playing around with what
you want to include, or how you want it to be configured, it may be
premature to create a spec. But at some point it will make sense. And
doing it will help *you*, in that outside people can then evaluate your
security. As a side benefit, whoever produces a useful design doc and
spec first will become the de facto standard that everybody else has to
compare against, because you'll be the one with written recommendations.

What I would love to do is head toward a handful of unified documents
that say "If you want to bundle Tor in context X, here are the
recommendations. Follow these and you'll be well on your way to doing
it right." Because it is very likely that if xerobank wrote a spec and
janusvm wrote a spec, you would have different recommendations. Wouldn't
it be neat if we could recognize those and then discuss them and decide
which recommendation is better for each issue?

Hopefully this short discussion helps to explain why the Tor website
doesn't yet recommend any of the bundled Tors -- JanusVM, Xerobank,
Incognito, Torpark, Anonym.OS, etc. And why when people ask me what
I think of a given project, my answer tends to be "They seem like nice
people, and the project has a lot of potential. But heck if I know. What's
it do really?"

--Roger