Re: Anonymity-preserving collection of usage data of a hidden service authoritative directory

[Roger Dingledine <arma@xxxxxxx>]

On Sat, May 05, 2007 at 12:10:08PM +0200, Karsten Loesing wrote:
> Maybe it's just a personal feeling (because I did not measure that yet), 
> but don't you think that introduction points change quite often? I 
> always thought that RSDs are republished so often, because it's anyway 
> unlikely that the set of IPos stays the same for more than one hour. 
> Thus, an RSD being 23 hours old simply cannot have any working IPos any 
> more.

Below are some data points from the hidden service I run on moria2. I've
filtered out everything except the publishes to tor26, so it's easier
to read. The server was started on May 04 06:25:04.044.

May 04 06:25:55.993 [notice] Making internal anonymized tunnel to 86.59.21.38:80 ...
May 04 07:13:55.535 [notice] Making internal anonymized tunnel to 86.59.21.38:80 ...
May 04 08:13:57.019 [notice] Making internal anonymized tunnel to 86.59.21.38:80 ...
May 04 09:13:57.798 [notice] Making internal anonymized tunnel to 86.59.21.38:80 ...
May 04 10:13:58.209 [notice] Making internal anonymized tunnel to 86.59.21.38:80 ...
May 04 11:13:59.748 [notice] Making internal anonymized tunnel to 86.59.21.38:80 ...
May 04 12:14:00.496 [notice] Making internal anonymized tunnel to 86.59.21.38:80 ...
May 04 13:03:12.085 [notice] Making internal anonymized tunnel to 86.59.21.38:80 ...
May 04 14:03:12.339 [notice] Making internal anonymized tunnel to 86.59.21.38:80 ...
May 04 15:03:13.389 [notice] Making internal anonymized tunnel to 86.59.21.38:80 ...
May 04 16:03:14.190 [notice] Making internal anonymized tunnel to 86.59.21.38:80 ...
May 04 17:03:15.536 [notice] Making internal anonymized tunnel to 86.59.21.38:80 ...
May 04 18:03:16.542 [notice] Making internal anonymized tunnel to 86.59.21.38:80 ...
May 04 19:03:17.095 [notice] Making internal anonymized tunnel to 86.59.21.38:80 ...
May 04 19:53:47.333 [notice] Making internal anonymized tunnel to 86.59.21.38:80 ...
May 04 20:53:48.344 [notice] Making internal anonymized tunnel to 86.59.21.38:80 ...
May 04 21:53:50.016 [notice] Making internal anonymized tunnel to 86.59.21.38:80 ...
May 04 22:53:50.788 [notice] Making internal anonymized tunnel to 86.59.21.38:80 ...
May 04 23:53:52.321 [notice] Making internal anonymized tunnel to 86.59.21.38:80 ...
May 05 00:53:53.691 [notice] Making internal anonymized tunnel to 86.59.21.38:80 ...
May 05 01:53:54.219 [notice] Making internal anonymized tunnel to 86.59.21.38:80 ...
May 05 02:53:55.770 [notice] Making internal anonymized tunnel to 86.59.21.38:80 ...
May 05 03:53:57.569 [notice] Making internal anonymized tunnel to 86.59.21.38:80 ...
May 05 04:53:58.125 [notice] Making internal anonymized tunnel to 86.59.21.38:80 ...
May 05 05:53:58.680 [notice] Making internal anonymized tunnel to 86.59.21.38:80 ...
May 05 06:54:00.045 [notice] Making internal anonymized tunnel to 86.59.21.38:80 ...
May 05 07:54:00.243 [notice] Making internal anonymized tunnel to 86.59.21.38:80 ...
May 05 08:54:01.635 [notice] Making internal anonymized tunnel to 86.59.21.38:80 ...
May 05 09:54:02.960 [notice] Making internal anonymized tunnel to 86.59.21.38:80 ...
May 05 10:54:03.371 [notice] Making internal anonymized tunnel to 86.59.21.38:80 ...
May 05 11:54:04.099 [notice] Making internal anonymized tunnel to 86.59.21.38:80 ...
May 05 12:54:05.122 [notice] Making internal anonymized tunnel to 86.59.21.38:80 ...
May 05 13:54:06.596 [notice] Making internal anonymized tunnel to 86.59.21.38:80 ...
May 05 14:54:07.875 [notice] Making internal anonymized tunnel to 86.59.21.38:80 ...
May 05 15:54:08.488 [notice] Making internal anonymized tunnel to 86.59.21.38:80 ...
May 05 16:54:09.505 [notice] Making internal anonymized tunnel to 86.59.21.38:80 ...
May 05 17:54:10.421 [notice] Making internal anonymized tunnel to 86.59.21.38:80 ...
May 05 18:54:11.488 [notice] Making internal anonymized tunnel to 86.59.21.38:80 ...
May 05 19:54:12.802 [notice] Making internal anonymized tunnel to 86.59.21.38:80 ...
May 05 20:02:11.932 [notice] Making internal anonymized tunnel to 86.59.21.38:80 ...
May 05 21:02:12.987 [notice] Making internal anonymized tunnel to 86.59.21.38:80 ...
May 05 22:02:13.801 [notice] Making internal anonymized tunnel to 86.59.21.38:80 ...
May 05 23:02:14.440 [notice] Making internal anonymized tunnel to 86.59.21.38:80 ...
May 06 00:02:15.903 [notice] Making internal anonymized tunnel to 86.59.21.38:80 ...
May 06 01:02:16.913 [notice] Making internal anonymized tunnel to 86.59.21.38:80 ...
May 06 02:02:17.906 [notice] Making internal anonymized tunnel to 86.59.21.38:80 ...
May 06 03:02:18.994 [notice] Making internal anonymized tunnel to 86.59.21.38:80 ...
May 06 03:24:43.176 [notice] Making internal anonymized tunnel to 86.59.21.38:80 ...
May 06 04:24:45.184 [notice] Making internal anonymized tunnel to 86.59.21.38:80 ...
May 06 05:24:46.250 [notice] Making internal anonymized tunnel to 86.59.21.38:80 ...
May 06 06:24:46.151 [notice] Making internal anonymized tunnel to 86.59.21.38:80 ...
May 06 07:24:48.077 [notice] Making internal anonymized tunnel to 86.59.21.38:80 ...
May 06 08:24:48.661 [notice] Making internal anonymized tunnel to 86.59.21.38:80 ...
May 06 09:24:49.442 [notice] Making internal anonymized tunnel to 86.59.21.38:80 ...
May 06 10:24:51.331 [notice] Making internal anonymized tunnel to 86.59.21.38:80 ...
May 06 10:29:51.751 [notice] Making internal anonymized tunnel to 86.59.21.38:80 ...
May 06 11:29:52.245 [notice] Making internal anonymized tunnel to 86.59.21.38:80 ...
May 06 12:29:53.650 [notice] Making internal anonymized tunnel to 86.59.21.38:80 ...
May 06 13:29:54.597 [notice] Making internal anonymized tunnel to 86.59.21.38:80 ...
May 06 14:29:55.343 [notice] Making internal anonymized tunnel to 86.59.21.38:80 ...
May 06 15:29:56.290 [notice] Making internal anonymized tunnel to 86.59.21.38:80 ...
May 06 16:29:57.957 [notice] Making internal anonymized tunnel to 86.59.21.38:80 ...
May 06 17:29:58.155 [notice] Making internal anonymized tunnel to 86.59.21.38:80 ...
May 06 18:29:59.763 [notice] Making internal anonymized tunnel to 86.59.21.38:80 ...
May 06 19:30:00.971 [notice] Making internal anonymized tunnel to 86.59.21.38:80 ...
May 06 20:30:01.849 [notice] Making internal anonymized tunnel to 86.59.21.38:80 ...
May 06 21:30:02.283 [notice] Making internal anonymized tunnel to 86.59.21.38:80 ...
May 06 22:30:03.434 [notice] Making internal anonymized tunnel to 86.59.21.38:80 ...
May 06 23:30:04.232 [notice] Making internal anonymized tunnel to 86.59.21.38:80 ...

It looks like we rotate at least one introduction point a few times a
day. But since we establish 3, it could be that at least one of them
lasts for the whole day or maybe much longer. Or not. :)

> >Well, if the client's clock is wrong by 23 hours, ...
> 
> >But you're right, the servers storing the descriptors should be
> >assumed to have better clocks, and they could just dump old ones to
> >save clients the trouble.
> 
> I am not sure if I get your arguments about clock skew right. Doesn't 
> clock skew only address two *different* clocks, e.g. a client's and a 
> directory node's clock? Then I agree that there should be some tolerance.

Well, there are three clocks involved. There's the hidden service
(who puts a timestamp in the rendezvous descriptor and signs it), the
HS directory server (who serves rendezvous descriptors and throws them
away if they're more than 3 days old), and the client.

The reason we're so lax in our clock enforcement is that the clock skew
between the hidden service (some random Tor user) and the client (some
other random Tor user) might be quite high.

But you're right that we could probably throw them out at the HS directory
authority more quickly.

> But when a directory node receives an RSD, it can note when that was and 
> discard it after 1.5 hours using its own clock. Regardless of a client's 
> clock, the descriptor is 1.5 hours old when discarding it and -- 
> possibly -- useless. The latter depends on how often IPos change. (I 
> think this would be the next thing to measure...)

Yep.

> >Though note in connection_ap_handshake_rewrite_and_attach() that
> >clients try to refetch a newer descriptor if the one they have cached
> >is more than 15 minutes old. Are you following all the details so
> >far? :)
> 
> Now that you ask... :) Why 15 minutes? So, clients consider RSDs to be 
> old after 15 minutes, servers after 60 minutes, but directories keep 
> them for 3 days?...

Well, the client doesn't throw it away after 15 minutes. It only pauses
while trying to fetch a new one to see if there is a new one -- if not
it goes ahead and uses the old one.

> >There is something that is making the
> >rendezvous itself be very slow. I'm not sure what it is. There's no
> >need for it to be as slow as it is. And I think it really reduces the
> >set of people who think hidden services are neat.
> 
> Then this might be one of the next things to investigate. I think it's 
> some timeout being too long or some operations that could/should be 
> performed twice/three times in parallel.

Yep. I think it's due to extremely high congestion on a few of the links
in the Tor network, which causes the rendezvous or introduction circuit
to fail and maybe we don't try again aggressively or quickly enough. The
"cannibalizing" step was supposed to take care of a lot of that, and
maybe it did, but there is still something wrong.

> These are the possible next tasks (arbitrary order):
> - Find out why connection establishment is that damn slow.
> - Measure how often IPos change.
> - Think about the format of RSDs (ASCII vs. binary), encryption of 
> contents and the related security implications.
> - Describe the protocol to receive/serve/replicate RSDs.
> 
> But enough measurements for the moment. I think I should think about 
> some concepts now and hence will start with the RSD format.

Sounds reasonable.

--Roger