Hi Karsten, Nick,
Thanks for your feedback!
I've removed the sections without your comments, to keep this email short.
> On 10 Feb 2020, at 20:49, Karsten Loesing <karsten@xxxxxxxxxxxxxx> wrote:
>
>> On 2020-02-10 07:36, teor wrote:
>
> I'm including some comments below.
>
>> Here is an initial draft of Proposal 313: Relay IPv6 Statistics.
>>
>> This proposal includes:
>> * logging the number of IPv6 relays in the consensus, and
>> * relays publishing IPv6 connection and consumed bandwidth statistics.
>>
>> This is the third of 3 proposals:
>> * Proposal 311: Relay IPv6 Reachability
>> * Proposal 312: Automatic Relay IPv6 Addresses
>> * Proposal 313: Relay IPv6 Statistics
>>
>> ...
>>
>> The full text is included below, and it is also available as a GitHub
>> pull request:
>> https://github.com/torproject/torspec/pull/108
>>
>> The related tickets are #33159 (proposal) and #33051 and #33052
>> (implementation):
>> https://trac.torproject.org/projects/tor/ticket/33159
>> https://trac.torproject.org/projects/tor/ticket/33051
>> https://trac.torproject.org/projects/tor/ticket/33052
>>
>> ...
>>
>> Filename: 313-relay-ipv6-stats.txt
>> Title: Relay IPv6 Statistics
>> Author: teor
>> Created: 10-February-2020
>> Status: Draft
>> Ticket: #33159
>>
>> ...
>>
>> 3. Logging IPv6 Relays in the Consensus
>>
>> We propose that relays (and bridges) log:
>> * the number of relays, and
>> * the consensus weight fraction of relays,
>> in the consensus that:
>> * have an IPv6 ORPort,
>> * support IPv6 reachability checks, and
>> * support IPv6 clients.
> On 11 Feb 2020, at 01:21, Nick Mathewson <nickm@xxxxxxxxxxxxx> wrote:
>
> I don't understand the motivation behind doing this in the Tor code,
> since it's not something that relay operators need to know about or
> take action on. To me, it seems more like something do do as part of
> metrics than in Tor per se.
I agree, we don't need these logs in tor. These calculations are
medium-term, and some of them only apply to Sponsor 55.
Also, as Karsten said, "Usable Guards" definition doesn't match Onionoo,
so these calculations really don't belong in metrics, either.
I've modified this section so we just do these calculations in a script:
https://github.com/torproject/torspec/pull/108/commits/91356f5db02b6a62afa3061278872b8d607db7ea
>> In order to test these changes, and provide easy access to these
>> statistics, we propose implementing a script that:
>> * downloads a consensus, and
>> * calculates and reports these statistics.
>>
>> As well as the statistics listed above, this script should also report the
>> following relay statistic:
>> * support IPv6 reachability checks and IPv6 clients.
>>
>> The following consensus weight fractions should divide by the total
>> consensus weight:
>> * have an IPv6 ORPort (all relays have an IPv4 ORPort), and
>> * support IPv6 reachability checks (all relays support IPv4 reachability).
>>
>> The following consensus weight fractions should divide by the
>> "usable Guard" consensus weight:
>> * support IPv6 clients, and
>> * support IPv6 reachability checks and IPv6 clients.
>>
>> "Usable Guards" have the Guard flag, but do not have the Exit flag. If the
>> Guard also has the BadExit flag, the Exit flag should be ignored.
>
> This definition is different from the one we're using in Onionoo for
> computing the "guard probability". There we include a relay with the
> Guard flag, regardless of whether it has the Exit and/or BadExit flag.
> Not sure if this matters and which definition is more useful, I just
> wanted to point out that they're different.
The Onionoo definition is long-term, see Nick's explanation:
> On 11 Feb 2020, at 01:21, Nick Mathewson <nickm@xxxxxxxxxxxxx> wrote:
>
> It seems to me that this rule should depend on the Wgd
> bandwidth-weights value ("Weight for Guard+Exit-flagged nodes in the
> guard Position"), right? (Right now that is zero, and I don't expect
> it to change.)
You're right, I've made that check part of the script design:
https://github.com/torproject/torspec/pull/108/commits/91356f5db02b6a62afa3061278872b8d607db7ea
Since I mainly expect to use the script for Sponsor 55 in 2020, I don't
propose a design for other values of Wgd. The script should just warn.
(These warnings might happen in chutney networks.)
>> We propose that these logs happen whenever tor:
>> * receives a consensus from a directory server, or
>> * loads a live, valid, cached consensus from disk.
>>
>> As an optional change, tor clients may also log this information. Some of
>> this information is not directly relevant to clients, but these logs may
>> help developers (and users).
>>
>> 4. Collecting IPv6 Consumed Bandwidth Statistics
>>
>> We propose that relays (and bridges) collect IPv6 consumed bandwidth
>> statistics.
>>
>> To minimise development and testing effort, we propose re-using the existing
>> "bw_array" code in rephist.c.
>>
>> In particular, tor currently counts these bandwidth statistics:
>> * read,
>> * write,
>> * dir_read, and
>> * dir_write.
>>
>> We propose adding the following bandwidth statistics:
>> * ipv6_read, and
>> * ipv6_write.
>> (The IPv4 statistics can be calculated by subtracting the IPv6 statistics
>> from the existing total consumed bandwidth statistics.)
>>
>> We propose adding a new BandwidthStatistics torrc option and consensus
>> parameter, which activates reporting of all these statistics. Currently,
>> the existing statistics are controlled by ExtraInfoStatistics, but we
>> propose using the new BandwidthStatistics option for them as well.
>>
>> The default value of this option should be "auto", which checks the
>> consensus parameter. If there is no consensus parameter, the default should
>> be 1. (The existing bandwidth statistics are reported by default.)
>>
>> TODO: Should we collect IPv6 bandwidth statistics on bridges?
>> On bridges, should bandwidth statistics be on or off by default?
>>
>> If we do want to collect bridge statistics, and we think it's safe,
>> modify proposals 311 and 312 to allow bridge statistics.
>>
>> ...
>
> Right now, bandwidth statistics are on by default on bridges. I'd think
> that turning them off by default as part of this proposal would be
> surprising. It would be better, in terms of less surprising, to have a
> replacement first before making that change.
We're focused on short-term tweaks in Sponsor 55, because it's a smaller
sponsor. So let's keep the changes small.
I've also moved the new BandwidthStatistics option to the
"Optional Changes" section:
https://github.com/torproject/torspec/pull/108/commits/e9a888f01ad86e8a9193de16ae956c8a62f3c6ae
> Regardless of defaults, I'd say that collecting IPv6 bandwidth
> statistics over a period of 24 hours is about as safe as collecting
> IPv4+IPv6 bandwidth statistics over a period of 24 hours.
Thanks, I've updated the proposal with this reasoning:
https://github.com/torproject/torspec/pull/108/commits/e9a888f01ad86e8a9193de16ae956c8a62f3c6ae
>> 5. Collecting IPv6 Connection Statistics
>>
>> We propose that relays (and bridges) collect IPv6 connection statistics.
>>
>> To minimise development and testing effort, we propose re-using the existing
>> "bidi" code in rephist.c. (This code may require some refactoring, because
>> the "bidi" totals are globals, rather than a struct.)
>>
>> In particular, tor currently counts these connection statistics:
>> * below threshold,
>> * mostly read,
>> * mostly written, and
>> * both read and written.
>>
>> We propose adding IPv6 variants of all these statistics. (The IPv4
>> statistics can be calculated by subtracting the IPv6 statistics from the
>> existing total connection statistics.)
>>
>> We propose using the existing ConnDirectionStatistics torrc option, and
>> adding a consensus parameter with the same name. This option will control
>> the new and existing connection statistics.
>>
>> The default value of this option should be "auto", which checks the
>> consensus parameter. If there is no consensus parameter, the default should
>> be 0. (The existing connection direction statistics are reported by
>> default.)
>>
>> TODO: Do enough relays report ConnDirectionStatistics, for accurate IPv6
>> connection statistics?
>> * at least 25% of relays have IPv6
>> * at the end of the project, we expect at least 33% of relays to have
>> deployed tor 0.4.4-stable
>>
>> If not, we should turn on ConnDirectionStatistics by default. (Or set the
>> consensus parameter for a few days, to collect these statistics.)
>
> It looks like these statistics are turned off by default, yet they are
> reported in 79,709 out of 80,468 recent extra-info descriptors I just
> looked at. Something's wrong in the current code, though I didn't spot
> it at a quick glance. If it's accidentally turned on by default, I think
> that changing it to a consensus parameter and turning that on for a few
> days only would be a good solution.
That is odd. I've checked the code as well. We disable
ConnDirectionStatistics by default, and on bridges.
I've opened a trac ticket for this issue here:
https://trac.torproject.org/projects/tor/ticket/33214
It could be a bug in tor, or some large distributions (or the sample torrc)
that sets "ConnDirectionStatistics 1".
Fixing this bug is optional for Sponsor 55, but we should be aware of it,
because we want to modify the ConnDirectionStatistics code.
I've referenced ticket 33214 in the proposal, and moved the ConnDirStatistics
consensus parameter to the "Optional Changes" section:
https://github.com/torproject/torspec/pull/108/commits/e31ded5bd7769fd86c488b678a74fe877a0fa50c
I've also made some consequential updates to proposals 311 and 312:
https://github.com/torproject/torspec/pull/108/commits/a0492dd33cfc9d210fd9a4f762b779f452591668
Thanks again for your feedback!
T
Attachment:
signature.asc
Description: Message signed with OpenPGP
_______________________________________________ tor-dev mailing list tor-dev@xxxxxxxxxxxxxxxxxxxx https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev