teor: > > On 20 Dec 2017, at 06:06, meejah <meejah@xxxxxxxxx> wrote: > > >> This project is not quite there yet, and will require some > >> non-trivial engineering time, but it's probably a much easier task > >> compared to peerflow due to the design being more understood and > >> already coded. > > > > I'm not convinced this part is completely accurate ;) because at TorDev > > MTL it seems to me the consensus was that nobody actually knows what > > torflow is doing and so answering the question "is bwscanner doing the > > same thing" is approximately NP-hard. > > I have some idea what torflow is doing, in a broad sense: > * launch 2 tor clients > * repeat as often as possible, running 9 different scanners: > * split relays into buckets by bandwidth percentile > * build two hop paths with a relay and exit from relays in the bucket > * download a file from a bandwidth server, choose the size based on the bucket > * measure how long it takes > * store the results in a database > * aggregate the results hourly: > * produce a consensus weight to advertised bandwidth ratio > * using a decaying weighted average > * and some form of feedback (PID) control > * and dump it to a file > Then authorities read this file and include it in their votes. Yes, all of this is correct. Technically though full PID feedback is disabled right now. The PID-based implementation itself is enabled via bwauthpid=1 in the consensus, but the PID constants are currently set such that there is no actual feedback happening. See Section 3 of the Bw authority spec for more info: https://gitweb.torproject.org/torflow.git/tree/NetworkScanners/BwAuthority/README.spec.txt#n353 If feedback is enabled (via consensus parameters), it drives relays to other forms of resource exhaustion which we do not currently measure (primarily CPU exhaustion, which we could approximate by circuit failure, but potentially also memory pressure, which we have no signal for). > I suspect that Mike Perry may remember more detail, or may want to > correct my summary, as he wrote most of torflow (I think?) Yes. > > (I think the best path to answering "does bwscanner do the same thing as > > torflow" is to Run It And See...) If any of these parties are having > > problems deploying bwscanner this is probably something I can help with. Karsten wrote some scripts that can produce CDF graphs of bw authority votes for all of the flag combinations. This was very useful for determining if different bw authorities were measuring the network similarly. It will also be useful to see how closely the bwscanner is coming to the bwauth votes: https://trac.torproject.org/projects/tor/ticket/2394 I am not sure what repo they are in, though. > Let's start by specifying what tor directory authorities expect from the > file format. This format is already specified in Sections 2.4 and 3.4 of the bwauth spec itself: https://gitweb.torproject.org/torflow.git/tree/NetworkScanners/BwAuthority/README.spec.txt#n332 https://gitweb.torproject.org/torflow.git/tree/NetworkScanners/BwAuthority/README.spec.txt#n447 (This output should not be confused with Section 1.6, which specifies the intermediate sub-process format before aggregating results). -- Mike Perry
Attachment:
signature.asc
Description: Digital signature
_______________________________________________ tor-dev mailing list tor-dev@xxxxxxxxxxxxxxxxxxxx https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev