[Author Prev][Author Next][Thread Prev][Thread Next][Author Index][Thread Index]
Re: [tor-bugs] #30693 [Circumvention/Snowflake]: Delete old unsanitized logs
#30693: Delete old unsanitized logs
-------------------------------------+------------------------------
Reporter: dcf | Owner: (none)
Type: task | Status: needs_review
Priority: Medium | Milestone:
Component: Circumvention/Snowflake | Version:
Severity: Normal | Resolution:
Keywords: | Actual Points:
Parent ID: | Points:
Reviewer: | Sponsor:
-------------------------------------+------------------------------
Changes (by dcf):
* status: new => needs_review
Comment:
I have prepared a candidate sanitized CSV file extracted from the
sanitized logs. I've placed it in /var/log/snowflake-broker/broker.csv.xz
for evaluation. It's 6 MB compressed but 1.8 GB uncompressed.
The sanitized CSV looks like this:
{{{
timestamp,event,proxyid,clientid,additional
2017-07-21 23:40:00,proxy-gets-none,157,,no-clients
2017-07-21 23:40:00,proxy-polls,157,,
2017-07-21 23:40:00,proxy-gets-none,149,,no-clients
2017-07-21 23:40:00,proxy-polls,149,,
2017-07-21 23:40:00,client-offers,,,
2017-07-21 23:40:00,proxy-gets-offer,,,
2017-07-21 23:40:00,proxy-answers,,,
2017-07-21 23:40:00,client-gets-answer,,,
2017-07-21 23:40:00,proxy-polls,160,,
2017-07-21 23:40:00,proxy-gets-none,159,,no-clients
2017-07-21 23:40:00,proxy-polls,159,,
2017-07-21 23:40:00,proxy-gets-none,157,,no-clients
2017-07-21 23:40:00,proxy-polls,157,,
}}}
Timestamps are truncated to multiples of 10 minutes. Client and proxy IDs
are replaced by sequential integers.
The `event` column can take on these values:
* `start` the broker was restarted.
* `client-offers` a client connects, sends an offer, and awaits an
answer.
* `client-gets-answer` a client receives a proxy's answer (successful
broker match).
* `client-gets-none` a client disconnects without receiving an answer,
whether because of a timeout or because there were no proxies.
* `proxy-polls` a proxy connects in order to receive an offer.
* `proxy-gets-none` a proxy disconnects without receiving a client offer
(no clients).
* `proxy-gets-offer` a proxy receives a client offer.
* `proxy-answers` a proxy sends an answer to the broker.
* `error` an error; the most common is "http: TLS handshake error". Other
possibilities are "http2: server: error", "http2: received GOAWAY", or a
bad HTTP request. The `additional` column distinguishes these cases.
Some of these have relations to each other. For example `proxy-polls` ≈
`proxy-gets-none` + `proxy-gets-answer`.
Using the sanitized CSV, I made a couple of graphs. The first shows shows
the number of broker outcomes per day, where an outcome is one of the four
possibilities:
* A client and proxy are successfully linked up.
* A proxy connects but doesn't get a client.
* A client connects but doesn't get a proxy.
* Some other error occurred.
Click to embiggen.
[[Image(broker-interactions.png,100%)]]
The second graph shows the estimated number of proxies. This is just 10 ×
`proxy-polls` / s. It's based on the assumption that each proxy polls
every 10 s. The assumption doesn't hold when there are actually clients,
but as you can see the estimate is pretty close to 3, which is the number
of fallback proxy-go instances.
Click to embiggen.
[[Image(broker-estimated-proxies.png,100%)]]
--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/30693#comment:3>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online
_______________________________________________
tor-bugs mailing list
tor-bugs@xxxxxxxxxxxxxxxxxxxx
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs