[Author Prev][Author Next][Thread Prev][Thread Next][Author Index][Thread Index]
[tor-bugs] #4178 [TorDNSEL/TorBEL]: Specify a TorBEL archive data format
#4178: Specify a TorBEL archive data format
-----------------------------+----------------------------------------------
Reporter: karsten | Owner:
Type: enhancement | Status: new
Priority: normal | Milestone:
Component: TorDNSEL/TorBEL | Version:
Keywords: | Parent:
Points: | Actualpoints:
-----------------------------+----------------------------------------------
Sebastian says that TorBEL only gives out ''current'' scan results in its
CSV file (and others). Once a relay disappears from the consensus, TorBEL
removes scan results for the relay, so that its IP address can get
unblocked by Wikipedia et al. as soon as possible. The CSV file is going
to be updated every five minutes.
Makes sense for TorBEL's main use case. But we want to archive TorBEL's
output files and use it as input for VisiTor, ExoneraTor, and similar
tools. We'll want to know whether a relay was found to exit via a given
IP address at a given time. But we want to avoid archiving every output
files that TorBEL publishes, which would be highly redundant.
How about we define an archive data format that extends TorBEL's CSV
format in Section 2.1 of
[https://gitweb.torproject.org/torbel.git/blob_plain/master:/doc/data-
spec.txt data-spec.txt]? The change would be that we never remove an
entry for a given ExitAddress and Router ID. There could be many such
entries, each with a distinct LastTestedTimestamp. The new uniqueness
criterion would be (ExitAddress, RouterID, LastTestedTimestamp).
We could define this archive data format in the same spec or only on
[https://metrics.torproject.org/formats.html formats.html]. We could
implement the new format by making the TorBEL host create a copy of the
CSV file whenever it changes and having metrics-db rsync and merge these
files into the archive data format.
Once we have this archive data format, we'll have to update VisiTor and
ExoneraTor to parse it.
--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/4178>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online
_______________________________________________
tor-bugs mailing list
tor-bugs@xxxxxxxxxxxxxxxxxxxx
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs