[tor-commits] [sbws/master] docs: replace explanation on how to store the measurements

[juga@xxxxxxxxxxxxxx]

commit dea4bb3fbf2519270faa4eb914c5c56654a820ae
Author: juga0 <juga@xxxxxxxxxx>
Date:   Thu Feb 7 12:16:07 2019 +0000

    docs: replace explanation on how to store the measurements
---
 docs/source/specification.rst | 30 ++++++++----------------------
 1 file changed, 8 insertions(+), 22 deletions(-)

diff --git a/docs/source/specification.rst b/docs/source/specification.rst
index e2b687b..06f1ca1 100644
--- a/docs/source/specification.rst
+++ b/docs/source/specification.rst
@@ -164,30 +164,16 @@ Selecting the data to download
 
 Source code: :func:`sbws.core.scanner._should_keep_result`
 
-Simple result storage
-~~~~~~~~~~~~~~~~~~~~~
+Writing the measurements to the filesystem
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-Internally, sbws has a hierarchy of ``Result`` classes for easy managing of
-different types of result (success, error-because-of-circuit-error,
-error-because-[...] etc.). These results get converted into JSON strings and
-stored -- **one per line** -- in text files in a data directory.
+For every measured relay, the measurement result is put in a queue.
+There's an independent thread getting measurements from the queue every second.
+Every new measurement is appended to a file as a json line
+(but the file itself is not json!).
+The file is named with the current date. Every day a new file is created.
 
-The text files are simply named after the date. For example:
-``2018-03-20.txt``.
-
-The sbws scanner only appends to these files, and it automatically starts a new
-file when the system's clock ticks past midnight UTC.
-
-To avoid any weird timezone-related issues, consumers of sbws scanner data (such
-as the generate and stats scripts) SHOULD read more files than strictly
-necessary. For example, if the validity period is 5 days, they should read 6
-days of files. Because all results have a Unix timestamp, consumers of sbws
-data can easily determine which results are just outside the validity period as
-they are reading them in.
-
-
-Simple result processing
-~~~~~~~~~~~~~~~~~~~~~~~~
+Source code: :func:`sbws.lib.resultdump.ResultDump.enter`
 
 Every hour the directory authorities vote to come to a consensus about the
 state of the Tor network.  The bandwidth authorities need to use the results



_______________________________________________
tor-commits mailing list
tor-commits@xxxxxxxxxxxxxxxxxxxx
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-commits