[Author Prev][Author Next][Thread Prev][Thread Next][Author Index][Thread Index]
[tor-commits] [stem/master] Example in our tutorial for writing descriptors to disk and reading them back
commit 23042f197de3aaed34f7c29c12359165aeba5ef3
Author: Damian Johnson <atagar@xxxxxxxxxxxxxx>
Date: Mon Nov 17 22:10:49 2014 -0800
Example in our tutorial for writing descriptors to disk and reading them back
Neat suggestion by mmcc on...
https://trac.torproject.org/projects/tor/ticket/13774
---
docs/change_log.rst | 4 ++
docs/contents.rst | 1 +
docs/tutorials/double_double_toil_and_trouble.rst | 4 ++
docs/tutorials/examples/persisting_a_consensus.rst | 72 ++++++++++++++++++++
docs/tutorials/mirror_mirror_on_the_wall.rst | 55 ++++++++++++++-
5 files changed, 134 insertions(+), 2 deletions(-)
diff --git a/docs/change_log.rst b/docs/change_log.rst
index 1724497..7fdd2bf 100644
--- a/docs/change_log.rst
+++ b/docs/change_log.rst
@@ -71,6 +71,10 @@ The following are only available within Stem's `git repository
* The /info command errored for relays without contact information.
+ * **Website**
+
+ * Added an example for `writing descriptors to disk and reading them back <tutorials/mirror_mirror_on_the_wall.html#saving-and-loading-descriptors>`_ (:trac:`13774`)
+
.. _version_1.2:
Version 1.2
diff --git a/docs/contents.rst b/docs/contents.rst
index 15283f1..f7a49d2 100644
--- a/docs/contents.rst
+++ b/docs/contents.rst
@@ -17,6 +17,7 @@ Contents
tutorials/examples/exit_used
tutorials/examples/list_circuits
tutorials/examples/outdated_relays
+ tutorials/examples/persisting_a_consensus
change_log
download
diff --git a/docs/tutorials/double_double_toil_and_trouble.rst b/docs/tutorials/double_double_toil_and_trouble.rst
index 19ff0c5..ce6c58a 100644
--- a/docs/tutorials/double_double_toil_and_trouble.rst
+++ b/docs/tutorials/double_double_toil_and_trouble.rst
@@ -103,3 +103,7 @@ Descriptors
Provides information about the current votes from Tor's Bandwidth
Authorities.
+* `Saving and Loading a Tor Consensus <examples/persisting_a_consensus.html>`_
+
+ Example for writing a Tor consensus to disk, and reading it back.
+
diff --git a/docs/tutorials/examples/persisting_a_consensus.rst b/docs/tutorials/examples/persisting_a_consensus.rst
new file mode 100644
index 0000000..d9c681b
--- /dev/null
+++ b/docs/tutorials/examples/persisting_a_consensus.rst
@@ -0,0 +1,72 @@
+Saving and Loading a Tor Consensus
+==================================
+
+.. image:: /_static/buttons/back.png
+ :target: ../double_double_toil_and_trouble.html
+
+Reading and writing a Tor consensus to disk is similar to `other descriptor
+types <../mirror_mirror_on_the_wall.html#saving-and-loading-descriptors>`_
+with one small difference.
+
+Most descriptors are just about a single relay. Server descriptors and
+microdescriptors, for instance, can be concatenated together and dumped to a
+file because they're each independent of each other.
+
+The Tor consensus, however, is a larger document containing information about
+the Tor network in addition to a little data on each of the relays.
+
+In Stem the overall document is a
+:class:`~stem.descriptor.networkstatus.NetworkStatusDocumentV3`, and the
+information on individual relays are
+:class:`~stem.descriptor.router_status_entry.RouterStatusEntryV3` instances.
+
+Why does this matter? By default when you read a consensus Stem provides you
+**just** the :class:`~stem.descriptor.router_status_entry.RouterStatusEntryV3`.
+This is for performance reasons, and because usually that's what developers
+want. But for writing the conssensus to disk we'll want the whole document
+instead.
+
+So how do we get it? Just tell Stem that's what you want. The
+:class:`~stem.descriptor.__init__.DocumentHandler` tells Stem how to read the
+consensus. For example, to write the consensus simply do the following...
+
+::
+
+ from stem.descriptor import DocumentHandler
+ from stem.descriptor.remote import DescriptorDownloader
+
+ downloader = DescriptorDownloader()
+ consensus = downloader.get_consensus(document_handler = DocumentHandler.DOCUMENT).run()[0]
+
+ with open('/tmp/descriptor_dump', 'w') as descriptor_file:
+ descriptor_file.write(str(consensus))
+
+Our *consensus* here is the current
+:class:`~stem.descriptor.networkstatus.NetworkStatusDocumentV3`. The
+**descriptor_dump** file now looks like...
+
+::
+
+ network-status-version 3
+ vote-status consensus
+ consensus-method 18
+ valid-after 2014-11-17 23:00:00
+ fresh-until 2014-11-18 00:00:00
+ valid-until 2014-11-18 02:00:00
+ voting-delay 300 300
+ ... etc...
+
+You can then read it back with :func:`~stem.descriptor.__init__.parse_file`...
+
+::
+
+ from stem.descriptor import DocumentHandler, parse_file
+
+ consensus = next(parse_file(
+ '/tmp/descriptor_dump',
+ descriptor_type = 'network-status-consensus-3 1.0',
+ document_handler = DocumentHandler.DOCUMENT,
+ ))
+
+ for fingerprint, relay in consensus.routers.items():
+ print "%s: %s" % (fingerprint, relay.nickname)
diff --git a/docs/tutorials/mirror_mirror_on_the_wall.rst b/docs/tutorials/mirror_mirror_on_the_wall.rst
index 8fc196b..26a158c 100644
--- a/docs/tutorials/mirror_mirror_on_the_wall.rst
+++ b/docs/tutorials/mirror_mirror_on_the_wall.rst
@@ -8,6 +8,7 @@ with what they are and where to get them then you may want to skip to the end.
* :ref:`where-can-i-get-the-current-descriptors`
* :ref:`where-can-i-get-past-descriptors`
* :ref:`can-i-get-descriptors-from-the-tor-process`
+* :ref:`saving-and-loading-descriptors`
* :ref:`putting-it-together`
.. _what-is-a-descriptor:
@@ -138,8 +139,58 @@ through Tor's control socket...
from stem.descriptor import parse_file
- for desc in parse_file(open("/home/atagar/.tor/cached-consensus")):
- print "found relay %s (%s)" % (desc.nickname, desc.fingerprint)
+ for desc in parse_file('/home/atagar/.tor/cached-consensus'):
+ print 'found relay %s (%s)' % (desc.nickname, desc.fingerprint)
+
+.. _saving-and-loading-descriptors:
+
+Saving and loading descriptors
+------------------------------
+
+Tor descriptors are just plaintext documents. As such, if you'd rather not use
+`Pickle <https://wiki.python.org/moin/UsingPickle>`_ you can persist a
+descriptor by simply writing it to disk, then reading it back later.
+
+::
+
+ from stem.descriptor.remote import DescriptorDownloader
+
+ downloader = DescriptorDownloader()
+ server_descriptors = downloader.get_server_descriptors().run()
+
+ with open('/tmp/descriptor_dump', 'wb') as descriptor_file:
+ descriptor_file.write(''.join(map(str, server_descriptors)))
+
+Our *server_descriptors* here is a list of
+:class:`~stem.descriptor.server_descriptor.RelayDescriptor` instances. When we
+write it to a file this looks like...
+
+::
+
+ router default 68.229.17.182 443 0 9030
+ platform Tor 0.2.4.23 on Windows XP
+ protocols Link 1 2 Circuit 1
+ published 2014-11-17 23:42:38
+ fingerprint EE04 42C3 6DB6 6903 0816 247F 2607 382A 0783 2D5A
+ uptime 63
+ bandwidth 5242880 10485760 77824
+ extra-info-digest 1ABA9FC6B912E755483D0F4F6E9BC1B23A2B7206
+ ... etc...
+
+We can then read it back with :func:`~stem.descriptor.__init__.parse_file`
+by telling it the type of descriptors we're reading...
+
+::
+
+ from stem.descriptor import parse_file
+
+ server_descriptors = parse_file('/tmp/descriptor_dump', descriptor_type = 'server-descriptor 1.0')
+
+ for relay in server_descriptors:
+ print relay.fingerprint
+
+For an example of doing this with a consensus document `see here
+<examples/persisting_a_consensus.html>`_.
.. _putting-it-together:
_______________________________________________
tor-commits mailing list
tor-commits@xxxxxxxxxxxxxxxxxxxx
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-commits