[Author Prev][Author Next][Thread Prev][Thread Next][Author Index][Thread Index]

Re: [tor-talk] Why is there a huge gap between Total BW retrieved by Stem, Total shown on Tor Metrics and in http://torstatus.blutmagie.de/?

To: tor-talk@xxxxxxxxxxxxxxxxxxxx
Subject: Re: [tor-talk] Why is there a huge gap between Total BW retrieved by Stem, Total shown on Tor Metrics and in http://torstatus.blutmagie.de/?
From: Karsten Loesing <karsten@xxxxxxxxxxxxxx>
Date: Sun, 29 Mar 2015 18:07:03 +0200
Delivered-to: archiver@xxxxxxxx
Delivery-date: Sun, 29 Mar 2015 12:07:16 -0400
Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:message-id:date:from:user-agent:mime-version:to:subject :references:in-reply-to:content-type:content-transfer-encoding; bh=LLtEpsIK5HRnQRpz9q/iMGWghtaR23L6qo0Xdfif6+w=; b=Bhly0U2PZj/zSeyb38d2Z6WqGtJlXWKrevLYSzAsHjKzDZJPuAKcaRlFv2tjJAOFne uMCuC+mK2h0CtO7gIq7eX9Pg1ixp6P18nMYkZ3Xvp19keHUopSuRYN7XqmKCmFPOzXJr ZO4XgV2J/94nklHn0sJHKw+nMH5dEdfqGDOmZSVecLEjghM04f0YWpPS3sUhAMMCQXxt D+hLSKEiJfZucrmeYbt6Zybr/cXjuTnDYmvWf8GjT83IdQHPUfKsPx7JSo6ncRQSjx+5 torf2iTLrtFt2RbRyIe3aa51s9DMzsIdz5+whBor62rCi5QzcMDeIZZs7Woq+2eoTwjN JmWw==
In-reply-to: <CALRdebwtsHCr2HrTga_D3sb=Vi1MBUp2Q+Ooy++3idpGtNitiw@xxxxxxxxxxxxxx>
List-archive: <http://lists.torproject.org/pipermail/tor-talk/>
List-help: <mailto:tor-talk-request@lists.torproject.org?subject=help>
List-id: "all discussion about theory, design, and development of Onion Routing" <tor-talk.lists.torproject.org>
List-post: <mailto:tor-talk@lists.torproject.org>
List-subscribe: <https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-talk>, <mailto:tor-talk-request@lists.torproject.org?subject=subscribe>
List-unsubscribe: <https://lists.torproject.org/cgi-bin/mailman/options/tor-talk>, <mailto:tor-talk-request@lists.torproject.org?subject=unsubscribe>
References: <CALRdebwtsHCr2HrTga_D3sb=Vi1MBUp2Q+Ooy++3idpGtNitiw@xxxxxxxxxxxxxx>
Reply-to: tor-talk@xxxxxxxxxxxxxxxxxxxx
Sender: "tor-talk" <tor-talk-bounces@xxxxxxxxxxxxxxxxxxxx>
User-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:31.0) Gecko/20100101 Thunderbird/31.5.0

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 29/03/15 11:39, Anonymous Kyoto wrote:
> Thank you Dr. Karsten for your answer. Following your instruction,
> I am now understanding meaning of three terms: "advertised
> bandwidth", "bandwidth history", and "consensus weight" on Metrics.
> So please let me link those term with terms in Stem, and please 
> correct me if I am wrong cos I am using Stem Library so I really
> need to know which term in Metrics is associated with which term in
> Stem: Syntax: <term in Metrics = term in Stem>
> 
> 1. advertised bandwidth = observed_bandwidth
> 
>> (observed_bandwidth function is in: 
>> https://stem.torproject.org/api/descriptor/server_descriptor.html)
>
>> 
> to check it: I look at atlas of the IP: 94.198.68.13, and found
> advertised
>> bandwidth = 70.72 KB/s, and also using desc.observed_bandwidth in
>> stem to fetch the same IP, and got 70720 (B/s) by using these
>> code lines:
> 
> from stem.descriptor.remote import DescriptorDownloader
>> downloader = DescriptorDownloader() try:
> 
> for desc in downloader.get_server_descriptors().run():
> 
> print '%s %i' %(IP, desc.observed_bandwidth)

Advertised bandwidth is the minimum of bandwidth rate, bandwidth
burst, and observed bandwidth.  So, it may be the case that it's the
same as observed bandwidth, but better also consider the other two
bandwidth values.

> 2. bandwidth history = Not Available (?)
> 
>> I cannot find what function should be used in Stem to retrieve
>> the bandwidth history value.

Relays report a bandwidth history in their extra-info descriptors.
You'll have to sum up all values for a given day to obtain the daily
total and compute the daily average.  But beware of duplicates.  I'm
afraid this is not exactly trivial to compute using Stem.

> 3. consensus weight = bandwidth or = measured
> 
> This one maybe the weirdest one, but please correct me if I
> misunderstand
>> the terms. As I look at the consensus file @ 
>> http://86.59.21.38/tor/status-vote/current/consensus and look at
>> the specification for "w" line @ 
>> https://gitweb.torproject.org/torspec.git/tree/dir-spec.txt#n1768
>>
>> 
I found they wrote that "w line is An estimate of the bandwidth of this
>> relay, in an arbitrary unit (currently kilobytes per second)", in
>> stead of unit-less as you answered and written in Stem @ 
>> https://stem.torproject.org/api/descriptor/router_status_entry.html

Ah,
>> 
sounds like dir-spec indeed claims that consensus weight is
currently in kilobytes per second, but I'd strongly suggest not to
rely on that.  Better consider it a unit-less value and don't compare
it to the other bandwidth metrics.

> Next what I tried to figure out the question myself by writing
> these code lines:
> 
> from stem.descriptor.remote import DescriptorDownloader
>> downloader = DescriptorDownloader()
> 
> try:
> 
> for desc in downloader.get_consensus().run():
> 
> print '%s %i' %(IP, desc.measured)
> 
> 
> and get
> 
> 95.215.44.189 None
>> 73.15.150.172 None 162.247.72.201 None ...and so on
> 
> 
> While if I use this code lines:
> 
> from stem.descriptor.remote import DescriptorDownloader
>> downloader = DescriptorDownloader()
> 
> try:
> 
> for desc in downloader.get_consensus().run():
> 
> print '%s %i' %(IP, desc.bandwidth)
> 
> 
> I would get this result:
> 
> 95.215.44.189 323
>> 73.15.150.172 14 162.247.72.201 6240
> 
> 
> which is exactly the same with values in consensus file @ 
> http://86.59.21.38/tor/status-vote/current/consensus Therefore, I
> am confused about consensus weight (in metrics VS in directory 
> specification) VS bandwidth and measured (in Stem). Which of them
> is unit-less, and which of them is in B/s or KB/s. And, which value
> one should use to calculate probability of a relay in path
> selection in proportion to its bandwidth.
> 
> I am so sorry if the way I raise and organize the question make
> you confused or hard to understand. Especially to Dr. Karsten, I am
> so sorry to ask you about Stem in related to Metrics cos I am aware
> that Stem in not under your management.

I'm afraid I don't know enough about Stem to give advice good here.

Let me ask, though, what exactly you're trying to achieve?  Maybe
there's an easier way to achieve that than writing your own Stem script.

All the best,
Karsten

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
Comment: GPGTools - http://gpgtools.org

iQEcBAEBAgAGBQJVGCMnAAoJEJD5dJfVqbCrLhQH/2QLXmvgWdo8Ij8qmyzKs6gR
+mIQWt/0lSUPiuyBXR+7A9mq3jPgCTdWSt5RL24hWzBqIQ3cS6sbDXCnk9r5q2pX
t8/YV3hlfKy+H9Abd+XiTsTjBRygU/URVeBx6y0l5HQT7K+bvEPGCkGnHxr1PZyl
kZEBZtFjbadGyOxzfYquvC+xEToEp1p94/hVnQlivFQ2E5Ww6kkz+l8gZvMtbtl8
z3igaWeaAw918ZDNBAGp3ASBhIM7rVXU5qddHqjHe4MPbpI4cjFFqsrtOQ1ll9uZ
BVOOctp9R8GKqgCK0JRf0jq80QK22+DUksMNJa5NUI7SwT/MPCMb/Lk0+ScU6Mo=
=5jE5
-----END PGP SIGNATURE-----
-- 
tor-talk mailing list - tor-talk@xxxxxxxxxxxxxxxxxxxx
To unsubscribe or change other settings go to
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-talk

Follow-Ups:
- Re: [tor-talk] Why is there a huge gap between Total BW retrieved by Stem, Total shown on Tor Metrics and in http://torstatus.blutmagie.de/?
  - From: Moritz Bartl

References:
- Re: [tor-talk] Why is there a huge gap between Total BW retrieved by Stem, Total shown on Tor Metrics and in http://torstatus.blutmagie.de/?
  - From: Anonymous Kyoto

Prev by Author: Re: [tor-talk] Why is there a huge gap between Total BW retrieved by Stem, Total shown on Tor Metrics and in http://torstatus.blutmagie.de/?
Next by Author: Re: [tor-talk] Games Without Frontiers: Investigating Video Games as a Covert Channel
Previous by thread: Re: [tor-talk] Why is there a huge gap between Total BW retrieved by Stem, Total shown on Tor Metrics and in http://torstatus.blutmagie.de/?
Next by thread: Re: [tor-talk] Why is there a huge gap between Total BW retrieved by Stem, Total shown on Tor Metrics and in http://torstatus.blutmagie.de/?
Index(es):
- Author
- Thread