[Author Prev][Author Next][Thread Prev][Thread Next][Author Index][Thread Index]

Re: [tor-dev] Trouble with onionperf visualize and S61 performance experiments

To: George Kadianakis <desnacked@xxxxxxxxxx>
Subject: Re: [tor-dev] Trouble with onionperf visualize and S61 performance experiments
From: Karsten Loesing <karsten@xxxxxxxxxxxxxx>
Date: Tue, 3 Nov 2020 17:16:06 +0100
Cc: "tor-dev@xxxxxxxxxxxxxxxxxxxx" <tor-dev@xxxxxxxxxxxxxxxxxxxx>
Delivered-to: archiver@xxxxxxxx
Delivery-date: Tue, 03 Nov 2020 11:16:24 -0500
Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:to:cc:references:from:message-id:date:user-agent :mime-version:in-reply-to:content-language:content-transfer-encoding; bh=Hxi/icTwhwsDSzeufmrJK/5CV56SbpUQUv3hDUTw1Mo=; b=BOxewIUbS9Wf1ON0dfernDG1UYTASE37+Cza37dgH/IgM06lgAWC+R+cJSZBcnKUv9 rcaLC7OhCa6U549CSHSVgRJvAUPV3uYN3Nk03xu6vxIsbLQduEZVbXVqJyy5KBRZlbD8 4NDYprOt2cntcnLPJdrohDm4UUySeOzDTKgOhdYiiP8ZKVXixfV305fR0Y7gOUYK8uCu q2GTMPYmhIe4M8gD4U/PZCnBGqNV/HXJJMNuBnUDw/dH7eC8qB+Q7GmS8ScLBDJEzr/Y jzci52IZ+aHo2kE5yRQYexG7fqPmhIuToyornuQvt17uqP28zWhZWH1qZeij1SzLQLDr MFig==
In-reply-to: <875z6mczeq.fsf@riseup.net>
List-archive: <http://lists.torproject.org/pipermail/tor-dev/>
List-help: <mailto:tor-dev-request@lists.torproject.org?subject=help>
List-id: discussion regarding Tor development <tor-dev.lists.torproject.org>
List-post: <mailto:tor-dev@lists.torproject.org>
List-subscribe: <https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev>, <mailto:tor-dev-request@lists.torproject.org?subject=subscribe>
List-unsubscribe: <https://lists.torproject.org/cgi-bin/mailman/options/tor-dev>, <mailto:tor-dev-request@lists.torproject.org?subject=unsubscribe>
References: <875z6mczeq.fsf@riseup.net>
Reply-to: tor-dev@xxxxxxxxxxxxxxxxxxxx
Sender: "tor-dev" <tor-dev-bounces@xxxxxxxxxxxxxxxxxxxx>
User-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0) Gecko/20100101 Thunderbird/78.3.3

On 2020-11-03 15:01, George Kadianakis wrote:
> Hello Karsten,

Hi George!

> hope you are doing well!
> 
> I've been working on the S61 performance experiments [0] and I would appreciate
> some help with onionperf.
> 
> I have done various onionperf measurements using something the following command:
>   $ onionperf measure -i --tgen ~/tgen/build/src/tgen --tor ~/onionperf/tor/src/app/tor --drop-guards 10
> 
> I put each of the measurements on a different directory and now I want
> to analyze them and derive the CDF-TTFB graphs etc. I attempted doing
> that using the following calls:
> 
>  $ onionperf analyze --tgen ./tgen-client/onionperf.tgen.log --torctl ./tor-client/onionperf.torctl.log
>  $ onionperf visualize --data onionperf.analysis.json.xz "test"
> 
> Unfortunately, the 'visualize' call can fail for the attached 'onionperf-mbps.json.xz':
> 
>   $ onionperf visualize --data onionperf.analysis.json.xz "Test Measurements"
>   2020-11-03 15:51:31 1604411491.540736 [onionperf] [INFO] loading analysis results from /user/tmp/onionperf/analysis/onionperf.analysis.json.xz
>   2020-11-03 15:51:31 1604411491.577864 [onionperf] [INFO] done!
>   2020-11-03 15:51:31 1604411491.586845 [onionperf] [INFO] NumExpr defaulting to 8 threads.
>   /user/.local/lib/python3.8/site-packages/OnionPerf-0.8-py3.8.egg/onionperf/visualization.py:251: UserWarning: Attempting to set identical left == right == -1e-06 results in singular transformations; automatically expanding.
>   /user/.local/lib/python3.8/site-packages/OnionPerf-0.8-py3.8.egg/onionperf/visualization.py:251: UserWarning: Attempting to set identical left == right == -1e-06 results in singular transformations; automatically expanding.
>   Traceback (most recent call last):
>     File "/user/.local/bin/onionperf", line 4, in <module>
>       __import__('pkg_resources').run_script('OnionPerf==0.8', 'onionperf')
>     File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 650, in run_script
>       self.require(requires)[0].run_script(script_name, ns)
>     File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 1453, in run_script
>       exec(script_code, namespace, namespace)
>     File "/user/.local/lib/python3.8/site-packages/OnionPerf-0.8-py3.8.egg/EGG-INFO/scripts/onionperf", line 622, in <module>
>     File "/user/.local/lib/python3.8/site-packages/OnionPerf-0.8-py3.8.egg/EGG-INFO/scripts/onionperf", line 382, in main
>     File "/user/.local/lib/python3.8/site-packages/OnionPerf-0.8-py3.8.egg/EGG-INFO/scripts/onionperf", line 522, in visualize
>     File "/user/.local/lib/python3.8/site-packages/OnionPerf-0.8-py3.8.egg/onionperf/visualization.py", line 48, in plot_all
>     File "/user/.local/lib/python3.8/site-packages/OnionPerf-0.8-py3.8.egg/onionperf/visualization.py", line 205, in __plot_throughput_ecdf
>     File "/user/.local/lib/python3.8/site-packages/OnionPerf-0.8-py3.8.egg/onionperf/visualization.py", line 235, in __draw_ecdf
>     File "/usr/lib/python3/dist-packages/pandas/core/frame.py", line 5000, in dropna
>       raise KeyError(list(np.compress(check, subset)))
>   KeyError: ['mbps']

Indeed, that's a bug in the visualize mode.

However, before it fails it writes a .csv file that tells us why: none
of the measurements are successful! I'm seeing lots of TOR/CANT_ATTACH
errors in that file. There's something wrong in your measurement setup.
If you fix that, you'll be able to visualize the results.

(We should still fix the bug and produce a nicer error message.)

> Then for another onionperf run (attached as 'onionperf-44028.json.xz') it gave me a different error:
> 
>   $ onionperf visualize --data onionperf.analysis.json.xz "Test Measurements"
>   2020-11-03 15:50:03 1604411403.946028 [onionperf] [INFO] loading analysis results from /user/tmp/onionperf/analysis/onionperf.analysis.json.xz
>   2020-11-03 15:50:03 1604411403.976088 [onionperf] [INFO] done!
>   Traceback (most recent call last):
>     File "/user/.local/bin/onionperf", line 4, in <module>
>       __import__('pkg_resources').run_script('OnionPerf==0.8', 'onionperf')
>     File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 650, in run_script
>       self.require(requires)[0].run_script(script_name, ns)
>     File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 1453, in run_script
>       exec(script_code, namespace, namespace)
>     File "/user/.local/lib/python3.8/site-packages/OnionPerf-0.8-py3.8.egg/EGG-INFO/scripts/onionperf", line 622, in <module>
>     File "/user/.local/lib/python3.8/site-packages/OnionPerf-0.8-py3.8.egg/EGG-INFO/scripts/onionperf", line 382, in main
>     File "/user/.local/lib/python3.8/site-packages/OnionPerf-0.8-py3.8.egg/EGG-INFO/scripts/onionperf", line 522, in visualize
>     File "/user/.local/lib/python3.8/site-packages/OnionPerf-0.8-py3.8.egg/onionperf/visualization.py", line 38, in plot_all
>     File "/user/.local/lib/python3.8/site-packages/OnionPerf-0.8-py3.8.egg/onionperf/visualization.py", line 129, in __extract_data_frame
>   KeyError: '44028'
> 
> Because of the two errors above I'm unable to proceed with producing any
> results.  Any idea what I might be doing wrong? Is there any other data
> you might need to help me with this?

In this case it's failing even before writing a CSV file. Still, when
looking at the .xz file I'm seeing only errors, too, so it's a related
issue here.

> Thanks a lot!
> 
> [0]: https://trac.torproject.org/projects/tor/wiki/org/roadmaps/CoreTor/PerformanceExperiments
>      https://gitlab.torproject.org/tpo/core/tor/-/issues/40157

Hope this helps!

All the best,
Karsten
_______________________________________________
tor-dev mailing list
tor-dev@xxxxxxxxxxxxxxxxxxxx
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

Follow-Ups:
- Re: [tor-dev] Trouble with onionperf visualize and S61 performance experiments
  - From: Karsten Loesing

References:
- [tor-dev] Trouble with onionperf visualize and S61 performance experiments
  - From: George Kadianakis

Prev by Author: Re: [tor-dev] Trouble with onionperf visualize and S61 performance experiments
Next by Author: [tor-dev] A new idea for email encryption on tor
Previous by thread: [tor-dev] Trouble with onionperf visualize and S61 performance experiments
Next by thread: Re: [tor-dev] Trouble with onionperf visualize and S61 performance experiments
Index(es):
- Author
- Thread