[Author Prev][Author Next][Thread Prev][Thread Next][Author Index][Thread Index]
Re: [tor-bugs] #28877 [Core Tor/Tor]: Paginate large controller commands like 'GETINFO desc/all-recent'
#28877: Paginate large controller commands like 'GETINFO desc/all-recent'
--------------------------+--------------------------
Reporter: wagon | Owner: (none)
Type: defect | Status: assigned
Priority: Medium | Milestone:
Component: Core Tor/Tor | Version:
Severity: Normal | Resolution:
Keywords: | Actual Points:
Parent ID: | Points:
Reviewer: | Sponsor:
--------------------------+--------------------------
Comment (by wagon):
Hello, atagar. Thank you for the explanation, now it is more clear for me.
> saturates the control connection, preventing further commands and events
from being transmitted.
What it actually saturates? Indeed, in some setups I did see this problem,
but `tor-prompt` doesn't have it. Is it some internal TCP or socket thing
that is saturated? Can that limit be increased by setting some preference
in the system?
> but I've been cautioned that any direct use of tor's data directory is a
bad idea.
I agree. Furthermore, you cannot use tor's data directory if you are not
running Nyx as `debian-tor` user. If your user is just a member of
`debian-tor` group (this is recommended setup, see discussion in #25890),
you still cannot read files in that directory because of #28356.
> We need some way of breaking up these responses. Pagination probably
isn't a good fit so ideas welcome.
> The above is a long time problem I've had with tor's 'get all
descriptor' commands
Could you explain why it is a problem? Yes, it may block controller for a
few seconds if user type this command in Nyx interpreter, but only in this
case. This blocking would happen anyway because printing the output to a
terminal always requires some time. If you need this output internally in
some functions in Stem or Nyx, you can get it almost immediately. In
addition you don't need to run such commands very often. Maybe you need to
run them only once, when Nyx is started (then response can be cached for
some time).
To be clear, let us check exact numbers. To be sure we are not wrong here
because of specifics in implementation of Nyx or Stem, we can use the
following simple script `test.sh` for these tests:
{{{
#!/bin/bash -e
cmd="$@"
pass="ControlPortPassword"
function test_tor() {
echo "$1" >&3
sed "/^250 OK\r$/q" <&3
echo QUIT >&3
exec 3<&-
}
exec 3<>/dev/tcp/127.0.0.1/9051
echo AUTHENTICATE \"$pass\" >&3
read -u 3
test_tor "$cmd"
}}}
Now fun begins. Let us check some simple fast command:
{{{
$ time tor-prompt --run 'GETINFO version' 1>/dev/null
0.14s user 0.04s system 77% cpu 0.227 total
$ time ./test.sh 'GETINFO version' 1>/dev/null
0.00s user 0.00s system 36% cpu 0.011 total
}}}
Compare it with more heavy output such as `ns/all`:
{{{
$ time tor-prompt --run 'GETINFO ns/all' 1>/dev/null
0.28s user 0.07s system 58% cpu 0.591 total
$ time ./test.sh 'GETINFO ns/all' 1>/dev/null
0.02s user 0.00s system 8% cpu 0.230 total
}}}
Now check it with the most resource consuming command, `desc/all-recent`:
{{{
$ time tor-prompt --run 'GETINFO desc/all-recent' 1>/dev/null
Traceback (most recent call last):
File "/path/to/stem/tor-prompt", line 8, in <module>
stem.interpreter.main()
File "/path/to/stem/stem/interpreter/__init__.py", line 151, in main
interpreter.run_command(args.run_cmd, print_response = True)
File "/path/to/stem/stem/util/conf.py", line 289, in wrapped
return func(*args, config = config, **kwargs)
File "/path/to/stem/stem/interpreter/commands.py", line 381, in
run_command
print(output)
UnicodeEncodeError: 'ascii' codec can't encode character u'\u021b' in
position 1237805: ordinal not in range(128)
$ time ./test.sh 'GETINFO desc/all-recent' 1>/dev/null
0.21s user 0.03s system 60% cpu 0.391 total
}}}
People always say that shell is too slow and inconvenient, while python is
really fast and convenient. However, as you see, simple shell is 20-30
times faster than python on simple commands, about 2-3 times faster than
python on heavy commands, and gets many megabytes output of `desc/all-
recent` within less that half of second. Old UNIX style still rocks. Had
you choose Bash or some other shell instead of python for writing Stem?
If I don't redirect output to `/dev/null`, terminal becomes a bottleneck
for these commands. It will take about 5-6 seconds to print `decs/all-
recent` output to a terminal. Nyx interpreter is even more slow than `tor-
prompt` (I think it is due to curses), it will take 15 seconds or like
that for this printing. If I don't redirect output to `/dev/null`, the
last `tor-prompt` command doesn't crash.
Thus, I still doubt we need to ask tor network people to do something with
it. As you see, well written tools work fast with current `ControlPort`
implementation.
> Feel free to file a separate ticket with the 'nyx --debug' output when
Nyx freezes so I can see what's up.
I've created #28902. Now we have many tickets where I didn't get reply
from you yet, and our discussion stopped, so I'm afraid I overload you
with many tasks or questions.
--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/28877#comment:6>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online
_______________________________________________
tor-bugs mailing list
tor-bugs@xxxxxxxxxxxxxxxxxxxx
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs