[Author Prev][Author Next][Thread Prev][Thread Next][Author Index][Thread Index]
Re: [tor-dev] Help with rebasing arma's n23-5 tor branch to current master
Hi Charlie,
did you have the chance to simulate your rebased n23-5 branch using Shadow?
If not, maybe I can help by simulating it once and writing down more
specific instructions for you?
Thanks!
Karsten
On 8/8/13 10:36 AM, Karsten Loesing wrote:
> (Moving this discussion back to tor-dev@ with Charlie's permission.)
>
> On 8/6/13 4:10 AM, Charlie Belmer wrote:
>> Hey Karsten,
>>
>> I've been reading up on Shadow & Scallion to do some performance testing on
>> N23. Before I go too far down that road, I wanted to check in with you.
>>
>> Are there existing blueprints or best practices I should be following?
>
> Thanks for looking into this!
>
> So, I wouldn't call the following notes blueprints or best practices,
> but they are a description of what I would do to run Shadow simulations
> of your branch.
>
> First of all, be sure to read the Shadow wiki which has contains lots of
> interesting stuff: https://github.com/shadow/shadow/wiki
>
> The next step would be to get a vanilla Tor master (that your branch is
> based on) running in a minimal or tiny Shadow network. It's probably
> easiest to use a local Ubuntu VM for this. Or you can start an m1.large
> EC2 instance for the tiny network, or an even smaller instance for the
> minimal network.
>
> Once that simulation succeeds, you'll want to run another simulation
> using your branch, extract performance metrics using Shadow's analyze
> script, produce a PDF with graphs also using the analyze script, and
> then wonder if things are faster than before.
>
> The minimal or tiny networks are mostly there to learn that things are
> working as expected. You'll probably want larger networks for real
> performance results, but starting small to get everything working is
> probably easier.
>
> Sounds doable, I guess. However, here's where things start to get
> tricky: in order to simulate Tor master (or a branch based on it) in
> Shadow, you'll have to perform quite a few extra steps like compiling
> your own CMake and Clang/LLVM. Not really hard, but you need to get all
> pieces together before Shadow does what you want.
>
> I'm pasting detailed instructions that I used to simulate Steven's uTP
> branch in Shadow below as an example. Not everything applies to your
> branch, but it should give you the general idea.
>
> If something remains unclear, please feel free to ask!
>
>> Depending how it goes, I'd also look at re-trying adaptive 23 or tuning the
>> existing implementation.
>
> Neat! We should probably include Mashael in that discussion once you
> have a working and simulated n23-5 branch. Maybe she has ideas what
> parts need tweaking. I'm cc'ing her in this mail so she knows what
> you're working on.
>
> In any case, having a working rebased n23-5 branch is already a big step
> forward.
>
> Thanks!
> Karsten
>
>
> === START OF INSTRUCTIONS ===
>
> Launch new EC2 instance running Ubuntu Server 13.04 64 bit
>
> Set ulimit -n to 25000, see Shadow wiki for details
> https://github.com/shadow/shadow/wiki/Preparing-your-machine#system-configs-and-limits
>
> $ sudo apt-get update
> $ sudo apt-get upgrade
>
> Clone and test-compile modified libutp
>
> $ sudo apt-get install build-essential git automake libssl-dev libevent-dev
> $ mkdir src
> $ cd src/
> $ git clone https://github.com/sjmurdoch/libutp
> $ cd libutp/
> $ make
> $ cd ../
>
> Clone and test-compile Tor branch utp
>
> $ git clone https://git.torproject.org/tor.git
> $ cd tor/
> $ git remote add sjm217 https://git.torproject.org/sjm217/tor.git
> $ git fetch sjm217
> $ git checkout -b utp sjm217/utp
> $ ./autogen.sh
> $ LDFLAGS="-L/home/ubuntu/src/libutp" CFLAGS="-I/home/ubuntu/src/libutp"
> LIBS="-lutp -lrt" ./configure --disable-asciidoc
> $ make
> $ make distclean
> $ cd ../
>
> Modify Tor to use uTP for all links (or "0 &&" for not using uTP for any
> link)
>
> diff --git a/src/or/channeltls.c b/src/or/channeltls.c
> index 0551b73..b7b36e1 100644
> --- a/src/or/channeltls.c
> +++ b/src/or/channeltls.c
> @@ -418,7 +418,7 @@ channel_tls_connect(const tor_addr_t *addr, uint16_t
> port,
> /* Create a uTP connection */
> tor_addr_to_sockaddr(addr, port, (struct sockaddr*)&sin, sizeof(sin));
> tor_addr_to_str(addr_str, addr, sizeof(addr_str), 0);
> - if (!strncmp(addr_str, "128.232.10.129", sizeof(addr_str))) {
> + if (1 || !strncmp(addr_str, "128.232.10.129", sizeof(addr_str))) {
> log_info(LD_CHANNEL,
> "Trying uTP connection to %s", addr_str);
> tlschan->utp = UTP_Create(tor_UTPSendToProc, tlschan, (const struct
> sockaddr*)&sin,
>
> Install CMake 2.8.10
>
> $ wget http://www.cmake.org/files/v2.8/cmake-2.8.10.2.tar.gz
> $ tar xf cmake-2.8.10.2.tar.gz
> $ cd cmake-2.8.10.2/
> $ ./configure
> $ make
> $ export PATH=~/src/cmake-2.8.10.2/bin:$PATH
> $ cd ../
> $ which cmake # result: /home/ubuntu/src/cmake-2.8.10.2/bin/cmake
>
> Build Clang and LLVM from source
>
> $ wget http://llvm.org/releases/3.2/llvm-3.2.src.tar.gz
> $ wget http://llvm.org/releases/3.2/clang-3.2.src.tar.gz
> $ tar xaf llvm-3.2.src.tar.gz
> $ tar xaf clang-3.2.src.tar.gz
> $ cp -R clang-3.2.src llvm-3.2.src/tools/clang
> $ cd llvm-3.2.src/
> $ mkdir build
> $ cd build/
> $ cmake -DCMAKE_INSTALL_PREFIX=/home/ubuntu/.local ../.
> $ make
> $ make install
> $ export PATH=~/.local/bin/:$PATH
> $ cd ../../
> $ which clang # result: /home/ubuntu/.local/bin//clang
>
> Install Shadow
>
> $ sudo apt-get install gcc xz-utils make automake autoconf cmake tidy
> libtidy-dev libglib2.0 libglib2.0-dev dstat pdftk python2.7
> python-matplotlib python-numpy python-scipy
> $ git clone https://github.com/shadow/shadow.git
> $ cd shadow
> $ ./setup dependencies
> $ ./setup build -g -i /home/ubuntu/src/libutp -i
> /usr/include/x86_64-linux-gnu/c++/4.7 -l /home/ubuntu/src/libutp
> --tor-prefix /home/ubuntu/src/tor --tor-lib utp
> $ ./setup install
> $ export PATH=~/.shadow/bin/:$PATH
>
> Simulate tiny network in Shadow
>
> $ cd resource/examples/scallion/
> $ tar xf tiny-4GB-m1.large.tar.xz
> $ cd tiny-4GB-m1.large/
> $ scallion
>
> Graph simulation results
>
> Add one line to beginning of analyze.py:
>
> import matplotlib; matplotlib.use('Agg')
>
> For additional traffic simulations:
>
> $ grep -v "\[traffic-" scallion.log > scallion-notraffic.log
>
> $ cd ~/src/
> $ mkdir results
> $ cd results/
> $ python ~/src/shadow/contrib/analyze.py parse --output uTP-1
> ~/src/shadow/resource/examples/scallion/tiny-4GB-m1.large/data-utp-1/scallion.log
> $ python ~/src/shadow/contrib/analyze.py parse --output uTP-0
> ~/src/shadow/resource/examples/scallion/tiny-4GB-m1.large/data-utp-0/scallion.log
> $ python ~/src/shadow/contrib/analyze.py parse --output vanilla
> ~/src/shadow/resource/examples/scallion/tiny-4GB-m1.large/data-vanilla/scallion.log
> $ python ~/src/shadow/contrib/analyze.py plot --title "sjm217's utp
> branch (#9166)" --prefix "sjm217-utp" --data vanilla/ "vanilla
> 0.2.4.4-alpha" --data uTP-1/ "uTP for all links" --data uTP-0/ "uTP for
> none of the links"
>
> === END OF INSTRUCTIONS ===
>
>
_______________________________________________
tor-dev mailing list
tor-dev@xxxxxxxxxxxxxxxxxxxx
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev