[Author Prev][Author Next][Thread Prev][Thread Next][Author Index][Thread Index]

Re: [tor-bugs] #6752 [Tor]: TestingTorNetwork doesn't lower the dir fetch retry schedules

To: undisclosed-recipients:;
Subject: Re: [tor-bugs] #6752 [Tor]: TestingTorNetwork doesn't lower the dir fetch retry schedules
From: "Tor Bug Tracker & Wiki" <blackhole@xxxxxxxxxxxxxx>
Date: Thu, 07 Mar 2013 13:40:33 -0000
Auto-submitted: auto-generated
Delivered-to: archiver@xxxxxxxx
Delivery-date: Thu, 07 Mar 2013 08:40:44 -0500
In-reply-to: <044.6bfc31bbf51050da56d143e920d606b2@xxxxxxxxxxxxxx>
List-archive: <http://lists.torproject.org/pipermail/tor-bugs>
List-help: <mailto:tor-bugs-request@lists.torproject.org?subject=help>
List-id: "auto: Tor bug tracker status mails" <tor-bugs.lists.torproject.org>
List-post: <mailto:tor-bugs@lists.torproject.org>
List-subscribe: <https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs>, <mailto:tor-bugs-request@lists.torproject.org?subject=subscribe>
List-unsubscribe: <https://lists.torproject.org/cgi-bin/mailman/options/tor-bugs>, <mailto:tor-bugs-request@lists.torproject.org?subject=unsubscribe>
References: <044.6bfc31bbf51050da56d143e920d606b2@xxxxxxxxxxxxxx>
Reply-to: tor-assistants@xxxxxxxxxxxxxx
Sender: tor-bugs-bounces@xxxxxxxxxxxxxxxxxxxx

#6752: TestingTorNetwork doesn't lower the dir fetch retry schedules
--------------------------------------+-------------------------------------
 Reporter:  arma                      |          Owner:                    
     Type:  enhancement               |         Status:  new               
 Priority:  major                     |      Milestone:  Tor: 0.2.4.x-final
Component:  Tor                       |        Version:                    
 Keywords:  tor-client small-feature  |         Parent:  #7172             
   Points:                            |   Actualpoints:                    
--------------------------------------+-------------------------------------

Comment(by karsten):

 I think I found out what prevented us from making new directory requests
 every 60 seconds as defined in the lowered refetch schedule: there's a
 hard-coded 5 minute timeout for letting a directory connection stall
 before expiring it: `DIR_CONN_MAX_STALL`.  We'll probably need to lower
 this timeout to, say, 30 seconds in a testing network.

 Also, I found a problem from lowered refetch schedules that we need to
 fix: there are (at least) four constants defining how many times we try a
 descriptor download before giving up:
 `CONSENSUS_NETWORKSTATUS_MAX_DL_TRIES`,
 `MAX_ROUTERDESC_DOWNLOAD_FAILURES`, `MAX_MICRODESC_DOWNLOAD_FAILURES`,
 `MAX_CERT_DL_FAILURES`. They're all set to 8, but we'll probably want to
 set them to 80 or 100 in a testing network.  If we don't do that,
 especially with retries every 60 seconds we may easily run into 7, 8, 9
 failed attempts, and then we stop trying.  This is particularly bad on
 non-clients which aren't triggered by an external application to reset the
 download schedule.  In my experiments I had some nodes not bootstrap at
 all within 1 hour if these constants are set to 8.

 If you like these changed constants in a testing network, I'll include
 them as five new torrc options in my patch.

 Want to see some results?  When I use the lowered refetch schedules and
 set the constants above as described, I get the following "Boostrapped
 100%" lines:

 {{{
 0:0:11:753547 [thread-0] 0:15:3:173889106 [scallion-message]
 [1uthority-73.1.0.0] [intercept_logv] [notice] Bootstrapped 100%: Done.
 0:0:11:763847 [thread-0] 0:15:3:289621246 [scallion-message]
 [2uthority-74.1.0.0] [intercept_logv] [notice] Bootstrapped 100%: Done.
 0:0:11:792273 [thread-0] 0:15:3:673929774 [scallion-message]
 [3uthority-75.1.0.0] [intercept_logv] [notice] Bootstrapped 100%: Done.
 0:0:12:794207 [thread-0] 0:15:21:967156005 [scallion-message]
 [nonexit2-84.1.0.0] [intercept_logv] [notice] Bootstrapped 100%: Done.
 0:0:12:859044 [thread-0] 0:15:22:882704137 [scallion-message]
 [exit1-78.1.0.0] [intercept_logv] [notice] Bootstrapped 100%: Done.
 0:0:13:226706 [thread-0] 0:15:28:358539690 [scallion-message]
 [webclient1-88.1.0.0] [intercept_logv] [notice] Bootstrapped 100%: Done.
 0:0:14:458496 [thread-0] 0:15:53:001822682 [scallion-message]
 [exit5-82.1.0.0] [intercept_logv] [notice] Bootstrapped 100%: Done.
 0:0:14:471866 [thread-0] 0:15:53:334226076 [scallion-message]
 [exit4-81.1.0.0] [intercept_logv] [notice] Bootstrapped 100%: Done.
 0:0:14:536720 [thread-0] 0:15:54:845187276 [scallion-message]
 [nonexit4-86.1.0.0] [intercept_logv] [notice] Bootstrapped 100%: Done.
 0:0:14:648466 [thread-0] 0:15:55:760034257 [scallion-message]
 [nonexit5-87.1.0.0] [intercept_logv] [notice] Bootstrapped 100%: Done.
 0:0:16:346193 [thread-0] 0:16:39:063729918 [scallion-message]
 [exit3-80.1.0.0] [intercept_logv] [notice] Bootstrapped 100%: Done.
 0:0:17:050978 [thread-0] 0:16:53:675256680 [scallion-message]
 [nonexit3-85.1.0.0] [intercept_logv] [notice] Bootstrapped 100%: Done.
 0:0:17:401589 [thread-0] 0:16:59:462751388 [scallion-message]
 [exit2-79.1.0.0] [intercept_logv] [notice] Bootstrapped 100%: Done.
 0:0:17:542252 [thread-0] 0:17:0:740213796 [scallion-message]
 [nonexit1-83.1.0.0] [intercept_logv] [notice] Bootstrapped 100%: Done.
 0:0:19:847435 [thread-0] 0:17:38:479260129 [scallion-message]
 [4uthority-76.1.0.0] [intercept_logv] [notice] Bootstrapped 100%: Done.
 0:0:23:362068 [thread-0] 0:20:14:179430946 [scallion-message]
 [5uthority-77.1.0.0] [intercept_logv] [notice] Bootstrapped 100%: Done.
 }}}

 The second timestamp in these log lines is simulated time
 (0:15:3:173889106 in the first line).  The very first consensus in this
 network is published at 0:15:0.  That means that the longest time to
 bootstrap is 2:38 minutes.  5uthority starts at time 0:20:2 and takes 12
 seconds to bootstrap.  This seems very acceptable.  For comparison, here's
 the situation before changing the five constants above:

 {{{
 0:0:11:532841 [thread-0] 0:15:3:107393402 [scallion-message]
 [1uthority-73.1.0.0] [intercept_logv] [notice] Bootstrapped 100%: Done.
 0:0:11:533525 [thread-0] 0:15:3:127368755 [scallion-message]
 [2uthority-74.1.0.0] [intercept_logv] [notice] Bootstrapped 100%: Done.
 0:0:11:675343 [thread-0] 0:15:4:268884973 [scallion-message]
 [3uthority-75.1.0.0] [intercept_logv] [notice] Bootstrapped 100%: Done.
 0:0:13:908788 [thread-0] 0:18:28:931568544 [scallion-message]
 [nonexit4-86.1.0.0] [intercept_logv] [notice] Bootstrapped 100%: Done.
 0:0:13:929487 [thread-0] 0:18:29:280680071 [scallion-message]
 [exit1-78.1.0.0] [intercept_logv] [notice] Bootstrapped 100%: Done.
 0:0:14:414258 [thread-0] 0:18:45:522758087 [scallion-message]
 [4uthority-76.1.0.0] [intercept_logv] [notice] Bootstrapped 100%: Done.
 0:0:16:414355 [thread-0] 0:20:13:955995654 [scallion-message]
 [5uthority-77.1.0.0] [intercept_logv] [notice] Bootstrapped 100%: Done.
 0:0:16:651176 [thread-0] 0:20:21:901359712 [scallion-message]
 [nonexit1-83.1.0.0] [intercept_logv] [notice] Bootstrapped 100%: Done.
 0:0:16:804762 [thread-0] 0:20:24:057107728 [scallion-message]
 [nonexit5-87.1.0.0] [intercept_logv] [notice] Bootstrapped 100%: Done.
 0:0:16:870519 [thread-0] 0:20:27:890874732 [scallion-message]
 [exit2-79.1.0.0] [intercept_logv] [notice] Bootstrapped 100%: Done.
 0:0:19:363695 [thread-0] 0:23:23:469750053 [scallion-message]
 [exit4-81.1.0.0] [intercept_logv] [notice] Bootstrapped 100%: Done.
 0:0:19:920079 [thread-0] 0:24:20:250686967 [scallion-message]
 [exit5-82.1.0.0] [intercept_logv] [notice] Bootstrapped 100%: Done.
 0:0:20:673098 [thread-0] 0:24:49:322543288 [scallion-message]
 [webclient1-88.1.0.0] [intercept_logv] [notice] Bootstrapped 100%: Done.
 0:0:21:749740 [thread-0] 0:25:38:208126411 [scallion-message]
 [nonexit3-85.1.0.0] [intercept_logv] [notice] Bootstrapped 100%: Done.
 }}}

 And finally, here's the situation before applying any patch:

 {{{
 0:0:11:448903 [thread-0] 0:15:2:582133247 [scallion-message]
 [3uthority-75.1.0.0] [intercept_logv] [notice] Bootstrapped 100%: Done.
 0:0:11:526322 [thread-0] 0:15:3:715646237 [scallion-message]
 [1uthority-73.1.0.0] [intercept_logv] [notice] Bootstrapped 100%: Done.
 0:0:11:527393 [thread-0] 0:15:3:736179239 [scallion-message]
 [2uthority-74.1.0.0] [intercept_logv] [notice] Bootstrapped 100%: Done.
 0:0:13:131711 [thread-0] 0:16:38:776506156 [scallion-message]
 [4uthority-76.1.0.0] [intercept_logv] [notice] Bootstrapped 100%: Done.
 0:0:13:502492 [thread-0] 0:17:20:866255937 [scallion-message]
 [nonexit2-84.1.0.0] [intercept_logv] [notice] Bootstrapped 100%: Done.
 0:0:13:580818 [thread-0] 0:17:22:338375323 [scallion-message]
 [exit1-78.1.0.0] [intercept_logv] [notice] Bootstrapped 100%: Done.
 0:0:13:709414 [thread-0] 0:17:23:833521695 [scallion-message]
 [nonexit5-87.1.0.0] [intercept_logv] [notice] Bootstrapped 100%: Done.
 0:0:15:607681 [thread-0] 0:19:12:118365753 [scallion-message]
 [exit3-80.1.0.0] [intercept_logv] [notice] Bootstrapped 100%: Done.
 0:0:15:917059 [thread-0] 0:19:19:662494634 [scallion-message]
 [exit5-82.1.0.0] [intercept_logv] [notice] Bootstrapped 100%: Done.
 0:0:17:402870 [thread-0] 0:20:14:413988393 [scallion-message]
 [5uthority-77.1.0.0] [intercept_logv] [notice] Bootstrapped 100%: Done.
 0:0:17:694102 [thread-0] 0:20:20:697006789 [scallion-message]
 [exit4-81.1.0.0] [intercept_logv] [notice] Bootstrapped 100%: Done.
 0:0:19:807030 [thread-0] 0:23:26:615529962 [scallion-message]
 [nonexit1-83.1.0.0] [intercept_logv] [notice] Bootstrapped 100%: Done.
 0:0:20:406078 [thread-0] 0:24:24:836237885 [scallion-message]
 [exit2-79.1.0.0] [intercept_logv] [notice] Bootstrapped 100%: Done.
 0:0:21:459315 [thread-0] 0:26:10:050116590 [scallion-message]
 [webclient1-88.1.0.0] [intercept_logv] [notice] Bootstrapped 100%: Done.
 0:0:25:253972 [thread-0] 0:29:31:480640102 [scallion-message]
 [nonexit4-86.1.0.0] [intercept_logv] [notice] Bootstrapped 100%: Done.
 0:0:35:308164 [thread-0] 0:38:43:258560918 [scallion-message]
 [nonexit3-85.1.0.0] [intercept_logv] [notice] Bootstrapped 100%: Done.
 }}}

-- 
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/6752#comment:35>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online
_______________________________________________
tor-bugs mailing list
tor-bugs@xxxxxxxxxxxxxxxxxxxx
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Prev by Author: [tor-bugs] #8426 [Tor bundles/installation]: GeoIP files missing from Vidalia Bundles
Next by Author: Re: [tor-bugs] #6752 [Tor]: TestingTorNetwork doesn't lower the dir fetch retry schedules
Previous by thread: [tor-bugs] #8426 [Tor bundles/installation]: GeoIP files missing from Vidalia Bundles
Next by thread: Re: [tor-bugs] #6752 [Tor]: TestingTorNetwork doesn't lower the dir fetch retry schedules
Index(es):
- Author
- Thread