[Author Prev][Author Next][Thread Prev][Thread Next][Author Index][Thread Index]
Re: [tor-bugs] #18910 [Metrics/CollecTor]: distributing descriptors accross CollecTor instances

Subject: Re: [tor-bugs] #18910 [Metrics/CollecTor]: distributing descriptors accross CollecTor instances
From: "Tor Bug Tracker & Wiki" <blackhole@xxxxxxxxxxxxxx>
Date: Thu, 20 Oct 2016 11:40:07 -0000
Auto-submitted: auto-generated
Delivered-to: archiver@xxxxxxxx
Delivery-date: Thu, 20 Oct 2016 07:40:20 -0400
In-reply-to: <046.efa4d874c0d1b588ace4f821ae2d5546@torproject.org>
List-archive: <http://lists.torproject.org/pipermail/tor-bugs/>
List-help: <mailto:tor-bugs-request@lists.torproject.org?subject=help>
List-id: "auto: Tor bug tracker status mails" <tor-bugs.lists.torproject.org>
List-post: <mailto:tor-bugs@lists.torproject.org>
List-subscribe: <https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs>, <mailto:tor-bugs-request@lists.torproject.org?subject=subscribe>
List-unsubscribe: <https://lists.torproject.org/cgi-bin/mailman/options/tor-bugs>, <mailto:tor-bugs-request@lists.torproject.org?subject=unsubscribe>
References: <046.efa4d874c0d1b588ace4f821ae2d5546@torproject.org>
Reply-to: no-reply@xxxxxxxxxxxxxx, tor-assistants@xxxxxxxxxxxxxx
Sender: "tor-bugs" <tor-bugs-bounces@xxxxxxxxxxxxxxxxxxxx>
#18910: distributing descriptors accross CollecTor instances
-------------------------------+---------------------------------
 Reporter:  iwakeh             |          Owner:  iwakeh
     Type:  enhancement        |         Status:  needs_review
 Priority:  High               |      Milestone:  CollecTor 1.1.0
Component:  Metrics/CollecTor  |        Version:
 Severity:  Normal             |     Resolution:
 Keywords:  ctip               |  Actual Points:
Parent ID:                     |         Points:
 Reviewer:                     |        Sponsor:
-------------------------------+---------------------------------

Comment (by karsten):

 Please take a look at the new commits in my updated task-18910-4 branch.

 Remaining tasks:

  - Find out why unit tests fail.  (See below.)
  - Test on a server.
  - Squash commits and merge.

 Here's the test output with 4 failing tests, both on OS X and on Linux:

 {{{
     [junit] Testsuite: org.torproject.collector.sync.SyncPersistenceTest
     [junit] Tests run: 40, Failures: 4, Errors: 0, Skipped: 0, Time
 elapsed: 30.708 sec
     [junit]
     [junit] Testcase: testDescWriteOutput[0] took 2.044 sec
     [junit] Testcase: testDescWriteRecent[0] took 0.523 sec
     [junit] Testcase: testOutFileContent[0] took 0.396 sec
     [junit] Testcase: testRecentFileContent[0] took 0.389 sec
     [junit] Testcase: testDescWriteOutput[1] took 0.454 sec
     [junit]     FAILED
     [junit] data used: relay-descriptors/server-
 descriptor/2016/10/e/3/e381ce74a1a592f6d375706665aba6d4d22923f1,
 relay-2016-10-02-16-05-00-server-descriptors, resulting list:
 [/tmp/junit1420314177894412746/out/relay-descriptors/server-
 descriptor/2016/10/c/8/c8c3588019f7c896eb4185cfc1074cfe5eb405ea,
 /tmp/junit1420314177894412746/out/relay-descriptors/server-
 descriptor/2016/10/4/1/4179c50d3c764bc85c9d719e14e55a6cc232a10d,
 /tmp/junit1420314177894412746/out/relay-descriptors/server-
 descriptor/2016/10/2/0/2091f76a8256e479cbe4f57be85f87909af07236,
 /tmp/junit1420314177894412746/out/relay-descriptors/server-
 descriptor/2016/10/b/b/bbca7ed70ba6ea88f995b067a004f5a4d0903d6e,
 /tmp/junit1420314177894412746/out/relay-descriptors/server-
 descriptor/2016/10/d/a/dae8966ca600b46bc75ed5efb97286481e9a6876,
 /tmp/junit1420314177894412746/out/relay-descriptors/server-
 descriptor/2016/10/e/3/e381ce74a1a592f6d375706665aba6d4d22923f1,
 /tmp/junit1420314177894412746/out/relay-descriptors/server-
 descriptor/2016/10/e/1/e1142337dee5b890393a0891acbde51577c2b743,
 /tmp/junit1420314177894412746/out/relay-descriptors/server-
 descriptor/2016/10/a/0/a0ed9227a9413f140445002ce412f8828591e7ec,
 /tmp/junit1420314177894412746/out/relay-descriptors/server-
 descriptor/2016/10/5/b/5b202650802a916f1ec3a1ef36b98706e3747701,
 /tmp/junit1420314177894412746/out/relay-descriptors/server-
 descriptor/2016/10/5/a/5a536243bf056cd7177ddfd8eb363fec978f3343]
 expected:<...-descriptor/2016/10/[e/3/e381ce74a1a592f6d375706665aba6d4d22923f1]>
 but
 was:<...-descriptor/2016/10/[c/8/c8c3588019f7c896eb4185cfc1074cfe5eb405ea]>
     [junit] junit.framework.AssertionFailedError: data used: relay-
 descriptors/server-
 descriptor/2016/10/e/3/e381ce74a1a592f6d375706665aba6d4d22923f1,
 relay-2016-10-02-16-05-00-server-descriptors, resulting list:
 [/tmp/junit1420314177894412746/out/relay-descriptors/server-
 descriptor/2016/10/c/8/c8c3588019f7c896eb4185cfc1074cfe5eb405ea,
 /tmp/junit1420314177894412746/out/relay-descriptors/server-
 descriptor/2016/10/4/1/4179c50d3c764bc85c9d719e14e55a6cc232a10d,
 /tmp/junit1420314177894412746/out/relay-descriptors/server-
 descriptor/2016/10/2/0/2091f76a8256e479cbe4f57be85f87909af07236,
 /tmp/junit1420314177894412746/out/relay-descriptors/server-
 descriptor/2016/10/b/b/bbca7ed70ba6ea88f995b067a004f5a4d0903d6e,
 /tmp/junit1420314177894412746/out/relay-descriptors/server-
 descriptor/2016/10/d/a/dae8966ca600b46bc75ed5efb97286481e9a6876,
 /tmp/junit1420314177894412746/out/relay-descriptors/server-
 descriptor/2016/10/e/3/e381ce74a1a592f6d375706665aba6d4d22923f1,
 /tmp/junit1420314177894412746/out/relay-descriptors/server-
 descriptor/2016/10/e/1/e1142337dee5b890393a0891acbde51577c2b743,
 /tmp/junit1420314177894412746/out/relay-descriptors/server-
 descriptor/2016/10/a/0/a0ed9227a9413f140445002ce412f8828591e7ec,
 /tmp/junit1420314177894412746/out/relay-descriptors/server-
 descriptor/2016/10/5/b/5b202650802a916f1ec3a1ef36b98706e3747701,
 /tmp/junit1420314177894412746/out/relay-descriptors/server-
 descriptor/2016/10/5/a/5a536243bf056cd7177ddfd8eb363fec978f3343]
 expected:<...-descriptor/2016/10/[e/3/e381ce74a1a592f6d375706665aba6d4d22923f1]>
 but
 was:<...-descriptor/2016/10/[c/8/c8c3588019f7c896eb4185cfc1074cfe5eb405ea]>
     [junit]     at
 org.torproject.collector.sync.SyncPersistenceTest.testDescWriteOutput(SyncPersistenceTest.java:179)
     [junit]
     [junit] Testcase: testDescWriteRecent[1] took 0.057 sec
     [junit] Testcase: testOutFileContent[1] took 0.1 sec
     [junit] Testcase: testRecentFileContent[1] took 0.058 sec
     [junit] Testcase: testDescWriteOutput[2] took 1.398 sec
     [junit] Testcase: testDescWriteRecent[2] took 0.505 sec
     [junit] Testcase: testOutFileContent[2] took 0.858 sec
     [junit] Testcase: testRecentFileContent[2] took 0.722 sec
     [junit] Testcase: testDescWriteOutput[3] took 0.434 sec
     [junit] Testcase: testDescWriteRecent[3] took 0.43 sec
     [junit] Testcase: testOutFileContent[3] took 0.673 sec
     [junit] Testcase: testRecentFileContent[3] took 0.67 sec
     [junit] Testcase: testDescWriteOutput[4] took 1.336 sec
     [junit] Testcase: testDescWriteRecent[4] took 0.861 sec
     [junit] Testcase: testOutFileContent[4] took 1.484 sec
     [junit] Testcase: testRecentFileContent[4] took 1.431 sec
     [junit] Testcase: testDescWriteOutput[5] took 0.867 sec
     [junit] Testcase: testDescWriteRecent[5] took 0.8 sec
     [junit] Testcase: testOutFileContent[5] took 1.601 sec
     [junit] Testcase: testRecentFileContent[5] took 1.7 sec
     [junit] Testcase: testDescWriteOutput[6] took 0.19 sec
     [junit]     FAILED
     [junit] data used: relay-descriptors/extra-
 info/2016/10/9/a/9a4b819baeeeb6952ba737b752471b8637e75a5c,
 relay-2016-10-02-08-05-00-extra-infos, resulting list:
 [/tmp/junit4173269913956546333/out/relay-descriptors/extra-
 info/2016/10/c/a/ca86eb96d22d188bb574b6b329ab21e0d9243516,
 /tmp/junit4173269913956546333/out/relay-descriptors/extra-
 info/2016/10/3/1/317586098443ed19b200417556a08ebc42133521,
 /tmp/junit4173269913956546333/out/relay-descriptors/extra-
 info/2016/10/3/6/36691feb7cec6a9630b9ecd11a9b5dc61c147c5d,
 /tmp/junit4173269913956546333/out/relay-descriptors/extra-
 info/2016/10/4/e/4ef90738e54a403b265120dcbab7b494e0c68d3b,
 /tmp/junit4173269913956546333/out/relay-descriptors/extra-
 info/2016/10/8/2/82471deac7b251089a0878d29a228d4e323b823f,
 /tmp/junit4173269913956546333/out/relay-descriptors/extra-
 info/2016/10/0/4/04219ada0be922fa7518d36b0d8e66afc55e8603,
 /tmp/junit4173269913956546333/out/relay-descriptors/extra-
 info/2016/10/e/b/eb73b59951bc1b0403be81220fb75be464954c31,
 /tmp/junit4173269913956546333/out/relay-descriptors/extra-
 info/2016/10/9/a/9a4b819baeeeb6952ba737b752471b8637e75a5c,
 /tmp/junit4173269913956546333/out/relay-descriptors/extra-
 info/2016/10/6/a/6a36d4ac36447e645c91ed63633a09197b7ad97e] expected
 :<.../extra-info/2016/10/[9/a/9a4b819baeeeb6952ba737b752471b8637e75a5c]>
 but was:<.../extra-
 info/2016/10/[c/a/ca86eb96d22d188bb574b6b329ab21e0d9243516]>
     [junit] junit.framework.AssertionFailedError: data used: relay-
 descriptors/extra-
 info/2016/10/9/a/9a4b819baeeeb6952ba737b752471b8637e75a5c,
 relay-2016-10-02-08-05-00-extra-infos, resulting list:
 [/tmp/junit4173269913956546333/out/relay-descriptors/extra-
 info/2016/10/c/a/ca86eb96d22d188bb574b6b329ab21e0d9243516,
 /tmp/junit4173269913956546333/out/relay-descriptors/extra-
 info/2016/10/3/1/317586098443ed19b200417556a08ebc42133521,
 /tmp/junit4173269913956546333/out/relay-descriptors/extra-
 info/2016/10/3/6/36691feb7cec6a9630b9ecd11a9b5dc61c147c5d,
 /tmp/junit4173269913956546333/out/relay-descriptors/extra-
 info/2016/10/4/e/4ef90738e54a403b265120dcbab7b494e0c68d3b,
 /tmp/junit4173269913956546333/out/relay-descriptors/extra-
 info/2016/10/8/2/82471deac7b251089a0878d29a228d4e323b823f,
 /tmp/junit4173269913956546333/out/relay-descriptors/extra-
 info/2016/10/0/4/04219ada0be922fa7518d36b0d8e66afc55e8603,
 /tmp/junit4173269913956546333/out/relay-descriptors/extra-
 info/2016/10/e/b/eb73b59951bc1b0403be81220fb75be464954c31,
 /tmp/junit4173269913956546333/out/relay-descriptors/extra-
 info/2016/10/9/a/9a4b819baeeeb6952ba737b752471b8637e75a5c,
 /tmp/junit4173269913956546333/out/relay-descriptors/extra-
 info/2016/10/6/a/6a36d4ac36447e645c91ed63633a09197b7ad97e] expected
 :<.../extra-info/2016/10/[9/a/9a4b819baeeeb6952ba737b752471b8637e75a5c]>
 but was:<.../extra-
 info/2016/10/[c/a/ca86eb96d22d188bb574b6b329ab21e0d9243516]>
     [junit]     at
 org.torproject.collector.sync.SyncPersistenceTest.testDescWriteOutput(SyncPersistenceTest.java:179)
     [junit]
     [junit] Testcase: testDescWriteRecent[6] took 0.038 sec
     [junit] Testcase: testOutFileContent[6] took 0.048 sec
     [junit] Testcase: testRecentFileContent[6] took 0.092 sec
     [junit] Testcase: testDescWriteOutput[7] took 0.086 sec
     [junit]     FAILED
     [junit] data used: bridge-descriptors/2016/10/extra-
 infos/9/f/9f88a7c2abe6665d204137ba8c2661d42e7c2829,
 bridge-2016-10-02-08-09-00-extra-infos, resulting list:
 [/tmp/junit4453404715943167674/out/bridge-descriptors/2016/10/extra-
 infos/c/a/cab78ea0ffe9a7bc00527fef19f546c47d59f01a,
 /tmp/junit4453404715943167674/out/bridge-descriptors/2016/10/extra-
 infos/3/4/3412a1dccd183a1c0bd1b748f34d88594be6ea52,
 /tmp/junit4453404715943167674/out/bridge-descriptors/2016/10/extra-
 infos/3/1/31bcea576e77ba66150f7903b588c919adad849c,
 /tmp/junit4453404715943167674/out/bridge-descriptors/2016/10/extra-
 infos/1/6/16d2b79fbd0a8567c6afd7585c775ac7745561e3,
 /tmp/junit4453404715943167674/out/bridge-descriptors/2016/10/extra-
 infos/f/a/fabd8f614633ec2d2d405f2554e14381bc33d9cb,
 /tmp/junit4453404715943167674/out/bridge-descriptors/2016/10/extra-
 infos/b/c/bcfcbb38b15e9b500b1a6e9b0bcbbce858660f17,
 /tmp/junit4453404715943167674/out/bridge-descriptors/2016/10/extra-
 infos/e/1/e11c5239494bad2f6f3759f1104a2f6182beab4d,
 /tmp/junit4453404715943167674/out/bridge-descriptors/2016/10/extra-
 infos/e/e/eee0dc51b9a0a71ba73610123b13cea212b5cf83,
 /tmp/junit4453404715943167674/out/bridge-descriptors/2016/10/extra-
 infos/9/f/9f88a7c2abe6665d204137ba8c2661d42e7c2829,
 /tmp/junit4453404715943167674/out/bridge-descriptors/2016/10/extra-
 infos/7/a/7a93ca1edc543e747f1157bc3a557890335311a4] expected:<...2016/10
 /extra-infos/[9/f/9f88a7c2abe6665d204137ba8c2661d42e7c2829]> but
 was:<...2016/10/extra-
 infos/[c/a/cab78ea0ffe9a7bc00527fef19f546c47d59f01a]>
     [junit] junit.framework.AssertionFailedError: data used: bridge-
 descriptors/2016/10/extra-
 infos/9/f/9f88a7c2abe6665d204137ba8c2661d42e7c2829,
 bridge-2016-10-02-08-09-00-extra-infos, resulting list:
 [/tmp/junit4453404715943167674/out/bridge-descriptors/2016/10/extra-
 infos/c/a/cab78ea0ffe9a7bc00527fef19f546c47d59f01a,
 /tmp/junit4453404715943167674/out/bridge-descriptors/2016/10/extra-
 infos/3/4/3412a1dccd183a1c0bd1b748f34d88594be6ea52,
 /tmp/junit4453404715943167674/out/bridge-descriptors/2016/10/extra-
 infos/3/1/31bcea576e77ba66150f7903b588c919adad849c,
 /tmp/junit4453404715943167674/out/bridge-descriptors/2016/10/extra-
 infos/1/6/16d2b79fbd0a8567c6afd7585c775ac7745561e3,
 /tmp/junit4453404715943167674/out/bridge-descriptors/2016/10/extra-
 infos/f/a/fabd8f614633ec2d2d405f2554e14381bc33d9cb,
 /tmp/junit4453404715943167674/out/bridge-descriptors/2016/10/extra-
 infos/b/c/bcfcbb38b15e9b500b1a6e9b0bcbbce858660f17,
 /tmp/junit4453404715943167674/out/bridge-descriptors/2016/10/extra-
 infos/e/1/e11c5239494bad2f6f3759f1104a2f6182beab4d,
 /tmp/junit4453404715943167674/out/bridge-descriptors/2016/10/extra-
 infos/e/e/eee0dc51b9a0a71ba73610123b13cea212b5cf83,
 /tmp/junit4453404715943167674/out/bridge-descriptors/2016/10/extra-
 infos/9/f/9f88a7c2abe6665d204137ba8c2661d42e7c2829,
 /tmp/junit4453404715943167674/out/bridge-descriptors/2016/10/extra-
 infos/7/a/7a93ca1edc543e747f1157bc3a557890335311a4] expected:<...2016/10
 /extra-infos/[9/f/9f88a7c2abe6665d204137ba8c2661d42e7c2829]> but
 was:<...2016/10/extra-
 infos/[c/a/cab78ea0ffe9a7bc00527fef19f546c47d59f01a]>
     [junit]     at
 org.torproject.collector.sync.SyncPersistenceTest.testDescWriteOutput(SyncPersistenceTest.java:179)
     [junit]
     [junit] Testcase: testDescWriteRecent[7] took 0.027 sec
     [junit] Testcase: testOutFileContent[7] took 0.071 sec
     [junit] Testcase: testRecentFileContent[7] took 0.054 sec
     [junit] Testcase: testDescWriteOutput[8] took 0.07 sec
     [junit]     FAILED
     [junit] data used: bridge-descriptors/2016/10/server-
 descriptors/a/8/a8a5509ad1393c8f36abd2d8f0de1bb751926872,
 bridge-2016-10-02-16-09-00-server-descriptors, resulting list:
 [/tmp/junit6983894424845948870/out/bridge-descriptors/2016/10/server-
 descriptors/c/3/c32acc52826b37e5319f1bac2f8812b33a772540,
 /tmp/junit6983894424845948870/out/bridge-descriptors/2016/10/server-
 descriptors/0/7/07d952e9020cb68a63d9156653a2e41af4af4d44,
 /tmp/junit6983894424845948870/out/bridge-descriptors/2016/10/server-
 descriptors/0/a/0a65c636a20631bd5deb2f10dc664d2032303c46,
 /tmp/junit6983894424845948870/out/bridge-descriptors/2016/10/server-
 descriptors/e/5/e5d46e88cb52d4fc0524398cfb7a6754394bc5e9,
 /tmp/junit6983894424845948870/out/bridge-descriptors/2016/10/server-
 descriptors/6/1/614414898ee133ee9bf6b10a9898cab518c5453e,
 /tmp/junit6983894424845948870/out/bridge-descriptors/2016/10/server-
 descriptors/7/8/784d6f9e82426072fbfc7a42f8f7133ba6ec1453,
 /tmp/junit6983894424845948870/out/bridge-descriptors/2016/10/server-
 descriptors/a/8/a8a5509ad1393c8f36abd2d8f0de1bb751926872,
 /tmp/junit6983894424845948870/out/bridge-descriptors/2016/10/server-
 descriptors/5/b/5b20316b03afc98a165a219044b6fa6ca34c58ab] expected
 :<.../server-descriptors/[a/8/a8a5509ad1393c8f36abd2d8f0de1bb751926872]>
 but was:<.../server-
 descriptors/[c/3/c32acc52826b37e5319f1bac2f8812b33a772540]>
     [junit] junit.framework.AssertionFailedError: data used: bridge-
 descriptors/2016/10/server-
 descriptors/a/8/a8a5509ad1393c8f36abd2d8f0de1bb751926872,
 bridge-2016-10-02-16-09-00-server-descriptors, resulting list:
 [/tmp/junit6983894424845948870/out/bridge-descriptors/2016/10/server-
 descriptors/c/3/c32acc52826b37e5319f1bac2f8812b33a772540,
 /tmp/junit6983894424845948870/out/bridge-descriptors/2016/10/server-
 descriptors/0/7/07d952e9020cb68a63d9156653a2e41af4af4d44,
 /tmp/junit6983894424845948870/out/bridge-descriptors/2016/10/server-
 descriptors/0/a/0a65c636a20631bd5deb2f10dc664d2032303c46,
 /tmp/junit6983894424845948870/out/bridge-descriptors/2016/10/server-
 descriptors/e/5/e5d46e88cb52d4fc0524398cfb7a6754394bc5e9,
 /tmp/junit6983894424845948870/out/bridge-descriptors/2016/10/server-
 descriptors/6/1/614414898ee133ee9bf6b10a9898cab518c5453e,
 /tmp/junit6983894424845948870/out/bridge-descriptors/2016/10/server-
 descriptors/7/8/784d6f9e82426072fbfc7a42f8f7133ba6ec1453,
 /tmp/junit6983894424845948870/out/bridge-descriptors/2016/10/server-
 descriptors/a/8/a8a5509ad1393c8f36abd2d8f0de1bb751926872,
 /tmp/junit6983894424845948870/out/bridge-descriptors/2016/10/server-
 descriptors/5/b/5b20316b03afc98a165a219044b6fa6ca34c58ab] expected
 :<.../server-descriptors/[a/8/a8a5509ad1393c8f36abd2d8f0de1bb751926872]>
 but was:<.../server-
 descriptors/[c/3/c32acc52826b37e5319f1bac2f8812b33a772540]>
     [junit]     at
 org.torproject.collector.sync.SyncPersistenceTest.testDescWriteOutput(SyncPersistenceTest.java:179)
     [junit]
     [junit] Testcase: testDescWriteRecent[8] took 0.025 sec
     [junit] Testcase: testOutFileContent[8] took 0.036 sec
     [junit] Testcase: testRecentFileContent[8] took 0.036 sec
     [junit] Testcase: testDescWriteOutput[9] took 0.192 sec
     [junit] Testcase: testDescWriteRecent[9] took 0.086 sec
     [junit] Testcase: testOutFileContent[9] took 0.1 sec
     [junit] Testcase: testRecentFileContent[9] took 0.109 sec
 }}}

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/18910#comment:67>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online
_______________________________________________
tor-bugs mailing list
tor-bugs@xxxxxxxxxxxxxxxxxxxx
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs
Prev by Author: Re: [tor-bugs] #20404 [Metrics/metrics-lib]: DescriptorIndexCollector should be default implementation
Next by Author: Re: [tor-bugs] #20253 [HTTPS Everywhere/EFF-HTTPS Everywhere]: Change Ruleset for HTTPS Everywhere
Previous by thread: Re: [tor-bugs] #18910 [Metrics/CollecTor]: distributing descriptors accross CollecTor instances
Next by thread: Re: [tor-bugs] #18910 [Metrics/CollecTor]: distributing descriptors accross CollecTor instances
Index(es):
- Author
- Thread