[Author Prev][Author Next][Thread Prev][Thread Next][Author Index][Thread Index]

[tor-bugs] #20412 [Metrics/Onionoo]: Don't let a single bad archived descriptor break the import

Subject: [tor-bugs] #20412 [Metrics/Onionoo]: Don't let a single bad archived descriptor break the import
From: "Tor Bug Tracker & Wiki" <blackhole@xxxxxxxxxxxxxx>
Date: Thu, 20 Oct 2016 12:51:04 -0000
Auto-submitted: auto-generated
Delivered-to: archiver@xxxxxxxx
Delivery-date: Thu, 20 Oct 2016 08:51:14 -0400
List-archive: <http://lists.torproject.org/pipermail/tor-bugs/>
List-help: <mailto:tor-bugs-request@lists.torproject.org?subject=help>
List-id: "auto: Tor bug tracker status mails" <tor-bugs.lists.torproject.org>
List-post: <mailto:tor-bugs@lists.torproject.org>
List-subscribe: <https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs>, <mailto:tor-bugs-request@lists.torproject.org?subject=subscribe>
List-unsubscribe: <https://lists.torproject.org/cgi-bin/mailman/options/tor-bugs>, <mailto:tor-bugs-request@lists.torproject.org?subject=unsubscribe>
Reply-to: no-reply@xxxxxxxxxxxxxx, tor-assistants@xxxxxxxxxxxxxx
Sender: "tor-bugs" <tor-bugs-bounces@xxxxxxxxxxxxxxxxxxxx>

#20412: Don't let a single bad archived descriptor break the import
---------------------------------+-----------------
     Reporter:  karsten          |      Owner:
         Type:  defect           |     Status:  new
     Priority:  Medium           |  Milestone:
    Component:  Metrics/Onionoo  |    Version:
     Severity:  Normal           |   Keywords:
Actual Points:                   |  Parent ID:
       Points:                   |   Reviewer:
      Sponsor:                   |
---------------------------------+-----------------
 The Onionoo mirror broke in late September for some reason I don't know,
 and the host didn't come back afterwards.  We only noticed two weeks later
 and had to reimport September and October data.  However, the September
 archives contain a bad descriptor that breaks the import.  Here's the bad
 descriptor (`3/8/384f93dbac20fdf293a731b391b3fc0757d9f78a` in `server-
 descriptors-2016-09.tar.xz`):

 {{{
 @type server-descriptor 1.0
 router Pegasus70 78.142.19.172 443 0 80
 platform Tor 0.2.5.12 on Linux
 protocols Link 1 2 Circuit 1
 published 2016-09-15 08:31:03
 fingerprint E9C5 8383 DB9A E52A DAF3 5F91 88B2 741A 05F5 A02F
 uptime 1016140
 bandwidth 8746942 8955284 7957930
 extra-info-digest C07612E283D3157219DA1DDDF3AE125268206412
 onion-key
 -----BEGIN RSA PUBLIC KEY-----
 MIGJAoGBAPNQHBnVRiPl7H5cRC/GOMKIeGRSAuM/3Jzuxrg0idlL1YPoQtKAfaqI
 LY9cGSEk88FGcOkgZdDiwSL9LAtBF1hpYB2ajGjNhTQkae00DC1NlWGzi89wkA/R
 4qxSCm4mjoY7EEmfOLI/X/Rp9FE8rL7X39XK6q+nv5uyHI+T/7GHAgMBAAE=
 -----END RSA PUBLIC KEY-----
 signing-key
 -----BEGIN RSA PUBLIC KEY-----
 MIGJAoGBAMojXIJSwwavFOc6afWILpgIc4sAXd8KSsOh966rLjuGZyUsN3+gqta9
 2QLV4HOBy9L24NRE3iXmySlfTiT2pxiSXo+h/B18Gw2clSewHx7xC1QnkT69xxL2
 6AvOu5NDbu5SxHOtOi95FEnuE6VmKZhBawHD3KG6j6euZINilojrAgMBAAE=
 -----END RSA PUBLIC KEY-----
 family $9D14BAC27FFE7170601FC0EC792A927E1FC11A1D
 hidden-service-dir
 ntor-onion-key vk7nDH5FVjFWxflyapUT+9+em+CGO/aaYjaO6LGJ3B0=
 reject 0.0.0.0/8:*
 reject 169.254.0.0/16:*
 reject 127.0.0.0/8:*
 reject 192.168.0.0/16:*
 reject 10.0.0.0/8:*
 reject 172.16.0.0/12:*
 reject 78.142.19.172:*
 accept *:20-21
 accept *:43
 accept *:53
 accept *:79-81
 accept *:88
 accept *:110
 accept *:143
 accept *:194
 accept *:220
 accept *:389
 accept *:443
 accept *:464
 accept *:531
 accept *:543-544
 accept *:554
 accept *:563
 accept *:636
 accept *:706
 accept *:749
 accept *:873
 accept *:902-904
 accept *:981
 accept *:1194
 accept *:1220
 accept *:1293
 accept *:1500
 accept *:1533
 accept *:1677
 accept *:1723
 accept *:1755
 accept *:1863
 accept *:2082
 accept *:2083
 accept *:2086-2087
 accept *:2095-2096
 accept *:2102-2104
 accept *:3128
 accept *:3389
 accept *:3690
 accept *:4321
 accept *:4643
 accept *:5050
 accept *:5190
 accept *:5222-5223
 accept *:5228
 accept *:5900
 accept *:6697
 accept *:8008
 accept *:8074
 accept *:8080
 accept *:8082
 accept *:8087-8088
 @uploaded-at 2016-09-15 08:32:03
 @source "79.134.255.35"
 router Beluga 79.134.255.35 1979 0 0
 identity-ed25519
 -----BEGIN ED25519 CERT-----
 AQQABkDiAfjGVdzeISYHVC86lkA1GRbNmgn80ndEWHoNfqq3apelAQAgBACmRaXq
 1UdBqrNx7dYOhs62167xULhT4QoThd/IgiZw18mYn19eCtf0qfGiDmYv1v4d1INq
 drh+i4yS1XGw8oypyLU27mt9BI5ezMXHMeKkEvRdgNwg5K2Levzw7PhK6go=
 -----END ED25519 CERT-----
 master-key-ed25519 pkWl6tVHQaqzce3WDobOtteu8VC4U+EKE4XfyIImcNc
 platform Tor 0.2.8.7 on Linux
 protocols Link 1 2 Circuit 1
 published 2016-09-15 08:31:58
 fingerprint EC69 7C3D 5819 B16B B899 D29A 18B9 E7B6 095D FAEC
 uptime 129602
 bandwidth 2097152 3145728 1286199
 extra-info-digest 387513B8F45EFB6711F40FA6869146DE62B058D5
 l+9BGbujAcdSANxivZN210RJSHsHSQCQMPqOYg4VNSA
 onion-key
 -----BEGIN RSA PUBLIC KEY-----
 MIGJAoGBALKRXi8ClqAACiYtBCF+Ot4154CxhykXufXQFGEYR2KkyEI4wPp2E/hV
 izLQjrjmIq+akyFUGNE/u/OY5seeUlcFtFnBHfsotrtBkL8yqMqmyheL5OG1CWX3
 ROKd6UtzMP1ebIcalS0hdc7nlpOlzxd91IJjlE5eI/jKJyTKl4C1AgMBAAE=
 -----END RSA PUBLIC KEY-----
 signing-key
 -----BEGIN RSA PUBLIC KEY-----
 MIGJAoGBALL8VZK/RpoV8XkaSFTjFfchYDeTrzToWgiE8fFm68Pato+iQ5xArjkW
 gKaj4DAqrUMZgR2rz0joiD9U6lEssEFhXM0laqlLpcuhoBB+6BbiLOFbcm6MHPcr
 eVuNRjcwRIr71SCASCih52LHuyOCDwqMnNpdIyCLse+AAtdqmJ3TAgMBAAE=
 -----END RSA PUBLIC KEY-----
 onion-key-crosscert
 -----BEGIN CROSSCERT-----
 FZg86wGaQqd4v8nBC0GQm1bHo0ooZFh79XnZPVYzw/gBUIguy4vbumIb07sXd7+1
 C7cNBlfpy/1UQH0t4l9Crqj6LqL9DNJr8xOKx0hbgw5dxDHJNu16+qMTsdo30RVQ
 O1XBS12Wh6Or/suW8D+wRQ8TT804c0wdc5TMTJIe9Wo=
 -----END CROSSCERT-----
 ntor-onion-key-crosscert 0
 -----BEGIN ED25519 CERT-----
 AQoABj/5AaZFperVR0Gqs3Ht1g6GzrbXrvFQuFPhChOF38iCJnDXAJ4T5u2HXRQF
 1B8/CNz0+VNfa1C+kP/CoAT+qxud/wJCpoUo3au/YzSZSrjTtstKTs8lv7chn+QT
 JiqklO+iegM=
 -----END ED25519 CERT-----
 hidden-service-dir
 contact BTC 1EfzsAj6rLnvYMuAZeTLBmhZ3gHcjxfUkp
 ntor-onion-key V8SPDfH90+Fa8Q21xYzG5qqovaOsKqP2aZKdHBWAWBI=
 reject *:*
 tunnelled-dir-server
 router-sig-ed25519
 4CUXRGMVxOXkR+qgv/W+Jsz7WgGWVloER9qZgfqbNNEA3UyAo2W7odoAcBpwrrPFarAYoza1T7I2WmsigneFDw
 router-signature
 -----BEGIN SIGNATURE-----
 aH+lYDSsISil+cUXxbXzv2H/M5rDguHOMKbMSZMBRSTIUAV6zrxaSEJgQweOJyr4
 CiqAJjxp4sTcnUvsaqFZwrO8xsv+LZHabIbNu+DnuPtS66Xngh3q/wEJbx/VLWfq
 yAexVH/GnxAfwplvB99GIiHqE20r+nLAdrgGnZ4x/9M=
 -----END SIGNATURE-----
 }}}

 The issue is that this file contains one partial descriptor and another
 full descriptor without a @type annotation.  Onionoo doesn't know how to
 process it.  However, when we attempt to process this descriptor, we abort
 the entire import process.  Even worse, when aborting the import process
 we're letting the descriptor reader thread continue reading descriptors
 until its queue runs full and it waits for us to accept more parsed
 descriptors.  The result is that the Onionoo process, which ran in
 `--single-run` mode, did not finish within a few days until I killed it.
 Oh wow.

 Quick fix: Skip the bad descriptor, that is, `continue;` rather than
 `break;`.

 Longer-term fix: Come up with better rules how to handle bad input data,
 which is somewhat related to #19834.

 Follow-up question: What went wrong that CollecTor produced this file?

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/20412>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online
_______________________________________________
tor-bugs mailing list
tor-bugs@xxxxxxxxxxxxxxxxxxxx
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Prev by Author: Re: [tor-bugs] #20379 [Applications/Tor Browser]: Can't initially connect to bridges after new network connection
Next by Author: Re: [tor-bugs] #20278 [Core Tor/Tor]: cert-spec.txt contains incomplete reference / documentation for certificate types
Previous by thread: Re: [tor-bugs] #20411 [Applications/Tor Browser]: Make sure that connections are closed before we shut tor in Tor Browser down
Next by thread: [tor-bugs] [Tor Bug Tracker & Wiki] Batch modify: #20287, #19934, #19831
Index(es):
- Author
- Thread