[Author Prev][Author Next][Thread Prev][Thread Next][Author Index][Thread Index]

Re: [tor-bugs] #16914 [Tor]: improve consensus download decompress performance

#16914: improve consensus download decompress performance
-----------------------------+---------------------------------------------
     Reporter:  teor         |      Owner:
         Type:  enhancement  |     Status:  needs_information
     Priority:  normal       |  Milestone:
    Component:  Tor          |    Version:  Tor: 0.2.6.10
   Resolution:               |   Keywords:  TorCoreTeam201509 Post027Freeze
Actual Points:               |  Parent ID:
       Points:               |
-----------------------------+---------------------------------------------
Changes (by teor):

 * status:  new => needs_information


Comment:

 Using `gzip -9 -v * 2>&1 | cut -f1 -d%` on some recent tor directory
 documents:
 {{{
 cached-consensus-20150827-1400-UTC:        66.0
 cached-descriptors-20150827-1400-UTC:      55.7
 cached-microdesc-consensus-20150826-1400-UTC:      56.3
 cached-microdesc-consensus-20150827-2300-UTC:      56.1
 cached-microdescs-20150826-1100-UTC:       51.9
 cached-microdescs-20150827-2300-UTC:       52.1
 }}}

 The microdescriptor consensus and descriptors are downloaded by most Tor
 clients (and, therefore, by most Tor instances). The descriptors are
 downloaded individually, so their ratios may be slightly lower. (The
 "full" consensus and descriptors aren't used by most clients, so they can
 be ignored for the purposes of this analysis.)

 I suggest we increase the expected ratio to 75%. We currently double the
 size of the buffer when we need to reallocate anyway, so we are already
 using that much RAM for every decompression, we're just allocating 50%,
 then reallocating 75%. (Except if the individual microdescriptor ratios
 fall under 50%, then we're only using 50%.)

 Alternately, we could increase the expected ratio to 70%, saving 1
 reallocation and 5% RAM in typical cases. This would be a win for both
 performance and RAM usage.

 Open questions:
 * What is the range of compression ratios on recent microdescriptor
 consensuses? Do they vary much?

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/16914#comment:3>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online
_______________________________________________
tor-bugs mailing list
tor-bugs@xxxxxxxxxxxxxxxxxxxx
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs