Re: BSD/etc Tor servers? (was Re: Tor takes too much RAM)

[Scott Bennett <bennett@xxxxxxxxxx>]

     On Tue, 4 Dec 2007 09:53:40 -0500 Roger Dingledine <arma@xxxxxxx>
wrote:
>On Fri, Nov 30, 2007 at 11:35:23PM -0600, Scott Bennett wrote:
>>      However, I would be interested in knowing why I see this:
>> 
>> -rw-------  1 _tor  _tor    365907 Nov 30 23:10 cached-consensus
>> -rw-------  1 _tor  _tor  39096827 Nov 30 23:11 cached-descriptors
>> -rw-------  1 _tor  _tor    174297 Nov 30 23:17 cached-descriptors.new
>> 
>> The cached-descriptors file contains *six* differing descriptors for my tor
>> server, five of which are obsolete.  What gives?
>
>The apparently obsolete ones are valid for some v2 networkstatus that
>is still live. That is, there might be a client out there with an old
>but still valid v2 networkstatus that lists one of those old descriptors.
>
>So since you're a cache, you need to keep copies of those around too,
>in case some client asks for them.
>
>You discard them once the networkstatus that listed them becomes obsolete.
>
>See https://www.torproject.org/svn/trunk/doc/spec/dir-spec-v2.txt

     Okay.  I guess I had the naive idea that there would be no more than
one networkstatus per directory authority at any given time.  Oh, well.
>
>Once the v2 directory design disappears in favor of v3 (which is used
>in the Tor 0.2.0.x branch), the number of descriptors the caches will
>need to cache should go down.
>
>https://www.torproject.org/svn/trunk/doc/spec/dir-spec.txt

     The sooner, the better, IMO.
>
>>  Are the obsolete descriptors
>> taking up memory in the tor address space?  Seems to me that 1300 - 1700 tor
>> server descriptors should not require over 37 MB, but if they appear multiple
>> times, I suppose that would account for the waste.
>
>No, Tor mmaps that file, so it isn't all in memory.
>
     I wasn't asking about real memory.  mmap() maps a file into the address
space, though, so it could well account for the apparent memory usage as
shown by top(1) (SIZE field) or ps(1) (VSZ field in FreeBSD), which was the
point I was trying to make.


                                  Scott Bennett, Comm. ASMELG, CFIAG
**********************************************************************
* Internet:       bennett at cs.niu.edu                              *
*--------------------------------------------------------------------*
* "A well regulated and disciplined militia, is at all times a good  *
* objection to the introduction of that bane of all free governments *
* -- a standing army."                                               *
*    -- Gov. John Hancock, New York Journal, 28 January 1790         *
**********************************************************************