Re: huge pages, was where are the exit nodes gone?

[Arjan <n6bc23cpcduw@xxxxxxxxxxxxxxxxxxxxxx>]

Scott Bennett wrote:
>      On Tue, 13 Apr 2010 19:10:37 +0200 Arjan
> <n6bc23cpcduw@xxxxxxxxxxxxxxxxxxxxxx> wrote:
>> Scott Bennett wrote:
>>>      BTW, I know that there are *lots* of tor relays running on LINUX
>>> systems whose operators are subscribed to this list.  Don't leave Olaf and
>>> me here swinging in the breeze.  Please jump in with your LINUX expertise
>>> and straighten us out.
>> I'm not an expert, but I managed to perform some google searches.
>>
>> http://libhugetlbfs.ozlabs.org/
>> >From that website:
>> libhugetlbfs is a library which provides easy access to huge pages of
>> memory. It is a wrapper for the hugetlbfs file system. Applications can
>> use huge pages to fulfill malloc() requests without being recompiled by
>> using LD_PRELOAD.
> 
>      [Aside to Olaf:  oh.  So forcing the use of OpenBSD's malloc() might
> prevent the libhugetlbfs stuff from ever knowing that it was supposed to
> do something. :-(  I wonder how hard it would be to fix the malloc() in
> libhugetlbfs, which is most likely derived from the buggy LINUX version.
> Does libhugetlbfs come as source code?  Or is the use of LD_PRELOAD simply
> causing LINUX's libc to appear ahead of the OpenBSD version, in which case
> forcing reordering of the libraries might work?  --SB]

If Olafs test shows that CPU usage is reduced and throughput stays the
same or improves, modifying Tor to support linux huge pages might be an
option. Part 2 of this article contains some information about the
available interfaces:
	http://lwn.net/Articles/374424/
Getting the wrapper to work with (or like) the OpenBSD version will
probably be easier.


>> Someone is working on transparent hugepage support:
>> http://thread.gmane.org/gmane.linux.kernel.mm/40182
> 
>      I've now had time to get through that entire thread.  I found it
> kind of frustrating reading at times.  It seemed to me that in one of
> the very first few messages, the author described how they had long
> since shot themselves in the feet (i.e., by rejecting the algorithm of
> Navarro et al. (2002), which had already been demonstrated to work on an early
> FreeBSD 5 system modified by Navarro's team) on emotional grounds (i.e.,
> "we didn't feel its [Navarro's method's] heuristics were right").
<snipped the remainder of the analysis>

Thanks for your analysis of the thread and the reference to the Navarro
paper.

I've located the paper and will read it when time permits:
http://www.usenix.org/events/osdi02/tech/full_papers/navarro/

***********************************************************************
To unsubscribe, send an e-mail to majordomo@xxxxxxxxxxxxxx with
unsubscribe or-talk    in the body. http://archives.seul.org/or/talk/