Re: (FWD) Problems with Tor in user-mode-linux

[Bernhard Wiedemann <wiedeman@xxxxxxxxxxxxxxxxxxxxxxx>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi

I now have a theory on why tor uses that much memory in server mode:
Say the parent process allocates about 50MB for normal operation. That is 
OK so far.
Then the parent notices that it needs a new dns-worker and forks.

> When linux does a fork, all of the memory        
> pages are inherited in a copy-on-write fashion
> and Tor doesn't change most of them.

Well... I guess Tor does not allocate 50MB of memory to never write
to it again... especially if we talk about running for days with nodes 
coming and going...

So a forked process gets a version of all 50MB memory, which are 
mapped to the same memory... until the parent changes the content of it's 
mem. As soon as the parent writes to it's version of memory
the kernel needs to create a real copy of the written memory page, as it 
does not know the child will never ever read all that mem again.
After a while another dns-worker is forked with the parent's now modified 
memory which then needs to be copied again thus using more real memory.
This theory would also explain, why the dns-worker processes had most of 
their memory swapped out: they dont use it.

At least that gives a maximum usage of $#processes * $parentmem
unlike ordinary memory-leaks.

there are many possible solutions to this issue (if my theory is correct):
a)
 free all unused mem in the child process (maybe easiest to try)
b)
 call exec after fork, using an external dns-worker
c)
 use threads (e.g. pthread_create) - might improve performance too.
d)
 fork a dns-parent right at the start, which then forks dns-workers later


hopefully helpful
Bernhard M.
-----BEGIN PGP SIGNATURE-----

iD8DBQFB9creSTYLOx37oWQRAttPAKDr00j1g1pMm/qzAI10fIq+PlLzAACfTyA7
5AGL0uxT56FB9FmR9eUqiYM=
=GEaY
-----END PGP SIGNATURE-----