Python Memory allocation

[Colin Tuckley <colin@xxxxxxxxxxx>]

Nick,

This was mentioned on a list where we were talking about various memory
allocation systems in the OS world.

It seems to be applicable to Mixminion (in that it's a long running task)

Were you aware of it and is it significant for Mixminion?

regards,

Colin

---------------------------------------------------------------------
<quote>
http://mail.python.org/pipermail/python-dev/2004-October/049480.html

If you can't be bothered to read it I'll summarise it simply:  python
_never_ frees memory used  by objects.  I'll say that again just so it
sinks in, Python never frees object memory (!!!!!!!).

The reasons are obscure (principally to do with pyalloc code not getting
the global interpreter lock and possible problems interacting with
multi-threaded systems) but the effect is important.  If you write a
python program that you expect to run for a long time then it's total
allocated memory will reach a peak and pretty much stay there, no matter
what you do.  Combined with a memory reuse policy that can result in a
fiar bit of fragmentation and it's swap till you drop time.

There is a patch under development to try and fix this problem in Python
2.5 (or at least alleviate it a little) but it's a long way from being
accepted and it seems that the Python developers haven't viewed this as
a priority because they think Python is a "scripting" langauge - i.e.
they don't expect people to write long running processes in it.
</quote>


-- 
Colin Tuckley      |  colin@xxxxxxxxxxx  |  PGP/GnuPG Key Id
+44(0)1903 236872  |  +44(0)7799 143369  |     0x1B3045CE

"Heisenberg may have slept here"