[Author Prev][Author Next][Thread Prev][Thread Next][Author Index][Thread Index]
Re: [pygame] numpy migration (again)
Hello,
The current plan is to support numeric, and numpy by using the python
'array' module from C, and hooking into that via python.
This way everyone who used the surfarray module just as a bridge to
other libraries can still do so with the python array module - which
has been around since before python 1.5. So people using it for
opengl integration, and interfacing to PIL, , wx, tk etc don't need
any new dependencies.
By default surfarray will work as it does and return the same things
as it does now. However there will be ways to also just get normal
python arrays, and also to get numpy stuff. Numpy, and numeric will
be supported at the python level, not C. You won't need numeric or
numpy installed at compile time, just at runtime.
This should ease compilation, allow backwards compatibility, and also
allow a choice of numeric or numpy - or neither. Since such a big use
case for surfarray is just as a bridge between other libraries -
supporting it without an extra dependency is double plus cool.
Also there will be a big warning not to use numeric, or numpy at all -
since it is such a big source of problems for people not having it
installed. If you use numeric, or numpy less people will be able to
play your games - and you will get more people asking you how to use
it.
I think this plan let's everyone be happy, and warm inside.
... now someone just has to code it.
On 7/19/07, Santiago M. Mola <coldwind@xxxxxxxxxx> wrote:
Hi all,
I have read some discussions in the mailing list about switching from
Numeric to NumPy. I'd like to come with this topic again.
Let's see our scenario. Numeric is unmaintained, and their devs are
putting all their efforts on NumPy. That's no more performance,
portability or security fixes for Numeric, which is... bad.
Furthermore, Numeric is broken on amd64 + Python 2.5 (the default on
most Linux distributions) and it won't be fixed by Numeric authors nor
have been fixed by any 3rd party. OK, maybe it gets fixed one day, but
what will happens when the next problem pops up?
Some people complained about numpy's status and the migration consequences...
First point was documentation being not free [1]. NumPy team sells a
book with full documentation, other projects do this, like Ruby (and
Rails), and it never was a problem. There is enough (I think) free
documentation for Numpy:
+ SciPy doc [2]
+ NumPy official API reference [3]
+ Non-official NumPy API reference (with another format, for all tastes ;-) [4]
+ NumPy Example List (very very handy) [5]
+ NumPy dev wiki (this have some details about internals and porting) [6]
Maybe it's not the optimal doc, but It's a good start.
Next point is performance. Kamilche pointed that Numeric was 3x faster
than NumPy [7]. Maybe that was true 7 months ago but I used his
benchmark script today and obtained very different results. NumPy and
Numeric performance was very similar. NumPy (with pygame patched to
support it) took ~0.6 seconds to do the test, and Numeric (with pygame
non patched) took ~0.7. It's not an in-depth benchmark but it gives an
idea.
And finally, backwards compatibility. This is the real problem, but
don't panic, things are easier than it seems. Games which use Numeric
themselves, need to be ported to NumPy. Migration is not painful.
Anyone who is interested in porting his game, don't hesitate to email
me (I'm not a NumPy expert, but I successfully migrated some stuff).
Some side notes:
It was discussed to maintain Numeric compatibility in pygame 1.8 [8].
This could seem a good idea, but I think it's a recipe for problems in
the mid-term and not needed at all. 1.8 release is a good moment to
drop Numeric so people migrate to pygame 1.8 and NumPy at once... less
pain for all. Also it's more maintainable and less error prone.
I'd even say to do a release in 1.7.x series with NumPy support, which
would ease transition and let distros to kick Numeric.
Finally, here's a patch against 1.7.1 to use NumPy _native_ API (not
oldnumeric):
http://personales.alumno.upv.es/~sanmove/stuff/pygame-1.7.1-numpy.patch_newapi
Thanks all for your work,
Santiago
[1] http://thread.gmane.org/gmane.comp.python.pygame/7150/focus=7166
[2] http://www.scipy.org/Documentation
[3] http://www.scipy.org/doc/numpy_api_docs/numpy.html
[4] http://docs.neuroinf.de/api/numpy/
[5] http://www.scipy.org/Numpy_Example_List_With_Doc
[6] http://projects.scipy.org/scipy/numpy/wiki/
[7] http://thread.gmane.org/gmane.comp.python.pygame/8882/focus=8918
[8] http://thread.gmane.org/gmane.comp.python.pygame/9944/focus=9979
--
Santiago M. Mola
Jabber ID: coldwind@xxxxxxxxxxxx