Re: [pygame] Python and Speed

[Greg Ewing <greg.ewing@xxxxxxxxxxxxxxxx>]

Ian Mallett wrote:
> Why not?  If C is faster, why can't Python be equally so?  If we assume 
> that C is the fastest a language available, then that implies that 
> Python /could/ be faster should it adopt the same procedures as C.

But "adopting the same procedures as C" would mean becoming
a statically-typed low-level language -- losing most of the
things that make Python nicer to program in than C!

C is fast because it's designed to be easily translatable
into very efficient machine code. When a C compiler sees

   int a, b, c;
   a = b + c;

it knows that a, b and c are all integers that fit in a
machine word, and it knows their memory addresses, so it
generates about 2 or 3 instructions to do the job.

However, when a Python implementation sees

   a = b + c

it has no idea what types of objects b and c will refer to
at run time. They might be ints, they might be long ints
that don't fit in a machine word, they might be strings,
they might be some custom object of your own with an
__add__ method. And they may be different for different
executions of that statement.

So every time the statement is executed, the names b
and c need to be looked up (namespaces can dynamically
gain or lose names, so we can't assume they're always
at a particular address), examine the types of the objects
they refer to, figure out what operation needs to be
done, and carry it out. Doing all that takes a great
many more machine instructions than adding a couple of
ints in C.

There are various ways of addressing the speed issue
with dynamic languages. One is what's known as "type
inferencing", where the compiler examines the whole
program, thinks about it very hard, and tries to convince
itself that certain variables can only ever hold values
of certain types; specialised code is then generated
based on that. This works best in languages that are
designed for it, e.g. Haskell; applying it post-hoc
to a dynamic language such as Python tends to be much
harder and not work so well.

Another is to use "just-in-time" techniques, where
you look at what types are actually turning up at run
time and generated specialised code for those cases.
Psycho is an attempt to apply this idea to Python;
reportedly it can produce useful speedups in some
cases.

There's a project around called PyPy which is using
type inferencing and various other strange and wonderful
techniques in an attempt to produce a self-hosted
Python implementation, i.e. written entirely in
Python. Last I heard, it was still somewhat slower
than CPython.

> I've gathered that Python is based on C code, so it translates your code 
> into C code on the fly.

No, it doesn't translate it into C, it just executes it
directly.

There is a translation step of sorts, into so-called
"bytecodes", which are instructions for a virtual machine.
But the instructions are very high-level and correspond
almost one-for-one with features of the Python language.
The interpreter then executes these virtual instructions.

The CPython interpreter happens to be written in C, but
it could have been written in any language that can be
compiled to efficient machine code, and the result would
be much the same.

> How can you run a C file from a Python script?

You can't, not directly. You need to write what's
known as an "extension module", which is a wrapper written
in C that bridges between the worlds of Python objects
and C data types. You then compile this into a dynamically
linked object file, which can be imported as though it
were a Python module.

Writing an extension module by hand is rather tedious
and not for the faint of heart. It has to handle all
conversion between Python objects and C data, manage
the reference counts of all Python objects it deals
with, and be scrupulous about checking for errors
and reporting them. If you want to get an idea of
what's involved, have a look at the "Extending and
Embedding" and "Python/C API" sections of the Python
documentation.

Fortunately, there are a variety of tools available
to make the task easier, such as SWIG, Boost Python
(for C++), Pyrex and Cython.

If you want to try this, my personal recommendation
would be Pyrex, although this is not exactly an
unbiased opinion, given that I wrote it. :-)

http://www.cosc.canterbury.ac.nz/greg.ewing/python/Pyrex/

--
Greg