[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [pygame] Fun with C modules



Ive have a similar story to tell. Im processing 3 x 40000 line logfiles in python. It took 120 seconds to complete. Using the profile module I found *all* that time was spend in the function that looped over every line of the file and processed them one by one. By optomising the functions within the loop (as dicatated by profile) by setting a lot of static stuff outside the loop (dot stuff, like member=class.member[key].member and then using member in the loop) I was able to shed the running time down to 80 seconds.

So I re-wrote the core processing function in C. Basically went...

proc=Processor()
proc.Process(file1)
proc.Process(file2)
proc.process(file3)
result=proc.PropagateResult(myclass)

In PropagateResult I pass it a class. It then instantiates this class over and over again filling in the data that it got from processing and returns a dictionary of the results with the values in the dictionary being instances of the class passed. The PropagateResult() C function basically goes...

{
...

	/* process args. This makes sure weve been passed a class type */
	if(!PyArg_ParseTuple(args, "O!", &PyClass_Type, &objecttype))
		return NULL;
	
	/* make our dict that well return */
	dict=PyDict_New();
	if(!dict)
		return NULL;

...

	for(...)
	{
		/* instantiate a new object of the passed class . The class should have no __init__ function */
		current=PyInstance_New(objecttype, NULL, NULL);
		if(!current)
			goto failure;

		... fill data into current ...
			
		/* add to the dict */
		if(PyDict_SetItemString(dict,keystring,current)==-1)
			goto failure;
	}
	
	return (PyObject *)dict;
	
failure:
	Py_XDECREF(dict);
	Py_XDECREF(current);
	Py_XDECREF(value);
	Py_XDECREF(value2);
	return NULL;		
}

Apon moving this core into the python processor, the time was slashed to just 8 seconds!!
This was quite remarkable, and there is still a vast amount of python code surrounding this core.

Pythons extensibility is wonderful :)

Kind Regards

Crispin

On Thu, Dec 20, 2001 at 04:14:52PM +1100, Richard Jones wrote:
> I've just spent a day converting some of my python classes over to C, with 
> some fun results.
> 
> In a nutshell, my application (ok, game ;) stores a 64x64 map grid, two 
> triangles per cell, in an octree. This is then culled for the frustum and 
> rendered as lit and textured triangles. The scene I'm getting the following 
> results from has 170 of the triangles from the octree being rendered.
> 
> I converted my frustum class (which has methods like calculate() and 
> isCubeInFrustum()), octree class (which just stores away objects with a 
> vertexes list and a render() method) and a simple lit/textured triangle 
> object which has the aforementioned vertexes array and render() method. The 
> python and C modules are functionally identical - they're completely 
> interchangeable (making this test quite easy :) but the C modules do have 
> several optimisations. For example, the C triangle class calls OpenGL 
> directly, not through the python bindings. I suspect my unpacking the 
> arguments to the gl calls once, when the object is created, has a large 
> impact...
> 
> I took a set of timings (render 100 frames and then time the next 100 
> frames), turning each of the C modules on and off:
> 
>              No C: 7fps
>          C octree: 7fps
>          C frusum: 8fps
>        C triangle: 10fps
>  C octree&frustum: 8fps
>     C frustum&tri: 11fps
>      C octree&tri: 26fps
>               all: 41fps
> 
> Well, what strange numbers. At 41fps my "game" is quite playable. Sure, most 
> of the work is done in a single call to a C module, but Python's still there 
> holding it all together :)
> 
> 
>     Richard
> 
> ps. I don't have a website at present - I'm between jobs - but I hope to have 
> something set up soon so I can share this stuff.
> ____________________________________
> pygame mailing list
> pygame-users@seul.org
> http://pygame.seul.org
> 
____________________________________
pygame mailing list
pygame-users@seul.org
http://pygame.seul.org