[pygame] Google Summer of Code: Testing

["Nicholas Dudfield" <ndudfield@xxxxxxxxx>]

Greetings all,

I am a participant in the Google Summer of Code. My application for writing tests for pygame was accepted. The goal, over and above having tests for their own sake, is to make pygame easier to port to new and different platforms and versions of Python. 

Py3k is due soon, and people have all sorts of ideas for pygame, such as deploying on flash VMs using pypy. Really cool ideas. All of this will be made much easier with more extensive testing in place. A rough estimate for current unittest coverage is about 20%. Ideally, at least 95% would be covered.

A brief outline
===============

1.  Write unittests for pygame
2.  Develop a speed regression framework
3.  Develop an interactive testing framework.

1)  Write unittests for pygame
==============================

Being a student, the fact is that I don't have that much experience writing tests. I haven't spent 10 years as a QA testing manager "in the real world" for example. My planned approach, that seems to have won endorsement from the PSF, is to do "breadth first" testing. Obviously you want bang for your buck, but where to start? By first getting a big picture view of pygame and becoming aware of the interconnections I will be better prepared to go into the next phase. That being that as time allows and with consultation from the more experienced pygamers in the community I will dive into more thorough pinpointed testing. Like all plans you can only really set your sails and adjust accordingly to changing conditions. 

2)  Develop a speed regression framework
========================================

"Ok, the change I made passes all the tests, great. Next....? "

As part of the tests I will also profile select areas so it's easy to judge when performance regressions are introduced.

3)  Develop an interactive testing framework
============================================

I like to think of testing as ESP; extra-sensory programming. You use the computer as an extra sense in the perception of "does it work like it should". Instead of trying all the different permutations yourself (haphazardly and slowly), you instruct your servant to do them signalling the result in a condensed form. "Does all that work?" Your computer signals "yes" (hopefully). The two of you zip through the tasks.

Some times it is hard to instruct your faithful servant though. "Jiles, Did you hear that mp3? Did it play twice?". Jiles can only "hear" the other world. You have to "tell" Jiles to help you as best he can; running through a batch of tests in a logical order, asking you if they passed, and carefully recording your response.

I will develop a framework for interactive testing. At this stage I am thinking it would be good to use pygame as the gui component but if something else in the python opensource world allows the job to be done faster I will probably use that. I have a little experience with PyQt4.

Conclusion
==========

There is lot's of work ahead and I will appreciate any help and advice.

Thanks,

akalias, aka Nicholas Dudfield

ps. My blog is at blog.akalias.net and my irc handle: akalias.