Re: [pygame] pygame unittests

["René Dudfield" <renesd@xxxxxxxxx>]

Why won't __ methods work?

eg. this __add__ method seems to work with the naming scheme?
test_Foo____add__():

also...

	<_raz_>	The test generator looks nice
	<_raz_>	though it'd need support for properties and interfaces
implemented by a module/class
	<illume>	ah, yeah... that could be good
	<illume>	maybe an option to supply a constructor... and then ouput
tests for the properties
	<illume>	eg. --create_for_instance="pygame.Color(1,2,3,4)" >> color_test.py
	<_raz_>	for the Color class there have to be additional tests for the
sequence and number interface for example
	<_raz_>	as it implements those
	<illume>	yeah
	<illume>	hrmm
	<illume>	I guess this thing is mainly to be used as a starting point...
	<_raz_>	sure
	<illume>	could add more features though
	<illume>	like, introspecting for more things
	<_raz_>	maybe this could be generated using dir()
	<_raz_>	and an exclude list or so for those methods or properties,
that do not need testing or are not implemented
	<_raz_>	e.g. Color.__rdiv__ (x) returns "NotImplemented"
	<_raz_>	so you know that it's not supported (currently) and can skip it
	<_raz_>	the only thing to keep in mind then would be: what are valid
arguments - this could be determined based on a script argument or so


I think these extra features could be added later - probably good to
start using it now... and maybe add these extra features later.


cu,




On Fri, May 30, 2008 at 3:52 PM, René Dudfield <renesd@xxxxxxxxx> wrote:
> hi,
>
> I think renaming the current tests is fine.
>
> There's 127 test methods to rename... which isn't too much.
> grep "def test" test/*  | wc -l
>
> I also think, don't worry about private methods... can add those if needed.
>
> I think go for it!  Anyone see any issues with it?
>
> This should give us a whole bunch of failing tests, so we can see a
> rough guide of how many tests left to implement.
>
> **NOTE: the code allows making generated tests pass instead of
> failing... so we can still see if any REAL tests fail.  I think the
> generated tests should pass by default... so our compile farm is still
> useful.  At least until pygame 1.8.1 is released.  To see the auto
> generated tests fail would just require changing a flag in
> run_tests.py.
>
> cu,
>
>
>
>
>
>
>
> On Fri, May 30, 2008 at 3:36 PM, Nicholas Dudfield <ndudfield@xxxxxxxxx> wrote:
>> Greetings all,
>>
>> akalias here, your GSOC student with the Pygame Testing project.
>>
>> One of the stated goals of my gsoc project is to have a really broad
>> coverage of tests. And in fact my first coding period was planned to be
>> spent on that. It's a fairly big task.
>>
>> To that end I have been working on a unittest stub generator that uses
>> introspection to create stubs of tests automatically.
>>
>> Eventually I want the stubber to be able to be used something like this:
>>
>>    gen_stubs.py --needed pygame.color >> color_test.py
>>
>> That would append stubs for all units (callables) from pygame.color that are
>> not currently tested. To keep it simple, I would just append the stubs and
>> people can spend the two seconds manually placing them.
>>
>> How would you know what functions aren't tested? An idea I am toying with is
>> to create a consitent naming scheme that we can all agree upon for the tests
>> (renaming existing tests) that makes it easy to determine which unit's are
>> already tested.
>>
>> The idea would be to inspect a module and based upon the names of it's tests
>> determine which units(functions) are tested.
>>
>> For example let's look at the method pygame.font.Font.get_ascent:
>>
>>    What information is needed?
>>
>>    pygame.       N: All unittests are for pygame
>>    font.         N: The tests are categorized into files, eg font_test.py
>>    Font.         Y: is a class and unique namespace
>>    get_ascent    Y:     what if there was pygame.font.get_ascent ?
>>
>> eg.
>>
>>    class + description optionals separated by double underscore
>>
>>        Class__method__description
>>
>>        pygame.font.Font.get_ascent -> becomes
>>
>>            def test_Font__get_ascent(self)
>>
>>        pygame.font.get_ascent -> becomes
>>
>>            test_get_ascent(self)
>>
>>            test_get_ascent__attacks_when_cornered(self)
>>
>>            ...
>>
>> So for pygame.font.Font.get_ascent if there was existing tests that started
>> with "test_Font__get_ascent", the proposed utility would skip creating that
>> stub.
>>
>> Issues
>> ======
>>
>> An issue with this naming scheme is methods prefixed or suffixed with
>> underscores.
>>
>> If we are only aiming at public methods this is a non issue.
>>
>> What "private" methods to test? What is the minimum coverage to aim for ?
>>
>> (the scope of the stub generator is all that's under discussion)
>>
>>
>> What are peoples thoughts. Is this a path worth venturing down? It would
>> mean renaming existing tests.
>>
>>
>