Re: gEDA-user: gerbv 2.3 -- make test or such

[Dan McMahill <dan@xxxxxxxxxxxx>]

Stefan Salewski wrote:
> For gerbv 2.3 some automated tests seems to exist -- I have to admit
> that I never care about such stuff...
> 
> But we got a bug report from our Gentoo QA team:
> 
> http://bugs.gentoo.org/show_bug.cgi?id=294911
> 
> http://bugs.gentoo.org/attachment.cgi?id=211363&action=view
> 
> Many messages like
> 
> "FAILED:  See mismatch/example_mentor_boardstation"
> 
> I am not sure how serious these messages are (maybe caused by gentoo's
> sandbox?).
> 
> Should the test work fine without these "Failed" messages?
> Or should we just ignore the test results.
> 

ignore the results and maybe file a bug report on sourceforge that 
reminds me to add a big warning about the test suite that says it is for 
developer use only.

The reason to ignore it is the following.  The test suite works by 
producing PNG output from an input file and then does a pixel by pixel 
check against a reference file.  What we have observed is that roundoff 
differences between different machines (alpha, i686, x86_64, 
-ffloat-store or not, etc) will cause variation which shows up as a 
change.  The larger use for the test suite is an individual developer 
can use a special flag to create a "before" set of reference files, make 
some code changes, and then check for output changes. This seems to do a 
better job of catching code change induced rendering changes.

I have not come up with a better way or a suitable command line way of 
ignoring changes which are "small enough".

-Dan


_______________________________________________
geda-user mailing list
geda-user@xxxxxxxxxxxxxx
http://www.seul.org/cgi-bin/mailman/listinfo/geda-user