"Test Tuesday"

[mark]

I'd like to propose that this coming Tuesday we spend the day doing things to fix tests. szabgab has done a lot of work on our tests to make them actually run, but they're noisy (with warnings) and some of them are skipped. We should work on improving that.

Testing should really be part of what we do every time we upload a patch to Bugzilla (and we are talking about adding a 'i-ran-the-tests' flag to patches) and not just something we look at on a particular day -- but we have to start somewhere.

So, those of you who want something to do, I challenge you to find a test that isn't working and make it work. You can comment here if you want to claim some of the tests.

Also acceptable: adding new tests!

But for now, focusing on fixing the ones we have is probably more bang for the buck. Up to you, though.

Comments

[wyntarvox]

I'm not entirely sure what I'm getting myself in to, but I'll work on t/console-changejournaltype.t

[yvi]

I have no idea what I am doing and I can't promise it'll be Tuesday (might be any weekday this week), but I will try to fix these ones that are being skipped:

t/console-suspend.t
t/console-tags.t

[kareila]

Is there a basic "what tests are and how to use them" howto anywhere?

[foxfirefey]

There was this post:

http://dw-dev.dreamwidth.org/20733.html

And this needs more filling out:

http://wiki.dwscoalition.org/notes/Dev_Testing

[kareila]

Yay, thanks for looking up that post. I tagged it so it won't get lost again.

[kareila]

I didn't have to look very far to find a test that didn't work. The very first one failed:

dh-alierak@hack:~/dw$ prove t/
t/00-compile.t .................... 37/1344 
#   Failed test 'STDERR of LJ/Splash.pm'
#   at t/00-compile.t line 76.
#          got: 'Base class package "Splash::Server" is empty.
#     (Perhaps you need to 'use' the module which defines that package first.)
#  at cgi-bin/LJ/Splash.pm line 6
# BEGIN failed--compilation aborted at cgi-bin/LJ/Splash.pm line 6.
# Compilation failed in require at -e line 1.
# '
#     expected: ''
Bailout called.  Further testing stopped:  Test failed.  BAIL OUT!.
FAILED--Further testing stopped: Test failed.  BAIL OUT!.

Um... do we even use LJ::Splash?

[mark]

That file has been gone a long time. Please make sure that you have removed all 'no longer in hg' files before you run the tests, especially the 'compile the files' test. :)

Edit: You removed it (http://hg.dwscoalition.org/dw-free/rev/81b487a3f369) as part of FotoBilder removal (yay!), so that file shouldn't be in your directories anymore.

I'm not sure how you do your development, but for me I always make sure that running bin/cvsreport.pl with no arguments shows up empty -- there should be no output.

If there is output, especially if it's a 'new-file' line, then those files have to be removed.

$ bin/cvsreport.pl -1 -n | xargs rm

That's the easiest way to remove everything marked 'new-file'. But even then, sometimes things don't show up if they're in a directory that vcv doesn't know about.

Every once in a while, I do a full clean.

$ rm -rf bin cgi-bin htdocs
$ bin/cvsreport.pl -i

But be wary of that if you store things in any of those directories... it's safe for me, but I don't know your development style, so it may not be entirely safe for you.

[kareila]

dh-alierak@hack:~$ cvsreport.pl | wc -l
882

Yikes.

Thanks for the tip. This is turning into Cleanup Tuesday...