Process documentation needed: automated test suite

[denise]

crschmidt pointed out in his bug walkthrough that the process of running the automated test suite is not very well documented! And I agree; I know I don't run the tests when I'm patching things since I don't really know much about it, and I don't contribute tests along with my code because I actually don't know how to write them or what they should do. I am probably not the only one! (The only thing I know about the test suite is that you shouldn't run it in production because it does weird and wonky things to the database.)

I do know that our test suite coverage is not extensive at all, and we've talked before about moving to a more test-driven devlopment mindset. If anybody wanted to write up some documentation on the Wiki about both how to run the tests (and what you should look for) and how to write new tests (and when you should), that would be awesome and I would love you forever. (Well, I mean, I love you all forever already. But I'd love you more forever.)

Comments

[mark]

It's great if people want to work on testing. I don't find it particularly useful though, in most cases it just takes time away from development when we already have a dearth of time available to develop things.

There are some situations in which writing tests makes a ton of sense -- I wrote some for the payment system when it was being built, and also for the big WTF changes. Those sorts of systems where getting it wrong makes for security risks and/or money loss is something really worth considering.

UI stuff, though, and other complicated systems? Eh. If people want, that's awesome, but I'm not going to do it or push it.

[exor674]

Yeah, I really really do not like most of our test suite. Even ignoring the DB corruption issues.

[fu]

UI stuff not so much (though a thing can be said for unit-testing JS), but even for relatively less security / money conscious things, testing can take a lot of the pain out of -- well things like making sure entries are parsed right. Because assuming we already had decent tests, it would be a lot easier to do something like add a test with an entry with <script> in the input and just run it, and you'd never have to double-check that case again assuming tests are run properly, than it would be to post a new entry and view it and make changes and refresh and etc.