Chewed on your comments for a little bit. Here we go...
dedicated Acceptance Tests
Alright... what are acceptance tests? (fails to see the distinction between them and integration tests).
no valid Unit Test for that class. Which by itself should be worrying
I think this drives the point home.
just be sure you have that functional unit under a set of Integration Tests in case the Unit Tests end up being less than ideal.
I guess I should take this to mean at a certain point, one should scrap their unit tests if performance reasons don't allow for the efficient testing of the unit?
it seems like a better way would be to use text/XML files to contain all this information and run it through a testing framework that reads the data and runs the tests.
I sometimes get this feeling, although I find it difficult to imagine a format that is flexible to get all of our testing needs (or even most of them). Not so savvy about SQL: most of the time, we try to abstract away into not writing it (heh heh).
In a sense a long drawn out test case is as bad as long drawn out documentation - it gets boring too quickly
I try to make my unit tests interesting. Case in point:
Code: Select all
function testRedefinition_define() {
CS::defineNamespace('Cat', 'Belongs to Schrodinger.');
CS::define('Cat', 'Dead', false, 'bool', $d1 = 'Well, is it?'); $l1 = __LINE__;
CS::define('Cat', 'Dead', false, 'bool', $d2 = 'It is difficult to say.'); $l2 = __LINE__;
$this->assertIdentical($this->our_copy->defaults['Cat']['Dead'], false);
$this->assertIdentical($this->our_copy->info['Cat']['Dead'],
new HTMLPurifier_ConfigDef_Directive('bool',
array($this->file => array($l1 => $d1, $l2 => $d2))
)
);
$this->expectError('Inconsistent default or type, cannot redefine');
CS::define('Cat', 'Dead', true, 'bool', 'Quantum mechanics does not know.');
$this->expectError('Inconsistent default or type, cannot redefine');
CS::define('Cat', 'Dead', 'maybe', 'string', 'Perhaps if we look we will know.');
}
Are you using TDD or retrospective Unit Testing?
It's interesting that you should mention that.
Most of the time I do TDD. Sometimes, however, I get a really good idea and jump straight to implementing it, because making a unit test for the idea would be too difficult to do.
In fact, I can see that the classes I did not TDD are the ones having the most problems right now!
In our case it's more a Unit Test problem - assuming even a high rate of coupling any Integrated Tests are less likely to need change since they're at a much higher level where coupling has a minimal impact (i.e. who cares which way the pieces are bolted together so long as the whole works).
But if the behavior of the whole changes, you'll end up changing integration tests and unit tests, quite possibly redundantly. Is redundancy bad in unit testing? On one hand, you don't want cascading errors. On the other hand, you shouldn't remove a test case without good reason.
Moving along...
I think we've established some important symptoms of naughty unit tests and naughty code. In a perfect world, code would be simple and tests would be simple, but this is not the case. Pushing redesigning the API of your actual code out of the picture of the moment, I would like to pose the question of
how to refactor duplicated/redundant code in unit tests.
Case-study: my original method of dealing with duplicated code was to... put it in a loop. I would create arrays of "input" and "output", and then a special loop that would due all the initialization necessary and perform the test. The first practical problem: all the failed assertions would come out to the same line, so there was no way of finding out which test failed! I "fixed this" by sending fail messages to the assertions that included the necessary machinery.
Some of this code is still hanging around, but in retrospect, it seems like it was a bad idea. I've since migrated to a new way of handling duplicated initialization: custom assertions. You'll find my unit tests littered with things like:
Code: Select all
function test() {
// valid ID names
$this->assertDef('alpha');
$this->assertDef('al_ha');
$this->assertDef('a0-:.');
$this->assertDef('a');
// invalid ID names
$this->assertDef('<asa', false);
$this->assertDef('0123', false);
$this->assertDef('.asa', false);
// test duplicate detection
$this->assertDef('once');
$this->assertDef('once', false);
// valid once whitespace stripped, but needs to be amended
$this->assertDef(' whee ', 'whee');
}
Totally unhelpful to someone who wants to figure out how to invoke the object being tested, but quite useful in determining the behavior of the class. (Tidbit: the assert function's implementation is *10* lines long. Only 1 function call is being tested. The rest is assertions or setup. I have another harness that's 60 lines long, once again, only one function call is being tested.)
Any wisdom in this area?