Page 1 of 2

Testing Class Hierarchy

Posted: Thu Jan 29, 2009 8:25 pm
by josh
How do you normally handle testing a hierarchy of classes? As far as I know there's 2 approaches, create an "abstract test suite" that has an array of classes to loop thru, insantiate, and run the tests on

or, you put the tests only once for one of the concrete implementations, and just "trust" yourself to make sure the other classes extend the same abstract class, etc.. and just keep all common functionality under 1 random classes' test suite ( chosen at random ) and not worry about testing hte other classes ( since they extend the same abstract class ).

Tradeoffs for the first method seem to be execution speed, in exchange for increased test harnesses. Are there better approaches? I'd have to assume others have ran into this conundrum.

Re: Testing Class Hierarchy

Posted: Tue Feb 03, 2009 3:14 am
by Chris Corbyn
This is a good question and I'm interested to hear what other approaches are.

Currently, if this class hierarchy is accompanied by a lot of tests I create a test case for the topmost superclass. For each subclass in the hierarchy, I create a test case which subclasses the test case for the superclass. That way I know that the functionality I don't want to override still works, and for any functionality I do want to override all I need to do is override the test method. It's a little dangerous to do this though since it can lead to fragile tests... you have to pay attention to the tests in the superclass and make sure you override anything that should not be tested.

The only alternative's I can see are:

1. Assume that because you've subclassed a fully tested superclass, you don't need to re-test that functionality (risky!)
2. Copy & paste the entire test for the superclass into a new test case and edit that according to whatever you changed (tangled to maintain).

Here's my outline:

Code: Select all

class Class A {
}
 
class ClassB extends ClassA {
  // ... Override methods of ClassA ...
}
 
class ClassATest extends UnitTestCase {
}
 
class ClassBTest extends ClassATest {
  // ... override methods of ClassATest where needed, or leave them to run as needed ...
}
I'll have a look what Gerard Meszaros has to say on the issue.

Re: Testing Class Hierarchy

Posted: Tue Feb 03, 2009 9:10 am
by josh
Thanks for the response, I also use some test inheritance as well, but im interested in different approaches. With inheritance your tests are gonna slow down drastically... I actually have been using test inheritance for other things ( providing test helper methods ). Going to try to start testing my hierarchies like that. Probably going to need to work on a more robust test runner to choose test suites ala carte since as is it takes like 15-20 seconds to run all my tests.

Re: Testing Class Hierarchy

Posted: Tue Feb 03, 2009 11:29 am
by josh
Hmm your idea made me think of doing this which I'm doing now, before I'd randomly pull models from the DB or instantiate them in the constructor, it just depended. Based on your comments I thought of the idea to make the creation of hte model a template method, so I'll run all the same tests 1x for the "manually" insantiated object, and then again for the "live" model, with maybe some extra methods to test writing back the data.

Re: Testing Class Hierarchy

Posted: Tue Feb 03, 2009 11:35 am
by Jenk
Chris Corbyn wrote:Assume that because you've subclassed a fully tested superclass, you don't need to re-test that functionality
Bingo.

It's no more risky than not writing your own tests for an external library. All you'd be accomplishing is duplicating tests. All you should be doing is testing the behaviour specific to that subclass.

Re: Testing Class Hierarchy

Posted: Tue Feb 03, 2009 5:04 pm
by Chris Corbyn
Jenk wrote:
Chris Corbyn wrote:Assume that because you've subclassed a fully tested superclass, you don't need to re-test that functionality
Bingo.

It's no more risky than not writing your own tests for an external library. All you'd be accomplishing is duplicating tests. All you should be doing is testing the behaviour specific to that subclass.
Except that more than likely you don't have access to the tests (or they don't play nicely with your test suite) for the external library. If you wrote the tests for the superclass I don't think there's much reason NOT to subclass the test case. If you're trying to demonstrate that your overrode some behaviour, a test subclass is the perfect way to achieve that.

But I'm curious to hear if people have other ways of doing this. For example, perhaps rather than a single test case your superclass is split across many tests cases, grouped into a test suite. If you could re-use that test-suite, substituting the bits you don't need (or want to change) then you'd achieve something similar.

I agree with what you're saying if we're talking about external libraries you don't have control over, but not when it's your own code.

EDIT | And while TDD is all about testing behaviour... isn't the whole point of a subclass to change behaviour? You're not duplicating tests as such, you're asserting that your modifications did not have a damaging effect on the integrity of the class. Say your subclass override a method that the superclass uses for something else... you want to know that your change broke that feature.

Re: Testing Class Hierarchy

Posted: Tue Feb 03, 2009 5:17 pm
by josh
Chris Corbyn wrote: EDIT | And while TDD is all about testing behaviour... isn't the whole point of a subclass to change behaviour? You're not duplicating tests as such, you're asserting that your modifications did not have a damaging effect on the integrity of the class. Say your subclass override a method that the superclass uses for something else... you want to know that your change broke that feature.
Yeah dude, I dig it. Ive been modifying my tests today... running all my assertions once for a manually insantiated class with dependencies stubbed out, and then in another subclass I override doSetUp() and ask the datamapper to build my model from my test database. I'm going to try to start using more inheritence in my tests... I do see both sides like Jenk said, for the most part it needlessly slows it down, but at the same time I think I would like to be able to know I haven't broken things in my subclass. Also you could easily strip out the test duplication with regex if you changed your mind, and I'm sure you could use reflection to conditionally run tests based on which class in the test hierarchy it was defined on, then you could run your tests short or long.

Re: Testing Class Hierarchy

Posted: Tue Feb 03, 2009 5:44 pm
by Chris Corbyn
Conditional test logic is generally seen as a bad thing. I'd avoid that at all costs.

As opposed to overriding the setUp() itself, you can also (possibly preferably) use Creation Methods:

To put it into more of a real-world perspective:

Code: Select all

class PdfTest extends TestCase {
  
  protected function createPdf($someDependency) {
    return new PdfDocument($someDependency);
  }
  
  protected function createWhateverDependency() {
    return $this->_mock('TheDependency');
  }
  
}
 
class PasswordProtectedPdfTest extends PdfTest {
  
  protected function createPdf($someDependency) {
    return new PasswordProtectedPdf($someDependency);
  }
  
}
So all you're overriding are parts of the test that create the SUT and it's dependencies.

Re: Testing Class Hierarchy

Posted: Tue Feb 03, 2009 7:40 pm
by josh
Yeah I just chose to name mine doSetUp(), what do you mean by conditional test logic? I went from 250 tests to 600 in a few hours of refactoring, and now my tests actually execute faster, ( by 2-5 seconds ), I am now testing things more thoroughly and already identified and fixed several issues due to not running the tests for subclasses.... I don't see any down sides yet.

Re: Testing Class Hierarchy

Posted: Tue Feb 03, 2009 7:59 pm
by Chris Corbyn
josh wrote:Yeah I just chose to name mine doSetUp(), what do you mean by conditional test logic? I went from 250 tests to 600 in a few hours of refactoring, and now my tests actually execute faster, ( by 2-5 seconds ), I am now testing things more thoroughly and already identified and fixed several issues due to not running the tests for subclasses.... I don't see any down sides yet.
I was referring to "conditional" test logic... i.e. if..else/switch inside your test cases/fixture setup. This is usually a bad idea. You were talking about something before (sorry I may have misread it) to do with checking the type of SUT automatically before running tests. This could more than likely lead to bugs in your test code. It's generally a test smell to find if..else in a test case.

The subclassing is not conditional logic though, I wasn't referring to that bit ;)

Re: Testing Class Hierarchy

Posted: Tue Feb 03, 2009 10:04 pm
by josh
Ah, yeah Im using a factory template method

Re: Testing Class Hierarchy

Posted: Wed Feb 04, 2009 7:23 am
by Jenk
Chris Corbyn wrote:
Jenk wrote:
Chris Corbyn wrote:Assume that because you've subclassed a fully tested superclass, you don't need to re-test that functionality
Bingo.

It's no more risky than not writing your own tests for an external library. All you'd be accomplishing is duplicating tests. All you should be doing is testing the behaviour specific to that subclass.
Except that more than likely you don't have access to the tests (or they don't play nicely with your test suite) for the external library. If you wrote the tests for the superclass I don't think there's much reason NOT to subclass the test case. If you're trying to demonstrate that your overrode some behaviour, a test subclass is the perfect way to achieve that.

But I'm curious to hear if people have other ways of doing this. For example, perhaps rather than a single test case your superclass is split across many tests cases, grouped into a test suite. If you could re-use that test-suite, substituting the bits you don't need (or want to change) then you'd achieve something similar.

I agree with what you're saying if we're talking about external libraries you don't have control over, but not when it's your own code.

EDIT | And while TDD is all about testing behaviour... isn't the whole point of a subclass to change behaviour? You're not duplicating tests as such, you're asserting that your modifications did not have a damaging effect on the integrity of the class. Say your subclass override a method that the superclass uses for something else... you want to know that your change broke that feature.
Which was my point that is entirely wrapped up in the last sentence of my initial reply..
All you should be doing is testing the behaviour specific to that subclass.
You should also not be running external tests in your own suite. Well, to say you shouldn't is a bit strong.. it's unnecessary. You aren't developing that library, so why are you running tests for it? (at 'install' I understand to check it hasn't goofed)

Same applies for sub/super classes. You are not developing the super class when you are developing the subclass, so why are you testing the superclass? The superclass has it's own test, no need to duplicate.

Re: Testing Class Hierarchy

Posted: Wed Feb 04, 2009 1:58 pm
by mickeyunderscore
I use PHPUnit to test classes that contain complex functionality, but I don't retest inherited methods unless they are overridden/extended.

More simple classes I don't tend to test in PHPUnit, usually there isn't time and I find it a bit of a pain writing unit tests out.

I'd like to practice TDD, but time constraints > programming methodology :(

Re: Testing Class Hierarchy

Posted: Wed Feb 04, 2009 5:49 pm
by Chris Corbyn
Jenk wrote:Same applies for sub/super classes. You are not developing the super class when you are developing the subclass, so why are you testing the superclass? The superclass has it's own test, no need to duplicate.
This we disagree on. By subclassing a class you are not testing the same class anymore. Like I say, you've effectively changed the behaviour of the superclass. You're making some very big assumptions that the subclass does not break the superclass behaviour and I think that is dangerous. I'm happy to remain in disagreement though...

Re: Testing Class Hierarchy

Posted: Thu Feb 05, 2009 3:22 am
by mickeyunderscore
Chris Corbyn wrote:This we disagree on. By subclassing a class you are not testing the same class anymore. Like I say, you've effectively changed the behaviour of the superclass. You're making some very big assumptions that the subclass does not break the superclass behaviour and I think that is dangerous. I'm happy to remain in disagreement though...
I'm not sure I follow this, would you retest all methods? Or just ones that have been extended/overridden, or involve the use of extended/overridden methods?