Page 1 of 1
Test fixtures
Posted: Wed Nov 26, 2008 8:32 am
by alex.barylski
A test fixture is typically setUp/tearDown for each unit test, correct?
That seems like a lot of work...I suppose through refactorings one could minimize the work required...my application and all it's dependencies are setup inside index.php so I could in theory just copy that code into a method and call that method before each unit test. Along with the setup routines responsible for creating tables, files, configurations, etc...
Seeing as I am more interested in classical TDD as opposed to mockist TDD/BDD does it not make sense to effectively create and identical environment to my live system and just tear it down and re-create at the end of each unit test?
About the only objects I would mock would be SMTP server (ie: Swift connection)...but I digress...
Surely it would be time consuming to re-create a "real" environment for each test (a few more seconds anyway) but I would prefer testing results as opposed to behavior...
My question then: "Would it make sense to *not* tearDown() a fixture in the case where I was testing a createRecord() method? For instance, as opposed to creating tables, creating a record, testing return code, tearing down fixture, then creating a fixture with existing records and testing retreiveRecord() -- would it not make sense to leave the fixture in a 'created' state and simply call retreiveRecord() immediately after but before the tearDown() occured?"
Cheers,
Alex
Re: Test fixtures
Posted: Wed Nov 26, 2008 8:48 am
by mintedjo
Sounds good to me. Anything to minimize the workdone or time spent on unit testing.
I hate unit tests! BOOOOO UNIT TESTS! Especially the stupid Java JUnit nonsense! BOOOOO!
Re: Test fixtures
Posted: Wed Nov 26, 2008 8:50 am
by Eran
A test fixture is the fixed state of the system on which you run tests. setUp/tearDown methods help reach the desired state of the system, but they are not the only way to do so.
For my test suite, I include the libraries and set up database connections etc. before running the test suite. Barring exceptional cases, those things don't change between tests and are not affected by tests. The setUp / tearDown methods are responsible for changing only the environment details relevant to the tests they are responsible for.
Also, I wouldn't create the database / tables independently for each test. Emptying a table should be enough to test cleanly on it. The most important thing is that tests can run independently of each other - they should never rely on other tests running previously to work correctly. If you have repeating code in your test suite (such as creating a certain record), abstract it into a separate method which isn't a test method, and run it in every instance it's needed.
Re: Test fixtures
Posted: Wed Nov 26, 2008 9:36 am
by Jenk
I would prefer testing results as opposed to behavior
Whoah.. slow down and think about this. It is the behaviour you are wanting to test. Your are testing that the behaviour achieves the desired result, and possibly also testing that it achieves nothing but the desired result. The only caveat is that you don't (or at least try not to) test the implementation.
Re: Test fixtures
Posted: Wed Nov 26, 2008 11:25 am
by alex.barylski
The setUp / tearDown methods are responsible for changing only the environment details relevant to the tests they are responsible for
Makes sense. My installation routines would be invoked once at test "startup" and destroyed when all tests were completed...
Just for the sake of clarity I would define (correct me if I'm wrong):
1. Unit test -- The smallest possible (most atomic) test one can perform.
2. Test case -- A collection unit tests to assert behaviour of single method
3. Test suite -- A collection of test cases to assert behavior of single class
4. System under test -- A collection of test suites
I'm not sure how accurate I am with the definitions but this is how distinguish each phase or level testing...so basically:
System under test would create the tables, connections, etc inside it's __construct and __destruct might then destroys the tables, files, etc...
A test case would prepare the fixtures for the given unit tests?
Again, what leaves me wondering, is what to do when I wish to test
create(), then retreive immediately after. Should
create() leave the tables in a known state and have
retreive() use that data as "fixture" data or should the tables be flushed and new fixtures initialized?
they should never rely on other tests running previously to work correctly
I can see the logic in that, but what about in the situation above...wouldn't you be killing two birds with one stone? As opposed to setting up two fixtures?
Cheers,
Alex
Re: Test fixtures
Posted: Wed Nov 26, 2008 11:32 am
by alex.barylski
Whoah.. slow down and think about this. It is the behaviour you are wanting to test. Your are testing that the behaviour achieves the desired result, and possibly also testing that it achieves nothing but the desired result. The only caveat is that you don't (or at least try not to) test the implementation
By behavior I meant the implementation...but yes you are right...the behavior that "results" in the expected return values is what I am testing...
The problem is...when I think BDD I can't help but think mock objects and coupling that with implementation behavior as opposed to interface level results. When you test the result you are indeed testing a implementation behavior (method will only eval conditional IF) but at the interface level, so I find it hard to say I am testing behavior, so much as I am testing the results of a behavior.
I need to think about this for a bit more
Cheers.,
Alex
Re: Test fixtures
Posted: Wed Nov 26, 2008 11:58 am
by Eran
1. Unit test -- The smallest possible (most atomic) test one can perform.
2. Test case -- A collection unit tests to assert behaviour of single method
3. Test suite -- A collection of test cases to assert behavior of single class
4. System under test -- A collection of test suites
Some definitions change depending on who you ask, but mine are mostly different than yours:
A test case is a collection of unit tests that tests a component of the application (for example, an entire class). What tests are included in a test case is usually related to how well they share the same setUp/tearDown routines.
A test suite is a collection of test cases for organizations purposes (simplifies running multiple test cases).
System under test is your application in a frozen static state. Your tests bring the system under test to the desired state, however they are not a part of it.
I can see the logic in that, but what about in the situation above...wouldn't you be killing two birds with one stone? As opposed to setting up two fixtures?
The whole point of tests being independent is that one breaking won't break the other. To have the ability to localize a failing test is very important - so you can easily track what went wrong. If suddenly 10 tests break, you'd have a much harder time knowing which one is the problem since it broke 9 others with it.
Re: Test fixtures
Posted: Wed Nov 26, 2008 12:13 pm
by arjan.top
PCSpectra wrote:
4. System under test -- A collection of test suites
SUT is a part of the application that is being tested, not collection of test suites
Re: Test fixtures
Posted: Wed Nov 26, 2008 12:18 pm
by Jenk
Just to add my definitions are similar to pytrin's.
It's difficult at first, given we are programmers and we think logically and programmatically, (and the next to useless auto-test generators put out there by idiots like Microsoft don't help the learning) but there does not need to be a 1:1 method:test relationship.
Infact, try to ignore that you are testing methods all together. Especially in BDD. You are testing a behaviour. This behaviour does not need to be encapsulated inside of one method, it would be nice if it can, but don't for the sake of this "rule" think it must be. Also don't think that just because a method exists, it must be tested. If it is a helper method, or a 'no-brainer' (such as an accessor) then why bother testing it anyway.
An example would be a test/spec for a collection object.
Code: Select all
public function testAdd () {
$collection = new Collection();
$this->assertEqual($collection->size(), 0);
$collection->add('foo');
$this->assertEqual($collection->size(), 1);
$this->assertEqual($collection->at(0), 'foo');
}
We're actually specifically testing 1 behaviour, and generically testing 3 methods there.. but we aren't breaking encapsulation by testing the implementation. The methods size() and at() will have specific tests, too, and add() will (should) be generically tested there, so as again not to break encapsulation.
One "rule" I simply don't allow in my environment is chaining of tests, such as your suggestion to not tearDown(). Tests/Specs are to be an entirely encapsulated controlled environment, with (ideally) no outside influences.
Re: Test fixtures
Posted: Wed Nov 26, 2008 12:23 pm
by arjan.top
Wouldn't it be better to have separate test for empty collection?
Re: Test fixtures
Posted: Wed Nov 26, 2008 12:43 pm
by Jenk
you could have a testInitialisation() or such if you like, that would be the specific test for the empty collection.
however the behaviour we are testing in my example, is that adding the object 'foo' will increase the size by one, and when we retrieve the object we just added at it's determined index, that it is the object we just added. We could be less descript and do something like this:
Code: Select all
public function testAdd() {
$collection = new Collection();
$size = $collection->size();
$object = 'foo';
$collection->add($object);
$this->assertEqual($collection->size(), $size + 1);
$this->assertIdentical($collection->at($size + 1), $object);
}
The assertIdentical() would differ if you use 0 indexing of course (i.e. no '+ 1'.)
Re: Test fixtures
Posted: Sun Nov 30, 2008 11:23 am
by josh
PCSpectra wrote:That seems like a lot of work...I suppose through refactorings one could minimize the work required...my application and all it's dependencies are setup inside index.php so I could in theory just copy that code into a method and call that method before each unit test.
You can use test suites to share functionality across units. You should strive to eliminate duplication within tests, within the program code, as well as duplication between the test and the program. For instance if you tested a method returns 5 in the test, and in your real code you just wrote return 5; the test passes, but there is duplication with the "5". This is a simplistic example but im sure you will get the idea. Read Kent Bent's TDD book for more recommendations like this
PCSpectra wrote:My question then: "Would it make sense to *not* tearDown() a fixture in the case where I was testing a createRecord() method? For instance, as opposed to creating tables, creating a record, testing return code, tearing down fixture, then creating a fixture with existing records and testing retreiveRecord() -- would it not make sense to leave the fixture in a 'created' state and simply call retreiveRecord() immediately after but before the tearDown() occured?"
In your setup() method you're probably going to re-instantiate the objects which will reset their states. But you don't want any state transfer from test to test, that would lead to coupled units.
Re: Test fixtures
Posted: Mon Dec 01, 2008 3:37 am
by Jenk
I'm not sure if it's the case with SimpleTest, but most test frameworks create a new instance of the test object, for each test method within. Ergo, a class MyTests, which has methods testA, testB and testC, will have 3 instances created, and each will only have one of those three tests run.
So not tearing down will be pointless, anyway.
Re: Test fixtures
Posted: Mon Dec 01, 2008 2:12 pm
by alex.barylski
I believe that is how PHPUnit works -- which is what I'm using...