Page 1 of 1
Decentralizing Integration tests
Posted: Thu May 03, 2007 5:35 pm
by Ambush Commander
Quotes are examples
A big part of unit testing is mocking object that a class depends on so that you are testing solely that class. However, there are cases where you want to ensure that an entire composition of objects is working together: usually, this is as simple as making unit tests that cover higher level interfaces... sans the mocks.
The class Clock depends on an internal time object. When testing the clock, the time object should be mocked, so that you're only testing clock. However, it's a good idea to also test the clock and the time object at the same time.
What I've discovered recently, however, is that when adding extra behaviors to a method using composition, the integration test ends up getting separated from the rest of the tests for that method.
We have a SushiMachine, which takes string orders and converts them into Sushi object. The machine is composed of SushiWidgets, which each are responsible for creating a certain type of sushi. A SushiWidget can be tested individually to determine whether or not it creates the right Sushi based on the parameters that the SushiMachine extracted out of the Order, but you also want to test the SushiMachine to make sure it gives back the right Sushi from an order, i.e.
Order -> SushiMachine -> ParsedOrderForm
ParsedOrderForm -> SushiWidget -> Sushi
Order -> SushiMachine -> Sushi !
Does the test for Order -> SushiMachine ->Sushi belong in the SushiMachine object, or the specific SushiWidget it's testing?
I hope I was comprehensible.
Posted: Fri May 04, 2007 4:02 am
by Maugrim_The_Reaper
Which object is starting and directing the process? If it's an integration test, then that's the object being tested which runs the entire process. I couldn't see the exact relationship between Order and SuchiMachine, but if Order is requesting SushiMachine to fetch a Sushi object, than Order is the highest level member in the chain of command and the one against which the integration tests are being done. The role is different if SushiMachine accepts an Order as the context upon which to determine a Sushi object to create - then SushiMachine is the level of integration being targeted by the tests.
Posted: Fri May 04, 2007 2:01 pm
by Ambush Commander
I would say the SushiMachine. The order is being passed to the SushiMachine as a parameter (I realize the flow charts weren't really clear in this respect). Your latter situation is the correct one.
I agree, SushiMachine is the object we're testing during the integration tests. But often, we're testing a specific sub-behavior covered by another object lower down the composition. If that object's behavior changes, the integration test changes, i.e. cascading changes.
Posted: Sat May 12, 2007 11:03 am
by Ollie Saunders
I have previously considered that this is a possible hole in unit testing. However I have never really found the need to plug it. Are you making this post because you think it might be a problem or because you have actually experienced a problem?
Posted: Sat May 12, 2007 2:34 pm
by Ambush Commander
It's a possible problem but hasn't really caused any problems yet.
Posted: Mon May 14, 2007 3:39 am
by Maugrim_The_Reaper
I agree, SushiMachine is the object we're testing during the integration tests. But often, we're testing a specific sub-behavior covered by another object lower down the composition. If that object's behavior changes, the integration test changes, i.e. cascading changes.
I never really got around to replying, did I?
There's really no easy solution. Once you change an object's behaviour, you change the unit tests, and the interface to other objects and therefore there's no escaping the integration test changes. I suppose at the end of the day you really need to minimise behaviour changes in so far as that's reasonable. A lot depends on the timing of integration tests - if you are testing early than early integration tests are going to change almost as often as your unit tests. Also what is your approach? To have all components/aggregates pass integration testing before delivery/release? Might not be always possible...
Posted: Mon May 14, 2007 3:46 pm
by Ambush Commander
My approach is to treat integration tests just like unit tests. Sometimes, I get too lazy (or too performance-oriented) to make sure my design is loosely coupled or figure out all the mocks I need to program, so the integration test is my only way of determining the validity of the system.
Integration tests are quite susceptible to cascading errors: if one component has problems, unrelated unit tests start failing. This seems dirty, but in practice, I don't find it to great of a hindrance. Debugging with all this error information is manageable: just focus on a little section, fix it, and usually everything else falls into place.