Decentralizing Integration tests

Discussion of testing theory and practice, including methodologies (such as TDD, BDD, DDD, Agile, XP) and software - anything to do with testing goes here. (Formerly "The Testing Side of Development")

Moderator: General Moderators

Post Reply
User avatar
Ambush Commander
DevNet Master
Posts: 3698
Joined: Mon Oct 25, 2004 9:29 pm
Location: New Jersey, US

Decentralizing Integration tests

Post by Ambush Commander »

Quotes are examples
A big part of unit testing is mocking object that a class depends on so that you are testing solely that class. However, there are cases where you want to ensure that an entire composition of objects is working together: usually, this is as simple as making unit tests that cover higher level interfaces... sans the mocks.
The class Clock depends on an internal time object. When testing the clock, the time object should be mocked, so that you're only testing clock. However, it's a good idea to also test the clock and the time object at the same time.
What I've discovered recently, however, is that when adding extra behaviors to a method using composition, the integration test ends up getting separated from the rest of the tests for that method.
We have a SushiMachine, which takes string orders and converts them into Sushi object. The machine is composed of SushiWidgets, which each are responsible for creating a certain type of sushi. A SushiWidget can be tested individually to determine whether or not it creates the right Sushi based on the parameters that the SushiMachine extracted out of the Order, but you also want to test the SushiMachine to make sure it gives back the right Sushi from an order, i.e.

Order -> SushiMachine -> ParsedOrderForm
ParsedOrderForm -> SushiWidget -> Sushi
Order -> SushiMachine -> Sushi !
Does the test for Order -> SushiMachine ->Sushi belong in the SushiMachine object, or the specific SushiWidget it's testing?

I hope I was comprehensible.
Last edited by Ambush Commander on Fri May 04, 2007 1:58 pm, edited 1 time in total.
User avatar
Maugrim_The_Reaper
DevNet Master
Posts: 2704
Joined: Tue Nov 02, 2004 5:43 am
Location: Ireland

Post by Maugrim_The_Reaper »

Which object is starting and directing the process? If it's an integration test, then that's the object being tested which runs the entire process. I couldn't see the exact relationship between Order and SuchiMachine, but if Order is requesting SushiMachine to fetch a Sushi object, than Order is the highest level member in the chain of command and the one against which the integration tests are being done. The role is different if SushiMachine accepts an Order as the context upon which to determine a Sushi object to create - then SushiMachine is the level of integration being targeted by the tests.
User avatar
Ambush Commander
DevNet Master
Posts: 3698
Joined: Mon Oct 25, 2004 9:29 pm
Location: New Jersey, US

Post by Ambush Commander »

I would say the SushiMachine. The order is being passed to the SushiMachine as a parameter (I realize the flow charts weren't really clear in this respect). Your latter situation is the correct one.

I agree, SushiMachine is the object we're testing during the integration tests. But often, we're testing a specific sub-behavior covered by another object lower down the composition. If that object's behavior changes, the integration test changes, i.e. cascading changes.
User avatar
Ollie Saunders
DevNet Master
Posts: 3179
Joined: Tue May 24, 2005 6:01 pm
Location: UK

Post by Ollie Saunders »

I have previously considered that this is a possible hole in unit testing. However I have never really found the need to plug it. Are you making this post because you think it might be a problem or because you have actually experienced a problem?
User avatar
Ambush Commander
DevNet Master
Posts: 3698
Joined: Mon Oct 25, 2004 9:29 pm
Location: New Jersey, US

Post by Ambush Commander »

It's a possible problem but hasn't really caused any problems yet.
User avatar
Maugrim_The_Reaper
DevNet Master
Posts: 2704
Joined: Tue Nov 02, 2004 5:43 am
Location: Ireland

Post by Maugrim_The_Reaper »

I agree, SushiMachine is the object we're testing during the integration tests. But often, we're testing a specific sub-behavior covered by another object lower down the composition. If that object's behavior changes, the integration test changes, i.e. cascading changes.
I never really got around to replying, did I? ;)

There's really no easy solution. Once you change an object's behaviour, you change the unit tests, and the interface to other objects and therefore there's no escaping the integration test changes. I suppose at the end of the day you really need to minimise behaviour changes in so far as that's reasonable. A lot depends on the timing of integration tests - if you are testing early than early integration tests are going to change almost as often as your unit tests. Also what is your approach? To have all components/aggregates pass integration testing before delivery/release? Might not be always possible...
User avatar
Ambush Commander
DevNet Master
Posts: 3698
Joined: Mon Oct 25, 2004 9:29 pm
Location: New Jersey, US

Post by Ambush Commander »

My approach is to treat integration tests just like unit tests. Sometimes, I get too lazy (or too performance-oriented) to make sure my design is loosely coupled or figure out all the mocks I need to program, so the integration test is my only way of determining the validity of the system.

Integration tests are quite susceptible to cascading errors: if one component has problems, unrelated unit tests start failing. This seems dirty, but in practice, I don't find it to great of a hindrance. Debugging with all this error information is manageable: just focus on a little section, fix it, and usually everything else falls into place.
Post Reply