Are you a Mockist or a Stubbist?

Not for 'how-to' coding questions but PHP theory instead, this forum is here for those of us who wish to learn about design aspects of programming with PHP.

Moderator: General Moderators

Post Reply
User avatar
Maugrim_The_Reaper
DevNet Master
Posts: 2704
Joined: Tue Nov 02, 2004 5:43 am
Location: Ireland

Are you a Mockist or a Stubbist?

Post by Maugrim_The_Reaper »

A question for the Mob.

Myself and Travis Swicegood kickstarted the PHPMock project recently. As a bit of background I'm developing a Behaviour-Driven Development (BDD) framework called PHPSpec, Travis is writing an updated PHPT framework for PEAR2, as well as continuing to help out on SimpleTest (he's also in the PHPSpec project). One of the mailing list lurkers is Sebastian Bergmann who seems to be considering ditching the current generation of PHPUnit Mock Objects if we get off our collective asses and write this thing.

PHPMock is intended to be an independent Mock Object and Stub (or you can just call it a Test Double) framework available across all testing/specing platforms.

Now the problem we're having is that we sit in two different camps. I'm a Mockist (unless it's a small cluster of objects, I will mock out everything else when testing - stubs only if mocks are clearly overkill or it's just a slow resource), Travis is a Stubbist (he writes Stubs for everything I'd usually Mock). So setting goals for PHPMock is like watching Labour sit down with the Conservatives and agree on something (add in a US metaphor if you prefer Wink). It's curious how the two approaches conflict.

I was wondering what folk here tend to prefer using? Are you a Mockist or a Stubbist at heart? Or do you use the other few options (real objects, test spys, etc). I wrote on my blog a few days ago that in PHP it's often partly linked to whichever Unit Testing library you grew up on - PHPUnit (no mocks until recently) or SimpleTest (has had Mocks forever) - but that only holds true so far.

Also why do you use your preferred option? There's another debate between whether development works better or not (or no difference at all) depending on if you verify outcomes (the results of an action) or verify behaviour (results + indirect outputs, i.e. method calls on other objects).
User avatar
feyd
Neighborhood Spidermoddy
Posts: 31559
Joined: Mon Mar 29, 2004 3:24 pm
Location: Bothell, Washington, USA

Post by feyd »

Considering one could write a generic stub/mock that handles it all (in PHP 5+), it doesn't really matter either way to me.

Honestly though, I've never liked the name Mock. I guess I'm just used to Stubs in coming from a background in languages which did not support (in any form) reflection. However, for the large part, I view them as fundamentally the same or rather, I'm ignorant of the differences between the two.
User avatar
Christopher
Site Administrator
Posts: 13596
Joined: Wed Aug 25, 2004 7:54 pm
Location: New York, NY, US

Post by Christopher »

Could you give some examples of how you and Travis perceive Mocks and Stubs? I think it would be helpful because there is some overlap and vagueness to the terms. You guys are a little deep in the esoterica of testing, so clear examples would help.

I think I use more stubs than mocks, but that just may be because you haven't finished PHPMock yet! ;) Likewise I think we need the ability to both verify outcomes and verify behaviour in different circumstances.
(#10850)
User avatar
Maugrim_The_Reaper
DevNet Master
Posts: 2704
Joined: Tue Nov 02, 2004 5:43 am
Location: Ireland

Post by Maugrim_The_Reaper »

Okay, a few examples.

The below code offers up two uses of PHPMock, first as a Mock Object setting expectations of it's use, which are later verified (a failed Mock expectation will fail the entire test), second as a Stub which is basically just created to churn out the defined return values for specified methods. Most of the Mocking API should be obvious, ordered() means the Mock should be called in precise order it's defined. Note a few expectations are optional - e.g. withNoArgs() is already the default assumption but adding it just makes the Mock behaviour clearer.

Borrowing from PHPSpec:

Code: Select all

class DescribeLoggerWithFileStrategy extends PHPSpec_Context
{
    public function itShouldWriteToFile_Mocked()
    {
        $fileWriter = PHPMock::generate('Logger_Writer_File');
        $fileWriter->shouldReceive('write')->with('Hello')->once()->ordered();
        $fileWriter->shouldReceive('readLast')->withNoArgs()->once()
                       ->andReturn('Hello')->ordered();

        $logger = new Logger($fileWriter);
        $logger->log('Hello');
        $this->spec($logger->lastLog())->should->equal('Hello');

        $fileWriter->verify();
    }

    public function itShouldWriteToFile_Stubbed()
    {
        $fileWriter = PHPMock::generate('Logger_Writer_File',
                                     array('write'=>null, 'readLast'=>'Hello'));

        $logger = new Logger($fileWriter);
        $logger->log('Hello');
        $this->spec($logger->lastLog())->should->equal('Hello');
    }
}
Definition time!

I suppose the main thing is to explain how a Mock Object differs from a Stub. I usually refer to Martin Fowler's essay "Mocks Aren't Stubs", or failing that do a search for xUnit Test Patterns (the book is an excellent read but there's a smaller Wiki formatted resource with much of the material). Just summarising here.

Stub: Designed to control indirect inputs to an object under test so isolation from the environment is maintained.

Mock: Designed to verify the object under test is using other objects correctly, as well the same reasoning for Stubs.

So the difference is that Mock Objects act just like Stubs, but also include behaviour verification. Applying TDD or BDD we'd say that we're specifying the indirect outputs of the objects (all those method calls on other objects) as well as its direct outputs (the outcome/resulting state of any action). One of the outcomes of applying Mock Objects is that we need to expose more understanding of how an object works, so that we can design the interfaces of the other objects it's interacting with (they probably won't exist in TDD/BDD yet) which lets us write later tests/specs for those objects on the spot, or a little later, with little effort. The downside is that Mock Objects since they pay attention to call order, method counts, and argument lists, is that they are inherently more brittle than Stubs (outside PHP I sometimes limit brittleness using a tactic called object interaction recording to keep default Mock Object expectations updated dynamically).

Back to me and Travis specifically. The differences above have led to a debate over design. Do we a) write a framework for Stubs, record interactions, and verify them against later Mock Expectations, or b) write a framework for Mocks and support Stubs as "dum-dum" systems with no intelligence/expectations. Behaviour of each is almost identical (talk about esoteric ;)) but each assumption has a small impact on the API which makes the other camp unhappy in some way. In my coded example, I've suggested adopting the Stub Generation approach there so Stubs aren't tied to the Expectation API of Mocks.

If any of that is still vague (arborint's quite right that I'm living in the esoterica - Travis and I are hip deep in testing systems and theory and having a weird sense of fun) don't sweat it. You can read the probably clearer definitions at:
Stub: http://xunitpatterns.com/Test%20Stub.html
Mock: http://xunitpatterns.com/Mock%20Object.html
User avatar
Christopher
Site Administrator
Posts: 13596
Joined: Wed Aug 25, 2004 7:54 pm
Location: New York, NY, US

Post by Christopher »

Maugrim_The_Reaper wrote:Back to me and Travis specifically. The differences above have led to a debate over design. Do we a) write a framework for Stubs, record interactions, and verify them against later Mock Expectations, or b) write a framework for Mocks and support Stubs as "dum-dum" systems with no intelligence/expectations. Behaviour of each is almost identical (talk about esoteric ;)) but each assumption has a small impact on the API which makes the other camp unhappy in some way. In my coded example, I've suggested adopting the Stub Generation approach there so Stubs aren't tied to the Expectation API of Mocks.
I thought you had almost talked yourself into something there... ;)

Honestly I think your implementation is heading in the right direction. And you are right that, in general, Mocks are really just intellegent Stubs. Both are useful and I can see where you might want to transition from one to the other, Stub -> Mock when you find you need more behavior, or Mock -> Stub when you realize you've started down the road to overkill. I think I would give the classes different names to indicate the decision made, but have them both extend a common base class because they are essentially the same. Make changing course easy especially Stub -> Mock.
(#10850)
User avatar
Jenk
DevNet Master
Posts: 3587
Joined: Mon Sep 19, 2005 6:24 am
Location: London

Post by Jenk »

My understanding of stub vs mock.

Stub method:

Code: Select all

public function doSomething () {
    return 'foo';
}
Mock method:

Code: Select all

public function __call ($name, $args) {
    $this->_incrementCallCountFor($name);
    $this->_validateArgsForCall($name, $args);
    $this->_checkOrderOf($name, $args);
    return $this->_returnValueFor($name, $args);
}
Mocks all the way, baby.
User avatar
Chris Corbyn
Breakbeat Nuttzer
Posts: 13098
Joined: Wed Mar 24, 2004 7:57 am
Location: Melbourne, Australia

Post by Chris Corbyn »

I'm a Mockist. Stubs don't generally crop up in my tests except in the form of a Mock object performing nothing but an actor role.

I'd really like to see a Mock object framework which doesn't bundle all its expectation methods into the mock object itself though. Something like JMock where you use a Mockery to create the object and tell the Mockery what you expect to happen on that object.

The only thing with that is that you almost certainly lose the ability to create Partial mocks since it likely needs to somehow act as a strategy on the Mockery (and therefore know about the Mockery) -- most likely through constructor injection. Partial Mocks scare me a bit though.
User avatar
Maugrim_The_Reaper
DevNet Master
Posts: 2704
Joined: Tue Nov 02, 2004 5:43 am
Location: Ireland

Post by Maugrim_The_Reaper »

Travis Swicegood seems to have some ideas in that direction so I'll see where he goes with it. At the moment we have 3-4 methods added to a Mock class declaration - two of them will have a unique prefix (to prevent name clashing and such) but the other two (shouldReceive/verify) would need to be reserved names.

The way the API is targeted, once you call shouldReceive() you get back a different object entirely for expectation setting so there's no class mangling for those (aren't fluent interfaces great ;)). If we can swing out those last two methods it would be absolutely brilliant but we'll see. Having reserved method names isn't the ideal way.
User avatar
Jenk
DevNet Master
Posts: 3587
Joined: Mon Sep 19, 2005 6:24 am
Location: London

Post by Jenk »

Sounding like a broken record.. use a proxy object to completely, 100% avoid naming collisons. I've even posted the code for it in another thread.
User avatar
Ollie Saunders
DevNet Master
Posts: 3179
Joined: Tue May 24, 2005 6:01 pm
Location: UK

Post by Ollie Saunders »

I'm a mockist for sure.
I can see where you might want to transition from one to the other, Stub -> Mock when you find you need more behavior, or Mock -> Stub when you realize you've started down the road to overkill.
Isn't it more logical to allow the programmer to start with sometime simple and expand on it rather than the other way round. That's how TDD has traditionally worked. Increasingly more detailed increment fleshing out the system in increasing detail.

I don't see any problem in using shouldReceive('foo')->andReturn('bar') for stubbing purposes. Are these methods being called on the actual mocks or on a builder object (I believe this is what arborint is referring to as a mockery)? Because that strikes me as a very good idea. You could then have multiple builders, one that approaches it from a mocking perspective and one that approaches from stubbing perspective.

Code: Select all

$mockDescription = new Mock_Description();
$mockDescription->shouldReceive('foo')->withNoArgs()->andReturn('bar');
$mockDescription->shouldReceive('zim')->with(5);
$mock = $mockDescription()->makeFromClass('Monkey');

// use $mock in stuff

$stubDescription = new Stub_Description(array('foo' => 'bar'));
$stub = $stubDescription->makeFromClass('Monkey');

// use $stub in stuff

// could continue to make more mocks/stubs from the existing Mock/Stub_Descriptions
// if descriptions are statey
You also won't have to worry about the issue of reserving names on anything more than a verify() or equivalent method on the mock. You could probably even mostly avoid that by having the mock's __destruct code report back to an observer.
User avatar
Christopher
Site Administrator
Posts: 13596
Joined: Wed Aug 25, 2004 7:54 pm
Location: New York, NY, US

Post by Christopher »

I agree with ole and think he makes some good points. As I said, going Stub -> Mock would be the more common path. After looking at the examples I think the naming is a little obscure. I think it could be clearer with something like:

Code: Select all

$mockDescription = new Mock_Description();
$mockDescription->shouldImplement('foo')->withNoArgs()->andReturn('bar');
$mockDescription->shouldImplement('zim')->withArgs('string', 'int')->andReturn(false);
$mock = $mockDescription()->makeFromClass('Monkey');
(#10850)
User avatar
Maugrim_The_Reaper
DevNet Master
Posts: 2704
Joined: Tue Nov 02, 2004 5:43 am
Location: Ireland

Post by Maugrim_The_Reaper »

Sounding like a broken record.. use a proxy object to completely, 100% avoid naming collisons. I've even posted the code for it in another thread.
Thread reference? I must have missed it...

Travis is looking at that side of things at the moment but I can put it on the mailing list for reference. When we buckle down to development I'll make sure it's referred to in the user stories. It seems it's pretty fascinating for everyone here so I'll elevate its importance and dump the class tag-along mock methods as early as possible.
I agree with ole and think he makes some good points. As I said, going Stub -> Mock would be the more common path. After looking at the examples I think the naming is a little obscure. I think it could be clearer with something like:
What's happening in the code is something I started calling a default mock. Basically a reusable set of descriptions any new Mock/Stub could integrate without needing to set up expectations in every single spec method. The code proposed is quite attractive.

One of the things we're aiming for here is that want to avoid any need to differentiate a Stub from a Mock too literally - both share the same behaviour of expecting method calls, and returning values so the Mocking API is optional (internally it would just trigger a conversion so Stub data is pushed into a collection of Expectation objects for verification purposes). This means you can indeed take an object which behaves like a Stub, and pass it expectations to add Mock behaviour - we just don't call anything a stub/mock if possible (I always find it's confusing to people throwing too much explicit definitions around in any API if not needed). Of course the problem here is that it returns to (proxy object would fix this no doubt) calling a method from the Stub to initiate the Mocking API via a "shouldReceive" or similar.
User avatar
Jenk
DevNet Master
Posts: 3587
Joined: Mon Sep 19, 2005 6:24 am
Location: London

Post by Jenk »

User avatar
Ollie Saunders
DevNet Master
Posts: 3179
Joined: Tue May 24, 2005 6:01 pm
Location: UK

Post by Ollie Saunders »

Here's another idea. Mock builds itself. Say we just call it mock for everything:

Code: Select all

$mock = new Mock::fromClass('foo');
$mock->expectCallTo('bar')->withNoArgs()->thatReturns(4);
$mock->expectCallTo('zim')->with(array(5, 6));
$mock->expectReturningCalls(array('gir' => true, 'pie' => false)); // adding pure a stubbing functionally
$mock->bar(); // throw exception, undefined method, not in mocking mode
$mock->mock(); // mock is no longer configurable
$mock->bar(); // call recorded, mocking is now in effect
$secondMock = clone $mock; // mock mode not in effect for secondMock and call recording reset, $secondMock has the same expectation
If you want more natural language you might consider

Code: Select all

$mock = new Mock::fromClass('foo');
$mock->bar()->shouldBeCalledWithNoArgs()->andReturn(4);
$mock->zim()->shouldBeCalledWithArgs(array(5, 6));
$mock->methods(array('gir', 'pie'))->willReturn(array(true, false));
$mock->mock(); // mock modes as before
User avatar
Chris Corbyn
Breakbeat Nuttzer
Posts: 13098
Joined: Wed Mar 24, 2004 7:57 am
Location: Melbourne, Australia

Post by Chris Corbyn »

ole wrote:If you want more natural language you might consider

Code: Select all

$mock = new Mock::fromClass('foo');
$mock->bar()->shouldBeCalledWithNoArgs()->andReturn(4);
$mock->zim()->shouldBeCalledWithArgs(array(5, 6));
$mock->methods(array('gir', 'pie'))->willReturn(array(true, false));
$mock->mock(); // mock modes as before
The withNoArgs() method and withArgs() methods are a little less fluent than just passing those args to bar() and zim().

I'm thinking almost like JMock:

Code: Select all

$mockery = new Mockery();

$mockEngine = $mockery->mock('Engine');
$mockery->expect($mockEngine)->turn(1000)->andReturnValue(true);

$car = new Car($mockEngine);
$car->setRpm(1000);
$car->drive();

$mockery->assertSatisfied();
Because you pass the mock object to the mockery, the same mockery can be used for all mock objects you create. And because the mockery returns something very similar to the mock object (maybe even a clone of it?) via expect() you can just "record" what you want to happen.

I personally love this approach. It means you can effectively do a dry-run of the events which you expect to follow.

//now wtf is that HUGE noisy insect in my room! 8O
Post Reply