Beyond TDD: Behaviour-Driven Development

Discussion of testing theory and practice, including methodologies (such as TDD, BDD, DDD, Agile, XP) and software - anything to do with testing goes here. (Formerly "The Testing Side of Development")

Moderator: General Moderators

User avatar
Maugrim_The_Reaper
DevNet Master
Posts: 2704
Joined: Tue Nov 02, 2004 5:43 am
Location: Ireland

Beyond TDD: Behaviour-Driven Development

Post by Maugrim_The_Reaper »

It's oddly counter-productive to post this to a testing category but if the shoe only half fits... ;)

I am basically looking for some opinions on the theory on Behaviour-Driven Testing (BDD) from the perspective of PHP developers. So I figured here would be a good bet. Folk here have one of the highest rates of TDD adoption I know of in a PHP community.

I posted a quick exploration of the basic theory (omitting several tag-alongs to BDD) on my blog. Please fell free to ask any remote question or criticism you may have below:
http://blog.astrumfutura.com/archives/3 ... lored.html

About a year ago now I started to learn Ruby, and during the course of that encountered the concept of BDD. BDD in Ruby is predominantly done with a library called RSpec. After using RSpec for 10+ months, and being a young library tracking its changing API, I came to quite enjoy it's approach to the core goals of TDD. On returning to PHP (I was Ruby focused up to June) for various work, I found PHP's lack of BDD frameworks hard to accept.

Recently, a tiny group of us (me + 1) have been seriously considering starting up a framework project, so I'm interested in seeing how Devnetwork members would view the concept. The more I see how PHP developers react and think, the more I can effectively put the message across. As a bonus, even if you think the idea lacks value you at least get some 1 on 1 coverage from someone practicing BDD.

I should note that the extraordinary thing I really found odd on linking PHP to BDD, is that there seems to be zero coverage of BDD on any PHP oriented site/forum or mailing list. Is the PHP community really so ignorant of trends in other languages?

Your thoughts are welcome!
User avatar
TheMoose
Forum Contributor
Posts: 351
Joined: Tue May 23, 2006 10:42 am

Post by TheMoose »

I don't think we're so ignorant to other languages so much as apathetic to their techniques. I think that a BDD framework would definitely be interesting, although as you said in the article, I don't see it being widely adopted as a great many people don't truly understand "testing" (whether it be behavioral or unit based) as a design aspect.

I can say personally that when I first started playing with unit tests, I though of them as just that, unit tests to make sure that specific portion of the code did it's job properly. As I drove deeper into it, it didn't seem logical to test whether one part worked by itself when I needed to see whether a stream of pieces worked as they should (a user object interacting with a database object, etc etc), which in essence, is this behavior stuff (in short that's what it is).
User avatar
Maugrim_The_Reaper
DevNet Master
Posts: 2704
Joined: Tue Nov 02, 2004 5:43 am
Location: Ireland

Post by Maugrim_The_Reaper »

I think that a BDD framework would definitely be interesting, although as you said in the article, I don't see it being widely adopted as a great many people don't truly understand "testing" (whether it be behavioral or unit based) as a design aspect.
That's the main hurdle. People just aren't understanding at the moment. So anything that makes that understanding easier to come by has quite a bit of value attached. Of course BDD is not just TDD with some language changes, it holds a few more benefits in addition that TDD doesn't fully marry together in XP.
As I drove deeper into it, it didn't seem logical to test whether one part worked by itself when I needed to see whether a stream of pieces worked as they should (a user object interacting with a database object, etc etc), which in essence, is this behavior stuff (in short that's what it is).
I'd say the example is flawed, but you're close. In terms of units, we test one method at a time. Now in terms of behaviours, we might specify against several methods. Maybe it takes several methods to get a useful behaviour, or several classes. And let's remember Refactoring, which does not alter behaviour, can itself split up or amalgamate units quite naturally anyway. So the unit focused perspective is largely an outcome of getting too enamoured with testing. It's cool if you're just unit testing, but TDD is not unit testing. Let the tests follow the behaviour (not the unit).

Thanks for taking the time out to read the blog ;).
User avatar
Christopher
Site Administrator
Posts: 13596
Joined: Wed Aug 25, 2004 7:54 pm
Location: New York, NY, US

Post by Christopher »

Honestly Pádraic, I think you drank a little too much of the Kool-aid. I agree that many of the goals of BDD are worthwhile, but for a number of reasons I think it is just one of the many experimentation phases in the post-TDD era of XP. My prediction is that the major Unit Testers will add the good stuff learned from BDD, but that BDD itself will go nowhere. I could be wrong though. ;)
(#10850)
User avatar
Maugrim_The_Reaper
DevNet Master
Posts: 2704
Joined: Tue Nov 02, 2004 5:43 am
Location: Ireland

Post by Maugrim_The_Reaper »

Been drinking for 10+ months and the taste remains quite pleasant, thank you ;).

Is BDD an experiment? I quite agree - it's barely over the 1 year mark since semi-popular libraries hit alpha, not even two since the primary BDD manifesto posts turned up. But I can't ignore that after using TDD for a while now, it's getting pretty frustrating using it outside a small team I have some influence over. First PHPUnit took a ridiculous amount of time to support Mock Objects. Currently I've spent months dancing around the Zend Framework proposals with folk discussing colour schemes for the bike shed and re-implementing the same ideas (with the exact same behaviour) a dozen ways over. ;) I'm motivated to do something to push TDD's goals.

So for a minimal investment of time, I feel quite happy to do a little blogging. I enjoyed using RSpec, so I'm enthusiastic about sharing the experience with the PHP community. Since that leads me to needing a PHP BDD framework, I'm pretty interested to seeing something turn up. Now a cool response would be to explain your reasons :). The problem with using the Kool-Aid argument?
Remember folks, we aren't going for user satisfaction. We aren't going for happy. We're going for all-out passion. And that comes with a price tag. Detractors. Lots of them. And they talk. For every passionate user out evangelizing you to everyone they meet, a koolaid-hunter will do his (or her) best to make sure everyone knows that your passionate users have lost their minds. That they're victim of marketing hype. Sheep.
Kathy Sierra http://headrush.typepad.com/

Have not gone totally over the edge...
User avatar
Christopher
Site Administrator
Posts: 13596
Joined: Wed Aug 25, 2004 7:54 pm
Location: New York, NY, US

Post by Christopher »

As I said, I also agree with the goals of BDD. But BDD is not the only evolutionary thing going on in testing -- even if it is the kool-aid-iest. Look at things like Java's TestNG you can see incremental improvements in different directions. And old tools like JUnit are not standing still. Neither BDD and TDD do not do enough with environment and preconditions. And neither really attack speeding up the rote part of coding tests.

The real question about BDD is which part is actually a systemic improvement in testing and which part is just nicer syntax (to some)?

I also think you need to look a the guys who build the current TDD tools. They are definitely not kool-aid drinkers -- in fact they are (perhaps unfortunately) the exact opposite. If you want to be effective in PHP, I would check with Marcus Baker or Perrick Penet about whether it is even possible to add BDD to SimpleTest (and if there is interest). That would be interesting.
(#10850)
User avatar
Maugrim_The_Reaper
DevNet Master
Posts: 2704
Joined: Tue Nov 02, 2004 5:43 am
Location: Ireland

Post by Maugrim_The_Reaper »

As I said, I also agree with the goals of BDD. But BDD is not the only evolutionary thing going on in testing -- even if it is the kool-aid-iest. Look at things like Java's TestNG you can see incremental improvements in different directions.
That's cool, because BDD is not an evolutionary thing in testing. It's an evolutionary thing in how to think about TDD properly by providing an accessible language. ;)
And old tools like JUnit are not standing still. Neither BDD and TDD do not do enough with environment and preconditions. And neither really attack speeding up the rote part of coding tests.
Never stated otherwise - BDD inherits several flaws with TDD since it's a relatively youthful evolution. As features improve TDD, no doubt they will move across to BDD very quickly. But since BDD is not TDD only, testing concerns are not as big a factor. Have a look at where JBehave and RSpec (both BDD, both at different levels) are going.
The real question about BDD is which part is actually a systemic improvement in testing and which part is just nicer syntax (to some)?
There is zero systemic improvement in testing, because we are not testing. Are you trying to prove a point for me? I just wrote a blog post outlining why BDD makes sense, and why the nomenclature of testing is interfering with people's understanding of how TDD is supposed to work. Improvements to testing do not, per se, improve the performance of TDD as a design discipline.

Is BDD a nicer syntax? One hopes so. Have you looked at some TDD efforts in PHP recently? Is it "just" a nicer syntax? BDD is being used as an ATDD tool for capturing executable examples of application requirements, it's sometimes described as a loose form of DDD's umbiquitous language, and it provides highly readable documents. There is added-value there over what TDD generally produces. From an XP perspective there's merit beyond TDD to where it's heading.
I also think you need to look a the guys who build the current TDD tools. They are definitely not kool-aid drinkers -- in fact they are (perhaps unfortunately) the exact opposite. If you want to be effective in PHP, I would check with Marcus Baker or Perrick Penet about whether it is even possible to add BDD
I haven't checked with Marcus or Perrick, but I have with Travis. Travis has a lot of TDD experience, and like me is a long time Phing user, so it's been an interesting few days.
User avatar
Jenk
DevNet Master
Posts: 3587
Joined: Mon Sep 19, 2005 6:24 am
Location: London

Post by Jenk »

I've been reading up on BDD for a few weeks (for Smalltalk) and tbh, you said it best in one line:

"If you are doing TDD well, you are already doing BDD."
(and on a sidenote, I thought it was a well written article :) )

I'd disagree with the line that it is picking up on the failings of the TDD world. More like "Making it easier to understand" (though this is a failing of TDD, I agree, but it's a bit brash to just state "picking up the failings of xyz") or "Simplifying the world of TDD"

Regarding the "acceptance into the world" hurdle.. who are you making this framework for? You, or the world? (ideally both, but bare with me) if it is useful to you, great, get on an make it. If others like it and use it as well, bonus. No point making/not making something just because it would be popular. Unless you'll be putting a price tag on it of course..

Also I think I read a comment on the chaining of commands in a reply above somewhere, to have an object/method return an instance of self/this/$this is the default behaviour in a number of languages, and is for lack of a better phrase, damned useful in many situations. It's also interesting that in your example Padraic, you mention the syntax being near to plain english. This is one of the joys of Smalltalk if you were not aware. :) [/plug]

P.S. Rather than churn out a new framework, why not contribute to/extend on an existing framework such as SimpleTest?
User avatar
Maugrim_The_Reaper
DevNet Master
Posts: 2704
Joined: Tue Nov 02, 2004 5:43 am
Location: Ireland

Post by Maugrim_The_Reaper »

I've been reading up on BDD for a few weeks (for Smalltalk) and tbh, you said it best in one line:

"If you are doing TDD well, you are already doing BDD."
(and on a sidenote, I thought it was a well written article Smile )
I actually thought about it this time around ;). I'm fairly sure it still suffered from brevity somewhat since it was geared up for BDD on low-level code, rather than high-level application specifications.

There just had to be someone else familiar with BDD on this forum - should have know it would be you!
I'd disagree with the line that it is picking up on the failings of the TDD world. More like "Making it easier to understand" (though this is a failing of TDD, I agree, but it's a bit brash to just state "picking up the failings of xyz") or "Simplifying the world of TDD"
I'd actually hoped to avoid (hard to so since BDD seems like criticism) the idea the TDD process had a failure, when the main failure is in how it's explained/presented not the process itself.

Can you quite from the blog? It's something I'd love to note for future avoidance. Or at least put it more delicately.
Regarding the "acceptance into the world" hurdle.. who are you making this framework for? You, or the world? (ideally both, but bare with me) if it is useful to you, great, get on an make it. If others like it and use it as well, bonus. No point making/not making something just because it would be popular. Unless you'll be putting a price tag on it of course..
Primarily, it's because I want one. Don't get me wrong, the idea of bringing BDD to PHP does have a certain ego-boosting appeal, but really it's because I liked using RSpec in Ruby and having it in PHP is a personal want.

I was plain lucky that Travis was even remotely interested and emailed me.
Also I think I read a comment on the chaining of commands in a reply above somewhere, to have an object/method return an instance of self/this/$this is the default behaviour in a number of languages, and is for lack of a better phrase, damned useful in many situations. It's also interesting that in your example Padraic, you mention the syntax being near to plain english. This is one of the joys of Smalltalk if you were not aware. Smile [/plug]
Sorry to say I barely know Squeak, and just enough to know Smalltalk is very neat. I think readability of code is often neglected in the braceful languages more so than in the braceless. A little fluent interface, and method aliasing, it's very useful. Plug away ;).
P.S. Rather than churn out a new framework, why not contribute to/extend on an existing framework such as SimpleTest?
Because we're not sure it's immediately possible. It's probably a simple matter to alias assert methods, but you still need to work through the fluent interface, matching, and behaviour sharing. So there's still a chunk of code needing to be written. At a bare minimum there would likely be a Runner compatible with SimpleTest to allow side-by-side existence (pretty useful too if someone doesn't want to migrate the whole way). In the future, perhaps the source can be integrated more tightly. Another much smaller factor is that SimpleTest will eventually migrate to subversion and PHP5 work commence - I'd rather let Marcus and co. focus on that and then we'll see.

Given the uncertainty and trends in adopting xSpec frameworks, a separate library seems the path most travelled so far. I think JUnit has a small aliasing extension but it's not all that known.
User avatar
feyd
Neighborhood Spidermoddy
Posts: 31559
Joined: Mon Mar 29, 2004 3:24 pm
Location: Bothell, Washington, USA

Post by feyd »

I like the idea and the conceptual change to more descriptive, meaningful methods/properties. I've been trying to shift there more myself as time moves on.

One thing that bugged me, maybe because I don't fully understand it all fluently just yet, or it isn't really important, is the "should" method. Personally, I'd go with an "is." "Should" feels less concrete than "is." It's probably ancillary though.
User avatar
Maugrim_The_Reaper
DevNet Master
Posts: 2704
Joined: Tue Nov 02, 2004 5:43 am
Location: Ireland

Post by Maugrim_The_Reaper »

From what I recall, Dave Astels was all for "must" to make it concrete. But as happens in open source projects, he was overruled by others. I can see a point for "should" since it makes sense in English - "I'm going to write a Bowling class and it should do X". So the must or should question is a hard one - must is less used since it pins you to a requirement without any leverage.

"Is" on the other hand... The problem with "is" (and it's getting unholy technical here ;)) is that it's present tense. Since BDD shifts the view point away from verification of what exists now (testing), to specifying what will exist (specs) later, you need a forward looking verb.
User avatar
Jenk
DevNet Master
Posts: 3587
Joined: Mon Sep 19, 2005 6:24 am
Location: London

Post by Jenk »

RE: Quote for "failings of TDD" I may have picked that up from a different page/blog now that I look for it.. :oops:
feyd wrote:I like the idea and the conceptual change to more descriptive, meaningful methods/properties. I've been trying to shift there more myself as time moves on.

One thing that bugged me, maybe because I don't fully understand it all fluently just yet, or it isn't really important, is the "should" method. Personally, I'd go with an "is." "Should" feels less concrete than "is." It's probably ancillary though.
I'm much the same, I prefer solid and absolute references/phrases when testing, but then this is pretty much the key for BDD. It's not happened yet, so by natural language we cannot say "Is" or "Isn't" we can only say "Should" or "Shouldn't" in a future tense. :)

EDIT: It's also, I guess, to shy away from the idiom that a failed test == a bad test.
User avatar
Christopher
Site Administrator
Posts: 13596
Joined: Wed Aug 25, 2004 7:54 pm
Location: New York, NY, US

Post by Christopher »

Maugrim_The_Reaper wrote:There is zero systemic improvement in testing, because we are not testing. Are you trying to prove a point for me? I just wrote a blog post outlining why BDD makes sense, and why the nomenclature of testing is interfering with people's understanding of how TDD is supposed to work. Improvements to testing do not, per se, improve the performance of TDD as a design discipline.
Well of course I am trying to prove a point to you! :) (just as you are to the universe! :)) So yo, let me get all up in your face about BDD... ;)

Again, I agree with the goals of BDD, but after rereading your blog post I find the Kool-aid a little disturbing:

- "TDD is not, will not, and never has been, a testing methodology." This is nice spin, but it is simply not true. It is both a design and testing methodology -- which is the beauty of it.

- "In TDD, which utilises Unit Testing, we usually end up testing on a per unit basis." Actually you build on a behavior-by-behavior basis and end up organizing in units. Maybe the syntax sucks, but that's what happens. It is Unit Testing that is focused on units, not TDD.

- Your three rules are to stop using the words "test", "unit" and "assert". Hence my comment above about syntax vs substance. And for the record I think using different words can be useful.

You really don't have to vilify TDD to promote BDD. I think that is your big mistake. Anyone who has embraced TDD understands the potential benefits of BDD, so why antagonize. Anyone who isn't using tests will have to use them in other areas even if they use BDD, so why send a negative message about testing.

If you focus too much on the fluent interfaces and natural language strings -- you end up with the smurf discussion above about "should" vs "must" vs "is" which is a Kool-aid induced distraction from focusing on Behavior during the design phase which is the interesting part.

I am actually interested in contributing to the creation of BDD tools for PHP, but I would like to get past the Rails is better than Java/PHP/Python style PR campaign and on to a reality-based discussion.
(#10850)
User avatar
Maugrim_The_Reaper
DevNet Master
Posts: 2704
Joined: Tue Nov 02, 2004 5:43 am
Location: Ireland

Post by Maugrim_The_Reaper »

Well of course I am trying to prove a point to you! Smile (just as you are to the universe! Smile) So yo, let me get all up in your face about BDD... Wink
<looks for dark corner to hide in...fails> ;)
- "TDD is not, will not, and never has been, a testing methodology." This is nice spin, but it is simply not true. It is both a design and testing methodology -- which is the beauty of it.
I disagree. It's entirely true. That it gives rise to test coverage is inevitable, but not the core goal. TDD does not test code, since such code does not yet exist. I was pointed to the following quote recently from Jimmy Nilsson:

"One way to describe TDD is to say that you use tests or examples for specifying behaviour."

Applying Domain-Driven Design and Patterns, was published in 2006. If you're not specifying, then you must be verifying. Verification is not a part of TDD.
- "In TDD, which utilises Unit Testing, we usually end up testing on a per unit basis." Actually you build on a behavior-by-behavior basis and end up organizing in units. Maybe the syntax sucks, but that's what happens.
And it's not entirely right. How does behaviour give a crap how many classes/methods you have? If three classes cooperate to give rise to one required behaviour, then does this mean you must test all three classes, or all three as a singular system? What if the three classes were the result of refactoring from a single class already covered - does this suggest we start expanding the specifications, or the tests, to create new class specific testing code? So no, such organisation of tests and production code is a testing approach, not a part of TDD. TDD already gave us a safety net before we refactored - the safety net never changes unless the behaviour does. So any extra code thereafter is for testing only, and unrelated to TDD.

Not sure I explained that brilliantly well - the point is that behaviour and units are rarely a simple 1:1 map.
- Your three rules are to stop using the words "test", "unit" and "assert". Hence my comment above about syntax vs substance. And for the record I think using different words can be useful.
Define substance? If BDD can help improve people's understanding and adoption of TDD's goals then is this not of value? Then there's the integration of DDD and ATDD. The language changes have more consequences than you're considering.
You really don't have to vilify TDD to promote BDD. I think that is your big mistake. Anyone who has embraced TDD understands the potential benefits of BDD, so why antagonize. Anyone who isn't using tests will have to use them in other areas even if they use BDD, so why send a negative message about testing.
Now, now. The entrance sign clearly states that Straw Men are barred from these proceedings ;). I have never vilified TDD (least not intentionally - see JCarts concerns earlier that had me worried). I have never criticised testing either. I have only noted that explaining TDD in terms of testing is not a good idea. Since TDD is not a testing methodology.
If you focus too much on the fluent interfaces and natural language strings -- you end up with the smurf discussion above about "should" vs "must" vs "is" which is a Kool-aid induced distraction from focusing on Behavior during the design phase which is the interesting part.
Missing the point. A fluid interface language that's natural is of a lot of value in Agile/XP. Where do you think acceptance criteria come from? Developers in isolation? Similar tactics, though with less utility (personal belief), are not uncommon in TDD. One such foray was using annotations in JUnit to escape the "test" method prefix for agile dox generation. Sounds great, but it's still leaving @test everywhere. BDD goes beyond this.
I am actually interested in contributing to the creation of BDD tools for PHP, but I would like to get past the Rails is better than Java/PHP/Python style PR campaign and on to a reality-based discussion.
Rails? I don't do Rails. I do Ruby (never learned Ruby just for Rails - I guess DHH would tell me to "F*** Off" in another stupid slide ;)). BDD was first implemented at a higher level in a Java framework. I used Rspec for Ruby a lot over the last ten months, but BDD is not tied to the language (JCart probably came across Smalltalk's variant of xSpec for example).

Last re-quote:

"BDD is what you're already doing, if you're doing TDD well."

The process is not in question, how it's approached and described is.
User avatar
Christopher
Site Administrator
Posts: 13596
Joined: Wed Aug 25, 2004 7:54 pm
Location: New York, NY, US

Post by Christopher »

Well considering that we disagree 100% ... it is interesting that we are both pro-BDD. I just simply don't need all the rationals you give (nor do I agree with many of them) to like BDD. It simply seems a natural enhancement to the toolbox and a better approach to getting the design right.
(#10850)
Post Reply