Multiple asserts per unit test

Discussion of testing theory and practice, including methodologies (such as TDD, BDD, DDD, Agile, XP) and software - anything to do with testing goes here. (Formerly "The Testing Side of Development")

Moderator: General Moderators

matthijs
DevNet Master
Posts: 3360
Joined: Thu Oct 06, 2005 3:57 pm

Post by matthijs »

Thanks for all the links, Hockey. That'll keep me (and others) occupied for a while!

Hadn't even seen that manual. Following the link to phpspec in maugrim's sig only leads to the code.
alex.barylski
DevNet Evangelist
Posts: 6267
Joined: Tue Dec 21, 2004 5:00 pm
Location: Winnipeg

Post by alex.barylski »

matthijs wrote:Thanks for all the links, Hockey. That'll keep me (and others) occupied for a while!

Hadn't even seen that manual. Following the link to phpspec in maugrim's sig only leads to the code.
Hey man no worries...I typically don't like sharing but because Maugrim has been so generous with mentoring me in BDD and you've helped me tons in CSS stuff...and the fact that it's XMAS eve...I figured WTH? :P

Unfortunately, I compiled that list quickly and missed about a dozen other blogs which were really well done (no strict reference to Booby or was that Ruby?) but hopefully that list helps anyways...It seems BDD is really in it's infancy (actually I like Maugrim mentions that in a previous post) so it's difficult to pickup anyt decent articles...the only good thing about that I bet...is that only people who actually understand the subject are writing articles...not just people who here about it once and decide to blog about it, like in OOP for instance. :)

Sadly, I'm considering actually writing an article on the subject (despite having less than a working-weeks insight) if for any other reason, when I write it helps me hash out ideas and ask more questions. When others read it, they can pick up on misconceptions and provide constructive feedback or argue until their teeth turn gold. :P
User avatar
Maugrim_The_Reaper
DevNet Master
Posts: 2704
Joined: Tue Nov 02, 2004 5:43 am
Location: Ireland

Post by Maugrim_The_Reaper »

http://dev.phpspec.org/manual/index.html

The manual is in draft which is why there's no direct link other than to the Google Code URL. Once PHPSpec 0.2 does have a release, and the manual is finalised for 0.2, then it would be more visible (hey, might even be a web page before redirection to Google ;)).

As to articles for BDD. They're scarce - even scarcer are accurate ones. BDD is mid-evolution so even my original blog entry is outdated to a fault. With the release recently of Rspec 1.0, the community of BDD users has solved a ton of the problems linking XP to Story Runner to low level Specs. It was such a significant update to BDD core ideals that the December Rubyconf slides were updated quite a bit in tone to address the differences between BDD and TDD.
User avatar
Jenk
DevNet Master
Posts: 3587
Joined: Mon Sep 19, 2005 6:24 am
Location: London

Post by Jenk »

re: Maugrims reply to me.

I'm not so sure that is accurate. I was already using TDD from a high level design (or rather, behaviour level) before I stumbled upon BDD. I guess it all depends on how "perfect" you (read: the developer deciding) took TDD to start with. I was already converting User Stories to tests, then after refactoring adding lower level tests. We've also been using FIT/FitNesse for a while, which I can say with absolute certainty makes the job of turning User Stories into tests/specs a lot easier as it just becomes logical/common sense.

(For those unaware of FIT/FitNesse, you enter a user story as an html table, then you can add extra levels of granularity to the top level test/spec and drill down. The end user [you or the product owner] can then click a button on a webpage and it will display green if all tests/specs for that user story pass, or red if not all pass.)
User avatar
Maugrim_The_Reaper
DevNet Master
Posts: 2704
Joined: Tue Nov 02, 2004 5:43 am
Location: Ireland

Post by Maugrim_The_Reaper »

That BDD is not TDD with a few word changes is entirely accurate. There is an element of that obviously since BDD evolved out of TDD. But generally contrasting a typical TDD example with a typical BDD example shows differences at almost every level of the process, from the small to the large. If your TDD and XP practices are comparable to BDD then congratulations - it makes you one of the few who've mastered the art the vast majority of users fail to achieve or even understand. But it is TDD + XP - not TDD.

Does BDD include practices which perfection in TDD strives for (why not perfection?) Definitely - again because BDD evolved out of TDD to solve it's shortcomings which are arguably numerous - read the blog of anyone who's tried to apply TDD and admitted failure. It's not uncommon. But yet again, BDD has a much expanded scope. TDD has never been an isolated practice, and those other practices over the years have gradually been invented, documented, and integrated into how we apply TDD - but they're not defined by TDD and never were. Some didn't even formally exist when TDD was coined. BDD's scope includes integrating those very closely - into the language, the core concepts, and the frameworks in use. Not just a few word tweaks there. Or is DDD also a word-tweaky crock? :)

Then there's the learning curve differences - the danger point to TDD really since it's poorly adopted (not only in PHP) or plain misinterpreted too easily. BDD is easier to learn. It spread like wildfire in Ruby to such an extent it's hard to remember if Ruby even has a Unit Testing library (or would be if it wasn't built in...;)). Ruby conferences have BDD sessions as a matter of course. The frameworks are deliberately designed for BDD which is good since TDD has no specific framework (they just borrowed Unit Testing frameworks from extreme testing which is why everything remains so test-obsessive). It has a shallow learning curve - not the 7 leaps of faith TDD requires. It's used in real projects even - something that was a huge doubt as little as a year ago when the idea was so new it still squeaked.

Does it sound like any BDD user would let the idea of BDD = TDD + 3 New Words stand?
User avatar
Jenk
DevNet Master
Posts: 3587
Joined: Mon Sep 19, 2005 6:24 am
Location: London

Post by Jenk »

Probably not, but the the average TDD user (on the grand scheme of things) probably isn't using TDD effectively, either. It still boils down to how you use(d) TDD compared to the ideal of BDD. The concepts brought to light with BDD have always been there for TDD, that is certain, BDD was evolved from TDD to make it easier for developers to grasp this. Dave Astel and a few others from the "Ruby Gang" were looking to improve TDD's use in the wide world, and after much deliberation decided it would be better to chop it right down and (almost) start over with a new framework and the inclusion of should, be, not etc. just to ease the translation from "As a User, when I submit X, I want to receive Y" to (in Smalltalk for sake of my memory):

Code: Select all

x submit should equal: y
I've been seeing people who can use TDD, not necessarilly in the ideal way, but with some degree of success, look at BDD and simply say "Why bother? That's what I already do." but they are actually not looking at either correctly. They are still testing private methods/properties, or verfiying needlessly because that is what they did before hand with TDD. It's all a matter of relevance.

I still stand that BDD is simply TDD re-branded. The process for effective TDD, and BDD, is translate requirements (e.g. user stories) into tests, complete code to pass the tests, then refactor into more tests, and code to complete the tests (rinse repeat ad nauseum) until you have completed all the requirements. What BDD has done very well is, like you say, shorten the learning curve of xDD, and rightly so. I believe the naming of Test Driven Design was simply brought about because someone, somewhere, decided it would be useful to have the tests coded before coding the actual application - so for simplicities sake called it TDD. Later we have looked and thought "That's silly, Tests are past-tense."

The seminars and conferences have exploded for BDD, mainly within the Ruby community because RSpec get's the most attention from the testing guru's, because BDD has accomplished what it set out to do.. making TDD easier to understand, so more people have jumped onto it and thus there is more discussion.

The two are achingly similar - you have a spec from a user. You need to ensure that what you create satisfies that spec. Both in TDD and in BDD, you create a programmatic module for ensuring that spec is met. TDD tended to throw people off because it was a common mistake to assume that if it is a class/object, it must have it's own test. This has some logic behind it, you have a collection of common objects/classes shared amongst many applications, so it is logical you will have a test/spec that will simply ensure the class/object does what it says on the tin. An Iterator, you will want to know that it iterates. But for application or business logic, you may want to know if ObjectA will cause an effect on ObjectB, ObjectC and Object22. This is where many familiar with UnitTesting were thrown off, they did not realise that a UnitTest (technically an Intergration test) can still be used here. BDD bridges this gap somewhat.

However, if by "BDD" and the "Evolution of TDD to BDD" you are including the extra publicity/emphasis the guru's are putting out there to let people know it is ok to do the above, then that is a different matter entirely. We would not be debating this case if they did not change the name of TDD, but simply gave all their seminars on TDD with the ethic of "X" (what you claim is the ethic of BDD, and I claim is the ethic of both TDD and BDD.)
User avatar
Maugrim_The_Reaper
DevNet Master
Posts: 2704
Joined: Tue Nov 02, 2004 5:43 am
Location: Ireland

Post by Maugrim_The_Reaper »

after much deliberation decided it would be better to chop it right down and (almost) start over with a new framework and the inclusion of should, be, not etc. just to ease the translation from "As a User, when I submit X, I want to receive Y" to (in Smalltalk for sake of my memory):
Completely new framework called RSpec using the Scenario DSL. Does JUnit support this? It's more then a changing 3-4 words.

Code: Select all

Scenario "savings account has enough money" do
    Given "my savings account balance is", 100 do |balance|
        @savings_account = Accounts::AccountFactory.create(:savings)
        @savings_account.add(balance)
    end
    And "my checking account balance is", 50 do |balance|
        @checking_account = Accounts::AccountFactory.create(:checking)
        @checking_account.add(balance)
    end
    When "I transfer", 20 do |amount|
        @savings_account.transfer(amount.to_i).to(@checking_account)
    end
    Then "my savings account balance should be", 80 do |balance|
        @savings_account.balance.should == balance
    end
    And "my checking account balance should be", 70 do |balance|
        @checking_account.balance.should == balance
    end
end
I still stand that BDD is simply TDD re-branded.
TDD does not include DDD or ATDD. BDD does. Can't simplify much further. BDD goes beyond TDD. Like I said before, if you've already linked the three into a process under XP then good for you. But TDD does not explicitly link them at all - if it did then TDD would never be based on a Unit Testing framework as such frameworks currently stand. We already have Unit Test, FIT, TestNG, BDD, Behave, etc. frameworks because TDD has become inextricably linked with Unit Testing alone.

I still hinge on the differences. TDD does not encompass DDD (BDD does - it has a DSL, it's readable, it has textual specs for lay persons). TDD does not encompass ATDD (BDD does - it has a Scenario encoder, GWT DSL, etc.). TDD does not enforce how to use test doubles. BDD does (Mock Objects, Inside-In development, 1:N mapping). TDD...

They are different - I honestly think those differences are very specific, and very unique to BDD. TDD just does not have them as it stands which is why I argue TDD will eventually become the dinosaur we point to ten years from now while everyone applies BDD (or whatever the next evolution in practice gets called). David Chelimsky put it plain and simple - he put a big red X graphic across a sentence reading "Test Driven Development with “should” instead of “assert”" because it is different. It is just too convenient to label a practice the twin of another with minor changes when it's obviously have some effect for some reason - are 3-4 word changes making that effect? I think not.
User avatar
Jenk
DevNet Master
Posts: 3587
Joined: Mon Sep 19, 2005 6:24 am
Location: London

Post by Jenk »

TDD does not include DDD or ATDD. BDD does. Can't simplify much further. BDD goes beyond TDD. Like I said before, if you've already linked the three into a process under XP then good for you. But TDD does not explicitly link them at all - if it did then TDD would never be based on a Unit Testing framework as such frameworks currently stand. We already have Unit Test, FIT, TestNG, BDD, Behave, etc. frameworks because TDD has become inextricably linked with Unit Testing alone.
This is where we differ then I guess. TDD to me does include those. Unit Testing is the "flaw" in that just because it was/is a class called "UnitTest" (or implied to be a Unit Test only) does not imply you cannot encompass anything else. This is what BDD has changed for me, and it is a small change in my eyes, soley down to the learning curve change (which I heartily agree isn't actually a small change, I only call it small because that is all it changes.) Perhaps my standards for TDD is just too high, or that I'm just simply arguing that anything that involves creating tests before creating the code under test is TDD.
alex.barylski
DevNet Evangelist
Posts: 6267
Joined: Tue Dec 21, 2004 5:00 pm
Location: Winnipeg

Post by alex.barylski »

This is where we differ then I guess. TDD to me does include those. Unit Testing is the "flaw" in that just because it was/is a class called "UnitTest" (or implied to be a Unit Test only) does not imply you cannot encompass anything else. This is what BDD has changed for me, and it is a small change in my eyes, soley down to the learning curve change (which I heartily agree isn't actually a small change, I only call it small because that is all it changes.) Perhaps my standards for TDD is just too high, or that I'm just simply arguing that anything that involves creating tests before creating the code under test is TDD.
First you cite him, now you spite him... :lol:

I'm just kidding...I'm sure you haven't spited anyone...although I find it funny that the student is arguing with the master...

Cheers :)
User avatar
Maugrim_The_Reaper
DevNet Master
Posts: 2704
Joined: Tue Nov 02, 2004 5:43 am
Location: Ireland

Post by Maugrim_The_Reaper »

Not a student/master thing. I've been on the forum long enough to consider JCart one of the forums foremost TDD/Testing experts. If not THE foremost. I think where we differ currently has gone to defining TDD. I belief it doesn't encompass DDD or ATDD which are separate practices but I think JCart is so foremost he really has put them into the TDD category for use which would IMO indicate he's so foremost he already does BDD and doesn't distinguish them as I do.

There may be hope for PHP yet ;).
alex.barylski
DevNet Evangelist
Posts: 6267
Joined: Tue Dec 21, 2004 5:00 pm
Location: Winnipeg

Post by alex.barylski »

Maugrim_The_Reaper wrote:Not a student/master thing
I know...I was just teasing...the way I see it...if he can be dumbfounded by the fact I don't agree with his opinion or see things differently...then I can tease him about first quoting you, then arguing with you...

Code: Select all

I've been on the forum long enough to consider JCart one of the forums foremost TDD/Testing experts. If not THE foremost. I think where we differ currently has gone to defining TDD. I belief it doesn't encompass DDD or ATDD which are separate practices but I think JCart is so foremost he really has put them into the TDD category for use which would IMO indicate he's so foremost he already does BDD and doesn't distinguish them as I do.
Jcart certainly has expressed his opinions on the TDD forum more than many others on here, but so has Jenk, which is who I was teasing... :P
User avatar
Jenk
DevNet Master
Posts: 3587
Joined: Mon Sep 19, 2005 6:24 am
Location: London

Post by Jenk »

Well, teasing or no teasing, I was getting tired of trying to convert others - something which I remembered was a past trait of mine that did not do me any favours. But I will also add, not regarding anyone here before I say it, there is a certain degree of evangelism amongst the testing commmunity. It does more good than bad, I will happily agree, as it is another reason for people to progress, but it does add a flavour of "BDD is newer than/an evolution of TDD, so TDD must suck and be so primitive." whilst I, as should be clear by now, hold the opinion that BDD does nothing more than make it easier to understand what the benefits of effective TDD are.

Anyway, I kindly offer that we agree to disagree should this provoke any more reactions.
User avatar
John Cartwright
Site Admin
Posts: 11470
Joined: Tue Dec 23, 2003 2:10 am
Location: Toronto
Contact:

Post by John Cartwright »

Maugrim_The_Reaper wrote:Not a student/master thing. I've been on the forum long enough to consider JCart one of the forums foremost TDD/Testing experts. If not THE foremost. I think where we differ currently has gone to defining TDD. I belief it doesn't encompass DDD or ATDD which are separate practices but I think JCart is so foremost he really has put them into the TDD category for use which would IMO indicate he's so foremost he already does BDD and doesn't distinguish them as I do.

There may be hope for PHP yet ;).
Very flattering coming from someone who has taught me so much. Although I have a sinking suspicion you meant to acknowledge Jenk and not myself :oops:
alex.barylski
DevNet Evangelist
Posts: 6267
Joined: Tue Dec 21, 2004 5:00 pm
Location: Winnipeg

Post by alex.barylski »

Jenk wrote:Well, teasing or no teasing, I was getting tired of trying to convert others - something which I remembered was a past trait of mine that did not do me any favours. But I will also add, not regarding anyone here before I say it, there is a certain degree of evangelism amongst the testing commmunity. It does more good than bad, I will happily agree, as it is another reason for people to progress, but it does add a flavour of "BDD is newer than/an evolution of TDD, so TDD must suck and be so primitive." whilst I, as should be clear by now, hold the opinion that BDD does nothing more than make it easier to understand what the benefits of effective TDD are.

Anyway, I kindly offer that we agree to disagree should this provoke any more reactions.
First off, try not to look at it as "converting" people...otherwise you sound like the Evangelist. :P

All joking aside...while I admit to not being en expert in the field of testing...I've been at it long enough that I have personally seen the similarities but also choose to distinguish the differences (whether they are philosophical, conceptual, nomenclature whatever - even the tiniest difference is differences enough for me). I can see and understand even, how you would see them as two the same, which is why I seen our argument as pointless - but human nature got the best of me and I argued anyways. :P

I think you misunderstood me when I attempted to disseminate the two...I wasn't implying that TDD is primitive...but the fact of the matter is...that anyone that accepts BDD for what it is will see it as a evolution not it's next of kin. If you don't agree with the philosophies of BDD...then obviously...you don't get it or in your case, don't want to bother...so you will never see them as different...

Personally, having just come into the game...I think I've saved myeslf months of learning because I choose to (perhaps through ignorance) see unit testing as TDD and specifying behavior as BDD. It allowed me to quickly grasp the idea of mocking objects (which for th longest time I thought was the same as stubbing). See a behavior as being different than a unit test has I think assisted me greatly - and any assistance I received from Maugrim and others like yourself. ;)

I don't think we need to agree to disagree...in a way...do I not (by virtue of BDD being the evolution of TDD) already agree with you, just choose to see the overall picture a littler differently. :P
Very flattering coming from someone who has taught me so much. Although I have a sinking suspicion you meant to acknowledge Jenk and not myself
I'm sure he would include you in that picture as well...anyways...isn't this sweet though...you guys all complimenting each other on Christmas? :lol:

Merry Christmas
Post Reply