PHP Developers Network

A community of PHP developers offering assistance, advice, discussion, and friendship.
 
Loading
It is currently Sun Dec 21, 2014 5:39 am

All times are UTC - 5 hours




Post new topic Reply to topic  [ 11 posts ] 
Author Message
 Post subject: Testing test methods
PostPosted: Mon Sep 27, 2010 11:45 pm 
Offline
DevNet Master

Joined: Wed Feb 11, 2004 4:23 pm
Posts: 4872
Location: Palm beach, Florida
Who writes test for their custom assertion methods and such, and can you share that code & tests and walk us through how you did it? Did you apply TDD to writing the test utility methods themselves?


Top
 Profile  
 
 Post subject: Re: Testing test methods
PostPosted: Tue Oct 26, 2010 8:04 am 
Offline
DevNet Master
User avatar

Joined: Mon Sep 19, 2005 6:24 am
Posts: 3587
Location: London
How complex are the custom assertions? Do they need to be custom assertions, or just a helper method on the test class (and thus can be born from abstraction rather than developed outright)

Have you got an example custom assertion you want to create?


Top
 Profile  
 
 Post subject: Re: Testing test methods
PostPosted: Tue Oct 26, 2010 12:10 pm 
Offline
DevNet Master

Joined: Wed Feb 11, 2004 4:23 pm
Posts: 4872
Location: Palm beach, Florida
Typically I create helpers that return true or false, out of abstractions, and then pass their return value to a built in assertion for verifying.


Top
 Profile  
 
 Post subject: Re: Testing test methods
PostPosted: Wed Oct 27, 2010 4:30 am 
Offline
DevNet Master
User avatar

Joined: Mon Sep 19, 2005 6:24 am
Posts: 3587
Location: London
Why not just use assertions in the helper instead of a return value?

Syntax: [ Download ] [ Hide ]
public function assertAreEqualUsers($expected, $returned) {
  $this->assertIsA("User", $returned);
  $this->assertEqual($expected->name, $returned->name);
  // etc.
}


Top
 Profile  
 
 Post subject: Re: Testing test methods
PostPosted: Wed Oct 27, 2010 8:20 pm 
Offline
DevNet Master

Joined: Wed Feb 11, 2004 4:23 pm
Posts: 4872
Location: Palm beach, Florida
Because then a test failure reports that helper method as the "failure" which is confusing. There is a way to build custom assertion methods in PHPUnit without delegating, but I heard the 'proper' way to do this is to develop your assertions using unit testing, which is what I'm asking for advice on :-)

It makes sense sort of. Assertions are often more complex than the test methods. Often changing one assertion could have the side effect of silently breaking a bunch of tests (so that they always pass or something). I guess really what I'm after is some reassurance that I'm on the right track, and possibly example code :D


Top
 Profile  
 
 Post subject: Re: Testing test methods
PostPosted: Thu Oct 28, 2010 7:44 am 
Offline
DevNet Master
User avatar

Joined: Mon Sep 19, 2005 6:24 am
Posts: 3587
Location: London
That makes perfect sense. :)

I think where I differ is that I don't see it as a need for custom assertions, I only see it as a need to refactor (i.e. extract a method). :)


Top
 Profile  
 
 Post subject: Re: Testing test methods
PostPosted: Fri Oct 29, 2010 1:39 am 
Offline
DevNet Master

Joined: Thu Jan 30, 2003 9:26 pm
Posts: 2893
Location: Glasgow, Scotland
josh wrote:
Who writes test for their custom assertion methods and such, and can you share that code & tests and walk us through how you did it? Did you apply TDD to writing the test utility methods themselves?


I think this is very important for basic assertions which you'll use again and again. If you can't trust your test framework debugging becomes much, much harder. If you're really stuck on something you'll find yourself wondering about the test code itself.

Even use-once assertions ought to be tested but it's interesting to note that you've already got some protection without doing anything. The code you're writing tests the new (untested) test just as much as the new test tests the code. Obviously that's not perfect but to get a false positive, they both have to fail in step. That can certainly happen but it's not usually a thing which can happen easily.

Obviously there are different levels of acceptable risk. Mistakes in a forum post count aren't so bad but mistakes in a financial app can be catastrophic for a business.

I think use your judgment. If the code is simple it might be OK but if in doubt, test it.


Top
 Profile  
 
 Post subject: Re: Testing test methods
PostPosted: Fri Oct 29, 2010 9:15 am 
Offline
DevNet Master

Joined: Wed Feb 11, 2004 4:23 pm
Posts: 4872
Location: Palm beach, Florida
So should assertions be developed with test first or test last? Does it matter? Is there pro(s)/con(s)? Same pros/cons as testing production code I would assume? Is the general consensus not to bother writing these "meta-tests" except in exceptional situations?

The only times I ever get false positives or negatives seems to be when I forgot to make the test fail before passing (Ex. I write the test, it passes on first run, and I move on not realizing I put assertTrue() instead of assertFalse()). I guess the only thing you can do there is try not to be so absent minded, eh? These are usually really easy to fix, but happen more often than an assertion method having a bug in it's implementation.


Top
 Profile  
 
 Post subject: Re: Testing test methods
PostPosted: Fri Oct 29, 2010 10:04 am 
Offline
DevNet Master

Joined: Wed Feb 11, 2004 4:23 pm
Posts: 4872
Location: Palm beach, Florida
So for example do you guys prefer:

Test Helper:
Syntax: [ Download ] [ Hide ]
$this->assertTrue( $this->levelExists('year', $year->getId()), 'when unlink year should delete year');
 


Or

Assertion Method:
Syntax: [ Download ] [ Hide ]
$this->assertLevelExists('year', $year->getId(), 'when unlink year should delete year');
 


And if so, do you use the facilities of your test framework to build a true assertion, with proper line # reporting, and robust failure message generation? Or simply delegate to a couple of lower level built in assertions?


Top
 Profile  
 
 Post subject: Re: Testing test methods
PostPosted: Fri Oct 29, 2010 3:20 pm 
Offline
DevNet Master
User avatar

Joined: Wed Jun 27, 2007 9:44 am
Posts: 4294
Location: Sofia, Bulgaria
I remember I've read somewhere that:
Syntax: [ Download ] [ Hide ]
$this->assertNull($var);

should be used instead of
Syntax: [ Download ] [ Hide ]
$this->assertTrue($var == null);


So, following this logic, I think that the assertion method should be better than the test helper in your examples. I'm new to TDD though ;)

_________________
Image
http://openfmi.net/projects/flattc/ Linux is better :)


Top
 Profile  
 
 Post subject: Re: Testing test methods
PostPosted: Fri Oct 29, 2010 8:01 pm 
Offline
DevNet Master

Joined: Wed Feb 11, 2004 4:23 pm
Posts: 4872
Location: Palm beach, Florida
Well if you have the assertion already developed (assertNull) its a given to use that one. However if there's a cost to creating an assertion method that doesn't yet exist. There's varying tradeoffs between test helpers & test assertion methods. Test helper methods can be conjoined into a criteria, or used from within more than one assertion method to scratch the surface

I agree the custom assertion method is always going to win in terms of readability. But what about cost is what I'm asking I guess?

I think the only 'hard & fast' rule for testing is your tests should read like documentation. So if the readability impact is negligible, what other criteria do you use to decide whether an assertion is warranted over a helper?


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 11 posts ] 

All times are UTC - 5 hours


Who is online

Users browsing this forum: No registered users and 0 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Jump to:  
Powered by phpBB® Forum Software © phpBB Group