Page 1 of 3
Tiny Unit Tester
Posted: Wed Feb 15, 2006 12:49 pm
by Christopher
I use Unit Test frameworks for my work, but there are a couple of competing ones and they all require some investment to use. The proponents and authors of them would say that they are easy to use if you just make a little effort -- but the fact that 99% of the members of these forums probably don't use them is a testament to the fact that the level of effort to use them is too high.
So I was wondering what a very easy-to-use Unit Tester might look like. My idea was to have something that was so easy-to-use that people who would not take the time to learn PHPUnit or SimpleTest might use it to create a little test coverage for their code. It also might be handy around there forums to have Unit Tester to use with code snippets. This gets more interesting when you use Unit Tests as the requirements/specification for code -- which is one of the best ways to use Unit Tests.
Here were my requirements:
- Did not require building test classes
- Only support/introduce the most basic "assert" cases
- Track the number of "passed" and "failed" tests
- Have a simple HTML Reporter to show either "passed" in green or list the file and line numbers of failed tests in red
So I wrote small Unit Tester that you could used like this:
Code: Select all
function doubler($number) {
return $number + $number;
}
include 'TinyTest.php';
$tester = new UnitTester();
// change these to incorrect values to see failed tests reported
$tester->assertTrue(doubler(2) == 4);
$tester->assertTrue(1 == 1);
$tester->assertFalse(1 == 2);
$reporter = new HTMLReporter();
$reporter->output($tester);
Here is the code for the Unit Tester:
Code: Select all
class UnitTester {
var $total_passed = 0;
var $total_failed = 0;
function TinyTest() {
}
function assertTrue($result) {
if ($result) {
$this->_testPassed();
} else {
$this->_testFailed();
}
}
function assertFalse($result) {
if ($result) {
$this->_testFailed();
} else {
$this->_testPassed();
}
}
function _testPassed() {
++$this->total_passed;
}
function _testFailed() {
++$this->total_failed;
$backtrace = debug_backtrace();
$this->failures[] = $backtrace[1];
}
}
class HTMLReporter {
var $passed_heading_style = 'color: white; background-color: green; margin-top: 2; padding: 4 4 4 4;';
var $failed_heading_style = 'color: white; background-color: red; margin-top: 2; padding: 4 4 4 4;';
var $failed_test_style = 'color: white; background-color: red; margin-top: 2; padding-left: 4;';
function output($test) {
if ($test->total_failed > 0) {
echo '<div style="' . $this->failed_heading_style . '">Failed ' . $test->total_failed . ' of ' . ($test->total_failed + $test->total_passed) . ' tests. </div>';
$n = 1;
foreach ($test->failures as $backtrace) {
$message = $n++ . ". Failed: {$backtrace['function']} on line {$backtrace['line']} of file {$backtrace['file']}. ";
echo '<div style="' . $this->failed_test_style . '">' . $message . '</div>';
}
} else {
echo '<div style="' . $this->passed_heading_style . '">Passed ' .$test->total_passed . ' tests.</div>';
}
}
}
I suppose I could make it even smaller by putting the reporter into to Unit Test class as a reporter() method. It is probably what would be HTML 99% of the time and if you want a CLI reporter. for example. you could just extend the Unit Test class and overload reporter().
So, is this enough to be useful or are there other features that even the most basic Unit Tester must have? Is this interesting or worthwhile?
Posted: Wed Feb 15, 2006 12:54 pm
by feyd
I can testify that every time I've tried reading the docs for SimpleTest, I've become even more confused. Which is extremely rare for myself.
Yours seems a LOT more straight forward about what it is and isn't doing. I'll have to play with it a bit, of course, but at least I can read it.

Posted: Wed Feb 15, 2006 1:02 pm
by Christopher
So would something like this be handy around here to introduce a little Unit Testing? If tests could be included with posted code, with a link over to Code Snippets where they could easily get this code, an occasional person might have the lightbulb come on.
Posted: Wed Feb 15, 2006 1:11 pm
by feyd
I think it would be handy. I know I'd like to incorporate Unit Tests into things I'm working on, and would be nice to use for regression testing among other things.
Posted: Wed Feb 15, 2006 2:00 pm
by Maugrim_The_Reaper
Wonders how many have used a testing framework outside of Unit Testing - phpt, or test-more (something interesting Chris Shiflett has blogged about in the past - based on a similar Perl method which I'm more familiar with).
Posted: Wed Feb 15, 2006 2:24 pm
by Christopher
I was going to make this a Poll, but I figured that very few members venture into the Unit Test forum. This is really the only PHP Unit Testing forum I know and it is a shame it is not used (I think it is one of the good things McGruff left).
I think it would be good to put a Poll in the Code forum at some point asking if people have or would use Unit Tests, or would use something like this. But I wanted to present it here to get a converstation going. Something like this that might be a stepping stone (or just useful) for DevNetwork members. If there are better ideas I am open to them as I just hacked this up quickly.
I get the sense that people will cut-and-paste a snippet -- if it works and they can see what it does they might use it. Do you think that phpt or test-more would be a better direction?
Posted: Wed Feb 15, 2006 9:54 pm
by Ambush Commander
I can testify that every time I've tried reading the docs for SimpleTest, I've become even more confused. Which is extremely rare for myself.
Nice pun, but actually, when I was reading the docs, it was really interesting and insightful for me. Sort of like reading Martin Fowler type stuff. Hmm...
As for the code, it's
really bare bones. Glad to see you kept the green box style. I'd have to say the biggest missing thing is mocking (dear God, I use that stuff everywhere). Also, while it's extremely simplistic, requiring very little code to get up and running, SimpleTest is much more manageable when you have, say, thirty test suites. It also is consistent with JUnit (which is basically the premiere unit testing framework in Java).
But of course, I think that scalability is a necessary sacrifice for convenience. You can always refactor later. But, I still have misgivings. The main thing is that when I do something of a certain complexity, I wouldn't dare develop it without a set of good unit tests. But when it's that complex, you probably want to go whole hog and use the whole framework anyway. Plus, unit tests make you actually
think about what you are trying to implement, and there's always the chance that you discover that this is going to be a lot harder than you thought.

(that's probably a good thing). I have to think about this. It's an interesting idea though.
Posted: Wed Feb 15, 2006 9:59 pm
by timvw
After trying out the examples at
http://www.lastcraft.com/simple_test.php i was more or less able to get started with testing. Well, i did have some problems with my UserAgent tests, but that is because the documentation was out of sync with the code. If i remember well it was because the simpletestbrowser did (not) follow redirections by default.
Getting the eclipse plugin for simpletest to work was a more challenging task. I must admit that i haven't used this one anymore, but when i was, i was about to fork the project (actually, i did have a custom version) because it didn't allow me to use the (cvs/custom) simpletest files and was forced to use the stable release delivered with the plugin.
I don't think the initial investment to learn about testing is really the problem. I've started with learning simpletest, but quickly discovered that junit and nunit are more or less the same. Good things seem to be copied very quickly from one language/environment into another
I think it's more the investment in writing unit tests for every interface/class/method that isn't very appealing. On the other hand, when i forced myself to write the tests, i discovered a couple of bugs i wouldn't have found that early otherwise.
What i really mis is the possibility to easily perform navigation testing.. Setting up a couple of sequences (and expected page contents) should be made easier.. The only packages that i've found untill know are commercial java products, not really useful for the average php programmer.. On the other hand, does it really matter in which language it is implemented as long as it's able to do the testing?
Another issue is that for writing unit-testable classes you're more or less forced to think about dependency injection.. Even when it doesn't really make sense..
Posted: Wed Feb 15, 2006 10:04 pm
by Ambush Commander
It seems that...
If you can't test that stuff with ease, there's something wrong with your API. Refactor so that you can test them, and then you'll not only have good unit tests but cleaner code.
Posted: Wed Feb 15, 2006 11:53 pm
by Christopher
Ambush Commander wrote:Nice pun, but actually, when I was reading the docs, it was really interesting and insightful for me. Sort of like reading Martin Fowler type stuff. Hmm...
I agree and have learned a lot from Marcus Baker (lastcraft).
Ambush Commander wrote:As for the code, it's really bare bones. Glad to see you kept the green box style. I'd have to say the biggest missing thing is mocking (dear God, I use that stuff everywhere). Also, while it's extremely simplistic, requiring very little code to get up and running, SimpleTest is much more manageable when you have, say, thirty test suites.
Bare bones was what I was going for. I was looking for something that someone with no knowledge of OOP and Unit Testing could get to work really easily. My hope is that is would be a stepping stone and they would move to a real Unit Tester once they understood the concepts and could see the limitations.
Ambush Commander wrote:It also is consistent with JUnit (which is basically the premiere unit testing framework in Java).
How could I make this more consistent with JUnit?
Ambush Commander wrote:I have to think about this. It's an interesting idea though.
I am very interested in feedback or even involvement from those who understand Unit Testing around here. This is just a hack of an idea. Think for example if in the Code section when someone posts some code that doesn't work, the response is to post some Unit Tests to show them where the problem is. They could grab some code like the above from Code Snippets and learn how to track the problems down themselves,
Posted: Wed Feb 15, 2006 11:58 pm
by Ambush Commander
I am very interested in feedback or even involvement from those who understand Unit Testing around here. This is just a hack of an idea. Think for example if in the Code section when someone posts some code that doesn't work, the response is to post some Unit Tests to show them where the problem is. They could grab some code like the above from Code Snippets and learn how to track the problems down themselves,
It does seem like a good "first, baby step." Then, once we've got them hooked ("This unit testing you speak of... what exactly is it?") then we can and (IMO) should hit them with the full test suite.
Posted: Wed Feb 15, 2006 11:59 pm
by Christopher
timvw wrote:I don't think the initial investment to learn about testing is really the problem. I've started with learning simpletest, but quickly discovered that junit and nunit are more or less the same. Good things seem to be copied very quickly from one language/environment into another

As I said, the code above is no replacement for a real Unit Tester. I see it more as a stepping stone and as something to use here in the forums that requires no installation.
timvw wrote:What i really mis is the possibility to easily perform navigation testing.. Setting up a couple of sequences (and expected page contents) should be made easier.. The only packages that i've found untill know are commercial java products, not really useful for the average php programmer.. On the other hand, does it really matter in which language it is implemented as long as it's able to do the testing?
Yeah ... Ambush Commander had a question in this fourm about testing complex pages and page sequences are a similar think. It seems like there needs to be a tool that is almost like a macro-recorder (like record mode in expect) where you go to the pages and it records what they are supposed to look like and what is supposed to be done.
Posted: Thu Feb 16, 2006 12:02 am
by Christopher
Ambush Commander wrote:It seems that...
If you can't test that stuff with ease, there's something wrong with your API. Refactor so that you can test them, and then you'll not only have good unit tests but cleaner code.
Exactly .. but what about the many programmers in the Code Forum who don't know what "Refactor" or "Unit Test" mean, or for that matter what " there's something wrong with your API' means.
Posted: Thu Feb 16, 2006 11:52 am
by timvw
arborint wrote:
It seems like there needs to be a tool that is almost like a macro-recorder (like record mode in expect) where you go to the pages and it records what they are supposed to look like and what is supposed to be done.
Something like
http://www.pushtotest.com/Downloads/features.html would be nice

Posted: Thu Feb 16, 2006 2:38 pm
by Christopher
timvw wrote:arborint wrote:
It seems like there needs to be a tool that is almost like a macro-recorder (like record mode in expect) where you go to the pages and it records what they are supposed to look like and what is supposed to be done.
Something like
http://www.pushtotest.com/Downloads/features.html would be nice

Looks interesting. Have you used it? I may download it and give it a try.
So what do you think about the Tiny Unit Tester? Do you think something like it would be a worthwhile thing to use around here to show unit tests and as a stepping stone to using a real unit tester?