Getting some HTML out of PHPUnit

Discussion of testing theory and practice, including methodologies (such as TDD, BDD, DDD, Agile, XP) and software - anything to do with testing goes here. (Formerly "The Testing Side of Development")

Moderator: General Moderators

User avatar
Ambush Commander
DevNet Master
Posts: 3698
Joined: Mon Oct 25, 2004 9:29 pm
Location: New Jersey, US

Post by Ambush Commander »

From what I hear, SimpleTest is currently the superior unit tester, but once PHPUnit3 is done the pendulum will swing back. Hmm...
alex.barylski
DevNet Evangelist
Posts: 6267
Joined: Tue Dec 21, 2004 5:00 pm
Location: Winnipeg

Post by alex.barylski »

No one has said why it's superior though and I can't remark as I know neither very well, but I am learning them concurrently :)

I'd like to hear a non-biased comparison of features, etc...so I don't have to figure all that out myself :)
User avatar
Ollie Saunders
DevNet Master
Posts: 3179
Joined: Tue May 24, 2005 6:01 pm
Location: UK

Post by Ollie Saunders »

PHPUnit it much more mature and fully featured in comparison to SimpleTest. It also has supposed for all the PHP 5 stuff which SimpleTest does not. I can't really tell you any more sorry.
So what was causing the problems? Was it the CLI issue?
I was just using it in completely the wrong way. You have to right a unit test and then run it via the shell using the phpunit command.
once PHPUnit3 is done the pendulum will swing back
Certainly looking that way.
User avatar
sweatje
Forum Contributor
Posts: 277
Joined: Wed Jun 29, 2005 10:04 pm
Location: Iowa, USA

Post by sweatje »

ole wrote:PHPUnit it much more mature and fully featured in comparison to SimpleTest. It also has supposed for all the PHP 5 stuff which SimpleTest does not. I can't really tell you any more sorry.
SimpleTest is not PHP5 strict code in it's own code base because Marcus is still supporting PHP4 and did not want to fork the code base. The major difference I see is PHPUnit supports code coverage. I believe Marcus never felt a burning need for this becuase he does TDD religiously, and therefore covers the majority of the code base by virtue of the methodology choosen. Aside from that, SimpleTest can catch exceptions, can Mock interfaces, etc. so even though you "can't really tell you any more" I would be interested in hearing why "PHPUnit it much more mature and fully featured in comparison to SimpleTest."
alex.barylski
DevNet Evangelist
Posts: 6267
Joined: Tue Dec 21, 2004 5:00 pm
Location: Winnipeg

Post by alex.barylski »

I was just using it in completely the wrong way. You have to right a unit test and then run it via the shell using the phpunit command.
Thats what I said :P

CLI = Shell (sorta)

You were running the phpUnit from a browser using HTTP instead of calling it directly from CLI (Command Line Interface: The prompt you see when using a shell)

At least this is what I assume

Cheers :)
User avatar
Ollie Saunders
DevNet Master
Posts: 3179
Joined: Tue May 24, 2005 6:01 pm
Location: UK

Post by Ollie Saunders »

sweatje wrote:The major difference I see is PHPUnit supports code coverage. I believe Marcus never felt a burning need for this becuase he does TDD religiously, and therefore covers the majority of the code base by virtue of the methodology choosen.
Yesterday at PHPLondon:
Marcus Baker wrote:So many people are asking for it, I'm going to have to put it in
sweatje wrote:Aside from that, SimpleTest can catch exceptions, can Mock interfaces, etc. so even though you "can't really tell you any more" I would be interested in hearing why "PHPUnit it much more mature and fully featured in comparison to SimpleTest."
Well I'm not claiming to be any great source of knowledge on this because I've only recently starting doing unit testing but here are my reasons
  • You have many more assertions to choose from, making failing tests more informative
  • You can mark tests as incomplete or skipped and have that reflected in the output
  • PHPUnit has been around longer, so by definition must be more mature. Unless I'm wrong.
  • PHPUnit has full support for PHP5 (this is important to me).
  • I've been having difficulties modifing the output appearance of a failing test in SimpleTes. Marcus has kindly agreed to help me with this via email, what a nice chap he is.
I really took all these points and assumed a correlation with the quality and completeness of the library as a whole leading to the larger conclusion that PHPUnit has more features. It was a face value hypothesis but you can't deny it doesn't seem that way.
User avatar
sweatje
Forum Contributor
Posts: 277
Joined: Wed Jun 29, 2005 10:04 pm
Location: Iowa, USA

Post by sweatje »

ole wrote:
  • You have many more assertions to choose from, making failing tests more informative
I guess this one never really occoured to me. You can do anything you want with assertTrue and PHP. For me it is very natural to add a few custom assertions to a test case when writing them. assertExpectation provides even more capabilities this way. It would then be reasonable to add your favorite custom assumptions to a base test case you work from on your projects.

for example, is there really much difference between:

Code: Select all

$this->assertTrue(in_array($val,$arr));
and

Code: Select all

$this->assertContains($val,$arr);
?

I think Marcus has specifically not expanded the list of available assertions to keep the SimpleTest API more managable.
ole wrote:
  • You can mark tests as incomplete or skipped and have that reflected in the output
abstract is your friend. SimpleTest will not run any abstract test cases.
ole wrote:
  • PHPUnit has been around longer, so by definition must be more mature. Unless I'm wrong.
I don't know the actual dates. I would say your definition is poor however. Sourceforge is littered with half started and abandoned projects, as well as many newer projects with great design and functionality. Simple time in existance is a very poor way to guage the maturity of a project.

For years SimpleTest had both Mock Objects and a WebTestCase (which I believe PHPUnit still lacks), which makes it more mature in my book. My rating is surely more subjective that a start date based one however.
ole wrote:
  • PHPUnit has full support for PHP5 (this is important to me).
I have been writing PHP5 exclusivly for several years now, and using SimpleTest every day when I write PHP code. I don't understand what you find lacking here. Perhaps you could elaborate.
ole wrote:
  • I've been having difficulties modifing the output appearance of a failing test in SimpleTes. Marcus has kindly agreed to help me with this via email, what a nice chap he is.
I really took all these points and assumed a correlation with the quality and completeness of the library as a whole leading to the larger conclusion that PHPUnit has more features. It was a face value hypothesis but you can't deny it doesn't seem that way.
You will never find me disagreeing that Marcus is a nice chap. I belive I have probably learned more from Marcus than any other individual PHP'r I've come in contact with.

BTW, I hope you don't find my tone argumentative. I am genuinely interested in the differences between these two pieces of software, and in understanding how SimpleTest might be improved as well.

Regards,
User avatar
Ollie Saunders
DevNet Master
Posts: 3179
Joined: Tue May 24, 2005 6:01 pm
Location: UK

Post by Ollie Saunders »

is there really much difference between assertTrue(in_array) and assertContains
I think there is. Because if the test fails for assertTrue(in_array) you are just told it was false where it should have been true. Whereas with assertContains the error message can be tailored more specifically. For instance showing you what the array does contain or telling you if you actually even supplied an array.
abstract is your friend. SimpleTest will not run any abstract test cases.
Well i've just been changing function names like testStrlen to skip_testStrlen, didn't know you could use abstract although presumibly only on classes not individual methods because you can't have a body for them. But as far as I can tell you aren't informed in the output of skipped or incomplete tests.
I don't know the actual dates. I would say your definition is poor however. Sourceforge is littered with half started and abandoned projects, as well as many newer projects with great design and functionality. Simple time in existance is a very poor way to guage the maturity of a project.
v3.0 > v1.0. SimpleTest first release on SourceForge appears to be 2005-05-30 21:10 and PHPUnit 0.3 on sourceforge appears to be 2000-11-10 06:00 although first stable PEAR release is 2002-07-11.
My rating is surely more subjective that a start date based one however.
Yes, maturity is, level of completeness not whether it has specifics or not. Of course I cannot argue which is the most complete because I do not know enough about either or TDD itself.
I have been writing PHP5 exclusivly for several years now, and using SimpleTest every day when I write PHP code. I don't understand what you find lacking here. Perhaps you could elaborate.
  • I can't, easily, get messages out of exceptions that have occurred.
  • I can't test whether they have occured without writing try catch blocks and using assertNull
  • I have to set error_reporting without E_STRICT, which means my code isn't tested with E_STRICT on which is a problem
Those are the only problems I've experienced to date. I'm expecting more.
You will never find me disagreeing that Marcus is a nice chap. I belive I have probably learned more from Marcus than any other individual PHP'r I've come in contact with.
Yeah he did a presentation on Agile Development yesterday at PHPLondon *ole looks at the time*, I mean two days ago, which I think may have just changed everything about the way I develop. It was really enjoyible. He must have been talking for an hour but I wasn't bored for a minute.

Apparently you are likely to make an appearance next month, is that right? Perhaps we can say hello.
BTW, I hope you don't find my tone argumentative. I am genuinely interested in the differences between these two pieces of software, and in understanding how SimpleTest might be improved as well.
Your tone did come across as argumentative. In fact I wasn't going to reply were it not for you saying that. I'm glad you did.
User avatar
sweatje
Forum Contributor
Posts: 277
Joined: Wed Jun 29, 2005 10:04 pm
Location: Iowa, USA

Post by sweatje »

ole wrote:
abstract is your friend. SimpleTest will not run any abstract test cases.
Well i've just been changing function names like testStrlen to skip_testStrlen, didn't know you could use abstract although presumibly only on classes not individual methods because you can't have a body for them. But as far as I can tell you aren't informed in the output of skipped or incomplete tests.
abstract works on whole test cases, I usuually just put and x infront of the test methods I do not want run, so I can later do a search and replace for xtest with test to re-enable them in the test case.
ole wrote:v3.0 > v1.0. SimpleTest first release on SourceForge appears to be 2005-05-30 21:10 and PHPUnit 0.3 on sourceforge appears to be 2000-11-10 06:00 although first stable PEAR release is 2002-07-11.
The only thing to be careful of there was there were about half a dozen different PHPUnit projects going a few years ago. You would need to be careful to make sure you were actually tracing the lineage of the current project, not one of the abandon ones.


ole wrote: I can't, easily, get messages out of exceptions that have occurred.
I can't test whether they have occured without writing try catch blocks and using assertNull
They are a little clunky, but just to make sure you are aware of the current capabilities:

Code: Select all

<?php

require 'simpletest/unit_tester.php';
require 'simpletest/reporter.php';

class subject {
  function bad() {
    throw new Exception('something bad happened');
  }
}

class testExceptions extends UnitTestCase {
  function testIt() {
    $s = new subject;

    $this->expectException();
    $s->bad();
  }
  function testNotExpected() {
    $s = new subject;
    $s->bad();
  }
  function testThisShouldFail() {
    $this->expectException();
  }

}

$test = new testExceptions;
$test->run(new TextReporter);
produces:

Code: Select all

testExceptions
Exception 1!
Unexpected exception of type [Exception] with message [something bad happened] in [/home/sweatje/public_html/testex.php line 8]
1) Failed to trap exception
        in testThisShouldFail
FAILURES!!!
Test cases run: 1/1, Passes: 1, Failures: 1, Exceptions: 1
So you can say "I am expecting an exception", If an exception is unexpectedly thrown, it shows up in the output, and if you are expecting an exception, and none is thrown, you get a failure. This covers a large number of case, but you are correct if you want to do more with the exception, you have to catch it yourself.

ole wrote:I have to set error_reporting without E_STRICT, which means my code isn't tested with E_STRICT on which is a problem
which can only be tackled once PHP4 is abandoned :(

ole wrote:Apparently you are likely to make an appearance next month, is that right? Perhaps we can say hello.
That sounds great, look me up. As soon as I mentioned I was coming, Marcus roped me into doing a presentation, so I should not be hard to find ;)
ole wrote:Your tone did come across as argumentative. In fact I wasn't going to reply were it not for you saying that. I'm glad you did.
Once I read over my response, I saw it kind of looked like a lawyer cross-examining someone. I certainly did not want to convey that sort of a feeling, but it is often hard to read intention into the written word since things can be taken multiple ways. I am glad you choose to respond.

Regards,
User avatar
Ollie Saunders
DevNet Master
Posts: 3179
Joined: Tue May 24, 2005 6:01 pm
Location: UK

Post by Ollie Saunders »

The only thing to be careful of there was there were about half a dozen different PHPUnit projects going a few years ago. You would need to be careful to make sure you were actually tracing the lineage of the current project, not one of the abandon ones.
Yeah the sourceforge ones might not be reliable but the PEAR one definately is.
They are a little clunky, but just to make sure you are aware of the current capabilities:
I wasn't aware of them but yes they are clunky.
That sounds great, look me up. As soon as I mentioned I was coming, Marcus roped me into doing a presentation, so I should not be hard to find
Haha, will do.
Post Reply