PHP unit-testing framework: which one do you use?

Discussion of testing theory and practice, including methodologies (such as TDD, BDD, DDD, Agile, XP) and software - anything to do with testing goes here. (Formerly "The Testing Side of Development")

Moderator: General Moderators

Post Reply

Which testing framework you are using?

Simpletest
15
83%
PHPUnit
3
17%
Test::Simple
0
No votes
Test::More
0
No votes
Something else
0
No votes
 
Total votes: 18

User avatar
Weirdan
Moderator
Posts: 5978
Joined: Mon Nov 03, 2003 6:13 pm
Location: Odessa, Ukraine

PHP unit-testing framework: which one do you use?

Post by Weirdan »

I had used to use Simpletest until I had to switch to PHP5 and it had been great. Nowadays I'm using PHPUnit3 for the reason of better PHP5 support (Simpletest produces too much noise when ran under the E_STRICT). With this poll I'd like to get a feeling of what framework is popular today, and why.
User avatar
RobertGonzalez
Site Administrator
Posts: 14293
Joined: Tue Sep 09, 2003 6:04 pm
Location: Fremont, CA, USA

Post by RobertGonzalez »

I use Simpletest, but that is primarily because the community here seems to really promote it. It is easy to use (but I cannot compare that to anything) and easy to understand.

Quite honestly I hadn't realized that there were that many test suites out there.
User avatar
Maugrim_The_Reaper
DevNet Master
Posts: 2704
Joined: Tue Nov 02, 2004 5:43 am
Location: Ireland

Post by Maugrim_The_Reaper »

You'll find SimpleTest is predominant on the forum, and likely over on Sitepoint I'd guess. PHPUnit has superior marketing and development support though it really is just catching up with SimpleTest and there will be little between them in a few months when the PHP5 SimpleTest conversion gets underway (ST is hitting RC1 of the next - and last - PHP4 oriented version shortly).

So the forum is likely not representative of global use. I have used both over the last few years, and I still very much prefer SimpleTest. Recently ST has also started a stretch of experimental add-ons for things like Selenium integration, continuous integration reporting, and auto test runners which is looking pretty cool. In a few days it will be migrated to Subversion (finally!). So its future is looking bright.

Note: I fix bugs in ST when time allows. Definitely biased ;).

As for the other options.
User avatar
dbevfat
Forum Contributor
Posts: 126
Joined: Tue Jun 28, 2005 2:47 pm
Location: Ljubljana, Slovenia

Post by dbevfat »

Maugrim is right about the general preference for Simpletest here and on Sitepoint. I myself was using ST for a long time, but have switched lately, because PHPUnit seems to have more to offer than Simpletest. Ever since it has Mock objects, I see no obvious reason to use Simpletest any more, not counting the community support, of course, where Simpletest wins by far. It had PHP5 support (E_STRICT compliant) when I was really needing it, phing and selenium support, code coverage and logging. Not sure how Simpletest stands on these now, but at that time it actually had very little of these features, if any at all. I still use Simpletest, mostly for legacy projects, but my heart is elsewhere ... :)

Nevertheless, it appears that better times are ahead (I'm quite sure Maugrim knows a lot more about what Simpletest can do than I), so my guess is that you can't be wrong choosing either. Give both a try and decide. I wouldn't go for the Perl-based testers, because I don't really know what's the point, but they may be ok too.

regards
User avatar
Ollie Saunders
DevNet Master
Posts: 3179
Joined: Tue May 24, 2005 6:01 pm
Location: UK

Post by Ollie Saunders »

Biased as charged. SimpleTest only here. I've considered leaving many times mainly because of how it has fallen behind PHPUnit (in terms of actual releases) as I result I use the CVS version of SimpleTest and that's not particularly stable.

One particularly frustrating thing only recently, where I couldn't throw exceptions in the callback function from preg_replace_callback because of a PHP bug and I couldn't test for trigger_errors because of a SimpleTest bug, led me to re-write this whole thing using explode() and lots of bog standard string processing functions. The re-write wasn't nearly as ugly as I expected but it was really annoying. Both bugs have since been fixed.

SimpleTest is great. Lastcraft really knows his OO so it's easily extended. The documentation is really good too. I don't know PHPUnit might be the same in both those regards. Although I'm particularly against the lack of class namespacing. SimpleTest fills the global namespace and this can complicate things. SimpleTest has its problems but, you know, I've written too many test cases too turn back now, I'm interested in watching it improve and I may help in future.
User avatar
Maugrim_The_Reaper
DevNet Master
Posts: 2704
Joined: Tue Nov 02, 2004 5:43 am
Location: Ireland

Post by Maugrim_The_Reaper »

I think the only current major issue in ST (from the perspective of features in CVS) is PHP5 E_STRICT compliance. We have Selenium in CVS, and I have Phing running tests and logging results from the ArrayReporter. Code coverage I'm not familiar with since I never really use it (it's more useful on bigger code bases than I usually work with) - I don't think it's in ST just yet. Some of the most recent code (e.g. Selenium RC support) is actually in PHP5 already.

If bugs do crop up for those using the CVS version be sure to report them. There's an RC1 of the last PHP4 version being released very soon :). You can PM me personally if you're not on the mailing list or posting to the Sourceforge bug tracker.
User avatar
Weirdan
Moderator
Posts: 5978
Joined: Mon Nov 03, 2003 6:13 pm
Location: Odessa, Ukraine

Post by Weirdan »

Maugrim, I remember the time when generated mocks in ST were all inherited from the base SimpleMock class... is it still that way?
User avatar
Ambush Commander
DevNet Master
Posts: 3698
Joined: Mon Oct 25, 2004 9:29 pm
Location: New Jersey, US

Post by Ambush Commander »

I've never heard of Test::Simple or Test::More. Anyone care to explain?
I think the only current major issue in ST (from the perspective of features in CVS) is PHP5 E_STRICT compliance.
Feyd released a PHP5 strict version of SimpleTest a while back...
lastcraft
Forum Commoner
Posts: 80
Joined: Sat Jul 12, 2003 10:31 pm
Location: London

Post by lastcraft »

Hi...
Weirdan wrote:Maugrim, I remember the time when generated mocks in ST were all inherited from the base SimpleMock class... is it still that way?
No.

This was done to ensure nothing leaked into the test. I wanted the tests to fail (false positives) rather than to lie. The difficulty of type hinting concrete classes forced a bit of a turnaround in the latest version. I consider hinting on a class name to be a bad practice, but then if the code was perfect you wouldn't need tests either. The mocks cope better with type hints now, but will drop back to a cloning strategy if there are final methods.

If you type hint on a class name that has a final method then there is really nothing I can do...

yours, Marcus
Post Reply