Unit Testing a Unit Testing Framework? :-\

Discussion of testing theory and practice, including methodologies (such as TDD, BDD, DDD, Agile, XP) and software - anything to do with testing goes here. (Formerly "The Testing Side of Development")

Moderator: General Moderators

Post Reply
User avatar
Chris Corbyn
Breakbeat Nuttzer
Posts: 13098
Joined: Wed Mar 24, 2004 7:57 am
Location: Melbourne, Australia

Unit Testing a Unit Testing Framework? :-\

Post by Chris Corbyn »

So this comes full-circle to a chicken/egg type situation.

I've written a Unit Testing Framework for JavaScript with Mock Objects and all that good old stuff. Along with writing it I've been writing acceptance tests which basically should all pass or fail depending upon what is being tested. Now to go whole hog and unit test the framework itself I'd need to use the framework to test the framework -- so how can people actually trust the results? I mean I'd trust them personally because I can tell what should and shouldn't be happening by making sure the tests fail where I say so, and they pass where they should do.

Should I stick with the acceptance tests or do you think using the framework which is being tested in order to run the tests is fair enough?
User avatar
neophyte
DevNet Resident
Posts: 1537
Joined: Tue Jan 20, 2004 4:58 pm
Location: Minnesota

Post by neophyte »

You wrote a unit testing framework for Javascript... Wow. This I gotta see.

How do you test your testing... Good question. I was just thinking that about that today while experimenting for the first time with SimpleTest.
User avatar
Chris Corbyn
Breakbeat Nuttzer
Posts: 13098
Joined: Wed Mar 24, 2004 7:57 am
Location: Melbourne, Australia

Post by Chris Corbyn »

neophyte wrote:You wrote a unit testing framework for Javascript... Wow. This I gotta see.
It kinda wasn't made for public release but I think I'll throw it out there anyway. Most people would probably prefer JSUnit but I prefer the interface of SimpleTest so I've attempted to recreate that fairly closely. I imagine a week or two and there will be an alpha release.

It is one of those weird concepts though, testing a testing tool with the testing tool. Seems a bit backward :P
User avatar
Ambush Commander
DevNet Master
Posts: 3698
Joined: Mon Oct 25, 2004 9:29 pm
Location: New Jersey, US

Post by Ambush Commander »

Does the GNU C compiler compile itself? It didn't when it first started off, but eventually, yeah, it used older versions of itself to compile itself. (I might be wrong, since I'm recalling this from something I read a long time ago)

'd say do the same: use a different testing framework first, then switch it over once your framework is substantially complete. If you wanted to be a real stickler, always use older versions to test your framework (might cause some namespace troubles though).

Another way is to test complex features only using simpler features. Test Mock objects with regular assert methods. Test super-duper mock objects with only mock objects. Etc.
alex.barylski
DevNet Evangelist
Posts: 6267
Joined: Tue Dec 21, 2004 5:00 pm
Location: Winnipeg

Post by alex.barylski »

Many projects use themselves, but like AC has said, it's usually a case of eventually...

subversion I believe is an example of a self-hosted application, in that they use subversion to basically SVN it's own source tree.

Visual C++ 5 was used to write Visual Studio 6 and so on...

I think it's quite common for libraries, applications, etc to follow this approach...I think your on the right track. :)

p.s-I'd like to see your JS unit test library...

I've been contemplating which PHP unit testing framework to starting using (or write my own) and mostly because I do an equal amount of JS and PHP. So having a similar interface for both client/server development would be a awesome bonus as possibly pursuade me into using SimpleTest. :)
User avatar
Maugrim_The_Reaper
DevNet Master
Posts: 2704
Joined: Tue Nov 02, 2004 5:43 am
Location: Ireland

Post by Maugrim_The_Reaper »

SimpleTest tests itself with itself. If something goes wrong then either an unexpected error/exception will occur or the tests will simply fail. Seems weird but it is a workable solution. ;)
User avatar
Chris Corbyn
Breakbeat Nuttzer
Posts: 13098
Joined: Wed Mar 24, 2004 7:57 am
Location: Melbourne, Australia

Post by Chris Corbyn »

Maugrim_The_Reaper wrote:SimpleTest tests itself with itself. If something goes wrong then either an unexpected error/exception will occur or the tests will simply fail. Seems weird but it is a workable solution. ;)
I think I'll go ahead and test it with itself. I can certainly test the Mock objects library in the Unit Tester since that's entirely separate anyway. It may lead to a few re-thinks in terms of factorisation which is always a good thing.

Cheers :)
Post Reply