Page 1 of 1

Inlined unit tests

Posted: Fri Nov 10, 2006 10:48 pm
by alex.barylski
I wonder if that technique is used in complied languages like C...

What I mean is...instead of writing external tests, you actually write the tests as members of an object (stripped in production code) this way, you could also test private and protected members as well :P

Your objects ctor and dtor act as the setup and teardown functions...and this way...you avoid the worst part of writing unit tests...keeping your tests synchronized with the actual API...when I design...and first implement everything is extremely volatile...I change like lighting...keeping Agile so to speak ;)

When the function signature or name, etc changes which again in my case is frequent...my unit tests have to change which is a PITA when your API changes as frequently as mine. This has one of two negative side effects.

1. Influences you to accept poorer quality API in order to reduce synchronizing tests with them
2. Influences you to give up on unit tests altogather because maintenance is a PITA

Now I know some are advocates of unit testing because it forces you to think about your design first...which admittedly it does...but it doesn't actually *force* you to do anything...just it helps to put some fore thought into an API before cranking out code, writing tests first then implementing...

The fact is...even with this fore thought my API still has a tendancy to change and keeping the windows open to rapidly keep my API/Tests synchronized is a double PITA.

Like writing API docs inline with the actual funciton makes life easier, so to would having your tests inlined as desciribed above...

Ignoring that in PHP it would add bloat - what is your opinion on this matter...can you see any down sides???

Cheers :)

Posted: Fri Nov 10, 2006 10:53 pm
by feyd
It would seem that you need more thought and less action.

Posted: Fri Nov 10, 2006 10:58 pm
by alex.barylski
feyd wrote:It would seem that you need more thought and less action.
Very elaborate of you ;) I have dumped considerable thought into the whole idea...it's why I'm coming here for others input... :)

Whats so wrong about it? MFC classes use a similar technique although not part of a framework perse...they simply dump messages to your output IDE window...

Edit: It would be nice to see little green lights go BOO!!! but as the old adage goes...in MFC's case...No news is good news hinting at the fact that although no little green lights appear...no messages in output would be almost equally as satisfying and both do the same job...

Unless enxternal tests have other benefits...but I can't see reusability being one of them... :?

Posted: Fri Nov 10, 2006 11:35 pm
by alex.barylski
Just to be fare, here are some faults with the inline approach:

1. You would only get test results of objects invoked during application execution...external tests test every single interface always so if something breaks your aware of it immediately not 2 months down the road...

2. Tight coupling which is both a positive and negative.
+ Easier to maintain (very important)
- Loss of modularity (maybe important) in team development when one guy writes tests another implemention

Others?

Posted: Sat Nov 11, 2006 1:09 pm
by Maugrim_The_Reaper
It seems to me you need to research unit testing before you go any further.
What I mean is...instead of writing external tests, you actually write the tests as members of an object (stripped in production code) this way, you could also test private and protected members as well
Unit Testing deliberately focuses on public methods and members. The reason is quite simple - you are not testing code, you are testing behaviour. You have a class which you expect to perform a specific task in a specific way using a specific API. Unit Tests allows you to test these concepts to ensure the behaviour you want is present - otherwise a test will fail signalling something is wrong.

Now since private and protected methods are only accessible within the parent class (and optionally a descendant) by testing the public methods you are also testing the private methods - the publics use the privates to give rise to the final behaviour. This is one reason why unit testing immediately puts a focus on refactoring - the better you're capable of refactoring, the better you can decompose public behaviour into focused public methods and private methods which are easily identified as being responsible for failed tests. For many test infected folk, a testable class is a good class (it means the class has followed at least some OOP best practices).

Does that make sense?

If you test individual private members you are simply duplicating test coverage and making far more work for yourself. Tests do not need to cover every minute implementation detail (who cares if you use a giant public method, or two dozen privates? The tests don't). That path will force you to update tests for every little refactoring. Unit tests shouldn't need updating. If you change the behaviour of classes, that's your doing - the unit tests are completely innocent of blame. If a bug still manages to creep in, guess what? Not the unit tests fault - *you* missed part of the behaviour when writing tests - see Regression Testing for this one.
Your objects ctor and dtor act as the setup and teardown functions...and this way...you avoid the worst part of writing unit tests...keeping your tests synchronized with the actual API...when I design...and first implement everything is extremely volatile...I change like lighting...keeping Agile so to speak
I came across this concept in the Zend Framework (a lot of developers have never practised TDD and it shows in spades) and its been infuriating at times at how poor some unit tests can be (minimal coverage, no exception tests, little realisation that unit tests are valid documentation). Unit Tests as a testing tool is the core purpose for their existence but there is so much more to unit testing. If you follow Agile practices, than Unit Tests are *never* written after the code - you always write the Unit Test *first*, then you write code to pass that test. Essentially, you never ever synchronise tests to your API - you actually synchronise your API to your newly updated tests ;). The practice is Test Driven Design. By writing the tests first you ensure classes are testable - i.e. you are forced to refactor early and often, and reduce coupling to a minimum from the start. Arguably (I am strongly pro TDD but not everyone is ;)) this saves time in the long run, but costs more time at the start writing tests.

Admittedly I have a serious test infection...;). When I started with the ZF 0.20 the first place I went to was the tests directory. Why read the source code when the API is laid out in detail as part of the unit tests??? ;)

A few other points.

Unit Tests do not *force* good practice, they strongly encourage them assuming you can recognise the signals (code smells) and know how to refactor. TDD is not a standalone skill - without refactoring it will ultimately fail as a testing strategy. An example is using Singletons in constructors. If you do not use a Registry, or do not pass Singletons as parameters, then testing the class will prove difficult. This often requires finnicky workarounds like extending test classes, or adding conditional constructor arguments. The extra test work is not the unit testing being difficult, it's the developer not recognising the design problems with their supposedly independent decoupled class.

Same goes for using statics in classes - it makes testing difficult without pulling in the static's parent class (which may or may not be unit tested separately).

Unit Tests require little forethought. You plan the class, sketch a preliminary interface, write the unit tests which capture the planned behaviour. Write the code, then assess for refactoring possibilities. It's a simple, incremental process - once you accept the upfront time spent on writing tests and keeping your code passing green.

Since it's Saturday and I'm incredibly tired (late night) forgive any duplication of comments...:)

Posted: Sat Nov 11, 2006 1:57 pm
by timvw
Hockey wrote:
feyd wrote:It would seem that you need more thought and less action.
Very elaborate of you ;) I have dumped considerable thought into the whole idea...it's why I'm coming here for others input... :)
I believe feyd ment to say that you should think better over what you're going to program... That way you can reduce the need to constantly refactor and keep both implementation and verification in sync....

Overhere you can't check your code in unless all the tests run... So if you refactor a class you're forced to update the tests too... In order to circumvent smartparts that think that they don't have to write tests (so nothing can fail either) we have another policy that requires that there is complete code coverage...

(Yes, it is an extreme approach.. But given that we're working with fairly disciplined developers anyway it doesn't require that much extra effort in order to comply with those policies... (And in emergencies we can always lower the policy rules))

Posted: Sat Nov 11, 2006 4:29 pm
by alex.barylski
Now since private and protected methods are only accessible within the parent class (and optionally a descendant) by testing the public methods you are also testing the private methods - the publics use the privates to give rise to the final behaviour.
Yes I realize that, but I wonder if that was intentional or by consequence. Testing the publics methods will no dought test the privates, etc, but if a public interface fails...and it's because of a private, you would have to manually determine that public method DoThis() failed because of a faulty DoThat() private method...

Seems to me, it would make sense to test app functions...not just publics.
If you test individual private members you are simply duplicating test coverage and making far more work for yourself.
Ahhh...ok...you got me there...touche :P I can't argue about that ;)

But in all fareness my argument wasn't *just* that you could test privates but rather the ease of having inlined unit tests over external tests...???

I hadn't considered using the unit tests as a form of documentation up until now...so that is an argument in favour of keeping your tests external...but thats what auto-document generators are for :P

Cheers :)

Posted: Sat Nov 11, 2006 4:37 pm
by alex.barylski
timvw wrote:I believe feyd ment to say that you should think better over what you're going to program
Personally I plan just enough to understand the problem not the solution.

Cheers :)