Testing Command Line Beautifier

Discussion of testing theory and practice, including methodologies (such as TDD, BDD, DDD, Agile, XP) and software - anything to do with testing goes here. (Formerly "The Testing Side of Development")

Moderator: General Moderators

Post Reply
User avatar
Ambush Commander
DevNet Master
Posts: 3698
Joined: Mon Oct 25, 2004 9:29 pm
Location: New Jersey, US

Testing Command Line Beautifier

Post by Ambush Commander »

I'm developing a Command Line program primarily because as progress goes on, you can keep track of how far you've gone. This is especially useful for programs that have an inherently long execution time.

So... in my quest for complete object seperation, I have seperated my output beautifier and muter into it's own class in which it can handle the problems of output.

The primary method is nothing fancy, it's a simple function that applies certain context sensitive formatting to text passed to it, for instance, you can indent text, and the outputter will automatically wrap long text for you.

That being said, here's the unit test:

Code: Select all

class SimpleTest_EZY_CommandLine extends UnitTestCase {
    
    var $obj;
    
    function setUp() {
        $this->obj =& new EZY_CommandLine;
        $this->obj->quiet = true;
    }
    
    function tearDown() {
        unset($this->obj);
    }
    
    function testOutput() {
        
        $backup =& $this->obj;
        $backup->quiet = false;
        
        ob_start();
            
            $backup->output('RESET', CL_OUTPUT_RESET);
            ob_clean();
            
            $backup->output('Bang', CL_OUTPUT_NONE);
            $result[0] = ob_get_contents();
            ob_clean();
            
            $backup->output('Bang', CL_OUTPUT_SUB);
            $result[1] = ob_get_contents();
            ob_clean();
            
            $backup->output('Bang', CL_OUTPUT_NONE);
            $result[2] = ob_get_contents();
            ob_clean();
            
            $backup->output('Bang', CL_OUTPUT_SUB);
            $result[3] = ob_get_contents();
            ob_clean();
            
            $backup->output('Boom', CL_OUTPUT_SUP);
            $result[4] = ob_get_contents();
            ob_clean();
            
            $backup->output('Bishie!', CL_OUTPUT_SUP);
            $result[5] = ob_get_contents();
            ob_clean();
            
            $backup->output('Bishie!', CL_OUTPUT_RESET);
            $result[6] = ob_get_contents();
            ob_clean();
            
            $backup->output('Too Far...', CL_OUTPUT_SUP);
            $result[7] = ob_get_contents();
            ob_clean();
            
            $backup->output('1', CL_OUTPUT_SUB);
            $backup->output('2', CL_OUTPUT_SUB);
            $backup->output('3', CL_OUTPUT_SUB);
            $backup->output('4', CL_OUTPUT_SUB);
            $result[8] = ob_get_contents();
            ob_clean();
            
            $backup->output('Bishie!', CL_OUTPUT_RESET);
            $result[9] = ob_get_contents();
            ob_clean();
            
        ob_end_clean();
        
        $this->assertEqual($result[0], "Bang\n");
        $this->assertEqual($result[1], "  Bang\n");
        $this->assertEqual($result[2], "  Bang\n");
        $this->assertEqual($result[3], "    Bang\n");
        $this->assertEqual($result[4], "  Boom\n");
        $this->assertEqual($result[5], "Bishie!\n");
        $this->assertEqual($result[6], "Bishie!\n");
        $this->assertEqual($result[7], "Too Far...\n");
        $this->assertEqual($result[8], "  1\n    2\n      3\n        4\n");
        $this->assertEqual($result[9], "Bishie!\n");
        
        ob_start();
            
            $backup->output('RESET', CL_OUTPUT_RESET);
            ob_clean();
            
            $backup->output('This is a title', CL_OUTPUT_TITLE);
            $result[10] = ob_get_contents();
            $expect[10] = str_repeat('=',75)."\nThis is a title\n".str_repeat('=',75)."\n";
            ob_clean();
            
            $backup->output('This is a subtitle', CL_OUTPUT_SUBTITLE);
            $result[11] = ob_get_contents();
            $expect[11] = str_repeat('-',75)."\nThis is a subtitle\n";
            ob_clean();
            
            $backup->output('Indented!', CL_OUTPUT_SUBTITLE | CL_OUTPUT_SUB);
            $result[12] = ob_get_contents();
            $expect[12] = "  ".str_repeat('-',73)."\n  Indented!\n";
            ob_clean();
            
        ob_end_clean();
        
        $this->assertEqual($result[10],$expect[10]);
        $this->assertEqual($result[11],$expect[11]);
        $this->assertEqual($result[12],$expect[12]);
        
        ob_start();
            
            $backup->output('RESET', CL_OUTPUT_RESET);
            ob_clean();
            
            $backup->output('Waiting...', CL_OUTPUT_SUSPENDNEWLINE);
            $backup->output('OK.');
            $result[13] = ob_get_contents();
            $expect[13] = "Waiting... OK.\n";
            ob_clean();
            
            $backup->output('Waiting...', CL_OUTPUT_SUSPENDNEWLINE && CL_OUTPUT_SUB);
            $backup->output('OK.');
            $result[14] = ob_get_contents();
            $expect[14] = "  Waiting... OK.\n";
            ob_clean();
            
        ob_end_clean();
        
        $this->assertEqual($result[13],$result[13]);
        $this->assertEqual($result[14],$result[14]);
        
    }
    
}
This test case smells, and that probably is a good thing from a design standpoint. The amounts of duplication and the gratuitous use of output buffering is astounding. Furthermore, the proliferate assertions are dependent on certain cross-output variables (such as the indent), and they are extremely intolerant to changes in formatting.

That being said, I have done some thinking about the SimpleTest reading, and I've come to this conclusion: The logic and the presentation must be seperated!

I think that what should happen is that one function should handle the context information, setting certain details based on the arguments passed to it. Then, it passes a less easy to use set of parameters to another presentation component which then echoes out the data. Using this method, I'll be able to mock the presentation method and then do assertions on what should be passed to it.

How do I unit test the presentation method (if I decide to change the way I present things, it would break my test cases, and regexps don't work because the output must be correct to a fine grain precision)? Possibly... you can't unit test it, instead, you create a standard "routine" and have it output everything and then you examine it and make sure everything works (sort of like those printer troubleshooter papers).

Other problems: how do I test calls that influence each other? Possibly... Package them in there own test*() methods and don't rely on CL_OUTPUT_RESET to reset everything. If one thing fails, cascade the error for the entire test function, but don't let it influence anything else.

The new way of testing the logic, by asserting certain types of parameters must be passed, also smells. Aren't I "peeking into the trunk" of how it works? It seems like I have to define an API for this presentation function, but then why not call it directly? Possibly... I'm barking up the wrong tree, and in fact, the maintenance of tabbing across outputs is for the logic function, but the actual tabbing of a message belongs to the presentation.

I've tried to answer my own questions, so if it seems like I'm going in the right direction, just say so. If I'm not, I'll try to figure out the right way without asking you to devote too much of your time trying to teach me.

What I really need is a mentor. For now, McGruff will have to do. ;)
User avatar
nielsene
DevNet Resident
Posts: 1834
Joined: Fri Aug 16, 2002 8:57 am
Location: Watertown, MA

Post by nielsene »

If it were me, I would at least start by splitting that up into many little test functions. Each function should test more or less one thing. So each of your arrays items should be pulled out to their own test.

Why are you using $backup, why not use use $this->obj?

I'ld probably move the ob_start into the setUp as well.
User avatar
Ambush Commander
DevNet Master
Posts: 3698
Joined: Mon Oct 25, 2004 9:29 pm
Location: New Jersey, US

Post by Ambush Commander »

nielsene wrote:If it were me, I would at least start by splitting that up into many little test functions. Each function should test more or less one thing. So each of your arrays items should be pulled out to their own test.
Good idea.
Why are you using $backup, why not use use $this->obj?
Err... a relic from cut and paste.
I'ld probably move the ob_start into the setUp as well.
Then the assertions wouldn't get painted. :?
User avatar
nielsene
DevNet Resident
Posts: 1834
Joined: Fri Aug 16, 2002 8:57 am
Location: Watertown, MA

Post by nielsene »

Ambush Commander wrote: Then the assertions wouldn't get painted. :?
Really? I don't use output buffering, but that seems wrong. Why would it matter if you call it in the setUp or in the test?
User avatar
Ambush Commander
DevNet Master
Posts: 3698
Joined: Mon Oct 25, 2004 9:29 pm
Location: New Jersey, US

Post by Ambush Commander »

nielsene wrote:Really? I don't use output buffering, but that seems wrong. Why would it matter if you call it in the setUp or in the test?
Well, I'd have to switch it off whenever I want to do an assertion. Some tests won't need output buffering too. So I think the test would be the best place to call it.

The comparisons, of course, would be pretty brittle though.
McGruff
DevNet Master
Posts: 2893
Joined: Thu Jan 30, 2003 8:26 pm
Location: Glasgow, Scotland

Re: Testing Command Line Beautifier

Post by McGruff »

Ambush Commander wrote:I've come to this conclusion: The logic and the presentation must be seperated!
Well-spotted. You'll go far :) Divide and conquer is what OOP is all about. Basically what you're doing is testing if output() formats the string correctly. If output() returns a string something else can print it and you can simply test the string contents. You'd only need one or two assertions: one with string "foo" and maybe one with an empty string since that's a special case. It might be handled differently. Or, if it's handled the same way, you might want to check that the code does work with an empty string.

From the command line you might be printing to STDOUT or STDERR so it makes sense to split off the formatting from the output target.
Ambush Commander wrote:How do I unit test the presentation method (if I decide to change the way I present things
I've never tested command line output before. Off the top of my head, I'd maybe run the script from within a test case send STDOUT to one file and STDERR to another, then make assertions on the file contents. Repeat for each use case.
Ambush Commander wrote:Other problems: how do I test calls that influence each other?


Test methods should certainly be fine-grained. If something fails you want the test to tell you exactly what went wrong so you can zero in on the problem quickly. You might test the primary class's methods one by one, possibly with several test methods for each. Or tests might focus instead on "use cases" for the class, making assertions for several primary class methods in each test case method. It all depends.
What I really need is a mentor.
Glad to help. Actually I also learn a lot by thinking about other people's testing problems. Don't take anything I say as gospel - just opinion. You'll also get some good advice on sitepoint.com - quite possibly from the author of SimpleTest & some of the other developers. There's also a SimpleTest mailing list.
Post Reply