Simpletest woes
Moderator: General Moderators
Re: Simpletest woes
Output buffering? I dont see any syntax errors in the code you posted either
- allspiritseve
- DevNet Resident
- Posts: 1174
- Joined: Thu Mar 06, 2008 8:23 am
- Location: Ann Arbor, MI (USA)
Re: Simpletest woes
The class under test is missing the first bracket {...
Edit: the problem was due to @include in my __autoload() function.
Edit: the problem was due to @include in my __autoload() function.
Last edited by allspiritseve on Tue Jan 20, 2009 9:53 am, edited 1 time in total.
Re: Simpletest woes
Hi..
No, SimpleTest cannot catch certain classes of errors from PHP. Script dead and SimpleTest reduced to collateral damage. If you have the standard error reporting turned off, you get the mystery behaviour above. The favourite one is running out of memory when running in the browser. SimpleTest just appears to stop when you run it's own self test suite.
I started working on the ability to run test cases in their own process, but stopped part way through because I didn't have enough requests for that feature. Got as far as the XML communication, and the DefaultReporter responding to the --xml flag. The last steps is the ability to shell out the test case on to the command line, and to respond properly to an entire test case going AWOL.
Frankly I've wanted this feature for ages and this conversation is a good excuse to finish it. I want to get the next beta release out first though.
yours, Marcus
No, SimpleTest cannot catch certain classes of errors from PHP. Script dead and SimpleTest reduced to collateral damage. If you have the standard error reporting turned off, you get the mystery behaviour above. The favourite one is running out of memory when running in the browser. SimpleTest just appears to stop when you run it's own self test suite.
I started working on the ability to run test cases in their own process, but stopped part way through because I didn't have enough requests for that feature. Got as far as the XML communication, and the DefaultReporter responding to the --xml flag. The last steps is the ability to shell out the test case on to the command line, and to respond properly to an entire test case going AWOL.
Frankly I've wanted this feature for ages and this conversation is a good excuse to finish it. I want to get the next beta release out first though.
yours, Marcus
- Chris Corbyn
- Breakbeat Nuttzer
- Posts: 13098
- Joined: Wed Mar 24, 2004 7:57 am
- Location: Melbourne, Australia
Re: Simpletest woes
Marcus, if you added that feature to simpletest I'd love that. SitePoint have created their own test suite to do this using AJAX, I've created my own test suite to do it with AJAX, or with command line processes and I've definitely seen others do it. It seems that the fact people are going ahead and building their own test suites with this functionality is reason enough to make it a part of the framework 
Reasons for doing it?
1) Scope: If each test case runs in it's own process you're ensuring a clean environment for each test.
2) Memory: Trying to run 3000 test cases in one process is basically not possible on the standard memory limit.
Gotchas:
If you test data uses multiple character sets (say you're testing i18n stuff) then the generated XML will have character set mismatches and will not be parseable. <![CDATA[ .. ]]> sections do not help because they expect the character character set.
If your tests make assertions regarding binary data, the XML generated is invalid and not parseable by any XML interpreter. We need some way of including binary data in XML. <![CDATA[ .. ]]> sections don't work for some byte values (the NUL byte for one).
The way I got around this way to add assertIdenticalBinary() to my test cases, where pack() is used and then hex strings are compared instead.
Reasons for doing it?
1) Scope: If each test case runs in it's own process you're ensuring a clean environment for each test.
2) Memory: Trying to run 3000 test cases in one process is basically not possible on the standard memory limit.
Gotchas:
If you test data uses multiple character sets (say you're testing i18n stuff) then the generated XML will have character set mismatches and will not be parseable. <![CDATA[ .. ]]> sections do not help because they expect the character character set.
If your tests make assertions regarding binary data, the XML generated is invalid and not parseable by any XML interpreter. We need some way of including binary data in XML. <![CDATA[ .. ]]> sections don't work for some byte values (the NUL byte for one).
The way I got around this way to add assertIdenticalBinary() to my test cases, where pack() is used and then hex strings are compared instead.
Re: Simpletest woes
I dont think ajax is needed, my feature request would be to simply flush the type casting error ( or any non-fatal errors ) to my browser as soon as it (they) occur(s), in case subsequent errors are "fatal" in nature. Honestly I could probably care less about ajax, not to say it wouldn't be "cool" but it doesn't solve any real needs I have currently with the library. I know you cant catch the fatal errors, but I'm saying if there's no fatal errors I get the type casting error caught by simpletest, my request would be to simply patch the html reporter so it was impossible for it to catch errors but have the script error out before they were outputted. Just my 2 cents, but hey if you say ajax is a better way to solve it then do that
It would be neat if there were just some way to get a "better" error output when you pass NULL to a type casted function like in the example I posted
I appreciate the otherwise wonderful testing framework, very KISS / elegant
I appreciate the otherwise wonderful testing framework, very KISS / elegant
Re: Simpletest woes
http://michaelkimsal.com/blog/php5-type-hinting/
I was naive to think its a buffering issue ( realized its not.. ). But check this out.
Edit: nvm I guess this doesn't really help either.. jesus why can't they make it just throw an exception. I'm pulling my hair out here there has to be a way
I was naive to think its a buffering issue ( realized its not.. ). But check this out.
Edit: nvm I guess this doesn't really help either.. jesus why can't they make it just throw an exception. I'm pulling my hair out here there has to be a way
Re: Simpletest woes
Alright I figured out a workable fix. I implemented a base class, overrode _call and thew an exception about undefined methods. Granted if an object adhere's to an interface this still allows things to seep thru the cracks but thats easier to live with then these darned fatal errors 
Edit: holy crap do my eyes deceive me, I'm getting the type casting error thrown now from simpletest:
Unexpected PHP error [Argument 1 passed to Ne8_Db_Mapper::save() must be an instance of Ne8_Model, instance of Ne8_Collection given, called in C:\Users\Josh\Desktop\ne8\metator\library\Ne8\Db\Mapper.php on line 634 and defined] severity [E_RECOVERABLE_ERROR] in [C:\Users\Josh\Desktop\ne8\metator\library\Ne8\Db\Mapper.php line 394]
WOOP WOOP WOOP
I was lookin into register_shutdown_function and how it exists in a magic "stack" (grrr ) and the same thing w/ fruitless output buffering callbacks. Saw your thread at sitepoint too.
Edit: holy crap do my eyes deceive me, I'm getting the type casting error thrown now from simpletest:
Unexpected PHP error [Argument 1 passed to Ne8_Db_Mapper::save() must be an instance of Ne8_Model, instance of Ne8_Collection given, called in C:\Users\Josh\Desktop\ne8\metator\library\Ne8\Db\Mapper.php on line 634 and defined] severity [E_RECOVERABLE_ERROR] in [C:\Users\Josh\Desktop\ne8\metator\library\Ne8\Db\Mapper.php line 394]
WOOP WOOP WOOP
I was lookin into register_shutdown_function and how it exists in a magic "stack" (grrr ) and the same thing w/ fruitless output buffering callbacks. Saw your thread at sitepoint too.
Re: Simpletest woes
Sorry to flood the boards... I'm thinking if I switch to linux xdebug will start working ( known issues with xamp / wamp ) and I'll get stack traces without having to make a base class, which means even if I pass NULL values I can still find out who the caller is
Re: Simpletest woes
Hi all.
Just to let you know I'm still following this thread, but I'm working on the SimpleTest docs at the moment ready for a beta release. I'll try to sneak the testing in a separate process into the beta after that.
I'll also backport it to the 1.0.* branch as I think this would be a cool way of testing mixed PHP4 and PHP5 code bases. Or even multiple versions of PHP so that I could use it for compatibility testing for Simpletest itself.
yours, Marcus
Just to let you know I'm still following this thread, but I'm working on the SimpleTest docs at the moment ready for a beta release. I'll try to sneak the testing in a separate process into the beta after that.
I'll also backport it to the 1.0.* branch as I think this would be a cool way of testing mixed PHP4 and PHP5 code bases. Or even multiple versions of PHP so that I could use it for compatibility testing for Simpletest itself.
yours, Marcus