http://code.google.com/p/phpmutagen/
SVN: http://phpmutagen.googlecode.com/svn/trunk/
To get over the explanation bit, imagine the following code:
Code: Select all
class Math {
public function add($number1, $number2) {
return $number1 + $number2;
}
}
class DescribeMathAdding extends PHPSpec {
public function itShouldAddTwoNumbers() {
$math = new Math;
$total = $math->add(1,1);
$this->spec($total)->should->equal(2);
}
}The second is Mutation Testing. In MT the idea is to make changes to source code (i.e. mutations) to check if the assumed broken code is detected by the test suite. If the tests do not fail - then it means that regardless of Code Coverage, we have a potential bug/code block undetectable by the test suite. So in essence, Mutation Testing tests your unit tests and specs to check if they the level of detection is sufficient.
How does it work? Taking the code above, the Mutation Testing engine would replace the original Math class with a mutated version. For example, replacing the "+" operator with "-". Presumably an add() method which now subtracts should be detected by your testing suite
Cutting to the chase, you want to see output like:
Code: Select all
.......
7 Mutants born out of the mutagenic slime!
7 Mutants exterminated!
No Mutants survived! Muahahahaha!Code: Select all
....M..
7 Mutants born out of the mutagenic slime!
6 Mutants exterminated!
1 Mutant escaped; the integrity of your suite may be compromised by the following Mutants:
1)
Index: /opt/mutesting/src/PHPMock/Expectation.php
===================================================================
@@ -147,9 +147,9 @@
public function verifyCall(array $args)
{
$this->_validateOrder();
- $this->_actualCallCount++;
+ $this->_actualCallCount--;
if (!is_null($this->_exceptionToThrow)) {
if (is_array($this->_exceptionToThrow)) {
$class = $this->_exceptionToThrow[0];
$message = $this->_exceptionToThrow[1];
Happy Hunting! Remember that some Mutants may just be Ghosts (or if you want to be boring, false positives).The theory of PHPMutagen is:
1. Identify source code
2. Run the local test suite to ensure it's clean (i.e. in a passing state)
3. Generate a list of possible mutations to source code files
4. Apply each mutation in sequence - re-run tests after each
5. Assemble mutations which did not cause the local test suite to fail.
6. Report the escaped Mutants for developer review (so can go Dalek on them
PHPMutagen currently has a dependency on PEAR::Text_Diff to generate unified diffs of the escaped Mutants so you see how the mutation changed the source code file. I omitted the usual /newline notice for the diff (not making patches with it are you
It is currently only supporting the impressive total of 2 mutations - plus operators and increment operators (+ and ++). It's pretty easy to add more - just add new PHPMutagen_Mutation subclasses and add the PHP token or string to mutate to the switch statements in PHPMutagen_MutableFile.
I'll be looking for code review later once I brush out a few final issues this evening. Anyone think this would be useful?