Mutation testing is a technique in which the software under test is modified in a controlled manner to produce a mutant. Then test cases are executed against each mutant. This helps answer the question "How good is our test suite?". For example if we have made the following change:
- if A and B:
+ if A or B:
and our test suite reports a PASS result that means we are not doing a good job at detecting possible errors.
I have been using mutation testing for production grade software in both Python and Ruby and I'm also the most active contributor to Cosmic Ray, the mutation testing tool for Python. In this talk I will explain how mutation testing works and what it can be used for. I will give practical examples of code which wasn't tested and how to test it and also examples of bugs that I've found. I will also mention some differences between the Python and Ruby tools.
Speakers: Alexander Todorov