Code coverage (the percentage of your code tested by your tests) is a great metric. However, coverage doesn’t tell you how good your tests are at picking up changes to your codebase - if your tests aren’t well-designed, changes can pass your unit tests but break production.
Mutation testing is a great (and massively underrated) way to quantify how much you can trust your tests. Mutation tests work by changing your code in subtle ways, then applying your unit tests to these new, "mutant" versions of your code. If your tests fail, great! If they pass… that’s a change that might cause a bug in production.
In this talk, I’ll show you how to get started with mutation testing and how to integrate it into your CI/CD pipeline. After the session, you’ll be ready to use mutation testing with wild abandon. Soon, catching mutant code will be a routine part of your release engineering process!
Developers often use metrics as targets. Code coverage is one of the most common, and most prone to corner-cutting and abuse. Goodhart's Law states: When a metric becomes a target, it ceases to be a good metric (and trust us, we have a lot of memes to share with you about this!)
Mutation score is different. Because it tests your tests, coverage isn't enough - you have to write tests that are resistant to all the mutated code that wants nothing more than to sneak through your CI/CD pipeline into production.
