Code coverage is a great metric - how much of your code is actually tested by your unit tests. However, this doesn’t tell you how good your tests actually are at picking up changes to your codebase - if your tests aren’t well thought-out, it’s possible for changes to get past your unit tests but break production. Mutation testing is a great (and massively underrated) way to understand how valuable your tests are. Mutation score quantifies how much trust you should put in your tests to stop mutant versions of your code escaping from your lab and wreaking havoc in production. In this talk, I’ll show you examples of how to get started with mutation testing and how to integrate it into your CI/CD pipeline. After the session, you’ll be ready to use mutation testing with wild abandon and you'll be ready to integrate it into your release engineering process, testing your mutant code without needing to set foot in the lab at all!
Developers often use metrics as targets. Code coverage is one of the most common, and most prone to corner-cutting and abuse. Goodhart's Law states: When a metric becomes a target, it ceases to be a good metric (and trust me, I have a lot of memes to share with you about this!) Mutation score is different. Because it tests your tests, coverage isn't enough - you have to write tests that are resistant to all the mutated code that wants nothing more than to sneak through your CI/CD pipeline into production.
Speakers: Max Kahan