Max Kahan

Developers often use code coverage as a target, which makes it a bad measure of test quality. Mutation testing changes the game: create mutant versions of your code that break your tests, and you'll quickly start to write better tests! Come and learn to use it as part of your CI/CD process. I promise, you'll never look at penguins the same way again!

Code coverage (the percentage of your code tested by your tests) is a great metric. However, coverage doesn’t tell you how good your tests are at picking up changes to your codebase - if your tests aren’t well-designed, changes can pass your unit tests but break production. Mutation testing is a great (and massively underrated) way to quantify how much you can trust your tests. Mutation tests work by changing your code in subtle ways, then applying your unit tests to these new, "mutant" versions of your code. If your tests fail, great! If they pass… that’s a change that might cause a bug in production. In this talk, I’ll show you how to get started with mutation testing and how to integrate it into your CI/CD pipeline. After the session, you’ll be ready to use mutation testing with wild abandon. Soon, catching mutant code will be a routine part of your release engineering process, and you’ll never look at penguins the same way again!

Code coverage is a great metric - how much of your code is actually tested by your unit tests. However, this doesn’t tell you how good your tests actually are at picking up changes to your codebase - if your tests aren’t well thought-out, it’s possible for changes to get past your unit tests but break production. Mutation testing is a great (and massively underrated) way to understand how valuable your tests are. Mutation score quantifies how much trust you should put in your tests to stop mutant versions of your code escaping from your lab and wreaking havoc in production. In this talk, I’ll show you examples of how to get started with mutation testing and how to integrate it into your CI/CD pipeline. After the session, you’ll be ready to use mutation testing with wild abandon and you'll be ready to integrate it into your release engineering process, testing your mutant code without needing to set foot in the lab at all!

Developers often use metrics as targets. Code coverage is one of the most common, and most prone to corner-cutting and abuse. Goodhart's Law states: When a metric becomes a target, it ceases to be a good metric (and trust me, I have a lot of memes to share with you about this!) Mutation score is different. Because it tests your tests, coverage isn't enough - you have to write tests that are resistant to all the mutated code that wants nothing more than to sneak through your CI/CD pipeline into production.

Code coverage (the percentage of your code tested by your tests) is a great metric. However, coverage doesn’t tell you how good your tests are at picking up changes to your codebase - if your tests aren’t well-designed, changes can pass your unit tests but break production.

Mutation testing is a great (and massively underrated) way to quantify how much you can trust your tests. Mutation tests work by changing your code in subtle ways, then applying your unit tests to these new, "mutant" versions of your code. If your tests fail, great! If they pass… that’s a change that might cause a bug in production.

In this talk, I’ll show you how to get started with mutation testing and how to integrate it into your CI/CD pipeline. After the session, you’ll be ready to use mutation testing with wild abandon. Soon, catching mutant code will be a routine part of your release engineering process!

Developers often use metrics as targets. Code coverage is one of the most common, and most prone to corner-cutting and abuse. Goodhart's Law states: When a metric becomes a target, it ceases to be a good metric (and trust us, we have a lot of memes to share with you about this!)

Mutation score is different. Because it tests your tests, coverage isn't enough - you have to write tests that are resistant to all the mutated code that wants nothing more than to sneak through your CI/CD pipeline into production.

Other people: Paco van Beckhoven

pip install malware: it’s that easy. Almost all projects depend on external packages, but did you know how easy it can be to install something nasty instead of the dependency you want?

You might remember classic typosquatting examples like goggle.com, but it’s now common to see malicious code hidden in spoofed or otherwise fraudulent PyPI packages or nested dependencies. Malware developers can also use techniques like starjacking to appear legitimate, so these unpleasant packages become even more difficult to spot. It’s estimated that over 3% of packages on PyPI could be using this technique.

By the end of this talk, you’ll know how to protect yourself when installing and updating dependencies and you’ll leave with a checklist to follow to help you stay safe in future.