For decades developers try to build a bug free software. Numerous techniques and metrics were developed. However, there is still a need in an approach that can both assert program domain and provide some reliable metrics. And such approach exists. It is called Mutation Testing. Here is a brief overview:
There is a function and a test for that function. Any semantic change in the function (so called 'mutant') should break the test ('kill mutant'). If that's not the case, then mutant is considered as survived.
No mutants should survive ever. If they are, then either the test is bad or the function is not correct.
Research in this area exists since 70's. However, this approach is not widely adopted. The computational cost is one of the blockers. Another one, is the way such systems are implemented: most of them do mutations at the AST level, hence they are tightly coupled to the particular languages.
We want to present you Mull: our implementation of Mutation Testing system. We use JIT and runtime compilation to speedup the system and make it accessible for everyday usage. Besides that we go down to the IR level, making the system applicable for any LLVM-based language such as C, C++, Rust, Objective-C, and many others.
- https://github.com/mull-project/mull
Abstract
For decades developers try to build a bug free software. Numerous techniques and metrics were developed. However, there is still a need in an approach that can both assert program domain and provide some reliable metrics. And such approach exists. It is called Mutation Testing. Here is a brief overview:
There is a function and a test for that function. Any semantic change in the function (so called 'mutant') should break the test ('kill mutant'). If that's not the case, then mutant is considered as survived.
No mutants should survive ever. If they are, then either the test is bad or the function is not correct.
Research in this area exists since 70's. However, this approach is not widely adopted. The computational cost is one of the blockers. Another one, is the way such systems are implemented: most of them do mutations at the AST level, hence they are tightly coupled to the particular languages.
We want to present you Mull*: our implementation of Mutation Testing system. We use JIT and runtime compilation to speedup the system and make it accessible for everyday usage. Besides that we go down to the IR level, making the system applicable for any LLVM-based language such as C, C++, Rust, Objective-C, and many others.
- https://github.com/mull-project/mull
Brief Outline
- General introduction into Mutation Testing
- Overview of our Mutation Testing system
- High-level overview of the system design
- Results and findings from LLVM's test suite
- Future work
- Q&A
