Elevator Pitch
Is all your code meaningful, or is some redundant or unreachable? Are your tests strict, or do they let code get away with undesired behavior? Mutation Testing helps root out these problems. This talk explains what Mutation Testing is, how it works, how to use it, and what to do about the problems.
Description
Would your test suite still pass if the tested code was changed? If so, you probably have gaps in your test suite, or unreachable, redundant, or otherwise ineffective code – maybe even both!
Mutation Testing reveals these cases, and provides some guidance about how to fix them. This talk will tell you what mutation testing is, how it works inside, how to use it, where in your development process to use it, and its benefits, drawbacks, and history. There will be multiple examples, and a list of tools for many popular languages (and some others).
You will come away equipped with a powerful technique for making sure your test suite is strict and your code is meaningful!
Notes
I’ve picked Advanced level, because it’s an introduction to an advanced topic, but it might also be suitable for some intermediates. If you mean the level of their knowledge within the topic, then it’s more introductory.
I have given this talk many times before, including being voted third best talk out of over 70 at Software Quality Days 2024, and am slated to do more soon. You can see videos of some of them (in lengths from 22 to 68 minutes) in my playlist at https://www.youtube.com/playlist?list=PLMrm16n64Bub8urB-bsyMyHiNPMLG7FAS
I have the code already in Elixir, Python, Ruby, and JavaScript, and with decent lead time I could translate it into other languages. If you don’t have a preference, I’ll pick whichever is convenient for me at the time.
Agenda: - What is mutation testing? - Why isn’t test coverage enough? - What are its pros and cons? - How does it work (overview and details)? - Simple example (finding and fixing bad test) - Complex example (finding and fixing bad/missing test) - Complex example (finding and fixing bad/missing test, and redundant code) - FAQs as time allows, such as: history, why is it so CPU/RAM intensive, why change code in only one way per mutant, where to fit it into dev process, etc. - Wrapup - Q&A