In this talk, we highlight how important automated testing is to sustain the upstream development model of Linux, especially when products are involved. We will then give you a tour of our system, how it integrates with the developer's workflow, what we are working on, and we managed to grow from a couple of thousands tests executed per week to over 4 millions!
We will also present the changes we have done to the IGT test suite to become less Intel-specific and serve the needs of multiple drivers.
Upstream development requires never regressing the features that were already present in previous versions. We believe that this is not only the right thing to do, it is also increasing in relevance as more and more products move towards this model. To embrace this model, we try to catch unintentional regressions as early as possible through our CI system. This improves the productivity of our developers (fewer bugs coming months after the code was committed), and provide users with a smoother upgrade path (either through the product manufacturer or by upgrading the kernel themselves).
The Intel GFX CI has grown massively in the past 1.5 years: the number of machine doubled; the test coverage went from 260 tests/machine up to over 4k; the number of tests executed went from 100k/week to over 4M; pre-merge testing time dropping to an average of 30 minutes despite the increased usage.
We will then tour the audience through our different services, how they integrate with the developers' workflow, and how we manage to keep track of failures (and file them).
Finally, we will share what are our current developments, our goals, and what we are doing to the IGT test suite in order make it useful for more drivers than Intel.
Speakers: Martin Peres Arek Hiler