Continuous Integration is a very important part in order to ensure a high quality software product. In order to be able to test the MariaDB server across multiple platforms, operating systems and configurations, the MariaDB Foundation has developed a continuous integration framework based on Buildbot. This talk will focus on describing the design decisions in order to be able to scale and test the MariaDB server on over 100 different configurations. We use a multi-master setup and combine Docker latent workers with LibVirt workers and classical Buildbot workers in order to achieve this.
In order to ensure that MariaDB runs smoothly, it needs to be tested on multiple platforms and configurations. In order to ensure this, the MariaDB Foundation has developed a new continuous integration framework, based on Buildbot, that thoroughly tests the server across multiple platforms, operating systems and configurations.
In order to achieve this, we use a mixture of builders, varying from Docker latent workers, to LibVirt workers but also the classical “always on” buildbot workers. For easier management, we have a separate master process that deals with each type of worker. In this talk, I will talk about our design decisions and some limitations that stand behind these. For example, the testing phase of the MariaDB server generates a very high amount of logs. Since Buildbot can deal with around 10k lines of logs per second, having multiple builds that run at the same time may saturate the master process. In order to deal with this, we use a master locking mechanism together with a multi-master setup.
Speakers: Vlad Bogolin