Testing applications intended to run on high-performance computing clusters can be a painful process, especially when you do not have access to the resources you need. Sometimes you just want to check if you have the right compile flag specified, if the script you wrote actually works, or if your application can successfully use multiple cores. One way to test your HPC software is by emulation; instead of wasting precious compute time on your bare-metal cluster, you use a micro-HPC cluster that provides the same functionality as the bare-metal cluster on a smaller scale.
This talk will discuss the concept and implementation of an open-source testing framework that builds upon micro-HPC clusters. We will explore how a micro-HPC cluster can be brought up using Python and deploy core software such as identity management, shared file systems, and environment modules. We will also be exploring the transferability of the testing framework, such as how we can ensure that tests can both be run in CI pipelines and on local development machines, or how can use pre-existing frameworks such as pytest or unittest to run the tests. Lastly, we will discuss limitations of the testing framework and future work that can be done to improve its capabilities.
Speakers: Jason Nucciarone