Containers are gaining significant traction as a means of deploying and distributing software in the scientific computing space. With a plethora of solutions to choose from, this talk will go into why there is a tendency for custom container runtimes in this area of computing. We will discuss portability, performance, security and ease of use, as well as direct hardware access as driving factors. As new processor architectures and specialized hardware for machine learning emerge, some of those factors will affect larger parts of the container community.
This talk will give an overview of (some of) the currently available container solutions and how they enable scientists to move their software stacks between machines and work in a reproducible way. The technical challenges and implementations of interfacing with the host system for access to hardware and highly optimized system libraries will be presented. Experiences of running containers on thousands of nodes at NERSC will be shared, which includes the good and the bad parts. The drawbacks of running portable but unoptimized code inside of containers and how that can have a profound impact on the performance of production workloads and what a solution for that could look like, will be addressed. Wrapping up I'd like to show how some of the work done in the scientific space can benefit the broader community and facilitate discussion about that.
Speakers: Georg Rath