HPC administration tasks require admins to run identical commands across their clusters efficiently and frequently. Cluster tools develop again and again their own set of commands to perform similar tasks. Admins often develop their own scripts trying to implement fast execution, not always successfully. ClusterShell proposes to address these problems by offering a new set of command-line tools and a Python framework, both relying on the same optimized code and features. It took the best of famous commands, like pdsh, and improved it. The library can be used to ease admin script development and remove the burden of implementing optimized parallelism. ClusterShell supports multiple execution backends like SSH or RSH variants. Tools from other projects already rely on ClusterShell for their efficient command execution like MilkCheck or Shine. Moreover, ClusterShell offers a powerful way to manage range of nodes which could be used in any tools using its Python API.
This presentation is intended to any cluster or server farm administrator, with minimal administration experience, but also Python tool developers.
More details about nodeset features and hierarchical architecture from Linux Symposium 2012 paper: https://www.kernel.org/doc/ols/2012/ols2012-thiell.pdf
Aurelien is HPC system admin and developer on large-scale supercomputers from CEA for 10 years. Highly involved in parallel filesystems and cluster management. He is involved in ClusterShell design and code since its inception in 2008.
Speakers: Aurélien Degrémont