Robert McLay

XALT is a tool run on clusters to find out what programs and libraries are run. XALT uses the environment variable LD_PRELOAD to attach a shared library to execute code before and after main(). This means that the XALT shared library is a developer on every program run under linux. This shared library is part of every program run. This talk will discuss the various lessons about routine names and memory usage. Adding XALT to track container usage presents new issues because of what shared libraries are available in the container.

XALT is an open source project under the LGPL licence. It is available at https://github.com/xalt/xalt and the documentation can be found at https://xalt.readthedocs.io. A presentation can be found at: https://www.youtube.com/watch?v=6QSKaAkbiWo?

Let's talk real, no-kiddin' supercomputer analytics, aimed at moving beyond monitoring the machine as a whole or even its individual hardware components. We're interested in drilling down to the level of individual tasks, users, and binaries. We’re after ready answers to the "what, where, how, when and why" that stakeholders are clamoring for: everything from which libraries (or individual functions!) are in demand, to preventing the problems that get in the way of successful science. This talk will show how XALT can provide this type of job-level insight.

XALT can provide a wide range of metrics and measures of job-level activity. There are benefits to both users and stakeholders: sponsoring institutions interested in strategic priorities; organizations concerned about meeting users' needs; and those seeking to study user activity to improve value and effectiveness.

We will show how this tool provides high value to centers and their users as it can provide documentation on how an application was built to provide reproducibility by reporting the exact environment in which jobs were run.

We have been running XALT at Texas Advanced Computing Center, one of the largest supercomputers in the world and it has become mission critical for us to know what programs to benchmark for new systems. It has also told us what programs that shouldn't be running on the large memory nodes. We will also describe using analytics on the big data generated through XALT.

XALT has a small but growing community. It is also tracking usage at major sites around the world: NICS, NCSA, Univ. of Utah, KAUST.

On today's supercomputers, chemists, biologists, physicists and engineers are sharing the same system and they all need different software. Environment Modules have been the way since the '90 that users select the software they need. They allow users to load and unload the packages they want. They get to control which version of the software they use, rather than the system administrators. Lmod, implemented in Lua, is a modern replacement for the venerable TCL/C based tool. Lmod offers many features to handle the vastly more dynamic software environment than the original tool was designed to handle.

Lmod is a modern, flexible, robust environment module system: the system tool that "sets the table" by helping supercomputer users load the software packages they need to do their work. They get to control which version of the software they use, rather than the system administrators. It is a Lua-based drop-in replacement for the venerable TCL/C environment module system. Because needs vary across the diverse computational research community, module systems like Lmod are mission-critical, ubiquitous, and expected on such systems: users need help selecting the correct combination of tools from among the thousands available on modern, large-scale systems.

This is not a talk about Lmod's features or design; instead, the focus is on process, community, and culture. What is it like to develop, maintain, and support a tool for a community that is often unaware that it (or Lua) even exists? How does one go about diagnosing and correcting problems on systems to which you cannot get access?

But the talk also addresses an important Lua-related issue. A module system loads a given software package by interpreting a 'modulefile' that describes the details of the environment that software package needs. Lmod's native modulefiles are Lua programs; Lmod parses legacy TCL modules by translating their instructions into Lua. This approach has been very successful, but has not been without its challenges: this talk includes an introduction to some of the associated technical issues and edge cases.

This has been an interesting ride, from a local tool deployed at a single site in Texas, to an open source mainstay that is mission critical at hundreds of supercomputer centers around the world. World domination through Lua and Lmod, one center at a time!