Ludovic Courtès

High-performance computing (HPC) is often seen as antithetical to “reproducibility”: one would have to choose between software that achieves high performance, and software that can be deployed in a reproducible fashion. This talk will discuss how GNU Guix lets users deploy software optimized for the target machines while preserving provenance tracking and reproducibility.

High-performance computing (HPC) is often seen as antithetical to “reproducibility”: one would have to choose between software that achieves high performance, and software that can be deployed in a reproducible fashion. This talk will discuss how GNU Guix lets users deploy software optimized for the target machines while preserving provenance tracking and reproducibility.

Why is HPC seen as antithetical to reproducibility in the first place? Maybe your cluster admins told you: if you want peak performance, you have to use the software stack that they themselves or the hardware vendors installed and tailored specifically to this cluster, and to recompile your code locally. Your software deployment becomes tied to this machine and hardly reproducible elsewhere.

However, by giving up on reproducibility, we would give up on verifiability, a foundation of the scientific process. How can we conciliate performance and reproducibility? Engineering work that has gone into performance portability has already proved fruitful, but some areas remain unaddressed when it comes to CPU tuning. This talk looks into package multi-versioning, a technique developed for GNU Guix, a tool for reproducible software deployment. We will show that it allows us to implement CPU tuning without compromising on reproducibility and provenance tracking.

You clone a Git repository, then pull from it. How can you tell its contents are “authentic”—i.e., coming from the “genuine” project you think you’re pulling from? With commit signatures and “verified” badges ✅ flourishing, you’d think this has long been solved—but nope! This is in essence the problem GNU Guix, as a software deployment tool and GNU/Linux distribution, had to solve as we will see in this talk.

A key element of supply chain security is updates: how can we make sure software updates are secure? That one doesn’t risk running malicious software when updating software their system? For free system distributions, The Update Framework (TUF) has become a reference on these matters. However, TUF is designed with binary distributions in mind—think Debian or even PyPI—and is not suite for “source distributions” like GNU Guix.

In this talk I will present how Guix distributes software packages and the mechanisms central to supply chain security in Guix: reproducible builds, builds from source (the “full-source bootstrap”), and provenance tracking. Software updates in Guix amount to ‘git pull’ so the security of updates translates to the ability to authenticate Git checkouts.

Believe it or not, this pretty fundamental problem was still in search of a solution. Guix developed a simple mechanism for Git authentication, which has been used in production for a couple of years. I will present it and, given that the solution is generic, show how it could benefit Git users alike. We’ll also reflect on how Guix’s approach compares to those developed by tools like slsa or in-toto.

GNU Guix is a software deployment tool that gives you what you declare. It provides abstractions to declare software packages, profiles containing several packages, operating systems and software services, and even fleets of machines.

Guix saw its 1.2.0 release in November 2020. In this talk I will present some of the latest features added to Guix or still cooking, showing how they all follow the same declarative pattern and how that eases the assembly of complex software pieces.

Jupyter has become a tool of choice for researchers willing to share a narrative and supporting code that their peers can re-run. This talk is about Jupyter’s Achille’s heel: software deployment. I will present Guix-Jupyter, which aims to make notebook self-contained and to support reproducible deployment.

Jupyter has become a tool of choice for researchers in data science and others fields. Jupyter Notebooks allow them to share a narrative and supporting code that their peers can re-run, which is why it is often considered a good tool for reproducible science.

However, Jupyter Notebooks do not describe their software dependencies, which significantly hinder reproducibility: What if your peer runs different Python version? What if your notebook depends on a library that your peer hasn’t installed? What will happen if you try to run your notebook in a few years?

All these issues are being addressed by tools such as Binder and its friend repo2docker. These solutions, though, do not address what we think is the core issue: that notebooks lack information about their software dependency.

In this talk I will present our take on this problem, Guix-Jupyter. Guix-Jupyter allows users to annotate their notebook with information about their run-time environment. Those annotations are interpreted and Guix takes care of deploying the dependencies described. Furthermore, Guix-Jupyter ensures that code runs in an isolated environment (a container) as a way to maximize reproducibility.

Guix-Jupyter is work-in-progress and we are eager to share our approach and get your feedback!

This talk will reflect on what GNU Guix has to offer to users and how it compares to other approaches—from CONDA and pip to Flatpak and Docker. Guix is not only a transactional package manager and declarative GNU/Linux distribution: it’s also an environment manager, a container provisioning tool, and more. We will describe these tools and our journey to 1.0, emphasizing key properties that set Guix apart: reproducibility, transparency, and hackability.

When it comes to software deployment, we are getting used to a new distribution of roles among many actors: traditional distros take care of core software pieces, “application bundles” à la Docker/Flatpak provide complex applications, Cabal, Gem, npm, pip, and friends take care of language-specific software, and Puppet/Ansible orchestrate the whole thing. Each of these tools has its justification, but the end result is a maze that’s hard to deal with.

In this talk I will present GNU Guix, a software deployment toolbox and its associated distro that feature transactional upgrades and rollbacks, declarative OS deployment inspired by Nix, and reproducible builds. I will show how Guix approaches a variety of use cases: “package management” à la apt-get, environment management à la VirtualEnv, Ansible-style declarative OS deployment, and container provisioning à la Docker.

Guix emphasizes programmability and one of its salient features is that it provides a unified Scheme programming environment to deal with with all the aspects of configuration, deployment, and system management—including service management by PID 1. I will illustrate how this works out on standalone Guix systems, and show the benefits of the approach by discussing applications that take advantage of Guix as a library to support reproducible software deployment.

Last, I will reflect on the road that led to Guix 1.0 six months ago and present some of the challenges ahead.

As GNU Guix reaches 1.0, this talk will reflect on what it has to offer to users and how it compares to other approaches—from CONDA and pip to Flatpak and Docker. Guix is not only a transactional package manager and declarative GNU/Linux distribution: it’s also an environment manager, a container provisioning tool, and more. We will describe these tools and our journey to 1.0, emphasizing key properties that set Guix apart: reproducibility, transparency, and hackability.

In a time where distros are sometimes viewed as “that thing you run inside a container”, where alternative approaches to software distribution are taking steam—from CONDA and pip to Flatpak and Docker—is there still room for the traditional approach to GNU/Linux distros?

As GNU Guix reaches 1.0 after six years of development, this talk will reflect on what Guix has to offer to users and how that compares to other approaches. You can view Guix as a package manager, but with transactional upgrades and roll-backs. Or you can view it as a standalone distribution like Debian but with a declarative interface à la Puppet. Guix is also an environment manager à la VirtualEnv but not limited to a single language, a container provisioning tool like—think ‘Dockerfile’ but with reproducibility and transparency built in, and a continuous integration/deployment tool—why write a lengthy ‘gitlab-ci.yml’ when you can reuse your package definition?

In this talk I will give an overview of all these tools Guix provides and discuss our journey from a “package manager” kind of project to a complete software deployment toolbox. I will present key properties that set Guix apart: transparency through reproducible builds and provenance tracking, and practical user freedom through a broad range of discoverable APIs that we hope invite users to hack. We believe Guix has shown that there’s still room for innovation in the distro space, and still value in distro-style software distribution.

GNU Guix is a package manager and GNU/Linux distribution that builds on Scheme, a Lisp dialect known for its emphasis on minimalism. This talk will be an exploration of Guix’ use of Scheme’s extensibility to build high-level abstractions and user interfaces in a broad sense.

GNU Guix is a package manager, GNU/Linux distribution, and more generally a “software deployment toolbox” featuring key features such as transactional upgrades and rollbacks, declarative operating system configuration, and more. Guix has become a relatively large piece of software but a key aspect of it is that Guix builds on Scheme, a Lisp dialect known for its emphasis on minimalism. From day one, use of Scheme as its single implementation language has shaped Guix.

This talk will be about what it means for Guix to be implemented as a set of libraries on top of this tiny core that Scheme is. Together we will have a guided tour through Guix’ interfaces, be they “programming” or “user” interfaces; we’ll look at its embedded domain-specific languages (EDSLs) that make it just as clear as your favorite YAML/JSON/XML thing while retaining the full power (and joy!) of a general-purpose programming language. We’ll discuss the features of Scheme that make it possible. I will share my thoughts on how such a design can empower users and how well this has worked in practice.

Package management, container provisioning, and workflow execution are often viewed as related but separate activities. This talk is about using Guix to integrate reproducible software deployment in scientific workflows.

In HPC, tools usually focus exclusively on one of these aspects: Spack or EasyBuild manage packages, Singularity or Shifter deal with containers, and SLURM, CWL, or Galaxy mostly leave it up to users to deploy their software.

While the initial tooling of GNU Guix is about package management, we have grown it into a toolkit that, broadly speaking, allows developers to integrate reproducible software deployment into their applications—as opposed to leaving it up to the user.

In this talk I will illustrate the benefits of this approach with examples from recent work from the Guix-HPC effort. This ranges from the “guix pack” container provisioning tool, to the Guix Workflow Language (GWL), which incorporates deployment as a key aspect of workflow management. I will discuss how we could make these tools key components of broader reproducible scientific workflows as demonstrated by projects such as ActivePapers, ReScience, or NextJournal.

A discussion on the future of GNU Guix and GuixSD.

What's a "system service"? How do we model that in GuixSD?

In most people’s mind, system services are a bunch of daemons that simply need to be started at boot time, or whenever they are actually needed. Possibly services form a dependency graph, and possibly they are actions other than spawning a daemon, such as mounting a file system.

As always, the devil is in the detail, and reality is that “system services” on a modern GNU/Linux system—with udev, dbus, polkit, along with more traditional Unix services—include lots of different “activities”, with lots of interactions among them. That naive picture of a graph of services no longer works.

This talk is going to tell the story of system services in GuixSD. GuixSD started from the naive visions of a “dependency graph of actions” to evolve into a generic model of service composition. I will describe what makes GuixSD’s service composition model unique, and how it helps users and sysadmins reason about the whole system.

Small languages update and discussion

HPC computing environment and especially super computing has its bag of challenges when it comes to software deployment. As scientists we often do not get root access which means that we either depend on what software is available or we build software in a dedicated directory using tools such as Brew, Conda or even from source. Unfortunately these solutions depend on already installed tools from an underlying distribution, often proprietary or dated compilers, and, for example, modules. Any binary that gets produced, therefore, tends to be totally unique, both in the generated binary and its set of dependencies. This is bad, both for trouble shooting and for pursuing reproducible science. With GNU Guix we are developing a fully reproducible free software stack including all dependencies targeting cross-builds for heterogeneous computing environments, such as the Beacon Intel PHI-based supercomputer, without having a requirement for containers. In this talk I will describe the challenges we are overcoming and what it will mean for the HPC community to have architecture optimized and fully reproducible build targets that can be deployed in a non-root, non-container environment.

“It used to work perfectly, then I upgraded something, and somehow…” Sounds like a déjà vu? Sometimes feel like software deployment is unpredictable? Ever wondered if you can trust your compiler or the integrity of those binary packages you have downloaded? Falling for VM/Docker images to address the shortcomings of “traditional distros”?

This talk introduces GNU Guix, a package manager that implements the functional package management paradigm pioneered by Nix to address these issues.

Guix supports transactional upgrades and rollbacks, as well as support for multiple software profiles. In this talk, I will introduce functional package management and demonstrate Guix on practical use cases. We will show that the approach provides better resource usage and transparency than "app bundles" as popularized by Docker. I will discuss the implications on (bit-)reproducible packages and environments, and how this can lead to verifiable binaries. Lastly, we will see how this extends to whole-system deployments with GuixSD, the Guix System Distribution.

Session with the GNU Guile and LUA communities

Session with the GNU Guile and LUA communities

GNU Guix and GuixSD provide a transactional package manager and purely declarative GNU/Linux distribution. Under the hood, Guix builds upon a unified set of Guile Scheme programming interfaces, making it "The Emacs of Distros." This talk will give an overview of those APIs and how they can be used.

The talk will cover things like store remote procedure calls (RPCs), multi-tier programming with G-expressions, the monadic store API, and more.

Introduction to the GNU Guile and GNU Guix projects

Introduction to the GNU Guile and GNU Guix projects

What's with Emacs? Its design makes user freedom more practical by putting all the mechanics at the user's fingertips. This talk will explore how GNU Guix and its distribution seek to empower users in a similar way.

The very design of Emacs empowers users: the whole documentation and source is at their fingertips, ready to be debugged or changed for immediate effect. Free software made practical!

What about following that Emacs meme for a complete distro? That's what the GNU Guix project has been trying to answer. The project has been developing a package manager and associated GNU/Linux distribution over the last two years. By using a general-purpose implementation language, Scheme, and by specializing it to offer high-level abstractions, Guix and the Guix System Distribution (GNU GSD) empower users by blurring the distinction between packagers and users, and by offering a uniform, hackable framework.

This talk gives an update of GNU Guix, and gives insight into how its design and implementation attempt to enhance software freedom.

We will see how users can fiddle with these foundations without having to cross barriers imposed by complex C systems and the historical baggage of a flurry of loosely integrated scripts and tools of varying languages. The talk will also demo features that set Guix apart from many other tools: supports transactional upgrades, rollback, per-user installations, reproducible operating system declarations, and more.

Guix is GNU's package manager and distribution. It seeks to empower users in several ways: by being a dependable system foundation, by providing the tools to formally correlate a binary package and the "recipes" and source code that led to it, by allowing users to customize the distribution, and by lowering the barrier to entry in distribution development.

This talk will describe the features and foundations of GNU Guix as a package manager. It will report on the current status of building a stand-alone GNU distribution, and outline design goals that set it apart from existing distros.

Guix is GNU's package manager and distribution. Its goal is to be a foundation for the GNU system that is dependable, hackable, and liberating.

Guix is a dependable package manager in that it supports transactional upgrades and roll-backs---valuable features pioneered by the Nix package manager it is based upon. The underlying functional package management paradigm, where packages are built in isolated containers, maximizes chances that builds are bit-reproducible---allowing users to compare independent builds of a given package, rather than relying on a single distribution point for binaries.

Packages are declared in a high-level fashion, using a domain-specific language embedded in the Scheme programming language. This is the first step in making it hackable to our eyes: packagers do not even need to know Scheme to write a package definition, yet the full power of Scheme, of GNU Guile, and of the Geiser programming environment is available. From a programming viewpoint, Guix and the GNU distribution are just a bunch of "normal" Guile modules, some of which export "package" objects---one can easily write Guile code that builds atop the distribution, customizes it, or otherwise fiddles with packages.

This feature set serves one goal: to put users in control. We are putting together an operating system made of software that respects the user's freedom---GNU's goal as stated 30 years ago. By allowing users to know exactly what dependencies and build scripts led to a given binary, the functional package management paradigm increases the control users have over the software they're running. By offering a consistent view, tool set, and APIs, we hope to make it easier for users to exert their freedom to adapt the software to their needs.

This talk will present these features, the underlying foundations, and our motivations. We'll report on the project's status after more than a year of development, and discuss what's cooking in the GNU system.