conferences | speakers | series

conda-forge, mamba, boa and quetz - the evolution of package management for data science and beyond

home

conda-forge, mamba, boa and quetz - the evolution of package management for data science and beyond
EuroSciPy 2022

Mamba is a fast, cross-platform and language independent package manager that is fully compatible with conda packages. It has enabled the conda-forge project to scale way beyond what was previously possible. In this talk we present further innovations in the mamba ecosystem, including boa, a new build tool based on mamba and quetz, an open-source and extensible package server for conda packages.

Mamba is widely used in the conda-forge community, a binary distribution of scienitific open-source packages for Windows, macOS and Linux. Today, mamba powers hundreds of automated builds for Python, C, C++ and many other programming languages every day. In this talk we will go into some of the details that make the conda-forge project and the mamba integration so successful. Furthermore, we will highlight some interesting new innovations coming to the mamba project. Thanks to a CZI grant we currently work on the sandboxing of environments on Linux and macOS, zchunk support for repodata, package metadata signing, OCI and S3 support and more. Additionally, we will take a look at a new build tool called boa. Boa comes with a new recipe format that is cleaner and faster to evaluate and makes building new packages a smooth process. It comes with an interactive mode to make packages easily debuggable. We will show how we have used boa to produce an experimental WASM distribution called "emscripten-forge" that builds WebAssembly enabled conda-packages that can readily be used in the web-browser with JupyterLite. Lastly, we will show recent improvements in quetz, the open-source conda package server. It is fully extensible, both in the front- and backend – something we want to use with the conda-forge project to add more features for users and package maintainers such as fast (metadata) package search.

Speakers: Wolf Vollprecht