As the number of production machine learning use-cases increase, we find ourselves facing new and bigger challenges where more is at stake. Because of this, it's critical to identify the key areas to focus our efforts, so we can ensure our machine learning pipelines are reliable and scalable. In this talk we dive into the state of production machine learning in the Python Ecosystem, and we will cover the concepts that make production machine learning so challenging, as well as some of the recommended tools available to tackle these challenges. This talk will cover key principles, patterns and frameworks around the open source frameworks powering single or multiple phases of the end-to-end ML lifecycle, incluing model training, deploying, monitoring, etc. We will be covering a high level overview of the production ML ecosystem and dive into best practices that have been abstracted from production use-cases of machine learning operations at scale, as well as how to leverage tools to that will allow us to deploy, explain, secure, monitor and scale production machine learning systems.

As the number of production machine learning use-cases increase, we find ourselves facing new and bigger challenges where more is at stake. Because of this, it's critical to identify the key areas to focus our efforts, so we can ensure our machine learning pipelines are reliable and scalable. In this talk we dive into the state of production machine learning in the Python Ecosystem, and we will cover the concepts that make production machine learning so challenging, as well as some of the recommended tools available to tackle these challenges. This talk will cover key principles, patterns and frameworks around the open source frameworks powering single or multiple phases of the end-to-end ML lifecycle, incluing model training, deploying, monitoring, etc. We will be covering a high level overview of the production ML ecosystem and dive into best practices that have been abstracted from production use-cases of machine learning operations at scale, as well as how to leverage tools to that will allow us to deploy, explain, secure, monitor and scale production machine learning systems. This talk will be relevant for any keen python practitioners or seasoned ML practitioners interested to get an updated overview of the state of the production ML ecosystem in the current year, covering a broad range of sub-fields in the space. This talk will benefit the Python ecosystem by providing cross-functional knowledge, bringing together best practices from data scientists, software engineers and DevOps engineers to tackle the challenge of machine learning at scale. During this talk we will shed light into some of the more popular and up-and-coming libraries to watch in this space, and we will provide a conceptual and practical hands on deep dive which will allow the community to both, tackle this issues and help further the discussion.

Identifying the right tools to enable for high performance machine learning may be overwhelming as the ecosystem continues to grow at break-neck speed. This becomes particularly emphasised when dealing with the ever growingly popular large language and image generation models such as GPT2, OTP and DALL-E, between others. In this session we will dive into a practical showcase where we will be productionising the large image generation model DALL-E, and showcase some optimizations that can be introduced as well as considerations as the use-cases scale. By the end of this session practitioners will be able to run their own DALL-E powered applications as well as integrate these with functionalities from other large language models like GPT2, etc. We will be leveraging key tools in the Python ecosystem to achieve this, including Pytorch, HuggingFace, FastAPI and MLServer.

Identifying the right tools to enable for high performance machine learning may be overwhelming as the ecosystem continues to grow at break-neck speed. This becomes particularly emphasised when dealing with the ever growingly popular large language and image generation models such as GPT2, OTP and DALL-E, between others. In this session we will dive into a practical showcase where we will be productionising the large image generation model DALL-E, and showcase some optimizations that can be introduced as well as considerations as the use-cases scale. By the end of this session practitioners will be able to run their own DALL-E powered applications as well as integrate these with functionalities from other large language models like GPT2, etc. We will be leveraging key tools in the Python ecosystem to achieve this, including Pytorch, HuggingFace, FastAPI and MLServer.

This talk will provide practical insights on high performance GPU computing in Python using the Vulkan Kompute framework. We will cover the trends in GPU processing, the architecture of Vulkan Kompute, we will implement a simple parallel multiplication example, and we will then dive into a machine learning example building a logistic regression model from scratch which will run in the GPU.

In more detail these are the topics of the talk:

• Motivations

• High level overview of the OSS Vulkan initative enabling cross-vendor GPU computing

• The Python Kompute Framework and its architecture which augments Vulkan

• A simple Python Kompute example implementing a parallel array multiplication

• An advanced Python Kompute example implementing a parallel array multiplication

• Further resources & further reading

A more in-depth version of this talk can be found in this blog post:

• https://towardsdatascience.com/beyond-cuda-gpu-accelerated-python-for-machine-learning-in-cross-vendor-graphics-cards-made-simple-6cc828a45cc3

If you are interested in the C++ internals, as well as further performance optimizations, you can join the deeper dive at the HPC & Data Science Room:

• https://fosdem.org/2021/schedule/event/gpu_vulkan/

Other useful links:

• Vulkan Kompute Repo: https://github.com/EthicalML/vulkan-kompute • Vulkan Kompute Docs: https://kompute.cc/

Other people: Matteo Bertucci

The lifecycle of a machine learning model only begins once it's in production. In this talk we provide a practical deep dive of the best practices, principles, patterns and techniques around production monitoring of machine learning models. We will cover standard microservice monitoring techniques applied into deployed machine learning models, as well as more advanced paradigms to monitor machine learning models through concept drift, outlier detector and explainability.

We'll dive into a hands on example, where we will train an image classification machine learning model from scratch, deploy it as a microservice in Kubernetes, and introduce advanced monitoring components as architectural patterns with hands on examples. These monitoring techniques will include AI Explainers, Outlier Detectors, Concept Drift detectors and Adversarial Detectors. We will also be understanding high level architectural patterns that abstract these complex and advanced monitoring techniques into infrastructural components that will enable for scale, introducing the standardised interfaces required for us to enable monitoring across hundreds or thousands of heterogeneous machine learning models.

Many advanced data processing paradigms fit incredibly well to the parallel-architecture that GPU computing offers, which has resulted in the continuously growing adoption of graphics card for general purpose computing. Exciting advancements in the open source Vulkan Project are enabling developers to take advantage of general purpose GPU computing capabilities in cross-vendor mobile and desktop GPUs including AMD, Qualcomm, NVIDIA & friends.

In this talk we will learn to write GPU accelerated algorithms which will be able to run on virtually any GPU hardware, including non-NVIDIA GPUs. We'll introduce an intuition and key concepts around GPU computing, as well as show how you can get started with the Vulkan Kompute framework with only a handful of lines of C++ or Python code.

Throughout the talk we will also dive into the GPU computing terminology around asynchronous & parallel workflow processing, cover the core principles of machine learning data parallelism, explain the hardware concepts of GPU queues & queueFamilies, and talk about how advancements in new and upcoming graphics cards will enable for even bigger speedups (such as the NVIDIA Ampere GA10x architecture which will support up to 3 parallel queue processing workloads).

In more detail these are the topics of the talk:

• GPU computing intuition, hardware and foundations

• Deeper dive into the OSS Vulkan C++ SDK enabling cross-vendor GPU computing

• The C++ Kompute Framework and its architecture which augments Vulkan

• Simple C++ Example with Kompute

• Deeper Optimizations (Batch Commands, Asynchronous and Parallel Workloads)

• FamilyQueues for Hardware-Parallel Workloads

• C++ Example for FamilyQueue Hardware-Parallel Workload

A more in-depth version of this talk can be found in this blog post:

• https://towardsdatascience.com/parallelizing-heavy-gpu-workloads-via-multi-queue-operations-50a38b15a1dc

If you are interested in the higher level use-cases, as well as machine learning examples, you can join the talk at the Python Room:

• https://fosdem.org/2021/schedule/event/python_cuda/

Other useful links:

• Vulkan Kompute Repo: https://github.com/EthicalML/vulkan-kompute • Vulkan Kompute Docs: https://kompute.cc/

Managing production machine learning systems at scale has uncovered new challenges that have required fundamentally different approaches to that of traditional software engineering and data science. In this talk, we'll provide key insights on MLOps, which often encompasses the concepts around monitoring, deployment, orchestration and continuous delivery for machine learning. We will be covering a hands on an example where we will be training, deploying and monitoring ML at scale. We'll be using Jenkins X (+ Prow & Tekton) to deploy/promote these models across multiple environments. We will use KIND (Kubernetes in Docker) to run integration tests in our development environment. Finally, we'll be using Seldon to orchestrate & monitor these models leveraging advanced ML techniques.

Managing production machine learning systems at scale has uncovered new challenges which have required fundamentally different approaches to that of traditional software engineering and data science. In this talk, we'll provide key insights on MLOps, which often encompasses the concepts around monitoring, deployment, orchestration and continuous delivery for machine learning. We will be covering a hands on an example where we will be training, deploying and monitoring ML at scale. We'll be using Jenkins X (+ Prow & Tekton) to deploy/promote these models across multiple environments. We will use KIND (Kubernetes in Docker) to run integration tests in our development environment. Finally, we'll be using Seldon to orchestrate & monitor these models leveraging advanced ML techniques.

Description

This talk will provide a practical overview of the state of production machine learning frameworks, tools and techniques that seem to have become a trend for the coming year. I will be mainly covering the open source tools and frameworks available in 2019 to help you expand your DataOps and MLOps infrastructure. This talk will cover the technologies available to support specifically around reproducibility, monitoring, compliance and orchestration of data and computations. I will present these technologies through the 8 principles of machine learning.

Talk Name

The state of machine learning operations in 2019: reproducibility, explainability, bias evaluation and beyond

Description

This talk will provide a practical overview of the state of production machine learning frameworks, tools and techniques that seem to have become a trend for the coming year. I will be mainly covering the open source tools and frameworks available in 2019 to help you expand your DataOps and MLOps infrastructure. This talk will cover the technologies available to support specifically around reproducibility, monitoring, compliance and orchestration of data and computations. I will present these technologies through the 8 principles of machine learning.

About me

Alejandro Saucedo is a technology leader with over 10 years of software development experience, he is currently the Chief Scientist at The Institute for Ethical AI & Machine Learning. Throughout his career, Alejandro has held technical leadership positins across hyper-growth scale-ups and tech giants including Eigen Technologies, Bloomberg LP and Hack Partners. Alejandro has a strong track record building multiple departments of machine learning engineers from scrath, and leading the delivery of numerous large-scale machine learnig systems across the financial, insurance, legal, transport, manufcturing and construcion sectors (in Europe, US and Latin America).