Swati Sehgal

Optimizing CPU management improves cluster performance and security, but is daunting to almost everyone. CPU management may seem complex, but it can be explained in such a way that even your inner toddler will comprehend. With this talk, we will give a path to success.

You may have a multi-socket node cluster where your AI/ML workloads care about the proximity of your CPUs to GPUs. You may be running scientific workloads where you want to pin in cores within containers instead of just a pod level. You may have a single-socket server where you want to save a single core outside of Kubernetes for a daemon dedicated to mining bitcoin, without affecting your other jobs (please do not do this). We will cover these and more, helping you understand the intricacies of CPU management within the kubelet and what Kuberenetes can and cannot currently do. We will also cover how you can help escalate the visibility of use cases not currently covered within Kubernetes.

Many clusters do not have CPU management because it is difficult to do correctly without impacting performance. While there are static and dynamic pinning abilities, we will consolidate the use cases and remove the pain it takes for users to deploy Kubernetes. This talk will help the audience gain an insight into features that can be used for resource management and orchestration of containerized applications with focus on CPU management. Users that have performance sensitive workloads such as AI/ML, Telco, 5G and Networking workloads will benefit from this talk. This talk will also help CTOs, system architects, developers and engineers in their planning to develop, test, or optimize their deployments in a Kubernetes environment. We also hope to use this forum to find other community members with specific needs in our desire to make the kubelet more flexible with regard to CPU management.

With Kubernetes gaining popularity for performance-critical workloads such as 5G, Edge, IoT, Telco, and AI/ML, it is becoming increasingly important to meet stringent networking and resource management requirements of these use cases. Performance-critical workloads like these require topology information in order to use co-located CPU cores and devices. Despite the success of Topology Manager, aligning topology of requested resources, the current native scheduler does not select a node based on it. It's time to solve this problem! We will introduce the audience to hardware topology, the current state of Topology Manager, gaps in the current scheduling process, and prior out-of-tree solutions. We'll explain the workarounds available right now: custom schedulers, creating scheduling extensions, using node selectors, or manually assigning resources semi-automatically. All these methods have their drawbacks. Finally, we will explain how we plan to improve the native scheduler to work with Topology Manager. Attendees will learn both current workarounds, and the future of topology aware scheduling in Kubernetes.

Kubernetes has taken the world by storm attracting unconventional workloads such as HPC Edge, IoT, Telco and Comm service providers, 5G, AI/ML and NFV solutions to it. This talk would benefit users, engineers, and cluster admins deploying performance sensitive workloads on k8s. Addition of newer nodes running alongside older ones in data centers results in hardware heterogeneity. Motivated by saving physical space in the data centers, newer nodes are packed with more CPUs, enhanced hardware capabilities. Exposing to use fine grain topology information for optimised workload placement would help service providers and VNF vendors too. We’ll explain numerous challenges encountered in efficiently deploying workloads due to inability to understand the hardware topology of the underlying bare metal infrastructure and scheduling based on it. Scheduler’s lack of knowledge of resource topology can lead to unpredictable application performance, in general under-performance, and in the worst case, complete mismatch of resource requests and kubelet policies, scheduling a pod where it is destined to fail, potentially entering a failure loop. Exposing cluster level topology to the scheduler empowers it to make intelligent NUMA aware placement decisions optimizing cluster wide performance of workloads. This would benefit Telco User Group in kubernetes, kubernetes and the overall CNCF ecosystem enabling improved application performance without impacting user experience.