Kubernetes is a great solution for hosting highly available endpoint applications, but network functions can still prove challenging. VPN gateways are notably tricky, with overlay and underlay conflicts making their deployment an interesting dance. Performance is also a challenging topic when speaking about scale, most importantly when common optimizations like GSO don't really apply.
Leveraging the Calico CNI with a VPP dataplane mixed with multus, allowed us exposing multiple k8s managed interfaces to pods, and thus building complex network functions that still benefit from k8s constructs (HA, service discovery, ...). In the end building an highly available Wireguard gateway gets as easy as building any other application. And it can even leverage accelerated interfaces and cryptographic engines, to reach multiple Gbps without hassle.
Hoping this architecture could benefit Kubernetes at large, we started drafting a KEP, proposing to upstream the concept of multiple isolated networks, and standardizing their interaction with existing k8s resources.
Speakers: Nathan Skrzypczak