I will show how to use BPFtrace to trace both function calls in your programs and kernel functions. Then, I will show how such a low-level tracing tool can be used in high-level environments like Kubernetes clusters with kubectl-trace.
kubectl-trace (https://github.com/fntlnz/kubectl-trace) schedules bpftrace programs on your kubernetes cluster using the kubectl.
BPFtrace (https://github.com/iovisor/bpftrace) is a new tracing tool announced in October 2018. You can write one-liner commands in your shell using a high-level tracing language inspired by awk and C, and predecessor tracers such as DTrace and SystemTap. Behind the scenes, it compiles this high-level language with LLVM into eBPF bytecode (extended Berkeley Packet Filter) available in recent Linux kernels. It allows you to trace different layers of the system: kernel dynamic tracing (kprobes), user-level dynamic tracing (uprobes), and tracepoints.
It can be used to trace containers, for example running on Kubernetes, the most popular container orchestration tool. BPFtrace is used as a base tool by kubectl-trace to build more complex tracing tools on Kubernetes.
Speakers: Alban Crequy