The Xilinx ZynqMP SoC includes a powerful H.264/H.265 Video Codec Unit (VCU) which Xilinx supports with an out-of-tree driver and user space library. The allegro driver from mainline Linux allows to use the VCU using the standard V4L2 encoder API without requiring any custom user space.
I will explain why I wrote the driver, show how the VCU is used from a driver perspective, and present the current state of the driver and which features are still missing.
The Zynq UltraScale+ MPSoC integrates several ARM cores and an FPGA into a single chip. One variant of this SoC includes an H.264/H.265 Allegro DVT Video Codec Unit (VCU).
Although Xilinx did a good job supporting the ZynqMP in mainline Linux, the VCU was originally only supported via an out-of-tree driver. It consists of a user space library and a firmware blob, which interact via a custom IOCTL interface. Unfortunately, this has security implications as the user space handles physical buffer addresses.
Fortunately, the V4L2 mem2mem device API for video codecs avoids this problem by using abstract DMA buffers. Many applications already use these APIs, and various drivers for hardware codecs already implement these APIs. Thus, a V4L2 driver for the VCU allows reusing existing applications such as FFmpeg and GStreamer on the ZynqMP.
Speakers: Michael Tretter