conferences | speakers | series

Tapping Into the Binary Log Change Stream

home

Tapping Into the Binary Log Change Stream
FOSDEM 2019

Row-based binary log is mostly used as a logical replication log for MySQL. However, ever since row-based replication was introduced, it has also been widely used as an integration point between a MySQL server and other components in an IT infrastructure. It is often used as a capture-data-changes stream, as a source of data for extract-transform-load operations or even as an event notification trigger (e.g., propagating transaction information to proxy layers).

Commonly deployed setups revolve around collecting/subscribing to data changes and propagating these to downstream consumers through a message bus, like Kafka for instance.

This session will present such use cases, highlighting the additional metadata added to the binary log in the latest releases, explain how to efficiently make the most out of these and how to optimize the implementation of a change capture procedure. We will also showcase a couple of example plugins that tap into the server's binary log stream and export it even before it reaches the binary log files.

Speakers: Luis Soares