In any block-oriented, update-in-place database with log-based recovery, two kinds of page flushing are needed. Eviction flushing is necessary when all least recently used blocks are dirty (modifications exist in the buffer pool). Checkpoint flushing (writing out the oldest modified page first) allows the log checkpoint to advance, shortening the potential recovery time by logically discarding the start of the log. Any write of a persistent page must be preceded by a corresponding log write and optional doublewrite, to guarantee that crash recovery works.
We will cover some improvements in this area in MariaDB Server's version of InnoDB storage engine. We will also discuss how a database could be configured to minimize write amplification.
MariaDB 10.5 simplified page flushing and fixed bottlenecks that had originally been worked around by introducing multiple buffer pool instances and multi-threaded flushing. It turns out that a single buffer pool and page flushing thread (assisted by a log flushing thread) can still saturate most contemporary I/O subsystems. Data structures were simplified, some synchronous writes replaced with asynchronous ones, and mutex contention reduced through more use of atomic memory operations.
Speakers: Marko Mäkelä