Shlomi Noach

Relational databases have invested in the performance of the relational model, but not as much in developer flows, creating an operational barrier driving developers away. We present an improved paradigm that brings back ownership into developers hands, illustrated by recent developments in Vitess.

The relational model is one of the oldest surviving models in computer science. But while relational databases have evolved to meet modern load, throughput and scalability needs, they have not evolved as much to meet developers' needs.

The schema, at the heart of the relational model, remain a major operational blocker in modern development flows. Developing and deploying schema changes is unlike any other development and deployment flow in practice today. Operational complexity and constraints, lack of conflict resolution, difficulty or inability to undeploy, and the need to understand database internals, all make relational schema development deter developers, who look for other solutions elsewhere.

In this session we will review these impediments and how they came to be, and offer a modern take, that gives developers back their ownership of their data and flows. Recent developments in Vitess, an open source CNCF project, introduce new capabilities that change the relational development paradigm. We will discuss:

• Development flow: hiding the operational complexity
• Deployments and scheduled schema migrations
• Undeploy: revertible changes
• Align schema and code: declarative schemas, idempotent deployments, and version control

This session reveals four experimental Vitess developments that automate away complex database operations. With these developments Vitess is able to run its own database infrastructure, transparently to the user, and take control of risky and elaborate situations and operations.

We will briefly explain the Vitess architecture and how it supports said control, and discuss the following developments:

• Throttling: pushback for massive writes.
• Table life cycle: safe and lazy DROP TABLE operations.
• Online DDL: automating, scheduling and managing online schema migrations.
• HA, failovers and cluster healing via vitess/orchestrator (aka vtorc).

Vitess is a CNCF open source database clustering system for horizontal scaling of MySQL.

Schema migration is more than running an ALTER TABLE. It is about designing, reviewing, approving, queuing, scheduling, executing, auditing, controlling and versioning the changes.

At GitHub we run multiple migrations per day, and much of this flow used to be manual, taking a significant toll from the databases team. In this session we illustrate how we automated away migration using free and open source solutions, and based on trusted development flow.

Schema migration is more than running an ALTER TABLE. It is about designing, reviewing, approving, queuing, scheduling, executing, auditing, controlling and versioning the changes.

At GitHub we run multiple migrations per day, and much of this flow used to be manual, taking a significant toll from the databases team. In this session we illustrate how we automated away migration using free and open source solutions, and based on trusted development flow.

We highlight the use of the skeema tool, with GitHub Actions, git flow and gh-ost.

The CAP theorem is often used to describe the tradeoffs of available versus consistent systems. But the CAP theorem is often misunderstood and misrepresented. In this session we investigate the CAP properties of various MySQL replication setups, and show that it is impossible to claim a setup is AP or that it is CP. We will see how the properties can be non binary, and change across time, geography and workload. We will also observe where CAP's terms can be made irrelevant. This session takes the practical engineer's perspective and shows how we can gain the upper hand with reasonable tradeoffs.

Un-split brain (aka Move Back in Time) MySQL. This brings to light some of our recent work on disaster mitigation at @github.

Please note that this talk replace Jean-François Gagné's talk entitled "The consequences of sync_binlog != 1" as he wasn't able to join us in Brussels.

Orchestrator operates Raft consensus as of version 3.x. This setup improves the high availability of both the orchestrator service itself as well as that of the managed topologies, and allows for easier operations.

This session will briefly introduce Raft, and elaborate on orchestrator's use of Raft: from leader election, through high availability, cross DC deployments and DC fencing mitigation, and lightweight deployments with SQLite.

Of course, nothing comes for free, and we will discuss considerations to using Raft: expected impact, eventual consistency and time-based assumptions.

orchestrator/raft is running in production at GitHub, Wix and other large and busy deployments.

GitHub's gh-ost is a MySQL online schema migration tool changing the operational paradigm. It is triggerless, controllable, auditable, and generates a low writeload on the master.

This session describes how gh-ost differs from existing online schema migration tools, and the many perks made available by its design.

gh-ost runs all of GitHub's migrations. It is released as open source and supported by GitHub at https://github.com/github/gh-ost

The nature of MySQL replication implies various crash scenarios of varying availability impact. Orchestrator is an open source project that discovers, manages and recovers your MySQL replication. Attend this talk to learn how Orchestrator detects failures with minimal false positives/negatives, and figures out the best method of recovery even in complex topologies.

The nature of MySQL replication implies various crash scenarios of varying availability impact. Orchestrator is an open source project that discovers, manages and recovers your MySQL replication. Attend this talk to learn how Orchestrator detects failures with minimal false positives/negatives, and figures out the best method of recovery even in complex topologies.

Pseudo GTID and easy replication management

This session introduces a technique called “Pseudo GTID” which allows easy refactoring of replication topologies, and makes balanced, deeply nested topologies achievable, safe and productive.

Managing large replication topologies introduces difficult questions: do you place all your slaves directly under the master? Do you setup intermediate masters thus creating deep replication trees? How do you synchronize sibling slaves when their master dies? How do you recover replication of a slave if its intermediate master is gone?

MySQL’s Global Transaction ID (GTID) comes to solve the above. However GTID comes with its own limitations and not always or not easily applicable.

Pseudo GTID is a non intrusive solution that utilizes standard replication, and turns your existing replication topology into a GTID-like topology, allowing you to repoint slaves however you like.

We will quickly discuss the problems incurred by standard binary log file & position setup, and the limitations of GTID, Pseudo-GTID injection. We will present the algorithm and implementation for utilizing Pseudo GTID to freely repoint slaves.