Multi-leader replication

With one leader, if you cannot reach it, you cannot write.

Multi-leader basics

In multi-leader (active/active) replication, several nodes take writes and send them to each other. Each leader is also a follower of the others. If they replicated synchronously it would act like one leader, so think of it as async. Inside one region it is rarely worth the trouble. It is often an add-on, with traps around auto-increment keys, triggers, and constraints.

Multi-region trade-offs

A region is one geographic place with several datacenters, called availability zones. Zones have their own power and fast links to each other, so a system can span them easily. Spreading over zones survives a zone outage, but surviving a whole region going down needs several regions, which costs latency and money. With a leader in each region:

• Writes are local, not a trip across the world.
• If a region goes down, the others keep going and it catches up later.
• A broken link between regions does not stop writes.
• But two leaders can each accept a write that is fine alone and wrong together, like two people taking the same username. If you need such rules, use one leader; many apps do not.

Replication topologies

A topology is the paths writes take between leaders: a circle, a star, or all-to-all. In a circle or star, nodes pass on others' writes, so each write is tagged with the IDs of nodes it passed, and a node skips writes carrying its own ID. One dead node in a circle or star can cut others off. All-to-all has many paths, so it copes better.

Out-of-order writes

On all-to-all, links differ in speed, so an update can arrive before the insert it changes. Clock timestamps cannot fix this, because clocks drift. Version vectors can, but many systems do not use them, so test yours.

Sync engines and local-first

A calendar on a phone is a leader that syncs when online, maybe days later. Each tab in a live shared editor is a replica too. A sync engine keeps data on the device and syncs in the background, so the screen updates at once and offline is just a long delay. Local-first apps also keep working if the maker shuts down its servers. This fits when a user's data is small enough to download.

Remember

• Use multi-leader for regions or offline, not by default.
• It cannot enforce unique values.
• Clocks cannot order writes.