Consensus
Consensus is a protocol pattern that lets a group of replicas choose one ordered history of state changes despite crashes, message delays, and network partitions. Systems such as Raft do this by electing a leader, replicating a log, and committing entries only when a quorum has agreed.
The problem is not copying data, but deciding which copy is authoritative when machines disagree. A node can pause, messages can arrive late, and a partition can make each side believe the other has failed. If both sides accept writes, clients see histories that cannot both be true. Consensus prevents that by making progress require evidence from enough replicas that two conflicting decisions cannot both be made.
In Raft-style consensus, time is divided into terms. A node that wants to lead asks for votes, and becomes leader only if it receives a majority. The leader appends client operations to its log and sends those entries to followers. An entry is committed when it is stored on a majority, such as leader plus one follower in a 3-replica group. Later leaders must contain the committed prefix, so index 7 cannot change from x to y.
Quorums work because they overlap. If read quorum R and write quorum W satisfy R + W > N, every read quorum intersects every write quorum, so a read can encounter the latest committed write or information about it. With N=3, W=2 and R=2 is common. R=1 and W=1 is commonly misunderstood as replication with availability, but it permits stale reads because the read may contact a replica that missed the write.
The cost is availability under some failures. Consensus needs a reachable majority, so a 3-replica group can tolerate one crash but not the loss of two replicas or a partition that leaves no majority. Two replicas are a trap for high availability: majority is still 2, so if either node is unavailable, writes cannot safely commit. Consensus also adds latency, because commits wait for quorum acknowledgements, not merely local disk.
Engineers meet consensus inside systems that maintain shared control-plane or metadata state. Kubernetes relies on etcd, which uses Raft. CockroachDB runs Raft for ranges. Kafka KRaft uses quorum controllers for metadata. Consul uses Raft for its service catalogue and key-value state. In practice, you debug leader changes, quorum loss, slow followers, and the difference between a replicated log entry and a committed one.
Common questions
- Why are three replicas preferred over two?
- With two replicas, a majority is still both replicas, so the system cannot safely accept writes after either one is lost. With three replicas, a majority is 2, so one replica can fail while the remaining replicas still elect a leader and commit entries. The extra node buys fault tolerance, not extra copies for their own sake.
- Does consensus mean every replica has the same data immediately?
- No. Consensus guarantees agreement on what is committed, not instant identical storage everywhere. Followers may lag behind the leader. A log entry can be present on some machines without being committed. Once a majority has accepted it and the protocol’s rules mark it committed, later leaders must preserve it.
- What does R + W > N actually protect against?
- It ensures quorum overlap. Any successful read set and write set share at least one replica, so a read cannot be completely disjoint from the replicas that accepted the latest write. Without that overlap, a client may read from replicas that never saw the write and receive a stale value while the system still appears healthy.
- Can consensus keep a system available during any network partition?
- No. The honest answer is that it depends on where the majority is. The side of a partition that can form a quorum may continue electing a leader and committing writes. A minority side must stop accepting writes, otherwise the system risks split-brain and conflicting histories.