Why Three Nodes?
Why three nodes keep appearing in production architectures — and why replication alone is not enough
If you have worked with distributed systems long enough, you have probably seen the same number appear again and again:
Three etcd members. Three MongoDB replica-set members. Three Kafka controllers. Three RabbitMQ quorum queue members. At some point, it starts to look like an industry convention.
It is not.
Behind that recurring number is one of the most important ideas in distributed systems: quorum.
How many independent participants must agree before the system is allowed to consider something authoritative?
That “something” might be the identity of a leader, the durability of a write, a replicated log entry, a metadata update, or even a change to cluster membership. Understanding that distinction is what separates having replicas from actually designing a fault-tolerant distributed system.
Why Three Nodes?
For a majority-based system, quorum is usually calculated as:
Q = floor(N / 2) + 1
where N is the number of voting members.
This immediately explains something interesting. A three-node cluster can tolerate one unavailable member. A four-node cluster can also tolerate only one.
So adding a fourth voter often gives you more infrastructure cost without increasing majority failure tolerance. To tolerate two failures while continuing to make decisions, you need five voters.
This is why odd-sized consensus groups appear so often.
But the real idea behind quorum is not simply “50% + 1.” It is intersection.
Imagine five nodes:
Members = {A, B, C, D, E}
Quorum 1 = {A, B, C}
Quorum 2 = {C, D, E}
Intersection = {C}
Two independently formed majorities cannot be completely separate from one another. That overlap creates a bridge between decisions. This principle is fundamental to protocols such as Paxos and Raft.
Quorum Is Not Consensus
This distinction is one of the easiest things to miss.
Quorum is a mechanism. Raft and Paxos are consensus protocols.
Quorum tells us how many participants are required for a decision. Consensus additionally defines rules governing things such as:
· who may become leader,
· which history is considered valid,
· how proposals are ordered,
· when a log entry becomes committed,
. and how information survives future elections.
Majority intersection alone is not enough. The participants must also follow protocol rules that prevent conflicting histories from becoming authoritative.
Suppose you replicated a record to two out of three servers. That sounds safe. But what happens during the next leader election? Which server is allowed to become leader? What if the candidate is missing that record? Can another partition create a competing version of history?
Without additional rules, “two copies exist” does not automatically mean the system is consistent.
Liveness Does Not Mean Authority
This leads to one of the most useful mental models for operating distributed systems:
A healthy node is not necessarily an authoritative node.
Imagine a three-node Raft cluster and a network partition:
A + B | C
A + B -> quorum -> progress allowed
C -> minority -> authoritative progress denied
Node C might still have healthy CPU, available memory, working disk, network connectivity to clients, and a running process. From a traditional monitoring perspective, it may appear perfectly healthy.
But C has only one vote. The other side has two.
Raft candidates need a majority to become leader, and committed log history is protected through majority replication together with Raft’s log-safety rules.
The minority stops not because the system failed to design for availability. It stops because continuing independently could create two different truths.
Sometimes refusing to write is the safety feature.
The Dangerous Question: “Do We Have Quorum?”
There is a better question:
Quorum of what?
Different systems use the word quorum for very different things. That matters enormously in production.
RabbitMQ: Quorum of Queue Members
RabbitMQ Quorum Queues use a Raft-based replication model. In a three-member queue, two members form the majority required for authoritative progress.
This means the queue can normally tolerate one unavailable member. Moving to five members increases failure tolerance to two, but also increases replication and coordination cost.
The important point is simple: the quorum belongs to the queue members that participate in replication and decision-making.
MongoDB: Votes Are Not Data
MongoDB highlights an important distinction between voting and durability. An arbiter can participate in elections without storing application data. So an additional vote does not necessarily mean an additional durable copy. For that reason, production replica sets typically favor data-bearing voting members, and w: "majority" is concerned with acknowledgement from the required data-bearing majority.
Get Bahadır Berk Bozdağ’s stories in your inbox
Join Medium for free to get updates from this writer.
A vote is not the same thing as a copy of your data.
Cassandra: A Different Kind of Quorum
Cassandra uses quorum differently from leader-based consensus systems. For replication factor N, let W be the number of replicas required for a write and R the number consulted for a read.
R + W > N
With N = 3, a common example is W = 2 and R = 2. The read and write sets must therefore overlap. But overlap alone does not make the entire database linearizable. Cassandra’s ordinary consistency model and its Paxos-based Lightweight Transactions provide different guarantees.
So:
Quorum intersection is not automatically strong consistency.
Kafka Has Two Different Quorum Stories
Kafka is a good example of why the word quorum needs context. KRaft controllers form a genuine quorum for cluster metadata. Topic data follows a different path. Partition durability depends on the leader, the in-sync replica set, producer acknowledgements, replication factor, and min.insync.replicas.
A common durability-oriented configuration is:
RF = 3min.insync.replicas = 2acks = all
The distinction matters:
KRaft quorum protects metadata. ISR rules protect partition data.
Calling both simply “Kafka quorum” hides an important operational difference.
Redis Shows Why Replication Is Not Consensus
Redis provides a useful counterexample to the assumption that several acknowledgements automatically mean consensus. WAIT can delay a client until a write has reached a requested number of replicas. That improves durability probability, but it does not transform Redis into a strongly consistent consensus system.
A safer description is:
Redis WAIT asks replicas to acknowledge a write; it does not create Raft-style quorum semantics.
Replication and consensus are related, but they are not the same thing.
Quorum Is Not Free
Agreement has a cost.
If several replicas must participate before an operation can complete, network latency, storage latency, and coordination become part of the request path.
PACELC captures this trade-off well:
During a partition: Availability or Consistency.
Otherwise: Latency or Consistency.
Quorum therefore converts part of your infrastructure topology into application latency.
That is not necessarily bad. It is the price paid for stronger agreement.
Operating Quorum in Production
The hardest quorum failures usually come from topology and operations rather than from misunderstanding the formula.
Three voters in the same availability zone are still exposed to a single shared failure. A stronger three-member design spreads those voters across independent failure domains.
Monitoring should also focus on the distributed system rather than individual processes. Useful questions include whether a majority is reachable, whether leadership is stable, whether replicas are falling behind, and whether durable commits are slowing down.
Membership changes deserve the same care. Adding or removing voters changes the system’s ability to form a majority, so the safest operational rule is:
Change one voting dependency at a time, verify convergence, then continue.
A healthy process is not enough.
A healthy quorum is what matters.
So Should Everything Use Three Nodes?
No.
For majority-based consensus, a useful sizing rule is:
N = 2F + 1
where F is the number of simultaneous voting-member failures the system must tolerate while continuing to make progress.
F = 1 → N = 3F = 2 → N = 5F = 3 → N = 7
But consensus-group size should not be used as a substitute for capacity scaling.
Need more throughput? Add brokers, partitions, shards, replicas, or workers where the product architecture allows it.
Consensus groups exist primarily to preserve authority, not to provide storage or compute capacity.
That is why three voters are such a common baseline, and why five only becomes worthwhile when surviving two simultaneous voting-member failures is an actual requirement.
The Real Meaning of Quorum
Quorum is not really about the number three.
It is about authority.
It defines how much independent agreement a distributed system requires before it is willing to treat something as true: a leader, a write, a committed log entry, a membership change, or an authoritative state.
Once you look at distributed systems this way, configurations such as:
3 nodes5 controllersRF = 3minISR = 2w: "majority"LOCAL_QUORUM
stop looking like arbitrary best practices.
They become consequences of deliberate choices about failure, consistency, durability, latency, and availability.
So the better production question is not:
“Is the node up?”
It is:
“Does the system still have enough independent agreement to know that what it is doing is safe?”
That is what quorum is really about.
References
Leslie Lamport — Paxos Made Simple
Diego Ongaro & John Ousterhout — In Search of an Understandable Consensus Algorithm (Raft)
Giuseppe DeCandia et al. — Dynamo: Amazon’s Highly Available Key-value Store
Seth Gilbert & Nancy Lynch — Brewer’s Conjecture and the Feasibility of Consistent, Available, Partition-Tolerant Web Services
Daniel J. Abadi — Consistency Tradeoffs in Modern Distributed Database System Design: CAP is Only Part of the Story
etcd Documentation — FAQ
