Kafka Replication and Fault Tolerance
A Kafka topic represents a stream of data, and a given topic can be identified by its name. A single Kafka topic contains a set of partitions. When messages are produced to a topic, they are directed to one of the topic’s partitions.
Kafka Broker
In simple terms, a Kafka broker can be identified as a server. A collection of Kafka brokers forms a Kafka cluster.
Get Pawara Gunawardena’s stories in your inbox
Join Medium for free to get updates from this writer.
A single broker has a set of topic partitions. Producer and consumer applications use these brokers to send and receive data.
Kafka Topic Replication Factor
Kafka topic partitions can be replicated across brokers. The replication factor of a topic defines the number of replicas for each partition in that topic.
Replicating Kafka topic partitions across brokers ensures fault tolerance.
Leader of a Partition
One broker is designated as the leader of a given partition, and producer applications that need to write data to that partition send data to the leader. Other brokers that are assigned as followers replicate data from the leader to maintain the In-Sync Replicas (ISR).
By default, producer applications send data only to the partition’s leader broker.
If a leader broker of a partition fails, one of the in-sync follower brokers becomes the new leader.
