Ultimate Guide On Kafka #Part 1: What is Kafka?

This course assumes that you have zero understanding of kafka.

Terms to know

• Poll — To regulary check and ask for any changes.
• Event — An event is a message that tells somethings happened. (For example: user ordered an iphone).

Let’s simplify everything step by step —

1. What is Kafka?

Kafka = a middleman 🤔

Imagine you are at a restaurant. There is a waiter between the kitchen and the customer.

Kitchen → Waiter → Customer

Kitchen does not need to find the customer directly. Kafka is exactly that kind of middleman for software.

App A → Kafka → App B
That’s it.

Let’s use a real world example —

You have an online shopping app.

A user clicks on the BUY button. Your backend receives the request.

USER click buy → backend receives the buy request.

Now the backend needs to do 3 things

1. Take Payment
2. Reduce inventory
3. Send Email

without kafka, you backend first need to communicate with payment server, inventory server, and then email server.

Your node server has to communicate with all three.

Get Nitesh Chowdhary’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

Now introduce kafka

Instead :
Your Node.js server simply tells kafka:

Hey Kafka, an order is created.

Kafka remembers that information.

Then the other application read it.

Now let’s understand what exactly Kafka remembers

suppose the user orders an iPhone with order id #123
So, node.js sends this event to kafka:

ORDER_CREATED
OrderId: 123
product: iPhone

Kafka stores this information.

Kafka acts as a notebook and stores all the event information sent by node.js server.

Kafka notebook

1. ORDER_CREATED - order 123
2. ORDER_CREATED - order 124
3. ORDER_CREATED - order 125
4. ORDER_CANCELLED - order 126
5. ORDER_CREATED - order 127

That’s the core idea.

Kafka stores a sequence of events.

Why is that useful? What is kafka suppose to do with the info and why do we need an intermediate layer between our node.js server and other services.

Imagine the payment service is temporarily broken.

Node.js --> Kafka --> Payment ❌

Since the payment is broken, kafka will store that information. When the payment service is back up, payment will poll the events from kafka.

kafka --> Payment ✅

Now, since payment is up and running so payment reads and process that event.

So, kafka acts like a buffer between the application.

In the next article, we will understand producers and consumers…

Note: First time learning Kafka? We’re keeping it simple for now. The deeper technical terms and interview questions will follow in upcoming articles!

Thanks for reading the article.