Then we look around.

WhatsApp delivers billions of messages. Netflix streams to millions of users. Twitter refreshes personalised timelines almost instantly.

These products don’t just work — they continue to work at scale. That isn’t luck. It’s engineering.

I spend my day job writing Laravel/PHP at a startup. This project is how I’m forcing myself to think in distributed systems — and I’m building it in the open, phase by phase, with a full write-up after each one.

Why Backend Engineers Should Build Systems, Not Clones

Two friends sketch a few screens, write some code, connect a database, and launch v1. Months later, millions of people use it every day.

Except that’s almost never how it happens.

Between the first line of code and the product we use today are years of failures, redesigns, bottlenecks, and thousands of engineering decisions. Systems that worked for a thousand users stopped working for a million. Databases were redesigned. New technologies were introduced — not because they were trendy, but because they solved real problems.

I think the same lesson applies to us as developers. There’s nothing wrong with building clones — they teach us what to build. Building systems teaches us why.

When something breaks, you’re forced to ask better questions. Why is this query slow? Should this run in the background? Do I really need a cache? Is this an application problem or a database problem? Over time, you stop copying architecture and start understanding it.

That’s the experience I’m after.

Why a Twitter Timeline?

On the surface, a timeline looks simple: you follow people, they post, you open the app and see their posts.

Behind that simple experience is one of the more interesting backend problems out there. Every user's timeline is different. New posts are constantly being created, users expect their feed to load instantly, and the system has to balance reads, writes, storage, and latency all at once.

A timeline isn't just:

sql

SELECT * FROM posts;

It's a distributed systems problem disguised as a social media feature. That's why I picked it — not for another Twitter clone, but because it naturally forces problems around database design, pagination, caching, background processing, and architectural trade-offs.

One Rule Before Writing Any Code

Before writing a single line, I made one promise to myself: don’t optimise before you measure.

It’s tempting to start every project with Redis, Kafka, and microservices because that’s what large companies use. But those companies didn’t start there — every piece of infrastructure was introduced because it solved a specific bottleneck. I want this project to follow the same philosophy: start with the simplest correct solution, and only add complexity once I understand exactly why it’s needed.

The Series

I’m building this in phases, and I’ll be writing up each one as it lands — architecture decisions, benchmarks, what broke, what I’d do differently.

Phase 1 — Foundation. Users, follow relationships, posts, a timeline API, cursor pagination, and a baseline benchmark. The question here: how far can a well-designed PostgreSQL app go before you need anything else?

Phase 2 — Scaling reads and writes. Redis, BullMQ, background workers, and the fan-out question: do you write to followers’ timelines when a post is made, or read and merge at request time?

Phase 3 — Thinking like a distributed system. Hybrid timeline generation for high-follower accounts, cache invalidation, observability, load testing, and horizontal scaling — not because they’re buzzwords, but because the earlier phases will force them.

Building in Public

This isn’t a tutorial and it isn’t a finished project — it’s a build log. I’ll share the decisions, the mistakes, the trade-offs, and the benchmarks along the way. Some ideas will hold up under load. Some will look fine in testing and fall apart the moment real traffic hits — and those are usually the ones worth writing about.

If you’re curious what happens past CRUD, follow along — Part 1 (the foundation and first benchmarks) is coming next.

I’ve been building backends for two years. Laravel, Node.js, Redis, PostgreSQL — real production systems, real traffic. I thought I had a decent mental model of how things fail.

Then I opened Designing Data-Intensive Applications, and Chapter 1 quietly dismantled something I didn’t even know I assumed.

The assumption I didn’t know I had

On a single machine, failure is binary.

A process either runs or it crashes. A database is either up or it’s down. Your server either responds or it times out. You always know — even if what you know is bad news.

This assumption is so deeply baked in that we don’t even think about it. It shapes how we debug. It shapes how we write error handling. It shapes what we reach for when something goes wrong: logs, stack traces, console.log.

And it works. Until it doesn’t.

Partial failure — the thing that breaks the mental mode

Here’s the scenario DDIA introduced that stopped me cold:

Node A is up. Node B is up. The network between them is silently dropping packets.

No exception is thrown. No alert fires. Both nodes are healthy by every metric you’re watching. But your system is producing wrong results — or worse, inconsistent results — and you have no idea.

This is a partial failure. And it’s a fundamentally different class of problem than anything you encounter on a single machine.

On one server, failure is an event. In a distributed system, failure is a state — one that your system can be in without knowing it.

Why this matters beyond theory

The moment this clicked, I finally understood something that had always felt like over-engineering to me: observability tooling.

I knew what Zipkin and OpenTelemetry were. I’d seen them in job descriptions. I’d nodded along when senior engineers talked about distributed tracing. But I never deeply understood why they were necessary — not just useful, but necessary.

Now I do.

When your system is distributed, a single request touches multiple services. Service A calls Service B calls Service C. Any one of those hops can degrade silently. The request completes — maybe slowly, maybe with a subtle data inconsistency — but no single service logged an error.

You can’t reconstruct what happened from individual service logs. The failure lives between the services, not inside any of them.

That’s what distributed tracing solves. A trace ID propagates through every hop. Zipkin correlates the spans. OpenTelemetry standardises the instrumentation. You get a timeline of the entire request across every service boundary.

You’re not debugging a process. You’re debugging a conversation between processes.

What does this change about how I think

Before DDIA: I thought about failure as “something broke, find the error.”

After DDIA Chapter 1: I think about failure as “something might be degrading silently right now, and the only way to know is to have built the observability before you needed it.”

That’s a different engineering posture entirely. It means:

• You instrument before incidents, not after
• You treat network calls as inherently unreliable, not just occasionally unreliable
• You design for partial availability, not just uptime
• You build tracing into your services because console.log only tells you what happened inside one box

Where I’m taking this

I’m currently building HookRelay — a webhook delivery service — in TypeScript and PostgreSQL. Webhook delivery is inherently distributed: your server, the queue, the delivery worker, the client’s endpoint.

Partial failure is not a theoretical concern for me. It’s the default operating condition.

DDIA didn’t just teach me vocabulary. It gave me a framework to think about what I’m actually building. Chapter 1 alone reframed two years of production intuition.

I’m reading the rest of the book the same way: not to memorise concepts, but to find the next thing I was wrong about without knowing it.

Reading DDIA 2nd edition and writing about it as I go. If this resonated, follow along — one chapter at a time.

Let’s break down these concepts through real-life analogies.

🧵 Synchronous JavaScript — The Queue Analogy

Imagine you’re standing in a bank queue. Each customer (task) is served one at a time. No matter how small or big the task is, the next person in line must wait until the current one is fully served.

In the same way, synchronous JavaScript executes code line by line, one statement after another. If a line takes time (like reading a file or fetching data), it blocks everything else.

Example:

console.log("1");
console.log("2");
console.log("3");

Output:

1
2
3

Each line runs in order. No skipping. No multitasking.

⏳ The Problem with Synchronous Code

Now imagine one customer in the queue is filling out a loan application that takes 10 minutes. Everyone behind them is stuck doing nothing.

That’s exactly the issue with synchronous JavaScript. If a task takes time (like downloading a file), everything else has to wait.

🎭 Asynchronous JavaScript — The Restaurant Analogy

Now imagine you’re at a restaurant. You place an order (a task), and the waiter (JavaScript engine) takes your request but doesn’t make you wait. Instead, you go back to your table and continue chatting (doing other tasks).

When your food is ready (task is done), the waiter comes back and serves it. That’s asynchronous behavior.

In JavaScript, you can use callbacks, promises, or async/await to tell the engine:
“Go do this, and when you’re done, let me know.”

Example with setTimeout:

console.log("1");

setTimeout(() => {
  console.log("2");
}, 2000); // 2-second delay

console.log("3");

Output:

1
3
2

Even though console.log("2") appears earlier in the code, it runs later because JavaScript continues without waiting for the 2-second timer.

🧠 What’s Really Happening Behind the Scenes?

JavaScript uses something called the Event Loop and a Task Queue to manage asynchronous tasks.

• Think of the Call Stack as the chef preparing food orders.
• The Task Queue is like a counter where finished dishes (callbacks) are placed.
• The Event Loop keeps checking: “Is the chef (call stack) free?” If yes, it grabs the next ready dish (task) from the counter (queue) and serves it.

📦 Promises — The Amazon Analogy

Think of a Promise like ordering a product from Amazon.

• Pending: You placed the order.
• Fulfilled: The product is delivered.
• Rejected: The order failed.

While you’re waiting for the delivery, you can do other things. When the delivery arrives, you get notified (via .then() or await).

Example:

fetch('https://api.example.com/data')
  .then(response => response.json())
  .then(data => console.log(data));

The request is sent, JavaScript moves on, and when the data is ready, it processes it.

🧘‍♂️ Async/Await — The Netflix Analogy

With async/awaitYour code looks synchronous but behaves asynchronously.

Imagine binge-watching a Netflix series. You watch one episode, wait for it to finish, then go to the next. You’re still watching at your pace, but each episode “awaits” the end of the previous one.

Example:

async function getData() {
  const response = await fetch('https://api.example.com/data');
  const data = await response.json();
  console.log(data);
}

getData();

It’s easier to read and understand than chaining .then() calls, but under the hood, it’s still non-blocking.

📏 Conclusion

Features: Synchronous, Asynchronous, Blocking, Yes, No. Execution: One at a time, can skip and come back later. Example Analogy Bank queue Restaurant or Amazon delivery JavaScript Tools Default behaviorsetTimeout, Promises, async/await

Understanding the difference helps you write faster, non-blocking code — a must for modern web apps. Just remember:

“Let JavaScript take the order, but don’t force it to cook before serving the next customer!”

🎥 Bonus Learning Resource

For a more detailed explanation, you can check out the beautifully explained video:
Understanding Sync vs Async JavaScript

1. .some(): "Koi toh hoga!" (At least one should match)

Imagine a group of friends discussing where to eat. One person suggests a pure vegetarian restaurant. The question is:
“Is there at least one person in the group who prefers vegetarian food?”

Using .some(), we check if at least one person meets the conditions:

Copy

Copy

javascriptCopyEditlet friends = ["Rohan", "Amit", "Pooja", "Neha"];
let prefersVegetarian = name => name === "Pooja"; 

console.log(friends.some(prefersVegetarian)); // true

In an Indian context, it’s like checking if at least one person in a family is a cricket fan during IPL season. If even one person watches, the TV is tuned to cricket!

2. .every(): "Sabko pasand hona chahiye!" (Everyone should match)

Imagine a family trip to Kedarnath. The trip will happen only if everyone in the family agrees to go.

Let’s check if all members are interested:

Copy

Copy

javascriptCopyEditlet family = ["Papa", "Mummy", "Bhaiya", "Didi"];
let wantsToVisitKedarnath = name => name !== "Bhaiya"; 

console.log(family.every(wantsToVisitKedarnath)); // false

Here, if even one person (Bhaiya) disagrees, the whole trip gets canceled!

3. .find(): "Ek hi banda chahiye!" (Find the first matching element)

Imagine searching for a chaiwala in a railway station. You don’t need all chaiwalas, just one to get your tea.

Copy

Copy

javascriptCopyEditlet vendors = [
    { name: "Raju", sells: "Pani" },
    { name: "Amit", sells: "Chai" },
    { name: "Vikas", sells: "Samosa" }
];

let chaiwala = vendors.find(vendor => vendor.sells === "Chai");

console.log(chaiwala); // { name: "Amit", sells: "Chai" }

This returns Amit, as he is the first chaiwala found.

4. .includes(): "List mein hai ya nahi?" (Check if an item exists)

Imagine you are making Aloo Paratha but aren’t sure if you have ajwain (carom seeds) in your kitchen. Instead of checking each item manually, .includes() helps:

Copy

Copy

javascriptCopyEditlet spices = ["Haldi", "Jeera", "Dhaniya", "Ajwain"];
console.log(spices.includes("Ajwain")); // true
console.log(spices.includes("Elaichi")); // false

This is just like checking if Amitabh Bachchan is in a Bollywood movie cast — if yes, it’s an instant hit! 🎬

5. .forEach(): "Sabke liye chai banao!" (Perform an action for every element)

Imagine a Halwai (sweet shop owner) preparing Laddoos for Diwali. He needs to apply silver foil (chandi ka warq) to each laddoo. Instead of doing it manually, .forEach() automates the task:

Copy

Copy

javascriptCopyEditlet laddoos = ["Besan", "Motichoor", "Coconut"];

laddoos.forEach(laddoo => console.log(`Applying silver foil on ${laddoo} laddoo`));

// Applying silver foil on Besan laddoo
// Applying silver foil on Motichoor laddoo
// Applying silver foil on Coconut laddoo

This ensures that every laddoo gets special treatment. Similarly, .forEach() is useful when performing actions on each element in an array.

Conclusion

These array methods can be powerful when used correctly:

MethodMeaning in the ContextUse Case.some()At least one cricket fanDoes at least one element satisfy the condition?.every()Everyone must agree to the Kedarnath tripDo all elements meet the condition?.find()Finding the first chaiwalaGet the first element that matches.includes()Checking if ajwain is in the kitchenDoes an item exist in the array?.forEach()Applying foil to every laddooExecute a function for every element

By associating these methods with relatable Indian scenarios, learning them becomes easier and more enjoyable! 🚀

1. map(): Transforming Every Element

Analogy: Preparing Masala Chai from Tea Leaves

Imagine running a tea stall (chai tapri) and having a batch of tea leaves. You aim to convert all these raw tea leaves into a masala chai.

• The input: Tea leaves
• The transformation: Boiling them with milk, sugar, and spices
• The output: A delicious cup of masala chai for each tea leaf batch

In JavaScript, map works similarly—it takes each item in an array, transforms it, and returns a new array.

const teaLeaves = ["Assam", "Darjeeling", "Masala", "Green"];
const masalaChai = teaLeaves.map(leaf => leaf + " Chai");

console.log(masalaChai);

// Output: ["Assam Chai", "Darjeeling Chai", "Masala Chai", "Green Chai"]

Each tea leaf is transformed into a flavored chai, just like map() applies a function to each element in an array.

2. filter(): Selecting What Matters

Analogy: Picking the Freshest Mangoes from a Market

In a mango market, a vendor has a pile of mangoes, but not all are good for sale. The vendor picks only the ripest mangoes to sell.

• The input: A pile of mangoes
• The condition: Only ripe mangoes are selected
• The output: A new pile of ripe mangoes

In JavaScript, filter is used to create a new array containing only elements that meet a certain condition.

const mangoes = [
  { name: "Alphonso", ripe: true },
  { name: "Dasheri", ripe: false },
  { name: "Langda", ripe: true },
  { name: "Totapuri", ripe: false }
];

const ripeMangoes = mangoes.filter(mango => mango.ripe);
console.log(ripeMangoes);
// Output: [{ name: "Alphonso", ripe: true }, { name: "Langda", ripe: true }]

Just like a vendor picks only the best mangoes, filter() selects only elements that satisfy a condition.

3. reduce(): Summarizing Data

Analogy: Collecting Donations for a Temple Festival

Suppose a temple committee is collecting donations from people for an annual festival. Each person contributes some amount, and in the end, the total sum needs to be calculated.

• The input: Individual donations
• The operation: Adding all the donations together
• The output: Total donation amount

In JavaScript, reduce helps in accumulating values into a single result.

const donations = [100, 500, 250, 150, 1000];
const totalDonation = donations.reduce((sum, amount) => sum + amount, 0);
console.log(totalDonation);
// Output: 2000

Just like the committee collects donations from individuals and sums them up, reduce() accumulates array values into a single output.

Conclusion

Each of these methods plays a unique role in JavaScript:

• map() transforms elements (like making chai from tea leaves).
• filter() selects elements based on a condition (like choosing ripe mangoes).
• reduce() combines elements into a single value (like totaling donations).

These functions make JavaScript arrays more powerful and help write cleaner, more efficient code. Next time you use map, filter, or reduce, just think about chai, mangoes, and temple donations!

In this article, I’ll walk you through how to implement a Stack using a Linked List in C++.

Why Use a Linked List for a Stack?

• Dynamic Size: Unlike arrays, linked lists allow stacks to grow or shrink dynamically.
• Memory Efficiency: No need to allocate extra memory upfront; each element is created only when needed.
• No Size Limit: You don’t have to worry about exceeding a predefined capacity.

Operations on a Stack

A stack supports three primary operations:

1. Push: Add an element to the top of the stack.
2. Pop: Remove and return the top element from the stack.
3. Peek (or Top): View the top element without removing it.
4. isEmpty: Check if the stack is empty.

Linked List Structure

To implement a stack using a linked list:

1. Each node in the linked list represents an element of the stack.
2. The head of the linked list acts as the top of the stack.

C++ Code Implementation:

#include <iostream>
using namespace std;
// Node structure for the linked list
struct Node {
 int data;
 Node* next;
};
// Stack class
class Stack {
private:
 Node* top; // Pointer to the top of the stack
public:
 // Constructor
 Stack() {
 top = nullptr;
 }
// Push an element onto the stack
 void push(int value) {
 Node* newNode = new Node();
 newNode->data = value;
 newNode->next = top;
 top = newNode;
 cout << "Pushed " << value << " onto the stack.\n";
 }
// Pop the top element from the stack
 int pop() {
 if (isEmpty()) {
 cout << "Stack underflow! Cannot pop from an empty stack.\n";
 return -1; // Return an invalid value to indicate error
 }
int poppedValue = top->data;
 Node* temp = top;
 top = top->next;
 delete temp;
 return poppedValue;
 }
// Peek at the top element without removing it
 int peek() {
 if (isEmpty()) {
 cout << "Stack is empty! No top element.\n";
 return -1; // Return an invalid value to indicate error
 }
return top->data;
 }
// Check if the stack is empty
 bool isEmpty() {
 return top == nullptr;
 }
// Destructor to free memory
 ~Stack() {
 while (!isEmpty()) {
 pop();
 }
 }
};

How It Works

Let’s break down each function:

1. push:

• Creates a new Node with the given value.
• Updates the next pointer of the new node to point to the current top.
• Sets the new node as the new top.

1. pop:

• Checks if the stack is empty.
• Stores the value of the current top node.
• Moves the top pointer to the next node.
• Deletes the old top node and returns the stored value.

1. peek:

• Simply returns the value of the top node without removing it.

1. isEmpty:

• Checks if the top pointer is nullptr.

Time and Space Complexity

The space complexity for the entire stack depends on the number of elements and is O(n), where n is the number of nodes in the linked list.

Advantages of Linked List-Based Stack

• No need for a predefined size.
• Dynamically grows as needed.

Disadvantages

• Slightly more memory usage compared to arrays due to pointer storage.
• Requires additional pointer management.

Conclusion

Using a linked list to implement a stack provides flexibility and dynamic memory management, making it an excellent choice in situations where the size of the stack cannot be determined beforehand. While arrays might be simpler, linked lists offer greater scalability for larger applications.

Now you can implement your stack using linked lists and explore its potential in your projects! 🚀

In the world of competitive programming and technical interviews, Data Structures and Algorithms (DSA) are essential skills to master. While many of us can grasp the intuition behind a problem, translating that understanding into working code often feels like a completely different battle, especially in a language like C++. This gap between intuition and implementation can be frustrating, but with the right mindset and strategies, it’s possible to bridge it. In this article, I’ll break down why this gap exists and offer actionable techniques to help you improve your implementation skills.

The Intuition-Implementation Gap: Why Does It Happen?

Understanding a problem, also called “intuition,” involves recognizing patterns, choosing the right data structure, and coming up with a high-level approach to solve it. You might even be able to visualize how the solution works in theory. However, once you sit down to write the code, things might fall apart. Here’s why:

• Over-simplification in Understanding: When we think about a problem abstractly, we often skip over the intricate details, such as edge cases, corner scenarios, or the actual mechanics of working with data structures.
• Lack of Familiarity with the Language: For those working in C++, understanding the problem is one thing, but knowing how to manipulate pointers, use STL containers effectively, or manage memory efficiently can be challenging. C++ is powerful but requires careful handling.
• Unrefined Coding Habits: Even with the right idea, poor habits like skipping planning, writing unstructured code, or neglecting to break the problem down into smaller tasks can derail your progress.
• Anxiety or Overthinking: Sometimes, knowing the solution in theory but not being able to implement it correctly can lead to self-doubt or over-complicating the problem, making the task feel overwhelming.
• The “I’ll Figure It Out” Trap: Often, we dive into coding without a concrete plan, assuming that we’ll figure out edge cases as we go. This leads to getting stuck midway through the implementation, causing frustration and time wastage.

Now that we know why this happens, let’s focus on bridging the gap.

Techniques for Improving Implementation Skills

1. Break Down the Problem Into Sub-Problems:

One of the key steps to move from intuition to implementation is to decompose the problem into smaller, more manageable tasks. For example, if you’re working on a linked list problem, start by outlining the major steps like:

• Iterating through the linked list.
• Identifying when a condition is met (e.g., detecting a loop).
• Applying the solution logic (e.g., reversing part of the list).

Write pseudocode for each part before jumping into C++ code. This will help you keep track of what you need to implement without getting lost in the details.

3. Learn to Identify Edge Cases Early

One common reason for implementation struggles is overlooking edge cases until you start writing code. To fix this:

• Analyze the problem deeply: Before writing the solution, list potential edge cases — empty inputs, large inputs, negative values, etc.
• Implement small checks: In C++, be cautious about handling these edge cases, especially when dealing with arrays or pointers. Write conditional checks early in your code to ensure you’re not accessing invalid memory or exceeding array bounds.

For example, when working with a linked list, always check if the head is null before proceeding.

4. Work Incrementally: Test as You Go

Instead of writing the entire solution and then running into a wall of bugs, break your code into chunks and test each chunk as you go:

• Write functions for individual tasks: Break down complex problems into smaller, manageable functions. In C++, separating your code into helper functions also makes debugging easier.
• Run simple test cases first: Don’t wait until the full implementation is done to run tests. Test simple cases (like the smallest input) and work your way up to more complex cases.

This approach helps you isolate issues and correct them without getting overwhelmed by multiple bugs at once.

5. Practice, Reflect, and Reimplement

You won’t bridge the gap by solving a problem once and moving on. Instead:

• Revisit old problems: Go back to problems you’ve already solved and try to re-implement them from scratch. Doing this repeatedly helps you internalize patterns and techniques.
• Analyze others’ solutions: After solving a problem, review other solutions in C++. You might discover more efficient or cleaner ways to implement the same logic.
• Reflect on your mistakes: Keep a log of the errors and challenges you faced, especially in C++ (e.g., memory management issues or infinite loops). Reflecting on these will help you avoid repeating the same mistakes.