Imagine You Walk Into Burger Mania on a Busy Evening

You walk into Burger Mania at peak hours.

There are 20 people waiting in line.
The restaurant has 5 chefs, 5 stoves, 5 staff at the counter, and 10 tables.

On paper, this place should run smoothly.

But something feels off.

There is only one queue.

One staff member is taking orders. One by one.

You are standing 5th in line, just waiting to say what you want. The person at the counter takes the first order, writes it down, sends it to the kitchen, waits a bit, then moves to the next person.

Nothing else is happening.

Now look around.

Four staff members are doing nothing.
Chefs are waiting because orders are not reaching them fast enough.
Stoves are sitting unused.
Tables are empty.

Everything is available, but everything is stuck.

The system is technically working, but it is slow, inefficient, and frustrating.

This is what a blocking system looks like.

Every step waits for the previous one to finish before anything else can begin.

What Changed in the Story, and What That Means in System Design

Let’s go back inside Burger Mania.

Nothing about the restaurant changed.

Still the same chefs.
Same stoves.
Same staff.
Same tables.

But suddenly, the place started working smoothly.

So what actually changed?

Asynchronous Processing (Handling Multiple Requests)

Splitting one queue into many means requests are handled independently. One customer placing an order no longer blocks the next.

In system design terms, this is asynchronous processing.

What you introduce:

• Async frameworks like FastAPI, aiohttp
• Event loop based execution

What changes:

• Requests are handled independently
• System handles multiple requests concurrently
• Throughput increases without adding hardware

Non-Blocking I/O (The Real Game Changer)

Earlier, order taking slowed down while the kitchen processed requests. Now, orders keep coming in while cooking happens separately.

That’s non-blocking I/O.

What you introduce:

• Async DB drivers like asyncpg
• Async HTTP clients like httpx

What changes:

• API doesn’t wait for DB or network
• Threads are not blocked
• System continues processing other requests

Concurrency (Handling Multiple Tasks Together)

Each chef picks an order independently.

That’s concurrency.

What you introduce:

• asyncio.gather
• Parallel task execution inside a request

What changes:

• One request can trigger multiple operations
• Tasks are in progress simultaneously

Parallelism and Threading (Actual Execution)

Multiple chefs cooking at the same time represents parallelism.

What you introduce:

• ThreadPoolExecutor
• Worker processes

What changes:

• CPU-bound work runs in parallel
• Better utilization of compute resources

Semaphore and Rate Limiting (Preventing System Overload)

Only 5 stoves and 10 tables. Not everything can run at once.

That’s a semaphore.

What you introduce:

• Semaphores in code
• Rate limiting
• API gateways

What changes:

• System enforces limits
• Prevents crashes under high load

Task Queues and Background Jobs

Each chef maintains their own list of orders.

That’s task queues and background processing.

What you introduce:

• Kafka, RabbitMQ, AWS SQS
• Celery workers

What changes:

• Work is distributed
• Tasks are decoupled
• System becomes scalable

gRPC and Fast Service Communication

Orders move instantly between counter and kitchen.

That’s efficient internal communication.

What you introduce:

• gRPC
• Protocol Buffers

What changes:

• Faster service-to-service calls
• Lower latency than REST
• Efficient data transfer

Connection Pooling and Database Performance

Instead of everyone waiting for a single notebook, access is shared.

That’s connection pooling + async DB access.

What you introduce:

• PgBouncer
• Async DB engines

What changes:

• Connections reused
• Queries don’t block each other
• Handles high concurrency efficiently

Why Asynchronous Systems Matter, How They Differ from Synchronous Ones, and Where They Can Hurt You

By now, the Burger Mania setup should feel intuitive.

Let’s translate that intuition into real system behavior.

1. The Core Idea Behind Asynchronous Systems

At a high level, asynchronous systems are not about making one task faster.

They are about making sure:

nothing sits idle while waiting.

In most modern applications, the slowest parts are:

• Database queries
• External API calls
• Disk operations

A synchronous system waits for these to complete.

An asynchronous system moves on and does something else.

That single shift changes everything.

2. Key Benefits of Asynchronous Processing

2.1 Better Resource Utilization

In a synchronous system:

• A thread makes a DB call
• It waits
• CPU sits idle

In an asynchronous system:

• That waiting time is reused
• The same thread handles other requests

What this really means is:
you can handle more traffic without increasing hardware.

This is one of the biggest performance gains highlighted in research on async frameworks combined with connection pooling. Systems handle higher concurrency without exhausting resources.

2.2 Higher Throughput

Throughput is how many requests your system can handle per second.

With async:

• Requests don’t block each other
• The system keeps moving

Result:

• More requests processed
• Better scalability

2.3 Lower Latency Under Load

Important nuance.

Async does not always make a single request faster.

But under load:

• Synchronous systems create queues
• Requests pile up
• Response time increases

Async systems keep processing in parallel.

Result:

• Lower average latency
• Much better worst-case latency

2.4 Efficient Handling of High Concurrency

If 1000 users hit your system:

Synchronous:

• Needs many threads
• High memory usage
• Context switching overhead

Asynchronous:

• Uses fewer threads
• Handles many tasks through an event loop

This is why async is widely used in:

• APIs
• Real-time systems
• Streaming applications

2.5 Works Best for I/O Heavy Systems

Async shines when your system spends time waiting.

Examples:

• Database-heavy apps
• Microservices calling other services
• File uploads and processing

This aligns with practical system design guidance:
async is ideal for I/O-bound workloads.

2.6 Works Well with Connection Pooling

Modern systems combine:

• Async processing
• Database connection pools

Instead of opening a new connection per request:

• Connections are reused
• Requests don’t block

Research shows this combination significantly improves performance in high-concurrency environments.

2.7 Improved User Experience

Because:

• The system does not freeze
• Background tasks don’t block responses

Example:

• User uploads a file
• Processing happens in background
• User continues using the app

3. Synchronous vs Asynchronous Systems

Let’s make the difference very clear.

Synchronous system

• One task at a time
• Each step waits for the previous one
• Easy to understand
• Poor scalability

Asynchronous system

• Multiple tasks in progress
• No waiting on slow operations
• More efficient
• More complex

A simple mental model

Synchronous:
wait, then work

Asynchronous:
work while waiting

4. Where Asynchronous Systems Fit

Use async when:

• You have frequent DB calls
• You depend on external APIs
• You expect high concurrent users

Avoid relying only on async when:

• Tasks are CPU-heavy

In that case:

• Use threads or processes

5. Drawbacks of Asynchronous Systems

This is where most people underestimate the cost.

5.1 Increased Complexity

Async code is harder to read and reason about.

Flow is not linear.

Instead of:

• Step 1 → Step 2 → Step 3

You now have:

• Multiple tasks running
• Execution happening at different times

This makes development and debugging harder.

5.2 Harder Error Handling

Errors in async systems:

• May occur later
• May occur in a different context

This can lead to:

• Missed exceptions
• Silent failures

You need proper logging and monitoring.

5.3 Not Suitable for CPU-Heavy Tasks

Async does not make computation faster.

If a task is CPU-heavy:

• It still blocks execution

You need:

• Threading
• Multiprocessing

5.4 Race Conditions and Data Issues

When multiple tasks run together:

• Shared data can get corrupted
• Execution order is unpredictable

This introduces:

• Race conditions
• Hard-to-reproduce bugs

5.5 Requires Strict Resource Control

If you allow unlimited async tasks:

• Too many DB queries
• Too many API calls
• System overload

That’s why:

• Semaphores
• Rate limiting
• Connection pools

are essential.

5.6 Debugging and Observability is Harder

Tracking issues becomes difficult because:

• Execution is not sequential
• Multiple tasks interact

You need:

• Structured logging
• Distributed tracing
• Metrics

6. The Balanced View

Asynchronous systems are powerful, but they are not a silver bullet.

The right approach is:

• Use async for I/O-heavy work
• Use threads or processes for CPU-heavy work
• Always control concurrency

7. Final Thought

The biggest mistake is thinking async is about speed.

It is not.

It is about removing wasted time.

Once you remove waiting, your system naturally becomes faster, more efficient, and more scalable.

Python Examples That Map to the System Design Concepts

We’ll keep this practical and aligned with the story.

Each example shows what changes when you move from blocking to async, and how you control concurrency.

1. Blocking vs Asynchronous Execution

Blocking version

What happens here:

• Each request waits for the previous one
• Total time is roughly 6 seconds

This is your single queue restaurant.

Asynchronous version

What changes:

• All tasks run together
• Total time drops to around 2 seconds

This is multiple order counters working at once.

2. Concurrency with asyncio

What this shows:

• Multiple requests are handled at the same time
• No one waits unnecessarily

This maps to concurrency in the restaurant.

3. Threading for Parallel Work (Chefs Cooking)

What this shows:

• Multiple tasks actually execute in parallel
• Useful for blocking operations

This is like multiple chefs cooking at the same time.

4. Controlling Load with Semaphore

What this shows:

• Only 2 tasks run at a time
• Prevents overload

This is your limited stoves in the kitchen.

5. Simulating Async Database Calls

What this shows:

• Database calls don’t block each other
• Multiple queries execute concurrently

This is non-blocking I/O in action.

6. Connection Pool Concept (Controlled DB Access)

What this shows:

• Only 3 DB connections used at a time
• Prevents database overload

This directly maps to connection pooling in real systems.

7. Background Tasks (Decouplig Work)

What this shows:

• User does not wait for long tasks
• Work continues in the background

This is how modern async systems improve user experience.

8. Parallel Service Calls (gRPC Style Thinking)

What this shows:

• Multiple services are called at the same time
• Faster overall response

This is how internal services communicate efficiently in real systems.

9. What You Should Take Away from These Examples

• asyncio.gather gives you concurrency
• await prevents blocking
• Semaphore protects your system
• Threads help with parallel execution
• Background tasks decouple work

Together, these patterns form the backbone of modern scalable systems.

Final Thought

Async is not about making things faster.

It’s about eliminating wasted time.

Once you remove waiting, your system naturally becomes faster, more efficient, and scalable.

🚀 Why is DRY important?

• Encourages code reusability
• Reduces redundant logic
• Improves maintainability & debugging
• Saves development time & effort

A Lesson from My Experience 🎯

While working on a GenAI chatbot at Amdocs, I had to scrape content for the D1 Domain and create an index for chatbot queries.

✔️ What I Did:

• Checked the existing code ✅
• My approach to scraping was different, so I wrote a new function from scratch instead of reusing existing logic.

❌ What Went Wrong?

• My approach failed after 3 days of effort in coding & testing.
• After discussing with a senior, I realized that a better, existing approach could have saved me all that time.

🔥 Lesson Learned:

✅ Discuss your approach with seniors (more than one, if possible) before diving into coding.
✅ Think like an engineer constructing a building — consult the architect before building!

📌 The Role of Modular Design

Breaking a feature into independent modules ensures:
🔹 Scalability — Each module works independently.
🔹 Maintainability — Easier debugging & updates.
🔹 Efficiency — Faster development & testing.

Example: A Modular Chatbot for D1 Domain

🔹 Data Collection

• Scraping Confluence pages or PDFs
  🔹 Storage
• Storing vector data in ChromaDB
  🔹 Indexing
• Organizing vector data by user-chosen domain
  🔹 Model Initialization
• Using LangChain & OpenAI to query the RAG model
  🔹 API Handling
• Creating APIs to handle train, query, and index requests

💡 The Takeaway? Always reuse existing solutions when possible and structure your code in a modular way to save time and effort.

#SoftwareDevelopment #DRYPrinciple #CodingBestPractices #AI #GenAI #CareerGrowth

🙋‍♂️ Hi guys , I always had a small doubt when I heard there are more than one API, I heard about GRAPHQL, and REST, so I got a doubt that which one should I use and which one is better to use.

👉In this post, I would like to give a brief on which is the best, I know only REST API, but not GRAPHQL, so I am using some reference materials for completing this post.

👉 REST API vs GRAPHQL

Let me start with which one is the best

✍️ GraphQL is the better than REST

👨‍💻 The main difference between GRAPHQL and REST

👉 A REST API is an architectural concept for network-based software. GraphQL, on the other hand, is a query language, a specification, and a set of tools that operates over a single endpoint using HTTP

👨‍💻 Where GraphQL is better than REST API

👉 When using a REST API to fetch information, you’ll always get back a complete dataset. For example, if you wanted to request information from two objects, you’d need to perform two REST API requests.

Lets say you need initial user data, and you need to fetch it from /users/:userID, and secondly you need posts data for that user, so another end point may be. /users/:userID/posts.

So when the end point is REST API. It would take two HTTP GET requests. They are (I am using json data in this example)

/users/:userID

{

         “user”:{

             “id”:”user1”,

             “firstName”:”rohan”,

              “lastName”:”devaki”

         }

}

And one more request for posts of userID with end point /users/:userID/posts

 {

     “posts”: [{

         “id” :”post1”,

         “title”: “REST_vs_GRAPHQL”,

         “content”:”graphql is better than REST”

      }]

}

In GRAPHQL, on the other hand, you need to simply send a query to the GRAPHQL server that includes the data requirements. Then the server responds with the JSON object where the requirements are fulfilled

Query{

  User(id:”user1”){

      Name

      Posts{

           Title     

      }

}

👉 So by this we can understand that the GRAPHQL is powerful, flexible and efficient , where as REST is not that much flexible and inefficient.

👉 GRAPHQL also fetches the data , that is actually required in the query. But REST in the other hand, it fetches all the data that is not even required.

👨‍💻 as in the above example you can see REST fetched, all of the user and the post details, where as GRAPHQL only fetched the user’s name and the post’s title.

Don’t just read it ,and get the knowledge, like and share so that others also get the knowledge 😃.

you are welcomed to tell the points which are need to be improved in the comments section 😃.

Follow me on Linkedin here , and I will be back with an informative post soon 😃.