Backend Communication Patterns

As backend systems scale, spanning microservices, APIs, and real-time updates, efficient communication becomes critical. Design patterns guide how, when, and who communicates across services, impacting performance, scalability, and reliability.

This article covers core patterns like request-response, long polling, pub/sub, and multiplexing. We’ll also explore synchronous vs asynchronous workflows, stateful vs stateless services, and sidecar architecture for improved resilience and observability.

Request-Response Model

The request-response model is the most foundational pattern in backend systems. It’s simple and well-supported across nearly every platform and language, making it the go-to choice for many client-server interactions.

A client sends a request, the server receives and processes it, and then sends back a response. The client then continues execution based on the response received. This synchronous flow — where the client waits for the server’s reply — is what makes request-response predictable and structured.

This model is the bedrock of many modern technologies:

• Web protocols such as HTTP, DNS, and SSH
• APIs built using REST, GraphQL, or SOAP
• Database access via SQL or NoSQL queries
• Remote Procedure Calls (RPC), including binary protocols like gRPC

The communication typically follows a well-defined format, governed by protocols and serialization standards. Common examples include:

• JSON or XML for REST APIs
• Protocol Buffers (Protobuf) for gRPC
• Binary formats for performance-sensitive services

Each message has explicit boundaries, which simplifies parsing and validation on both ends. For instance, a standard HTTP request contains headers, method, URL, and body — all of which are structured and standardized, making it easier to handle across distributed systems.

Example: Image Uploads

Consider an application feature that allows users to upload images. The request-response model fits naturally here:

• For smaller images, a single HTTP POST request can transfer the entire file at once.
• For larger images or when network reliability is a concern, chunked uploads can be used. The image is broken into smaller parts, each sent in a separate request. The server then reconstructs the image from these chunks.

This pattern ensures that clients receive immediate feedback after each step, whether it’s a success acknowledgment or an error code, making it ideal for user-facing workflows.

Limitations of the Model

Despite its ubiquity, the request-response pattern is not suitable for all scenarios. Its synchronous nature means the client is blocked while waiting for the response, which can be inefficient or even problematic in certain contexts. Consider the following examples:

• Real-time applications like chat systems or live dashboards
• Long-running operations such as video processing or data analytics
• Unreliable client connectivity, where users may drop off mid-request

For such use cases, patterns like WebSockets, Server-Sent Events (SSE), or long polling offer better performance and user experience by supporting asynchronous or streaming communication.

Synchronous vs Asynchronous Workloads

One of the core decisions in backend system design is understanding when and how work gets done, especially in relation to how a system handles waiting. This boils down to whether the workload is synchronous or asynchronous, and the choice has significant implications for performance, scalability, and responsiveness.

Synchronous Workloads

In a synchronous workload, the caller halts execution until it receives a response from the receiver. The two systems — caller and callee — operate in lockstep. No other progress can be made until the current task completes.

A classic example comes from the operating system level. Imagine a program requesting a disk read. In a synchronous model, the main thread of execution is suspended until the data is fetched from the disk. During this wait, the program is effectively paused and can’t perform any other meaningful work.

This model is easier to reason about and is often used in simple applications or tasks that require immediate results. However, in high-performance systems, blocking like this can quickly become a bottleneck, wasting valuable compute resources.

Asynchronous Workloads

In contrast, asynchronous workloads decouple the initiation of a task from the moment its result is available. After sending a request, the caller is free to continue executing other operations. The response, once ready, is handled via a notification mechanism, allowing the system to make progress without waiting idly.

There are several ways this can be implemented:

• Polling: The system periodically checks to see if the response has arrived (e.g., via epoll in Linux).
• Callbacks: A designated function is invoked when the result is ready (as seen in io_uring or event-driven frameworks).
• Multithreading: A separate thread handles the blocking operation while the main thread remains responsive and continues execution.

Take Node.js, for example. When it initiates a disk read, the actual I/O is handled by a background thread. The main event loop continues running other code, and once the read is complete, a callback is triggered with the result. This is the foundation of Node.js’s non-blocking I/O model and is key to its ability to handle thousands of concurrent operations efficiently.

Asynchronous Design in Modern Systems

Asynchronous patterns are pervasive in modern backend architectures. You’ll find them across different layers and technologies:

• Programming models: Promises, async/await, futures
• Task scheduling: Job queues that return job IDs for polling results later
• Databases: Asynchronous commits and write-ahead logging in systems like PostgreSQL
• Operating systems: Asynchronous file and network I/O using epoll, io_uring, and fsync buffering
• Distributed systems: Replication mechanisms that synchronize in the background without blocking writes

The benefit of asynchronous execution is clear: it allows systems to be non-blocking, concurrent, and highly scalable. However, it often introduces additional complexity, such as managing callbacks, tracking state across operations, or dealing with eventual consistency.

Push Communication Pattern

When immediate delivery of information to the client is critical, the Push communication model becomes a powerful backend design choice. Unlike traditional request/response patterns, where the client initiates every interaction, push allows the server to actively send data to the client without being asked.

Why Not Always Use Request/Response?

Request/response patterns fall short in scenarios that require real-time communication. For example:

• Notifying users the moment someone logs in
• Delivering a new message as soon as it’s received

In such cases, constantly polling the server is inefficient. That’s where the push pattern is useful.

How Push Works

• The client establishes a connection with the server.
• The server pushes data to the client as soon as it’s available.
• The client remains passive — no need to send new requests.

This requires a bidirectional protocol (e.g., WebSockets), enabling the server to initiate communication.

For example, RabbitMQ supports push-based delivery to consumers.

Pros of Push

• True real-time experience
• Ideal for time-sensitive updates

Cons of Push

• Clients must stay online
• Not all clients can handle push load or state
• Requires a bidirectional communication protocol
• Less suitable for lightweight or battery-constrained devices

For clients with limited capabilities or intermittent connectivity, polling may still be the better fit.

Push Communication Pattern

In scenarios where immediacy is essential, like real-time notifications, chat applications, or collaborative editing, the traditional request-response model can become inefficient.

This is where the push communication pattern shines. It allows servers to proactively send data to clients without being asked, enabling responsive, real-time experiences that feel instantaneous.

Why Not Always Use Request/Response?

The request-response model, while robust and widely adopted, is inherently pull-based. The client must ask the server for updates, even if there’s nothing new to report. This constant polling introduces unnecessary load, latency, and resource usage, especially in situations where data changes frequently or unpredictably.

Imagine these scenarios:

• A user needs to be notified the moment someone sends them a message.
• A dashboard should update immediately when new data is available.
• A stock trading app must reflect market changes in real time.

Polling for such updates would be wasteful and often too slow. Push solves this by flipping the control.

How Push Communication Works

In a push model:

1. The client establishes a persistent connection with the server.
2. The server holds on to that connection.
3. When new data becomes available, the server pushes it down to the client.

This approach requires bidirectional communication — the server needs a way to initiate data transfer. Technologies like WebSockets, Server-Sent Events (SSE), or even HTTP/2 push enable this behavior. On the message queue side, systems like RabbitMQ also support push-based delivery, where consumers are notified as soon as messages are available.

In this setup, the client remains passive once the connection is established — there’s no need for repeated polling or round-trips to check for new data.

Advantages of Push

• Real-Time Updates: Perfect for use cases like live chat, collaborative tools, multiplayer games, or instant notifications.
• Efficient Communication: Reduces unnecessary traffic compared to polling, especially when updates are sporadic but time-sensitive.
• Better User Experience: Users get updates immediately, making apps feel fast and responsive.

Challenges and Trade-offs

Despite its strengths, push communication also comes with limitations:

• Persistent Connections: Clients must stay online and maintain a live connection. This isn’t always feasible for mobile or battery-constrained devices.
• Infrastructure Overhead: Managing long-lived connections for thousands (or millions) of clients requires robust infrastructure and careful resource management.
• Compatibility: Not all environments or clients support bidirectional protocols, particularly older browsers or restricted networks.

In these cases, hybrid models are sometimes used, combining long polling or periodic polling for less capable clients, while using push for others.

Long Polling

In many backend systems, you often encounter a trade-off between responsiveness and efficiency. Real-time push communication is ideal but not always feasible, either due to infrastructure limitations or client constraints. On the other hand, short polling (frequent, repeated client requests) creates unnecessary load and network noise. This is where long polling emerges as a practical alternative.

Why Long Polling?

Long polling is best suited for situations where an event or result might take a while, but you still want the client to stay informed as soon as something changes, without overwhelming the server with constant requests.

Typical scenarios include:

• Waiting for a background job to complete
• Tracking asynchronous user actions
• Monitoring message availability in a chat or queue

In these cases, short polling could introduce too much overhead, and full push might be overkill or unsupported. Long polling gives you the benefit of delayed responsiveness with fewer requests.

How Long Polling Works

The flow of long polling looks similar to regular polling, but with one crucial difference — the server holds the request open until it has something meaningful to return.

Here’s how it typically works:

1. The client sends a request to the server.
2. The server does not respond immediately; instead, it waits until the result is ready (or a timeout is reached).
3. Once the data or event is available, the server replies with the result.
4. The client processes the response and, if needed, initiates another long poll to wait for the next update.

Sometimes, this flow involves an intermediate job or task handler. For example, the client may first send a job and receive an ID. Then, it polls the server using that ID until the job is finished and a result is available.

To prevent indefinite waiting, most implementations set a timeout on the server or client side, after which the connection is closed and can be retried.

Where Long Polling is Used

Long polling has been a go-to solution in systems that needed pseudo-real-time capabilities before technologies like WebSockets became mainstream. It’s still relevant in many cases, especially when working with:

• Message queues like Kafka (where consumers may wait for new data)
• Legacy systems or constrained environments
• Browsers that don’t support more modern, persistent connection protocols

Benefits of Long Polling

• Reduces Noise: Fewer client requests compared to short polling means less network traffic and server load.
• More Scalable: It handles many clients more efficiently, especially when events are infrequent.
• Tolerant to Disconnects: If a client drops the connection, it can safely reconnect and continue polling.

Drawbacks to Consider

While more efficient than short polling, long polling still isn’t truly real-time. The client must initiate communication, and while the server waits to respond, some delay is still inherent. Additionally, it introduces complexity around timeout handling, connection management, and retry logic, especially under unreliable network conditions.

Long polling strikes a smart balance between real-time needs and technical constraints. It’s not as seamless as push, but it’s a pragmatic choice for many systems needing near-real-time updates without the architectural weight of full bidirectional protocols.

Server-Sent Events (SSE)

In a world increasingly shaped by real-time experiences — think chat apps, dashboards, or live notifications — backend systems must push updates to clients with minimal delay. While technologies like WebSockets offer powerful bidirectional communication, they also introduce complexity and overhead. Server-Sent Events (SSE) offer a simpler alternative — a real-time, one-way stream from server to client over plain HTTP.

Why Use Server-Sent Events?

Traditional request-response patterns break down when the server needs to notify the client of events as they happen. Consider scenarios like:

• A user logs in, and others should be notified instantly.
• A new message arrives in a group chat.

While polling can work, it’s inefficient. WebSockets solve the problem but often require custom protocol handling and infrastructure support. SSE hits the sweet spot as it’s a lightweight, event-driven mechanism built right into HTTP.

SSE uses plain HTTP, making it easy to implement and debug. Modern browsers provide native APIs like EventSource, that handle reconnects and parsing out of the box. Since it’s based on HTTP, it works well with common infrastructure like load balancers, proxies, and monitoring tools.

How SSE Works

SSE operates on a very simple principle: one HTTP request initiates a continuous stream of events from the server to the client.

Here’s the typical flow:

1. The client makes a standard HTTP GET request to an SSE endpoint.
2. The server keeps the connection open and responds with a stream of events over time.
3. Each message is sent in a special text/event-stream format, which the browser or client can continuously parse and handle.
4. The server never formally ends the response — it keeps sending updates as new data becomes available.

Each chunk of the response is a structured, self-contained event. For instance:

data: {"message": "New chat message"}

data: {"notification": "User X logged in"}

This makes SSE ideal for delivering continuous, event-driven updates in near real-time.

Limitations to Keep in Mind

SSE isn’t perfect for every use case:

• One-way only. Unlike WebSockets, clients can’t send messages back over the same stream.
• Persistent connection needed. If the client disconnects (e.g., poor mobile networks), the stream breaks, though most clients can automatically retry.
• Browser connection limits. Especially under HTTP/1.1, browsers often limit the number of simultaneous connections to the same server (usually 6). This can be a bottleneck when multiple tabs or services are open.
• HTTP/2 improves this significantly with connection multiplexing.

In short, Server-Sent Events offer a clean, minimal-effort way to stream events from backend to client — perfect for live feeds, dashboards, alerts, or system health checks. For many applications, SSE provides just enough “real-time” without the complexity of WebSockets or long polling.

Publish-Subscribe (Pub/Sub)

In modern distributed systems, it’s common for multiple services to react to the same event. For instance, when a user uploads a file, one service may store it, another might generate a thumbnail, and a third could log the activity. Coordinating this using traditional request-response patterns would lead to tightly coupled, hard-to-maintain systems.

Get AkashSDas’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

This is where Publish-Subscribe (Pub/Sub) shines.

What Is Pub/Sub?

At its core, Pub/Sub is a messaging pattern that enables one-to-many communication without requiring direct knowledge between senders and receivers.

• Publishers send messages to a topic.
• Subscribers listen to that topic and receive relevant messages.
• A broker (like Kafka, RabbitMQ, or Google Pub/Sub) handles the delivery, decoupling publishers from subscribers.

This makes Pub/Sub ideal for event-driven architectures, allowing systems to react asynchronously to changes.

Why Not Just Use Request-Response?

While the request-response model works well for simple client-server communication, it falls short in scenarios where:

• Multiple services need to consume the same message.
• Services should operate independently and not block each other.
• Message delivery must be resilient, even if some services are temporarily down.

With request-response:

• Systems become tightly coupled. Each service must know who to talk to.
• Scalability suffers as more receivers are added.
• Failures propagate; if one service is down, the whole workflow may stall.

Pub/Sub avoids these pitfalls by introducing a layer of indirection via the message broker.

How Pub/Sub Works

Here’s a high-level breakdown:

1. A publisher sends a message to a broker under a specific topic.
2. The broker stores or routes the message depending on the system.
3. One or more subscribers receive the message: Push model: Broker actively delivers messages (e.g., RabbitMQ). Pull/long polling: Subscribers poll the broker for messages (e.g., Kafka).

This decouples timing — publishers don’t wait for consumers. Messages can be delivered immediately or stored for later consumption.

Benefits of Pub/Sub

• Loose Coupling: Publishers and subscribers are unaware of each other.
• Horizontal Scalability: Multiple consumers can process the same event in parallel.
• Offline Tolerance: Subscribers can consume messages later if they were offline.
• Event-Driven Architecture: Encourages systems to react to events asynchronously.
• Extensibility: Adding new subscribers doesn’t require changing publisher logic.

Trade-Offs and Challenges

Despite its power, Pub/Sub introduces complexity:

• Reliability: Guarantees vary — messages might be lost or duplicated without proper configuration (e.g., the “Two Generals Problem”).
• Broker Overhead: Requires running and managing a broker service (Kafka, RabbitMQ, etc.).
• Delivery Semantics: Systems must deal with at-least-once, at-most-once, or exactly-once delivery models.
• Latency vs. Throughput: Polling-based systems can delay messages or cause network noise if not tuned correctly.

Whether you’re building a notification system, logging pipeline, or distributed task execution engine, Pub/Sub helps systems communicate cleanly without becoming entangled.

Multiplexing and Demultiplexing

Modern backend systems must strike a balance between performance, scalability, and efficient resource usage. Two foundational techniques — multiplexing and demultiplexing — play a key role in optimizing communication between services, especially in environments using HTTP/2, QUIC, connection pooling, or high-throughput APIs.

Although these are networking concepts at heart, they’re widely applied in backend architectures to manage connections, streamline data flow, and improve system responsiveness.

What Is Multiplexing?

Multiplexing is the process of combining multiple independent streams, such as HTTP requests, TCP sessions, or events, into a single shared connection or communication channel.

This allows several logical interactions to share the same physical connection, reducing overhead and improving throughput.

For example:

• HTTP/2 uses multiplexing to send multiple HTTP requests over a single TCP connection. Unlike HTTP/1.1, where each request often requires a new TCP connection or gets blocked behind others (head-of-line blocking), HTTP/2 interleaves streams, allowing them to flow concurrently.
• In backend systems, a proxy server may accept numerous client requests, multiplex them into one stream, and forward that stream to an upstream service, reducing connection management overhead.

Multiplexing improves resource utilization and latency under normal conditions, but it introduces complexity in parsing interleaved streams and handling shared flow control. A slow or congested stream can affect others sharing the same connection.

What Is Demultiplexing?

Demultiplexing is the reverse process — it separates a combined stream into its original, independent parts. This is essential when a backend service needs to route, process, or track requests individually.

For instance:

• An API gateway or load balancer might receive a multiplexed HTTP/2 stream and demultiplex it into discrete HTTP/1.1 requests, forwarding them to different microservices.
• A backend database layer may demultiplex incoming queries to assign them to individual connection pool threads or services.

Multiplexing and de-multiplexing in action in case of Chrome

Demultiplexed communication ensures better isolation between requests — failures or delays in one do not impact others. However, it requires managing more connections and threads, increasing resource consumption.

Why It Matters

The trade-off between multiplexing and demultiplexing is a classic backend design decision:

In multiplexed setups:

• All logical streams share the same connection and flow control.
• If one stream stalls or overloads the system, others suffer (e.g., shared TCP congestion window).
• Efficient for low-latency use cases with many short-lived interactions.

Example of proxy multiplexing request into one HTTP/2 connection

In demultiplexed setups:

• Each request or stream is handled independently, offering better isolation.
• Higher cost in terms of sockets, memory, and thread management.
• Favored when precise control, throughput, or reliability is needed.

Example of proxy de-multiplexing request into three HTTP/1.1 connections

Connection Pooling and Backend Integration

A practical extension of these concepts is connection pooling — a resource optimization strategy where clients or services reuse a pool of pre-established connections rather than opening a new one for every request.

• Used extensively in databases, message brokers, and backend APIs.
• If the pool is full, new requests wait for a connection to be released.
• Helps reduce connection overhead and improves latency under load.

In hybrid systems, backends may receive multiplexed HTTP/2 requests, which must be demultiplexed and routed through a connection pool (e.g., a pool of database connections). Balancing this workflow is key to high-throughput systems.

Example of connection pooling

Handling Multiple Queries Over a Single Connection: Sending multiple SQL queries over the same database connection introduces its challenges:

• For systems like PostgreSQL, results can arrive out of order.
• Without tagging each query or response, the backend may not match them correctly, leading to data corruption or logic errors
• PostgreSQL 14+ introduced query pipelining, allowing multiple queries on a single connection with ordered responses. This helps backend services optimize throughput while maintaining correctness.

Real-World Example: Browsers and HTTP/1.1 Limitations

Before HTTP/2, browsers had to deal with HTTP/1.1’s limitations:

• Browsers typically opened six parallel TCP connections per domain.
• Requests were demultiplexed across those six channels.
• If all were in use, new requests stalled, causing delays and degraded performance.

This example highlights the need for multiplexing at scale and the importance of understanding flow control and concurrency limits at each layer of a system.

Multiplexing and demultiplexing aren’t just low-level network strategies — they’re foundational to backend architecture. They influence API responsiveness, database performance, to cloud-native scalability, etc.

Stateful vs Stateless

In system design, the concepts of stateful and stateless architectures are often discussed in black-and-white terms. But in reality, the distinction is more nuanced. It’s not just about whether state exists, but how it’s managed, where it’s stored, and how much the system depends on it to function.

At its core, this dichotomy defines whether a server — or more generally, a system — remembers anything from previous interactions. And that has profound implications for scalability, fault tolerance, and user experience.

What Is a Stateful System?

A stateful system retains client-specific information in memory across requests. It relies on stored context to operate correctly. Every new interaction builds upon the previous one, making the system more tightly coupled with the client’s session.

For example, consider a web server that maintains a user’s session state in its local memory. If the server crashes or the user is routed to a different instance, that session context is lost, and the workflow breaks.

In stateful systems:

• Each client’s journey is tied to the server instance.
• Load balancers must use sticky sessions to route a client back to the same server.
• Scaling horizontally becomes more complex, as it requires replicating or synchronizing session state.

What Is a Stateless System?

In contrast, stateless systems do not store client context internally. Each request is self-contained, carrying all the information the server needs to fulfill it. The server doesn’t need to remember anything between requests.

Statelessness doesn’t mean there is no state at all — it simply means the system doesn’t depend on local, in-memory state. Any required data is either included in the request (e.g., session tokens, parameters) or stored externally in databases or caches.

For instance, if you restart a stateless backend, ongoing client sessions won’t break — because the backend isn’t holding any session-related data locally. This makes such systems naturally more resilient, scalable, and easy to maintain.

Stateless ≠ No State

It’s important to emphasize that stateless systems can still use state — they just manage it differently. They often offload state to external systems such as Redis for caching, PostgreSQL for storage, or S3 for file persistence. This makes restarting or scaling services safer since no session continuity is broken.

A good litmus test: If I restart the backend, will the client’s session still work? If the answer is yes, you’re working with a stateless system.

The Role of Load Balancers and Sticky Sessions

In stateful environments, load balancers need to support sticky sessions (also known as session affinity). This ensures that all requests from a particular user are routed to the same backend instance. Without this, the loss of session context can lead to unexpected behavior or data loss.

Stateless systems don’t have this problem. Since each request is independent, any backend instance can serve it, making horizontal scaling simpler and more effective.

Protocols: Are They Stateful or Stateless?

Just like applications, communication protocols can be either stateful or stateless:

• Stateful protocols (like TCP) maintain ongoing session state. They track sequence numbers, acknowledgments, and connection details throughout the interaction.
• Stateless protocols (like UDP) treat each message as a separate entity with no memory of previous communication. Any needed context must be included in the message itself.

Interestingly, you can build stateful systems on stateless protocols — for example, QUIC, which runs over UDP, maintains its own session state. Conversely, stateless systems can operate over stateful protocols, like HTTP over TCP, while treating each HTTP request as independent.

True Stateless Systems: Are They Possible?

While many systems call themselves stateless, truly stateless architectures are rare, but not impossible. In such systems, the server holds no session-related memory, and everything it needs is included in the request.

Examples include:

• A microservice that returns whether a given number is prime. All it needs is the number; no memory is required between calls.
• JWT (JSON Web Tokens), which encode all user authentication and metadata directly in the token. The server simply validates the token and proceeds — no sessions, no lookups, no stateful dependencies.

Understanding the difference between stateful and stateless systems isn’t just academic — it’s essential when designing resilient, scalable backend architectures. Whether you’re building APIs, managing sessions, or architecting microservices, how you manage state directly affects performance, reliability, and maintainability.

Sidecar Pattern

In modern backend architectures, managing communication between services is a challenge. With the growing variety of protocols — HTTP/1.1, HTTP/2, HTTP/3, gRPC, TLS, TCP — the application codebase often ends up bloated with networking logic, upgrades, and edge-case handling.

This is where the Sidecar Pattern comes in: it offloads protocol complexity to a dedicated proxy, keeping your core application lean, focused, and easier to maintain.

The Problem: Protocol Complexity in Your App

Every time your application needs to communicate over the network, it must understand and speak a particular protocol. Whether it’s HTTP/2’s multiplexing or TLS’s encryption handshakes, handling these correctly requires:

• Protocol-specific libraries (like OpenSSL, gRPC, libcurl, etc.)
• Low-level socket communication
• Constant upgrades and debugging

This results in thick clients and servers — applications overloaded with transport-layer concerns. For example, something as simple as axios.get() hides a lot of complexity: constructing headers, handling TLS handshakes, parsing HTTP responses, and managing retries.

Now imagine switching from HTTP/1.1 to HTTP/3, or adding mutual TLS (mTLS). That means integrating and testing new libraries, dealing with backward compatibility, and often rewriting parts of your codebase.

The Solution: The Sidecar Pattern

What if you could separate the communication layer from the application logic? That’s exactly what the Sidecar Pattern does.

In the Sidecar architecture:

• A proxy process runs alongside your service (often in the same container or pod).
• The app communicates with the proxy via simple, local HTTP (typically over localhost).
• The proxy takes care of the protocol complexity, including:
• Upgrading HTTP versions
• TLS encryption and mutual authentication
• Retries, timeouts, and circuit breaking
• Load balancing and service discovery

This setup allows your application to remain focused on business logic, while the sidecar handles everything related to transport and resilience.

Why Is It Called a “Sidecar”? Co-located (Runs in the same container, VM, or pod), Transparent (Communicates over localhost, invisible to the outside world), and Interchangeable (Can be upgraded or replaced independently of the core app).

How It Works: Client and Server Proxies

Both the client and server services have their own sidecars. When the client app makes a request:

• It talks to its local proxy using HTTP/1.1 or another simple protocol.
• The proxy upgrades the request to HTTP/2, gRPC, or another protocol and sends it over the network.
• The server-side proxy receives the request, demultiplexes it, and passes it to the server’s application over localhost.

This enables protocol changes without touching your core service logic. Want to switch from HTTP/2 to HTTP/3? Just upgrade the proxies.

Real-World Examples

Several service mesh tools use the Sidecar Pattern to manage service communication:

• Istio — Built on Envoy, feature-rich, widely adopted.
• Linkerd — Lightweight, written in Rust, designed for simplicity.
• Envoy — A standalone proxy used in many modern systems for observability, security, and routing.

These tools provide advanced features:

• Protocol negotiation and upgrades — Sidecars can automatically handle different protocol versions (like HTTP/1.1, HTTP/2, or gRPC) without requiring changes to the application. When a client and server use incompatible protocols, the sidecar proxies on both ends negotiate and upgrade the connection seamlessly, allowing smooth communication even in heterogeneous environments.
• TLS and mTLS encryption — The sidecar proxy manages Transport Layer Security (TLS) for both outbound and inbound communication. This includes support for mutual TLS (mTLS), where both client and server authenticate each other using certificates. This ensures secure, zero-trust communication across services without forcing every service to manage certificates or encryption logic.
• Tracing, logging, and metrics — Sidecars can inject tracing headers (like X-Request-ID or trace-id) into every request and collect detailed telemetry data. This provides out-of-the-box distributed tracing, structured logs, and performance metrics that help you monitor, debug, and optimize your services, especially useful in large-scale, microservices-based systems.
• Dynamic service discovery — Instead of hardcoding service addresses, sidecars integrate with service discovery systems (like Kubernetes DNS, Consul, or etcd). This allows proxies to automatically route requests to the correct service instance, even as services scale up, down, or move across the cluster. It enables true infrastructure-level flexibility.
• Retries, circuit breakers, and caching — Sidecars implement built-in reliability patterns to improve fault tolerance. Retries automatically re-attempt failed requests, circuit breakers prevent cascading failures by halting requests to unstable services, and caching can reduce load on frequently accessed endpoints. These policies are configurable at the proxy level — no code changes needed in the application.

Pros of the Sidecar Pattern

• Language Agnostic — Works with any language.
• Flexibility — Upgrade or switch protocols without touching app code.
• Security — Centralized TLS/mTLS with fine-grained control.
• Observability — Built-in support for logging, tracing.
• Resilience — Handles retries, timeouts, rate-limiting, and failovers.
• Service Discovery — Easily integrates with DNS or service registries

Cons of the Sidecar Pattern

• Increased Complexity — More moving parts — each service now has an extra process.
• Debugging Overhead — Network issues might be due to the proxy, not your app.
• Latency Cost — Slight overhead due to proxy hop and parsing layers.

The Sidecar Pattern is a powerful design strategy that abstracts communication concerns from your services. It helps you build systems that are secure, observant, and resilient, without turning every app into a networking expert.