Handling Cache Stampede in High-Traffic Systems (with Redis)

In large-scale, high-traffic systems, cache (for example, Redis) is often seen as the performance “savior.” But with the wrong patterns, cache can actually become the source of serious problems: cache stampede. This issue often goes unnoticed until the system starts to struggle, latency spikes, and database CPU hits 100%.

This article explains how to understand and handle cache stampede, especially in distributed systems using Redis, based on real experience with hundreds of thousands of requests per minute.

What Is a Cache Stampede?

In simple terms, cache stampede happens when:

1. A cache key expires or is considered invalid.
2. Many requests arrive at the same time for that same key.
3. Because the cache is empty, all those requests hit the database (or another expensive data source) simultaneously.
4. This creates a burst in QPS (queries per second) to the database, CPU usage spikes, latency explodes, and it may trigger downtime.

In smaller systems, the effect might not be noticeable. But in high-traffic systems with many instances (20–50 service instances, autoscaling, load balancers, etc.), this pattern can show up as:

• Small database spikes every few seconds,
• Occasionally strange latency,
• A system that looks fine… until it’s not.

The core point: Redis only stores data. It does not automatically prevent 50 workers from hammering the database at the same time when the cache is empty or expired.

A Real-World Example

In one of the systems discussed:

• Only one Redis key expired.
• There were about 40 instances of a Go service.
• As soon as the key expired, all 40 instances fired 40 expensive queries to the database within about ±200 ms.
• Under certain traffic patterns, a single key like this could cause hundreds of expensive queries in 200 ms.

If this pattern repeats every few seconds, the system is actually fragile, even if it appears to be running fine.

Core Strategies to Handle Cache Stampede

Three key strategies:

1. Cache with Mutex / Single-Flight Pattern
2. Logical Expiration (Logical Expiry)
3. Staggered Expiration (TTL with jitter)

Let’s go through them one by one.

1. Cache with Mutex / Single-Flight Pattern

Main idea:
When many requests ask for the same key and that key has already expired:

Only one request is allowed to hit the database and refresh the cache. The other requests wait for that single “representative.”

Typical Implementation

• In Go, you can use the singleflight package (from golang.org/x/sync/singleflight).
• With Redis, you can implement a simple lock using:

SET key value NX EX <seconds>

Explanation:

SET ... NX = Set If Not Exists
Redis will only create the key if it doesn’t exist yet. That means:

• The first request that succeeds in setting the lock becomes the “representative” that runs the database query.
• Other requests fail to get the lock and thus wait for the result from the representative.
• EX <seconds> = TTL for the lock, e.g., 3 seconds.
  This is important to prevent the lock from hanging forever if the instance that holds the lock crashes.

What’s a Good Lock TTL?

Principles:

• Longer than the average time it takes to refresh the cache.
• Shorter than the request timeout of your service.

In practice:

• 1–5 seconds is usually reasonable.
• A very common choice is 3 seconds:
• Long enough to finish a heavy query + set the cache.
• Short enough to avoid deadlock if something fails.
• Safe for high traffic so requests don’t wait too long.

A practical rule:

• Choose lock duration so that
  (\text{lock_TTL} > \text{average cache refresh time})
  and
  (\text{lock_TTL} < \text{service request timeout}).

2. Logical Expiration (Logical Expiry)

This strategy is especially useful for expensive data such as:

• Machine learning recommendations,
• Aggregations across several tables,
• Complex heavyweight queries.

Concept of Logical Expiration

Instead of storing only the value in Redis, store two fields:

1. The data value,
2. An expired_at timestamp (when the data should be considered stale).

When a request comes in:

1. If now < expired_at
   → The data is still fresh → return the cached value.
2. If now >= expired_at
   → The data is logically expired, but:

• Still return the old data to the user (so the user doesn’t wait), and
• Trigger a background refresh of the data (via goroutine, job, or worker).

The effect:

• Users receive responses quickly, even when data is technically expired.
• The system refreshes data gradually in the background, rather than all at once.
• This is highly effective for expensive data that doesn’t need to be “ultra real-time.”

3. Staggered Expiration (TTL with Jitter)

The common problem:
If you have millions of users and you set every cache entry to the same TTL, say 5 minutes, then:

• Many keys will expire at roughly the same time,
• A burst of cache refreshes happens at once,
• This can trigger huge database spikes.

Solution: add jitter / randomness to TTL.

Example:

• Base TTL: 300 seconds (5 minutes)
• Add random jitter: 0–60 seconds

Final TTL = 300 + random(0–60)

Get Arif Rahman’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

Impact:

• Key expirations are spread out over a window of 5–6 minutes.
• Cache refresh load is more evenly distributed.
• Database spikes can be dramatically reduced (in the e-commerce example from the video, by around ±80%).

Common Mistakes in Cache Implementation

Besides the three strategies above, several anti-patterns often make systems fragile:

1. Assuming “Using Redis = Problem Solved”

Redis is just one component of a caching solution. If:

• Your expiry pattern is wrong,
• TTLs are arbitrary,
• You have no mechanism to prevent stampede,

then you’re only delaying the problem, not solving it.

2. Arbitrary / Too-Short Cache TTL

TTL that’s too short:

• Forces frequent cache expiry,
• Causes more frequent database hits,
• Increases the risk of cache stampede.

A More Reasonable Way to Choose TTL

TTL should be:

1. Longer than the update frequency of the data
   (if data changes every 1 minute, don’t cache it for only 10 seconds).
2. Longer than the average time required to recompute/refresh the data.
3. Wider than typical traffic spike intervals.

Common practice:

• TTL = 3–10× the average time of the query being cached.

Example:

• The query takes ~200 ms on average.
• TTL of 1–2 seconds is too short (you’ll refresh it constantly).
• A better TTL is several seconds up to minutes, depending on how frequently the data changes.

Additional principle:

• TTL should follow the cost and volatility of data:
• Expensive-to-compute data → longer TTL,
• Frequently changing data → shorter TTL,
• Always add jitter (staggered TTL).

3. Using Redlock Everywhere

Redlock is a distributed lock algorithm across multiple Redis nodes, designed by Redis’s creator.

Its purpose:

• A lock is considered acquired only if it’s successfully obtained on a majority of Redis nodes.
• Suitable for highly critical operations that require strong consistency across a multi-node Redis setup.

However:

• It’s not a silver bullet.
• It depends heavily on stable, low-latency network conditions.

If the network:

• Sometimes has 2 ms latency and sometimes 80 ms,
• Some nodes respond slowly,
• Clients receive lock confirmations late,

then you may experience:

• Locks that appear to fail even though they actually succeeded, or
• Even worse: locks that appear valid even after they’ve effectively expired.

This can become a new source of inconsistency.

Recommendations:

• Use Redlock only for very critical operations.
• Don’t blindly apply it to every cache refresh.
• Understand your network and Redis cluster characteristics before relying on Redlock.

4. Running Heavy Queries Directly on Cache Miss

A common pattern:

• In an API handler:
• Check cache → miss,
• Immediately run a heavy query (joins multiple tables, heavy aggregation),
• Or call an external API with strict rate limits.

Without protection, during a stampede:

• Dozens of instances might simultaneously:
• Execute heavy queries against the database, or
• Hammer an external API until you hit rate limits, causing widespread errors.

Extra Protection for Heavy Queries

At minimum, do three things:

1. Single-Flight / Mutex Lock
   Ensure that only one goroutine/worker runs the heavy query when there is a cache miss.
2. Fallback to Old Data
   If refresh fails or takes too long, use:

• Old data from cache (logical expiration), or
• Another lighter fallback.

3. Rate Limiting / Token Bucket for External APIs
For external services:

• Add a rate limiter or token bucket,
• Make sure your caching mechanism doesn’t cause your system to flood the external API.

Final Summary

Cache stampede is not a minor issue. It’s often the hidden reason behind:

• Sudden database spikes,
• Occasionally weird latency,
• Systems crashing during traffic surges even when business logic seems normal.

Key points to remember:

1. Redis is not an automatic savior
   The patterns you use (TTL, locks, expiry handling, etc.) determine whether your system is solid or fragile.
2. Three main strategies to handle cache stampede:

• Cache with mutex / single-flight
  Only one request hits the database to refresh; others wait.
• Logical expiration
  Return old data to users; refresh in the background.
• Staggered TTL (TTL + jitter)
  Spread out expirations to avoid synchronized spikes.

3. Avoid common mistakes:

• Arbitrary or too-short TTLs,
• Blindly relying on Redlock without understanding your network,
• Firing heavy DB queries or external API calls directly on every cache miss with no protection.

Implement these three strategies correctly, and your system will be much more ready to scale and handle high traffic without your monitoring dashboard giving you a heart attack every time traffic climbs.