Cache Stampede — When One Expired Key Takes Down Your Database
There’s something strange about performance bugs.
Sometimes, your application works perfectly for hours. Response times are fast, the database is healthy, and nobody is complaining.
Then suddenly, everything slows down.
The database CPU spikes. Requests start timing out. And your monitoring dashboard looks like something went terribly wrong.
But the trigger might be surprisingly small.
A single cache key expired.
Imagine an e-commerce website where thousands of users are viewing the same popular product. To avoid querying the database every time, the application stores the product details in a cache.
As long as the cached data exists, everything is fine.
But when that cache entry expires, the next request has to fetch the data from the database.
Now imagine a thousand requests arriving at almost exactly the same moment.
Instead of one request refreshing the cache, all thousand requests discover that the data is missing. They all query the database, and they may all try to rebuild the same cache entry.
Get thatqagirl’s stories in your inbox
Join Medium for free to get updates from this writer.
That’s a cache stampede.
The cache was supposed to protect the database. Instead, its expiration creates a sudden wave of traffic that overwhelms it.
And the frustrating part? The application might pass every test before this happens.
A basic test checks whether the cache works. Another checks whether expired data gets refreshed. Both pass.
But what happens when hundreds of requests hit the same expired key simultaneously?
That’s where testing needs to become more realistic.
I’d want to test what happens when a popular cache entry expires under load, whether multiple requests trigger duplicate database queries, and whether the database remains responsive while the cache is being rebuilt.
There are several ways to reduce this risk. A system can allow only one request to refresh a particular key while others wait, serve slightly stale data while refreshing in the background, or spread expiration times so thousands of entries don’t expire together.
But each approach introduces its own behavior to verify.
For example, if the application serves stale data during a refresh, how stale is acceptable? If one request owns the refresh lock and then crashes, can other requests recover? And if the database is already struggling, does the cache-refresh mechanism make things worse?
These are the questions that separate a cache that works in isolation from one that survives real traffic.
The lesson for QA is simple: don’t just test whether a cache expires. Test what your entire system does when it expires at the worst possible moment.
Because sometimes, the biggest outage doesn’t start with a complicated failure.
It starts with a timer reaching zero.
