Cache Invalidation: The Problem That Makes Senior Engineers Lose Sleep
TTL is not a strategy. It is a starting point. Here is what actually happens when your cache lies to your users. — By Sharad Bharadwaj
I once spent an entire Friday night debugging a production issue where users were seeing prices that had been updated six hours ago.
The cache was working perfectly. The database had the right data. The TTL was set to 3600 seconds, which the previous team had decided was “reasonable.” Nobody had questioned it since. The business had just run a flash sale, slashed prices by 40%, and thousands of users were checking out at the old price — the cache was serving it confidently, unaware that the world had moved on.
That night taught me something I wish someone had told me earlier: TTL is not a cache invalidation strategy. It is a best-effort guess about how long your data is probably still correct.
There is a famous quote in computer science — “There are only two hard things in Computer Science: cache invalidation and naming things.” Phil Karlton said it decades ago, and every engineer laughs when they first hear it. Then they hit production and stop laughing.
This blog is about why cache invalidation is genuinely hard, what TTL actually does and does not solve, and how to think through the trade-offs depending on what your system is actually doing.
Why We Cache in the First Place
Before getting into the problem, the reason we cache is worth stating plainly.
Your database is slow relative to memory. Not slow in absolute terms — modern databases are engineering marvels — but slow compared to serving a pre-computed response from Redis or Memcached. A typical Postgres query might take 15–50ms under normal load. A Redis hit takes under 1ms. At any meaningful scale, that difference collapses your infrastructure budget.
So we cache. We store a copy of the data closer to the application, serve requests from that copy, and skip the database most of the time.
The moment you have two copies of the same data, you have a consistency problem. The cache and the database can disagree. The question is never whether they will disagree — they will, always — the question is for how long, and whether your users will feel it.
That is the entire game.
What TTL Actually Does
TTL — Time-To-Live — is an expiry timer attached to a cache entry. You tell the cache: “hold onto this value for N seconds, then throw it away.” When the next request comes in after expiry, the cache misses, fetches fresh data from the database, and stores the new value.
Simple. Clean. Widely misunderstood.
Here is what TTL does not do: it does not know when your data changed. It has no awareness of the underlying database. It does not care that you updated a product price at 2:47 PM. It just counts down from whatever number you gave it, and expires when the clock runs out — not when the data became stale.
This is the fundamental mismatch. TTL is time-based. Staleness is event-based. You are solving an event-driven problem with a timer.
The Three Scenarios Where TTL Breaks Down
I have seen TTL fail in three distinct patterns. Each one has a different cause and a different fix.
Scenario 1 — The Flash Sale Problem (High-Frequency Updates)
This is what hit me that Friday night. When data changes faster than your TTL window, you are always serving old data.
An e-commerce product price sitting behind a 1-hour TTL is fine for normal operations. The moment marketing runs a time-sensitive promotion, that TTL becomes a liability. The data changed. The cache does not know. Users see the wrong price for up to an hour.
The instinct is to lower the TTL. Set it to 60 seconds instead of 3600. Problem solved, right?
Not quite. Now you are hitting your database 60x more often for every cached key. At any scale, that matters. You traded staleness for database pressure. The data is more accurate but your infrastructure bill went up and your latency under load got worse.
Lowering TTL is not a fix. It is just moving the dial between two bad outcomes.
Scenario 2 — The Thundering Herd (Mass Expiry)
Here is one that kills systems at 3 AM when nobody is watching.
If you cache a thousand product records with the same TTL — say, all set at midnight during a deployment — they all expire at the same time. Midnight plus TTL seconds, every single one of them falls off the cache simultaneously.
The next wave of requests hits. Every single one is a cache miss. Every one goes to the database. Your database, which was comfortably serving 200 queries per second from cache, suddenly gets hit with 5,000 simultaneous queries.
This is called the thundering herd. I have watched it take down two separate production databases. Both times the on-call engineer initially thought it was a DDoS. It was not. It was a cache deployment done at midnight where someone set identical TTLs for all records.
The fix is TTL jitter — adding randomness to your expiry times. Instead of TTL = 3600, you set TTL = 3600 + random(0, 300). Now your keys expire in a spread-out window. The herd thins. The database never sees a wall of requests all at once.
It is a small change. It has saved me from multiple incidents.
Scenario 3 — The Distributed Cache Inconsistency (Multi-Region Systems)
This one is subtle and it lives in the gap between architecture diagrams and reality.
Get Sharad Bharadwaj’s stories in your inbox
Join Medium for free to get updates from this writer.
When you run a multi-region setup — say, US-East and EU-West both have their own Redis clusters — and a user updates their account details in US-East, the EU-West cache still has the old value. Both caches have their own TTLs ticking independently.
A user who updates their billing address in the US and immediately logs in from a VPN resolving to EU-West will see their old address for up to however long your TTL is. In most cases, users do not cross regions mid-session. But in some systems — financial dashboards, user profile updates, anything security-sensitive — this inconsistency window is a real problem.
TTL alone cannot fix this because TTL has no awareness of what other cache nodes hold. You need a cross-region invalidation mechanism. TTL is just the safety net that cleans up whatever the invalidation signal missed.
The Architect’s Decision Framework for TTL
Over the years I have settled into a mental model that I use every time someone asks “what should our TTL be?”
The answer always starts with the same question: what is the business cost of serving stale data, and for how long?
Let me walk through each branch.
Long TTL — when staleness is cheap. A user’s avatar, a blog post body, a list of country codes, a product category tree that changes once a quarter. None of this is time-sensitive. Set a long TTL, add jitter, and forget about it. The cost of a user seeing a slightly old avatar is zero.
Medium TTL with write-through — when data changes occasionally. A user’s display name, a product description, an article title. These change sometimes but not constantly. A sensible TTL combined with explicitly invalidating or updating the cache on every write covers most cases. When the write happens, you either push the new value into cache immediately (write-through) or delete the key and let the next read repopulate it (cache-aside with active deletion).
Active invalidation — when staleness has a real cost. Product prices during sales, account balances, inventory counts, session tokens, feature flags. These are situations where the TTL is just a safety net — the real mechanism is your application explicitly telling the cache “this key is now wrong, throw it away” every time the underlying data changes.
Active Invalidation vs. TTL-Only: The Real Trade-off
This is where most architecture discussions get muddy, so I want to be direct about it.
TTL-only is simple. No extra infrastructure. No event publishing. No invalidation logic in your application. You set a number, you move on. The cost is that your cache is stale for up to TTL seconds after every write. Whether that is acceptable depends entirely on your data.
Active invalidation is precise. The moment data changes, the cache reflects it. The cost is complexity: you need a reliable mechanism to publish invalidation signals, every write path must participate, and you have to handle failure cases where the invalidation signal gets lost.
Here is the failure case nobody talks about. In active invalidation, the write to your database succeeds but the cache invalidation message gets dropped — network blip, Redis connection timeout, whatever. Now your database has fresh data and your cache has stale data with no TTL expiry coming for another 50 minutes. You are worse off than pure TTL because at least TTL would have eventually cleaned it up.
This is why you always use active invalidation plus TTL as a fallback, never active invalidation alone.
The Write-Through and Write-Behind Patterns
Two caching write strategies come up constantly in this discussion, and they solve different parts of the problem.
Write-through: When your application writes to the database, it also writes to the cache at the same time. The cache is always warm. No cold misses on fresh writes. The trade-off is that every write goes to two places, which adds latency and creates a two-write atomicity problem — what if the database write succeeds and the cache write fails?
Write-behind (write-back): Your application writes to the cache first, returns success to the user, and asynchronously flushes to the database later. Extremely fast for write-heavy workloads. The trade-off is data loss risk — if your cache dies before it flushes, that write is gone. This is appropriate for analytics counters and session data, never for anything you cannot afford to lose.
Most production systems I have built use cache-aside with active invalidation on writes. Write-through is viable for user-facing data that is read immediately after being written. Write-behind I have only used for metrics pipelines where occasional data loss is genuinely acceptable.
What TTL Should Actually Be Set To
I want to give concrete numbers because architecture blogs that avoid numbers are not actually useful.
The Lesson I Carry from That Friday Night
What that flash sale incident taught me was not “use shorter TTLs.” It was that nobody on the team had ever explicitly asked the question: “what happens to our cache when the business needs to change data urgently?”
The answer was sitting there waiting: nothing good. The cache would keep serving old prices for up to an hour, because the TTL was designed around normal operations, not emergency ones.
After that night we built an explicit cache invalidation endpoint that the pricing service would call on every price change. TTL stayed at 3600 seconds as a fallback. The invalidation signal was the primary mechanism. We also added monitoring to alert if a high-velocity write event — more than 50 price changes in 5 minutes — did not produce a corresponding spike in cache invalidation calls. That alert would catch the case where the invalidation pipeline silently failed.
Two layers of defense. TTL for normal drift. Active invalidation for known writes. Monitoring to catch gaps in both.
That is the architecture I have run in production ever since, across different stacks, different cloud providers, different scale levels. The specifics change. The pattern holds.
Final Thought
Cache invalidation is hard not because the technical mechanisms are complicated — they are not. It is hard because it forces you to reason about time, data ownership, failure modes, and business impact simultaneously.
TTL is seductive because it feels like it solves the problem. Set a number, the cache refreshes itself, everything is fine. And for a lot of data, it actually is fine. The danger is assuming it is always fine, without asking what your staleness window actually costs.
The engineers who get this right are not the ones who find the perfect TTL. They are the ones who know exactly which data in their system is worth the complexity of active invalidation, and they have the monitoring in place to know when either mechanism is failing silently.
Everything else is just tuning dials until the production alert stops.
