<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Samiran De  on Medium]]></title>
        <description><![CDATA[Stories by Samiran De  on Medium]]></description>
        <link>https://medium.com/@sam.de.721166?source=rss-368a9cb37eff------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*p0sv0r9C7gm3hTOne2ijJw.jpeg</url>
            <title>Stories by Samiran De  on Medium</title>
            <link>https://medium.com/@sam.de.721166?source=rss-368a9cb37eff------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 17:49:39 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@sam.de.721166/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Redis from first principle: Everything I Wish I Knew Before Using It]]></title>
            <link>https://medium.com/@sam.de.721166/redis-from-first-principle-everything-i-wish-i-knew-before-using-it-a59e2cd16a74?source=rss-368a9cb37eff------2</link>
            <guid isPermaLink="false">https://medium.com/p/a59e2cd16a74</guid>
            <category><![CDATA[software-engineering]]></category>
            <category><![CDATA[redis]]></category>
            <category><![CDATA[caching]]></category>
            <category><![CDATA[web-development]]></category>
            <category><![CDATA[backend]]></category>
            <dc:creator><![CDATA[Samiran De ]]></dc:creator>
            <pubDate>Sun, 18 Oct 2026 16:42:17 GMT</pubDate>
            <atom:updated>2026-10-18T16:42:17.738Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*eDnerBh1O_wmC6RXT_VXNQ.png" /></figure><p>When I first started learning backend development, I kept seeing Redis mentioned everywhere. Caching, rate limiting, session management, queues apparently, Redis could do a little bit of everything.</p><p>But I had one question: <strong>“Why do we even need Redis when we already have databases like MongoDB and PostgreSQL?”</strong></p><p>So let’s start from the very beginning</p><h3>1. Why does Redis exist?</h3><p>Before understanding Redis, we need to understand why we use this weird thing called “Redis”</p><p>let’s suppose a problem that appears in almost every backend application: <strong>retrieving and processing data costs time and resources.</strong></p><p>Imagine you’re building a website. Your website displays a list of products. When a user opens the products page, the frontend sends a request to your backend. Your backend queries MongoDB and returns the results.</p><p>The architecture looks like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*b9SVW5rDian66R-rxQbMMg.png" /></figure><p>A simple implementation might look like this:</p><pre>app.get(&quot;/products&quot;, async (req, res) =&gt; {<br>  const products = await Product.find();<br><br>  res.json(products);<br>});</pre><p>Nothing is wrong with this code. For a small application, it might be perfectly works.</p><p>But think if your application grows to 10k, 50k, 100K+ users. Your backend keeps asking database for the same product data, even when the data hasn’t changed. You start asking yourself: <strong>“Why do I need to retrieve the same information repeatedly if I already retrieved it a moment ago?”</strong></p><p>That’s the problem that introduces us to caching.</p><h3>1.1 Now what is caching?</h3><p>Caching means keeping a copy of data that you expect to reuse, allowing your application to avoid repeating an expensive operation.</p><p>Think about opening a website you’ve visited before. Your browser may reuse cached images and other resources instead of downloading everything again. That’s one form of caching. Backend applications use the same general idea. Instead of querying the database for every request, your backend can reuse a previously retrieved result.</p><h3>1.2 But don’t databases already use caching?</h3><p>good question.</p><p>MongoDB and PostgreSQL both use memory to improve performance. They have their own caching and query optimization mechanisms. Therefore, the problem isn’t simply that databases are slow or that they always retrieve everything from disk.</p><p>The actual question is: “<strong>Can I avoid repeating expensive database queries or computations by storing reusable results somewhere convenient<em>?</em></strong><em>”</em></p><p>For example, consider these application features:</p><ul><li>A homepage displaying the same promotional banner to everyone.</li><li>A product catalogue that changes only occasionally.</li><li>An analytics dashboard that calculates expensive aggregate results.</li><li>A public API endpoint receiving thousands of identical requests.</li><li>A leaderboard that repeatedly retrieves ranked scores.</li></ul><h3>1.3 Why not store everything in application memory?</h3><p>You could use a JavaScript object or a Map.</p><p>For example:</p><pre>const cache = new Map();<br><br>cache.set(&quot;theme&quot;, &quot;dark&quot;);<br><br>console.log(cache.get(&quot;theme&quot;));</pre><p>This works.</p><p>So why introduce another technology?</p><p>Consider a backend running on three server instances.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xmbYsgWaLGTo-WmRsWDzdQ.png" /></figure><p>Each instance has its own memory. If API 1 stores a value in its local Map, API 2 and API 3 cannot automatically access that value.</p><p>Redis provides a separate data store that multiple application instances can communicate with.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*B18uehzhiEPQ0gDlUhRAvw.png" /></figure><p>Well now you probably know why we use redis, now let’s talk about it a more</p><h3>2. What exactly is Redis?</h3><p>Redis is an in-memory data store that supports several data structures and operations for working with them.</p><h3>2.1 Is Redis a database or a cache?</h3><p>Both descriptions can be correct, depending on how it’s being used.<strong> Redis is a data store.</strong></p><p>When an application uses Redis to keep temporary copies of records from MongoDB, Redis is acting as a <strong>cache</strong>. When an application stores its primary data in Redis, Redis is acting as a primary <strong>data store</strong>.</p><p>These architectures have different requirements. For a disposable cache, losing the data might simply mean fetching it again from MongoDB. For a primary data store, persistence, replication, recovery, and durability become much more important.</p><h3>3. How does Redis actually store data?</h3><p>Let’s start with the simplest possible example. Imagine a dictionary. You look up a word and retrieve its meaning.</p><p>Redis uses a similar key-value model. Each key identifies a value. You can retrieve that value without performing a traditional database query across an entire collection.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*utWQS1xZXfiee8GC188UcA.png" /></figure><h3>3.1 Understanding Redis Data Types</h3><p>One of the first things that confused me about Redis was that it wasn’t just a database where you stored JSON documents or rows in tables.</p><p>Redis supports different data types, and each type is designed for a particular kind of problem. You don’t need to learn every Redis command to use it effectively. Start with the data types you’ll encounter most often.</p><h4>3.1.1 Strings: The simplest and most commonly used type</h4><p>A Redis string can store text, numbers, JSON, or other binary data. For example, imagine you’re building a blog application. Every time someone opens a popular article, your backend fetches it from MongoDB.</p><p>Instead of querying MongoDB every time, you can cache the article in Redis.</p><pre>SET article:101 &#39;{&quot;title&quot;:&quot;Getting Started with Redis&quot;,&quot;views&quot;:120}&#39;<br><br>GET article:101</pre><p>The key is article:101, and its value is the article data serialized as a JSON string.</p><p>In a real application, you’d usually use your programming language to serialize and deserialize the data rather than writing JSON manually.</p><p><strong>Where are strings useful?</strong></p><ul><li>Caching API responses.</li><li>Storing temporary verification codes.</li><li>Maintaining counters.</li><li>Storing feature flags or small configuration values.</li><li>Implementing simple rate-limiting counters.</li></ul><p>One important detail: use expiration for temporary values whenever appropriate.</p><pre>SET otp:user123 482916 EX 300</pre><p>Here this “<strong>EX</strong>” means this stores a value which is an OTP here for 3<strong>00 seconds</strong>, or five minutes.</p><p>For a real OTP system, you should also implement <strong>attempt limits</strong>, <strong>secure generation</strong>, and <strong>appropriate verification rules</strong>. Redis expiration alone doesn’t make an OTP system secure.</p><h4>3.1.2 Hashes: Storing related fields together</h4><p>Redis hashes let you store multiple fields under one key.</p><p>For example:</p><pre>HSET user:42 name &quot;Sam&quot; role &quot;developer&quot; plan &quot;free&quot;<br>HGET user:42 name<br>HGETALL user:42</pre><p>Conceptually, the data looks like this:</p><p>user:42<br>├── name: Sam<br>├── role: developer<br>└── plan: free</p><p>You can retrieve or update individual fields without replacing the entire value. This can be useful for compact application state, counters, and objects whose fields you access independently.</p><p>However, <strong>a Redis hash is not automatically a replacement for a MongoDB document or a PostgreSQL table</strong>. Redis doesn’t provide the same relational queries, constraints, or general-purpose persistence model as those databases.</p><p>For example, if your application needs to find every user whose email ends in @xyz.com, MongoDB or PostgreSQL is the appropriate place to perform that query.</p><p>Use Redis hashes when you have a specific reason to represent data that way, not simply because they exist.</p><h4>3.1.3 Sets: Keeping unique values</h4><p>A Redis set stores unique members.</p><pre>SADD post:101:likes user:1 user:2 user:3<br>SADD post:101:likes user:2<br>SCARD post:101:likes<br>SISMEMBER post:101:likes user:2</pre><p>The set contains three unique users, even though user:2 was added twice.</p><ul><li><strong>SADD</strong> adds members.</li><li><strong>SCARD</strong> returns the number of members.</li><li><strong>SISMEMBER</strong> checks whether a member exists.</li></ul><p>This is useful when you need to represent membership or uniqueness.</p><p>For example:</p><ul><li>Users who liked a post.</li><li>Users who joined a group.</li><li>IDs of users who have already received a particular notification.</li></ul><p>Be careful, though. If likes are important application records, Redis shouldn’t necessarily be their only storage location. A permanent likes system may need durable database records, transactional behavior, and reliable recovery.</p><h4>3.1.4 Sorted Sets: Ranking things by score</h4><p>Sorted sets are especially useful because each member has an associated score, and Redis keeps the members ordered by that score.</p><p>Suppose you’re building a leaderboard for your application.</p><pre>ZADD leaderboard 120 user:1<br>ZADD leaderboard 250 user:2<br>ZADD leaderboard 180 user:3</pre><p>Each user has a score.</p><p>To retrieve the highest-scoring users:</p><pre>ZREVRANGE leaderboard 0 9 WITHSCORES</pre><p>This returns up to ten users, starting with the highest score.</p><p>To update a user’s score:</p><pre>ZINCRBY leaderboard 10 user:1</pre><p>Redis increases the user’s score by ten. This is much more convenient than repeatedly fetching every user from a database and sorting them in your application which is more expensive operatioon and taek much more time to execute.</p><p><strong>Real-world use cases:</strong></p><ul><li>Gaming leaderboards.</li><li>Top-performing posts.</li><li>Trending content.</li><li>Ranked search results.</li><li>Time-sensitive ranking systems.</li></ul><h4>3.1.5 What about Lists and Streams?</h4><p>yeah Redis also supports lists and streams. At this point this weird thing can do a lot more than normally people think about it.</p><p><strong>Lists</strong> are ordered collections that can be useful for simple queues and recent-item lists. For example, you could maintain a list of recently viewed products.</p><p><strong>Streams</strong> are append oriented data structures designed for event records and stream-processing workflows. They support consumer groups, which allow multiple workers to process entries from a stream.</p><p><strong>For most beginners, strings, hashes, sets, and sorted sets are enough to start building useful applications.</strong></p><p>( Learn lists and streams when a project actually calls for them. For reliable background-job processing in Node.js, a library such as <strong>BullMQ</strong> is often a more practical starting point than implementing your own queue directly with Redis commands. )</p><h3>4. How Redis Makes Applications Faster</h3><p>Enough talking about data types and how it stores data</p><p>Now let’s see the interesting part that how is this thing make your application 10X faster.</p><p>Imagine you have a blog with a popular article. Every time someone visits that article, your backend executes a database query:</p><pre>const article = await Article.findById(articleId);</pre><p>That query might be fast enough for a small application. But what happens when thousands of people request the same article?</p><p>Your database may repeatedly perform the same work, even though the article hasn’t changed.</p><p>This is where caching helps.</p><h4>The basic idea</h4><p>Instead of fetching the article from MongoDB on every request, your backend can follow this process:</p><ol><li>Check whether the article exists in Redis.</li><li>If it exists, return the cached data.</li><li>If it doesn’t, fetch the article from MongoDB.</li><li>Store the result in Redis.</li><li>Return the result to the user.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*dksSbUPCdK_jJm0mWtvq-w.png" /><figcaption><strong>Cache miss</strong> (when data is not found in redis and need to fetch from db)</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*iZzeeodRoNhOarvvU1Bsdg.png" /><figcaption><strong>Cache hit</strong> (when data is found in redis &amp; directly send response)</figcaption></figure><p>The first request might take longer because it needs to fetch and cache the data. Later requests may be faster because the result is already available.</p><p>However, caching doesn’t automatically make every application faster. If your database query is already extremely cheap, or your cache hit rate is low, Redis may add overhead without delivering much benefit.</p><p><strong>The goal isn’t to add Redis everywhere. It’s to reduce expensive, repeated work.</strong></p><h4>Here are some most used Redis methods :</h4><p><strong>set</strong> : set one data in redis</p><p><strong>get</strong> : get one data in redis</p><p><strong>del</strong> : delete data in redis</p><p><strong>ttl</strong>: to see the expiry time of the craeted element in redis</p><p><strong>hset</strong> :set object of data ( multiple updatable data in redis )</p><p><strong>hgetall</strong> : get all data</p><h3>5. Building a Real Cache with Node.js, Express, MongoDB, and Redis</h3><p>Ok, enough yapping about Redis, now let’s implement it in real world scenarios :</p><p>Imagine we have a product API:</p><pre>GET /products/101</pre><p>MongoDB stores the product, and Redis caches it.</p><p>We’ll use ioredis to communicate with Redis.</p><h4>Step 1: Install the dependencies</h4><pre>npm install express ioredis mongoose dotenv</pre><p>Make sure MongoDB and Redis are running and configure their connection strings in your environment.</p><p>For example, your .env file might contain:</p><pre>PORT=3000<br>MONGODB_URI=mongodb://127.0.0.1:27017/redis_demo<br>REDIS_URL=redis://127.0.0.1:6379</pre><p>These are local development values. In production, use the connection details provided by your database and Redis services.</p><p>Never commit secrets or production credentials to your repository.</p><h4>Step 2: Connect to MongoDB and Redis</h4><pre>import express from &quot;express&quot;;<br>import mongoose from &quot;mongoose&quot;;<br>import Redis from &quot;ioredis&quot;;<br>import dotenv from &quot;dotenv&quot;;<br><br>dotenv.config();<br><br>const app = express();<br>app.use(express.json());<br><br>const redis = new Redis(process.env.REDIS_URL);<br><br>const Product = mongoose.model(<br>  &quot;Product&quot;,<br>  new mongoose.Schema({<br>    name: String,<br>    price: Number,<br>  })<br>);<br><br>await mongoose.connect(process.env.MONGODB_URI);</pre><p>This example assumes your project uses ES modules.</p><p>The Redis client maintains a connection to Redis. In a larger application, you would generally create the client once and reuse it rather than creating a new connection for each request.</p><h4>Step 3: Implement the cache-aside pattern</h4><pre>app.get(&quot;/products/:id&quot;, async (req, res) =&gt; {<br>  const { id } = req.params;<br><br>  // creating an id for redis<br>  const cacheKey = `product:${id}`;<br><br>  try {<br>    // 1. Check Redis first<br>    const cachedProduct = await redis.get(cacheKey);<br><br>    if (cachedProduct !== null) {<br>      return res.json({<br>        source: &quot;cache&quot;,<br>        product: JSON.parse(cachedProduct),<br>      });<br>    }<br><br>    // 2. Cache miss: query MongoDB<br>    const product = await Product.findById(id).lean();<br><br>    if (!product) {<br>      return res.status(404).json({<br>        message: &quot;Product not found&quot;,<br>      });<br>    }<br><br>    // 3. Store the result in Redis for 5 minutes<br>    await redis.set(<br>      cacheKey,<br>      JSON.stringify(product),<br>      &quot;EX&quot;,<br>      300<br>    );<br><br>    // 4. Return the product<br>    return res.json({<br>      source: &quot;database&quot;,<br>      product,<br>    });<br>  } catch (error) {<br>    console.error(&quot;Product request failed:&quot;, error);<br><br>    return res.status(500).json({<br>      message: &quot;Internal server error&quot;,<br>    });<br>  }<br>});</pre><p>And start the server:</p><pre>app.listen(process.env.PORT || 3000, () =&gt; {<br>  console.log(&quot;Server is running&quot;);<br>});</pre><p>For a beginner’s local project, this demonstrates the core pattern. There is an important production improvement to make before relying on it, which we’ll cover shortly.</p><h4>Understanding the important parts</h4><ol><li><strong>redis.get(cacheKey)</strong></li></ol><p>The application checks Redis before querying MongoDB.</p><p>The key might look like this: <strong>product:101</strong></p><p>Using consistent key names makes your Redis data easier to understand and maintain.</p><p><strong>2. </strong><strong>JSON.parse(cachedProduct)</strong></p><p>Redis returns the cached value as a string. Since we stored the product using JSON.stringify(), we need to parse it back into a JavaScript object.</p><p><strong>3. </strong><strong>Product.findById(id).lean()</strong></p><p>Mongoose’s lean() returns a plain JavaScript object instead of a full Mongoose document.</p><p>For a read-only response like this, that can reduce unnecessary processing.</p><p><strong>4. </strong><strong>redis.set(..., &quot;EX&quot;, 300)</strong></p><p>This stores the product in Redis with a five-minute expiration.</p><p>After the key expires, the next request becomes a cache miss and fetches fresh data from MongoDB.</p><h4>How do you know the cache is working?</h4><p>Send the same request twice:</p><pre>GET http://localhost:3000/products/101</pre><p>The first response should report:</p><pre>{<br>  &quot;source&quot;: &quot;database&quot;<br>}</pre><p>The second response should report:</p><pre>{<br>  &quot;source&quot;: &quot;cache&quot;<br>}</pre><p>The actual responses will also contain the product object.</p><p>You can verify the key directly in Redis:</p><pre>GET product:101<br>TTL product:101</pre><p>The <strong>TTL</strong> command tells you approximately <strong>how many seconds remain before expiration.</strong></p><p>If you see <strong>-2</strong>, the <strong>key doesn’t exist</strong>. If you see <strong>-1</strong>, <strong>the key exists but has no expiration</strong>.</p><h4>Why isn’t this implementation production-ready yet?</h4><p>Everything is working in Redis implementation right? right?</p><p>Nope, there’s a subtle problem.</p><p>The entire request is inside one try block. If Redis fails, the application returns an errorev, en if MongoDB is perfectly healthy and could still return the product.</p><p>For a cache, this is often undesirable.</p><p>A cache should generally improve performance, not become a new dependency that prevents your application from serving otherwise available data.</p><p>We’ll fix this in the production section below.</p><h3>6. Cache Invalidation: The Problem Every Redis Beginner Should Understand</h3><p>Here things get more interesting which beginners often miss at first.</p><p>Caching data is relatively easy.</p><p>Keeping cached data correct is harder.</p><p>Imagine your product is priced at ₹999.</p><p>Your application stores it in Redis:</p><pre>product:101 → {&quot;name&quot;:&quot;Keyboard&quot;,&quot;price&quot;:999}</pre><p>Now, an admin updates the product price in MongoDB to ₹1,199.</p><p>But Redis still contains the old value. Users may continue seeing ₹999 until the cached entry expires.</p><p>This is called <strong>“stale data”</strong>.</p><h4>The simplest solution: Use a TTL</h4><p><strong>TTL</strong> stands for <strong>time to live</strong>. It determines how long a key remains available before Redis automatically expires it.</p><p>For example:</p><pre>SET product:101 &#39;{&quot;name&quot;:&quot;Keyboard&quot;,&quot;price&quot;:999}&#39; EX 300</pre><p>Here the TTL is <strong>“EX 300”</strong>, the key expires after <strong>five minutes</strong>.</p><p>TTL is useful because it limits how long cached data can remain stale when the application stops updating it.</p><p>However, TTL doesn’t guarantee that users will always see the latest data. A user could still receive an old value just before the key expires.</p><h4>A better approach: Invalidate the cache after an update</h4><p>When a product is updated in MongoDB, delete its cached copy</p><pre>app.put(&quot;/products/:id&quot;, async (req, res) =&gt; {<br>  const { id } = req.params;<br><br>  try {<br>    const product = await Product.findByIdAndUpdate(<br>      id,<br>      req.body,<br>      {<br>        new: true,<br>        runValidators: true,<br>      }<br>    ).lean();<br><br>    if (!product) {<br>      return res.status(404).json({<br>        message: &quot;Product not found&quot;,<br>      });<br>    }<br><br>    await redis.del(`product:${id}`);<br><br>    return res.json(product);<br>  } catch (error) {<br>    console.error(&quot;Product update failed:&quot;, error);<br><br>    return res.status(500).json({<br>      message: &quot;Internal server error&quot;,<br>    });<br>  }<br>});</pre><p>The next request won’t find the old cache entry. It will fetch the updated product from MongoDB and cache the new value.</p><p>This pattern is called <strong>cache invalidation</strong>.</p><p>Notice that the database update happens before the cache deletion. This ensures that the application doesn’t intentionally repopulate the cache with the old database value during the normal update flow.</p><p>But, but, but</p><p>There are still failure cases to consider.</p><p>What happens if MongoDB successfully updates the product but Redis is unavailable when the application tries to delete the cached entry?</p><p>The database update has succeeded, but the stale cache entry may remain.</p><p>For a simple application, a short TTL limits the problem. For more demanding systems, you may need <strong>retries</strong>, <strong>reconciliation</strong>, or <strong>an event-driven invalidation mechanism</strong>.</p><p>how to use it properly then?</p><h4>Should you update Redis or delete the key?</h4><p>There are two common approaches.</p><p><strong>Option A: Delete the cache key.</strong></p><pre>await redis.del(`product:${id}`);</pre><p>The next read fetches the current product from MongoDB.</p><p>This is simple and works well for many applications.</p><p><strong>Option B: Update the cache with the new value.</strong></p><pre>await redis.set(<br>  `product:${id}`,<br>  JSON.stringify(product),<br>  &quot;EX&quot;,<br>  300<br>);</pre><p>This can avoid a cache miss immediately after an update.</p><p>However, it introduces more opportunities for inconsistent data, especially when multiple application instances update the same record concurrently.</p><p>For beginners, <strong>deleting the cache entry after a successful database update is usually the simpler default</strong>.</p><h3>7. Making Redis Failures Safe in Production</h3><p>Now let’s talk about some big league stuffs about production :</p><h4><strong>graceful degradation :</strong></h4><p>This is one of the most important lessons when introducing Redis into a real application.</p><p>Suppose your application uses MongoDB as its primary database and Redis only for caching.</p><p>What happens if Redis goes down?</p><p>Ideally, users should still be able to read products from MongoDB.</p><p>The application may become slower, but a temporary cache failure shouldn’t automatically make the entire product API unavailable.</p><p>This approach is often called <strong>graceful degradation</strong>.</p><h4>Handling Redis failure</h4><p>Consider this simplified helper:</p><pre>async function getCachedValue(key) {<br>  try {<br>    return await redis.get(key);<br>  } catch (error) {<br>    console.error(&quot;Redis read failed:&quot;, error);<br>    return null; <br>  }<br>}</pre><p>Returning <strong>null</strong> allows the caller to treat a Redis failure like a cache miss.</p><p>But this helper only handles reads. Writing to Redis needs separate handling too.</p><p>A more complete cache-aside route can look like this:</p><pre>app.get(&quot;/products/:id&quot;, async (req, res) =&gt; {<br>  const { id } = req.params;<br>  const cacheKey = `product:${id}`;<br><br>  let cachedProduct = null;<br><br>  // Try Redis, but don&#39;t let a cache failure block MongoDB.<br>  try {<br>    cachedProduct = await redis.get(cacheKey);<br>  } catch (error) {<br>    console.error(&quot;Redis read failed:&quot;, error);<br>  }<br><br>  // Cache hit<br>  if (cachedProduct !== null) {<br>    try {<br>      return res.json({<br>        source: &quot;cache&quot;,<br>        product: JSON.parse(cachedProduct),<br>      });<br>    } catch (error) {<br>      console.error(&quot;Invalid cached product:&quot;, error);<br><br>      // Best-effort cleanup of the corrupt cache entry.<br>      try {<br>        await redis.del(cacheKey);<br>      } catch (deleteError) {<br>        console.error(&quot;Redis cleanup failed:&quot;, deleteError);<br>      }<br>    }<br>  }<br><br>  // Cache miss or Redis failure: query MongoDB.<br>  try {<br>    const product = await Product.findById(id).lean();<br><br>    if (!product) {<br>      return res.status(404).json({<br>        message: &quot;Product not found&quot;,<br>      });<br>    }<br><br>    // Cache writes are best-effort.<br>    try {<br>      await redis.set(<br>        cacheKey,<br>        JSON.stringify(product),<br>        &quot;EX&quot;,<br>        300<br>      );<br>    } catch (error) {<br>      console.error(&quot;Redis write failed:&quot;, error);<br>    }<br><br>    return res.json({<br>      source: &quot;database&quot;,<br>      product,<br>    });<br>  } catch (error) {<br>    console.error(&quot;MongoDB query failed:&quot;, error);<br><br>    return res.status(500).json({<br>      message: &quot;Internal server error&quot;,<br>    });<br>  }<br>});</pre><p>This example makes Redis failures non-fatal for product reads.</p><p>It still has limitations: it doesn’t implement request timeouts, protection against a large number of simultaneous cache misses, or comprehensive production observability. Those should be addressed based on your application’s traffic and reliability requirements.</p><h3>One important warning: Redis timeouts</h3><p>A Redis request can fail, but it can also take too long to return.</p><p>If Redis is struggling, thousands of requests could spend time waiting for the cache before falling back to MongoDB.</p><p>A production application should use appropriate connection and command timeouts, monitor Redis latency, and avoid unbounded retries.</p><p>Also remember that falling back to MongoDB increases database traffic. If Redis fails during a traffic spike, the database may suddenly receive far more requests than usual.</p><p><strong>Graceful degradation is useful only if your primary database can handle the additional load.</strong></p><p>For a small application, start by handling <strong>cache failures</strong> correctly. Add more sophisticated fallback and load-shedding strategies when your traffic and reliability requirements justify them.</p><p>Now let’s see some small real world use cases in your projects :</p><h3>8. Redis for Rate Limiting: Protecting Your API</h3><p>Imagine you’ve built a login endpoint: <strong>POST /login</strong></p><p>An attacker or simply a badly written script can send thousands of requests in a short period.</p><p>Without rate limiting, your API might spend unnecessary resources processing those requests.</p><p>Rate limiting restricts how frequently a client can perform an action within a given period.</p><h4>But why Redis?</h4><p>Why not simply store request counts in regular JavaScript variables inside Express?</p><pre>// never do this in production<br>rateLimit = 0</pre><p>If your application scales across multiple servers, worker processes, or serverless containers, local in-memory variables are isolated to that single instance. A user could hit Server A, Server B, and Server C, dodging your limits entirely.</p><p><strong>Redis</strong> solves this:</p><ul><li><strong>Centralized:</strong> Every server checks the exact same counter in real time.</li><li><strong>Lightning Fast:</strong> Redis operations run in RAM (sub-millisecond latency).</li><li><strong>Auto-Expiring Keys (TTL):</strong> Old counters clean up automatically without filling your disk.</li><li><strong>Atomic Operations:</strong> Guarantees no two requests can race each other and exploit timing gaps.</li></ul><p>Now let’s see how to setup a rate limiter in code</p><h4>Step 1: Project Setup &amp; Installing Dependencies</h4><p>A production implementation should set expiration safely and atomically, or use a well-tested rate-limiting library like “<a href="https://www.npmjs.com/package/rate-limiter-flexible"><strong>rate-limiter-flexible</strong></a><strong>”</strong>.</p><p>Initialize a clean Node.js project and install the required libraries:</p><pre>npm init -y<br>npm install express ioredis rate-limiter-flexible</pre><h4>Step 2: Initialize the Limiter</h4><pre>import { RateLimiterRedis } from &quot;rate-limiter-flexible&quot;;<br><br>const rateLimiter = new RateLimiterRedis({<br>  storeClient: redis,<br>  keyPrefix: &quot;login_limit&quot;,<br>  points: 10,<br>  duration: 60,<br>});</pre><p>Here:</p><ul><li>storeClient is your Redis connection.</li><li>keyPrefix identifies the rate-limit keys.</li><li>points defines the number of allowed requests.</li><li>duration defines the time window in seconds.</li></ul><h4>Step 3: You can then apply the limiter to your login route:</h4><pre>app.post(&quot;/login&quot;, async (req, res, next) =&gt; {<br>  const clientKey = req.ip;<br><br>  try {<br>    await rateLimiter.consume(clientKey); // applying rate limiter here<br><br>    return next();<br><br>  } catch (error) {<br>    if (error &amp;&amp; error.remainingPoints !== undefined) {<br>      return res.status(429).json({<br>        message: &quot;Too many login attempts. Try again later.&quot;,<br>      });<br>    }<br><br>    // Redis or another unexpected infrastructure failure.<br>    console.error(&quot;Rate limiter failed:&quot;, error);<br><br>    return res.status(503).json({<br>      message: &quot;Login service temporarily unavailable&quot;,<br>    });<br>  }<br>});</pre><p>This is a example of basic implementation, but it needs a couple of production considerations.</p><ol><li>First, make sure Express’s IP information is configured correctly if you’re running behind a trusted reverse proxy. Blindly trusting forwarded IP headers can let clients manipulate their apparent IP address.</li><li>Second, rate limiting only by IP can affect multiple legitimate users behind the same network like all the students in same college wifi trying to login in same time. For login endpoints, you may want layered limits based on IP and account identifier, with careful handling of account enumeration and denial-of-service risks.</li><li>Finally, decide explicitly what should happen if Redis becomes unavailable. This example fails closed by temporarily rejecting requests when the limiter cannot operate. Some endpoints may instead use a controlled fallback. The correct choice depends on the risk of allowing unthrottled requests.</li></ol><p>However you don’t need to add rate limiters everywhere,</p><p><strong>Good places to use rate limiting include:</strong></p><ul><li>Login and password-reset endpoints.</li><li>OTP generation and verification.</li><li>Public APIs.</li><li>Expensive search endpoints.</li><li>AI endpoints that consume paid API credits.</li></ul><p>This is a good enough resource to make your first rate limiter with Redis, other than that just fu*k around and find out things to learn more by your own.</p><h3>9. Redis for Sessions and Authentication</h3><p>Now let’s talk about another use cases of Redis</p><p>Redis can also be used to store session data.</p><p>Before discussing implementation, let’s clarify the difference between <strong>sessions</strong> and <strong>JWTs</strong>.</p><h4>Traditional server-side sessions</h4><p>With a session-based authentication system, the server stores <strong>information about the authenticated user</strong>.</p><p>The browser receives a session identifier, often in a cookie.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*XQkiaAa04V3sHxwCSAWZzw.png" /></figure><p>For example, Redis might store:</p><pre>session:abc123 → {&quot;userId&quot;:&quot;42&quot;,&quot;role&quot;:&quot;admin&quot;}</pre><p>When the user makes another request, the backend reads the session identifier from the cookie and retrieves the associated session.</p><p>Redis is useful because sessions need to be accessed frequently, and they often need to expire automatically.</p><p>For example, you could configure sessions to expire after a period of inactivity or according to your application’s session policy.</p><h4>What about JWT authentication?</h4><p>With JWT-based authentication, the server can verify a signed token without looking up a session record on every request.</p><p>That doesn’t mean JWTs are always better or that Redis is unnecessary.</p><p>JWTs can still be combined with Redis for example, to maintain revocation lists, refresh-token records, or session metadata.</p><p><strong>But adding Redis to a JWT application just because it’s available isn’t automatically beneficial.</strong></p><h4>When should you use Redis for authentication?</h4><p>Redis-backed sessions make sense when you need centralized session management, rapid revocation, or shared state across multiple application instances.</p><p>For a project already using JWTs, consider whether you actually need server-side session tracking before adding another component.</p><p>And regardless of the approach, authentication needs secure cookie settings, appropriate token expiration, authorization checks, and protection against common web attacks. Redis alone doesn’t provide those guarantees.</p><h3>10. Building a Leaderboard with Redis</h3><p>Now let’s talk about this part where Redis offers more than just caching.</p><p>Imagine you’re building a gaming application with thousands of users. Each user earns points, and the application needs to display the top ten players.</p><p>Now your’e gonna think we can just <strong>query a database</strong>, <strong>sort the players by score</strong>, <strong>and return the top ten</strong>.</p><p>That can work well, especially with appropriate database indexes.</p><p>However, if you frequently update scores and request rankings, Redis sorted sets provide a convenient data structure for this workload.</p><p>Suppose there is a list of players :</p><pre>ZADD leaderboard 120 user:1<br>ZADD leaderboard 250 user:2<br>ZADD leaderboard 180 user:3</pre><h4>How to get the top three players?</h4><pre>ZREVRANGE leaderboard 0 2 WITHSCORES</pre><p>Possible result:</p><pre>user:2<br>250<br>user:3<br>180<br>user:1<br>120</pre><p>The highest-scoring player appears first.</p><p>Now suppose after some time the score for user:1 increases from 120 to 170. in that case Redis handles the score update and maintains the sorted-set ordering.</p><pre>ZREVRANK leaderboard user:1</pre><p>This returns the player’s zero-based position when ranked from highest to lowest. A result of 0 means first place.</p><p>One question might appear in your mind that <strong>“Does Redis replace your database here?”</strong></p><p>Not necessarily.</p><p>For a real application, you might store authoritative scores or match results in PostgreSQL or MongoDB, while using Redis to serve the current leaderboard quickly.</p><h3>11. Redis for Background Jobs and Queues</h3><p>This is an another popular use cases of Redis.</p><p>Imagine a user signs up for your application. Your backend might need to:</p><ol><li>Save the user to MongoDB.</li><li>Send a welcome email.</li><li>Generate a report.</li><li>Notify another service.</li></ol><p>If the API waits for all these operations, the request can become slow. Some tasks might also fail temporarily and need to be retried.</p><p>Background jobs let you separate the main request from work that can happen later.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*u9zpcRbhEb6Tn9c3gMR8Fw.png" /></figure><p>Redis can provide the storage and coordination used by a job queue.</p><p>You don’t need to write everything manually, In Node.js, ⁠<a href="https://docs.bullmq.io/"><strong>BullMQ</strong></a> is a popular library for implementing Redis-backed background jobs.</p><p><strong>A real production level background-job system should consider these edge cases:</strong></p><ul><li>Retries for temporary failures.</li><li>Backoff between retries.</li><li>Job timeouts.</li><li>Concurrency limits.</li><li>Duplicate job handling and idempotency.</li><li>Monitoring and failed-job recovery.</li></ul><p>For example, if a worker sends an email but crashes before recording success, a retry could send the email twice. Idempotency and appropriate job design help prevent duplicate side effects.</p><h4>When should you use a queue?</h4><p>Here are some use cases :</p><p>Sending emails, Generating reports, Processing uploaded files, Running scheduled tasks, Processing webhooks, Performing slow operations that don’t need to block the user’s request.</p><p><strong>Don’t add a queue to every application. For a simple contact form, a direct email call may be enough initially. Introduce background jobs when you need retries, asynchronous processing, better control over throughput, or separation between API requests and slow tasks.</strong></p><h3>12. What Happens When Redis Restarts? Understanding Persistence under the hood</h3><p>Enough examples now let’s talk about <strong>Persistence.</strong></p><p>So far, we’ve mostly treated Redis as a temporary cache. But Redis can also be configured to persist data to disk.</p><p>Now you’re gonna ask “How?”</p><p>Because Redis primarily keeps its <strong>working dataset</strong> in <strong>memory</strong>. Without persistence, restarting Redis can cause its in-memory data to be lost.</p><p>Persistence refers to the writing of data to durable storage, such as a solid-state disk (SSD). Redis supports two major persistence mechanisms:</p><h4>RDB (Redis Database):</h4><p>RDB persistence performs point-in-time snapshots of your dataset at specified intervals.</p><p><strong>Example of RBD: </strong>Think of your data like a whiteboard in a classroom:</p><ul><li>With <strong>AOF</strong>, you write down every single stroke of the marker.</li><li>With <strong>RDB</strong>, you don’t care about the strokes. Instead, you simply take a <strong>photo of the board every 15 minutes</strong>.</li><li>If someone accidentally wipes the board, you look at the most recent photo and redraw it exactly as it appeared at that moment.</li></ul><h4>AOF (Append Only File):</h4><p>AOF persistence logs every write operation received by the server. These operations can then be replayed again at server startup, reconstructing the original dataset. Commands are logged using the same format as the Redis protocol itself.</p><p><strong>Example of AOF : </strong>suppose a bank account, Instead of taking a photo of the final balance every hour, an accountant writes down <strong>every single transaction</strong> in a page as it happens:</p><ul><li><em>Deposit ₹500</em></li><li><em>Withdraw ₹200</em></li><li><em>Deposit ₹1,000</em></li></ul><p>If the power goes out and the computer resets, they just open the page, replay all the lines from top to bottom, and rebuild the exact current balance: ₹1,300.</p><p>Although enabling persistence doesn’t automatically make every Redis-backed queue safe against every failure. You still need appropriate configuration, backups where necessary, and a recovery strategy.</p><p>For a deeper explanation, see the ⁠<a href="https://redis.io/docs/latest/operate/oss_and_stack/management/persistence/">official Redis persistence documentation</a>.</p><h3>13. Redis Memory Management: What Happens When It Fills Up?</h3><p>Well now you know how Redis works and how to implement it,</p><p>But what if Redis storage got filled?</p><p>Suppose you cache every product, every API response, and every user’s profile without considering how much data you’re storing.</p><p>Eventually, Redis may approach its configured memory limit. What happens next depends on the configuration.</p><p>Pretty good question actually. and the answer is :</p><h4>Set a maximum memory limit</h4><p>Redis supports a maxmemory setting that limits how much memory it can use for its dataset.</p><p>For example, a deployment might configure:</p><pre>maxmemory 256mb</pre><p>This is only an example value. Your actual limit should depend on the available memory, workload, overhead, and deployment environment.</p><h4>Choose an eviction policy</h4><p>When Redis reaches its memory limit, the configured eviction policy determines how it handles new writes that need additional memory.</p><p>For a cache, common policies include:</p><ul><li><strong>allkeys-lru</strong>: Evicts keys that haven’t been used recently, approximately.</li><li><strong>allkeys-lfu</strong>: Evicts keys that are used less frequently, approximately.</li><li><strong>volatile-ttl</strong>: Chooses among keys that have expiration times, based partly on their remaining TTL.</li></ul><p>Redis supports other policies too.</p><p>The right choice depends on how your data is used. For example, a cache where a small group of items receives most of the traffic may benefit from a frequency-oriented policy.</p><p>Read the ⁠<a href="https://redis.io/docs/latest/develop/reference/eviction/">Redis eviction documentation</a> before selecting a policy for a production deployment.</p><h3>14. Running Redis Locally with Docker</h3><p>Now you know most of the necessary things about Redis, but if you wanna fu*k around a more and want to set it up in Docker then I can hear you</p><p>You don’t need to install and manage Redis manually to start learning it.</p><p>If you already use Docker, you can run Redis in a container.</p><p>For a quick local experiment:</p><pre>docker run --name redis-dev -p 6379:6379 -d redis</pre><p>This starts a Redis container and exposes port 6379 on your machine.</p><p>Your Node.js application can connect to:</p><pre>redis://localhost:6379</pre><p>Check that the container is running:</p><pre>docker ps</pre><p>Connect to Redis:</p><pre>docker exec -it redis-dev redis-cli</pre><p>Then try:</p><pre>PING<br>SET message &quot;Hello Redis&quot;<br>GET message</pre><p>You should receive:</p><pre>PONG<br>&quot;Hello Redis&quot;</pre><p>But this setup does not save your data, for that use docker desktop.</p><p>For a development setup where you want to preserve Redis data it’s a better approach to use docker desktop and ioredis</p><p>In docker you need to make a docker-compose.yml file and write the required Database and Redis services so that your services run in a port.</p><h4>here are some required terminal commands</h4><pre>docker init<br>npm init -y<br>npm i dotenv ioredis express mongoose<br>docker compose up -d<br>npm run dev</pre><p>for example :</p><pre>services:<br>  mongodb:<br>    image: mongo:8<br>    container_name: practice-mongodb<br>    restart: unless-stopped<br>    ports:<br>      - &quot;127.0.0.1:27017:27017&quot;<br>    environment:<br>      MONGO_INITDB_ROOT_USERNAME: admin<br>      MONGO_INITDB_ROOT_PASSWORD: practice123<br>    volumes:<br>      - mongo_data:/data/db<br><br>  redis:<br>    image: redis:8<br>    container_name: practice-redis<br>    restart: unless-stopped<br>    ports:<br>      - &quot;127.0.0.1:6379:6379&quot;<br>    volumes:<br>      - redis_data:/data<br><br>volumes:<br>  mongo_data:<br>  redis_data:</pre><p>Basic file structure :</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/422/1*Ai3HnpmlSmkjuXN9rqvofQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kG92j-aIsC5Ck0oOsxOI1w.png" /><figcaption>Docker desktop</figcaption></figure><p>This example enables AOF persistence and stores Redis’s data directory in a named Docker volume. It illustrates local development, not a complete production deployment. For production, pin an appropriate supported image version, secure access, configure memory limits and persistence deliberately, and plan for backups and recovery.</p><p>Also, don’t expose an unauthenticated Redis instance to the public internet. Keep it on a trusted private network and configure authentication and network controls appropriate to your environment.</p><p>For more information, see the ⁠<a href="https://docs.docker.com/compose/">official Docker Compose documentation</a>.</p><h3>15. Production Lessons: What You Should Know Before Shipping Redis</h3><p>Look, I know this article is longer than your attention span, but before you clone some massive open-source repo and deploy your unhinged pull request straight to production, sit down, my son. let me share you some some wisdom.</p><h4>15.1 Don’t cache everything</h4><p>Caching has a cost.</p><p>You need to maintain cache entries, handle expiration, invalidate outdated data, and monitor memory usage.</p><p>A good starting point is to cache data that is:</p><ul><li>Read frequently.</li><li>Expensive to compute or retrieve.</li><li>Relatively stable.</li><li>Safe to serve from a potentially slightly outdated copy.</li></ul><h4>15.2 Understand cache stampedes</h4><p>Imagine a popular product’s cache entry expires just as thousands of users request it.</p><p>All those requests may discover that the key is missing and query MongoDB simultaneously. This can create a sudden spike in database load.</p><p>This problem is commonly called a “<strong>cache stampede”</strong>.</p><p>Possible solutions :</p><ul><li>Adding a small amount of randomness to cache TTLs.</li><li>Coalescing concurrent requests for the same key within an application instance.</li><li>Using short-lived locks carefully when appropriate.</li><li>Refreshing popular cache entries before they expire.</li></ul><p>You don’t need to implement all these techniques in your first project. Start by understanding the problem, then introduce protection when your workload warrants it.</p><h4>15.3 Avoid caching errors accidentally</h4><p>Suppose a product doesn’t exist.</p><p>Your application returns null from MongoDB. If it caches that result indefinitely, newly created products may remain invisible until the cache entry is removed.</p><p>One solution is to avoid caching missing records initially. Another is to use a short TTL for negative results, often called <strong>negative caching</strong>.</p><p>The right choice depends on how often records are created and how quickly the application needs to reflect those changes.</p><h4>15.4 Use consistent key naming</h4><p>A messy key structure becomes difficult to maintain as an application grows.</p><p>Instead of arbitrary names like: x1, data, temp, 123</p><p>Use names that explain what the data represents:</p><pre>product:101<br>user:42:profile<br>leaderboard:weekly<br>rate_limit:login:192.0.2.10</pre><p>For larger applications, key prefixes and documented naming conventions help prevent accidental overlap between features.</p><h4>15.5 Monitor the right things</h4><p>If Redis is part of your production application, you should know whether it is healthy.</p><p>Useful things to monitor include:</p><ul><li>Memory usage.</li><li>Cache hit and miss rates.</li><li>Command latency.</li><li>Evicted keys.</li><li>Connection failures.</li><li>Rejected writes.</li><li>CPU usage.</li><li>The effect of Redis failures on your database.</li></ul><p>For a cache, a low hit rate may indicate that the TTL is poorly chosen, the keys aren’t reused often, or the application is caching the wrong data.</p><p>Monitoring should help answer a practical question:</p><p><strong>“Is Redis actually improving the application, and what happens when it doesn’t work?”</strong></p><h4>15.6 Don’t trust Redis as an authorization system by itself</h4><p>Redis can store permissions, session information, or temporary authorization state. But the application still needs to validate the data and enforce access control rules.</p><p>For example, caching a user’s role for an hour may cause the application to continue treating a demoted administrator as an administrator after their permissions change.</p><p>Use an appropriate invalidation strategy or a shorter cache lifetime when permission changes need to take effect quickly.</p><p><strong>Security sensitive data deserves more careful cache design than a public article or product description.</strong></p><h3>17. A Practical Project to Make Redis Click</h3><p>If you want to understand Redis beyond tutorials, build a small application that uses it for several distinct purposes. You don’t need to build a massive project. A product API is enough.</p><p>Build a simple <strong>Node.js + Express API</strong> using MongoDB as the primary database and Redis for caching, rate limiting, and leaderboards.</p><ul><li>Implement product CRUD with Redis caching and cache invalidation.</li><li>Add Redis-backed rate limiting and a leaderboard using sorted sets.</li><li>Test how your application behaves when Redis goes down using Docker.</li></ul><p>The goal is to understand how Redis works in a real application, not just learn its commands.</p><p>Congratulations! Your CRUD app now has Redis, Docker, caching, and a leaderboard. It still has 5 users, but at least it’s production-ready. Probably.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a59e2cd16a74" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>