Memcached Explained: Architecture, Internals, Caching Patterns, and Best Practices

A database that answers the exact same query a thousand times a minute is doing a thousand times more work than it actually needs to. Memcached exists to fix exactly that, a simple, deliberately minimal, in memory cache sitting in front of whatever is genuinely slow, a database, an expensive computation, a call to a third party API, and returning an answer in microseconds instead of milliseconds, for every single repeated request that does not actually need fresh work done all over again.

This guide covers Memcached genuinely completely, its design philosophy, the slab allocator that makes it so fast, the protocol underneath it, and the caching patterns and pitfalls that actually matter once you put it in front of a real, production system.

Here is what we will cover, in order:

• What Memcached actually is, and the problem it solves
• The design philosophy, deliberately simple
• Architecture, a dumb server and a smart client
• The memory model, slab allocation
• Installing Memcached
• The protocol, basic commands
• Client libraries and multi get
• Consistent hashing, how a client actually distributes keys
• Expiration and LRU eviction, per slab class
• CAS, avoiding race conditions without locks
• Caching patterns, cache aside, write through, and write behind
• The thundering herd problem and cache stampede prevention
• Memcached compared against Redis
• Security considerations
• Monitoring Memcached
• Common pitfalls
• Putting it all together, a complete caching layer
• Final thoughts

Let’s start with the actual problem Memcached solves.

What Memcached Actually Is, and the Problem It Solves

Picture a web page that shows a product’s details, its price, its description, its current stock level. Generating that page genuinely requires querying a database, and if that page is popular, the exact same query runs again, and again, and again, for every single visitor, even though the underlying data has not actually changed at all between one request and the next.

Memcached is a distributed, in memory key value store, built specifically to hold exactly this kind of already computed result, entirely in RAM, and hand it back in microseconds, dramatically reducing the load on whatever slower system actually produced that result the first time. It was originally built at Danga Interactive to keep LiveJournal genuinely responsive under heavy load, and its core idea has remained essentially unchanged since, a simple, extremely fast cache, not a database, not a message queue, not a general purpose data store.

One of the most common places you’ll find Memcached is in front of product catalogs, user profiles, recommendation engines, and feature flags. These datasets may receive millions of reads per day while changing only a handful of times. Caching them dramatically reduces database load without sacrificing freshness.

The Design Philosophy: Deliberately Simple

Teams sometimes ask why Memcached doesn’t support replication, persistence, transactions, or complex data structures. The answer is simple: those features make systems more capable, but they also make them slower and more operationally complex. Memcached intentionally avoids them to optimize one thing, serving cached data as quickly as possible.

Memcached’s entire design is built around doing genuinely one thing, caching, and doing it about as fast and as simply as possible, deliberately leaving out an enormous amount of functionality other systems consider essential.

This is not a limitation that crept in by accident, it is a genuinely deliberate design choice, and it is exactly why Memcached remains so fast, and so simple to actually operate, decades after it was first built. Every single feature left out is one less thing that can genuinely go wrong, and one less reason the actual cache lookup path needs to do any extra work at all.

Architecture: A Dumb Server, a Smart Client

It’s surprisingly common for engineers to assume adding a new Memcached server automatically rebalances existing data. It doesn’t. Existing keys remain where they are until they’re requested again or expire. Immediately after scaling out, you’ll often see a temporary increase in cache misses as the new server gradually fills with data.

The single most important architectural fact about Memcached, and the one that trips up more people than anything else, is that individual Memcached server instances genuinely do not talk to each other at all.

Each individual Memcached server is a genuinely isolated, independent in memory store, and the actual work of deciding which specific server a given key should live on, and consistently routing requests for that key to that exact same server every time, happens entirely inside the client library, not inside Memcached itself at all. This is a genuinely important, deliberate inversion of where the intelligence in the system actually lives, compared to a broker or a database that manages its own clustering internally, Memcached pushes that entire responsibility out to whoever is actually calling it.

The Memory Model: Slab Allocation

Engineers are sometimes confused when Memcached reports free memory while evictions continue increasing. The reason is usually slab allocation. One slab class has filled up with small objects and is constantly evicting items, while another slab class allocated for larger objects still has free space.

Memcached’s genuine speed comes partly from a deliberately specific way it manages the raw memory holding your actual cached values, called slab allocation, and understanding it explains a real, practical quirk you will eventually run into using Memcached at any real scale.

Rather than using a general purpose memory allocator, which tends to badly fragment memory over time as differently sized values are constantly stored and removed, Memcached divides its available memory into slabs, and each slab is further divided into fixed size chunks, with chunk sizes growing by a configurable growth factor across a series of slab classes. When you store an item, Memcached finds the smallest chunk size genuinely large enough to hold it, and stores it there, some space inside that chunk is typically wasted, since an item rarely fills its chunk exactly, but this deliberate tradeoff avoids the far more serious problem of external fragmentation an ordinary allocator would eventually suffer from after storing and evicting a genuinely enormous number of differently sized items over time.

memcached -vv -f 1.25

The -f flag controls that growth factor between slab classes, a smaller factor creates more, more finely spaced slab classes, wasting less space per item but consuming more memory on bookkeeping overhead, while a larger factor creates fewer, more coarsely spaced classes.

Installing Memcached

Memcached is available as a package on essentially every Linux distribution, and running it through Docker is genuinely the fastest way to get started locally.

docker run -d --name memcached -p 11211:11211 memcached:latest --memory-limit=256

That command starts a single Memcached instance, listening on its standard port, 11211, with a 256 megabyte memory limit, beyond which it will genuinely begin evicting older items to make room for new ones, exactly the eviction behavior covered in real depth shortly.

apt install memcached
memcached -m 256 -p 11211 -u memcache

The -m flag sets the memory limit in megabytes, and -u specifies which user the process should actually run as, since Memcached, sensibly, refuses to run as root by default.

The Protocol: Basic Commands

Memcached’s protocol is genuinely simple enough to interact with directly over a plain TCP connection, which is exactly why it is worth seeing at least once before reaching for a client library.

echo -e "set greeting 0 900 5\r\nhello\r\n" | nc localhost 11211

STORED

That command stores a value under the key greeting, with flags of 0, an expiration of 900 seconds, and a body genuinely 5 bytes long, hello. Retrieving it looks like this.

echo -e "get greeting\r\n" | nc localhost 11211

VALUE greeting 0 5
hello
END

echo -e "add greeting 0 900 5\r\nhowdy\r\n" | nc localhost 11211

NOT_STORED

That add call genuinely fails, returning NOT_STORED, precisely because the key already exists from the earlier set call, exactly the distinction that makes add genuinely useful for a first writer wins pattern, only ever actually storing a value the very first time, and leaving every subsequent attempt alone.

Client Libraries and Multi Get

One service initially performed more than a dozen individual cache lookups to render every product page. Switching to Multi Get reduced the number of network round trips to a single request and noticeably lowered page latency without changing any business logic.

In genuine, everyday practice, almost nobody talks to Memcached directly over a raw socket, a client library handles the actual protocol details, and, genuinely importantly, the key distribution across multiple servers covered in the next section.

<dependency>
    <groupId>net.spy</groupId>
    <artifactId>spymemcached</artifactId>
    <version>2.12.3</version>
</dependency>

import net.spy.memcached.AddrUtil;
import net.spy.memcached.MemcachedClient;

public class MemcachedExample {

    public static void main(String[] args) throws Exception {

        // Connect to multiple Memcached servers (uses consistent hashing)
        MemcachedClient client = new MemcachedClient(
                AddrUtil.getAddresses(
                        "cache-a.internal:11211 " +
                        "cache-b.internal:11211 " +
                        "cache-c.internal:11211"
                )
        );

        // Store a value with a TTL of 15 minutes (900 seconds)
        client.set(
                "product:4821",
                900,
                "{\"name\":\"Blue Shirt\",\"price\":24.99}"
        );

        // Retrieve the value
        String value = (String) client.get("product:4821");

        System.out.println(value);

        client.shutdown();
    }
}

A genuinely important performance technique available directly through the protocol, and exposed by essentially every client library, is multi get, retrieving several keys in a single request rather than issuing a separate round trip for each one.

import net.spy.memcached.AddrUtil;
import net.spy.memcached.MemcachedClient;

import java.util.Arrays;
import java.util.Map;

public class MemcachedBulkGetExample {

    public static void main(String[] args) throws Exception {

        // Connect to multiple Memcached servers
        MemcachedClient client = new MemcachedClient(
                AddrUtil.getAddresses(
                        "cache-a.internal:11211 " +
                        "cache-b.internal:11211 " +
                        "cache-c.internal:11211"
                )
        );

        // Fetch multiple keys in a single request
        Map<String, Object> values = client.getBulk(
                Arrays.asList(
                        "product:4821",
                        "product:4822",
                        "product:4823"
                )
        );

        values.forEach((key, value) -> System.out.println(key + " -> " + value));

        client.shutdown();
    }
}

Fetching a page that genuinely needs a dozen separate cached values, a product, its reviews, its related items, is dramatically faster issuing one single multi get than issuing twelve separate, sequential get calls, each one paying its own full network round trip cost individually.

Consistent Hashing: How a Client Actually Distributes Keys

Since Memcached servers genuinely never talk to each other at all, the client has to decide, on its own, which specific server a given key should actually live on, and it has to keep making that exact same decision consistently, every single time that same key is looked up again later.

Get Deep Awasthi’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

The naive approach, hashing a key and taking the result modulo the number of servers, works, but breaks badly the moment the actual number of servers ever changes.

Consistent hashing, almost universally implemented through an algorithm called ketama, solves this considerably more gracefully, mapping both servers and keys onto the exact same conceptual ring, and assigning each key to the nearest server found moving clockwise around that ring.

import net.spy.memcached.AddrUtil;
import net.spy.memcached.MemcachedClient;

public class MemcachedExample {

    public static void main(String[] args) throws Exception {

        MemcachedClient client = new MemcachedClient(
                AddrUtil.getAddresses(
                        "cache-a.internal:11211 " +
                        "cache-b.internal:11211"
                )
        );

        // Use the client...

        client.shutdown();
    }
}

The genuine advantage of this approach is that adding or removing one single server only actually affects the small slice of the ring immediately around it, roughly one over N of all keys, where N is the total number of servers, rather than remapping nearly the entire key space all at once the way plain modulo hashing does. Losing one out of ten servers under consistent hashing costs you roughly ten percent of your cache, a genuinely graceful, partial degradation, rather than the near total cache wipe plain modulo hashing would inflict on you instead.

Don’t expect adding a cache server to improve performance immediately. New servers start completely empty, so cache misses often increase briefly while popular keys are repopulated. This temporary “warm-up” period is completely normal.

Expiration and LRU Eviction, Per Slab Class

A one-hour TTL doesn’t guarantee an object survives for an hour. Under heavy memory pressure, Memcached may evict it after only a few minutes. Applications should always treat expiration as a maximum lifetime, not a promise.

Every item stored in Memcached can carry an expiration time, after which it is treated as though it never existed at all, and, genuinely importantly, Memcached also evicts items well before their actual expiration whenever it genuinely runs out of memory.

set session:abc123 0 3600 42

That expiration, 3600 seconds in this example, is a maximum lifetime, not a guarantee the item will actually stick around that long. When memory genuinely fills up, Memcached evicts the least recently used item to make room for a new one, exactly the behavior the name LRU refers to.

Here is the genuinely important, easily missed detail. Eviction happens per slab class, not globally across the entire cache. If one specific slab class, holding items of a particular size range, is genuinely full, Memcached evicts the least recently used item from that exact slab class specifically, even if an entirely different slab class, holding a differently sized range of items, still has plenty of completely free space sitting unused.

This is a genuinely real, practical gotcha, an application storing a mix of very small and very large values can end up with the small item slab classes evicting constantly under memory pressure, while large item slab classes sit comparatively empty, simply because memory, once allocated to a specific slab class, is not automatically reclaimed and handed to a different class that might genuinely need it more. Newer versions of Memcached include a slab reassignment mechanism specifically to address this, automatically moving memory between slab classes based on actual, real demand, but it is genuinely worth knowing this can happen at all before you are the one confused by an oddly uneven eviction rate across what looks, from the outside, like one single, unified cache.

CAS: Avoiding Race Conditions Without Locks

Inventory systems are a common example. If two checkout requests try to purchase the last item simultaneously, a simple read-modify-write sequence can oversell inventory. CAS detects that another request updated the value first, allowing the application to retry safely.

So far we’ve focused on storing and retrieving data. The next challenge is updating cached values safely when multiple clients are writing at the same time.

A genuinely common, real problem arises when two separate processes read the exact same cached value, each compute a new version of it based on what they read, and both write their own result back, with whichever write happens to land last silently overwriting the other’s, entirely unnoticed.

CAS, Check And Set, solves this without needing any actual distributed lock at all. A gets call, rather than an ordinary get, returns the value alongside a unique CAS token representing its exact current version.

import net.spy.memcached.AddrUtil;
import net.spy.memcached.CASResponse;
import net.spy.memcached.CASValue;
import net.spy.memcached.MemcachedClient;

public class MemcachedCASExample {

    public static void main(String[] args) throws Exception {

        MemcachedClient client = new MemcachedClient(
                AddrUtil.getAddresses(
                        "cache-a.internal:11211 " +
                        "cache-b.internal:11211"
                )
        );

        // Get value along with CAS token
        CASValue<Object> result = client.gets("inventory:4821");

        int value = Integer.parseInt(result.getValue().toString());
        long casToken = result.getCas();

        int newValue = value - 1;

        // Update only if value hasn't changed
        CASResponse response = client.cas(
                "inventory:4821",
                casToken,
                newValue
        );

        if (response != CASResponse.OK) {
            System.out.println("Someone else modified this value first, retry");
        }

        client.shutdown();
    }
}

The subsequent cas call only actually succeeds if the value’s CAS token is genuinely still the exact same one that was originally read, meaning nobody else modified it in between. If it has changed, the call fails outright, and the application can simply retry, reading the new, current value and attempting its update again, a genuinely lightweight, optimistic alternative to holding an actual, heavier distributed lock for the same underlying purpose.

Caching Patterns: Cache Aside, Write Through, and Write Behind

Memcached itself has genuinely zero awareness of your actual database at all, it never automatically stays in sync with anything, which means your own application code is entirely responsible for deciding exactly when data actually moves into, and out of, the cache. A handful of established patterns cover the overwhelming majority of real, practical situations.

Cache aside, sometimes called lazy loading, is genuinely the most common pattern used with Memcached specifically. The application checks the cache first, and only on a miss does it actually query the database, storing that result in the cache afterward for next time.

Cache Aside is by far the most widely used caching strategy because it only caches data that’s actually requested. Rarely accessed records never occupy cache memory, allowing the cache to naturally optimize itself around the application’s hottest data.

import net.spy.memcached.AddrUtil;
import net.spy.memcached.MemcachedClient;

public class ProductService {

    private static final MemcachedClient client;

    static {
        try {
            client = new MemcachedClient(
                    AddrUtil.getAddresses(
                            "cache-a.internal:11211 " +
                            "cache-b.internal:11211"
                    )
            );
        } catch (Exception e) {
            throw new RuntimeException(e);
        }
    }

    public static String getProduct(int productId) {

        String key = "product:" + productId;

        // Check cache first
        String cached = (String) client.get(key);
        if (cached != null) {
            return cached;
        }

        // Cache miss -> Fetch from database
        String product = queryProduct(productId);

        // Store in cache for 15 minutes (900 seconds)
        client.set(key, 900, product);

        return product;
    }

    // Simulated database call
    private static String queryProduct(int productId) {
        return "{\"id\":" + productId + ",\"name\":\"Blue Shirt\",\"price\":24.99}";
    }

    public static void main(String[] args) {
        System.out.println(getProduct(4821));
        client.shutdown();
    }
}

Write through keeps the cache and the database updated together, as one single operation, every single write goes to both at once, keeping the cache genuinely always fresh, at the cost of every write now being somewhat slower, since it has to touch two systems rather than one.

Financial applications sometimes prefer Write Through because serving stale balances is worse than paying a slightly higher write latency.

Write behind writes to the cache immediately, and updates the actual database asynchronously afterward, genuinely faster for the write itself, but carrying real risk, if the cache is lost before that asynchronous write to the database actually completes, that update is genuinely gone forever.

Write Behind works well when occasional data loss is acceptable, such as analytics counters or recommendation signals. It’s a poor choice for orders, payments, or inventory updates where every write must be preserved.

The Thundering Herd Problem and Cache Stampede Prevention

One common mistake is assigning every object exactly the same TTL. If thousands of objects expire at the same second, the database suddenly receives thousands of cache misses. Many systems avoid this by adding a small random amount of jitter to each TTL so expirations are spread over time.

A genuinely nasty, real problem shows up the moment a single, extremely popular cached value expires. If a thousand concurrent requests all check the cache at essentially the same instant, all a thousand of them see a miss, and all a thousand of them turn around and hit the database simultaneously, for what was, moments earlier, a single, perfectly cacheable value.

A genuinely effective mitigation has the very first request to actually experience a miss take out a lightweight lock, recompute the value, and populate the cache, while every other concurrent request either waits briefly for that same lock to be released, or is simply served a slightly stale value in the meantime, rather than every single one of them independently, redundantly hitting the database at once.

import net.spy.memcached.AddrUtil;
import net.spy.memcached.MemcachedClient;
import net.spy.memcached.internal.OperationFuture;

public class ProductService {

    private static final MemcachedClient client;

    static {
        try {
            client = new MemcachedClient(
                    AddrUtil.getAddresses(
                            "cache-a.internal:11211 " +
                            "cache-b.internal:11211"
                    )
            );
        } catch (Exception e) {
            throw new RuntimeException(e);
        }
    }

    public static String getProduct(int productId) throws Exception {

        String key = "product:" + productId;

        // Check cache first
        String cached = (String) client.get(key);
        if (cached != null) {
            return cached;
        }

        String lockKey = "lock:product:" + productId;

        // Try to acquire lock (succeeds only if lock doesn't already exist)
        OperationFuture<Boolean> future = client.add(lockKey, 10, "1");
        boolean gotLock = future.get();

        if (gotLock) {
            try {
                // Cache miss -> Fetch from database
                String product = queryProduct(productId);

                // Store in cache for 15 minutes
                client.set(key, 900, product);

                return product;
            } finally {
                // Release lock
                client.delete(lockKey);
            }
        } else {
            // Another thread is rebuilding the cache
            Thread.sleep(50);
            return getProduct(productId);
        }
    }

    // Simulated database call
    private static String queryProduct(int productId) {
        return "{\"id\":" + productId + ",\"name\":\"Blue Shirt\",\"price\":24.99}";
    }

    public static void main(String[] args) throws Exception {
        System.out.println(getProduct(4821));
        client.shutdown();
    }
}

That add call is genuinely doing real work here, since add only succeeds for the very first caller, every single other concurrent request sees it fail, and backs off briefly rather than piling directly onto the database at the exact same moment.

Memcached Compared Against Redis

This comparison comes up constantly, and the honest answer is that the two tools genuinely started from the same basic idea, a fast, in memory cache, and grew in deliberately different directions from there.

It’s common to see both technologies in the same architecture. Redis might handle sessions, queues, and distributed locks, while Memcached is used exclusively for high-volume application caching where raw throughput is the primary concern.

A genuinely practical rule of thumb, reach for Memcached when you need nothing more than a fast, simple cache, and genuinely want the operational simplicity of a tool with almost no features to actually misconfigure, and reach for Redis the moment you need anything beyond that, durability, richer data structures, or built in replication and clustering, capabilities Memcached deliberately, permanently leaves out.

Security Considerations

A surprising number of cloud incidents still begin with an internal service accidentally exposed to the public internet. Memcached should almost never be reachable directly from outside your private network.

Memcached genuinely has no authentication enabled by default at all, and this is not a theoretical concern, it caused a real, serious problem in 2018, when a large number of Memcached instances left accidentally exposed directly to the public internet were abused to launch some of the largest recorded distributed denial of service amplification attacks up to that point.

The genuinely correct, standard practice is straightforward, Memcached should never be directly reachable from the public internet at all, it should sit entirely behind a firewall, reachable only from the specific application servers that actually need it, on a private network. SASL authentication support does exist for situations genuinely requiring it, but network isolation is, and should remain, the actual primary line of defense, not an afterthought layered on top of an otherwise open, exposed instance.

Monitoring Memcached

A steadily falling cache hit rate is often one of the earliest indicators that an application has outgrown its allocated cache memory. Database CPU usage usually increases shortly afterward, making cache metrics an excellent early warning signal.

The stats command, sent directly over the same protocol already covered earlier in this guide, returns a genuinely detailed set of internal counters worth watching continuously.

echo "stats" | nc localhost 11211

A genuinely healthy cache maintains a high hit rate, and a low, stable eviction rate, a hit rate that has been steadily dropping, or an eviction rate that has been steadily climbing, is exactly the kind of early, genuine warning sign that your actual working set has outgrown the memory currently allocated to it, well before that same problem shows up instead as a sudden, painful spike in database load.

Common Pitfalls

During rolling deployments, entire Memcached clusters are sometimes restarted. Applications that mistakenly treat Memcached as permanent storage quickly discover that every cached object disappears at once, forcing the database to handle every request until the cache warms up again.

A handful of specific mistakes show up constantly enough in real, everyday Memcached use to deserve calling out directly.

Treating Memcached as a genuine data store, rather than a disposable cache, is a serious, common mistake, since it has no persistence at all, a restart, a crash, or simply running low on memory can silently erase data an application was incorrectly relying on being permanently there. Anything Memcached holds should always be genuinely, fully reconstructible from some other, authoritative source.

Storing wildly different sized values without any real awareness of the slab allocation behavior covered earlier can lead to a genuinely confusing situation where one slab class evicts constantly while another sits comparatively empty, precisely the practical gotcha already covered in depth earlier in this guide.

Forgetting that a cache miss is not an error at all, but a perfectly normal, expected outcome that application code must genuinely handle gracefully every single time, falling back cleanly to the actual, authoritative source, rather than treating an empty cache as some kind of unexpected, exceptional failure.

Exposing Memcached directly to an untrusted network, exactly the security mistake covered in the previous section, remains a genuinely common, real world misconfiguration worth actively guarding against from the very first day an instance is actually deployed.

Putting It All Together: A Complete Caching Layer

Let’s close the loop with one real, complete example, tying together cache aside loading, multi get, and stampede protection into a single, coherent caching layer for a product catalog page.

import time
from pymemcache.client.hash import HashClient
client = HashClient([
    ('cache-a.internal', 11211),
    ('cache-b.internal', 11211),
])
def get_products(product_ids):
    keys = [f'product:{pid}' for pid in product_ids]
    cached = client.get_many(keys)
    missing_ids = [pid for pid in product_ids if f'product:{pid}' not in cached]
    if missing_ids:
        fresh_products = {}
        for pid in missing_ids:
            lock_key = f'lock:product:{pid}'
            if client.add(lock_key, '1', expire=10):
                product = database.query_product(pid)
                client.set(f'product:{pid}', product, expire=900)
                fresh_products[f'product:{pid}'] = product
                client.delete(lock_key)
            else:
                time.sleep(0.05)
                fresh_products[f'product:{pid}'] = database.query_product(pid)
        cached.update(fresh_products)
    return cached

This one function genuinely demonstrates the majority of this entire guide at once. It fetches every requested product in a single multi get round trip rather than one request per product. Anything genuinely missing falls back to the database, using cache aside. And a lock, using add, protects against a stampede on any single, individually popular product, without ever needing an actual distributed locking system layered on top of Memcached to achieve it.

Final Thoughts

If you have made it this far, you now have a genuinely complete, practical understanding of Memcached, from the deliberately minimal design philosophy that makes it so fast, through the slab allocator underneath it, the protocol, consistent hashing, and the caching patterns that actually matter once real traffic hits it. That is not a small thing. A huge number of engineers use a cache for years knowing only set and get, without ever genuinely understanding why an eviction rate suddenly spiked, or why adding a new cache server briefly made everything slower rather than faster.

The best way to make this permanent is to actually run it yourself. Stand up two or three Memcached instances locally, wire up a client using consistent hashing, and watch directly how few keys actually move when you add a fourth. Deliberately let a popular key expire under simulated concurrent load, with and without a lock protecting it, and watch the very real difference in how hard your database actually gets hit in that exact moment.

One of the easiest ways to understand Memcached is to break it on purpose. Run two local instances, remove one, restart another, lower the memory limit, or let a popular key expire under concurrent load. Watching these scenarios happen is often more educational than reading about them.