Rate Limiting with LyteCache(Python)

If you’ve ever needed to rate limit an API, you’ve probably reached for Redis. It works, but it’s one more thing to run. I built LyteCache partly for exactly this: you get atomic counters and TTLs, the two things a rate limiter actually needs, without a server. The counters live in a SQLite file on the same machine, so they’re fast and they don’t vanish when you restart.

It’s a good fit when:

• You want to cap requests per user, API key or IP for a service running on one machine (even with several worker processes).
• You need rate limiting in dev, CI or a small deployment and don’t want to spin up Redis just for that.
• Each of your services manages its own limits. That’s how I run it across my own microservices: every service gets its own cache volume.

All the examples use the Python library, lytecache 0.2.0 on PyPI. You’ll need Python 3.9 or newer, then just pip install lytecache. The Java, TypeScript and Go versions have the same operations (named the way each language prefers), and they all read and write the same file format.

Why use LyteCache for rate limiting

Here’s the pitch. LyteCache gives you the correctness you’d expect from a proper shared counter, with the simplicity of something that just lives inside your app. No server, no network, no setup.

In practice, that means:

• There’s nothing to set up. No connection strings, passwords, TLS or extra container. The limiter just ships inside your service.
• Your limit means exactly what it says. Run 4 Gunicorn workers and set a limit of 100 a minute, and you get 100 a minute, because every worker on the machine shares the same count.
• Limits survive restarts and deploys. Counters are on disk, so a deploy doesn’t hand everyone a fresh quota.
• It’s fast. Every check is a local, in-process write. There’s no network round trip on your hot path.
• Nothing new to learn. The patterns below use INCR and EXPIRE exactly the way you already know them from Redis.
• You can see what’s going on. The CLI lists every limiter key and how long it has left, and you can unblock someone with a single del.
• Same setup everywhere. Dev, CI and production all run the same code. pip install lytecache and you're done.

Core primitives

You only need four operations for everything in this guide. The one that matters most is incr. Under the hood it's a single SQL UPSERT inside a transaction, so two requests hitting it at the same moment, even from different processes, never lose a count.

• incr(key, amount=1) (like Redis INCR): counts a request and hands back the new total, atomically.
• expire(key, ttl) (like EXPIRE): makes a window's counter disappear when the window ends.
• ttl(key) (like TTL): tells you how many seconds are left until the window resets, which is perfect for a Retry-After header.
• get / set(key, value, ttl=...) (like GET / SET EX): reads or stores a bit of state, such as a token bucket.

I’d recommend giving the limiter its own namespace, so its keys never get mixed up with whatever else you’re caching:

from lytecache import LyteCache

limits = LyteCache(namespace="ratelimit")   # zero-config: no path needed

Two habits will save you trouble in every pattern below:

1. Put the window number in the key, like rl:user42:28912345. That way an old key can never be counted by mistake, even if its TTL somehow didn't get set.
2. Think of TTLs as cleanup, not as the thing keeping your limiter correct. Expired keys are cleared when they’re read and by the background sweep, which runs every sweep_interval seconds (60 by default).

Pattern 1: fixed window

If you’re not sure which pattern to use, start here. A fixed window lets a client make up to N requests per window, say 100 a minute, and it costs you one incr per request. It's simple, and for most APIs it's all you need.

import math
import time
from dataclasses import dataclass

@dataclass
class Decision:
    allowed: bool
    limit: int
    remaining: int
    reset_after: float   # seconds until the window resets

def fixed_window(cache, identity: str, limit: int, window: int) -> Decision:
    now = time.time()
    window_id = int(now // window)
    key = f"rl:fw:{identity}:{window_id}"

    count = cache.incr(key)               # atomic across threads and processes
    if count == 1:
        cache.expire(key, window + 1)     # first hit opens the window; +1s slack

    reset_after = (window_id + 1) * window - now
    return Decision(
        allowed=count <= limit,
        limit=limit,
        remaining=max(0, limit - count),
        reset_after=reset_after,
    )

A few things worth knowing about how this behaves:

• The first request in a window creates the counter and sets its TTL. After that, requests just bump the number.
• Blocked requests still count. That’s on purpose: if a client keeps hammering you, they stay blocked until the window is over.
• If your process happens to crash between incr and expire, the key just sits there without a TTL. Nothing breaks, because the next window uses a different key. It's a tiny leak, and eviction or a manual delete will clear it.

If you’d like the limit to feel smoother right at the edge of each window, Pattern 2 builds on this with one extra read.

Pattern 2: sliding window (approximate)

This one smooths out that edge burst. The trick is to look at the previous window’s count as well as the current one, and blend them. It costs one incr and one get per request, and you don't need any fancy data structures.

Get Samson Ilemobayo’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

The previous window counts in proportion to how much of it still falls inside the last window seconds:

def sliding_window(cache, identity: str, limit: int, window: int) -> Decision:
    now = time.time()
    window_id = int(now // window)
    elapsed = now - window_id * window

    cur_key = f"rl:sw:{identity}:{window_id}"
    prev_key = f"rl:sw:{identity}:{window_id - 1}"

    current = cache.incr(cur_key)
    if current == 1:
        cache.expire(cur_key, 2 * window + 1)   # must outlive the next window too

    previous = cache.get(prev_key, 0)     # get() returns the default on miss
    weight = 1 - (elapsed / window)
    estimate = previous * weight + current

    return Decision(
        allowed=estimate <= limit,
        limit=limit,
        remaining=max(0, int(limit - estimate)),
        reset_after=window - elapsed,
    )

A couple of notes:

• The TTL is twice the window length because each counter gets read again, as previous, during the window that follows.
• This is the same weighted-counter approach plenty of big API gateways use. It’s accurate enough for real traffic and very cheap to run.

Pattern 3: token bucket

Reach for a token bucket when your clients have a good reason to send bursts, like syncing a batch of records, but you still want a steady average. Because it reads and updates two values at once, it runs inside a LyteCache lock. Expect it to cost roughly two to three times as much as Pattern 1.

def token_bucket(cache, identity: str, capacity: int, refill_per_sec: float) -> Decision:
    key = f"rl:tb:{identity}"
    full_after = capacity / refill_per_sec          # seconds to refill from empty

    with cache.lock(f"lock:{key}", timeout=1):      # one writer per bucket
        now = time.time()
        state = cache.get(key) or {"tokens": capacity, "ts": now}

        elapsed = max(0.0, now - state["ts"])
        tokens = min(capacity, state["tokens"] + elapsed * refill_per_sec)

        allowed = tokens >= 1
        if allowed:
            tokens -= 1

        cache.set(key, {"tokens": tokens, "ts": now}, ttl=math.ceil(full_after) + 1)

    wait = 0.0 if allowed else (1 - tokens) / refill_per_sec
    return Decision(
        allowed=allowed,
        limit=capacity,
        remaining=int(tokens),
        reset_after=wait,
    )

A few notes:

• The TTL is set to however long the bucket takes to refill from empty. So when the key expires, the bucket would have been full anyway, and “no key” correctly means “full bucket”.
• As an example, capacity=20, refill_per_sec=5 lets a client burst 20 requests, then settle at 5 per second.
• The lock stops two processes from spending the same token. Keep its timeout short, so that if a process dies while holding it, nobody waits long.

Integrating as middleware

To use any of these in a real app, wrap it in middleware. The middleware works out who’s calling, sends back 429 Too Many Requests when they're over the limit, and always includes the usual rate-limit headers so clients know where they stand. Here's what that looks like in FastAPI with the fixed window:

from fastapi import FastAPI, Request
from fastapi.responses import JSONResponse
from lytecache import LyteCache

app = FastAPI()
limits = LyteCache(namespace="ratelimit")

LIMIT, WINDOW = 100, 60     # 100 requests per minute

def identity_of(request: Request) -> str:
    api_key = request.headers.get("x-api-key")
    return f"key:{api_key}" if api_key else f"ip:{request.client.host}"

@app.middleware("http")
async def rate_limit(request: Request, call_next):
    d = fixed_window(limits, identity_of(request), LIMIT, WINDOW)
    headers = {
        "X-RateLimit-Limit": str(d.limit),
        "X-RateLimit-Remaining": str(d.remaining),
        "X-RateLimit-Reset": str(math.ceil(d.reset_after)),
    }
    if not d.allowed:
        headers["Retry-After"] = str(math.ceil(d.reset_after))
        return JSONResponse({"error": "rate_limited"}, status_code=429, headers=headers)

    response = await call_next(request)
    response.headers.update(headers)
    return response

Some things I’ve learned the hard way:

• Think about who you’re limiting. A user ID or API key is better than an IP address. If you’re behind a proxy or load balancer, read the client IP from a X-Forwarded-For hop you trust, not straight from the header, or anyone can spoof it.
• Give sensitive routes their own limits. Add the route group to the key (login:ip:1.2.3.4) so things like login or OTP can be much stricter than your read endpoints.
• Mind the event loop. LyteCache calls are synchronous SQLite writes and usually take well under a millisecond. If you’re pushing serious traffic through an async server, run the check with run_in_threadpool.
• Decide what happens if the cache fails. If the cache file isn’t available, do you let traffic through or block it? Letting it through keeps you up. Blocking it is safer for things like payment endpoints. Pick one on purpose.

Deployment and operations (Docker)

If you run in containers, give each service its own volume for the cache file. Your limits then survive restarts, and services never end up sharing counters by accident.

services:
  merchant:
    environment:
      LYTECACHE_PATH: /var/cache/lytecache/cache.db
    volumes:
      - lytecache-merchant:/var/cache/lytecache

volumes:
  lytecache-merchant:

You can peek at the limiter’s state any time with the CLI. Pass the same Compose project name (-p) your stack runs under, so the CLI opens the right volume.

# every active limiter key, with TTLs
docker compose -p sandbox run --rm lytecache-cli --namespace ratelimit keys 'rl:*' --long

# hit/miss, key count and size
docker compose -p sandbox run --rm lytecache-cli --namespace ratelimit stats

# unblock a client manually
docker compose -p sandbox run --rm lytecache-cli --namespace ratelimit del 'rl:fw:key:abc123:28912345'

A few tips for running it well:

• Share one file across your workers. Point every process on the machine at the same cache file. LyteCache uses SQLite’s WAL mode, so incr and lock stay atomic across processes.
• Give the limiter its own instance. Create a dedicated LyteCache(namespace="ratelimit") without max_keys or max_bytes, so your counters are always kept.
• Keep the file on local disk. That’s where SQLite is fastest and its locking is most reliable.
• Keep your clock in sync. Windows are based on wall-clock time, so run NTP like you normally would.
• Tidy up now and then. TTLs and the background sweep keep the key count in check. Call vacuum() occasionally to reclaim disk space.

Sources

• lytecache on PyPI (v0.2.0): API, configuration and concurrency notes
• lytecache Documentation: API, configuration for Python and other Language