Part 1 built the thing: a cache in front of the database, a queue for the clicks, a redirect that returns in a few milliseconds. What I didn’t have was a number: how many redirects per second that shape sustains, and which component runs out of capacity first.

So I pointed k6 at it and pushed until something gave. It wasn’t the component I’d designed the path around.

Key takeaways:

• A fixed-VU load test measures concurrency, not throughput, and reports latency inflated by its own queueing.
• Use k6’s ramping-arrival-rate executor when the target is stated as a rate.
• On a ramped run, the reported average RPS is diluted by the ramps. Read the P95 at the hold instead.
• At 3,000 req/s the single Node process saturated one CPU core while Redis and Postgres sat at ~25%.

🔗 Live: linkpulse.thedeepak.dev

📦 Repo: github.com/i-am-chitti/linkpulse.

🧪 What’s covered:

• The hot path being measured
• First test: fixed VUs, wrong measurement
• Results
• Past the target: one CPU core
• Limitations

The hot path, briefly

What GET /:shortCode costs on a warm key (redirect.ts):

1. A Lua rate-limit call to Redis.
2. A regex gate on the code.
3. A Redis GET for the cached destination.
4. A 302.
5. After the response is sent, not awaited: an LPUSH of the click event to Redis. A separate worker drains that in batches of 100 into Postgres.

Two synchronous Redis calls, no database. Redis should be the ceiling, if anything is. That was my bet.

Setup: Node 24, Express 5, one API process; Redis 7; PostgreSQL 16; k6. Everything ran in Docker Compose on one laptop (Apple M2 Pro, 10 cores, 16 GB), not the live EC2 instance. One overlay file (docker-compose.loadtest.yml) raises the rate limiter’s ceilings, because k6 traffic comes from one IP and the real anonymous limit would throttle the load generator before the hot path was under any load.

First test: fixed VUs, wrong measurement

The target: over 2,500 RPS on the redirect, P95 under 50ms.

The first k6 script was the obvious one: 500 virtual users in a closed loop, each firing a request, getting the response, firing the next. It blew past 2,500 RPS and reported a P95 around 170ms.

That’s not the system failing. It’s the wrong measurement.

A closed-loop VU test controls how many clients are connected. Each VU sends the instant the previous request returns, so throughput is whatever latency permits: 500 VUs at 2ms each is 250,000 RPS in theory, at 200ms it’s 2,500. The request rate isn’t something you set; it’s an output. And with 500 clients waiting on one single-process API, the latency they see includes time queued behind each other. The 170ms didn’t mean the redirect path is slow. It meant 500 clients with no think-time will saturate whatever they hit, then measure the saturation.

The question I wanted answered was: if 2,500 requests arrive per second, is each answered in under 50ms? Arrival rate is the input, latency is the output, client count is an implementation detail. k6’s ramping-arrival-rate executor asks for the rate directly and allocates however many VUs it needs (redirect-load-test.js):

export const options = {
  scenarios: {
    redirect_throughput: {
      executor: 'ramping-arrival-rate',
      timeUnit: '1s',
      startRate: 0,
      preAllocatedVUs: 300,
      maxVUs: 1000,
      stages: [
        { target: 1250, duration: '20s' },
        { target: 2500, duration: '20s' },
        { target: 2500, duration: '2m' },  // hold at target
        { target: 0,    duration: '20s' },
      ],
    },
  },
  thresholds: {
    http_req_duration: ['p(95)<50'],
    http_req_failed: ['rate<0.01'],
  },
};

No sleep() in the iteration; the executor owns the rate.

The other three scenarios (create, analytics, mixed) keep fixed VUs, because their targets are “stable with N concurrent users”, which is a closed-loop question. Only the redirect target is phrased as a rate.

Results

All thresholds passed (RESULTS.md). Two notes on the redirect row.

The 2,077 is a whole-run average, diluted by 40 seconds of ramp-up and 20 of ramp-down. The two-minute hold at 2,500 supplies about 80% of the requests, so the P95 is the number to trust: the service held 2,500 req/s at 40ms.

The gap between P50 and P95. 0.8ms median, 40ms at the 95th. A warm redirect is sub-millisecond. The tail is what happens when 2,500 of them a second land on one thread.

Past the target: one CPU core

Passing a threshold says the system is fine at the number you chose. It doesn’t say why, or how far past that number it stays fine. So I raised the hold to 3,000 req/s and watched docker stats.

P95 crossed 50ms. Container CPU:

• api: ~110-130%
• redis: ~20-26%
• postgres: ~20-26%

Node runs JavaScript on a single thread. 110–130% means that one core is full, with a little extra from garbage collection and I/O helper threads. Redis and Postgres were barely awake.

The cache I’d designed the path around had three-quarters of its capacity spare. The database that isn’t even on the warm path was idle. The bottleneck was the box doing the least work per request (parse HTTP, two middleware, two Redis calls, a 302), because it was doing it 3,000 times a second on one thread. The architecture diagram has a box labelled API, and nothing in it says that the box is one core.

The fix is horizontal: API replicas behind a load balancer. The design already allows it. JWTs are stateless, refresh tokens live in Postgres and rate-limit state is in Redis, so there are no sticky sessions and no code change. I haven’t built it. The diagnosed limit and the documented next step were the point; a second replica would move the same question one layer out.

Limitations

• The load generator shared the machine with the system under test. k6 was running 300+ VUs on the same 10 cores as the API container, so some of the API’s contention was k6. On a dedicated host the ceiling is probably somewhat higher. The shape of the finding survives; the exact number doesn’t.
• These are not numbers from the live deployment. Production is one EC2 instance behind Caddy with an external Neon Postgres. I haven’t measured it and I’m not quoting a laptop number as a production number.
• The cache was warm and held only 20 keys, so every redirect request was a hit. Real traffic has misses, expiries and scanners hitting the negative cache, all of which touch Postgres.
• These are one run’s numbers, not medians across repeats.

Takeaways

• The executor is part of the hypothesis. “Handles 2,500 RPS” and “handles 500 concurrent users” are different sentences, and a test that measures one can’t prove the other. My first test was a correct measurement of a question I hadn’t asked.
• Pass the threshold, then break it. The passing run said the design works at target. The 3,000 req/s run said why, and it was a different why than I’d have written down beforehand.
• Write the limit down. A table with four green ticks is marketing. The same table with “the ceiling is one core, here’s docker stats, here's the next step" is the only version that told me something I didn't already believe.

Part 1: Building a URL shortener: six design decisions and the alternatives I rejected.

Benchmark scripts and reporter: linkpulse/benchmarks.

The URL shortener is the hello world of system design: hash the URL, store the mapping, cache the hot ones, redirect. Four boxes.

I built one and put it online, with a real login, real analytics, and a public URL that anyone (and any bot) can hit. Running it turned up a second layer of decisions underneath those four boxes. Which redirect status code to send. Where the click gets counted. What happens when two requests race the rate limiter. What a stolen refresh token looks like from the server’s side.

This post is six of those decisions, each with the alternative I rejected and why.

🔗 Live: linkpulse.thedeepak.dev

📦 Repo: github.com/i-am-chitti/linkpulse.

Key takeaways:

• Send a 302, not a 301. Browsers can cache a 301 indefinitely, so the service stops seeing the clicks it is trying to count.
• Cache the misses as well as the hits, or a bot scanning random codes turns every probe into a database query.
• Do not rate limit with INCR and EXPIRE. A fixed window lets through twice the configured rate at the boundary.
• Enrich clicks in a separate process. The geo-IP database costs 110 MB of memory that the redirect path never needs.

🧪 What’s covered:

• What it does, and the performance targets
• Architecture
• Redirect hot path: cache, negative cache, 302 vs 301
• Click tracking: async queue and a separate worker process
• Rate limiting: sliding window in a Lua script, and why a rate limit alone isn’t enough
• Auth: refresh token rotation with reuse detection
• What I left out: Safe Browsing API and horizontal scaling
• Deployment: one VM with Docker Compose

What it does

LinkPulse shortens a URL to a random base62 code or a custom alias. Registered users get per-link analytics: clicks over time, unique visitors, countries, devices, browsers and referrers. Visitors can also shorten a link from the landing page without an account, and those guest links expire after 24 hours. Every create and edit passes through a rate limiter, a per-account link quota and a malicious-URL blocklist, and the anonymous browser forms are behind a Cloudflare Turnstile challenge.

Targets, set before writing code:

• Redirect GET /:shortCode: over 2,500 RPS, P95 under 50ms
• Create POST /api/links: over 200 RPS, P95 under 200ms
• Analytics read: over 100 RPS, P95 under 300ms

The redirect is the only operation that happens at machine speed; everything else happens as fast as people fill in forms. Every decision below follows from keeping the redirect path cheap.

Architecture

Two things worth pointing out in this diagram. The redirect path is Browser → API → Redis, and on a cache hit that’s all: Postgres and the worker are not involved. Redis is hit twice per redirect, once for the rate limiter and once for the cache, so the rate limiter is on the hot path too. And the worker is a separate process, not a background function in the API, for a reason I’ll measure below.

Redis does three jobs here: cache, click queue, rate-limiter state. Fine at this scale, and one fewer thing to run. It’s the first thing I’d split if Redis showed up as the bottleneck. It didn’t.

1. Redirect hot path

redirect.ts does, in order:

1. Rate-limit check. One Lua script call to Redis (decision 3 below).
2. Regex gate. A valid code is 3–32 characters of [a-zA-Z0-9_-] (generated codes are 7; custom aliases can be longer). Anything else is a 404 before the cache is consulted, so scanners can't fill it with keys that could never be valid.
3. Cache lookup (cacheService.ts). On a hit, that’s the second and last Redis call.
4. On a miss, Postgres, then write back. Misses are cached too: a code that doesn’t exist is stored as missing for 60 seconds, so a bot probing random codes costs one query per code per minute instead of one per request. “Gone” (expired or disabled) is cached separately so the response can say which.
5. Respond with a 302.

Why 302 and not 301. A 301 is permanent, and browsers cache it, sometimes indefinitely, so subsequent clicks never reach the service. That freezes the destination (breaking edit) and silently stops counting clicks. So 302, plus Cache-Control: no-store.

Rejected: querying Postgres on every redirect. That is fine at low traffic, but it ties the one latency-critical path to whatever else the database is doing.

Short codes are 7 characters from a 62-character alphabet, 6²⁷ ≈ 3.5 trillion, generated randomly rather than by hashing the URL. Hashing would give the same URL the same code, so two users couldn’t each have their own analytics for the same link. A collision on insert regenerates and retries (urlService.ts, base62.ts). In practice it doesn’t fire.

2. Click tracking

Every click needs a timestamp, IP → country, user agent → device and browser, and the referrer. None of that belongs in the request.

The handler sends the 302, then pushes a small JSON event onto a Redis list without awaiting it (clickQueue.ts). If the push fails, the visitor was still redirected; a lost analytics row is the right thing to lose. A separate worker (worker.ts) pops in batches of 100, enriches (clickProcessor.ts), and batch-inserts into Postgres.

Why a separate process rather than an async function in the API: the geoip-lite database is about 110 MB resident once loaded. Measured live, the API container sits at ~50 MB and the worker at ~170 MB. Loading geo-IP into the API would tax every redirect for a lookup the redirect never performs, and a burst of clicks would compete with redirects for the same event loop.

The trade-off: the worker pops a batch, then writes it, so a crash in between loses up to 100 clicks. For analytics that is acceptable. An acknowledged queue is the fix if it ever isn’t.

The worker also runs the hourly sweep of expired guest links and stale refresh tokens. It already touches Postgres on its own schedule, so the housekeeping was free.

3. Rate limiting

The textbook Redis rate limiter is a fixed window: INCR a key named after the current minute, EXPIRE it, reject over the limit. Three lines, and wrong by up to 2x.

With a 100-per-minute limit, a client sends 100 requests at 11:59:59.9 and 100 more at 12:00:00.1. Both windows see 100, both allow, and the client has done 200 requests in 200 milliseconds.

The fix is a sliding window counter: the current window’s count exactly, plus the previous window’s count weighted by how much of it still overlaps the trailing minute. At 12:00:15 the previous window contributes 75%. A client who maxed the tail of the last window is still bounded.

The second problem is concurrency. If the check is a GET, a compare in Node, then an INCR, two requests can both read 99 and both pass. So the whole decision is one Lua script on the Redis side (rateLimiter.ts; per-route tiers in rateLimit.ts):

local current  = tonumber(redis.call('GET', KEYS[1]) or '0')
local previous = tonumber(redis.call('GET', KEYS[2]) or '0')
local limit    = tonumber(ARGV[1])
local elapsed  = tonumber(ARGV[2])   -- fraction of current window elapsed, 0..1
local ttl      = tonumber(ARGV[3])

local weighted = previous * (1 - elapsed) + current

if weighted >= limit then
  return { 0, math.floor(weighted + 0.5) }
end

current = redis.call('INCR', KEYS[1])
redis.call('EXPIRE', KEYS[1], ttl)
return { 1, math.floor(weighted + 0.5) + 1 }

Loaded once, called via EVALSHA, one round-trip. Redis runs one script at a time, so two requests can't interleave inside it.

Two more choices sit on top of that script:

• Two tiers per route. Anonymous callers are keyed by IP with a tight limit; authenticated callers by user ID with a looser one. Configured separately for redirects, link creation, general API and auth, because someone hammering the login form and someone’s link going viral are different problems.
• Fail open. If Redis is down, requests go through. The limiter exists to protect Redis and Postgres from abusive load, and blocking everyone because the thing it protects is unavailable gets the priorities backwards.

A rate limit bounds speed, not identity

The per-user limit has a hole in it that took me a while to see. Accounts are free, so an identity is not a scarce resource. One machine that registers ten accounts gets ten times the per-user creation budget, and every link it creates is permanent. The limiter is working exactly as designed and the table still fills up.

Three layers close that, and each does something the other two cannot:

• A per-IP link-creation budget on `POST /api/links`, charged on top of the per-user limit and keyed by IP no matter who is signed in. Registering more accounts buys no extra creation rate from one machine.
• A per-account link quota (`MAX_LINKS_PER_USER`, 500 by default). Rate limits bound how fast links appear; this bounds how many exist. Without it an account can sit just under the rate ceiling forever and still grow the table without limit.
• A Cloudflare Turnstile challenge on the anonymous browser forms: register, login and guest shorten. Optional, and off unless a secret key is configured.

4. Auth

The requirement: sessions that survive a refresh, can be revoked, and don’t hand an attacker a long-lived credential after one XSS.

Access tokens are 15-minute JWTs held in browser memory only. Not localStorage, not a cookie. A script injected via XSS can’t read what isn’t stored.

Refresh tokens are opaque random strings, stored in Postgres as a SHA-256 hash so a database dump isn’t a session dump, delivered in an httpOnly cookie that the browser only sends to the refresh endpoint.

Every refresh rotates (authService.ts, tokens.ts): present a refresh token, get a new one, the old one is revoked. The interesting part is a revoked token showing up again. That means two parties hold it, the user and whoever copied it, and the server can’t tell which is which. So it revokes the whole token family and forces both to log in. The user gets a login prompt; the attacker gets one they can’t answer. This is the standard rotation-with-reuse-detection pattern from the OAuth 2.0 Security BCP (RFC 9700); the only decision here was to implement it directly rather than through a library.

Rejected: NextAuth.js. It issues and verifies sessions inside the Next.js server. Here sessions are owned by a separate Express API that also serves k6, curl and anything else with a bearer token. NextAuth would have meant two session systems, or restructuring auth around Next.js. The hand-rolled version is one design, checked the same way everywhere: rate-limit tiers, ownership checks, the OAuth callback.

One consequence: the system does not support account linking. Sign up with GitHub, then register with the same email via password, and the second attempt is rejected rather than merged. Merging safely needs its own confirmation flow, and a quick version of it is a way to take over someone else’s account.

5. What I left out

I skipped the Google Safe Browsing API. The malicious-URL check is a static domain blocklist (urlBlocklist.ts), shipping Google’s own documented Safe Browsing test domains by default so it demonstrably works without an API key. A real deployment would want live threat intelligence, but this project needs to run with docker compose up and no external account.

I also skipped horizontal scaling. The API runs as one process. The design is already stateless, since JWTs verify anywhere, refresh state is in Postgres and rate-limit state is in Redis, so a second replica needs no sticky sessions and no code change. I left it out because I didn’t yet know whether it was needed. Part 2 is about finding out.

6. Deployment

One EC2 instance running the same docker compose shape as local dev, minus Postgres (external, on Neon) and plus Caddy for HTTPS. Caddy routes /api/* and short codes to the API container and everything else to Next.js, so the browser talks to one origin.

Deploys are a manually triggered GitHub Actions workflow (deploy.yml, docker-compose.prod.yml, Caddyfile): build and push images to GHCR, run prisma migrate deploy against Neon, SSH in and restart the stack. Secrets come from GitHub, not a laptop.

Rejected: a PaaS. It is faster on day one, but its deploy config doesn’t carry to another provider, so moving later means building this VM-plus-Compose recipe anyway.

Also rejected: RDS. The point of an external database is that it stays put when the compute moves. RDS is still an AWS resource, so it would have to move too. Neon never does.

Summary

The four-box diagram is accurate as far as it goes. Each decision in this post sits one level below it:

• A cache in front of the database also has to cache the misses, or a bot scanning random codes reaches Postgres on every request.
• The redirect has to be a 302, or browsers stop sending the clicks you are counting.
• The rate limiter needs a sliding window, since INCR and EXPIRE let through twice the configured rate at a boundary.
• The click queue needs its own process, because the geo-IP lookup costs 110 MB of memory the redirect path never uses.

None of these needed traffic to find. They all came up while building on one machine.

What did need traffic was the question of which component runs out of capacity first. I assumed Redis. Part 2 measures it.

Part 2: Load-testing a URL shortener: the bottleneck was Node, not Redis or Postgres

Stack: Node 24, Express 5, TypeScript, PostgreSQL 16 via Prisma, Redis 7, Next.js 16; Vitest and Supertest for tests, k6 for load. Full write-up, including testing and security notes: ARCHITECTURE.md.

A practical, run-tested guide to unit testing WP-CLI commands with PHPUnit and Behat.

A migration command reassigns 40,000 posts from a legacy author ID to the WordPress user that owns them. It runs unattended in a deploy step. It processes 14,203 posts cleanly. Then it hits one post whose legacy ID isn’t in the mapping.

What happens next — does it stop, skip the post and keep going, write post_author = 0? Nobody knows, because nobody has run this exact case. The command works: it runs, it prints output, and for most CLI scripts that's where testing stops.

A broken web request breaks one page for one visitor, and you hear about it in minutes. A command that mishandles one row in a 40,000-row batch can quietly corrupt every row after it — running with full database access, unattended, often straight against production, with nobody reading the output. The failure mode isn’t a crash. It’s wrong data that looks completely fine.

This post covers what to prove about a command like that before it runs unattended, and the specific ways WP_CLI_Command classes resist PHPUnit more than the rest of a WordPress codebase.

📦 Full repo: github.com/i-am-chitti/wp-cli-command-testing

🧪 Table of Contents

1. The Command — What one unmapped row actually costs
2. WP_CLI_Command Doesn't Exist Until You Load It — WP-CLI as a Composer dependency
3. The Test That Watches Nothing — Why expectOutputString() never sees WP-CLI output
4. Proving the Command Stops — WP_CLI::$capture_exit is private; what to do instead
5. What runcommand() Is Actually For — And why PHPUnit can't reach it
6. What Unit Tests Can’t See — Behat against the real wp binary
7. Wiring It Into CI — Matrix floors, credentials, and versions that don’t vary
8. Key Takeaways

1. The command, and what’s actually at stake

The command, trimmed to the relevant part:

class Migrate_Authors_Command extends WP_CLI_Command {

    public function run( $args, $assoc_args ) {
        $mapping  = $this->load_mapping( $assoc_args['map'] );
        $post_ids = get_posts( [
            'post_type'      => 'post',
            'fields'         => 'ids',
            'posts_per_page' => -1,
        ] );

        $migrated = 0;

        foreach ( $post_ids as $post_id ) {
            $legacy_id = get_post_meta( $post_id, '_legacy_author_id', true );

            if ( ! isset( $mapping[ $legacy_id ] ) ) {
                WP_CLI::error( "No mapping for legacy author ID {$legacy_id} (post {$post_id})" );
            }

            wp_update_post( [ 'ID' => $post_id, 'post_author' => $mapping[ $legacy_id ] ] );
            $migrated++;
        }

        WP_CLI::success( "{$migrated} posts migrated." );
    }
}

WP_CLI::error() inside the loop does all the safety work here. It halts the command on the first bad row.

That safety is one refactor away from disappearing. Change it to log a warning and continue — a reasonable-looking change, so one bad row doesn't kill a long run — and every post after that row keeps its old author, the command still exits 0, and the deploy reports success.

So the property worth testing is: an incomplete mapping stops the command, and posts already migrated stay migrated. Proving that means working around three things WP-CLI does deliberately.

2. WP_CLI_Command doesn't exist until you load it

Run a plain PHPUnit test against that command and it fails before the first assertion:

Error: Class "WP_CLI_Command" not found

WP_CLI_Command only exists inside the wp binary, and PHPUnit never touches that code path.

The fix is to install wp-cli/wp-cli as a normal Composer dev dependency. Its own composer.json already points at the classes, so vendor/autoload.php loads WP_CLI_Command for you — no hunting for a wp binary on disk.

{
  "require-dev": {
    "wp-cli/wp-cli": "^2.11",
    "wp-phpunit/wp-phpunit": "^7.1",
    "yoast/phpunit-polyfills": "^2.0",
    "phpunit/phpunit": "^9.6",
    "roots/wordpress": "^7.1"
  },
  "extra": {
    "wordpress-install-dir": "wordpress"
  }
}

wp-phpunit is only WordPress's test suite; roots/wordpress is what puts core itself on disk, so composer install is the only setup step.

PHPUnit is pinned to ^9.6, not the ^9.6 || ^10.5 you'll see suggested elsewhere. The 10.5 release I tried removed an internal PHPUnit\Util\Test method that wp-phpunit's test-case class still calls — not an assertion, so phpunit-polyfills doesn't cover it. Check that combination yourself before trusting it.

tests/bootstrap.php ties it together:

require_once dirname( __DIR__ ) . '/vendor/autoload.php';

putenv( 'WP_TESTS_PHPUNIT_POLYFILLS_PATH=' . dirname( __DIR__ ) . '/vendor/yoast/phpunit-polyfills' );

$_tests_dir = getenv( 'WP_TESTS_DIR' ) ?: dirname( __DIR__ ) . '/vendor/wp-phpunit/wp-phpunit';
require $_tests_dir . '/includes/functions.php';

tests_add_filter( 'muplugins_loaded', function () {
    require dirname( __DIR__ ) . '/plugin.php';
} );

require $_tests_dir . '/includes/bootstrap.php';

yoast/phpunit-polyfills needs a word. WordPress's test tools call assertion methods that newer PHPUnit versions renamed or removed; the polyfill restores the old names as thin wrappers, and WordPress's bootstrap picks it up via WP_TESTS_PHPUNIT_POLYFILLS_PATH. Without it the suite breaks differently depending on which PHPUnit is installed, and local and CI don't always match.

🔗 composer.json

🔗 tests/bootstrap.php

3. The test that watches nothing

This test looks reasonable and passes:

public function test_run_reports_success() {
    ( new Migrate_Authors_Command() )->run( [], [ 'map' => $this->fixture_map() ] );

    $this->expectOutputString( '3 posts migrated.' );
}

It also asserts nothing.

WP_CLI::success(), log(), and error() don't use echo. They go through a Logger — a small class whose only job is printing to your terminal — and the default one writes straight to php://stdout, skipping PHP's output buffer entirely. expectOutputString() and ob_start() only see that buffer, so the test above was comparing against an empty string the whole time. A command that logs nothing and one that logs correctly look identical to it.

set_logger() swaps in a Logger you can read from:

class Spy_Logger extends \WP_CLI\Loggers\Base {
    public array $messages = [];

    public function info( $message, $newline = true ) { $this->messages['info'][] = $message; }
    public function success( $message ) { $this->messages['success'][] = $message; }
    public function warning( $message ) { $this->messages['warning'][] = $message; }
    public function error( $message ) { $this->messages['error'][] = $message; }
}

It’s extends \WP_CLI\Loggers\Base, not an interface called Logger. Base only forces info(), success() and warning() on you — but WP_CLI::error() calls $logger->error() directly, so leave that one out and you get "call to undefined method" the first time a real error fires.

Spy_Logger.php

public function test_run_migrates_every_post_with_a_valid_mapping() {
    $logger = new Spy_Logger();
    \WP_CLI::set_logger( $logger );

    $post_ids = $this->create_posts_with_legacy_authors( [ 101, 102, 103 ] );
    ( new Migrate_Authors_Command() )->run( [], [ 'map' => $this->fixture_map() ] );

    foreach ( $post_ids as $post_id ) {
        $this->assertNotSame( 0, get_post( $post_id )->post_author );
    }
    $this->assertStringContainsString( '3 posts migrated', $logger->messages['success'][0] );
}

(fixture_map() is a stand-in here — the actual test builds the mapping file inline with tempnam(): Migrate_Authors_Command_Test.php.)

The first assertion checks post_author in the database, not the log message. The log line reports what the command says it did; the database holds what it did.

The Logger isn’t the only output path. --format=json, --format=csv and --format=yaml use plain echo, so ob_start() captures those normally. Only log, success, warning and error messages bypass the buffer.

4. Proving the command actually stops

This is the test that catches a hard stop being turned into a skip. Two approaches don’t work.

Setting WP_CLI::$capture_exit = true before calling run() would make error() throw a catchable WP_CLI\ExitException instead of calling exit(). The property exists and controls exactly that, but it's private — a test file can't set it.

The only thing that flips it is runcommand() internally, with exit_error => false. Routing the call through runcommand() hits a harder wall: it goes through WP-CLI's Runner, and a Runner that was never started has a null config, so it dies on in_array('migrate-authors', null, true). Populating that by hand means reimplementing WP-CLI's boot sequence.

The alternative is to change what the test touches. The loop that matters — walk the posts, stop on the first unmapped one — doesn’t need WP_CLI at all:

public function migrate( array $mapping ): int {
    $post_ids = get_posts( [ 'post_type' => 'post', 'fields' => 'ids', 'posts_per_page' => -1 ] );
    $migrated = 0;

    foreach ( $post_ids as $post_id ) {
        $legacy_id = get_post_meta( $post_id, '_legacy_author_id', true );

        if ( ! isset( $mapping[ $legacy_id ] ) ) {
            throw new Mapping_Error( "No mapping for legacy author ID {$legacy_id} (post {$post_id})" );
        }

        wp_update_post( [ 'ID' => $post_id, 'post_author' => $mapping[ $legacy_id ] ] );
        $migrated++;
    }

    return $migrated;
}

run() becomes a thin wrapper around it:

public function run( $args, $assoc_args ) {
    $mapping = $this->load_mapping( $assoc_args['map'] );

    try {
        $migrated = $this->migrate( $mapping );
    } catch ( Mapping_Error $e ) {
        WP_CLI::error( $e->getMessage() );
        return;
    }

    WP_CLI::success( "{$migrated} posts migrated." );
}

Now the test that matters doesn’t touch WP_CLI:: at all:

public function test_migrate_stops_before_touching_posts_past_an_unmapped_id() {
    $mapped_author   = self::factory()->user->create();
    $original_author = self::factory()->user->create();

    // Post A has a mapped legacy author. Post B does not. Post C comes after B.
    [ $post_a, $post_b, $post_c ] = $this->create_posts_with_legacy_authors( [ 101, 999, 102 ] );
    foreach ( [ $post_a, $post_b, $post_c ] as $post_id ) {
        wp_update_post( [ 'ID' => $post_id, 'post_author' => $original_author ] );
    }

    $this->expectException( Mapping_Error::class );

    try {
        ( new Migrate_Authors_Command() )->migrate( [ '101' => $mapped_author, '102' => $mapped_author ] );
    } finally {
        $this->assertSame( $mapped_author, (int) get_post( $post_a )->post_author, 'Post before the failure should be migrated.' );
        $this->assertSame( $original_author, (int) get_post( $post_c )->post_author, 'Post after the failure should be untouched.' );
    }
}

Change the error handling to skip-and-continue and this fails immediately, long before a real 40,000-row run. No ExitException, no capture_exit, no simulated runtime — a plain exception and a database assertion. What it doesn't cover is run()'s own error path, whether it really calls WP_CLI::error() and exits. That's Behat's job, below.

🔗 Migrate_Authors_Command.php

🔗 Migrate_Authors_Command_Test.php

Note the casts. post_author comes back as a numeric string — a raw DB row property WordPress doesn't convert — so assertSame() fails without them. assertEquals() would hide the difference.

5. What runcommand() is actually for

run should still refuse to start on a mapping nobody has verified — say, by calling a separate wp migrate-authors verify-mapping. WP_CLI::runcommand() exists for exactly that: one command invoking another by name, with no hard class dependency.

Testing that composition from PHPUnit hits the same wall as Section 4: no started Runner. That constraint is tolerable, because runcommand() is worth its cost mainly when the two commands live in different, independently-versioned packages — where a direct call isn't available anyway. That case only executes inside a live wp process, which makes it Behat's to verify.

When both commands are in the same package, a plain method call does the job and stays testable:

public function run( $args, $assoc_args ) {
    $mapping = $this->load_mapping( $assoc_args['map'] );

    $invalid = $this->find_invalid_mapping( $mapping );

    if ( null !== $invalid ) {
        WP_CLI::error( $invalid );
        return;
    }

    // ...
}

It calls find_invalid_mapping(), not verify_mapping(). The subcommand ends in WP_CLI::success( '...mappings verified.' ), which is correct when someone runs it by hand and noise as a pre-flight check. Keeping the check in a private method that returns a message or null lets each caller report it appropriately.

The rule: use runcommand() when you don't control the other command's class. Otherwise call the method — that's the version PHPUnit can reach.

6. What unit tests can’t see

Everything so far tests the command’s logic with WP-CLI’s internals simulated around it. None of it verifies that wp migrate-authors run --help produces sensible help text, or that the command behaves when invoked exactly as it would be on a real server.

WP-CLI ships an official tool for that gap: wp-cli/wp-cli-tests, which runs your command through the real wp binary against a throwaway WordPress install, using Behat. Scenarios read like plain English — set up state, run a real command, check the output:

Feature: migrate-authors

  Background:
    Given a WP install
    And I run `wp user create migrated-author migrated-author@example.com --porcelain`
    And save STDOUT as {MIGRATED_AUTHOR_ID}

  Scenario: Halting on a post whose legacy author ID isn't mapped
    Given I run `wp post create --post_title="Unmapped post" --post_status=publish --porcelain`
    And save STDOUT as {POST_ID}
    And I run `wp post meta update {POST_ID} _legacy_author_id 999`
    And a wp-content/mu-plugins/author-mapping.csv file:
      """
      101,{MIGRATED_AUTHOR_ID}
      """

    When I try `wp migrate-authors run --map=wp-content/mu-plugins/author-mapping.csv`
    Then STDERR should contain:
      """
      No mapping for legacy author ID 999
      """
    And the return code should be 1

Every step there ships with wp-cli-tests' FeatureContext — nothing custom. There's no shorthand for "a post with a given meta value"; you compose it from wp post create and wp post meta update, same as you would at a terminal.

Use I try, not I run. I run fails the scenario as soon as the command exits non-zero, which is what this scenario asserts.

(Trimmed — the feature file also covers the successful path.)

Behat runs alongside PHPUnit, not instead of it. PHPUnit runs in milliseconds and takes every edge case; Behat boots an actual WordPress site, so it covers only the scenarios that need the real binary.

Two things to know before starting. WP_CLI_TEST_DBTYPE=sqlite removes the database server requirement entirely — FeatureContext skips its mysql and mysqldump shell-outs — which reduces setup to composer behat. And wp-cli/wp-cli alone doesn't provide wp post or wp user for fixtures; those live in wp-cli/entity-command.

7. Wiring it into CI

A suite that only runs when someone remembers to run it isn’t a safety net. The full workflow is in the repo; three things in it aren’t obvious from reading it.

The PHP matrix has a floor. wp-cli/wp-cli-tests pulls in behat/behat, which requires PHP 8.2+. Include 8.1 and composer install fails outright — a dependency resolution failure, not a test failure — and GitHub's default fail-fast cancels the remaining legs before they report.

Database credentials belong at the job level, not on the step that creates the database. Scoped to that step, creation succeeds and the test steps then fail with “Error establishing a database connection”, which reads like a credentials problem and isn’t.

Verify the matrix actually varies something. A wp: ['6.7', '6.4'] matrix here tested identical WordPress on both legs: WP_VERSION reaches only Behat, while PHPUnit takes its copy from roots/wordpress in composer.json. Pin both sources together and confirm the version that ran.

wp-env is the lighter alternative — WordPress's own tool, running PHPUnit inside Docker with none of this setup, and a --ci=github flag for the logs.

🔗 CI Actions

🔗 Test CI Workflow

Key Takeaways

• A crash isn’t the real risk. Silently wrong data is — a bad row deep in a big batch, with nobody noticing.
• Install wp-cli/wp-cli as a Composer dependency and WP_CLI_Command just loads.
• WP-CLI’s Logger writes to php://stdout, so expectOutputString() and ob_start() never see it. Swap it with set_logger(), and assert against the database, not just the log line. --format=json and --format=csv are the exception — those go through plain echo.
• WP_CLI::$capture_exit is private and runcommand() needs a live Runner, so neither works from a bare PHPUnit test. Pull the exit-prone logic into a method that throws a plain exception, and leave the real exit behavior to Behat.
• Reach for runcommand() only when you don't control the other command's class. Same package, call the method.
• PHPUnit for logic, Behat for the real binary, both in CI.

Related, on a different layer of the same stack: building type-safe Gutenberg blocks at scale.

Part 2 of two. Part 1 covers the architecture — push, pull, and why a hybrid is the only thing that scales. This part is the measurement.

The standard advice about feed systems is easy to find and easy to agree with:

• Push makes reads fast, because timelines are built in advance.
• Push gets expensive when an account has many followers.
• Pull makes writes cheap, because nothing is copied.
• Pull gets expensive when a user follows many accounts.

Every one of those is true. None of them is a number.

That bothered me while writing Part 1, because the whole design hinges on quantities nobody states. How expensive is “expensive”? Where exactly should the line between a normal account and a celebrity sit? Is the read advantage large or is it a rounding error?

So I built both models over the same data and measured them.

GitHub Repo Codebase — https://github.com/i-am-chitti/twitter-feed-benchmark

What’s covered:

• How the benchmark was set up, and what makes the dataset honest
• What fan-out costs per follower, and what that extrapolates to
• Where the celebrity cutoff comes from once you have a real number
• The memory bill, measured rather than estimated
• The read result I did not expect
• What this benchmark cannot prove

What I Built

Two endpoints, /feed/push and /feed/pull, sitting over one shared dataset. The push endpoint reads a pre-computed Redis sorted set and merges in a live query for followed celebrities. The pull endpoint runs the naive query: find who you follow, fetch their recent tweets, sort, return twenty.

Both return the same twenty tweets. That mattered more than I expected, and I’ll come back to it.

The stack

• API: Node.js 24 with Express, in Docker
• Cache: managed Redis
• Database: managed PostgreSQL 15
• Hosting: Render, each component as its own service — a starter web instance, starter Redis, and a basic-256mb Postgres
• Load generator: autocannon, running on my laptop against the deployed endpoints over the public internet

That last point matters and I’ll return to it in the limitations.

The dataset

• 1,000 users, 30,000 tweets, 20,000 follow edges
• Generated from a fixed PRNG seed, so the same input reproduces exactly — it produced identical graph statistics across two independent runs and across both SQLite and PostgreSQL
• Followees drawn from a Zipf distribution, not uniformly

That last choice is the one that makes the experiment mean anything. My first version had every user follow twenty random accounts, which gives every account roughly the same follower count. That erases celebrities from the dataset entirely, so there is nothing left to measure. With a power law, the follower counts came out as: median 7, 90th percentile 34, 99th percentile 281, maximum 960, mean 20.

Celebrity accounts weren’t hardcoded. They were derived from the graph — any account with 200 or more followers, which selected 14 of the 1,000.

How I measured

• Read tests: concurrency swept at 1, 10, 50, 100 and 400 connections, 30 seconds per run, three repeats per level
• Warm-up and ordering: the two models alternated which one ran first between repeats, so warm-cache and warm-query-plan effects cancel out
• Access pattern: userId randomised per request, rather than hammering one hot key
• Reported values: medians across the three repeats, with the spread recorded
• Errors: zero non-2xx responses across every run
• Server-side timings: a Server-Timing header breaks each request into stages, so latency could be attributed rather than guessed at. Medians of five samples, discarding a cold first request.
• Write tests: eleven probe accounts spanning 4 to 960 followers, five publishes each, median reported. Timed server-side inside the request, so my network latency doesn’t pollute it.

One thing I added before trusting any number: a test asserting both endpoints return the identical page. Comparing two architectures only means something if they’re doing equal work. That test caught a bug where a tweet reachable from both halves of the hybrid appeared twice in the page, which would have quietly made the comparison unequal. I ran it before every benchmark.

Result 1: Fan-Out Cost Is Linear

This is where push is supposed to break, so it went first. I published from accounts across the follower range and timed only the copying step.

• 4 followers: 1.05 ms
• 63 followers: 1.50 ms
• 191 followers: 2.79 ms
• 570 followers: 7.07 ms
• 960 followers: 13.64 ms

That fits a straight line: roughly 0.8 ms of fixed cost plus 12 microseconds per follower.

The linearity matters more than the constant, because a linear relationship extrapolates and a mysterious one doesn’t. Twelve microseconds sounds like nothing until you multiply it out:

• 100,000 followers → about 1.2 seconds
• 1 million followers → about 12 seconds
• 80 million followers → about 16 minutes

These are extrapolations from this benchmark, not production figures. They assume the linear relationship holds four orders of magnitude beyond anything I measured, on hardware I didn’t test, with no batching or sharding of the fan-out work. Treat them as an illustration of the shape of the problem rather than a prediction.

Even so, the shape is the point. Sixteen minutes of copying for one tweet is why a large account cannot use push, and it gives the celebrity threshold something better than folklore to stand on.

One constant, two very different values. The same fan-out code cost 4.5 microseconds per follower against a Redis container on my laptop, and 12 microseconds against the managed Redis. Nearly three times the difference, from identical code.

So this coefficient describes your infrastructure, not the algorithm. Anyone reusing the 12 µs figure without measuring their own setup is doing the exact thing this article is arguing against.

Result 2: The Cutoff Becomes Arithmetic

Part 1 left the celebrity threshold as a formula with a hole in it:

cutoff = fan-out time budget ÷ cost per follower

The measurement fills the hole. With a five-second fan-out budget:

5 seconds ÷ 12 microseconds ≈ 417,000 followers

To be precise about what that number is: it’s the cutoff for this hardware, this Redis, this fan-out implementation, and a five-second budget I chose. It is not an industry threshold, and if you ran the same calculation on your own stack you would get a different answer.

The transferable part isn’t 417,000. It’s that the threshold is a division you can perform, using one number you can measure in an afternoon and one number your product team can decide. Most write-ups on this topic pick a threshold and never say where it came from.

Result 3: The Memory Bill

Push costs memory that pull doesn’t, and at scale that becomes money.

I measured actual key sizes with MEMORY USAGE against live feed keys, rather than reasoning about Redis internals:

• a 330-entry timeline: 27,555 bytes → about 83 bytes per entry
• a 360-entry timeline: 28,899 bytes → about 80 bytes per entry
• a 420-entry timeline: 32,827 bytes → about 78 bytes per entry

So roughly 80 bytes per timeline entry. That’s an observed figure on this Redis version and configuration, not a Redis constant — sorted set encoding changes with size, and the numbers will shift with your setup and your Redis version.

Taking 80 bytes and an 800-entry timeline gives about 64 KB per user, so:

100 million users × 64 KB ≈ 6.4 TB

That’s timeline structure alone. Cached tweet bodies measured about 130 bytes each, which means the half that feels expensive is the cheap half.

6.4 TB is also the number that makes “don’t build timelines for inactive users” non-negotiable rather than a nice optimisation. At one in ten users active, the same design needs around 640 GB.

Result 4: The Read Path, Which Did Not Go As Expected

Push exists to make reads cheap. I expected the clearest result of the three here. I got the murkiest one.

Under load, the cached push timeline and the naive SQL pull query finished within about 1% of each other. Below saturation I could not reliably tell them apart. Push did pull ahead once the server was genuinely saturated, sustaining roughly 1.5 times the throughput — that ratio held across two independent runs, at 1.48 and 1.58.

Then the server-side timings said something stranger. Push was doing more work per request:

• Pull: 1.10 ms finding who you follow, plus 1.21 ms fetching their tweets — 2.31 ms, all of it PostgreSQL
• Push: 0.50 ms reading the sorted set, 0.34 ms fetching tweet bodies, 1.76 ms querying celebrities, 0.03 ms merging — 2.63 ms, of which 1.76 ms is PostgreSQL

Push pays two cache round trips and still queries the database for celebrities. Per request it is slower. Yet it sustained 1.5× the traffic.

Both measurements are correct, and reconciling them is the most useful thing I got out of this:

The cache didn’t make requests faster. It made them cheaper. It removed one of the two database round trips a read needs, moving load off the one resource that was running out of room.

Which gives a lesson that generalises well beyond feeds: an unloaded per-request timing tells you about latency, not about capacity. My stage timings predicted pull would win. The concurrency sweep said the opposite. They disagree because they measure different things, and under load it’s the second one that matters.

What This Benchmark Cannot Tell You

This section matters more than the results, because every number above has a boundary.

The dataset is small. 30,000 tweets and twenty follows per user. Pull only truly collapses when a single read merges hundreds of index ranges across millions of rows. I measured the gap opening, not the gap at full width — so treat the 1.5× as a floor, not a verdict.

The load generator sat on my laptop, across the internet. Roughly 500 ms of round-trip time and TLS went into every read sample. It’s identical for both models so it doesn’t invalidate the comparison, but it dilutes the difference and makes the low-concurrency latency figures useless for anything except establishing that the difference was small.

The 1.5× came from a saturated test. At 400 connections the system is well past its knee, and once you’re there average latency is arithmetically determined by throughput — they are one measurement, not two. I’m quoting the throughput ceiling for that reason, and not quoting a latency improvement alongside it as though it were independent evidence.

The database connection pool was left at its default of ten, uncontrolled. Pull makes two round trips per request where push makes one, so that default could plausibly amplify the gap. I intended to test it and didn’t.

Fan-out ran inline, not through a queue. Production systems enqueue and let workers drain it, which changes the write-latency picture completely. The 12 µs per follower is the cost of the work, not the cost of the request.

The fan-out coefficient is hardware-specific, by a factor of nearly three between two environments I actually tried.

No sharding, no replication, no multiple regions, single instance of everything. Production feed systems are batched, sharded and replicated, and each of those changes the arithmetic.

What I’d Take Away

Two of the three results are clean, and both are about writing rather than reading. That surprised me, because the read path is the part everyone argues about.

The reason is structural. Fan-out cost and memory cost both scale with follower count, which means you can characterise them without a production-sized database. Push’s read advantage only appears once your data is genuinely large, which is exactly why it was the one I couldn’t pin down.

So if you’re deciding whether to build this, the write and memory numbers are the ones to decide on. They’re also the ones you can get in an afternoon.

Which leads to the thing I’d tell anyone starting out:

The common failure isn’t skipping this design. It’s building it too early.

At thirty thousand tweets, a pre-built timeline system gains you almost nothing measurable and hands you a cache to keep in sync forever. A well-indexed SQL query stays the right answer for far longer than most articles admit.

If I were starting again, I’d measure my own fan-out cost before writing any of the architecture. It takes an afternoon, and it tells you more than any article can — including this one.

Part 1 covers the architecture this benchmark was testing: Celebrities vs. The Rest of Us — Designing a Twitter-Style Feed.

Code, including the seeding, both endpoints, the benchmark runner and the parity test: github.com/i-am-chitti/twitter-feed-benchmark

Two things happened on Twitter in the last second. Someone with eleven followers posted a photo of their lunch. Someone with sixty million followers posted a single word.

The same servers handled both.

And that’s the part that’s actually hard — not the scale. Plenty of systems serve a hundred million people. What makes a feed awkward is that those hundred million look nothing like each other. Whatever you build for the person with eleven followers is the wrong thing to build for the person with sixty million.

I’d read the standard explanation before — push, pull, hybrid models. What I could never find was a number attached to any of it. Nobody tells what copying one tweet costs, or where the “celebrity” line should sit. So I built it and measured it.

What’s covered:

• What we are building
• The numbers we are designing for
• Two ways to build a feed: push and pull
• The hybrid model, and why celebrities are treated differently
• Two caches, not one
• What production adds
• But is any of this worth it?
• Takeaway

The design is below. The measurements are in Part 2, including one result that undercut the thing I set out to prove. No system design background needed, and the code is on GitHub if you’d like to poke at it.

1. What We Are Building

Only three things happen in this system:

• A user posts a short message. Let’s call it a tweet.
• A user follows other users.
• A user reads a timeline: recent tweets from everyone they follow, newest first, twenty at a time.

That third one is the whole problem, and everything below answers a single question. When you open the app, where does that list of twenty tweets come from?

The Pieces

• Client — your phone or browser, asking for a timeline.
• Load balancer — spreads requests across a pool of identical servers.
• API servers — the application code. They hold no data; they fetch it and assemble a timeline, which is why you scale them by simply adding more.
• Database — the permanent record. It holds the tweets and the follow graph, meaning who follows whom. Reliable but slow, because it reads from disk.
• Cache — Redis. It keeps data in memory, so reads are roughly a hundred times faster than disk, but memory costs far more per gigabyte and a cache can lose what it holds. Ready-made timelines live here.

Five Words You’ll Need

Timeline - The list of tweets shown to one user, newest first.

Follower and followe - If Sita follows Ram, then Sita is the follower and Ram is the followee. Ram’s tweets show up in Sita’s timeline.

Fan-out - Copying one tweet out to many places. Ram posts once, but that tweet has to reach every one of Ram’s followers, and the spreading out is what we call fan-out. The central question in this post is when it happens: at the moment Ram posts, or at the moment Sita reads.

QPS - Queries per second, which is how we size servers.

Sorted set - A Redis structure where every item carries a number called a score, and Redis keeps them ordered by it. If the score is a tweet’s timestamp, the timeline sorts itself for free.

2. The Numbers We Are Designing For

Suppose we’re designing for a hundred million daily active users, each posting about 0.1 times a day and refreshing their feed about ten times a day. These are assumptions for the exercise rather than Twitter’s real traffic — the point is the shape of the result, not the precision of the inputs. There are 86,400 seconds in a day, so:

• Writes: 100M × 0.1 ÷ 86,400 = roughly 115 tweets per second
• Reads: 100M × 10 ÷ 86,400 = roughly 11,500 reads per second

Reads outnumber writes a hundred to one, which is a number to keep in mind.

That write figure is worth a second look. 115 tweets per second is nothing at all — a laptop would handle it without noticing. So why does the simple approach fall over?

Because if you build timelines ahead of time, each of those tweets has to be copied into every follower’s timeline. The figure that matters isn’t how many people post, it’s how many copies that turns into:

timeline writes per second = tweets per second × average followers

At 200 followers on average, those 115 tweets per second become 23,000 timeline writes per second, which is still perfectly manageable. The trouble lives in the tail. When one account with 80 million followers posts a single time, that’s 80 million writes from one tweet. To be clear about what that means: it’s 80 million writes of work, not one request performing 80 million operations. Section 6 covers how that work gets spread across background workers. But the work still has to happen, and somebody has to pay for it.

Cheap on average and brutal at the top end. That gap is the reason for everything in Section 4.

3. Two Ways To Build A Feed

There are two options, and they differ on one decision: do you do the work when the tweet is posted, or when someone reads it?

Option A: Push (fan-out on write)

Ram posts, and the system immediately looks up all of Ram’s followers and writes the tweet ID into each of their timelines. All the work happens here, at the moment of posting, so that reading is left with nothing to do.

The upside is that reads are fast and predictable. It makes no difference whether Sita follows ten people or ten thousand, because fetching her timeline costs the same either way. You’ll often see this called O(1). It isn’t, strictly — the range read is O(log N + M) in the size of the set and the page. But the useful part holds: the cost doesn’t grow with how many people you follow.

The downsides all come from that copying, and there are three of them.

Posting gets expensive - One tweet from an account with 80 million followers means 80 million cache writes. While those are running, every other post is queued behind them, so one celebrity slows the whole platform down.

You pay for memory you may never read - Every user needs a stored timeline whether they open the app or not, and that memory bill arrives monthly.

Corrections are brutal - Delete a tweet, unfollow someone, block an account, and the wrong data is already sitting in millions of timelines. Undoing a copy is far harder than making one.

Option B: Pull (fan-out on read)

Ram posts and the system saves one row, and that’s the end of it. Nothing gets prepared in advance, so the work happens later, when Sita opens the app. At that point the system has to find who Sita follows, fetch recent tweets from each of those accounts, then merge and sort the results.

The upside is that posting is cheap, and it costs the same whether you have four followers or four million. Nothing is copied, so there’s no cache to keep in sync and no stale data to correct. Deletes and unfollows just work.

The downsides are the mirror image.

Every read does real work - If Sita follows 500 people, that’s 500 index lookups to merge and sort, and she’ll do it ten times today along with everyone else. Given the 100:1 ratio from Section 2, this is the expensive side to be slow on.

It gets worse as you grow, in two directions at once - More tweets in the table means each of those lookups scans more, and users following more accounts means more lookups per read. Push’s cost stays flat as your data grows. Pull’s doesn’t.

Neither One Wins

Side by side, they look like this:

• Cost — Push is expensive to write and cheap to read. Pull is the exact reverse.
• Freshness and memory — Push lags slightly while copying, and needs a stored timeline for every user. Pull is always current and stores nothing extra.
• Where it falls apart — Push on accounts with millions of followers. Pull on users who follow a lot of busy accounts.

That last line is the way through, and it’s what the next section is about.

4. The Hybrid Model

So we use both. Which raises the question I got stuck on for a while: which model goes where, and why that way round rather than the other?

The Two Costs Point In Opposite Directions

The answer comes from asking where each cost actually gets paid.

Push costs one write per follower of whoever is posting. An account with 400 followers costs 400 writes per tweet.

Pull costs one lookup per account the reader follows. Sita follows 500 accounts, so a single read costs her 500 lookups.

Those are two different directions through the follow graph, and the two numbers behave nothing alike. Followee counts have a natural ceiling, because a human has to sit there and tap follow, so almost nobody follows more than a few thousand accounts. Follower counts have no ceiling at all — one account can have eighty million.

So Pull’s worst case is bounded and Push’s isn’t. That single asymmetry decides the design: use Pull exactly where Push runs off the end of the scale.

Which Gives You The Split

Normal accounts get Push — They have few followers, so copying is cheap. There are millions of them, so you’d rather pay that cost once when they post than on every read for the rest of time.

Celebrity accounts get Pull — Copying their tweets is hopeless, and pulling them is cheap for a reason that’s easy to miss: Sita may follow 500 accounts, but only a handful of those are celebrities. Her read splits into 495 accounts already sitting in her cache, plus a live query against maybe five.

And it only works this way round. Flip it — push the celebrities, pull everyone else — and you get the worst of both: eighty million writes every time a celebrity posts, and a 495-account query on every single read. The split isn’t a compromise between the two models. It’s the one arrangement of them that works.

Reading now takes two steps instead of one:

1. Read Sita’s ready-made timeline from the cache, which holds tweets from the normal accounts she follows.
2. Work out which of the accounts Sita follows are celebrities.
3. Fetch those celebrities’ recent tweets from the database.
4. Merge the two lists, sort by time, and return the top twenty.

One trap here caught me out. The two halves can overlap, because an account that only just crossed into celebrity status still has older tweets sitting in people’s caches, so the same tweet arrives twice — once from the cache and once from the live query. My first version happily returned duplicates, and a test comparing the two endpoints was what caught it. So the merge needs to dedupe by tweet ID (nine lines, here), or your readers see doubles.

Where Does The Cutoff Come From?

Most articles pick a number at this point. Ten thousand followers, or twenty-five thousand, or just “a lot”. Almost none of them explain where it came from.

You don’t have to guess though, because it’s a division:

cutoff = how long fan-out may take ÷ cost per follower

It starts with a budget. Suppose a tweet has to reach its followers within five seconds. You then measure what a single copy costs on your servers, and divide. Anything below the resulting line gets Push, anything above it gets Pull.

Both halves of that are yours to control. The budget is a product decision, and the cost per follower depends entirely on your hardware — which is why the answer has to be measured rather than looked up. Part 2 measures it on one particular stack and shows how far the number moves between environments.

5. Two Caches, Not One

A tweet has text, an author, and maybe a link or an image attached. If you store all of that inside every follower’s timeline, you end up keeping millions of copies of the same thing, so the cache gets split in two.

Tier one holds the timeline. One sorted set per user, named feed:{user_id}, where the score is the timestamp and the value is nothing more than the tweet ID. Because the score is a timestamp, the list keeps itself in newest-first order.

Tier two holds the content. The tweet itself lives here, stored exactly once, under tweet:{tweet_id}.

Reading a page then means one range query for the IDs, followed by one bulk fetch for the text.

The timeline half is the expensive half, and the arithmetic behind it is simple:

total memory = users × timeline length × bytes per entry

Each of those three is a lever you can pull, and they pull with very different force.

• Bytes per entry is why we store bare numbers and not full text.
• Timeline length is why you’d keep a few hundred entries per user instead of full history. A few hundred is the usual choice — 800 entries is about forty pages, which is a reasonable bet on how far anyone scrolls before giving up or switching to a profile view. Past that depth you fall back to querying the database.
• Users is the biggest lever, and it leads to a trick most large feeds rely on: don’t build timelines for people who aren’t there. You build them for whoever opened the app this month, and leave the rest to be built on demand.

Section 6 covers that on-demand rebuild, and Part 2 puts a number on what it saves.

6. What Production Adds

The design above works on paper. Four things are missing before it works in real life.

Copying Must Happen In The Background

The Push diagram is quietly misleading on one point. It implies Ram’s post request walks through all of his followers before returning, which for a large account isn’t a request at all — it’s a batch job holding a connection open for minutes. Real systems keep the posting path almost empty:

You save the tweet, drop its ID onto a queue, and return success. A pool of workers drains the queue and updates timelines afterwards, so followers see the tweet a moment later — and that moment is what the budget in Section 4 pays for. Celebrity tweets get their own lower-priority queue, so one viral account can’t block everyone else.

Timelines Must Be Rebuildable

A cached timeline is a copy, built from the tweets and follows tables, and any copy needs a way back to its source. So think about what happens when feed:sita isn't there — Redis dropped it under memory pressure, or Sita hasn't logged in for a year, or you just added a cache node. The naive version finds nothing, returns an empty timeline, and reports success. Sita sees a blank screen and nothing looks broken.

So the fix is to rebuild on a miss. If the cache comes back empty, you fall back to Pull, write the result into the cache, then return it. Once that works, skipping inactive users costs you nothing, because they take the rebuild path on their next visit.

Deleting Is Where Push Hurts

Someone deletes a tweet that already reached 80 million timelines, and you aren’t going to run 80 million deletes. So you don’t. You mark it deleted in the database and filter it out when a page is read — the ID stays in the sorted sets and gets dropped once the text comes back missing. Unfollows and blocks work the same way.

So Push buys fast reads by making writes expensive, then hands some of that back as filtering on every read. A trade, not a free win.

Paging Needs A Cursor

LIMIT 20 OFFSET 200 makes the database walk past two hundred rows and throw them away, getting slower the deeper you scroll. The sorted set offers something better: pass the oldest timestamp you've seen and ask for the next twenty, so every page costs the same however far you go. This is why the score is a timestamp and not a counter. Past a few hundred entries the timeline runs out and you fall back to Pull.

7. But Is Any Of This Worth It?

Everything above is the design as it’s usually taught, and I notice it asks you to take a lot on faith. Copying is expensive — how expensive? The cutoff is a division — divided by what? Push makes reads faster — by how much, and starting when?

I didn’t want to pick the cutoff from folklore, so I built both models over the same dataset and measured them. Three things came out of it.

Fan-out cost turned out to be linear, at roughly 12 microseconds per follower on the hardware I used, which is what finally gave the cutoff formula a real number to work with.

The memory bill was bigger than I expected. Around 80 bytes per timeline entry, which at a hundred million users is measured in terabytes — and that’s what turns “skip inactive users” from an optimisation into a requirement.

And the read result went the wrong way. Push was supposed to win comfortably. At the scale I tested, the pre-computed timeline and the naive SQL query landed within about 1% of each other. Push only pulled ahead once the server was saturated, and even then the reason wasn’t what I assumed.

The setup, the numbers, what surprised me, and — importantly — what the benchmark can’t prove are all in Part 2: I Built a Twitter-Style Feed to Find Out What Fan-Out Actually Costs.

What To Take Away

• The lopsided data is the whole story. Every decision here — two strategies, the cutoff, the split cache, skipping dormant users — comes from the fact that follower counts are wildly unequal.
• Write amplification is the real limit, not write volume. 115 tweets a second is nothing, but one celebrity tweet is 80 million writes.
• Push and Pull fail at opposite ends of the follower range, which is exactly why a hybrid works: the case that breaks one is the rare case for the other.
• The cutoff is a division, not folklore — your fan-out budget divided by your measured cost per follower. Both numbers are yours to find out.
• Split the cache, and only build timelines for active users. Timeline structure is the expensive half, and skipping dormant accounts changes the memory bill by an order of magnitude.
• Build it when your own numbers say to. Measuring your own fan-out cost takes an afternoon, and it decides the question better than any article can.

Further Reading

• Redis sorted sets — the structure the whole timeline design rests on, and worth reading properly if you’ve only used them for leaderboards.
• ZRANGE — the actual complexity of a range read, for anyone who wants to check my arithmetic.
• Timelines at Scale by Raffi Krikorian — Twitter’s own engineering talk on how they ran fan-out. The closest thing to a primary source on the real architecture, as opposed to the reconstructions that circulate in interview prep.
• Part 2 of this pair — the measurements, and where they contradict what I expected.

🏗️ Table of Contents

1. Introduction — The problem with scaling Gutenberg blocks
2. Anatomy of a Gutenberg Block — Directory structure & Static vs. Dynamic blocks
3. Comparing Setup Architectures — Standard wp-scripts vs. Enterprise Webpack
4. Step-by-Step Block Implementation: Testimonials Slider
5. The PHP Counterpart & Inline Schema Injection
6. Styling Architecture: Modular SCSS with Editor Context
7. Plugin Bootstrap: Enqueuing Compiled Bundles
8. Key Takeaways

1. Introduction

We had 40+ custom blocks spread across an enterprise WordPress theme. A single attribute rename broke 12 blocks silently — no compile errors, no warnings. The editor loaded fine but the frontend rendered empty divs. The root cause? No type system. Every block attribute was a loose Record<string, any>.

This guide documents the architecture we adopted to prevent this:

• TypeScript for compile-time attribute safety and IDE autocomplete across all @wordpress/* packages.
• A custom Webpack pipeline that compiles dozens of blocks into a single editor bundle instead of producing 40+ separate scripts.
• A modular SCSS system with context-aware compilation that shares styles between the frontend and the Gutenberg editor iframe.

We’ll walk through building a dynamic Testimonials Slider block end-to-end — from block.json to PHP rendering with inline Schema.org injection.

📦 The complete working plugin is available on GitHub: github.com/i-am-chitti/type-safe-blocks

2. Anatomy of a Gutenberg Block

The Directory Structure

A clean, modular block folder typically contains these core files:

src/editor/blocks/testimonials-slider/
├── block.json      # Metadata, settings, and attribute schemas
├── types.ts        # Shared TypeScript interfaces
├── index.ts        # Registers block settings locally
├── edit.tsx        # React interface in the WordPress editor
├── save.tsx        # Defines the saved post output (or dynamic placeholder)
├── style.scss      # Frontend layout styles
├── editor.scss     # Editor-specific style overrides
└── index.php       # PHP backend renderer (Server-side rendering & bootstrap)

Depending on how these files are configured, WordPress splits blocks into two categories: Static and Dynamic.

Static vs. Dynamic Blocks

Static Blocks

• Output Source: Generated by JavaScript in the editor and saved directly as static HTML inside the post content in the database.
• PHP Counterpart: Needs index.php only to register block metadata on the server. No custom render callback is required.
• Ideal For: Content-heavy structures that rarely change (e.g. Callouts, Cards, Buttons).
• Schema Changes: If the HTML layout is updated in block code, existing posts in the database will trigger Gutenberg validation errors when loaded in the editor.

In save.tsx, a static block generates the final markup. Notice how we type the props with BlockSaveProps<BlockAttributes> for compile-time safety:

export default function Save(props: BlockSaveProps<BlockAttributes>) {
  const { title, content, buttonText, buttonUrl } = props.attributes;
  const blockProps = useBlockProps.save({ className: 'promo-card-output' });
  return (
    <div {...blockProps}>
      <RichText.Content tagName="h2" value={title} />
      <RichText.Content tagName="p" value={content} />
      {buttonText && buttonUrl && (
        <a href={buttonUrl} className="promo-card-button">{buttonText}</a>
      )}
    </div>
  );
}

Dynamic Blocks

• Output Source: Saved as raw attributes only. The final HTML is compiled on the server side by PHP at runtime.
• PHP Counterpart: Requires a custom render_callback function inside index.php to generate the HTML.
• Ideal For: Layouts showing dynamic data (e.g., Latest Posts, Product Sliders, Review feeds).
• Schema Changes: Layout code updates render instantly across all pages without triggering validation errors.

In save.tsx, a dynamic block simply returns null — no markup is stored:

export default function Save() {
  return null;
}

3. Comparing Setup Architectures

Approach A: The @wordpress/scripts Setup (Best for Plugins)

The standard WordPress build system (@wordpress/scripts) is a zero-configuration CLI wrapper.

• How it works: It expects an entrypoint for each block and compiles them separately, producing individual .js, .css, and .asset.php files per block.
• Pros: Quick setup, excellent for isolated blocks in a plugin.
• Cons: Bulky for large codebases.

Why @wordpress/scripts Should Be Avoided in Larger Codebases:

1. Asset Inflation: 40 blocks = 40 JS files + 40 CSS files + 40 .asset.php files. Enqueuing these individually triggers dozens of HTTP requests and disk reads.
2. Code Duplication: Shared code (components, utilities) is compiled into every block’s bundle. No chunk splitting.
3. Rigid Configuration: Extending the internal Webpack/Babel/PostCSS config is fragile. No native support for path aliasing or custom TypeScript configs.
4. Degraded DX: Watching dozens of independent entry points concurrently is CPU-intensive with slow rebuilds.

Approach B: Custom Webpack Bundling (Best for Enterprise Setups)

For enterprise setups, compiling everything into unified bundles is far more efficient.

• How it works: Webpack compiles block scripts into a single dist/editor.js, and compiles styles into two distinct output stylesheets: dist/style.css (for shared frontend and editor layout styling) and dist/editor-styles.css (for editor-only overrides like form containers and toggle states).
• The iFrame Isolation Caveat: Modern WordPress (6.3+) renders the editor canvas inside an isolated <iframe>. Traditional admin enqueues (like wp_enqueue_style hooked to admin_enqueue_scripts) load CSS on the parent page, not inside the iframe. However, enqueue_block_assets does run inside the iframe context, making it the correct hook for plugins. For themes, add_editor_style() is the standard approach.
• Conditional Sass Compilation: By configuring sass-loader with additionalData, we dynamically inject a $is-editor context variable. This lets blocks share a single styling system while compiling editor-specific rules conditionally.

The key parts of our Webpack config:

entry: {
  editor: './src/editor/index.ts',            // Single editor JS bundle
  style: './src/style.scss',                  // Frontend CSS
  'editor-styles': './src/editor/editor.scss', // Editor-only CSS
},
plugins: [
  new RemoveEmptyScriptsPlugin(),  // Prevents empty .js for CSS-only entries
  new MiniCssExtractPlugin({ filename: '[name].css' }),
  new DependencyExtractionWebpackPlugin(),
],

The $is-editor injection happens inside the sass-loader options:

{
  loader: 'sass-loader',
  options: {
    additionalData: (content, loaderContext) => {
      const isEditor = loaderContext.resourcePath.includes('src/editor');
      return `$is-editor: ${isEditor};\n` + content;
    }
  }
}

See the full Webpack configuration → webpack.config.js on GitHub

4. Step-by-Step Block Implementation: Testimonials Slider

Let’s build our type-safe dynamic Testimonials Slider block. Managing testimonials as a structured array of objects (rather than loose child blocks) prevents authors from accidentally breaking the slider’s layout.

Step 4.1: Defining Metadata (block.json)

We define a testimonials attribute of type array, plus an isPreview boolean that controls the editor's preview/edit toggle (defaulting to true so the block loads in preview mode):

{
  "$schema": "https://schemas.wp.org/trunk/block.json",
  "apiVersion": 3,
  "name": "custom/testimonials-slider",
  "title": "Testimonials Slider",
  "category": "theme",
  "icon": "format-quote",
  "attributes": {
    "testimonials": {
      "type": "array",
      "default": []
    },
    "autoPlay": {
      "type": "boolean",
      "default": false
    },
    "isPreview": {
      "type": "boolean",
      "default": true
    }
  },
  "supports": {
    "anchor": true
  }
}

Step 4.2: Defining TypeScript Interfaces (types.ts)

To compile type-safely in WordPress, we must add an index signature ([key: string]: any;) to the block attributes type. This satisfies Gutenberg's internal Record<string, unknown> constraint:

export interface TestimonialItem {
  name: string;
  role: string;
  quote: string;
  rating: number;
}

export type BlockAttributes = {
  testimonials: TestimonialItem[];
  autoPlay: boolean;
  isPreview: boolean;
  anchor?: string;
  [key: string]: any;
};

Step 4.3: Building the Editor UI (edit.tsx)

The editor implements a Preview/Edit toggle pattern. By default (isPreview: true), the block renders a live preview matching the frontend. Toggling the sidebar switch reveals a spacious data-entry form.

The sidebar configuration — note how both the preview toggle and autoplay setting live in InspectorControls:

<InspectorControls>
  <PanelBody title="Testimonials Configuration">
    <ToggleControl
      label="Preview Mode"
      help="Toggle between editing and viewing the live frontend design."
      checked={isPreview}
      onChange={(val) => setAttributes({ isPreview: val })}
    />
    <ToggleControl
      label="AutoPlay Slides"
      checked={autoPlay}
      onChange={(val) => setAttributes({ autoPlay: val })}
    />
  </PanelBody>
</InspectorControls>

The preview branch renders with the same CSS classes as the PHP frontend output, so the editor is a true WYSIWYG representation:

{isPreview ? (
  <div className="testimonials-slider-container">
    <div className="testimonials-track">
      {testimonials.map((item, index) => (
        <div key={index} className="testimonial-card">
          <blockquote className="testimonial-quote">
            "{item.quote || 'Quote text...'}"
          </blockquote>
          <div className="testimonial-meta">
            <strong className="testimonial-name">{item.name || 'Author Name'}</strong>
            <span className="testimonial-role"> - {item.role || 'Position'}</span>
          </div>
          <div className="testimonial-rating">
            {'★'.repeat(item.rating || 5)}
          </div>
        </div>
      ))}
    </div>
  </div>
) : (
  /* Edit form with TextControl, TextareaControl, RangeControl per item */
)}

The immutable update pattern for array items uses keyof TestimonialItem to prevent typos at compile-time:

const updateTestimonial = (index: number, key: keyof TestimonialItem, value: string | number) => {
  const updated = [...testimonials];
  updated[index] = { ...updated[index], [key]: value };
  setAttributes({ testimonials: updated });
};

See the full edit.tsx with both preview and edit branches → edit.tsx on GitHub

Step 4.4: Defining the Save Placeholder (save.tsx)

Because this block’s markup is rendered dynamically by PHP, we return null:

export default function Save() {
  return null;
}

Step 4.5: Registering the Block (index.ts)

To make registration type-safe, explicitly parameterize BlockConfiguration<BlockAttributes> and registerBlockType<BlockAttributes>:

import { BlockConfiguration, registerBlockType } from '@wordpress/blocks';
import edit from './edit';
import save from './save';
import metadata from './block.json';
import { BlockAttributes } from './types';

export const { name }: { name: string } = metadata;

export const settings: BlockConfiguration<BlockAttributes> = {
  ...metadata,
  edit,
  save,
};

export const init = (): void => {
  registerBlockType<BlockAttributes>(name, settings);
};

5. The PHP Counterpart & Inline Schema Injection

The PHP file bootstraps the block and handles server-side rendering. The key design decisions:

1. register_block_type_from_metadata reads block.json directly — no manual attribute re-declaration.
2. Schema.org JSON-LD is injected inline inside the block’s HTML, not via wp_head. Since the render_callback fires during the_content (after wp_head has already executed), hooking into wp_head from inside render() would silently fail.

The bootstrap and schema compilation:

namespace CustomTheme\Blocks\TestimonialsSlider;

function bootstrap(): void {
  register_block_type_from_metadata(
    __DIR__,
    [ 'render_callback' => __NAMESPACE__ . '\\render' ]
  );
}
add_action( 'init', __NAMESPACE__ . '\\bootstrap' );

Inside the render callback, we compile the JSON-LD schema from testimonial data and inject it inline:

$schema_html = '';
if ( ! empty( $schema_reviews ) ) {
  $schema = [
    '@context' => 'https://schema.org',
    '@type'    => 'Product',
    'name'     => get_the_title() . ' Testimonials',
    'review'   => $schema_reviews,
  ];
  $schema_html = '<script type="application/ld+json">' . wp_json_encode( $schema ) . '</script>';
}

The HTML output uses the same CSS classes as the editor preview, keeping frontend and editor visually in sync:

<div class="testimonials-slider-container">
  <?php echo $schema_html; ?>
  <div class="testimonials-track">
    <?php foreach ( $testimonials as $item ) : ?>
      <div class="testimonial-card">
        <blockquote class="testimonial-quote">"<?php echo esc_html( $item['quote'] ); ?>"</blockquote>
        <div class="testimonial-meta">
          <strong class="testimonial-name"><?php echo esc_html( $item['name'] ); ?></strong>
          <span class="testimonial-role"> - <?php echo esc_html( $item['role'] ); ?></span>
        </div>
        <div class="testimonial-rating"><?php echo esc_html( str_repeat( '★', $item['rating'] ) ); ?></div>
      </div>
    <?php endforeach; ?>
  </div>
</div>

See the full index.php → index.php on GitHub

6. Styling Architecture: Modular SCSS with Editor Context

Each block has two SCSS files:

• style.scss — Layout styles shared between the frontend and the editor canvas.
• editor.scss — Editor-only overrides (edit form borders, mode badges, etc.).

These are imported into two global entrypoints (src/style.scss and src/editor/editor.scss) that Webpack compiles into dist/style.css and dist/editor-styles.css.

The $is-editor variable (injected by Webpack's sass-loader based on the file path) enables conditional compilation:

// style.scss — shared layout
.testimonials-slider-container {
  width: 100%;
  .testimonials-track {
    display: flex;
    flex-direction: column;
    gap: 15px;
  }
  .testimonial-card {
    width: 100%;
    border: 1px solid #ddd;
    padding: 15px;
    border-radius: 5px;
    background: #fff;
  }
  .testimonial-rating {
    color: #ffb400;
    margin-top: 5px;
  }
}

// editor.scss — editor-specific overrides
.testimonials-slider-editor {
  .testimonials-edit-form .form-item {
    border: 1px solid #e0e0e0;
    padding: 20px;
    margin-bottom: 15px;
    border-radius: 6px;
    background: #fafafa;
  }
  .form-item-header {
    display: flex;
    justify-content: space-between;
    align-items: center;
    margin-bottom: 15px;
  }
}

See the full SCSS files → style.scss | editor.scss

7. Plugin Bootstrap: Enqueuing Compiled Bundles

The main plugin file (type-safe-blocks.php) ties everything together. It loads each block's PHP bootstrap, enqueues the compiled CSS bundles, and registers the editor script.

namespace TypeSafeBlocks;

// Load PHP counterparts for each block
require_once __DIR__ . '/src/editor/blocks/testimonials-slider/index.php';
require_once __DIR__ . '/src/editor/blocks/promo-card/index.php';

Frontend and editor stylesheets are enqueued via enqueue_block_assets — the only hook that runs inside both the frontend and the Gutenberg editor iframe:

function enqueue_global_block_assets() {
  wp_enqueue_style(
    'type-safe-blocks-frontend',
    plugins_url(https://medium.com/'dist/style.css',%20__FILE__),
    [],
    filemtime( __DIR__ . '/dist/style.css' )
  );

  // Editor-only overrides — only loaded in the admin iframe
  if ( is_admin() ) {
    wp_enqueue_style(
      'type-safe-blocks-editor-overrides',
      plugins_url(https://medium.com/'dist/editor-styles.css',%20__FILE__),
      [],
      filemtime( __DIR__ . '/dist/editor-styles.css' )
    );
  }
}
add_action( 'enqueue_block_assets', __NAMESPACE__ . '\\enqueue_global_block_assets' );

The unified editor script is loaded only in the block editor, with dependencies auto-extracted from the .asset.php manifest:

function enqueue_editor_assets() {
  $asset_file = require __DIR__ . '/dist/editor.asset.php';
  wp_enqueue_script(
    'type-safe-blocks-editor',
    plugins_url(https://medium.com/'dist/editor.js',%20__FILE__),
    $asset_file['dependencies'] ?? [],
    $asset_file['version'] ?? '1.0.0',
    true
  );
}
add_action( 'enqueue_block_editor_assets', __NAMESPACE__ . '\\enqueue_editor_assets' );

See the full plugin bootstrap → type-safe-blocks.php on GitHub

8. Key Takeaways

• Choose the right builder. Use @wordpress/scripts for standalone plugins with a few blocks. Write a custom Webpack config when you need unified bundles, SCSS compilation, and fine-grained TypeScript control.
• Type your block attributes end-to-end. Define interfaces in types.ts, parameterize BlockEditProps<T>, BlockSaveProps<T>, and registerBlockType<T>. A single [key: string]: any index signature satisfies Gutenberg's internal type constraints.
• Static blocks save HTML, dynamic blocks save nothing. Static blocks commit markup to the database via save.tsx. Dynamic blocks return null and rely on PHP's render_callback. Choose dynamic when your layout depends on runtime data or when you need to inject structured schema.
• The editor is an iframe. Styles enqueued via admin_enqueue_scripts load on the admin wrapper, not inside the editor canvas. Use enqueue_block_assets for plugins or add_editor_style() for themes.
• Inject Schema.org inline, not via wp_head. The block's render_callback fires during the_content — after wp_head has already executed. Inline <script type="application/ld+json"> is fully supported by Google's crawler and avoids the silent lifecycle mismatch.
• Share styles, split concerns. Use a shared style.scss for layout and a separate editor.scss for editor-only overrides. Webpack's sass-loader can inject context variables ($is-editor) dynamically based on file paths, enabling conditional compilation from a single codebase.

📦 Full Source Code

The complete working plugin — with both blocks (Promo Card + Testimonials Slider), the custom Webpack pipeline, modular SCSS architecture, and inline Schema.org injection — is available on GitHub:

→ github.com/i-am-chitti/type-safe-blocks

Clone it, run npm install && npm run build, and activate the plugin in any WordPress 6.3+ installation.

Architectural patterns in software are high-level strategies that define the structure and organization of a system. They serve as blueprints for designing software systems that are scalable, maintainable, and robust. Not all patterns have everything equally. Here are some of the most common architectural patterns:

Layered Architecture

Also called n-tier architecture.

Organizes software into layers with distinct responsibilities (e.g., presentation, business logic, data access). Each layer has a job, no mixing and it’s clean.

Use-cases: simple CRUD app

✅ Keeps your code as organized as your kitchen drawers after a weekend cleaning spree.

🔴 Eventually, every bug gets stuck in the layer you swore you weren’t responsible for.

Client Server Architecture

Splits the system into two parts: client (frontend) and server (backend). It’s the best friend of a web developer.

Use-cases: web applications, networked systems

✅ A clean breakup: one does the thinking, the other just looks pretty.

🔴 If the server goes down, the client is basically a confused toddler staring at a blank wall

Monolithic Architecture

Classic all-rounder software pattern — everything together. The UI, business logic, database calls, caching, etc. All components are interconnected and interdependent.

Use-cases: Simple applications

✅ One repo to rule them all

🔴 Deploying it in enterprise is like trying to move a mountain… with a toothpick.

Microservice Architecture

Divides an application into small, tiny, loosely coupled services that communicate over APIs. Each service does its best job. It’s highly scalable and teams can independently work.

Use-cases: Large scale app needing high availability, independent deployments.

✅ Divide and conquer — every team gets their own mess to clean!

🔴 I love microservices, they’re like a thousand tiny bugs instead of one big one.

Event Driven Architecture

Systems use events to trigger and communicate between decoupled components. We have producers (who fires event), consumers (who react) and an event broker (central intermediary).

Use-cases: Real-time apps, asynchronous processing

✅ Like a surprise birthday party — everything responds to something unexpected!

🔴 Eventually, no one knows who’s throwing the party or why it’s on fire.

Service Oriented Architecture (SOA)

Emphasizes building systems from independent, reusable services. Communication between services is facilitated by ESB (enterprise service bus). ESB is a centralized service that routes service requests to the appropriate service. It also transforms the request into a format that is acceptable for the service’s underlying platform and programing language.

Use-cases: Integration between legacy systems

✅ Enterprise therapy: teaching old systems to talk to each other again.

🔴 Spend more time configuring the message bus than coding the actual app.

Model-View-Controller (MVC)

Separates application into three components: Model (data), View (UI), and Controller (logic).

Use-cases: Web and desktop applications.

✅ Separates concerns like a good therapist.

🔴 Eventually, you’ll need a controller just to manage your other controllers.

Peer-to-Peer (P2P) Architecture

All nodes have equal responsibility and can act as both clients and servers. This design allows the direct sharing of resources, be it bandwidth, storage, or processing power, creating cooperative networking and efficient resource discovery without relying on a central server.

Use-cases: File sharing, blockchain, decentralized applications.

✅ Everyone’s equal — communism for computers!

🔴 Unless one peer forgets to share… then it’s just chaos.

A caching proxy server is a type of intermediary server that forwards client requests to the actual server while storing (or caching) the responses. When the same request is made again, the cached response is returned instead of forwarding the request to the server, significantly reducing latency and improving performance. This concept, often referred to as a ‘web accelerator,’ helps speed up website access, reduce server load, and enhance user experience.

In this blog, we’ll build a CLI tool to implement a caching proxy server step by step inspired by roadmap.sh project. This tool will act as a gateway between clients and servers, showcasing how caching mechanisms can optimize web applications.

GitHub Repo — https://github.com/i-am-chitti/caching-proxy-server

Table of Contents

• Requirements
• Basic Setup
• Cache Preparation
• File Lookup Challenge
• Serve Cache
• Cache Purge

Requirements

1. First request should hit the origin server. Receive the response and cache locally. Return the response with X-CACHE:MISS header.
2. Subsequent request should be retrieved from the local cache and served with header X-CACHE:HIT header.

We’ll develop this project in Node.js. Read how to build a CLI app with Node.js here.

Basic Setup

Create a directory and initialize project —

mkdir caching-proxy
cd caching-proxy
npm init
npm install express commander

Create a index.js file. Register the CLI command —

#!/usr/bin/env node

import { program } from "commander";

program.version('1.0.0').description('A caching proxy server');

// Add command to start proxy server.
program.command('start').description('Start the proxy server')
    .requiredOption('-p, --port <port>', 'Port to run the server on')
    .requiredOption('-o, --origin <origin>', 'Origin server to which requests will be forwarded to' ).action((options) => {
        // Validate the port number
        const port = parseInt(options.port);
        if (isNaN(port) || port < 1 || port > 65535) {
            console.error('Invalid port number');
            return;
        }

        // Validate the origin URL
        try {
            new URL(options.origin);
        } catch (error) {
            console.error('Invalid origin URL');
            return;
        }

        // @todo startServer(options);
    });

program.parse(process.argv);

It contains basic validation of port number and URL address. We will soon define the startServer function.

Link the project -

npm link

To run the command, open terminal and do -

caching-proxy start -p 3000 -o https://example.com

Our next step is to hit the origin sever and prepare the cache.

Cache Preparation

Create a server.js file where our startServer code would reside. Using express js, start the server -

import fs from "fs";
import path from "path";
import express from "express";

const app = express();

const startServer = async (options) => {
    // Check if the port is present.
    if ( ! options.port ) {
        console.error('Port is required');
        return;
    }

    // Check if the origin is present.
    if ( ! options.origin ) {
        console.error('Origin is required');
        return;
    }

    app.listen(options.port, () => {
        console.log(`Server is running on port ${options.port}`);
    });

    app.get("*", async (req, res) => {
        // @TODO
    });
}

export default startServer;

In the route, I’ve added the wildcard * meaning all GET requests would be handled.

Now, we’ll extract the path from the incoming request. This path resembles a file path. We’ll leverage this and use the file system to store and retreive the cache response. Meaning, the responses will be written to file on hitting the origin server. When a request comes again, it will be retreived from the same file.

There are two structures of path here —

/example
/example.png

One path has the file extension while other doesn’t. The one which doesn’t have, we’ll store the file as index.{fileExtension}. The file extension part would be deduced from the content-type header of the response.

app.get("*", async (req, res) => {
    const requestPath = req.path;

    // @todo Check in cache.

    // Get file extension is present
    const fileExtension = path.extname(requestPath);

    // File path.
    let filePath = `cache${requestPath}`;

    // Parse the request path with cache directory and append index.
    if ( ! fileExtension ) {
        filePath = path.join(filePath, 'index');
    }

    // Remote URL.
    const urlPath = `${options.origin}${requestPath}`;
    
    const response = await fetch(urlPath);

    // Check if the response is successful
    if (!response.ok) {
        res.status(response.status).send(response.statusText);
        return;
    }

    // Initialize the headers object
    const headers = {};

    // Forward the headers from the origin server to the client
    response.headers.forEach((value, key) => {
        const lowerCaseKey = key.toLowerCase();

        // Skip the content-encoding header and headers containing the word cache
        if (lowerCaseKey !== 'content-encoding' && !lowerCaseKey.includes('x-cache')) {
            headers[key] = value;
            res.setHeader(key, value);
        }
    });

    // Add the x-cache header.
    headers['x-cache'] = 'HIT';
    
    // Deduce the file extension.
    const contentType = response.headers.get('content-type');
    if ( ! fileExtension ) {

        // Get file extension from content type
        const ext = mime.extension(contentType);

        // Add file extension to the file path
        filePath = `${filePath}.${ext}`;
    }

    // Create the directory if it doesn't exist
    fs.mkdirSync(path.dirname(filePath), { recursive: true });

    // Save the file to the cache
    const buffer = Buffer.from(await response.arrayBuffer());
    fs.writeFileSync(filePath, buffer);

    // Send the file to the client
    res.setHeader('x-cache', 'MISS');
    res.sendFile(path.resolve(filePath));
});

Here, we’ve made a remote call to origin server and saved the returned response as a file. If a file extension already exists, for example an image or .doc file, then it will be saved as it is. But, for HTML response or json response where the request path doesn’t contain the file extension, we’re saving it as index.{fileExtension}.

Also, note that we are excluding the content-encoding header as other wise the client browser will try to incorrectly decode ther response. Though origin server will be sending the encoded content (gzip, br etc), but once it’s saved to file, it plain content without any encoding. So, the ehader is stripped off. Moreover, the x-cache header is also stripped off to add our custom header.

To deduce the file extension from the content type, we’re using mime-types library. It can be installed as -

npm install mime-types

Now, this way our response gets saved to the cache directory -

File lookup challenge

Next challenge is to serve the cached file. We’ve the request path and there is an existing file. But we don’t know the file extension always. For example, if request was for static content like an image or doc, it would contain the file extension and we can directly read the file and pass to client in O(1). Else, between /api/v1/posts and /solutions/gt-matrix , we can’t differenciate if the file is a jsonor html or any other type as all files are saved as prefix index .

To solve this, we’ll introduce a database layer — sqlite. You can use Redis as well. But the DB should be very performant. I’m preferring sqlite over Redis as Redis is a server which involves IPC, network roundtrips. But sqlite is file based. So, only one IO is required to fetch the record.

I’ll use the Prisma ORM to move fast -

npm install prisma --save-dev

Go through this offical doc where you can understand how to setup the prisma with sqlite.

Create a db.js file and initialize the prisma client -

import { PrismaClient } from "@prisma/client";

const prisma = new PrismaClient();

export default prisma;

We’ll have a Path model in schema —

model Path {
  id          Int    @id @default(autoincrement())
  requestPath String @unique
  contentType String
  filePath    String @unique
  headers     String
}

• requestPath — the incoming path from request object.
• contentType — content-type from header
• filePath — the file path where the cached response is stored
• headers — all the headers received from the origin server except the content-encoding and x-cache

Let’s store the info after hitting the origin server using upsert -

// Create the directory if it doesn't exist
fs.mkdirSync(path.dirname(filePath), { recursive: true });

// Save the file to the cache
const buffer = Buffer.from(await response.arrayBuffer());
fs.writeFileSync(filePath, buffer);

// Store the file path in the database
try {
    await prisma.path.upsert({
        where: {
            requestPath
        },
        update: {
            filePath,
            contentType,
            headers: headersString
        },
        create: {
            requestPath,
            filePath,
            contentType,
            headers: headersString
        }
    });
} catch (error) {
    res.send('Error');
    return;
}

Serve Cache

Finally, we we’ll implement the logic for cached resposne —

// Check if the file path is present in the database
const pathData = await prisma.path.findUnique({
    where: {
        requestPath
    }
});

if (pathData) {
    const absolutePath = path.resolve(pathData.filePath);

    // Check if this path exists
    if ( fs.existsSync(absolutePath) ) {
        // Send headers
        const headers = JSON.parse(pathData.headers);
        Object.keys(headers).forEach(key => {
            res.setHeader(key, headers[key]);
        });

        // Send the file to the client
        res.sendFile(absolutePath);
        return;
    }
}

Here, if we found that there is a record in DB and a file actually exits, we’ll send the file along with headers.

So, final code becomes — https://github.com/i-am-chitti/caching-proxy-server/blob/main/server.js

Try this in action. Start server with CLI command. Hit a url, inspect the response header, it should be X-CACHE: MISS . Next request should be X-CACHE: HIT .

Cache Purge

There is one more requirement. One should be able to purge the cache — empty the DB and remove the cache directory. Let’s register one more command in index.js —

// Add command to clear the cache
program.command('clear').description('Clear the cache').action(async () => {
    try {
        // Clear the database
        await prisma.path.deleteMany();

        // Remove the cache directory
        fs.rmSync('cache', { recursive: true, force: true });

        console.log('Cache cleared');
    } catch (error) {
        console.error('Error clearing cache');
    }
});

This way, we are deleting all the records from the DB. Any further requests will directly hit the origin server and get cached. Also, the cache directory is emptied.

One common example is the Git CLI, which offers more flexibility and speed than its GUI counterpart, GitHub Desktop. Using the CLI, you can chain commands, apply conditions, and quickly navigate through complex workflows that might otherwise require multiple clicks in a UI.

In this guide, we’ll explore how to create a CLI application using Node.js, leveraging libraries and tools to make a versatile, developer-friendly command-line tool. Let’s dive in!

Table of Content

• Prerequisite
• Kickoff
  - Bootstrap
  - Add commander.js
  - Implement command
• User experience
  - Beautifying CLI
  - Prompting CLI
  - Cool Loaders
  - ASCII art
• Project
• Kick Off
  - Bootstrap
  - Update package.json
  - Add index.js
  - Implement handler
  - Event parser
  - End Result
• Best Practices

Prerequisite

Node.js

Kick Off

Step 1: Bootstrap

Create project folder and init npm -

mkdir github-user-activity
cd github-user-activity
npm init

Step 2: Install commander.js

commander.js is a popular JS library for parsing, handling errors, help text on CLI.

npm install commander

Step 3: Implementing command

Create index.js in root directory. This is where the CLI starts.

#!/usr/bin/env node

import { program } from "commander";

program
  .version("1.0.0")
  .description("My CLI")
  .option("-n, --name <type>", "Add your name")
  .action((options) => {
    console.log(`Hey, ${options.name}!`);
  });

program.parse(process.argv);

Add bin to your package.json — https://docs.npmjs.com/cli/v10/configuring-npm/package-json#bin

"bin": {
    "my-cli": "./index.js"
},
"type": "module"

Link the project — https://docs.npmjs.com/cli/v10/commands/npm-link

npm link

Now run from your terminal

my-cli --name YourName

User Experience

Beautifying CLI

Chalk is perfect for styling CLI’s output pop with color.

npm install chalk

Let’s improve

#!/usr/bin/env node

import { program } from "commander";
import chalk from "chalk";

program
  .version("1.0.0")
  .description("My CLI")
  .option("-n, --name <type>", "Add your name")
  .action((options) => {
    console.log(chalk.blue(`Hey, ${options.name}!`));
    console.log(chalk.green(`Hey, ${options.name}!`));
    console.log(chalk.red(`Hey, ${options.name}!`));
  });

program.parse(process.argv);

Prompting CLI

Let’s add some interactivity with Inquirer.js

npm install inquirer

Asking user -

#!/usr/bin/env node

import { program } from "commander";
import chalk from "chalk";
import inquirer from "inquirer";

program.version("1.0.0").description("My CLI");

program.action(() => {
  inquirer
    .prompt([
      {
        type: "input",
        name: "name",
        message: "What's your name?",
      },
    ])
    .then((answers) => {
      console.log(chalk.green(`Hey there, ${answers.name}!`));
    });
});

program.parse(process.argv);

You can use this library to ask user for various input types — password, checkbox, expand, rawlist, list, confirm

Cool Loaders

Loading spinner animation should be shown for async actions. ora is popular for this.

npm install ora

Add the loader that takes time —

#!/usr/bin/env node

import { program } from "commander";
import chalk from "chalk";
import inquirer from "inquirer";
import ora from "ora";

program.version("1.0.0").description("My CLI");

program.action(() => {
  inquirer
    .prompt([
      {
        type: "input",
        name: "name",
        message: "What's your name?",
      },
    ])
    .then((result) => {
      const spinner = ora(`Doing ${result.choice}...`).start(); // Start the spinner

      setTimeout(() => {
        spinner.succeed(chalk.green("Done!"));
      }, 3000);
    });
});

program.parse(process.argv);

ASCII art

Using Figlet.js, we can add cool arts -

npm install figlet

Adding this -

import figlet from "figlet";

console.log(
  chalk.yellow(figlet.textSync("My CLI", { horizontalLayout: "full" }))
);

Project

True learning comes from putting knowledge into practice. So, let’s jump in and build a CLI app with Node.js. We can make use of a fantastic Roadmap project to both learn and create at the same time. The task is straightforward: develop a CLI that retrieves and displays a GitHub user’s recent activity.

github-user-activity <username>

Kick Off

Step 1: Bootstrap

Create project folder and init npm, install libraries

mkdir github-user-activity
cd github-user-activity
npm init
npm install commander ora chalk figlet

Step 2: Update package.json

"bin": {
    "my-cli": "./index.js"
},
"type": "module"

Step 3: Add index.js

Create index.js file and add the welcome message -

#!/usr/bin/env node

import chalk from 'chalk';
import { program } from 'commander';
import figlet from 'figlet';
import ora from 'ora';
import eventParser from './src/utils/event-parser.js';

console.log(chalk.yellow( figlet.textSync( 'GH Activity')) );

program.version('1.0.0').description('A simple CLI for getting GitHub user activity');

program.argument('<username>').action(displayGitHubActivity);

program.parse(process.argv);

The displayGitHubActivity is the action handler. <> around the argument means that it’s required while, you can use [] to add the optional argument.

Step 4: Implement handler

Add username validation in order to avoid empty username. We’re using the Events API endpoint to fetch the GitHub events. By default, it provides the public events. You can add an optional argument for taking token string in CLI and pass it while making API call, you’ll also get the private events. Maybe you can expand on this!

const displayGitHubActivity = async (username) => {
    username = username.trim();

    if ( ! username ) {
        console.log( chalk.red('Please provide a valid GitHub username.') );
        return;
    }

    const apiURL = `https://api.github.com/users/${username}/events`;

    const spinner = ora('Loading user activity...').start();

    try {
        const response = await fetch(apiURL);
        const data = await response.json();

        spinner.stop();

        if ( ! Array.isArray(data) ) {
            console.log( chalk.red('An error occurred while fetching user activity. Check if the username is valid.') );
            return;
        }

        if ( ! data.length ) {
            console.log( chalk.red(`No activity found for user ${username}.`) );
            return;
        }

        console.log( chalk.green(`Hey, ${username}! Here are your recent activities on GitHub:`) );

        console.log( chalk.white('---') );

        data.forEach((activity) => {
                const { message } = eventParser(activity);

                if ( ! message ) {
                    return;
                }

                console.log( chalk.white(`- ${message}`) );
        });

        console.log( chalk.white('---') );
    } catch (error) {
        spinner.stop();
        console.log( chalk.red('An error occurred while fetching user activity. ') );
    }
};

This has reference to eventParser which returns a human-readable message to be displayed on CLI.

Step 5: Event parser

Create a event-parser.js file under src/utils directory -

const eventParser = (activity) => {
    const { type, actor, repo, created_at } = activity;

    let message = '';

    switch(type) {
        case 'PushEvent':
            if ( activity.payload.size === 0 ) {
                break;
            }
            message = `Pushed ${activity.payload.size} ${activity.payload.size > 1 ? 'commits' : 'commit'} to ${activity.payload.ref.replace('refs/heads/', '')} at https://github.com/${repo.name}`;
            break;
        case 'CreateEvent':
            message = `Created a new ${activity.payload.ref_type} ${activity.payload.ref} at ${repo.name}`;
            break;
        case 'DeleteEvent':
            message = `Deleted ${activity.payload.ref_type} ${activity.payload.ref} at ${repo.name}`;
            break;
        case 'WatchEvent':
            message = `Starred ${repo.name}`;
            break;
        case 'IssuesEvent':
            message = `${activity.payload.action} issue #${activity.payload.issue.number} at ${repo.name}. More at ${activity.payload.issue.html_url}`;
            break;
        case 'IssueCommentEvent':
            message = `Commented on issue #${activity.payload.issue.number} at ${repo.name}. More at ${activity.payload.comment.html_url}`;
            break;
        case 'ForkEvent':
            message = `Forked ${repo.name}`;
            break;
        case 'PullRequestEvent':
            message = `${activity.payload.action} pull request #${activity.payload.number} at ${repo.name}. More at ${activity.payload.pull_request.html_url}`;
            break;
        case 'PullRequestReviewCommentEvent':
            message = `Commented on pull request #${activity.payload.pull_request.number} at ${repo.name}. More at ${activity.payload.comment.html_url}`;
            break;
        case 'ReleaseEvent':
            message = `Released ${activity.payload.release.tag_name} at ${repo.name}`;
            break;
        case 'PublicEvent':
            message = `Open sourced ${repo.name}`;
            break;
        case 'MemberEvent':
            message = `Added ${activity.payload.member.login} as a collaborator to ${repo.name}`;
            break;
        case 'GollumEvent':
            message = `Created or updated a wiki page in ${repo.name}`;
            break;
        case 'CommitCommentEvent':
            message = `Commented on a commit at ${repo.name}`;
            break;
        case 'PullRequestReviewEvent':
            message = `${activity.payload.action} a review on pull request #${activity.payload.pull_request.number} at ${repo.name}. More at ${activity.payload.review.html_url}`;
            break;
        case 'ProjectCardEvent':
            message = `${activity.payload.action} a project card at ${repo.name}`;
            break;
        case 'ProjectColumnEvent':
            message = `${activity.payload.action} a project column at ${repo.name}`;
            break;
        case 'ProjectEvent':
            message = `${activity.payload.action} a project at ${repo.name}`;
            break;
        case 'SecurityAdvisoryEvent':
            message = `${activity.payload.action} a security advisory at ${repo.name}`;
            break;
        case 'DeploymentEvent':
            message = `${activity.payload.action} a deployment at ${repo.name}`;
            break;
        case 'DeploymentStatusEvent':
            message = `${activity.payload.action} a deployment status at ${repo.name}`;
            break;
        case 'OrganizationEvent':
            message = `${activity.payload.action} an organization`;
            break;
        case 'MarketplacePurchaseEvent':
            message = `${activity.payload.action} a marketplace purchase`;
            break;
        case 'LabelEvent':
            message = `${activity.payload.action} a label at ${repo.name}`;
            break;
        case 'ContentReferenceEvent':
            message = `${activity.payload.action} a content reference at ${repo.name}`;
            break;
        case 'CheckRunEvent':
            message = `${activity.payload.action} a check run at ${repo.name}`;
            break;
        case 'CheckSuiteEvent':
            message = `${activity.payload.action} a check suite at ${repo.name}`;
            break;
        case 'StatusEvent':
            message = `${activity.payload.state} the commit status at ${repo.name}`;
            break;
        default:
            message = `Performed a ${type} event at ${repo.name}`;
            break;
    }
    
    return {
        type,
        actor: actor.display_login,
        repo: repo.name,
        created_at,
        message
    };

}

export default eventParser;

It parses various events and constructs the message.

Step 6: End Result

Best Practices

Before diving into the CLI logic, I highly recommend checking out this repository by Liran Tal, which has earned over 3,000 stars and is packed with best practices for building CLI applications.

One key takeaway from Liran’s repo is the idea of providing a stateful experience for users. Instead of requiring them to re-enter the same information each time, you can use a package like conf to save data such as usernames, email addresses, or API tokens. This approach enhances usability and makes for a much smoother, more efficient experience.

Ready to see it all in action? Head over to my GitHub page to explore the complete project, including all example files. Dive in, experiment, and feel free to fork or star the repository if you find it useful!