<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Nishant Garg on Medium]]></title>
        <description><![CDATA[Stories by Nishant Garg on Medium]]></description>
        <link>https://medium.com/@nishant.gargvii?source=rss-d18da405456a------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/proxy/1*TGH72Nnw24QL3iV9IOm4VA.png</url>
            <title>Stories by Nishant Garg on Medium</title>
            <link>https://medium.com/@nishant.gargvii?source=rss-d18da405456a------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Tue, 17 Nov 2026 19:31:55 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@nishant.gargvii/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[FIFA WC Is NOT the Best Soccer Tournament]]></title>
            <link>https://medium.com/@nishant.gargvii/fifa-wc-is-not-the-best-soccer-tournament-119b76908726?source=rss-d18da405456a------2</link>
            <guid isPermaLink="false">https://medium.com/p/119b76908726</guid>
            <dc:creator><![CDATA[Nishant Garg]]></dc:creator>
            <pubDate>Wed, 08 Jul 2026 19:56:39 GMT</pubDate>
            <atom:updated>2026-07-08T19:56:39.740Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*gBL2ReAJ-5b96tGB.jpg" /></figure><p>Ask what a viewer actually wants from soccer, and the answer is simple: good soccer. Sharp tactics, high intensity, elite players tested against elite players. That — the spectacle itself — is the only fair yardstick for calling something the best of the sport. A tournament earns that title by the quality of what happens on the pitch, not by how much of the world happens to be watching. Judged that way, the World Cup is not the top of the game. The Premier League and the Champions League are.</p><h3>Soccer is a synchronized game, and sync takes time</h3><p>Soccer is one of the most coordinated team sports there is. It runs on patterns rehearsed until they’re instinctive: pressing triggers, passing lanes, when the full-back tucks in, who drops to cover, how the line steps up as one. None of that is improvised in the moment — it is the residue of months of repetition, until eleven people move like a single organism.</p><p>That is exactly what a season buys. Premier League players spend roughly ten months a year together — training, drilling, playing, failing, adjusting — and the understanding compounds week over week. A national team gets a few weeks. It asks world-class individuals to manufacture a shared identity in real time, on top of habits they’ve each built at different clubs playing different systems. The talent is there; the synchronization can’t be. And because soccer’s ceiling is set by coordination, not by individual brilliance, the tactical peak of a World Cup is capped before a ball is kicked. What the viewer sees is gifted players approximating a team, rather than a team operating at full resolution.</p><h3>Best-vs-best, sustained</h3><p>For the watcher, the height of the spectacle is the world’s finest playing <em>alongside</em> and <em>against</em> the world’s finest, orchestrated by the sharpest coaching. That is the weekly reality of the EPL and the UCL, where quality is concentrated rather than scattered — where even the opposition is drilled, coached, and dangerous, so there is nowhere on the pitch to hide. The intensity, the tempo, the razor-thin margins that decide elite matches are not occasional there; they are the baseline.</p><p>There is also the matter of rhythm. Quality reveals itself through repetition, not through a single night. A league puts the same players under the same scrutiny again and again, so what you’re watching is proven, not fluky — and it renews itself on a cadence a viewer can actually live inside, rather than arriving once every four years and vanishing.</p><h3>The platform depends on who you are</h3><p>For a player from a country without an elite league, the World Cup is a genuine gift — the one stage where the world finally watches, the one platform that can lift a career. That is real, and worth honoring. But for someone already operating at the summit, the league <em>is</em> the best place to play. They don’t need a tournament to find elite opposition or elite teammates; they meet both every week, in a setting built to get the maximum out of them.</p><h3>Pride and trophies decorate; quality drives</h3><p>The obvious counter is national pride — the idea that the World Cup carries a meaning no league can, because a whole country plays as one. There’s something to it, but less than it seems. Pride of that kind moves a genuine few very deeply; for most people it’s a shirt worn for a month and then folded away. People will say the right things and paint the flag on their faces, but outward display isn’t the same as what actually drives them.</p><p>And what drives them — players and viewers alike — is the same thing at the core: the quality of the play itself. The deepest motivation was never the cup or the flag. It’s the hunger to measure yourself against the strongest and to perform at your own ceiling. A trophy is a tag any competitor wants, and national pride is a banner worth waving, but both are consequences of great soccer, not the substance of it. The substance is the play and the tactics themselves — pride and silverware only decorate the spectacle, they don’t create it. And on that, the thing the viewer truly comes for, the EPL is far ahead, week after week.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=119b76908726" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Quality Time With My Toodler]]></title>
            <link>https://medium.com/@nishant.gargvii/quality-time-with-my-toodler-0e150afcd465?source=rss-d18da405456a------2</link>
            <guid isPermaLink="false">https://medium.com/p/0e150afcd465</guid>
            <category><![CDATA[quality]]></category>
            <category><![CDATA[kid-expenses]]></category>
            <category><![CDATA[toodlers]]></category>
            <category><![CDATA[parenting]]></category>
            <category><![CDATA[time-management]]></category>
            <dc:creator><![CDATA[Nishant Garg]]></dc:creator>
            <pubDate>Wed, 08 Jul 2026 15:10:31 GMT</pubDate>
            <atom:updated>2026-07-08T15:10:31.567Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Li32cSVi8QuS2_2t" /></figure><p>I read the post “No, I don’t spend ‘quality time’ with my family.” (<a href="https://www.linkedin.com/pulse/i-dont-spend-quality-time-my-family-praveen-gupta/">https://www.linkedin.com/pulse/i-dont-spend-quality-time-my-family-praveen-gupta/</a>) The argument: quality time is nonsense — a phrase invented to buy a clean conscience for parents who don’t show up. There’s no quality time, only time, and the magic happens on its own while you fix a sink or dry the dishes.</p><p>Reality: “there is only time” is the real nonsense — and it was invented by people who don’t know how to tell whether an hour was quality or not. As a working parent of a 15-month-old, I disagree with the core of it.</p><p>My experience</p><p>I get roughly one focused hour a day with my son on weekdays, and about two on weekends. That’s it. When time is that scarce, “just be around and magic will happen” is a luxury belief — it assumes an abundance of hours I simply don’t have. So I decide, in advance, how that hour is spent. That isn’t me easing a guilty conscience — it’s me taking the hour seriously.</p><p>Because <em>how</em> I spend it changes who my son becomes. At 15 months he’s building strength, balance, and coordination at a pace he’ll never hit again. A phone-free, activity-packed hour isn’t “forced happiness” — it’s development, and I’ve watched him grow sharper, stronger, and more confident week over week because the hour was built for it.</p><p>There’s a second thing he’s learning that no scheduled outing could teach: how his father works. A parent who works hard isn’t stealing time from their kid — they’re modeling work ethic and the value of time. Toddlers are imitation machines. When my son sees me focus, finish, and <em>then</em> turn to play, he absorbs the earliest form of discipline: first this, then that. That’s an argument <em>for</em> being intentional with our time together, not against it.</p><p>How I spend quality time</p><p>I keep it simple, physical, and on repeat:</p><ul><li>Bricks and blocks — build a tower, let him knock it down. Endless focus and joy.</li><li>Water and swim — splashing, kicking, blowing bubbles in the pool or the bath.</li><li>Races and chase — crawling and walking races across the floor, climbing, “I’m gonna get you!”</li><li>Ball games — rolling and kicking back and forth, which quietly teaches turn-taking.</li><li>Real work — a sock into the laundry, a cloth to “wipe” the table, stirring a bowl in the kitchen. He feels useful, and responsibility takes root early.</li></ul><p>But here’s the part most people miss: I play with him so he learns to play <em>without</em> me. The goal isn’t to be his entertainment — it’s to make him self-sufficient for the hours I’m not around. So I seed games he can carry on solo: hit the ball against the wall and catch it as it comes back, put every item back where it belongs as a game, join Lego bricks and break them apart. I start it with him, then step back and let him run it alone. A child who can happily occupy himself is building focus, patience, and independence — not falling apart the moment a parent leaves the room.</p><p>I also use that time to teach him what’s dangerous and what to do about it — the hot stove, the sharp edge, the socket, the stairs, named as “no” and “hot” and “careful.” If something ever goes wrong — something catches fire, breaks, or spills — he should come and get us, loudly, right away. It sounds heavy for a 15-month-old, but this is exactly the age it starts: a toddler who’s been shown the danger and the response can raise the alarm in the moment I’m <em>not</em> watching. That’s not fear — that’s safety, and it only gets built during time spent together.</p><p>And all of it runs on a strict schedule — blocks alone, blocks with mom, blocks with dad, each at the same hour every day. He’s learning something most people don’t grasp until adulthood: that a day has a shape, that time is divided on purpose. Teaching the value of scheduling from childhood is a gift. He won’t remember these exact days, but the rhythm will be wired into how he thinks about his time for life.</p><p>And none of this means I run the hour like a drill. I set the window — phone away, activity ready — then let him lead what we actually do inside it. He drives the hour; my job is to make sure the hour is worth driving. For a toddler, a predictable structure is exactly what makes him feel safe enough to be spontaneous.</p><p>When time is scarce, quality isn’t a marketing word. It’s the whole game.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=0e150afcd465" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[FAANG Staff Level: Ad Click Aggregator — System Design]]></title>
            <link>https://medium.com/@nishant.gargvii/faang-staff-level-ad-click-aggregator-system-design-c665af08b62e?source=rss-d18da405456a------2</link>
            <guid isPermaLink="false">https://medium.com/p/c665af08b62e</guid>
            <dc:creator><![CDATA[Nishant Garg]]></dc:creator>
            <pubDate>Mon, 06 Jul 2026 00:53:09 GMT</pubDate>
            <atom:updated>2026-07-06T17:20:41.672Z</atom:updated>
            <content:encoded><![CDATA[<blockquote><em>Full round: Requirements → Core Entities → API → High-Level Design (incl. data flow) → 15 Deep Dives. Data-processing problem — lead with the interface + numbered data flow, not CRUD entities. Scale anchor: ≈100M clicks/day, ≈1.2K/s avg, ≈20K/s peak, write-heavy (≈1:1M R/W). Thesis: writes ≫ reads → optimize the ingest path (durable buffer + pre-aggregate); reads are cheap off pre-aggregated tables. The load-bearing tension is low-latency vs money-accurate, resolved by a two-layer / Lambda architecture (fast approximate speed layer + exact batch-reconciliation layer).</em></blockquote><h3>1. Requirements</h3><h4>Functional — P0</h4><ul><li>Capture the click event — record every click (redirect the user to the advertiser).</li><li>Real-time dashboard — near-real-time aggregated metrics, 99% accuracy (approximate OK).</li><li>Billing-accurate counts — the number used for advertiser billing must be 99.99% accurate (exact/reconciled).</li><li>Complex advertiser queries — aggregate by time range, region, ad/campaign, with drill-down and ≥1-min granularity.</li><li>Idempotency — never double-count; a single served impression isn’t counted multiple times within a &lt;5-min dedup window.</li></ul><h4>Functional — P1 (named, not designed)</h4><p>System/ops metric dashboard · fraud/bot detection (behavioral, DDoS, JS-click, device fingerprinting) — <em>a separate upstream ML system; explicitly out of the core round</em> (dedup ≠ fraud detection).</p><h4>Out of scope</h4><p>Weighted clicks (per-source billing weights) · dry-run / shadow mode · premium/priority tiers · ad targeting/serving (a black-box Ad Placement Service exists) · cross-device / conversion tracking.</p><h4>Non-functional (quantified, per-access-pattern — the part that scores)</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/968/1*HqRLgCQrqKEVCXXsn0EX6Q.png" /></figure><h4>The two insights that drive the whole design</h4><ol><li>Write-heavy (≈1:1M). Consequence: <em>optimize the ingest/write path (buffer at 20K/s, pre-aggregate); reads are trivial because they hit pre-aggregated tables.</em> This is the inverse of a read-heavy fan-out problem — here the firehose is the hard part.</li><li>99% (dashboard) vs 99.99% (billing) is not one requirement — it’s two, for two consumers. A single approximate stream can’t produce a money-exact number, and a slow exact batch can’t feed a real-time dashboard. → forces a Lambda architecture: a fast approximate speed layer (stream) for the dashboard + an exact batch-reconciliation layer (source of truth) for billing that periodically corrects the speed layer.</li></ol><h4>Three clarifications worth stating (avoid conflating guarantees)</h4><ul><li>Availability ≠ accuracy. Ingest must be highly available AND no-loss durable (endpoint up + no click dropped); the <em>count</em> must be 99.99% accurate (reconciled). Related, distinct NFRs.</li><li>Idempotency ≠ fraud. In scope: dedup (don’t count the same click twice). Out of scope: deciding a click is <em>fraudulent</em> (bots/behavioral) — a separate system.</li><li>“Per-user” dedup vs the per-impression mechanism (be precise). The requirement is worded “single user not counted twice,” but the <em>mechanism</em> dedups on a per-serve impression_id — so it collapses double-click / refresh / retry of one served impression (the common case) into one count. It does not treat a user served the ad twice who clicks both as one (that&#39;s 2 distinct impressions → 2 clicks, which is correct). If true per-(user,ad) collapsing is wanted, that&#39;s a <em>different, looser</em> key (user:ad within a window) — name which one you mean. The &lt;5-min window = the TTL on the dedup store (bounds seen-ids held: ≈20K/s × 300s ≈ 6M ids).</li></ul><h3>2. Core Entities</h3><blockquote><em>Entities = nouns you persist, not views. The insight is the SoT-vs-derived split and that “Click” lives at two grains (raw event vs per-minute aggregate)</em></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/970/1*dLMKoABllkLOSgs0q6VLtg.png" /></figure><p>The load-bearing distinction: Click = raw, SoT, archived (what the batch layer re-aggregates for exact billing); AdAggregate = derived, rebuildable (the approximate speed-layer number the dashboard reads). This split <em>is</em> why the two-accuracy-tier requirement is satisfiable — the exact number is always recoverable from the raw archive even if the stream drifts.</p><h3>3. API / Interface</h3><blockquote><em>Frame as inputs → outputs. Input = click events from users; output = aggregated metrics for advertisers. The </em>tell<em> is the server-side redirect.</em></blockquote><pre># 1) Click capture — a GET (browser follows a link), record THEN redirect<br>GET /v1/clicks/{adId}?imp=&lt;signed_impression_id&gt;<br>      → 302 Location: &lt;advertiser_url&gt;     # record click server-side FIRST, then redirect<br>      dedup key = the signed impression_id embedded in the URL (NOT a client header)<br># 2) Real-time dashboard - pre-aggregated, fast path (~99%)<br>GET /v1/dashboard?adIds=[…]&amp;from=&amp;to=&amp;region=&amp;granularity=minute|hour|day<br>      Authorization: Bearer &lt;jwt&gt;          # advertiser derived from token; 403 cross-advertiser<br>      → [{ bucketStart, clicks }]<br># 3) Historical / drill-down analysis - structured query (NOT raw SQL)<br>GET /v1/analysis   Authorization: Bearer &lt;jwt&gt;<br>      { from, to, adIds, region, campaign, groupBy[], granularity }  → rows[]     403 on cross-advertiser</pre><p>Signals baked in:</p><ul><li>GET + server-side 302 — the click endpoint IS the redirect hop; recording happens on the reliable server path <em>before</em> the redirect, so a blocked/lost client call can&#39;t drop a click (client-side redirect would). This is the canonical decision for the problem.</li><li>Dedup via a server-minted, signed impression_id embedded in the click URL — <em>not</em> a client Idempotency-Key header (a link-click GET can&#39;t carry one) and <em>not</em> a client userId (untrusted, privacy, kills legit repeat clicks).</li><li>Structured query, not raw SQL — advertisers send fixed dimensions/filters (injection-safe, schema-decoupled, bounded cost).</li><li>granularity param — one endpoint serves live-minute <em>and</em> month-rollup drill-down.</li><li>Advertiser from the JWT on every read (403 on cross-advertiser) — the &quot;no cross-user access&quot; authz boundary; never trust adIds alone.</li></ul><h3>4. High-Level Design</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*r6eBG3UokqiJ8oR2k4fl7A.png" /></figure><p>D1: Server-side redirect — record the click, then 302</p><ul><li>Rejected: Client-side redirect (client hits ad network, gets URL, navigates)</li><li>Why rejected: Click capture depends on a client call that can be blocked/lost/ad-blocked → lost clicks = lost money</li><li>Chosen: GET /clicks/{id} records on the reliable server path first, then 302. Capture and redirect in one hop</li></ul><p>D2: Durable log (Kafka) as the buffer, in front of everything</p><ul><li>Rejected: Click Processor writes straight to the DB / calls Flink directly</li><li>Why rejected: At 20K/s a DB melts; if a consumer is down, in-flight clicks are dropped (no replay); a spike hammers consumers directly</li><li>Chosen: Kafka absorbs the 20K/s burst, is replayable (retention ~7d), and decouples ingest from processing → no-loss + backpressure</li></ul><p>D3: Flink behind Kafka (reads from the log)</p><ul><li>Rejected: Click Processor → Flink directly</li><li>Why rejected: Flink crash with no buffer = the in-flight window is lost, unreplayable → breaks the no-loss NFR on the speed layer</li><li>Chosen: Flink replays from its Kafka offset on restart → zero loss; the log, not the processor, owns the clicks</li></ul><p>D4: Stream processor (Flink) does the aggregation</p><ul><li>Rejected: Feed raw Kafka straight into OLAP, aggregate at query time</li><li>Why rejected: Aggregating 100M raw rows/day per query is slow + contends with ingest (the “store-and-query-one-DB” anti-pattern)</li><li>Chosen: Flink pre-aggregates per-(ad,min,region) → OLAP stores tiny rollups → sub-second reads. (Alt compute — Kafka Streams / ksqlDB / Spark Streaming — is fine; keep a stream processor for event-time + dedup)</li></ul><p>D5: 1-min event-time tumbling window + watermark (~30–60s)</p><ul><li>Rejected: (a) processing-time windows; (b) a much smaller window (e.g. 10ms/100ms) as the stored grain</li><li>Why rejected: (a) a click at 09:59:58 arriving 10:00:03 lands in the wrong minute; (b) a tiny bucket grain = 600–6000× more OLAP rows (100ms→600×, 10ms→6000× per minute) + write-amplification for data no one queries (metric granularity is ≥1 min), and it seals before late clicks arrive</li><li>Chosen: Window = the metric grain (1 min) — bucket by the click’s own timestamp so late clicks land in their true minute; watermark waits for stragglers. Freshness is a separate knob (see note) — don’t shrink the window to get real-time</li></ul><p>D6: OLAP columnar store (ClickHouse/Druid/Pinot) for serving</p><ul><li>Rejected: Row-store / OLTP (Postgres); or a time-series DB</li><li>Why rejected: OLTP row-store is slow for wide range-aggregation over billions of rows</li><li>Chosen: Columnar = built for time-range aggregation → hits the ≤500ms query NFR; pre-rollups (min→hr→day) for big/cold ranges</li></ul><p>D7: Lambda: speed layer + batch reconciliation as SoT</p><ul><li>Rejected: (a) speed layer only (Flink numbers are billing-truth); (b) batch only (nightly)</li><li>Why rejected: (a) stream drift/late events silently lose money — unacceptable for billing; (b) not real-time — no live dashboard</li><li>Chosen: Fast approximate stream (99% dashboard) + exact batch re-aggregate of the raw archive (99.99% billing) that overwrites drift. Two consumers, two accuracy tiers</li></ul><p>D8: Raw archive teed off Kafka = billing source of truth</p><ul><li>Rejected: Reconcile from Flink’s output</li><li>Why rejected: Checking Flink against Flink can’t catch Flink’s own errors — no independent ground truth</li><li>Chosen: Tee raw clicks to an immutable lake; Spark re-aggregates that → an independent check that heals the OLAP</li></ul><p>D9: Idempotency via server-minted signed impression_id (Redis SET NX EX, distributed)</p><ul><li>Rejected: Dedup on a client-supplied userId; or a per-process in-memory set</li><li>Why rejected: Client userId is untrusted + kills legit repeat clicks + privacy; in-memory set misses dups hitting a different processor</li><li>Chosen: Dedup on the signed impression id in a distributed cache → best-effort speed-layer dedup; batch reconciler does authoritative dedup → billing exact even if Redis drops a key</li></ul><p>D10: Structured query API (fixed dims/filters)</p><ul><li>Rejected: Let advertisers send raw SQL</li><li>Why rejected: Injection, runaway/unbounded queries, schema coupled to the API</li><li>Chosen: Fixed dimensions + granularity param → injection-safe, bounded cost, schema-decoupled; advertiser scoped from the JWT (403 cross-advertiser)</li></ul><p>How to read it, in one breath:</p><ul><li>Ingest (left, sync): user click → API GW → Click Processor (dedup via SET NX EX) → the one synchronous write: emit to Kafka → 302 redirect. Kafka (partition by adId) fans out to two consumers.</li><li>Speed layer (99%): Flink (1-min event-time windows + watermark) writes live counts → Redis and per-(ad,min,region) aggregates → OLAP (upsert-SUM).</li><li>Reconcile layer (99.99%): the tee archives raw clicks to the immutable lake (SoT); a periodic Spark job re-aggregates and overwrites/corrects OLAP — the billing-accurate number that heals stream drift.</li><li>Read (right): advertiser → same API GW (auth, rate-limit, 403 cross-advertiser) → Dashboard Svc reads Redis (live, last min) ∪ OLAP (older buckets); Query Svc reads OLAP + cold rollups. Advertisers only ever touch the derived read stores — never Kafka, the lake, or the write path. That isolation is why reads stay ≤100ms regardless of the 20K/s ingest load.</li></ul><p>Every arrow, named (write vs read):</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/841/1*5143djmug5F-gJIhm455RA.png" /></figure><p>Legend / load-bearing points:</p><ul><li>Click Processor → Kafka is the only synchronous ingest write. Everything downstream reads from Kafka.</li><li>Kafka fans out to BOTH the speed layer (Flink) and the raw archive (tee) — <em>either</em> can crash and replay from its offset ⇒ no-loss on both paths.</li><li>Flink writes the stores; services only read them (never Flink → service).</li><li>Two layers: Flink→Redis/OLAP = fast/approximate (99%); Append-log→Reconciliation→OLAP = exact (99.99% billing SoT) that heals stream drift.</li></ul><h4>Design decisions (alternative → why it loses → what we pick + why)</h4><p>D1: Server-side redirect — record the click, then 302</p><ul><li>Rejected: Client-side redirect (client hits ad network, gets URL, navigates)</li><li>Why rejected: Click capture depends on a client call that can be blocked/lost/ad-blocked → lost clicks = lost money</li><li>Chosen: GET /clicks/{id} records on the reliable server path first, then 302. Capture and redirect in one hop</li></ul><p>D2: Durable log (Kafka) as the buffer, in front of everything</p><ul><li>Rejected: Click Processor writes straight to the DB / calls Flink directly</li><li>Why rejected: At 20K/s a DB melts; if a consumer is down, in-flight clicks are dropped (no replay); a spike hammers consumers directly</li><li>Chosen: Kafka absorbs the 20K/s burst, is replayable (retention ~7d), and decouples ingest from processing → no-loss + backpressure</li></ul><p>D3: Flink behind Kafka (reads from the log)</p><ul><li>Rejected: Click Processor → Flink directly</li><li>Why rejected: Flink crash with no buffer = the in-flight window is lost, unreplayable → breaks the no-loss NFR on the speed layer</li><li>Chosen: Flink replays from its Kafka offset on restart → zero loss; the log, not the processor, owns the clicks</li></ul><p>D4: Stream processor (Flink) does the aggregation</p><ul><li>Rejected: Feed raw Kafka straight into OLAP, aggregate at query time</li><li>Why rejected: Aggregating 100M raw rows/day per query is slow + contends with ingest (the “store-and-query-one-DB” anti-pattern)</li><li>Chosen: Flink pre-aggregates per-(ad,min,region) → OLAP stores tiny rollups → sub-second reads. (Alt compute — Kafka Streams / ksqlDB / Spark Streaming — is fine; keep a stream processor for event-time + dedup)</li></ul><p>D5: 1-min event-time tumbling window + watermark (~30–60s)</p><ul><li>Rejected: (a) processing-time windows; (b) a much smaller window (e.g. 10ms/100ms) as the stored grain</li><li>Why rejected: (a) a click at 09:59:58 arriving 10:00:03 lands in the wrong minute; (b) a tiny bucket grain = 600–6000× more OLAP rows (100ms→600×, 10ms→6000× per minute) + write-amplification for data no one queries (metric granularity is ≥1 min), and it seals before late clicks arrive</li><li>Chosen: Window = the metric grain (1 min) — bucket by the click’s own timestamp so late clicks land in their true minute; watermark waits for stragglers. Freshness is a separate knob (see note) — don’t shrink the window to get real-time</li></ul><p>D6: OLAP columnar store (ClickHouse/Druid/Pinot) for serving</p><ul><li>Rejected: Row-store / OLTP (Postgres); or a time-series DB</li><li>Why rejected: OLTP row-store is slow for wide range-aggregation over billions of rows</li><li>Chosen: Columnar = built for time-range aggregation → hits the ≤500ms query NFR; pre-rollups (min→hr→day) for big/cold ranges</li></ul><p>D7: Lambda: speed layer + batch reconciliation as SoT</p><ul><li>Rejected: (a) speed layer only (Flink numbers are billing-truth); (b) batch only (nightly)</li><li>Why rejected: (a) stream drift/late events silently lose money — unacceptable for billing; (b) not real-time — no live dashboard</li><li>Chosen: Fast approximate stream (99% dashboard) + exact batch re-aggregate of the raw archive (99.99% billing) that overwrites drift. Two consumers, two accuracy tiers</li></ul><p>D8: Raw archive teed off Kafka = billing source of truth</p><ul><li>Rejected: Reconcile from Flink’s output</li><li>Why rejected: Checking Flink against Flink can’t catch Flink’s own errors — no independent ground truth</li><li>Chosen: Tee raw clicks to an immutable lake; Spark re-aggregates that → an independent check that heals the OLAP</li></ul><p>D9: Idempotency via server-minted signed impression_id (Redis SET NX EX, distributed)</p><ul><li>Rejected: Dedup on a client-supplied userId; or a per-process in-memory set</li><li>Why rejected: Client userId is untrusted + kills legit repeat clicks + privacy; in-memory set misses dups hitting a different processor</li><li>Chosen: Dedup on the signed impression id in a distributed cache → best-effort speed-layer dedup; batch reconciler does authoritative dedup → billing exact even if Redis drops a key</li></ul><p>D10: Structured query API (fixed dims/filters)</p><ul><li>Rejected: Let advertisers send raw SQL</li><li>Why rejected: Injection, runaway/unbounded queries, schema coupled to the API</li><li>Chosen: Fixed dimensions + granularity param → injection-safe, bounded cost, schema-decoupled; advertiser scoped from the JWT (403 cross-advertiser)</li></ul><blockquote><em>Dedup primitive note: use Redis </em><em>SET imp 1 NX EX 300 — atomic set-if-absent + 5-min TTL in </em>one<em> command; not </em><em>SETNX+</em><em>EXPIRE (two commands → crash-between-them bug). It&#39;s </em>best-effort<em> (a key can be evicted) — fine, because the batch reconciler dedups authoritatively over the raw archive, so billing stays exact.</em></blockquote><blockquote><em>Window size vs freshness (two independent knobs — don’t conflate): Flink windows can be </em>any<em> size (ms → hours); 1 minute here is a deliberate choice = the metric granularity requirement (advertisers query ≥1-min), not a Flink limit. Do NOT shrink the </em>window<em> to get a faster dashboard — a 100ms/10ms </em>bucket grain<em> = 600×/6000× more OLAP rows + write-amplification for data nobody queries, and seals before late clicks arrive. Dashboard freshness comes from a separate knob: the </em>emit frequency<em> — use Flink early-fire triggers to push the running partial of the </em>current<em> 1-min window every ~100ms–1s. So: window = stored grain (1 min); emit = freshness (sub-second). The early-fire partial is approximate (speed layer, 99%); the sealed window (after watermark) is the finalized number; reconciliation is the exact authority. If a requirement ever demanded per-second </em>stored<em> granularity, </em>then<em> you’d use a smaller window — Flink handles it, the cost is OLAP row/write volume.</em></blockquote><h3>Data flow (the pipeline, end-to-end)</h3><ol><li>Ingest: click → API GW → Click Processor dedups (SET NX EX on impression_id, distributed) → if new, emit to Kafka → 302 redirect. Record-before-redirect ⇒ never lose a click.</li><li>Buffer: Kafka, partition by adId (~7d retention) fans out to two independent consumers.</li><li>Speed layer: Flink (1-min event-time tumbling windows + watermark) → upsert-SUM to OLAP + live counts to Redis. (approximate, 99% — powers the dashboard)</li><li>Serve: Dashboard = Redis (live, last min) ∪ OLAP (older); Analysis = OLAP + cold pre-rollups (min→hr→day), coarsest table that answers the range.</li><li>Reconcile: Kafka tee → immutable raw lake (★ SoT) → periodic Spark re-aggregate → overwrites OLAP with the exact number. (99.99% — powers billing)</li></ol><h3>5. Deep-Dive Model Answers</h3><blockquote><em>Each dive: frame the crux → core design → hard case → edges → tradeoff.</em></blockquote><blockquote><em>Index: DD1 late/out-of-order · DD2 hot ad (write-shard) · DD3 peak spike · DD4 abuse: dedup vs fraud · DD5 big-range queries · DD6 exactly-once · DD7 do-you-need-checkpointing · DD8 multi-dim cube · DD9 geo/residency · DD10 consumer lag · DD11 schema evolution · DD12 reconciliation mechanics · DD13 cost · DD14 fast-redirect vs durable · DD15 hot advertiser (read-shard).</em></blockquote><blockquote><em>Recurring reflexes across the dives: </em>inline guarantee &gt; backstop<em> (DD6/DD7) · </em>“do you need X?” = challenge X<em> (DD7/DD13) · </em>rollups are directional<em> (DD8) · </em>split data by sensitivity<em> (DD9) · </em>a buffer has finite depth<em> (DD10) · </em>SoT outlives your code<em> (DD11) · </em>disjoint ownership for two writers<em> (DD12) · </em>durable = survives a crash now<em> (DD14) · </em>ask “hot on which side?”<em> (DD2/DD15).</em></blockquote><h4>DD1. Out-of-order / late clicks — event-time + watermark (late ≠ fraud)</h4><p>Crux. A click happens at 09:59:58 but arrives at 10:00:30 — slow mobile network, spotty wifi, ret/replay. This is the common case, not suspicious. If we bucket by <em>processing time</em> (arrival), it lands in the wrong minute and 09:59’s number is silently short.</p><p>Design — bucket by event-time, decide live-vs-reconciled with a watermark. Each click carries its own click-timestamp; Flink windows on event-time so a late click lands in its true minute regardless of arrival time. A watermark (~30–60s bounded out-of-orderness) decides only <em>where it’s counted</em>: within the grace → still in the live window; past the grace → misses the <em>live dashboard</em> (fine — dashboard is 99%/approximate) but is still written to Kafka + the raw lake, and reconciliation counts it for the exact billing number.</p><p>The key correctness point + tradeoff. Lateness is a <em>timing</em> property, not a <em>trust</em> property — do not reject late clicks or flag them potentialfraud; that would drop/mislabel honest slow-network users and violate no-loss. Fraud (bots, click-spam) is orthogonal and lives in the P1 detection system — never inferred from &quot;arrived late.&quot; Watermark length is a freshness-vs-completeness knob; keep it tight and let the batch layer catch stragglers. → [[flink]].</p><h4>DD2. Hot ad — a viral ad melts one Kafka partition</h4><p>Crux. Partitioning Kafka by adId gives clean per-ad aggregation, but a viral ad sends all its clicks to one partition + one Flink task → hot shard → lag, worst case backup/loss. This is a write-distribution hot key (not a popular-read one), so caching doesn&#39;t help — we must spread the writes.</p><p>Design — dynamic key-salting on the hot ad only. For ads flagged hot (by spend / rolling click-rate), change the partition key from adId to adId_0..N — spreads the firehose across N partitions + N tasks; scale N up/down as the ad heats/cools. Recombine on read — all salted sub-keys are summed back under the base adId (strip-and-SUM on the OLAP write, or sum at query time). Optionally batch events through the pipeline to cut per-event overhead (small hit to dashboard freshness, acceptable).</p><p>Edges + tradeoff. Salt only hot ads — salting all destroys per-ad locality for the 99.9% normal ads for no gain. A detector flips an ad to salted when it heats up and back when it cools (mode-flip). Rate-limiting guards against cascade. Tradeoff: salting buys write-throughput at the cost of a recombine step, paid only where one key would otherwise bottleneck. → [[scaling_writes]], [[sharding]].</p><h4>DD3. Peak hours — surviving the 20K/s spike</h4><p>Crux. ~1.2K/s average, ~20K/s at peak (business hours / time-zone clustering), and higher on a big event. A synchronous DB-write path would fall over; we can’t drop a (money) click.</p><p>Design — the log absorbs the burst, autoscale the stateless tiers. Kafka is the shock absorber: producers write at 20K/s, consumers drain at their own pace, retention covers the lag — so even an <em>unexpected</em> spike is survivable, not just a pre-scaled one. Shard the topic across enough partitions (≥~20 + headroom). Click Processor is stateless → horizontal autoscale; Flink scales tasks with partitions; OLAP managed/auto-scaled. Peaks are largely predictable → pre-scale after load-testing.</p><p>Edges + tradeoff. A <em>global</em> spike is fine if evenly spread; a <em>single viral ad</em> concentrates it → that’s the hot-shard (DD2). Backpressure lives in the log (it never pushes back on the user-facing redirect, which must stay fast). Tradeoff: over-provisioning partitions wastes money, too few caps throughput — size to peak × headroom + autoscale. → [[scaling_writes]].</p><h4>DD4. Abuse / repeat clicks — dedup (P0) vs fraud (P1), name the boundary</h4><p>Crux. “Prevent a user clicking an ad many times” bundles two <em>different</em> problems with different homes.</p><p>Design — the two are separate mechanisms.</p><ul><li>Accidental repeats (double-click, refresh, retry of <em>one</em> served impression) → idempotency, in scope: dedup on the signed impression_id (SET NX EX, distributed, 5-min window) → drop repeats before counting.</li><li>Deliberate click-fraud (a user/bot clicking across <em>many distinct impressions</em>, click farms, JS bots) → NOT caught by dedup (each is a distinct impression-id) → this is the P1 fraud-detection system (behavioral/ML, device fingerprinting), explicitly out of the core round.</li></ul><p>Tradeoff / the point. Dedup guarantees <em>exactly-once per impression</em> cheaply; it does not decide <em>intent</em>. Stating that boundary — “impression-id dedup handles accidental repeats; intentional fraud is the separate P1 system” — is the senior move (don’t half-answer a fraud question with a dedup mechanism).</p><h4>DD5. Low-latency queries over huge ranges — pre-rollup</h4><p>Crux. Live counts are easy (Redis); but “clicks for this campaign over the last year, by week” aggregates over huge ranges — slow even on columnar OLAP at query time.</p><p>Design — precompute the common windows. Live path: Redis (last-minute counts, generated in real time by Flink) → sub-100ms dashboard. Historical path: OLAP holds per-minute aggregates; a cron pre-rolls minute→hour→day into coarser tables. The Query Optimiser picks the coarsest table that answers the range (duplicates data, but avoids scanning billions of raw rows). Hot (≤90d) per-minute; cold (&gt;90d) from rollups / cold columnar. Billing tolerates latency for exactness.</p><p>Tradeoff. Pre-rollup trades storage for query speed (a form of caching) — worth it because query latency is the NFR and storage is cheap. → [[scaling_reads]], [[time_series_db]].</p><h4>DD6. Exactly-once vs at-least-once — where does the pipeline actually count once?</h4><p>Crux. Transport is at-least-once (a producer retry or a Flink replay can re-deliver a click). ⚠️ Kafka is NOT exactly-once by default — its exactly-once is <em>only</em> Kafka→Kafka (idempotent producer + transactions committing offsets &amp; output atomically) and does not extend to external sinks (OLAP/Redis/lake). So “Kafka guarantees it” is wrong for the parts that matter. Exactly-once counting is engineered across the pipeline, not granted by one box.</p><p>Design — trace the click; name the guard at every hop:</p><ol><li>Ingest (accidental client dups): SET NX EX on the signed impression_id (5-min window) drops double-click/refresh/retry <em>before</em> Kafka.</li><li>Stream → OLAP (the crux — replay safety): Flink holds the running per-window total in its windowed state and writes the absolute recomputed bucket total via UPSERT keyed by (ad, minute, region) — not an increment. So OLAP[A,09:59,US] = 4732; a Flink crash + replay of the 09:59 window just re-writes 4732 over 4732 → no effect. This is <em>idempotent by construction</em>: emit the <em>answer</em>, not a <em>change to</em> the answer. (Contrast: += windowClicks double-counts on replay — the bug.)</li><li>Batch (final authority): reconciliation re-aggregates the raw lake, dedups on impression_id, and overwrites OLAP → exact billing even if the stream drifted.</li><li>ZSET / live dashboard: a transient dup is tolerable — it’s 99%/approximate and self-heals on the next window/reconcile. Don’t over-engineer the speed layer.</li></ol><p>Key distinction + reflex. Two <em>different</em> dup sources, two <em>different</em> guards: duplicate source click → impression-id dedup (stops it entering the total); stream replay of a window → idempotent upsert (re-writing the same total is a no-op). The upsert does not dedup a genuine duplicate click, and impression-id dedup does not make replay safe — you need both, plus reconciliation as backstop. Reflex: name the <em>inline</em> guarantee (idempotent upsert), not just the backstop (reconciliation) — “reconciliation fixes it” is fragile/stale between runs; the upsert keeps the number right in real time. And: cite a component’s guarantee at its exact scope — Kafka’s exactly-once is Kafka→Kafka only; touch an external store and you engineer idempotency yourself. → [[kafka]], [[flink]].</p><h4>DD7. Do you even need Flink checkpointing? (the seniority tell)</h4><p>Crux. The question tests whether you reflexively add checkpointing or challenge it. The staff move: question it → quantify what you’d actually lose without it → name the boundary where it <em>does</em> earn its keep.</p><p>Design — for 1-min windows, mostly NO. A Flink crash loses at most ~1 minute of in-flight aggregation — and that minute isn’t gone: raw clicks are still in Kafka (offsets advance only on successful process→write, so unprocessed events remain) and in the append-only lake (SoT). On restart Flink replays that minute and re-aggregates. Crucially, replay is <em>safe</em> because the OLAP write is an idempotent upsert of the recomputed total (DD6) — re-processing the minute overwrites 4732 with 4732, never double-counts. And if a write never landed, reconciliation fills it from the SoT. So correctness comes from replay + idempotent upsert + reconciliation — checkpointing would only save re-computing ≤1 cheap minute, at the cost of latency + state-store overhead on every checkpoint. Redundant here.</p><p>The boundary + reflex. Checkpointing earns its keep for large/expensive-to-rebuild state — a <em>daily/weekly</em> window holds hours-days of in-memory aggregation; losing that and replaying a week of Kafka is impractical (Kafka may not even retain it) → <em>then</em> checkpoint. Reflex: “do you even need X?” is an invitation to challenge X — quantify what you lose without it; if it’s cheap to rebuild (≤1 min, replayable from a log you already keep), drop it. DD7 leans on DD6: the idempotent upsert is <em>why</em> you can skip checkpointing. → [[flink]], [[kafka]].</p><h4>DD8. Multi-dimensional aggregation explosion (region × campaign × device …)</h4><p>Crux — why you can’t pre-aggregate everything. Advertisers filter on ad × minute × region × campaign × device × creative × placement …. Materializing every combination is a combinatorial cube — even at modest cardinality (1K ads × 1440 min × 50 regions × 100 campaigns × 10 devices) the row count is orders of magnitude beyond the base per-minute aggregate (ad×minute). Can&#39;t store the full cube — choose what to materialize.</p><p>The key distinction — two axes that behave differently:</p><ul><li>Time (min → hour → day) is hierarchical + <em>additive</em> → composes cheaply. “2-hour” = sum two 1-hour buckets; “1 day” = 24 hours. Pre-roll a few grains, combine at read.</li><li>Dimensions (region/campaign/device) do NOT compose that way — rollups are <em>directional</em>. You can sum <em>out</em> a dimension you <em>stored</em> ((ad,min,region) → &quot;ignore region&quot; by summing regions), but you cannot recover a dimension you <em>didn&#39;t</em> store ((ad,min,region) + filter by campaign → campaign was collapsed away → not derivable from the pre-agg). Aggregate is a one-way collapse: coarser is free, finer needs the raw data.</li></ul><p>Design — materialize the hot, compute the cold from raw, adapt over time.</p><ol><li>Pre-aggregate the common, low-cardinality dims + time grains — e.g. (ad, minute, region) + rolled hour/day. Covers the vast majority of dashboards in one lookup.</li><li>Time composition at read: odd windows (2h, 7d) = sum the coarsest pre-rolls that tile the range (Query Optimiser picks the grain).</li><li>Cold-tail dimension not in the pre-agg (e.g. campaign) → go to the raw lake / columnar OLAP, NOT an in-memory filter of the pre-agg — the pre-agg already dropped campaign, so it can’t be recovered there; the columnar store retains all dims and is built to scan+group them. Slower, but it’s the rare path.</li><li>Adaptive materialization (the staff move): watch the query-access pattern; when a cold dimension becomes hot, promote it — start maintaining a (ad, minute, campaign) rollup so it moves from the slow raw path to a fast pre-agg. The materialized set is <em>driven by observed queries</em>, not fixed upfront.</li></ol><p>Tradeoff + reflex. Per-dimension storage-vs-query call: materialize hot dims (fast, bounded storage), compute cold combos from raw (slower, no blowup), let query patterns promote dims between the two. Reflex: rollups are directional — always aggregate coarser (sum out a stored dim / add time buckets), never recover a dropped dim without raw data. “Which dims do I store?” = “which breakdowns must be <em>fast</em>?” — the rest live in the raw/columnar tail until the access pattern says otherwise. → [[scaling_reads]], [[time_series_db]].</p><h4>DD9. Region / geo — where is region stamped, and one pipeline or one per region?</h4><p>Crux — the stamp is the <em>user’s origin</em>, decoupled from <em>where it’s processed</em>. A click routes to the nearest DC (overflow to the next if busy), so it can be <em>processed</em> anywhere — but the region stamp = the user’s location, written onto the event at capture. Overflow-to-another-DC is safe <em>because the stamp travels on the event</em>, so aggregation groups by origin regardless of processing DC.</p><p>Design — regional ingest, and split data by sensitivity.</p><ul><li>Regional ingest pipelines (edge PoP → regional Kafka/Flink) for low-latency capture + availability.</li><li>The load-bearing split — raw stays, aggregates travel:</li><li>Raw clicks = PII, residency-bound → stay in-region. EU raw clicks cannot leave EU (GDPR); US in US. This is <em>why</em> multi-region ingest exists — compliance, not just latency.</li><li>Aggregates = bare counts, no PII → roll up globally. “ad X = 4,732 clicks in EU at 09:59” is just a number → ship the aggregate (not the raw) to a global OLAP. An advertiser in NYC queries the global store, which already holds EU’s numbers → fast and compliant, <em>because only counts crossed the border, not raw clicks</em>. (If even aggregates are restricted, keep them regional and scatter-gather at query time.)</li><li>Reconciliation is per-region: each region’s batch re-aggregates its local raw lake (raw never leaves), then the corrected regional aggregates sum into the global store. Don’t attempt one global cross-region transaction — reconcile locally, aggregate globally.</li></ul><p>Tradeoff + reflex. Global aggregation adds replication latency (regional counts land in the global store slightly late) — fine for dashboards. Reflex: in a geo/residency problem, split data by sensitivity — raw/PII is residency-bound and stays local; derived aggregates are PII-free and can travel. The whole “fast global query + strict residency” trick lives in that split; and the region stamp rides the event so processing-DC overflow never corrupts per-region counts. → [[sharding]].</p><h4>DD10. Backpressure &amp; consumer lag — Flink lags Kafka by 10 minutes</h4><p>Crux — what the user sees, and why nothing is lost. The dashboard goes stale (up to the lag) — but no clicks are lost and the counts stay correct, and the <em>reason</em> is the buffer: Flink is just a consumer reading behind the log’s head; Kafka absorbs the backlog while ingest (Click Processor → Kafka) is completely unaffected and the redirect stays fast. Event-time windowing means late-processed clicks still land in their true minute, so when Flink catches up the numbers are right. The log decouples “click recorded” from “click processed” — that’s the entire reason it’s there (D2). Sharp secondary impact: a human tolerates a stale dashboard, but an advertiser’s automated bidding/budget script keyed off “current” data makes wrong decisions — the real blast radius is downstream automation, not the human view.</p><p>The hard limit (the point of the dive). The buffer is safe only while lag &lt; Kafka retention (~7d). If Flink lags <em>beyond</em> retention, un-processed events age out and are deleted → permanent data loss (the replay source is gone). So consumer lag is the SLO to alarm on, and retention must exceed worst-case recovery time — alert <em>well</em> before lag approaches retention. A buffer has finite depth = retention; crossing it is the one way this design actually loses data.</p><p>Recovery + reflex. Diagnose the cause before scaling — it’s not always capacity: (a) under-provisioned → autoscale Flink (tasks scale with partitions); (b) one stuck/hot partition (viral ad, DD2) → key-salt; © a bad server → evict + respawn; (d) slow sink (OLAP write backpressure) → the bottleneck is downstream, scaling Flink won’t help. Reflex: when a buffer “saves” you, state its finite depth and what happens when it fills — for Kafka that boundary is retention, and consumer-lag-vs-retention is the alarm that matters. → [[kafka]], [[scaling_writes]].</p><h4>DD11. Schema evolution — the click event format changes</h4><p>Crux — the governing principle. Adding device_type isn&#39;t a per-store hack; it&#39;s one rule applied everywhere: a schema registry (Avro/Protobuf) enforcing backward/forward compatibility, with the discipline: only ever ADD fields (optional, with defaults) — never rename, remove, or retype. Forward-compatible = old readers skip unknown fields; backward-compatible = new readers default missing ones. Every component&#39;s handling is an <em>instance</em> of this rule.</p><p>Why the rule exists — the lake is the binding constraint. The raw lake is the billing SoT with 7-year retention, and it’s immutable — you can’t rewrite years of archive. Reconciliation in 2031 will re-aggregate clicks written in 2024, so old-format events must stay readable forever. That’s <em>why</em> the rule is add-only: rename/remove/retype silently breaks reading years of data you can’t fix. The immutable multi-year SoT is the hard case — not OLAP.</p><p>Per-store (all instances of the rule):</p><ul><li>Kafka + registry: producer can’t ship a breaking change; new field is optional → old consumers unaffected.</li><li>Flink: reads new field if present, else default — exactly forward-compatibility.</li><li>Cassandra lake: additive column; old rows read the new column as null. Immutable + add-only = always readable.</li><li>OLAP (the <em>easy</em> case — it’s derived/rebuildable): add a nullable column, or dual-write and swap; if needed, re-materialize from the lake — no “downtime,” because a derived store is never the migration bottleneck. (Reframe: the SoT is unforgiving; the derived store you can always rebuild.)</li></ul><p>Reflex. Your source of truth outlives your code — when data is immutable + long-retained, schema evolution is add-only, registry-enforced, because you can never rewrite the archive and the newest code will read the oldest data. Derived stores (OLAP) are rebuildable and easy; the immutable SoT is where compatibility is non-negotiable.</p><h4>DD12. Reconciliation mechanics — how does “overwrite OLAP” stay consistent?</h4><p>Crux — two writers on the same store. The stream (live) and the batch (reconciliation) both write OLAP. Two hazards: (1) a dashboard read hits <em>mid-overwrite</em> (torn read); (2) batch recomputes a minute the stream is <em>still filling</em> (writers clobber each other).</p><p>Design — disjoint ownership by time + atomic versioned swap.</p><ul><li>Sealed windows only (the key): reconciliation touches only closed windows (older than watermark − margin); it never recomputes the actively-filling minute. So the two writers have disjoint ownership by time — the stream owns the open/recent window (provisional), batch owns sealed windows (final) — they never write the same bucket. Overlapping writers on one key is the bug; partitioning <em>what each may write</em> removes the conflict.</li><li>Immutable versioned writes + atomic swap (handles the torn read): don’t mutate in place — write a new version of the bucket, then make it visible atomically (flip a pointer / reads select MAX(version)). A reader therefore sees old-complete or new-complete, never a mix. (Bonus: versions give a billing audit trail — what the number was before correction.)</li><li>Provisional → final lifecycle: mark recent buckets provisional (speed-layer, may still adjust) and reconciled buckets final (billing-exact). The advertiser sees a monotonic settle — live approximate → finalized exact — not flicker.</li></ul><p>Reflex. When two writers touch the same store, give them disjoint ownership — here by time (stream = open window, batch = sealed windows). Combine with atomic versioned swaps so reads are never torn, and a provisional→final flag so consumers know which numbers can still move. → [[flink]].</p><h4>DD13. Cost — is a 24/7 Flink + Kafka + OLAP + Spark stack justified?</h4><p>Crux — the question wants the <em>fork</em>, not just “yes.” Don’t only defend the spend; show when you’d NOT build it, which proves the expensive stack is <em>deliberate</em>, not reflexive.</p><p>Design — justified only by three requirements together; drop any one → cheaper design. This stack is warranted only because (a) real-time dashboard + (b) high volume (20K/s) + © billing-exact all hold. Remove one and it’s over-built:</p><ul><li>No real-time need (daily/hourly reports suffice) → delete the entire speed layer (Flink + Redis); land clicks in the lake and run batch/Spark only — that alone gives billing-exact numbers. ~Half the stack gone.</li><li>Low volume (&lt; ~1K/s) → skip Kafka+Flink entirely; a simple queue + periodic aggregation, or DB-native rollups, is enough.</li><li>No exact-billing requirement (approximate analytics only) → drop reconciliation + the raw lake; the speed layer alone suffices.</li></ul><p>Cost levers (given the stack is justified): autoscale stateless tiers off-peak (Click Processor, Flink) to the avg (~1.2K/s) not peak; managed services (MSK/Kinesis, managed Flink, BigQuery/Snowflake) trade $ for ops/headcount; storage tiering — hot OLAP (≤90d) is pricey, cold → cheap object store (already in the design — it’s a <em>cost</em> lever too); retention sizing — Kafka 7d costs money, size to recovery need; reconciliation cadence — hourly vs daily Spark, less frequent = cheaper if the billing SLA allows.</p><p>Reflex. Justify cost by naming what you’d remove if a requirement vanished. “It’s needed because NFRs” is senior; “it’s needed <em>because all of real-time + high-volume + exact-billing hold — drop any one and here’s the cheaper design</em>” is staff. Confirm those three requirements are real before committing the spend; given they are, the cost buys guarantees you’re contractually on the hook for (billing). → [[scaling_writes]].</p><h4>DD14. Redirect must be fast AND the click must be durable — resolve the tension</h4><p>Crux — the ordering trap. The redirect is on the user’s critical path (must be ~ms), but the click is money (no-loss). The tempting-but-wrong answer: <em>return the 302, then async-push to Kafka.</em> The bug: between the 302 and the async push succeeding, the click lives only in memory — a crash in that window loses a paid click, and you’ve already redirected the user. Async-then-durable has a loss window.</p><p>Design — durable <em>first</em> (locally), then redirect, then forward. Reorder so the click is durable <em>before</em> the ack: local durable write (append-only WAL/disk, ~ms) → 302 redirect → async forward WAL→Kafka with retry. Now: the redirect stays ~ms (local append is fast); the click is durable the instant it hits the local WAL — a crash after the 302 doesn’t lose it (on restart the forwarder drains the WAL); and Kafka slowness/blips cost <em>latency-to-Kafka</em>, not data. Key insight: “durable” = hit a local durable medium, not “reached Kafka.”</p><p>The tradeoff spectrum (name where you sit + why). (a) Strict no-loss: local WAL → 302 → forward — adds disk + forwarder + WAL GC per click box, but zero loss window → billing-critical picks this. (b) Pragmatic: a synchronous Kafka produce with a tight timeout (acks are single-digit ms) that falls back to local WAL only on timeout — blocks the redirect a few ms, simpler. © Loosest: fire-and-forget async — smallest latency, but the crash-window loss exists. For clicks=money, (a) or (b).</p><p>Reflex. “Durable” means survives a crash <em>right now</em>, not eventually. If you return success before writing to <em>any</em> durable medium, you have a loss window. Put a fast local durable step <em>before</em> the ack; forwarding to the real log can be async <em>because the data is already safe</em>. → [[kafka]].</p><h4>DD15. Hot advertiser on the READ side (mirror of DD2)</h4><p>Crux — what breaks. A huge advertiser (millions of ads) opens a dashboard querying <em>all</em> their ads at once → a giant GROUP BY scan over a massive OLAP slice → slow query that hogs cluster resources and can starve other advertisers&#39; queries. This is a read hot key — opposite side of DD2&#39;s write hot key.</p><p>Design — layered, and cache alone is NOT enough.</p><ol><li>Pre-aggregate at the advertiser grain (not just per-ad) — so “all my ads” is a cheap pre-computed lookup, not a scan-and-sum of millions (same “materialize the hot access pattern” as DD8). <em>This is the primary fix.</em></li><li>Cache the big advertiser’s dashboard result (short TTL) — kills repeat reads (200 refreshes → 1 scan). But cache is secondary: the cold/first query and every miss still run the scan, so caching <em>needs</em> the pre-agg underneath it.</li><li>Paginate / limit — never return millions of ad rows.</li><li>Shard OLAP by advertiserId — one advertiser&#39;s data co-located → a heavy query hits one shard, not the whole cluster, so it can&#39;t starve others.</li></ol><p>The mirror (the point of the dive). DD2 = write-hot (viral ad → all writes to one Kafka partition → key-salt to <em>spread</em> the writes). DD15 = read-hot (big advertiser → one giant query → pre-agg + cache + shard-by-advertiser to <em>cheapen/isolate</em> the reads). Same word “hot key,” opposite side, opposite tool — salting doesn’t help a read-hot key; caching doesn’t help a write-hot key. Reflex: when you hear “hot,” first ask “hot on which side?” — write-hot → spread the writes; read-hot → cheapen/spread the reads. The side picks the tool. → [[caching]], [[sharding]], [[scaling_reads]].</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c665af08b62e" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[FAANG Staff Level: Google Calendar — System Design]]></title>
            <link>https://medium.com/@nishant.gargvii/faang-staff-level-google-calendar-system-design-0a6f884811c9?source=rss-d18da405456a------2</link>
            <guid isPermaLink="false">https://medium.com/p/0a6f884811c9</guid>
            <category><![CDATA[google-calendar]]></category>
            <category><![CDATA[software-engineering]]></category>
            <category><![CDATA[software-design]]></category>
            <category><![CDATA[system-design-interview]]></category>
            <dc:creator><![CDATA[Nishant Garg]]></dc:creator>
            <pubDate>Sun, 05 Jul 2026 14:22:52 GMT</pubDate>
            <atom:updated>2026-07-05T18:55:58.788Z</atom:updated>
            <content:encoded><![CDATA[<blockquote><em>Full round: Requirements → Core Entities → API → High-Level Design → Deep Dives. Scale anchor: 10M DAU / 100M total, 10M event-writes/day, ~100:1 read:write. Thesis: reads dominate → pay on write (fan-out, denormalize, precompute) so the hot read is one cheap lookup. SoT stores stay canonical; everything read on the hot path is derived + rebuildable behind a CDC spine. Rooms are the </em>only<em> strong-consistency path; people are advisory.</em></blockquote><h3>1. Requirements</h3><h4>Functional — P0</h4><ul><li>View events filtered by date range (the dominant read).</li><li>Create events with invitees; check invitee + room availability while scheduling.</li><li>Edit events (including recurring-scope edits).</li><li>Recurring events (incl. open-ended / infinite series).</li><li>RSVP + view acceptance status.</li><li>Reminders / alerts (upcoming event; N-min-before).</li><li>Sync across devices.</li><li>Timezone-correct display and recurrence.</li></ul><h4>Functional — P1 (named, not designed)</h4><p>Public/private calendars &amp; privacy · permissions/delegation · tasks (not just events) · metrics dashboard · full-text search.</p><h4>Out of scope</h4><p>Shadow mode, priority tiers, dry-run, A/B testing.</p><h4>Non-functional (quantified, per-access-pattern — the part that scores)</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*l4Pyh1IW6Pew3yoN7QGZuQ.png" /></figure><h4>The one insight that drives the whole design</h4><p>R:W ≈ 100:1 (read-heavy). Consequence stated out loud: <em>optimize the read path; we can afford real work on write (fan-out, denormalize, precompute) to make reads trivial.</em> This is the calendar analog of Twitter’s fan-out-on-write.</p><h3>2. Core Entities — store per access pattern</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xn3bd1OBB9eE0KqLhEM46Q.png" /></figure><p>Store roles (resolve the ambiguity explicitly): point-read by id → Event DB; date-range read → Redis ZSET (score=start_ts); free-text → Elasticsearch. Three query shapes, three stores. SoT = User, Event(+RRULE+overrides), Attendee/RSVP, Room Booking. Everything else is rebuildable from SoT by replaying CDC — the property that makes “eventual everywhere except rooms” safe.</p><h3>3. API</h3><p>REST, /v1, plural nouns. User always derived from the JWT — never a body/query param (spoofing red flag).</p><pre>POST  /v1/events                          # create + invitee list + room + recurrence<br>      headers: Authorization: Bearer &lt;jwt&gt;, Idempotency-Key: &lt;uuid&gt;<br>      body: { title, start, end, tz, invitees[], room_id?, recurrence? (RRULE) }<br>      → 201 {event_id, version}     errors: 400 / 401 / 403 / 409 / 429(+Retry-After) / 503<br>GET   /v1/events?start=&amp;end=&amp;cursor=&amp;limit=    # dominant read; date-range, cursor-paginated<br>GET   /v1/events/{id}                           # point read (hydration)<br>PATCH /v1/events/{id}?scope=this|this_and_following|series<br>      headers: If-Match: &lt;version&gt;              # optimistic concurrency<br>POST  /v1/events/{id}/rsvp    body:{status: yes|no|maybe}   # responder from token<br>GET   /v1/freebusy?users=[u1,u2]&amp;start=&amp;end=    # busy INTERVALS only, never titles<br>GET   /v1/events?sync_token=&lt;opaque&gt;            # delta sync → {changes[], next_sync_token}</pre><p>Signals baked in: Idempotency-Key on create (safe retries) · PATCH ?scope= for recurring-edit semantics · If-Match for concurrent-edit conflicts (→409) · cursor pagination (stable under mutation) · freebusy returns intervals only (privacy) · 429 + Retry-After (response headers, not request).</p><h3>4. High-Level Design</h3><h4>Component diagram</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2MjvNtc6GQbm2dC0XAb9Xg.jpeg" /></figure><p>Legend:</p><ul><li>Two synchronous SoT writes: Event Service → Event DB (returns 201, RYW) and RSVP Service → RSVP log (idempotent, latest-wins). Both then emit to CDC.</li><li>The one strong, synchronous claim: Room Booking DB (Spanner) is written by the Availability Service via a strong CAS at claim time — NOT through CDC. It’s SoT for bookings.</li><li>Everything off the Change Stream is async. Consumers: Search Indexer (ES), Reminder Scheduler, Sync Change Log, Fan-out Worker (→ per-user ZSET), and the Avail. Updater (→ User Availability, from RSVP-yes).</li><li>User Availability is derived + eventual + NOT SoT — rebuilt from the RSVP log. The Availability Service <em>reads</em> it (eventual) for free/busy, and <em>reads+CAS-writes</em> Room Booking (strong) for claims. Only the room claim is strong; reading availability is eventual.</li></ul><h4>Design decisions (alternative → why it loses → what we pick + why)</h4><p>D1: API Gateway at the edge</p><ul><li>Rejected: Clients call services directly</li><li>Why rejected: authN/rate-limit/TLS get duplicated in every service; no central throttle point</li><li>Chosen: One gateway owns cross-cutting concerns; services stay focused</li></ul><p>D2: Split into services (Event / RSVP / Calendar-read)</p><ul><li>Rejected: One monolith</li><li>Why rejected: Read service needs 100× the replicas of RSVP; a monolith forces them to scale together</li><li>Chosen: Split by load profile → scale each independently (read tier fans wide)</li></ul><p>D3: Only the SoT write is synchronous on create (return 201, then async)</p><ul><li>Rejected: Synchronously fan out to all N attendees before returning</li><li>Why rejected: Create latency becomes O(attendees) → a 500-person invite blows the 500ms budget; partial-failure mid-fan-out = corrupt state</li><li>Chosen: One bounded sync write → 201 (RYW for creator); fan-out is async. Latency is O(1) regardless of guest count</li></ul><p>D4: CDC / Kafka log as the async spine</p><ul><li>Rejected: Event Service directly RPCs the downstream workers</li><li>Why rejected: A down worker = lost update (no replay); tight coupling; a write spike hammers workers directly</li><li>Chosen: Durable ordered log → replay on crash, decoupled consumers, backpressure absorbed by the log</li></ul><p>D5: Materialized per-user Calendar View (Redis ZSET, score=start_ts)</p><ul><li>Rejected: Query Event+Attendee and join+filter-by-date on every read</li><li>Why rejected: A multi-table join on the 100:1 hot path is the bottleneck; can’t hit ≤100ms at scale</li><li>Chosen: Precomputed sorted set → date-range read is one ZRANGEBYSCORE hop; pay on write (D3) so read is trivial</li></ul><p>D6: Redis sorted set specifically</p><ul><li>Rejected: Document store keyed by event_id, or SQL with a range index</li><li>Why rejected: Doc store can’t answer “my events this week” without a scan; SQL range-scan on the hot path is heavier + harder to shard by user</li><li>Chosen: ZSET gives native in-memory range-by-score, keyed per user → matches the dominant query exactly</li></ul><p>D7: CDC-fresh + TTL-fallback + rebuild-on-miss for the view</p><ul><li>Rejected: (a) rebuild on a fixed timer; (b) live-read the SoT every time</li><li>Why rejected: Timer → stale by up to the interval (violates ≤500ms freshness); live-read → abandons the 100:1 precompute win, hammers SoT</li><li>Chosen: Hybrid: CDC keeps it fresh; TTL is only a safety net; cold/stale → rebuild that window from SoT. Cache is an optimization, never the authority</li></ul><p>D8: Rooms is the one strong, synchronous CAS (Spanner), NOT via CDC</p><ul><li>Rejected: Make rooms eventual like everything else; or make everything strong</li><li>Why rejected: Eventual rooms → two people both “book” the same slot (double-book, real-world harm); everything-strong → pay strong-consistency latency/cost/availability everywhere for an invariant only rooms have</li><li>Chosen: Confine strong consistency surgically: atomic compare-and-set on (room_id, slot); log gets 409/degrade. Everything else stays cheap+eventual</li></ul><p>D9: People are advisory (eventual free/busy)</p><ul><li>Rejected: Treat a person like a room (strong availability)</li><li>Why rejected: A human being double-booked is legal/normal — no hard invariant; strong here is gold-plating</li><li>Chosen: Eventual; the human resolves conflicts via RSVP. (Boundary: an auto-scheduler removes the human → people become exclusive → then use the D8 strong path — see C1)</li></ul><p>D10: RSVP as an append-log (Cassandra), current status = latest-wins</p><ul><li>Rejected: In-place mutable status column</li><li>Why rejected: Overwriting loses history/audit; harder to make idempotent under retries</li><li>Chosen: Append-only → free audit trail, idempotent, and the count is a derived aggregate off the log (see C2)</li></ul><p>D11: Device sync = server change-log + client cursor</p><ul><li>Rejected: Server tracks per-device sync state</li><li>Why rejected: 10M users × N devices of server-side cursors = write-heavy state to maintain + GC</li><li>Chosen: Server owns a durable per-user change-log; device holds a throwaway cursor; loss → full resync (see B2)</li></ul><p>D12: Recurrence = store the RULE, materialize a rolling window</p><ul><li>Rejected: Materialize every occurrence as a row</li><li>Why rejected: “Forever” = infinite rows; even capped = 10Ks of rows/event, slow create, huge “edit all future”</li><li>Chosen: One RULE row + a bounded rolling window expanded by a background job (see B4). Store the generator, not the output</li></ul><h4>Write path — create</h4><ol><li>POST /v1/events → gateway derives user_id from JWT, checks Idempotency-Key.</li><li>Synchronously write Event DB (SoT) → return 201 (RYW). <em>Only sync step for the event</em> — protects 500ms even for a 500-person invite.</li><li>Room (if requested) → synchronous strong CAS claim on Room Booking DB (Spanner) before 201 — direct, not via CDC (exclusive resource; loser gets 409/degrade).</li><li>Emit to CDC → all async downstream: Fan-out Worker → per-attendee ZSET + Change Log; Reminder Scheduler → time-buckets; Search Indexer → ES.</li></ol><h4>Write path — RSVP</h4><ol><li>POST /events/{id}/rsvp (responder from token) → RSVP Service appends to RSVP log (Cassandra, SoT) — idempotent, latest-wins.</li><li>Emit to CDC → async: the Availability Updater consumer, on status=yes, writes User Availability (now busy for others&#39; free/busy), and the Fan-out Worker confirms the responder&#39;s ZSET view.</li><li>So User Availability is derived + eventual — updated via CDC from the RSVP log, never written synchronously. (This is the ~200–500ms lag window in dive C1; acceptable because a person is advisory, not exclusive.)</li></ol><h4>Read paths (the 100:1 majority)</h4><ul><li>GET /events?start=&amp;end= → one hop ZRANGEBYSCORE cal:{user} start end. Fresh via CDC incremental update; TTL (1h day / 4h night) is a fallback only — past TTL / cache miss → rebuild window from SoT (CDC can drop). Redis holds ±90-day hot window; 7-yr lives in durable stores.</li><li>GET /freebusy → busy intervals only (privacy); draft-scheduling reads the availability store directly (skipping the cache) — note this is still eventual (User Availability is CDC-derived), which is fine: a person is advisory, so a slightly-stale free/busy at most yields a soft conflict the human resolves (see C1). The <em>room</em> claim, by contrast, is strong (B1).</li><li>GET /events?sync_token → server-owned durable Change Log + device-held throwaway cursor; no-token→full, valid→delta, stale→410.</li></ul><h4>NFR scorecard</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Ele1AO3VduBNWOcKMR1Y9w.png" /></figure><p>Whiteboard one-liner: <em>Read-heavy → fan-out-on-write into per-user ZSETs (CDC-fed, TTL-fallback, rebuildable); reminders via sharded time-buckets that timezones flatten; rooms are the lone strong-consistency claim; recurrence = RRULE + override rows expanded on read in wall-clock+tz.</em></p><h3>5. Deep-Dive Model Answers</h3><blockquote><em>Each dive: frame the crux → core design → hard case → edges → tradeoff.</em></blockquote><blockquote><em>Index: A1 large-invite fan-out · A2 CDC outage · A3 concurrent edits · B1 room booking · B2 device sync · B3 reminder herd (9am) · B4 recurrence · B5 reminder invalidation · B6 clock skew · B7 infinite-recurrence reminders · C1 auto-scheduler N-way hold · C2 RSVP count consistency · C3 tz-rule change · D1 storage sizing · D2 exec hot calendar · D3 search over history · E1 full-trace (move a recurring instance).</em></blockquote><h4>A1. The 500-person all-hands, then the CEO invites all 100K employees</h4><p>Crux: the celebrity / hot-key problem (Twitter fan-out analog). A 100K-invite event is a write-amplification bomb: one create → 100K ZSET writes + 100K reminder rows + 100K availability updates. Fix = hybrid fan-out with a threshold, applied to <em>all three</em> fan-out paths.</p><p>Design. Below threshold (~500–1K): fan-out-on-write into each attendee’s ZSET (99.9% of events). Above: skip fan-out, store one event + attendee list, large=true. Pull, concretely: an attendee&#39;s view = ZSET range-scan ∪ lookup of large-events they&#39;re on in [start,end] (via GSI attendee_id → (event_id, start_time)), merge-sorted — materialize once, read by many. The mode-flip is the edge: an event that <em>grows</em> past the threshold flips push→pull and backfill-removes early ZSET entries (or lets them TTL + dedup on read).</p><p>Other two fan-outs + tradeoff. Reminders: 100K firing at one instant = one hot bucket, and tz-smear does NOT help (one event, one absolute time) → shard event_id#shard or treat as a broadcast through the notification pipeline. Availability: does an all-hands even <em>block</em> free/busy? Often no (or only after RSVP-yes) → avoid 100K availability writes. Fan-out runs async off a priority queue so a 100K job can&#39;t starve a 3-person meeting. Tradeoff: push for the common small event (cheap reads), pull for the rare huge one (cheap writes) — pull&#39;s read-merge is paid only for the few large events a user is on.</p><h4>A2. CDC pipeline (Event DB → fan-out) down for 20 minutes — what do users see, how do you recover?</h4><p>Crux. CDC feeds only derived stores; SoT writes don’t ride CDC. So the outage degrades freshness, never durability or the strong path. Answer both clauses.</p><p>What users see (ranked by consistency class). Reads work (ZSET within TTL, else rebuild-from-SoT). SoT writes succeed (creator RYW; invitee views lag). User availability stale → wrong free/busy, tolerable (advisory). Room — the sharp one: stale browse → users <em>attempt</em> conflicts, but the Spanner CAS still rejects the loser (409) → failed attempts / bad UX, NOT double-bookings (the strong claim is CDC-independent). Reminders due to be <em>scheduled</em> in the window don’t land.</p><p>Recovery + principle. Heartbeat detects the stall → replay from last-acked checkpoint (ordered log) → idempotent, version-aware apply (drop-if-older, per-key order) → batch + throttle the backlog (avoid a second outage) → reminders: suppress past-due, schedule future → room reconciliation notice. HA via replicated CDC. Closing principle: everything downstream is rebuildable from SoT → lag, not loss. Reflexes: trace failures through your <em>actual</em> consistency paths; ask “what scheduled side-effects were silently swallowed?”</p><h4>A3. Two organizers edit the same event at the same time</h4><p>Crux. Lost-update / concurrent-write. Collisions are rare → optimistic concurrency, not pessimistic locks.</p><p>Design. Every Event row carries a version; GET returns it, PATCH sends If-Match; the write is an atomic CAS (UPDATE … WHERE version=?). First writer wins (v2→v3); second gets 409 → refetch + reapply. Field-level merge (the UX ceiling): disjoint fields (location vs time) → auto-merge; same field → surface conflict. Only a true same-field collision reaches a human.</p><p>Recurring twist + backstop. scope=this (override row) vs scope=series (rule row) touch different entities → version-check independently; on precedence, most-specific scope wins (instance override beats series edit for that occurrence). Derived stores apply version-stamped idempotent so a stale edit can&#39;t overwrite a newer one end-to-end. Tradeoff: optimistic = no lock latency on the common path; occasional retry softened by auto-merge.</p><h4>B1. Concurrent Room booking</h4><p>Crux: a room is the one exclusive resource — invariant “no two overlapping bookings.” Isolate the strong-consistency claim to the smallest scope.</p><p>Design. Two phases, two consistency levels. Browse/hint = eventual cached view (high-QPS, stale-ok). Claim = CAS transaction on the strongly-consistent (room_id, slot) Spanner row (read overlaps → none → insert → commit; loser 409). Discretize into fixed slots + uniqueness constraint so overlap is a point collision. Holds get TTL + sweeper (never a lock without expiry).</p><p>Graceful degradation + tradeoff. Lose the claim → don’t fail event creation; create it, mark room: none, notify &quot;pick another.&quot; Strong only on the claim path (low QPS); eventual on browse (high QPS). (People are advisory — double-booking legal, RSVP resolves; only auto-scheduling N attendees makes people exclusive → N-way atomic hold / saga.)</p><h4>B2. Device Sync</h4><p>Crux: keep N devices consistent with minimal transfer, survive device loss — without per-device server state.</p><p>Design. Server owns a durable ordered per-user Change Log (PK=user_id, SK=version monotonic; ~30–90-day retention). Device owns only a throwaway cursor (opaque sync_token = &quot;seen up to V&quot;). GET /events?sync_token: no token → full sync; valid → delta (SK &gt; v) + next_sync_token; older than retention → 410 Gone → discard + full-sync.</p><p>Trigger + tradeoff. Default stateless pull on foreground (no server cursor to GC); optional device registry (user → push_endpoint) sends a silent push = &quot;wake and pull&quot; for closed-app propagation (stores <em>how to reach you</em>, not <em>what you&#39;ve seen</em>). Device loss is free — just the no-token case; nothing important lived on the device.</p><h4>B3. Alert 30-min-before (and the 9am thundering herd)</h4><p>Crux: fire millions of reminders within 5s <em>without scanning</em>, surviving extreme skew.</p><p>Design. Time-bucketed schedule table keyed time_bucket#shard, SK=notif_id; inserted async off CDC. Firing = N stateless pollers, each owns a shard range; at t2 pull buckets (t1,t2] and dispatch — O(due). Delivery idempotent (event_id:user_id:fire_ts), fire-and-forget.</p><p>Surviving the 9am herd — two stacked mitigations. (1) Shard the bucket key → spreads 10M/min across N partitions + workers. (2) Timezones smear it for free — “9am” is 9am <em>local</em>, stored in UTC → fans across ~24 offsets, turning one spike into dozens. Math: 10M/60s ≈ 170K/s ÷ N shards ÷ 24 offsets → a few K/s per worker. (Clusters like India/US aren’t uniform → smear helps but isn’t sufficient, which is <em>why</em> you still shard.)</p><h4>B4. Recurrence event</h4><p>Crux: represent a possibly-infinite series compactly, render many occurrences at once, preserve history, allow single-instance exceptions, stay 9am-local across DST. Trap = materializing the unbounded thing.</p><p>Design — rule + rolling bounded window. Store one row = the RULE (RRULE + DTSTART + UNTIL/COUNT or nothing = infinite) — compact, infinite-safe. A background job generates occurrences into a bounded window (next ~N months into the ZSET, each a concrete (event_id, occurrence_date)) and re-arms forward; out-of-window ranges expand on read. &quot;Standup forever&quot; = 1 rule + ~1yr materialized, not 27K rows.</p><p>Exceptions + timezone. “Move <em>this</em> Monday” = override row (series_id, original_start) → {modified | CANCELLED}; read = generated ∪ overrides. ?scope=this writes an override; this_and_following splits the series; series edits the rule. DST-safety: store recurring series as wall-clock local + IANA tz-id, convert to UTC at expansion. Reconciles with reminders: one-off + concrete reminders → UTC; recurring series → wall-clock+tz, convert at expansion.</p><h4>B5. Reminder invalidation — event moves 3pm→4pm, a “remind 2:55” row is already bucketed</h4><p>Crux. Reminders are pre-materialized into buckets; on edit the old row is stale (fires wrong; new one never fires). Invalidate old + schedule new — eagerly (at edit) or lazily (at fire)?</p><p>Design — eager, because fires ≫ edits. Every reminder fires (~10M/day); few events are edited. Pay on the rare path: on edit, tombstone old + insert new; finding the old row is cheap via the event_id → reminder_ids index (kept anyway for delete-cascade). Hot fire path stays read-free. (Lazy validate-at-fire — stamp event_version, drop mismatches — is correct but taxes the common path; it wins only if edits are frequent.)</p><p>Edges + tradeoff. Old/new have different fire_ts → dedup key keeps them distinct. Lazy&#39;s fire-read is a batched multi-get per bucket (not 2×-per-event). Verdict: eager here; flip to lazy if edit churn dominates. Reflex: <em>stale pre-materialized state → invalidate eagerly or validate lazily by which event is more frequent</em> (inverse of Calendar-View, where reads dominate so you rebuild lazily).</p><h4>B6. Clock skew — poller clock is 90 seconds fast. What breaks?</h4><p>Crux (correctness first). Poller decides “due” from local now() → +90s pulls (t1, now+90] → every reminder fires ~90s early = direct SLA violation + user-visible bug. Mixed skews → inconsistent fire times. Secondary: bucket-boundary overlap/gaps.</p><p>Root cause + right fix. A physical-time firing decision trusting an unsynchronized clock — a time-sync problem, <em>not</em> event-ordering (logical/vector clocks can’t say “it’s 2:55 in the real world”). Fix: NTP-sync + bounded skew; take the fire boundary from a trusted time source (Spanner TrueTime or a central coordinator handing out the current bucket-id).</p><p>Defense-in-depth. Idempotent dispatch (double-processed bucket can’t double-send) + per-shard lease (one owner) + clock-drift health-checks (self-evict past the bound). NTP + trusted-time <em>prevent</em>; idempotency + leasing + eviction <em>contain</em>. Reflex: answer user-visible correctness first; match the clock to the problem.</p><h4>B7. “Remind 5 min before” on an event recurring forever — how many reminder rows?</h4><p>Crux + count. Materializing a reminder per occurrence forever = the unbounded trap. Decouple reminder materialization from event creation: a background job rolls a short window forward just-in-time. Count for infinite daily standup = occurrences in the horizon (~1 pending row), never infinite.</p><p>Two windows + crash-safety. Reuses the recurrence rolling-window pattern, <em>smaller</em> horizon: occurrences need ~1yr (render views); reminders only fire soon → max(reminder_lead_time)+buffer (a &quot;1 week before&quot; needs a ≥1-week window). Edits outside the window are no-ops. The job is checkpointed + idempotent (&quot;materialized up to T&quot;) → a missed run delays but never drops. Reflex: <em>when a new problem rhymes with a solved one, reuse the pattern and only adjust the parameter.</em></p><h4>C1. Auto-scheduler double-booking across the free/busy CDC lag</h4><p>Crux — the human-in-the-loop boundary flips the resource class. For <em>human</em> scheduling, a person is advisory: accept a meeting, CDC lags ~200–500ms, someone books you again in that window → double-booked → the human resolves it (decline one, attend the other). Eventual is correct there — a human absorbs the conflict. But an interview-panel auto-scheduler that commits without human confirmation removes the resolver — so each attendee becomes an exclusive resource for that booking (like a room), and a stale free/busy read now produces a <em>silently broken panel</em> nobody catches until the interview starts. A post-commit notification can’t un-break a confirmed 8-person panel — so this case needs prevention at claim time, not detect-and-compensate.</p><p>Design — strong reads + atomic N-way hold. The auto-scheduler must not trust the eventual free/busy view; it reads each candidate slot strongly-consistent and performs an all-or-nothing hold across all 8 attendees — the room-claim CAS generalized from 1 resource to N. Because the 8 holds are independent, this is a saga / 2PC: place a short-TTL hold on each of the 8; if all 8 succeed → commit the panel; if any fails (a 7th just got booked) → release the holds already taken (compensating action) and either retry a different common slot or report “no common time.” The TTL guarantees a crashed scheduler can’t leak permanent holds. Strictly harder than a room (one atomic claim) because it’s N-way atomic across N independent people.</p><p>Tradeoff + reflex. Reserve this heavy path for the narrow case that needs it — auto-commit + must-all-be-present. Everything else (normal human invites) stays on the cheap optimistic path: book, detect conflict on CDC, notify + one-click reschedule. Match the consistency tier to (a) whether a human is in the loop and (b) whether the commit is reversible — human + reversible → optimistic + compensate; no-human + must-be-correct-at-commit → strong reads + atomic N-way hold with compensating release. The staff signal: “advisory vs exclusive” isn’t a property of the entity (a person) — it’s a property of <em>how it’s being booked</em>.</p><h4>C2. Guest sees “12 accepted” but the organizer sees “11.” Where did consistency break?</h4><p>Crux — split the two things wearing the word “accepted.” The trap: 12 vs 11 looks like one problem but bundles two, with <em>different</em> consistency needs. The count (&quot;12 accepted&quot;) is a derived, event-cached aggregate updated async off the RSVP append-log (SoT) → eventual / approximate, accepted by design (nobody makes a decision on the exact integer; converges in ~200-500ms). Two observers simply read the cached count at slightly different lag points — or two RSVPs raced — neither is a real defect for a count.</p><p>Name the consistency model <em>per actor</em>. Reaching for “read-your-writes” here is a category slip: RYW protects the <em>writer</em> seeing their own write — but the guest and organizer are observers of a third person’s RSVP, so RYW says nothing about them; they correctly get eventual. RYW <em>does</em> live in this feature — for the RSVP-er, who must see their <em>own</em> Accept flip to “going” instantly. So: count → eventual (observers); own RSVP status → RYW (the writer); others’ statuses → eventual.</p><p>Where it “broke” + reflex. The event-cached count lagged the RSVP append-log (SoT) — the async derived copy hadn’t caught up (or an in-flight RSVP landed between the two reads). Not a bug — expected convergence. Reflex: when you name a consistency model, name the actor it protects — “eventual for whom? RYW for whom?” RYW = the writer reading their own write; eventual = everyone else observing that write. Same per-actor precision as strong-for-room-claim / eventual-for-browse.</p><h4>C3. Timezone politics — a country changes its DST rules next year; millions of recurring events stored as wall-clock+tz</h4><p>Crux — this is the case that <em>justifies</em> wall-clock+tz over UTC. Each recurring event stores the intent — 9:00 + America/Sao_Paulo, not a baked UTC instant. The offset is <em>computed</em> from the IANA tz database at expansion time, not frozen at create. So when a country drops DST and the tz-DB ships the new rule, the next expansion looks up the <em>current</em> rules → correct UTC → meeting stays 9am local, automatically, zero events touched. Had you stored UTC (bake 9am-local → 12:00 UTC), that snapshot silently renders as 8am/10am after the offset change and needs a fleet-wide rewrite. Storing intent turns a migration into a free re-resolution.</p><p>What actually needs doing (bounded, not fleet-wide). Future not-yet-materialized occurrences: nothing — they expand under the new rules for free (no migration, no ±-offset patch — there’s no stored offset to patch). The <em>only</em> stale data is occurrences already materialized into the ZSET + reminder window under the <em>old</em> rules (~1yr horizon) → re-expand that bounded window for events in the affected tz when the update lands — a targeted refresh of a derived cache (same “derived is rebuildable from SoT” property used everywhere). Operationally: pin the tz-DB version, roll updates deliberately, trigger the re-expansion as part of the rollout.</p><p>Edge + reflex. A rule change can leave a wall-clock time in a discontinuity — a 9am that now <em>doesn’t exist</em> (spring-forward gap) or <em>exists twice</em> (fall-back overlap) → documented policy (shift forward to next valid instant; pick first of a doubled instant). Reflex: store intent, not a snapshot of it — a government/DST rule change then updates the <em>rules engine</em> (IANA DB) and events re-resolve against it, instead of forcing a data migration. (Same “store the generator, not the output” instinct as recurrence: the rule survives change; a materialized answer doesn’t.)</p><h4>D1. Do the math: 7-year retention, 10M events/day. How big is the Event store? Does it change the design?</h4><p>BOTE. 10M/day × 365 × 7 ≈ 25B events. At ~1KB/event (10 fields × ~100B), that’s 25B × 1KB = 25 TB (watch the unit conversion — 25×1⁰⁹ × 1⁰³ B = 25×1⁰¹² B = 25TB, <em>not</em> 2.5TB). Storable on sharded DocumentDB without drama.</p><p>Does it change the design? Yes — but the driver is access-pattern skew, not raw volume. 25TB is <em>storable</em>; the problem is that ~99% of it is far-past events read almost never, so you’d be paying hot-store $/GB + index overhead for cold data. So tier by access frequency: the Redis ZSET already holds only the ±90-day hot window; the Event DB holds recent SoT; cold storage (object store / Parquet / cold Cassandra) holds the multi-year archive, paged in only on explicit “show me 2021” history queries.</p><p>Reframe + reflex. The 7-year figure is a compliance/retention floor, not a hot-query requirement — so size the hot store for ~1yr of activity and the cold store for retention. (Coherent with the rest: cold storage becomes SoT for old events, so “rebuild the ZSET from SoT” for an old window pages from cold.) Reflex: do the math <em>and sanity-check the units</em> (a 40× slip on a headline number is what an interviewer remembers), and “does it change the design?” is usually about access-pattern skew (recent-hot / old-cold), not total bytes — storage is cheap, hot storage + indexing isn’t.</p><h4>D2. Exec’s shared calendar — 50K events, 200 delegates all watching it</h4><p>Crux — the ZSET is what’s <em>stressed</em>, not the answer. “We have a ZSET” is the substrate; the scenario hits two of its failure modes: a read hot key (200 delegates hammering one cal:{exec} key on one shard) and write amplification (one exec edit potentially fanning to 200 delegate views). Plus 50K events is a pagination matter (never fetch the whole set), distinct from the hot key.</p><p>Design — one mechanism solves both. Read hot key → replicate the exec’s shared ZSET across read replicas + short TTL (rejected key-splitting key_0/key_1 — it adds <em>write</em> load; replication eases reads at the cost of eventual consistency). Write amplification → delegates <em>pull</em> the exec&#39;s single shared ZSET directly (with an authz check), do NOT fan out the exec&#39;s edits into 200 delegate views. The elegant part: these are the same fix — the replicas added for the hot key <em>are</em> what the 200 delegates pull from, so one shared replicated copy kills both problems (no 200× write fan-out, no single-shard read storm).</p><p>50K events + tradeoff. 50K is not a hot-key problem — it’s pagination discipline: reads are always range-bounded (ZRANGEBYSCORE for the visible window) + cursor-paginated; a month view touches hundreds, never 50K. Tradeoff: replication → eventual consistency on the shared view (fine for a calendar). Reflex: name a fix <em>and</em> its cost (replication → eventual), then look for when two problems share one fix — replicate-once + pull-from-replicas beats paying for two separate mechanisms.</p><h4>D3. Search: “find my lunch with Priya last year.” Does ES only hold the hot window?</h4><p>Crux — reject the premise. A hot-only index can’t answer “last year,” so the trap is copying the ZSET’s hot-windowing onto search. But search is a different access pattern: rare, latency-tolerant, and <em>inherently historical</em> (you search precisely because you don’t remember when). So it should NOT inherit the ZSET’s ±90-day window — ES indexes the full 7 years. ES is already derived + rebuildable + off the hot read path, so holding all history never touches latency-critical reads.</p><p>Design — one full-history <em>lean</em> index, because search QPS is tiny. Search volume is ~1/100 of reads, so you <em>don’t</em> hot/cold-split ES — a single cluster indexes all 7 years (low QPS argues for <em>simpler</em>, not a second tier). Index lean docs — only searchable fields + event_id (title, description, attendee names, start_time), not the full event payload, so &quot;7 years in ES&quot; is a compact text index, not a second copy of everything. ES is fed via CDC off the Event SoT (same spine); the multi-year archive is indexed from the cold store (D1). Flow: &quot;lunch with Priya last year&quot; → ES query scoped to my events (authz filter — security NFR) → event_ids → hydrate full details from Event DB / cold store. Some latency, acceptable by SLA (nobody needs a year-old lunch in 100ms). <em>Only</em> tier ES hot/cold if the index size itself becomes a cost problem.</p><p>Store roles — three query shapes, three stores (the ZSET does NOT do search). Each store indexes by a <em>different</em> key, so each answers a <em>different</em> question and none can do another’s job: point-read by id → Event DB; date-range → Redis ZSET (numeric range on start_time); free-text → Elasticsearch (inverted index over words). &quot;Find Priya&quot; is a keyword query → ES only; the ZSET is scored by time and has no idea what&#39;s <em>inside</em> an event.</p><p>Reflex. When a question frames a constraint (“ES only holds hot?”), check whether the constraint even <em>should</em> apply — the ZSET is hot-windowed because of scroll traffic; search has different economics (rare, slow-ok), so it doesn’t inherit that windowing. Match the tiering to <em>each access pattern’s</em> economics — don’t copy one pattern’s tiering onto another.</p><h4>E1. Full-trace: drag ONE instance of a recurring meeting (30 guests + a room) across a DST boundary to a new time</h4><p>Crux — every noun in the prompt is a checklist item. “one instance” → override, not the rule. “recurring” → don’t touch the base RRULE. “30 guests” → per-attendee fan-out. “a room” → the one strong-consistency path (can’t just async fan-out). “across a DST boundary” → wall-clock+tz re-resolution. Tick each off explicitly and nothing gets dropped.</p><p>The one synchronous, strong step — the room (easy to forget). PATCH /events/{id}?scope=this + If-Match → Event Service writes an override row (series_id, original_start) in Event DB; the base rule is untouched. The room move is the <em>only</em> piece that is NOT async: release the old slot + strong CAS-claim the new slot on Spanner (a room is exclusive). This can fail — new time&#39;s room may be taken → degrade (create the move, mark room: none, notify &quot;pick another&quot;), don&#39;t silently double-book. DST: the override stores wall-clock + tz-id, and the occurrence&#39;s new UTC instant is re-derived through the tz-DB — so crossing the DST boundary doesn&#39;t shift it by an hour.</p><p>Everything else is async off CDC (per-attendee ×30). Emit to the Change Stream → ES re-indexes this occurrence; reminders re-bucketed at the new DST-correct instant (old rows drop-stale at fire — B5/B6); 30 ZSETs updated (the materialized occurrence moves; ±90-day window); 30 Change-Log entries so each guest’s device syncs the delta; 30 User-Availability intervals shift (old slot freed, new marked busy). Product calls: re-notify all 30 (“meeting moved”) and decide whether a time change resets RSVPs. Reflex: in a full-trace, walk the nouns — and remember the lone strong path (room) sits <em>outside</em> the async fan-out, while the DST crossing forces wall-clock+tz re-resolution of both the occurrence and its reminder.</p><h4>Cross-cutting lessons (what separates staff from senior)</h4><ol><li>Re-derive a named pattern’s mechanics in <em>this</em> domain — say what “pull” <em>is</em> here; what a stale browse view actually causes.</li><li>Trace failures through your <em>own</em> consistency decisions — the strong room claim can’t double-book even with CDC down.</li><li>Ask “what side-effects were silently swallowed?” — reminders during an outage; disjoint vs same-field edits.</li><li>When a thing is unbounded, store the <em>generator</em>, materialize a <em>bounded window</em> on demand — recurrence rule, not ∞ rows (same instinct as not storing every GPS ping / not fanning out to 100M followers).</li><li>Consistency is per-access-pattern, not global — strong only where there’s a true exclusive invariant (rooms); eventual + human-in-the-loop everywhere it’s advisory (people).</li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=0a6f884811c9" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[FAANG Staff Level: Designing API Endpoints for System Design Guide]]></title>
            <link>https://medium.com/@nishant.gargvii/faan-staff-level-designing-api-endpoints-for-system-design-guide-db6cc9705148?source=rss-d18da405456a------2</link>
            <guid isPermaLink="false">https://medium.com/p/db6cc9705148</guid>
            <category><![CDATA[software-engineering]]></category>
            <category><![CDATA[backend-development]]></category>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[api-design]]></category>
            <category><![CDATA[system-design-interview]]></category>
            <dc:creator><![CDATA[Nishant Garg]]></dc:creator>
            <pubDate>Sat, 04 Jul 2026 15:02:51 GMT</pubDate>
            <atom:updated>2026-07-04T15:14:03.339Z</atom:updated>
            <content:encoded><![CDATA[<blockquote><em>Distilled from Staff-level API-design interview practice across </em><strong><em>9 systems</em></strong><em>: ticketing, file storage, chat, ride-sharing, payments, notifications, collaborative docs, news feed, and video. This is the “API signatures” slice of a system design round — what makes a set of endpoints read as </em>Staff-level<em> rather than CRUD.</em></blockquote><h3>Table of Contents</h3><ol><li>The API-design rubric</li><li>Foundations: resource modeling</li><li>Where things go: path vs query vs body vs header</li><li>Reads: list, search, pagination</li><li>Reads: single resource, batch, versions</li><li>Writes: idempotency, status codes, async</li><li>Idempotency key vs dedup key</li><li>State machines</li><li>Concurrency &amp; consistency (ETag, optimistic vs pessimistic)</li><li>Error model</li><li>Real-time APIs (WebSocket)</li><li>Security</li><li>Large data &amp; media (pre-signed URLs, resumable upload, manifests)</li><li>Scale patterns (fan-out, ingestion, approximate counts, geo)</li><li>Money APIs</li><li>Bulk operations &amp; partial failure</li><li>Versioning</li><li>The 9 worked domains (cheat sheet)</li><li>Pre-submit checklist</li><li>Recurring anti-patterns</li><li>Appendix A — the 9 full model answers</li></ol><h3>1. The API-design rubric</h3><p>When you’re scored on API signatures, these are the dimensions. “Good” = 7; the bar behind the number is what shifts with level.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*YWVslIweesEBcUWz4q3COg.png" /></figure><p><strong>F</strong>or every endpoint, ask <em>“who consumes this?”</em> (end-user vs another service) and <em>“what’s the consistency/retry contract?”</em> Those two questions drive most of the rubric.</p><h3>2. Foundations: resource modeling</h3><h4>Plural nouns, not verbs</h4><p>A path segment names a collection; appending an id names a member:</p><pre>/v1/events            the collection (all events)<br>/v1/events/{eventId}  one member</pre><ul><li>The path is a noun; the HTTP method is the verb. The same path does different operations:</li></ul><pre>GET    /events        list<br>POST   /events        create<br>GET    /events/{id}   read one<br>PUT    /events/{id}   update<br>DELETE /events/{id}   delete</pre><ul><li>Anti-patterns: /createEvent, /getEvent, /shorten, /book, /share — verbs in the path. The verb belongs in the method.</li><li>Plural for consistency. /events, /orders, /holds, /users — always plural. /event (singular) only reads right for one item and lies about the list operation.</li><li>Nesting reads naturally: /events/{id}/seats/{seatId} = &quot;this event&#39;s seat.&quot;</li><li>Exception — singletons: when there’s exactly one per parent, singular is fine: /users/{id}/profile, /events/{id}/settings.</li></ul><h4>Stable identity, not mutable keys</h4><p>Use a stable id as identity, not a mutable path/name. A rename must not break references:</p><ul><li>Good: /files/{fileId} (path is mutable metadata)</li><li>Bad: /files/{path} (renaming the file breaks every link)</li></ul><h3>3. Where things go</h3><p>The single most important modeling decision: path vs query vs body vs header.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ewpXaaMG5crjGtwfP_PRxw.png" /></figure><p>Rules that fall out of this:</p><ul><li>Identity → path, filter → query. /events/{id} (fetch one) vs /events?venueId= (filter many).</li><li>Never put filters in the path (/events/city/NYC falsely implies a sub-resource).</li><li>Never put secrets/PII/large inputs in the URL (logs, caches) — put them in the body or a header. (See §12.)</li><li>Identity comes from the auth token, never a client-supplied userId header/param — otherwise a caller can impersonate.</li></ul><h3>4. Reads: list, search, pagination</h3><h4>Anatomy of a list/search endpoint</h4><pre>GET /v1/events?q=&amp;city=&amp;from=&amp;to=&amp;cursor=&amp;limit=50</pre><p>Every list query is a mix of four kinds of params:</p><ol><li>Free-text search — q (opaque fuzzy match).</li><li>Structured filters — city, from, to (exact/range; range params come in pairs; ISO-8601 dates).</li><li>Pagination — cursor + limit (below).</li><li>Sorting — sort=startsAt:asc (optional).</li></ol><ul><li>Use flat params, not a filter={json} blob — flat is cacheable, validatable, debuggable.</li><li>Multi-value: repeat the key (category=rock&amp;category=jazz).</li></ul><h4>Cursor pagination, not offset</h4><p>Offset (?offset=100&amp;limit=50) breaks on live data:</p><pre>Page 1: offset=0,limit=3 -&gt; [A,B,C]<br>  ...item X inserted at top...<br>Page 2: offset=3,limit=3 -&gt; list is [X,A,B,C,D,E] -&gt; [C,D,E]   # C shown twice</pre><p>Deletes cause skips; deep offsets are O(n) scans.</p><p>Cursor (?cursor=&lt;opaque&gt;&amp;limit=50) fixes both:</p><ul><li>The cursor encodes the last-seen sort key (e.g. base64 of (startsAt, id)).</li><li>The DB does an indexed seek (WHERE (startsAt,id) &gt; (?,?) ORDER BY ... LIMIT 50).</li><li>Opaque so you can change the encoding later.</li><li>Response returns nextCursor (null when done).</li></ul><h4>Response envelope</h4><p>Never return a bare array — wrap it so you can add metadata later:</p><pre>200 { &quot;events&quot;: [ {...} ], &quot;nextCursor&quot;: &quot;eyJ...&quot; }</pre><h4>Defaults &amp; caps</h4><ul><li>limit: default (e.g. 50), server-enforced max (e.g. 100). Clamp or 400 if exceeded.</li><li>Bad cursor → 400. Add Cache-Control/ETag for hot, filter-light queries.</li></ul><h3>5. Reads: single resource, batch, versions</h3><h4>Get one member</h4><pre>GET /v1/events/{eventId}<br>200  ETag:&quot;7&quot;  { id, name, ... }     # FULL representation (vs summary in a list)<br>404  if missing</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9uDAKX37AfwXBHprVpVbdg.png" /></figure><p>{eventId} is template notation — the real URL is /v1/events/e_123. Path segments are positional; you never write ?id= grammar in a path.</p><h4>Batch reads</h4><p>Fetching a <em>set</em> is a filter, not an identity → query or POST body:</p><pre>GET  /v1/events?ids=e_1,e_2,e_3            # small, cacheable (cap ~100; URL length limit)<br>POST /v1/events:batchGet { ids:[...] }     # large payload (loses GET caching)</pre><p>The defining decision is partial failure. Don’t 404 the whole batch — return per-item results:</p><pre>200 { &quot;events&quot;:[ {id:e_1,...} ], &quot;notFound&quot;:[&quot;e_3&quot;] }      # or a map { &quot;e_1&quot;: {...}, &quot;e_3&quot;: null }</pre><h4>Optional version selector</h4><pre>GET /v1/docs/{id}                  -&gt; latest<br>GET /v1/docs/{id}?version=42       -&gt; revision 42   (omitted = latest)</pre><ul><li>Always echo the version you served (+ ETag) so the client knows what it got.</li><li>For a system with real history, prefer versions as a sub-resource: GET /docs/{id}/revisions/{rev} + GET /docs/{id}/revisions?cursor=.</li><li>Bad version → 404; malformed → 400; pruned → 410.</li></ul><h4>Sparse fieldsets</h4><p>Use ?fields=status,holder for selective fields — not GraphQL-in-a-GET-body.</p><h3>6. Writes: idempotency, status codes, async</h3><h4>Idempotency keys (the #1 write primitive)</h4><p>Every state-changing POST takes an Idempotency-Key header:</p><pre>POST /v1/orders   Headers: Authorization, Idempotency-Key: &lt;uuid&gt;</pre><ul><li>Server stores key → response (e.g. 24h). Same key → returns the same result, performs the action once. A dropped response + client retry never double-charges / double-books.</li><li>Same key + different body → 422 IDEMPOTENCY_CONFLICT.</li><li>This is the difference between a draft and a real write API. Put it on <em>every</em> unsafe write.</li></ul><h4>Status codes for writes</h4><ul><li>201 Created — resource created synchronously (return the id).</li><li>202 Accepted — accepted for async processing (return a handle/status).</li><li>204 No Content — success, nothing to return (idempotent PUT/DELETE).</li></ul><h4>Async writes</h4><p>When work is slow (transcoding, fan-out, external settlement), don’t block:</p><pre>POST /v1/videos/{id}/... -&gt; 202 { jobId, status:&quot;PENDING&quot; }<br>GET  /v1/.../{jobId}      -&gt; { status:&quot;PENDING&quot;|&quot;DONE&quot;|&quot;FAILED&quot; }     # poll<br>POST /v1/webhooks/...     (signed) -&gt; transitions the job                # or webhook</pre><p>Pattern: submit → 202 → status lifecycle → poll and/or webhook on completion. Gate dependent operations on the terminal state (e.g. “watch” returns 409 NOT_READY until READY).</p><h3>7. Idempotency key vs dedup key</h3><p>They look redundant; they operate at different layers.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Ys8T56GDiK9_OKtG-q8TMw.png" /></figure><ul><li>IKey = “did THIS request already happen?” (transport correctness).</li><li>dedupKey = “did this NOTIFICATION/EVENT already go out?” (business rule). Built deterministically from the event identity — {eventType}:{entityId}:{userId}[:{timeBucket}], or the upstream event id. Never random; never from rendered content. (Content varies for the same event — locale/AB/timestamps — and can collide for different ones; dedup by <em>meaning</em>, not bytes.)</li></ul><p>At-least-once message buses guarantee the same logical event arrives as separate requests — that’s why you need both.</p><h3>8. State machines</h3><p>When a resource moves through a lifecycle (a trip, an order, a hold), model transitions as action sub-resources, each idempotent and actor-authorized:</p><pre>POST /v1/rides/{id}/accept    (driver)  -&gt; 200 | 409 ALREADY_TAKEN<br>POST /v1/rides/{id}/arrive    (driver)<br>POST /v1/rides/{id}/start     (driver)  -&gt; IN_PROGRESS<br>POST /v1/rides/{id}/complete  (driver)  -&gt; COMPLETED<br>POST /v1/rides/{id}/cancel    (rider|driver) {reason}</pre><ul><li>Prefer explicit action endpoints over PATCH {status} — they make authorization (&quot;only the driver can accept&quot;) and idempotency (&quot;repeat returns current state&quot;) natural.</li><li>The race (two actors advancing the same resource) resolves to 409 Conflict.</li></ul><h3>9. Concurrency &amp; consistency</h3><h4>ETag — version fingerprint</h4><p>The server stamps a resource version; the client echoes it back under a different header name:</p><pre>Server -&gt; Client:  ETag: &quot;7&quot;                 (response only)<br>Client -&gt; Server:  If-None-Match: &quot;7&quot;        (conditional read -&gt; 304 Not Modified)<br>Client -&gt; Server:  If-Match: &quot;7&quot;             (conditional write -&gt; 412 Precondition Failed)</pre><ul><li>If-None-Match → caching: skip re-downloading unchanged data (304).</li><li>If-Match → optimistic concurrency: refuse a write if the resource changed under you (412), turning a silent lost-update into a loud retry.</li></ul><h4>Optimistic vs pessimistic</h4><ul><li>Optimistic (ETag/If-Match): assume no conflict, detect at write time. Good for low-contention edits (admin settings, doc metadata).</li><li>Pessimistic (a hold/lock with TTL): grab the resource first. Good for high-contention scarce resources (a seat). POST /events/{id}/holds -&gt; { holdId, expiresAt }.</li></ul><h4>Consistency tiers</h4><ul><li>Strong where correctness demands it: balances, seat availability, transfers (read-after-write).</li><li>Approximate / eventually-consistent where scale demands it: like counts, view counts, follower counts at billions/day. Don’t model likeCount as a transactional read — it&#39;s a cached counter. Say so.</li></ul><h4>Conflict resolution (collaborative editing)</h4><p>You don’t PUT the whole doc — you send operations against a base revision:</p><pre>C-&gt;S { type:&quot;edit&quot;, baseRevision:7, ops:[{op:&quot;insert&quot;, afterCharId, char, charId}, ...] }<br>S-&gt;C { type:&quot;edit.ack&quot;, committedRevision:10, transformed:true }     # server transformed + assigned<br>S-&gt;C { type:&quot;edit&quot;, revision:10, ops:[...] }                          # broadcast to others</pre><p>Two edits against rev 7 with the server at rev 9 → server transforms the later op (OT) or merges via char-ids (CRDT), assigns a monotonic revision, and everyone converges. Char-ids double as stable anchors for comments/cursors that survive edits.</p><h3>10. Error model</h3><h4>4xx (caller’s fault — fix the request) vs 5xx (your fault — often retryable)</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Tuex2A9h8QZ4-kEdf7GpQQ.png" /></figure><p>Confusing pairs:</p><ul><li>401 vs 403 — authN vs authZ.</li><li>400 vs 422 — can’t parse vs parsed-but-invalid.</li><li>409 vs 422 — 409 depends on current state (may succeed later); 422 is wrong about the request itself.</li><li>410 vs 404 — gone-and-known vs never/unknown.</li></ul><h4>Structured envelope + retryable flag</h4><pre>{ &quot;error&quot;: { &quot;code&quot;: &quot;SEAT_ALREADY_HELD&quot;, &quot;message&quot;: &quot;...&quot;, &quot;retryable&quot;: false, &quot;details&quot;: {...} } }</pre><ul><li>Retryable (backoff + same IKey): 429, 503, 504, often 500.</li><li>Terminal (fix request): 400/401/403/404/409/422.</li></ul><p>Pick the most specific status; add a typed code (one 409 has many causes).</p><h3>11. Real-time APIs (WebSocket)</h3><p>A WebSocket API is not request/response — it’s a bidirectional message catalog over one long-lived connection. You spec: (1) how to connect, (2) client→server messages, (3) server→client messages.</p><h4>Shared base (every WS design)</h4><pre>POST /v1/realtime/ticket   (Bearer) -&gt; { ticket, expiresIn:30 }       # auth (see §12)<br>WSS  /v1/connect?ticket=&lt;short&gt;&amp;since=&lt;cursor&gt;                         # since = resume point</pre><pre>Envelope:   { &quot;type&quot;: &quot;...&quot;, &quot;id&quot;: &quot;&lt;uuid&gt;&quot;, &quot;payload&quot;: {...} }       # id = correlation/dedup<br>Lifecycle:  S-&gt;C {connected}  C&lt;-&gt;S {ping/pong}  S-&gt;C {error:{code,retryable}}</pre><h4>Three reusable templates</h4><ul><li>Pub/Sub (subscribe → receive) — notifications, dashboards, live scores, “chat receive.”</li></ul><pre>C-&gt;S {subscribe, {topic}}   S-&gt;C {event, {topic, seq, data}}</pre><ul><li>Reliable full-duplex (send + receive, ack + resume) — chat, collab editing, ride-offer→accept.</li></ul><pre>C-&gt;S {send, {clientMsgId, ...}}   S-&gt;C {ack, {clientMsgId, seq}} S-&gt;C {message, {seq, ...}}        C-&gt;S {ack, {upToSeq}}     # resume via ?since=&lt;seq&gt;</pre><ul><li>Telemetry (high-freq, lossy, no ack) — ride location, live cursors, sensors.</li></ul><pre>C-&gt;S {telemetry, {lat,lng,ts}}    S-&gt;C {update, {entityId,...}}</pre><h4>What makes it real-time (REST gives these for free)</h4><ol><li>Envelope with type + id (async, bidirectional → need discriminator + correlation).</li><li>Application-level acks (TCP delivery ≠ app processed).</li><li>Heartbeat (ping/pong detects dead sockets).</li><li>Resume (reconnect with last seq/cursor; server replays the gap).</li></ol><h4>Hybrid: REST for writes, WS for push</h4><p>Keep POST /messages for sending (idempotency, retries, load balancing) and use the WS only for server→client push. Don&#39;t run everything over the socket.</p><p>Two cursors for messaging: a forward sync cursor (offline catch-up + live) and a backward history cursor (?before=&lt;seq&gt;) — different feeds, different cursors. Account-level /sync (all conversations) ≠ per-conversation history; don&#39;t make catch-up N calls (one per conversation).</p><h3>12. Security</h3><h4>Auth</h4><ul><li>Identity from the Bearer token (and scopes like wallet:write, admin) — never a client-supplied userId/followerid header or param.</li><li>Service-to-service APIs (platform consumers) use service auth + scopes.</li></ul><h4>Never put secrets/PII in a URL</h4><p>URLs are logged everywhere (LB, proxy, CDN, access logs) in plaintext, even over TLS. So:</p><ul><li>Tokens → Authorization header (or a short-lived ticket, below).</li><li>PII / precise location → request body (POST even for a “read” like a fare estimate).</li><li>Only non-sensitive identifiers/filters → path/query.</li></ul><h4>The WebSocket ticket pattern</h4><p>Browsers can’t set headers on new WebSocket(url), so you can&#39;t send the bearer token cleanly, and you must not put it in the URL. Instead:</p><pre>POST /v1/realtime/ticket  (Bearer real token)  -&gt; { ticket, expiresIn:30 }   # secret in header<br>WSS  /v1/connect?ticket=&lt;short-lived single-use&gt;                             # disposable in URL</pre><p>If the ticket leaks into logs it’s expired in 30s and single-use. (Native clients can use the Authorization header on the upgrade; browsers can use an httpOnly cookie + Origin check.)</p><h4>Webhook signatures (HMAC)</h4><p>A webhook endpoint is public — anyone can POST. Verify the sender with an HMAC signature:</p><pre>POST /v1/webhooks/...   X-Signature: sha256=&lt;hmac&gt;   X-Timestamp: ...<br>verify:  hmac_sha256(shared_secret, raw_body) == X-Signature   (constant-time compare)</pre><ul><li>HMAC = keyed hash; only someone with the shared secret can sign; the secret is never transmitted (only a per-request signature). Proves authenticity + integrity.</li><li>Sign the raw body (not re-serialized JSON); constant-time compare; include a timestamp and reject stale requests (replay defense); idempotent dedup on a provider ref; support secret rotation.</li></ul><h3>13. Large data &amp; media</h3><h4>Pre-signed URLs / data-plane offload</h4><p>Don’t funnel bytes through your API tier. The control plane (your API) hands out a pre-signed URL; the data plane (bytes) flows client ↔ blob store / CDN directly:</p><pre>GET /v1/files/{id} -&gt; { downloadUrl, expiresAt }    # client GETs bytes from the blob store/CDN</pre><h4>Resumable upload (multi-part session)</h4><p>A single giant POST can’t survive a dropped connection at 47 GB. Model upload as a session:</p><pre>POST /v1/files                       -&gt; { fileId, uploadId, partSize }     # 1. initiate<br>GET  /v1/uploads/{uploadId}/parts/{n}/url -&gt; { url, expiresAt }            # 2. pre-signed per part<br>   (client PUTs each part&#39;s bytes directly to the blob store -&gt; gets an ETag per part)<br>GET  /v1/uploads/{uploadId}/parts    -&gt; { uploadedParts:[{n,etag}] }       # 3. RESUME = list parts<br>POST /v1/uploads/{uploadId}/complete { parts:[{n,etag}] } -&gt; 201 { fileId, version }   # 4. finalize</pre><p>Resume works because the part list is queryable and each part is content-addressed by ETag. complete is also where you&#39;d trigger downstream async work (e.g. transcoding).</p><h4>Manifest-based delivery (adaptive bitrate)</h4><p>Streaming media returns a manifest, not a file or “chunks”:</p><pre>GET /v1/videos/{id}/manifest -&gt; { manifestUrl(HLS/DASH), renditions:[{height,bitrate}], expiresAt }<br>   409 NOT_READY  (gated on transcoding status)</pre><p>A manifest is one document the player reads to fetch rendition segments from the CDN itself — your API does not expose per-chunk endpoints. (Upload chunks ≠ stream segments.)</p><h3>14. Scale patterns</h3><h4>Fan-out (feeds, notifications)</h4><ul><li>Fan-out-on-write — push a post into each follower’s precomputed feed. Fast reads, expensive for huge follower counts.</li><li>Fan-out-on-read — assemble the feed at query time. Cheap writes, slow reads.</li><li>Hybrid (the answer): push-on-write for normal users; celebrities (huge follower counts) are pulled at read time and merged in. The GET /feed endpoint is simple — the design is the fan-out strategy + the hot-key/celebrity exception.</li></ul><h4>High-frequency ingestion</h4><p>1M+ writes/sec (driver locations, cursors) can’t be a fat REST call per ping (auth + TLS + headers dominate; data is ephemeral). Instead:</p><ul><li>Push over the existing WebSocket (auth once at connect; ~30-byte frames).</li><li>Batch/coalesce; binary/lossy protocol (latest-wins, no per-ping ack).</li><li>Write to an in-memory geo/state index, not a durable DB per ping.</li></ul><h4>Approximate counts</h4><p>At billions/event-day, counts are cached / eventually-consistent, not transactional reads. Recording is fire-and-forget (POST /views -&gt; 202), deduped per user/window.</p><h4>Geo queries — internal vs public</h4><p>“Find near me” is usually an internal matching primitive the dispatch service calls, <em>not</em> a public “browse drivers” endpoint. A public version (map dots) is anonymized, display-only. Always label each endpoint’s consumer: public (user) vs internal (service-to-service).</p><h4>Thundering herd / on-sale</h4><p>For 1M users hitting one resource at t=0: 429 + Retry-After is backpressure, but the real answer is admission control — a virtual waiting room: POST /events/{id}/queue -&gt; { queueToken }, and the contended write requires an admitted token.</p><h3>15. Money APIs</h3><p>Two rules are non-negotiable:</p><ol><li>Money = integer minor units + currency, never a float.</li></ol><pre>{ &quot;amount&quot;: 1999, &quot;currency&quot;: &quot;USD&quot; }     # 1999 cents = $19.99; NEVER 19.99</pre><p>2. Floats can’t represent decimal money (0.1 + 0.2 != 0.3); errors accumulate; the ledger won&#39;t balance. JSON numbers are IEEE-754 doubles, so 19.99 on the wire is already lossy — use integer cents (or a decimal string &quot;19.99&quot;), and pair with currency (minor-unit count varies: USD=2, JPY=0, KWD=3).</p><p>3. The ledger is append-only / immutable. Never PATCH/DELETE a transaction. Corrections are new reversing entries (a refund references the original; the original never changes).</p><p>Plus:</p><ul><li>Idempotency = correctness here — same key → same transaction, never a second debit.</li><li>Atomic transfer is one resource with both sides: POST /transfers { fromAccountId, toAccountId, amount } (debit + credit commit together) — not two calls.</li><li>Async settlement via signed webhook — top-up/withdraw return 202 PENDING; the PSP webhook flips state. State transitions are never client-callable.</li><li>Available vs pending balance; double-entry (Σdebits = Σcredits).</li></ul><h3>16. Bulk operations &amp; partial failure</h3><p>For “do X to many” (send to millions, get many ids):</p><ul><li>At large scale, make it async: 202 { batchId }, then GET /.../{batchId}/results?cursor=.</li><li>Per-item result model — never a single 200/fail for the whole batch:</li></ul><pre>{ results:[ { id, status:&quot;DELIVERED&quot;|&quot;FAILED&quot;|&quot;SUPPRESSED&quot;, reason } ], nextCursor }</pre><ul><li>Channel/status are filters, not path segments — ?channel=sms&amp;status=failed, not /results/sms, /results/whatsapp (avoid the N×M endpoint explosion).</li><li>For platform APIs (notifications), enforce preferences/opt-out server-side: an opted-out recipient is accepted but SUPPRESSED — the caller can&#39;t override it. Send carries a category (TXN vs MARKETING) matched against preferences.</li></ul><h3>17. Versioning</h3><ul><li>URI versioning (/v1/...) — simplest for external partners; what most APIs use.</li><li>Additive fields are non-breaking; removals/renames are breaking → new major version.</li><li>Signal deprecation with a Sunset header.</li><li>State the strategy explicitly; pick URI vs header and justify.</li></ul><h3>18. The 9 worked domains (cheat sheet)</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*mmF7j_RTwxxOyf1Djiexsg.png" /></figure><h3>19. Pre-submit checklist</h3><p>For every endpoint set, verify:</p><ul><li>Resource-oriented (plural nouns, no verbs in path); stable ids</li><li>Each value in the right place (identity→path, filter→query, payload→body, cross-cutting→header)</li><li>Idempotency-Key on every unsafe write; replay returns the same result</li><li>Cursor pagination on every list; server-capped limit; nextCursor in an envelope</li><li>Typed errors with retryable; most-specific status; the right 4xx (409/410/422 not lazy 400/500)</li><li>Auth from token + scopes; no secrets/PII in URLs</li><li>Async work returns 202 + status lifecycle + poll/webhook; dependents gated on terminal state</li><li>Concurrency expressed (ETag/If-Match, hold+TTL, or action transitions)</li><li>Consistency stated (strong where it matters; approximate counts at scale)</li><li>Internal vs public labeled; webhooks signed (HMAC, raw body, timestamp, dedup)</li><li>Money = integer minor units + currency; ledgers append-only</li><li>Bulk → async batch + per-item partial-failure results</li><li>Versioned (/v1)</li></ul><h3>20. Recurring anti-patterns</h3><p>These showed up again and again across the 9 designs — check for them:</p><ol><li>Verbs in the path (/create, /book, /shorten) — the method is the verb.</li><li>Query grammar in the path (/events/eventid=) — path segments are positional.</li><li>Offset pagination on live/large data — use opaque cursors.</li><li>Bare arrays as list responses — wrap in an envelope for nextCursor/metadata.</li><li>No idempotency key on writes — double-charge/double-book on retry.</li><li>userId in a header/param instead of from the token — impersonation.</li><li>Secrets/PII/tokens/location in the URL — they get logged. Body/header/short-lived ticket.</li><li>Float money / unit-less amounts — integer minor units + currency.</li><li>DELETE/PATCH on a transaction — ledgers are append-only; reverse with a new entry.</li><li>Client-callable state transitions (POST /transaction/failed/{id}) — use a signed webhook.</li><li>Synchronous “Delivered”/”Created” for async work — return 202 + a status the client polls.</li><li>Channel/type as path segments (/results/sms, /results/whatsapp) — use ?channel= filters.</li><li>Per-chunk / whole-doc transfer for editing or streaming — ops+baseRevision; manifest, not chunks.</li><li>One status code for a whole batch — per-item results with partial failure.</li><li>Exact transactional reads for counts at scale — they’re approximate/cached; say so.</li><li>A WebSocket modeled as REST endpoints — it’s a bidirectional message catalog.</li><li>“Find near me” as a public browse endpoint — usually an internal matching primitive.</li><li>Mutable path as identity (/files/{path}) — a rename breaks references; use a stable id.</li><li>Recipient-less / destination-less writes (a transfer with no “to”, a notification with no user) — every write needs its target.</li><li>A filter={json} blob in the query string — flat, typed params instead.</li></ol><h4>The one-sentence summary</h4><p>A Staff-level API surface is resource-oriented, idempotent on writes, cursor-paginated, typed in its errors, explicit about consistency and async, secure with no secrets in URLs, and honest about who consumes each endpoint — and the hard 80% of any design lives in a <em>few</em> decisions (idempotency, the consistency/conflict model, the async/real-time transport, and the one scale bottleneck), not in the CRUD.</p><h3>Appendix A — The 9 full model answers</h3><p>Complete, endpoint-by-endpoint Staff-level reference solutions for each system. Conventions from earlier sections apply throughout: /v1 versioning, Bearer auth (identity from the token), Idempotency-Key on every unsafe write, opaque cursors with a server-capped limit, and a typed error envelope { error: { code, message, retryable, details } }.</p><h3>A.1 — Event Ticketing</h3><pre>GET  /v1/events?q=&amp;city=&amp;from=&amp;to=&amp;cursor=&amp;limit=50<br>     -&gt; { events:[{id,name,venue,startsAt,minPrice,status}], nextCursor }<br>GET  /v1/events/{eventId}                 -&gt; ETag:&quot;7&quot; { id,name,venue,startsAt,onSaleAt,status,priceTiers[] } | 404<br>GET  /v1/events/{eventId}/seats?status=available&amp;cursor=&amp;limit=<br>     -&gt; { seats:[{seatId,section,row,number,priceTier,status}], nextCursor }     # structured availability<br>GET  /v1/events/{eventId}/venue-map       -&gt; { mapAssetUrl }                      # static, CDN-cached</pre><pre>POST /v1/events/{eventId}/holds   Headers: Auth, Idempotency-Key, X-Queue-Token<br>     Req: { seatIds:[&quot;s1&quot;,&quot;s2&quot;] }<br>     201: { holdId, seatIds, expiresAt, status:&quot;HELD&quot; }                           # pessimistic lock + TTL<br>     409 SEAT_ALREADY_HELD {conflictSeats} | 410 NOT_ON_SALE | 429 + Retry-After<br>POST /v1/orders   Headers: Auth, Idempotency-Key<br>     Req: { holdId, paymentMethodId }<br>     201: { orderId, status:&quot;CONFIRMED&quot;|&quot;PENDING_PAYMENT&quot;, tickets:[{ticketId,seatId,qrToken}], amount }<br>     409 HOLD_EXPIRED | 402 PAYMENT_FAILED                                        # idempotent replay -&gt; same order<br>GET  /v1/orders?status=&amp;cursor=&amp;limit=50   (Bearer; user-scoped) -&gt; { orders:[...], nextCursor }<br>GET  /v1/orders/{orderId}<br>POST /v1/admin/events   Headers: Auth(scope:admin), Idempotency-Key<br>     Req: { name, venueId, startsAt, onSaleAt, priceTiers[] } -&gt; 201 { eventId }<br>PUT  /v1/admin/events/{eventId}/inventory  Req:{ seats:[...] } -&gt; 202 Accepted   # async ingest for big venues<br>POST /v1/events/{eventId}/queue -&gt; { queueToken, position, etaSeconds }          # on-sale admission control</pre><p>Key ideas: a hold is a pessimistic lock with a TTL; the queue token gates write QPS during the on-sale; checkout is idempotent and the PSP makes it async (PENDING_PAYMENT).</p><h3>A.2 — File Storage &amp; Sync</h3><pre># Resumable upload (multi-part session)<br>POST /v1/files   Headers: Auth, Idempotency-Key<br>     Req: { parentId, name, size, contentHash? }<br>     201: { fileId, uploadId, partSize, status:&quot;PENDING&quot; }<br>GET  /v1/uploads/{uploadId}/parts/{n}/url -&gt; { url, expiresAt }                  # pre-signed per part<br>GET  /v1/uploads/{uploadId}/parts         -&gt; { uploadedParts:[{n,etag,size}], partSize }   # RESUME = list parts<br>POST /v1/uploads/{uploadId}/complete  Headers: Idempotency-Key<br>     Req: { parts:[{n,etag}] } -&gt; 201 { fileId, version, status:&quot;COMMITTED&quot; } | 409</pre><pre># Read (pre-signed download + metadata)<br>GET  /v1/files/{fileId}?version=          -&gt; { downloadUrl, expiresAt, size, contentHash } | 403 | 404<br>GET  /v1/folders/{folderId}/children?cursor=&amp;limit= -&gt; { entries:[{id,type,name,version}], nextCursor }<br># Sharing<br>POST   /v1/files/{fileId}/permissions  Req:{ granteeUserId, role:&quot;VIEWER&quot;|&quot;EDITOR&quot; } -&gt; 201 {permissionId}<br>DELETE /v1/files/{fileId}/permissions/{permissionId}<br>POST   /v1/files/{fileId}/links        Req:{ role, expiresAt?, password? } -&gt; 201 { linkUrl, linkId }<br>DELETE /v1/files/{fileId}/links/{linkId}<br># Delta / sync (cursor-based change feed)<br>GET  /v1/delta?cursor=&amp;limit=<br>     -&gt; { changes:[{fileId, path, changeType:&quot;ADDED&quot;|&quot;MODIFIED&quot;|&quot;DELETED&quot;, version}], nextCursor, hasMore }<br># Move / rename / delete<br>PATCH  /v1/files/{fileId}  Req:{ name?, parentId? }<br>DELETE /v1/files/{fileId}                                                         # soft-delete -&gt; trash</pre><p>Key ideas: stable fileId (path is mutable metadata); bytes flow client&lt;-&gt;blob store via pre-signed URLs; resume works because the part list is queryable; If-Match+version for edit conflicts.</p><h3>A.3 — Chat / Messaging</h3><pre># Real-time push (WS) + offline catch-up + history = two cursors<br>POST /v1/realtime/ticket (Bearer) -&gt; { ticket, expiresIn:30 }<br>WSS  /v1/connect?ticket=&lt;short&gt;<br>  C-&gt;S: { type:&quot;subscribe&quot;|&quot;typing&quot;|&quot;ack&quot;|&quot;ping&quot;, id, payload }<br>  S-&gt;C: { type:&quot;message&quot;|&quot;receipt&quot;|&quot;typing&quot;|&quot;ack&quot;|&quot;error&quot;|&quot;pong&quot;, payload:{ ...seq... } }</pre><pre># Send (REST; idempotent; returns the ordering seq)<br>POST /v1/conversations/{cid}/messages   Headers: Auth, Idempotency-Key<br>     Req: { clientMessageId, text|mediaId }<br>     201: { messageId, seq, serverTs, status:&quot;SENT&quot; } | 409 dup clientMessageId -&gt; original<br>GET  /v1/sync?cursor=&amp;limit=                                   # FORWARD: reconnect catch-up (per-device cursor)<br>     -&gt; { events:[{type, conversationId, message|receipt, seq}], nextCursor, hasMore }<br>GET  /v1/conversations/{cid}/messages?before=&lt;seq&gt;&amp;limit=50    # BACKWARD: history scroll-back<br>     -&gt; { messages:[ newest..oldest ], nextCursor }<br>POST /v1/conversations/{cid}/receipts  Req:{ upToSeq, type:&quot;DELIVERED&quot;|&quot;READ&quot; } -&gt; 200<br>POST   /v1/conversations  Req:{ type:&quot;DIRECT&quot;|&quot;GROUP&quot;, name?, memberIds[] } -&gt; 201 {cid}<br>GET    /v1/conversations?cursor=&amp;limit=                        # inbox<br>GET    /v1/conversations/{cid}<br>POST   /v1/conversations/{cid}/members  Req:{ userId, role } -&gt; 201<br>DELETE /v1/conversations/{cid}/members/{userId}<br>DELETE /v1/conversations/{cid}</pre><p>Key ideas: seq is server-assigned, monotonic per conversation (ordering source of truth); WS for live + /sync for reconnect; send returns SENT, delivery/read arrive async as receipts.</p><h3>A.4 — Ride-Sharing / Dispatch</h3><pre>POST /v1/fare-estimates   (Auth; location in BODY, not URL, for privacy)<br>     Req: { pickup:{lat,lng}, dropoff:{lat,lng} } -&gt; { quoteId, fare, surge, etaSec, expireAt }</pre><pre>POST /v1/rides   Headers: Auth, Idempotency-Key<br>     Req: { quoteId, pickup, dropoff } -&gt; 202 { rideId, status:&quot;SEARCHING&quot; }      # match is async (push)<br># Trip state machine (action sub-resources; idempotent + actor-authorized)<br>POST /v1/rides/{rideId}/accept   (driver)        -&gt; 200 | 409 ALREADY_TAKEN<br>POST /v1/rides/{rideId}/arrive   (driver)        -&gt; ARRIVED<br>POST /v1/rides/{rideId}/start    (driver)        -&gt; IN_PROGRESS<br>POST /v1/rides/{rideId}/complete (driver)        -&gt; COMPLETED { fare }<br>POST /v1/rides/{rideId}/cancel   (rider|driver)  Req:{ reason } -&gt; CANCELLED<br>GET  /v1/rides/{rideId}  -&gt; { status, driver:{lat,lng,etaSec}, fare, paymentStatus }   # live-track snapshot<br>GET  /v1/rides?role=rider|driver&amp;cursor=&amp;limit=   (Bearer)                              # history<br># Driver location ingestion (1M+/sec): a WS frame, NOT a fat REST call<br>WS  C-&gt;S { type:&quot;location&quot;, lat, lng, heading, ts }      # -&gt; in-memory geo index (not durable DB)<br># Geo matching is INTERNAL (dispatch service), not a public browse endpoint:<br>#   GeoIndex.queryNearby(pickup, radiusM, filters) -&gt; ranked candidate drivers<br># Optional public map (anonymized, display-only): GET /v1/drivers/nearby?lat=&amp;lng= -&gt; { cars:[{lat,lng}] }</pre><p>Key ideas: quoteId locks the fare; request -&gt; 202 SEARCHING, match via push; location over WS into an in-memory index; transitions are idempotent + actor-authorized; the geo query is internal.</p><h3>A.5 — Payments / Wallet</h3><pre>Money = { amount: 1999, currency: &quot;USD&quot; }            # integer minor units (cents), NEVER a float</pre><pre>POST /v1/accounts  Headers: Auth, Idempotency-Key  Req:{ type, holder, currency } -&gt; 201 { accountId, status }<br>GET  /v1/accounts/{id}/balances -&gt; { balances:[{ currency, available:Money, pending:Money }] }<br>POST /v1/accounts/{id}/topups   Headers: Auth, Idempotency-Key<br>     Req:{ amount:Money, source:{paymentMethodId} } -&gt; 202 { topupId, transactionId, status:&quot;PENDING&quot; }<br>POST /v1/transfers   Headers: Auth, Idempotency-Key                              # atomic, two-sided<br>     Req:{ fromAccountId, toAccountId, amount:Money }<br>     201: { transferId, status:&quot;COMPLETED&quot;|&quot;PENDING&quot;, debitTxnId, creditTxnId }<br>     409 INSUFFICIENT_FUNDS | 422 INVALID_AMOUNT | 423 ACCOUNT_FROZEN<br>POST /v1/accounts/{id}/withdrawals  Headers: Auth, Idempotency-Key<br>     Req:{ amount:Money, destination:{bankAccountId} } -&gt; 202 { withdrawalId, status:&quot;PENDING&quot; }<br>GET  /v1/transactions/{id} -&gt; { id, type, direction, amount:Money, balanceAfter:Money, status,<br>                                relatedTransactionId?, createdAt }                # immutable ledger entry<br>GET  /v1/accounts/{id}/transactions?cursor=&amp;limit= -&gt; { transactions:[...], nextCursor }<br>POST /v1/transactions/{id}/refunds  Headers: Auth, Idempotency-Key<br>     Req:{ amount? } -&gt; 201 { refundId, reversesTransactionId, amount:Money, status }   # NEW reversing entry<br>POST /v1/webhooks/settlement  X-Signature                                        # PSP -&gt; us; NOT client-callable<br>     Body:{ providerRef, transactionId, outcome:&quot;SETTLED&quot;|&quot;FAILED&quot; } -&gt; 200<br>PATCH /v1/accounts/{id}  (scope:admin; If-Match)  Req:{ status:&quot;FROZEN&quot;|&quot;ACTIVE&quot; } -&gt; 200 | 412</pre><p>Key ideas: integer cents + currency; idempotency = correctness (same key -&gt; same txn, no double-debit); ledger is append-only (refund = new reversing entry); state transitions come from the signed webhook, never the client.</p><h3>A.6 — Notification Platform</h3><pre>POST /v1/notifications/batch   Headers: Service-Auth, Idempotency-Key<br>     Req:{ templateId, locale?, category:&quot;TXN&quot;|&quot;MARKETING&quot;, channelStrategy:&quot;FALLBACK&quot;|&quot;ALL&quot;,<br>           sendAt?, dedupKey?,<br>           recipients:[{ userId, channels:[&quot;PUSH&quot;,&quot;EMAIL&quot;], variables:{...} }] }<br>     202: { batchId, accepted, status:&quot;QUEUED&quot; }</pre><pre>GET  /v1/notifications/batches/{batchId} -&gt; { status:&quot;PARTIAL&quot;|&quot;COMPLETED&quot;, counts:{delivered,failed,suppressed} }<br>GET  /v1/notifications/batches/{batchId}/results?channel=&amp;status=&amp;cursor=&amp;limit=  # channel/status are FILTERS<br>     -&gt; { results:[{ notificationId, userId, channel, status:&quot;DELIVERED&quot;|&quot;FAILED&quot;|&quot;SUPPRESSED&quot;, reason }], nextCursor }<br>GET  /v1/notifications/{notificationId}<br>POST /v1/templates  Req:{ name, channel, subject?, body, variables:[...] } -&gt; 201 { templateId, version }<br>GET/PUT /v1/templates/{id}<br>GET   /v1/users/{userId}/preferences -&gt; { channels:{...}, categories:{MARKETING:false}, quietHours }<br>PATCH /v1/users/{userId}/preferences<br>POST  /v1/unsubscribe  Req:{ token }                                             # one-click, signed token<br>POST /v1/webhooks/delivery  X-Signature<br>     Body:{ providerRef, notificationId, event:&quot;DELIVERED&quot;|&quot;OPENED&quot;|&quot;BOUNCED&quot;|&quot;FAILED&quot; } -&gt; 200</pre><p>Key ideas: bulk = async batch + per-recipient partial failure; opt-out enforced server-side (opted-out -&gt; SUPPRESSED, caller can&#39;t override); category matched against preferences; IKey + dedupKey; templated content; signed delivery webhook.</p><h3>A.7 — Collaborative Document Editing</h3><pre># REST control plane<br>POST /v1/docs  Auth, Idempotency-Key  {name} -&gt; 201 { docId, revision:0 }<br>GET  /v1/docs/{docId}?version=               -&gt; { docId, revision, content, permissions } | 404   # content+revision<br>GET  /v1/docs/{docId}/revisions?cursor=&amp;limit= -&gt; { revisions:[{rev,author,ts}], nextCursor }<br>GET  /v1/docs/{docId}/revisions/{rev}        -&gt; { revision, content }<br>GET  /v1/docs/{docId}/diff?from=&amp;to=         -&gt; { ops:[...] }<br>POST /v1/docs/{docId}/revert  {toRevision}   -&gt; { revision }                      # revert = a new revision<br>GET    /v1/docs/{docId}/permissions<br>POST   /v1/docs/{docId}/permissions  {granteeId, role:&quot;VIEW&quot;|&quot;COMMENT&quot;|&quot;EDIT&quot;} -&gt; 201<br>DELETE /v1/docs/{docId}/permissions/{userId}<br>POST /v1/docs/{docId}/comments  {anchor:{startCharId,endCharId}, text} -&gt; 201 {commentId}   # range anchor<br>GET  /v1/docs/{docId}/comments?cursor=</pre><pre># WebSocket live plane (message catalog)<br>POST /v1/realtime/ticket -&gt; { ticket, expiresIn:30 }<br>WSS  /v1/connect?ticket=&lt;short&gt;&amp;docId=&amp;since=&lt;revision&gt;<br>  C-&gt;S { type:&quot;edit&quot;, id, payload:{ baseRevision:7, ops:[{op:&quot;insert&quot;,afterCharId,char,charId},{op:&quot;delete&quot;,charId}] } }<br>  S-&gt;C { type:&quot;edit.ack&quot;, payload:{ clientOpId, committedRevision:10, transformed:true } }   # resolution OUTPUT<br>  S-&gt;C { type:&quot;edit&quot;,     payload:{ revision:10, authorId, ops:[...] } }                      # others&#39; ops<br>  C-&gt;S { type:&quot;cursor&quot;, payload:{ charId } }    S-&gt;C { type:&quot;cursor&quot;|&quot;presence&quot;, ... }</pre><p>Key ideas: edits are ops carrying a baseRevision (not whole-doc PUTs); the server transforms the op (OT) or merges via charIds (CRDT), assigns a monotonic revision, acks + broadcasts -&gt; everyone converges; charIds are stable anchors for comments/cursors; REST control + WS live plane.</p><h3>A.8 — News Feed / Timeline</h3><pre>GET  /v1/feed?cursor=&amp;limit=25<br>     -&gt; { posts:[{postId, authorId, text, mediaUrl, likeCount(~approx), commentCount, likedByMe, createdAt}], nextCursor }<br>     # hybrid fan-out: push-on-write for normal authors; celebrities pulled at read time + merged</pre><pre>POST /v1/posts  Auth, Idempotency-Key, dedupKey  {text, mediaId?, privacy} -&gt; 202 { postId, status:&quot;PUBLISHING&quot; }<br>GET  /v1/posts/{postId}<br>GET  /v1/users/{id}/posts?cursor=&amp;limit=<br># Follow graph (idempotent; paginated -- a celebrity has 100M followers)<br>PUT    /v1/users/{id}/following/{targetId} -&gt; 204<br>DELETE /v1/users/{id}/following/{targetId} -&gt; 204<br>GET    /v1/users/{id}/followers?cursor=&amp;limit= -&gt; { users:[...], nextCursor }<br>GET    /v1/users/{id}/following?cursor=&amp;limit=<br># Engagement (like = idempotent toggle; who-liked; comments)<br>PUT/DELETE /v1/posts/{postId}/likes/me -&gt; 204<br>GET  /v1/posts/{postId}/likes?cursor=&amp;limit= -&gt; { users:[...], nextCursor }<br>POST /v1/posts/{postId}/comments  Idempotency-Key {text} -&gt; 201 {commentId}<br>GET  /v1/posts/{postId}/comments?cursor=&amp;limit=</pre><p>Key ideas: hybrid fan-out + celebrity pull-at-read; POST /posts is 202 (async fan-out); counts are approximate/cached (not transactional); likes idempotent; ranked cursor stable under reranking; paginate every unbounded list.</p><h3>A.9 — Video Platform</h3><pre># Resumable upload -&gt; complete triggers async transcoding<br>POST /v1/videos  Auth, Idempotency-Key  {title} -&gt; 202 { videoId, uploadId, partSize }<br>GET  /v1/uploads/{uploadId}/parts/{n}/url -&gt; { url, expiresAt }<br>GET  /v1/uploads/{uploadId}/parts         -&gt; { uploadedParts:[...] }              # resume = list parts<br>POST /v1/uploads/{uploadId}/complete      -&gt; 202 { videoId, status:&quot;TRANSCODING&quot; }</pre><pre># Status lifecycle + transcoding webhook (transcoder -&gt; us)<br>GET  /v1/videos/{videoId} -&gt; { videoId, status:&quot;UPLOADED&quot;|&quot;TRANSCODING&quot;|&quot;READY&quot;|&quot;FAILED&quot;, title, duration?, viewCount(~approx) }<br>POST /v1/webhooks/transcoding  X-Signature  { providerRef, videoId, outcome:&quot;READY&quot;|&quot;FAILED&quot;, reason? } -&gt; 200<br># Watch = MANIFEST (ABR), gated on READY -- not chunks<br>GET  /v1/videos/{videoId}/manifest -&gt; { manifestUrl(HLS/DASH), renditions:[{height,bitrate}], expiresAt }<br>     409 NOT_READY | 404<br># (the player fetches segments from the CDN via the manifest; the API does NOT expose per-chunk reads)<br># View = client fire-and-forget event (deduped, approximate count)<br>POST /v1/videos/{videoId}/views  { sessionId, positionSec? } -&gt; 202<br># Search + engagement<br>GET  /v1/videos?q=&amp;uploadedAfter=&amp;cursor=&amp;limit= -&gt; { videos:[{videoId,title,thumbnailUrl,duration,viewCount}], nextCursor }<br>PUT/DELETE /v1/videos/{videoId}/likes/me -&gt; 204<br>GET  /v1/videos/{videoId}/likes?cursor=&amp;limit=<br>POST /v1/videos/{videoId}/comments  Idempotency-Key {text} -&gt; 201 {commentId}<br>GET  /v1/videos/{videoId}/comments?cursor=&amp;limit=<br>DELETE /v1/videos/{videoId}</pre><p>Key ideas: the video is an async job with a status lifecycle; complete triggers transcoding; the transcoder webhook flips it to READY; watch returns a manifest (renditions on CDN), gated on READY; views are fire-and-forget with approximate counts.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=db6cc9705148" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Beyond the FIRE Moment: Embracing a Well-Rounded Life]]></title>
            <link>https://medium.com/@nishant.gargvii/beyond-the-fire-moment-embracing-a-well-rounded-life-651688b5decd?source=rss-d18da405456a------2</link>
            <guid isPermaLink="false">https://medium.com/p/651688b5decd</guid>
            <category><![CDATA[fire]]></category>
            <category><![CDATA[finance]]></category>
            <category><![CDATA[happiness]]></category>
            <dc:creator><![CDATA[Nishant Garg]]></dc:creator>
            <pubDate>Tue, 30 May 2023 17:31:13 GMT</pubDate>
            <atom:updated>2023-05-30T17:43:20.867Z</atom:updated>
            <content:encoded><![CDATA[<p>The FIRE (Financial Independence, Retire Early) movement has surged in popularity, advocating for aggressive savings and early retirement. However, it is imperative to dispel the myth that life revolves solely around money. True fulfillment encompasses more than financial security, emphasizing happiness, personal growth, and the significance of work.</p><p>I. The Illusion of Money as the Sole Measure of Success</p><ul><li>Exploring the limitations of equating success with monetary wealth alone.</li><li>Recognizing the multifaceted aspects of a fulfilling life beyond financial abundance.</li></ul><p>II. The Importance of Working and Personal Growth</p><ul><li>Highlighting the significance of meaningful work in fostering personal development and self-worth.</li><li>The sense of purpose and fulfillment that comes from contributing to society and pursuing passions.</li></ul><p>III. The Harsh Reality of Early Retirement:</p><ul><li>When individuals retire early without a clear sense of purpose, they can quickly find themselves adrift. Work often provides structure, goals, and a sense of meaning. Without it, one may experience a profound loss of purpose, leading to feelings of emptiness and a lack of direction.</li><li>Diminished Social Connections: Workplaces serve as social hubs, fostering relationships and connections with colleagues. Early retirement can sever these vital social ties, leaving individuals feeling isolated and disconnected from the vibrant social fabric they were once a part of.</li><li>The human mind thrives on stimulation and challenge. By retiring prematurely, individuals deprive themselves of the mental stimulation that work often provides. This can lead to cognitive decline, decreased mental agility, and a general sense of intellectual stagnation.</li></ul><p>IV. Understanding the Long-Term Nature of Investments and Financial Markets</p><ul><li>Acknowledging that financial strategies, like life itself, require patience, resilience, and a long-term perspective.</li><li>In the late 1980s, Japan experienced an economic bubble, with its stock market reaching unprecedented heights. However, in the following decades, the market underwent a prolonged period of stagnation, known as the “Lost Decades.” This serves as a stark reminder that even strong economies can experience prolonged periods of underperformance in the stock market.</li><li>China’s stock market has witnessed significant volatility over the years. For instance, in 2015, the Chinese stock market experienced a sharp crash, eroding trillions of dollars in market value. It exemplifies how even emerging markets can be subject to unpredictable fluctuations, highlighting the need for a long-term perspective in investments.</li></ul><p>Conclusion:</p><p>As a software engineer, I’ve come to understand that the pursuit of financial independence shouldn’t consume our entire lives. Sure, it’s important to work hard and secure our financial future, but we must also remember to let loose and enjoy the journey.</p><p>Life isn’t just about money. It’s about the moments we create, the relationships we build, and the personal growth we experience. So, while we grind away at our keyboards, let’s not forget to take breaks and immerse ourselves in the richness of life outside of work.</p><p>Work hard, party hard, as they say. Find a balance that allows you to excel in your career while also enjoying the fruits of your labor. Seek opportunities for personal growth, whether it’s learning new skills, pursuing hobbies, or exploring the world around you. True success lies in finding happiness and fulfillment in both your professional and personal life.</p><p>Remember, the true wealth of life is measured by the memories we make, the connections we forge, and the person we become along the way. So, embrace a balanced approach, work passionately, and savor the moments that make life truly worth living.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=651688b5decd" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>