<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Sujal Charati on Medium]]></title>
        <description><![CDATA[Stories by Sujal Charati on Medium]]></description>
        <link>https://medium.com/@sujalc024?source=rss-817bdde51adb------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*xzrzd1FQDXbcozd_</url>
            <title>Stories by Sujal Charati on Medium</title>
            <link>https://medium.com/@sujalc024?source=rss-817bdde51adb------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 21:58:23 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@sujalc024/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Building a BPE Tokenizer from Scratch in JavaScript]]></title>
            <link>https://medium.com/@sujalc024/building-a-bpe-tokenizer-from-scratch-in-javascript-4245e96f4671?source=rss-817bdde51adb------2</link>
            <guid isPermaLink="false">https://medium.com/p/4245e96f4671</guid>
            <category><![CDATA[tokenizer]]></category>
            <category><![CDATA[encoding]]></category>
            <category><![CDATA[byte-pair-encoding]]></category>
            <category><![CDATA[token]]></category>
            <category><![CDATA[llm]]></category>
            <dc:creator><![CDATA[Sujal Charati]]></dc:creator>
            <pubDate>Fri, 16 Oct 2026 08:00:56 GMT</pubDate>
            <atom:updated>2026-10-16T08:00:56.181Z</atom:updated>
            <content:encoded><![CDATA[<p>How GPT-style models turn text into numbers in about 120 lines of code.</p><p>So, before going deep dive into this, let&#39;s understand why behind this tokenizer?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*CnsohGcn70dnTjTCJOXFaw.png" /><figcaption>Byte Pair Encoding</figcaption></figure><p>language models at its core are neural networks which can only consider numbers as its input and do computation on top of it.</p><p>There are two obvious approaches, and both have problems:</p><ul><li>Character-level. The vocabulary is tiny, but sequences get very long, and the model has to relearn that “t”, “h”, “e” often go together.</li><li>Word-level. Sequences are short, but the vocabulary is huge, and any word the model hasn’t seen (a typo, a name, an emoji) breaks it.</li></ul><p>Byte Pair Encoding (BPE) sits between the two. It starts with raw bytes and learns which pairs occur most often. It then merges those pairs into single tokens, so common words become one token and rare words split into smaller pieces. Nothing is ever “unknown”, because everything can fall back to bytes.</p><p>The idea at a high level:</p><p>1. Convert the text to UTF-8 bytes. Your starting vocabulary is the 256 possible byte values.<br>2. Count every adjacent pair of tokens.<br>3. Take the most frequent pair, give it a new ID (256, then 257, and so on), and replace every occurrence with that ID.<br>4. Repeat until you reach your target vocabulary size.</p><p>The ordered list of merges is the tokenizer. That’s all there is to it.</p><h4>Step 1: Counting pairs</h4><pre>function getPairCounts(id) {<br>  const adjacentmaps = new Map();<br>  for (let i = 0; i &lt; id.length - 1; i++) {<br>    const key = `${id[i]},${id[i + 1]}`;<br>    adjacentmaps.set(key, (adjacentmaps.get(key) ?? 0) + 1);<br>  }<br>  return adjacentmaps;<br>}</pre><p>JavaScript Map keys compare arrays by reference, so I turn each pair into a string like “97,116”. It’s a small trick that makes lookups easy.</p><p><strong>Step 2: Merging a pair</strong></p><pre>function merge(id, pair, newid) {<br>  const out = [];<br>  let i = 0;<br>  while (i &lt; id.length) {<br>    if (i &lt; id.length - 1 &amp;&amp; id[i] === pair[0] &amp;&amp; id[i + 1] === pair[1]) {<br>      out.push(newid);<br>      i += 2;<br>    } else {<br>      out.push(id[i]);<br>      i += 1;<br>    }<br>  }<br>  return out;<br>}</pre><p>Walk the list left to right. When you see the pair, emit the new ID and skip both items. Otherwise copy the item through.</p><p><strong>Step 3: Training</strong></p><pre>function train(text, vocabSize) {<br>  const ids = Array.from(new TextEncoder().encode(text));<br>  const merges = new Map();<br>  let current = ids;<br>  let newid = 256;<br><br>  for (let i = 0; i &lt; vocabSize - 256; i++) {<br>    const counts = getPairCounts(current);<br>    const sorted = Array.from(counts).sort((a, b) =&gt; b[1] - a[1]);<br>    if (sorted.length === 0 || sorted[0][1] &lt; 2) break;<br><br>    const pair = sorted[0][0].split(&quot;,&quot;).map(Number);<br>    current = merge(current, pair, newid);<br>    merges.set(sorted[0][0], newid);<br>    newid++;<br>  }<br>  return merges;<br>}</pre><pre>Each round finds the most common pair, merges it, and records the merge.<br>The loop stops early if no pair appears at least twice, because merging<br>a pair that occurs once saves nothing.<br><br>Training on &quot;the cat sat on the mat the cat is fat&quot; (37 bytes) gives this:<br><br>round 1: merged [97,116]  -&gt; 256, length 32     // &quot;a&quot;+&quot;t&quot; → &quot;at&quot;<br>round 2: merged [256,32]  -&gt; 257, length 28     // &quot;at&quot;+&quot; &quot; → &quot;at &quot;<br>round 3: merged [116,104] -&gt; 258, length 25     // &quot;t&quot;+&quot;h&quot; → &quot;th&quot;<br>round 4: merged [258,101] -&gt; 259, length 22     // &quot;th&quot;+&quot;e&quot; → &quot;the&quot;<br>round 5: merged [259,32]  -&gt; 260, length 19     // &quot;the&quot;+&quot; &quot; → &quot;the &quot;<br>round 6: merged [260,99]  -&gt; 261, length 17     // &quot;the &quot;+&quot;c&quot; → &quot;the c&quot;<br>round 7: merged [261,257] -&gt; 262, length 15     // &quot;the c&quot;+&quot;at &quot; → &quot;the cat &quot;<br><br>You can see the vocabulary building up: first at, then at ,<br>then th → the → the . By round 7 the phrase &quot;the cat &quot; is a single token. <br>The text shrank from 37 tokens to 15.</pre><p><strong>Step 4: Encoding new text</strong></p><p>Training only learns the merges. To tokenize new text, you replay them in the order they were learned.</p><pre>function encode(text, merges) {<br>  let ids = Array.from(new TextEncoder().encode(text));<br><br>  while (ids.length &gt;= 2) {<br>    const pairs = getPairCounts(ids);<br>    let bestkey = &quot;&quot;, bestid = Infinity;<br><br>    for (const key of pairs.keys()) {<br>      if (merges.has(key) &amp;&amp; merges.get(key) &lt; bestid) {<br>        bestkey = key;<br>        bestid = merges.get(key);<br>      }<br>    }<br>    if (bestkey === &quot;&quot;) break;<br><br>    ids = merge(ids, bestkey.split(&quot;,&quot;).map(Number), bestid);<br>  }<br>  return ids;<br>}</pre><p>At each step, look at the pairs in the current sequence and pick the one with the lowest merge ID, meaning the one learned earliest. Apply it and repeat until nothing more can merge. Following the training order is what makes encoding deterministic and consistent with training.</p><p><strong>Step 5: Decoding</strong></p><p>Decoding is the reverse. Build a lookup table from each token ID to the bytes it stands for, then concatenate and decode as UTF-8.</p><pre>function decode(ids, merges) {<br>  const vocab = new Map();<br>  for (let i = 0; i &lt; 256; i++) vocab.set(i, [i]);<br><br>  for (const [key, newid] of merges) {<br>    const [a, b] = key.split(&quot;,&quot;).map(Number);<br>    vocab.set(newid, [...vocab.get(a), ...vocab.get(b)]);<br>  }<br><br>  const bytes = [];<br>  for (const id of ids) bytes.push(...vocab.get(id));<br>  return new TextDecoder().decode(new Uint8Array(bytes));<br>}</pre><p>Because merges are stored in order, each new token is built from two tokens that already exist in the table. A single pass is enough.</p><p>Trying it out</p><pre>const sentence = &quot;the cat sat on the mat the cat is fat&quot;;<br>const merges = train(sentence, 266);<br><br>console.log(encode(sentence, merges));<br>// [262,115,257,111,110,32,260,109,257,262,105,115,32,102,256]<br><br>console.log(decode(encode(&quot;the cat sat&quot;, merges), merges));<br>// &quot;the cat sat&quot;</pre><p>The round trip works, and the 37-byte sentence is now 15 tokens.</p><p><strong>Limitations</strong></p><p>This is a teaching implementation. Real tokenizers like GPT-2’s or tiktoken differ in a few ways:</p><ul><li>Pre-tokenization. Real tokenizers first split text with a regex (words, numbers, punctuation) so merges never cross word boundaries. Here, “the cat “ became one token, which production tokenizers avoid.</li><li>Speed. Re-counting every pair each round is O(n × merges). Real implementations update counts incrementally.</li><li>Training data. One sentence is a toy. Real vocabularies train on gigabytes of text and have 50k–200k merges.</li></ul><p><strong>Takeaways</strong></p><ul><li>A tokenizer is just an ordered list of merges.</li><li>Starting from bytes means there’s no “unknown token” problem, and any string round-trips.</li><li>The core algorithm is small. The hard parts in production are speed and good pre-tokenization.</li></ul><p>you can refer to the whole code here: <a href="https://github.com/sujalcharati/bpt-tokenizer/blob/main/main.js">https://github.com/sujalcharati/bpt-tokenizer/blob/main/main.js</a> and refer <a href="https://github.com/karpathy/minbpe">https://github.com/karpathy/minbpe</a> , connect me here : <a href="https://github.com/sujalcharati">https://github.com/sujalcharati</a> , <a href="https://x.com/suj25887">https://x.com/suj25887</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4245e96f4671" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to make your website’s LCP under 2.5s]]></title>
            <link>https://medium.com/@sujalc024/how-to-make-your-websites-lcp-under-2-5s-792f4a461d23?source=rss-817bdde51adb------2</link>
            <guid isPermaLink="false">https://medium.com/p/792f4a461d23</guid>
            <category><![CDATA[google-core-web-vitals]]></category>
            <category><![CDATA[loading-optimization]]></category>
            <category><![CDATA[website-performance]]></category>
            <category><![CDATA[seo]]></category>
            <category><![CDATA[user-experience]]></category>
            <dc:creator><![CDATA[Sujal Charati]]></dc:creator>
            <pubDate>Thu, 01 Oct 2026 07:47:07 GMT</pubDate>
            <atom:updated>2026-10-01T07:47:07.031Z</atom:updated>
            <content:encoded><![CDATA[<p>Before diving deep into how I managed to reduce the metrics in my codebase (mobile view of website ) lets see at first what is LCP and why we even need it.</p><p><strong>What is it?</strong></p><p>LCP is the time from when the user starts navigating to when the largest image or text block in the viewport finishes rendering. That element is usually a hero image, a big headline, or a video poster. The browser reports it through the PerformanceObserver API, using the largest-contentful-paint entry type.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gGLgu7_ib3RpBb1GauzxCA.png" /><figcaption>src : <a href="https://web.dev/articles/lcp">https://web.dev/articles/lcp</a></figcaption></figure><p><strong>Why do we even need this?</strong></p><p>A page doesn’t load at a single moment, and for years the metrics we used, like <em>DOMContentLoaded</em> and <em>window. onload</em>, measured what the browser was doing rather than what the user could see. A page could fire load with a blank hero, or look finished long before load because a tracking script was still downloading. Users judge speed by one thing: when the main content shows up. Earlier attempts to guess that moment, like First Meaningful Paint and Speed Index, were inconsistent and hard to explain. So Google introduced <em>Largest Contentful Paint</em>, which uses a simple rule the browser can compute: the time until the largest element in the viewport is painted. <strong>A shared, measurable number matters because slow loads raise bounce rates, hurt conversions, and affect search ranking, and for paid traffic it means ad clicks that were paid for and never seen.</strong></p><p>Below is a before and after image / metrices that demonstrate the optimization in the Core Web Vitals.</p><p><strong>before :(</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/951/1*xOtDrv3MM2ILJSf5gU9b7A.png" /></figure><p><strong>after :)</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/505/1*9s7WdCkpKtmd7mUv3ND62g.png" /></figure><p>Some factors that helped me to achieve this are :</p><ol><li><strong>Identifying the real LCP element</strong>. I used a PerformanceObserver on the live site and found the LCP element was the navbar logo, not the hero image I had assumed. Two earlier fixes had barely moved the number because they targeted the wrong element.</li></ol><p>2. <strong>Removing loading=” lazy”</strong> from the logo. An above-the-fold image marked lazy delays its discovery and fetch. <strong>Adding priority and fetchPriority=” high” fixed that</strong>.</p><p>3. Making the <strong>hero image discoverable early</strong>. I added a <strong>static &lt;link rel=” preload” as=” image” fetchpriority=” high”&gt; </strong>in the server-rendered HTML. The browser’s preload scanner finds it before any JavaScript runs, instead of after hydration.</p><p>4. <strong>Serving right-sized images per device</strong>. Mobile now gets a w_828 image instead of the 1920px desktop file. Mobile and desktop layers are toggled by CSS, because browsers skip fetching display: none elements.</p><p>5. Using <strong>modern image formats</strong>. Moving to AVIF and WebP, and lowering quality where the image is mostly covered by an overlay, cut file sizes sharply. One hero went from 22KB to 8KB.</p><p>6. <strong>Deferring third-party scripts. GA, Meta Pixel, and Clarity</strong> load only after the page is visible or on first interaction. This kept about 300KB of scripts out of the LCP window.</p><p>7. Code-splitting below-the-fold sections. Using <strong>next/dynamic stopped the main thread from processing </strong>every homepage section before painting the hero.</p><p>8. <strong>Cutting main-thread and bandwidth waste</strong>. I removed the 52KB GIF on mobile, compressed the videos, moved the marquee animation from React state to direct DOM updates, and throttled the navbar scroll handler with requestAnimationFrame.</p><p><strong>Conclusion</strong></p><p>Getting LCP from 4.2s to under 2s didn’t come from one clever trick. It came from measuring first, and from making the browser’s job easier. The biggest lesson was that my assumption was wrong. I spent time optimizing the hero image, and the real LCP element was the navbar logo. A PerformanceObserver on the live site showed that in minutes.</p><p>After that, most of the gains came from the same idea: don’t make the browser wait. Discover the critical image early, fetch it at high priority, serve the right size in a modern format, and keep non-essential JavaScript and third-party scripts out of the way until the page has painted.</p><p>A few things I’d take away:</p><ul><li>Measure before you optimize. Find the real LCP element instead of guessing.</li><li>Discovery and priority matter more than bytes. A small image that starts late loses to a larger one that starts early.</li><li>Lab scores aren’t the whole story. Track field data from real users, because that is what your visitors actually experience ( I used Google Analytics 4 for this: the browser’s PerformanceObserver API measured LCP, FCP, CLS, INP and TTFB on each visitor’s device, and a small script sent them to GA4 as custom events. Looking at real sessions, including visitors on slow mobile connections who arrive from Instagram and Meta ads, showed me what people actually experienced, which a lab test on a fast machine can’t).</li><li>Not every fix sticks. Some changes, like inlining CSS, helped one page and hurt another, so test each one.</li></ul><p>Performance work is never fully done. New sections, scripts, and images can undo these gains, so I now treat LCP as something to keep monitoring rather than a one-time fix.</p><p>If you’re working on a similar problem or have a different approach, I’d like to hear about it.</p><p>Connect with me here:</p><ul><li>GitHub: <a href="https://github.com/sujalcharati">https://github.com/sujalcharati</a></li><li>X (Twitter): <a href="https://x.com/suj25887">https://x.com/suj25887</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=792f4a461d23" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[I Found Out My DNS Server Was Myself — So I Traced the Whole Thing]]></title>
            <link>https://medium.com/@sujalc024/i-found-out-my-dns-server-was-myself-so-i-traced-the-whole-thing-9bab4b6623d8?source=rss-817bdde51adb------2</link>
            <guid isPermaLink="false">https://medium.com/p/9bab4b6623d8</guid>
            <dc:creator><![CDATA[Sujal Charati]]></dc:creator>
            <pubDate>Sun, 21 Jun 2026 17:37:16 GMT</pubDate>
            <atom:updated>2026-06-21T17:37:16.488Z</atom:updated>
            <content:encoded><![CDATA[<p>I was on my phone’s hotspot, poking around my laptop’s network config for no good reason, when I ran this:</p><pre>$ cat /etc/resolv.conf<br># This is /run/systemd/resolve/stub-resolv.conf managed by man:systemd-resolved(8).<br>nameserver 127.0.0.53<br>options edns0 trust-ad<br>search .</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/854/1*_FebuStURDkQesown2JJSg.png" /></figure><p>127.0.0.53. Loopback. My DNS server was… <em>me</em>. My own machine.</p><p>I’d used DNS for years. I could say “it turns google.com into an IP” in an interview. But I could not have told you why my resolver was a loopback address, or what actually happens between me hitting Enter and a server somewhere answering. So I went down the rabbit hole — and traced the entire path on my own laptop, on a phone hotspot, with real commands.</p><p>This is that trace. We go from the browser to the final IP and we <strong>stop there</strong>. No TLS, no HTTP, no CDN — those come after we have the IP, and they’re a separate story. Here’s the chain:</p><ol><li>Browser cache</li><li>The OS resolver (systemd-resolved on 127.0.0.53)</li><li>The gateway (my phone hotspot)</li><li>The ISP / carrier resolver</li><li>Root servers</li><li>TLD servers</li><li>Authoritative servers</li><li>The answer comes back — and gets cached everywhere</li></ol><h3>1. The browser never asks if it already knows</h3><p>Before any network machinery wakes up, the browser checks itself. Chrome keeps its own in-memory DNS cache — you can see it at chrome://net-internals/#dns. If you loaded google.com thirty seconds ago, the hostname → IP mapping is probably still sitting there and the lookup ends before it starts.</p><p>This is the cheapest possible answer: no syscall, no socket, nothing. It’s also the first liar you’ll meet when debugging “why isn’t my DNS change taking effect” — the browser is happily serving you a stale copy. Everything below only runs on a <strong>miss</strong>.</p><h3>2. 127.0.0.53 — my machine is the DNS server (sort of)</h3><p>On a miss, the browser asks the OS to resolve the name (the classic getaddrinfo() call). The OS reads /etc/resolv.conf to find out <em>who</em> to ask — and on a modern systemd Linux box, that file points at itself: nameserver 127.0.0.53.</p><p>That loopback address isn’t a real DNS server out on the internet. It’s systemd-resolved, a local daemon — a <em>stub resolver</em> — listening on 127.0.0.53:53. So the surprise finally made sense: my machine isn&#39;t <em>answering</em> DNS from any real knowledge. It&#39;s a local middleman that caches answers and forwards the misses upstream. It&#39;s the front desk, not the destination.</p><p>To see what it’s actually doing, ask it directly:</p><pre>$ resolvectl status<br>Global<br>       Protocols: -LLMNR -mDNS -DNSOverTLS DNSSEC=no/unsupported<br>Link 2 (wlan0)<br>    Current Scopes: DNS<br>         Protocols: +DefaultRoute +DNS ...<br>       DNS Servers: 192.168.43.1</pre><p>The lines that matter:</p><ul><li><strong>Current Scopes: DNS</strong> and the <strong>per-link section</strong> (Link 2 (wlan0)) — resolved tracks DNS <em>per interface</em>. My hotspot link has its own server. This is exactly why VPNs and split-DNS work.</li><li><strong>DNS Servers</strong> — the upstream resolved forwards to. Here it&#39;s 192.168.43.1. That&#39;s not a public DNS server. That&#39;s my phone.</li></ul><p>So the real shape of step 2: browser → getaddrinfo → systemd-resolved at 127.0.0.53 → checks its own cache → on a miss, forwards to whatever upstream the network handed it. The loopback was never where the question was going.</p><h3>3. The gateway — where “upstream” actually points</h3><p>Where did resolved get 192.168.43.1? It didn&#39;t pick it. When my laptop joined the hotspot, DHCP handed it an IP, a default gateway, and one or more DNS server addresses — and on a hotspot, that DNS address usually <em>is</em> the gateway.</p><p>You can see the gateway directly:</p><pre>$ ip route | grep default<br>default via 192.168.43.1 dev wlan0 proto dhcp metric 600</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/798/1*MZ-QgbRI9G0197vvmt-64g.png" /></figure><p>That default via 192.168.43.1 is the next hop for everything leaving my machine — and on a hotspot, it&#39;s the phone. The phone is playing router. Most phones and home routers run a tiny DNS forwarder (dnsmasq and friends): they take your query, check a small local cache, and on a miss forward it to <em>their</em> upstream — the carrier/ISP resolver.</p><p>This is the layer everyone skips on the whiteboard, and it’s another cache, and another place a stale answer can come from. A shocking amount of “DNS is broken on my network” actually ends right here, on the little forwarder, not anywhere out on the internet. Important detail: the gateway does <strong>not</strong> do the full recursive walk itself. It forwards. The heavy lifting is one layer up.</p><h3>4. The ISP / carrier resolver — the one that actually works</h3><p>Now we hit the <strong>recursive resolver</strong> — the first machine in the chain that, on a miss, will go out and walk the entire DNS tree on your behalf. On hotspot that’s my carrier’s resolver; on home broadband it’s the ISP’s. (Or 1.1.1.1 / 8.8.8.8 if you&#39;ve overridden it — same role, different operator.)</p><p>Everything before this was caches hoping to dodge work. This is the machine that <em>does</em> the work, then holds the answer for its TTL so the next subscriber asking for google.com gets it instantly.</p><p>You can stand in for it and watch a single query resolve:</p><pre>$ dig @1.1.1.1 google.com</pre><pre>;; ANSWER SECTION:<br>google.com.		146	IN	A	142.250.193.110</pre><pre>;; Query time: 18 msec<br>;; SERVER: 1.1.1.1#53(1.1.1.1)</pre><p>Read the <strong>ANSWER SECTION</strong> — that&#39;s the IP, with a TTL (146) counting down. <strong>Query time</strong> tells you if this was a cache hit or a real walk. Run it twice: the TTL drops and the query time collapses on the second run. That&#39;s caching, live.</p><p>But @1.1.1.1 hides the journey — it just hands you the final answer. To <em>see</em> the walk, you trace it.</p><h3>5. Root servers — the top of the tree</h3><pre>$ dig google.com +trace</pre><pre>.			IN	NS	a.root-servers.net.<br>.			IN	NS	b.root-servers.net.<br>...<br>;; Received 811 bytes from 127.0.0.53#53(127.0.0.53) in 4 ms</pre><p>+trace tells dig to do the recursion <em>itself</em>, hop by hop, and print every step. The first block is the <strong>root</strong>.</p><p>There are <strong>13 root server identities</strong> — a.root-servers.net through m.root-servers.net. Not 13 machines: 13 <em>names</em>. Behind them are 1,800+ physical servers worldwide, all reachable by <strong>anycast</strong> — the same IP announced from many locations at once, with the network routing you to the nearest one. That&#39;s how 13 addresses serve the whole internet without melting.</p><p>The root knows almost nothing, on purpose. It doesn’t know where google.com lives. It knows <em>who is responsible for </em><em>.com</em>. So this block answers one question — &quot;where do I ask about .com?&quot; — and hands back the .com TLD nameservers. The root is a switchboard that only knows which department handles each top-level domain, then gets out of the way.</p><h3>6. TLD servers — the .com department</h3><p>The next block in the trace is the <strong>TLD</strong> level. The resolver took the root’s referral and asked a .com nameserver about google.com:</p><pre>com.			IN	NS	a.gtld-servers.net.<br>com.			IN	NS	b.gtld-servers.net.<br>...<br>;; Received 1174 bytes from 192.5.6.30#53(a.root-servers.net) in 12 ms</pre><p>.com is run by <strong>Verisign</strong>. .in is run by <strong>NIXI</strong>. Every TLD has an operator running its nameservers, and they&#39;re anycast too. Like the root, the TLD servers don&#39;t know google.com&#39;s IP — they know the next referral: <em>here are the authoritative nameservers for </em><em>google.com</em> (the ns1.google.com-style NS records).</p><p>So we’ve narrowed twice. Root said “ask the .com people.&quot; .com said &quot;ask Google&#39;s own nameservers.&quot; Each level knows exactly one thing: who&#39;s responsible for the level below it.</p><h3>7. Authoritative servers — where the IP actually lives</h3><p>The final block. The resolver asks one of google.com&#39;s <strong>authoritative</strong> nameservers, and this one doesn&#39;t refer you anywhere — it <em>answers</em>:</p><pre>google.com.		300	IN	A	142.250.183.206<br>;; Received 55 bytes from 216.239.34.10#53(ns2.google.com) in 24 ms</pre><p>That A record is the real IP, straight from the zone Google publishes. This is the only server in the entire chain that actually <em>knows</em> the answer instead of knowing who to ask. Everyone upstream was just narrowing the search.</p><p>Notice this IP can differ from what dig @1.1.1.1 gave back in step 4 — different anycast/geo answers, different moments in time. Same name, different valid answer depending on who&#39;s asking and from where. DNS isn&#39;t a static phone book; it&#39;s a question answered fresh per requester.</p><p>And that’s the whole thing. Root → TLD → authoritative. Three referrals, one answer — the foundation of the entire internet, fitting in one command’s output.</p><h3>8. The answer comes back — and everyone keeps a copy</h3><p>The IP doesn’t just go to you. On the way back, every layer that forwarded the question caches it, each honoring the record’s TTL:</p><ul><li>The <strong>recursive resolver</strong> (carrier/ISP) caches it — the next subscriber gets it instantly.</li><li>The <strong>gateway/hotspot</strong> forwarder may cache it.</li><li><strong>systemd-resolved</strong> caches it locally (resolvectl statistics shows the hit rate).</li><li>The <strong>browser</strong> caches it in memory.</li></ul><p>So the full walk you just watched in +trace happens <em>rarely</em>. The first person to ask pays for the trip; everyone behind them rides the cache until the TTL expires. That&#39;s the whole reason DNS scales to millions of queries a second — the tree is the cold path, and caches are the warm path you actually hit almost every time. That TTL ticking down in your dig output? It&#39;s the timer on every one of those copies.</p><p>you can connect with me here : <a href="https://github.com/sujalcharati">https://github.com/sujalcharati</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9bab4b6623d8" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Idempotency Keys : Why Your API Needs Them]]></title>
            <link>https://medium.com/@sujalc024/idempotency-keys-why-your-api-needs-them-379c2202096c?source=rss-817bdde51adb------2</link>
            <guid isPermaLink="false">https://medium.com/p/379c2202096c</guid>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[backend-development]]></category>
            <category><![CDATA[api-design]]></category>
            <category><![CDATA[nodejs]]></category>
            <category><![CDATA[system-design-interview]]></category>
            <dc:creator><![CDATA[Sujal Charati]]></dc:creator>
            <pubDate>Thu, 02 Apr 2026 17:10:09 GMT</pubDate>
            <atom:updated>2026-04-02T17:10:09.787Z</atom:updated>
            <content:encoded><![CDATA[<p>Have you ever clicked the “Pay Now” button and the page froze? Your first instinct — click it again. And again. Now imagine each click actually charged your card. That’s exactly the problem <strong>idempotency keys</strong> solve.</p><h3><strong>What is Idempotency?</strong></h3><p>In simple terms: <strong>doing something once or doing it 10 times should give the same result.</strong></p><p>Think of it like a light switch. You flip it ON — the light turns on. You flip it ON again — nothing changes. It’s already on. That’s idempotent behavior.</p><p>In APIs, a <strong>GET</strong> request is naturally idempotent — fetching data doesn’t change anything. But <strong>POST</strong> requests? They create things. A second POST to /Payments could mean a second charge. That’s a real problem.</p><h3><strong>The Real-World Problem</strong></h3><p>Let’s say a user is paying Rs. 500 on your app:</p><p>1. User clicks “Pay”</p><p>2. Request goes to the server</p><p>3. Network is slow — user sees no response</p><p>4. User clicks “Pay” again</p><p>5. <strong>Both requests hit the server</strong></p><p>6. User gets charged Rs. 1000 instead of Rs. 500</p><p>This isn’t a rare edge case. It happens due to:</p><ul><li>Network retries</li><li>User impatience (double clicks)</li><li>Mobile app background refresh</li><li>Webhook retries from payment gateways</li></ul><p>Companies like <strong>Stripe, Razorpay, and PayPal </strong>all handle this — and the solution is idempotency keys.</p><h3><strong>The Solution: Idempotency Keys</strong></h3><p>The idea is simple:</p><p>1. The <strong>client generates a unique key</strong> for each intended action</p><p>2. This key is sent in the request header</p><p>3. The <strong>server checks</strong>: “Have I processed this key before?”</p><ul><li><strong>Yes</strong> → Return the cached response (no duplicate processing)</li><li><strong>No</strong> → Process the request and store the result against this key</li></ul><p>Let’s build this from scratch.</p><h3><strong>Let’s Build It : Step by Step</strong></h3><h4><strong>Setting Up the Server</strong></h4><pre>import express from “express”;<br>const app = express();<br>const port = 3000;<br>app.use(express.json());</pre><p>Nothing fancy: a basic Express server.</p><p><strong>In-Memory Storage</strong></p><pre>const payments = {};<br>const idempotencyKeys = {};</pre><p>We’re using two objects:</p><ul><li>Payments: stores all processed payments</li><li>idempotencyKeys: maps each idempotency key to its request hash and response</li></ul><p>&gt; In production, you’d use Redis or a database. In-memory works fine for understanding the concept.</p><p><strong>Generating a Unique Payment ID</strong></p><pre>const generateUniqueID = () =&gt; {<br>const val = Math.floor(Math.random() * 1000000);<br>return `pay_${val}`;<br>};</pre><p>Each payment gets a unique ID like pay_482910. In production, you’d use UUIDs.</p><p><strong>The Payment Endpoint — Where the Magic Happens</strong></p><pre>app.post(&quot;/payments&quot;, (req, res) =&gt; {</pre><p><strong>Step 1: Extract the Idempotency Key</strong></p><pre>const key = req.headers[“idempotency-key”];<br><br>if (!key) {<br>return res.status(400).json({ error: &quot;idempotency key is not found!&quot; });<br>}</pre><p>The client <strong>must</strong> send an idempotency-key header. No key? Reject the request.</p><p><strong>Step 2: Validate the Request Body</strong></p><pre><br><br>const { userId, amount } = req.body;<br><br>if (typeof amount != &quot;number&quot; || amount &lt;= 0 || !amount) {<br>return res.status(400).json({ error: &quot;amount is not found or invalid!&quot; });<br>}<br><br>if (!userId) {<br>return res.status(400).json({ error: &quot;userId is not found!&quot; });<br>}</pre><p>Basic validation : we need a valid userIdand a positive amount.</p><p><strong>Step 3: Check if the Key Already Exists (The Core Logic)</strong></p><pre>if (key in idempotencyKeys) {<br> const currentHash = JSON.stringify({ userId, amount });<br><br>if (idempotencyKeys[key].requestHash != currentHash) {<br>return res.status(409).json({ error: &quot;conflict in orders keys..&quot; });<br>}<br><br>return res.status(200).json(idempotencyKeys[key].response);<br>}</pre><p>This is the heart of idempotency. Two things happen here:</p><p><strong>Case A — Same key, same data:</strong> The user (or network) retried the exact same request. We return the <strong>cached response</strong>. No duplicate payment.</p><p><strong>Case B — Same key, different data:</strong> Someone is reusing a key for a different request. That’s a <strong>conflict</strong>— we return 409. This prevents misuse of keys.</p><p>The trick is the <strong>request hash</strong>— we stringify {userId, amount} and compare it. If the hash matches, it’s a retry. If it doesn’t, it’s a conflict.</p><p><strong>Step 4: Process the Payment (First Time Only)</strong></p><pre>const paymentId = generateUniqueID();<br><br>const payment = {<br>paymentId,<br>userId,<br>amount,<br>status: &quot;pending&quot;<br>};<br><br>payments[paymentId] = payment;<br>payment.status = &quot;success&quot;;<br>const resultdata = { amount, paymentId, status: payment.status };</pre><p><strong>Step 5: Cache the Response Against the Key</strong></p><pre>idempotencyKeys[key] = {<br>requestHash: JSON.stringify({ userId, amount }),<br>response: resultdata<br>};<br>return res.status(200).json(resultdata);<br>});</pre><p>We store:</p><ul><li>requestHash— to verify future requests with the same key</li><li>response — to return instantly on retries</li></ul><p><strong>Bonus: Fetching a Payment</strong></p><pre>app.get(“/payments/:id”, (req, res) =&gt; {<br><br>const { id } = req.params;<br><br>if (payments[id]) {<br>return res.status(200).json(payments[id]);<br>}<br><br>return res.status(404).json({ message: &quot;Payment id is not found!&quot; });<br>});</pre><p>A simple GET endpoint to verify a payment exists. Since GET is naturally idempotent, no special handling needed.</p><p><strong>Key Takeaways</strong></p><p>1. <strong>Always use idempotency keys for POST endpoints with side effects</strong> : payments, orders, bookings</p><p>2. <strong>Hash the request body</strong> : to detect retries vs. conflicts</p><p>3. <strong>Return cached responses</strong> : don’t reprocess, just replay</p><p>4. <strong>Add expiry in production: </strong>keys shouldn’t live forever (Stripe uses 24 hours)</p><p>5. <strong>Use a persistent store</strong> : Redis or database, not in-memory</p><p><strong>What’s Next?</strong></p><p>This implementation is a great starting point. To make it production-ready, you’d want to:</p><ul><li>Use <strong>Redis</strong> for storing idempotency keys (with TTL/expiry)</li><li>Use <strong>UUIDs</strong> instead of random numbers for payment IDs</li><li>Add <strong>database transactions</strong> so payment creation is atomic</li><li>Handle <strong>race conditions</strong> with distributed locks</li></ul><p><em>If you found this helpful, follow me for more backend engineering deep-dives. Drop a comment if you’ve dealt with duplicate payment bugs before — I’d love to hear your war stories.You can connect with me in github: </em><a href="https://github.com/sujalcharati">https://github.com/sujalcharati</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=379c2202096c" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>