<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Sanjai Kumar on Medium]]></title>
        <description><![CDATA[Stories by Sanjai Kumar on Medium]]></description>
        <link>https://medium.com/@sanjaikumar.ask?source=rss-b3420bf86988------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*G4wl3a4hf032XXM0COkdkw@2x.jpeg</url>
            <title>Stories by Sanjai Kumar on Medium</title>
            <link>https://medium.com/@sanjaikumar.ask?source=rss-b3420bf86988------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 17:25:26 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@sanjaikumar.ask/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Hesitation Is the Real Killer at Work]]></title>
            <description><![CDATA[<div class="medium-feed-item"><p class="medium-feed-image"><a href="https://medium.com/@sanjaikumar.ask/hesitation-is-the-real-killer-at-work-f9b332089197?source=rss-b3420bf86988------2"><img src="https://cdn-images-1.medium.com/max/2600/1*2VKSP2sMWTqK1Cr56GcHOA.png" width="2720"></a></p><p class="medium-feed-link"><a href="https://medium.com/@sanjaikumar.ask/hesitation-is-the-real-killer-at-work-f9b332089197?source=rss-b3420bf86988------2">Continue reading on Medium »</a></p></div>]]></description>
            <link>https://medium.com/@sanjaikumar.ask/hesitation-is-the-real-killer-at-work-f9b332089197?source=rss-b3420bf86988------2</link>
            <guid isPermaLink="false">https://medium.com/p/f9b332089197</guid>
            <category><![CDATA[career-growth]]></category>
            <category><![CDATA[work]]></category>
            <category><![CDATA[public-speaking]]></category>
            <category><![CDATA[imposter-syndrome]]></category>
            <category><![CDATA[software-engineering]]></category>
            <dc:creator><![CDATA[Sanjai Kumar]]></dc:creator>
            <pubDate>Fri, 20 Nov 2026 10:45:45 GMT</pubDate>
            <atom:updated>2026-11-20T10:45:45.656Z</atom:updated>
        </item>
        <item>
            <title><![CDATA[Idempotency is more than just an answer to a Interview question]]></title>
            <link>https://medium.com/@sanjaikumar.ask/idempotency-is-more-than-just-an-answer-to-a-interview-question-d7a8be820f84?source=rss-b3420bf86988------2</link>
            <guid isPermaLink="false">https://medium.com/p/d7a8be820f84</guid>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[software-engineering]]></category>
            <category><![CDATA[interview-questions]]></category>
            <category><![CDATA[idempotency]]></category>
            <category><![CDATA[system-design-interview]]></category>
            <dc:creator><![CDATA[Sanjai Kumar]]></dc:creator>
            <pubDate>Thu, 24 Sep 2026 15:06:09 GMT</pubDate>
            <atom:updated>2026-09-24T15:06:09.631Z</atom:updated>
            <content:encoded><![CDATA[<p>We mostly heard the word Idempotency in every interview question and what HTTP methods follows it, But it is beyond that.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*MUm9Ad9rLubD-E_stnlqMA.png" /></figure><h3>What is Idempotency:</h3><p>It means doing an action that gives you exact same result even after doing it many times. In software engineering, it is a fundamental principle that ensures reliablity and stability withing distributed system or APIs.</p><p>For example, If you put first gear from neutral, A Car will move in that gear’s speed limit. If you again try to put first gear from neutral, still it is the same</p><h3>Where it is used in real world:</h3><p><strong><em>Payment Retries:</em></strong> You make payment in UPI, The request hit the backend system but suddenly you lost your internet connection before even the request is comitted, then your app makes a retry. If it is an idempotent system your retry will be considered as duplicate event and thus prevent extra deductions and saves your moeny 😃.</p><p><strong><em>Shopping Cart submission twice:</em></strong> When you try to place an order from your cart, suddenly the button froze and if you hit it multiple times before the button becomes normal, those <em>“n” </em>hits will considered a single.</p><p><strong><em>LLM tool calling / agent retries:</em></strong> When you run an agent to do a task, post your LLM processing it, If your agent stopped uncertainly, then re-request will cost your tokens/bills, So agent will re-request with idempotent key and your model will reply back with previously processed result sets.</p><h3>When can you choose it in your architecture:</h3><p><strong>The Trigger that comes outside from your system’s control:</strong></p><p>Whenever a call that comes from outside of your system that can potentially retry e.g Payment gateway, Third Party’s integration, Webhook, then you have to handle such calls in idempotent in nature.</p><p><strong>The operation carries a real cost when duplicated:</strong></p><p>Some actions which are fragile in nature to repeat such as Charging a customer twice or dispatching same cold mail more times or duplicating a data without realising cost management etc…</p><p><strong>Messaging system’s <em>“at-least-once”</em> delivery:</strong></p><p>Messaging system such as Kafka, RabbitMQ are explicit in their documentation that a message may be delivered more than once under certain failure conditions, So Any consumer built on top of such infrastructure must treat idempotency as a baseline requirement, not an optimization.</p><p><strong>Retries happen automatically and without human oversight:</strong></p><p>Cron jobs, Autonomous AI Agents are all capable of re-attempting an operation without a person present to notice that it already succeeded. As systems move toward greater autonomy, the absence of a human checkpoint makes idempotent design less of a best practice and more of a structural necessity</p><p><strong>Where idempotency can reasonably be deprioritized:</strong></p><p>Not every operations need this treatment. Pure read operations, actions that are trivially reversible, and calls confined to a tightly controlled internal boundary with minimal retry exposure often don’t need this. Cause Idempotency introduces its own overhead, additional storage for keys or logs, extra lookups before every write, so the decision should weigh the cost of implementing it against the cost of a duplicate actually occurring, rather than being applied as a blanket rule.</p><h3>Final Conclusion:</h3><p>At its core, idempotency is what lets a system tell the truth about its own state. When the same request can land once, twice, or five times and still leave your data exactly as it should be, you’ve bought yourself <strong>data integrity,</strong></p><p>If no phantom charges, no duplicate rows, no orphaned records to explain in a postmortem. That integrity is what makes <strong>reliability</strong> possible in the first place, because a system you can retry safely is a system you can trust to recover from failure instead of compounding it. And that trust is what brings <strong>stability</strong> into the system.</p><p>If you like this article, please feel free to go through my others and give your comments and suggestions on the content 💪. I can improvise it 🙌.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d7a8be820f84" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Postgres MCP Server]]></title>
            <link>https://medium.com/@sanjaikumar.ask/postgres-mcp-server-dd56488028aa?source=rss-b3420bf86988------2</link>
            <guid isPermaLink="false">https://medium.com/p/dd56488028aa</guid>
            <category><![CDATA[mcp-server]]></category>
            <category><![CDATA[generative-ai-tools]]></category>
            <category><![CDATA[postgresql]]></category>
            <category><![CDATA[ai-agent]]></category>
            <category><![CDATA[cursor]]></category>
            <dc:creator><![CDATA[Sanjai Kumar]]></dc:creator>
            <pubDate>Wed, 05 Aug 2026 16:32:37 GMT</pubDate>
            <atom:updated>2026-08-05T16:32:37.844Z</atom:updated>
            <content:encoded><![CDATA[<h4>You can see the Power of a Postgres MCP server on your local machine</h4><p>In our last article we have seen what MCP is and What it does. If you have not gone through it. You can use this <a href="https://medium.com/@sanjaikumar.ask/what-is-mcp-why-it-is-needed-9aec6a770b6c">link</a> to get a quick look. Ok!, Can we now jump into how we can utilise MCP’s power in connecting it with Postgres and fuel your day to day development activity.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*XcBjAsPECRqFFakOj5nR0Q.png" /></figure><p>Let’s Hop-On and start 🚀</p><h3>What You need:</h3><ul><li>Node.js — Runtime for the MCP server</li><li>npm — Package manager for installing dependencies</li><li>TypeScript + tsx — We’ll write the server in TypeScript and run it directly without a build step</li><li>PostgreSQL — A running Postgres instance (local or remote) that you want to expose via MCP</li><li>An MCP client — The app that will connect to your server e.g Cursor, Claude Desktop, or any MCP-compatible client</li></ul><h3>Project Setup:</h3><h4>Initialise the project</h4><pre>mkdir postgres-mcp &amp;&amp; cd postgres-mcp<br>npm init -y</pre><h4>Install dependencies</h4><p>We need three runtime packages:</p><ul><li>@modelcontextprotocol/sdk — the official Javascript/Typescript Software Development Kit (SDK) for MCP.</li><li>pg — PostgreSQL client for Node.js</li><li>zod — schema validation library (used to define tool parameters)</li></ul><pre>npm install @modelcontextprotocol/sdk pg zod<br>npm install -D typescript tsx @types/node @types/pg</pre><h4>Configure TypeScript</h4><p>Create a tsconfig.json in the project root:</p><pre>{<br>  &quot;compilerOptions&quot;: {<br>    &quot;target&quot;: &quot;ES2022&quot;,<br>    &quot;module&quot;: &quot;NodeNext&quot;,<br>    &quot;moduleResolution&quot;: &quot;NodeNext&quot;,<br>    &quot;strict&quot;: true,<br>    &quot;esModuleInterop&quot;: true,<br>    &quot;skipLibCheck&quot;: true<br>  }<br>}</pre><h3>Server core concept &amp; Setting it Up:</h3><p>An MCP server is typically created with McpServer and connected to a transport such as Streamable HTTP or stdio</p><ul><li><strong>Streamable HTTP</strong> for remote servers (recommended).</li><li><strong>HTTP + SSE</strong> for backwards compatibility only.</li><li><strong>stdio</strong> for local, process-spawned integrations.</li></ul><p>Import required packages as below and set up postgres connection</p><pre>import { McpServer } from &quot;@modelcontextprotocol/sdk/server/mcp.js&quot;;<br>import { StdioServerTransport } from &quot;@modelcontextprotocol/sdk/server/stdio.js&quot;;<br>import { z } from &quot;zod&quot;;<br>import pg from &quot;pg&quot;;<br><br>// Initialises Postgres by creating a connection pool<br>const { Pool } = pg;<br><br>const pool = new Pool({<br>  host: process.env.PGHOST ?? &quot;localhost&quot;,<br>  port: Number(process.env.PGPORT ?? 5432),<br>  database: process.env.PGDATABASE,<br>  user: process.env.PGUSER,<br>  password: process.env.PGPASSWORD,<br>});</pre><p>You can then create MCP server and connect it via StdioServerTransport</p><pre><br>const server = new McpServer({<br>  name: process.env.MCP_NAME || &quot;postgres&quot;,<br>  version: &quot;1.0.0&quot;,<br>});<br><br>/* <br>Here comes the code for registering tools(functionalities) <br>on the server which can be utilised by LLMs. e.g test_connection, list tables<br>*/<br><br>const transport = new StdioServerTransport();<br>await server.connect(transport);</pre><p>You can register Postgres user functionalities as a tool to server and let LLMs consume it based on End-User interaction</p><pre>// Test connection by listing current database, user and version<br>server.tool(<br>  &quot;test_connection&quot;,<br>  &quot;Test connection to local PostgreSQL&quot;,<br>  {},<br>  async () =&gt; {<br>    const result = await pool.query(&quot;select current_database(), current_user, version()&quot;);<br>    return asText(result.rows);<br>  }<br>);<br><br>/* <br>To list tables in selected database. LLM can figure out <br>which database to connect based on User Prompt<br>*/<br>server.tool(<br>  &quot;list_tables&quot;,<br>  &quot;List tables in the public schema&quot;,<br>  {},<br>  async () =&gt; {<br>    const result = await pool.query(`<br>      select table_name<br>      from information_schema.tables<br>      where table_schema = &#39;public&#39;<br>      order by table_name<br>    `);<br><br>    return asText(result.rows);<br>  }<br>);<br><br>/*<br>  Read query operation only exposes select query db operation<br>  by validating with basic regex<br>*/<br>server.tool(<br>  &quot;read_query&quot;,<br>  &quot;Run a SELECT query only&quot;,<br>  {<br>    sql: z.string(),<br>  },<br>  async ({ sql }) =&gt; {<br>    if (!/^\s*select\b/i.test(sql)) {<br>      throw new Error(&quot;Only SELECT queries are allowed in read_query.&quot;);<br>    }<br><br>    const result = await pool.query(sql);<br>    return asText(result.rows);<br>  }<br>);<br><br>/*<br>  Execute query operation only exposes CUD db operation<br>  by validating with basic regex.<br>*/<br>server.tool(<br>  &quot;execute_write&quot;,<br>  &quot;Run INSERT, UPDATE, or DELETE query&quot;,<br>  {<br>    sql: z.string(),<br>    values: z.array(z.any()).optional(),<br>  },<br>  async ({ sql, values = [] }) =&gt; {<br>    if (!/^\s*(insert|update|delete)\b/i.test(sql)) {<br>      throw new Error(&quot;Only INSERT, UPDATE, or DELETE are allowed.&quot;);<br>    }<br><br>    const result = await pool.query(sql, values);<br>    return asText({<br>      rowCount: result.rowCount,<br>      command: result.command,<br>    });<br>  }<br>);</pre><h3>Configuring the MCP Server in Your Client:</h3><p>An MCP client is any AI application that supports the Model Context Protocol — examples include <a href="https://cursor.com/">Cursor</a>, <a href="https://claude.ai/download">Claude Desktop</a>, and <a href="https://openai.com/chatgpt/desktop">ChatGPT Desktop</a>. The client spawns your server as a child process and communicates with it over stdio.</p><p>Each client has its own config file location:</p><ul><li>Cursor — ~/.cursor/mcp.json (global) or .cursor/mcp.json (per-project)</li><li>Claude Desktop — ~/Library/Application Support/Claude/claude_desktop_config.json (macOS)</li><li>ChatGPT Desktop — Settings → MCP Servers (via UI)</li></ul><blockquote>~/.cursor/mcp.json</blockquote><pre>{<br>  &quot;mcpServers&quot;: {<br>    &quot;local-postgres&quot;: {<br>      &quot;command&quot;: &quot;/Users/user/.local/share/mise/installs/node/20/bin/node&quot;,<br>      &quot;args&quot;: [<br>        &quot;/Users/user/Documents/postgres-mcp/node_modules/tsx/dist/cli.mjs&quot;,<br>        &quot;/Users/user/Documents/postgres-mcp/src/index.ts&quot;<br>      ],<br>      &quot;env&quot;: {<br>        &quot;PGHOST&quot;: &quot;localhost&quot;,<br>        &quot;PGPORT&quot;: &quot;5432&quot;,<br>        &quot;PGDATABASE&quot;: &quot;postgres&quot;,<br>        &quot;PGUSER&quot;: &quot;postgres&quot;,<br>        &quot;PGPASSWORD&quot;: &quot;postgres&quot;,<br>        &quot;MCP_NAME&quot;: &quot;local-postgres&quot;<br>      }<br>    }<br>  }<br>}</pre><h3>What each field does:</h3><ul><li>command — The executable to run</li><li>args — Arguments passed to the command — here we use tsx to run our TypeScript file directly</li><li>env — Environment variables passed to the server process. Our server reads these to connect to Postgres</li></ul><blockquote>Troubleshooting: If you see an ENOENT error mentioning npm path resolution, your client may not resolve npx correctly from its own environment. In that case, use absolute paths to both node and tsx:</blockquote><pre>{<br>  &quot;command&quot;: &quot;/absolute/path/to/node&quot;,<br>  &quot;args&quot;: [<br>    &quot;/absolute/path/to/postgres-mcp/node_modules/tsx/dist/cli.mjs&quot;,<br>    &quot;/absolute/path/to/postgres-mcp/src/index.ts&quot;<br>  ]<br>}</pre><blockquote><em>Find your paths with: </em><em>which node and </em><em>realpath node_modules/.bin/tsx</em></blockquote><p>A couple of things to note:</p><ol><li>I showed npx tsx as the primary config because it is simpler and works for most readers. The absolute-path version (which we needed for your Cursor setup) is in a troubleshooting callout so it doesn&#39;t scare off while you start.</li><li>Added a list above containing config file locations across clients which you can use it based on your client.</li><li>You can use MCP_NAME which can be more useful while debugging in client side.</li></ol><p>That’s a wrap. You went from zero to a working MCP server that lets any AI client query your Postgres database. If you have any Questions? Ideas? Roast my SQL? The comments are open 📥.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=dd56488028aa" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[What is MCP? Why it is needed?]]></title>
            <link>https://medium.com/@sanjaikumar.ask/what-is-mcp-why-it-is-needed-9aec6a770b6c?source=rss-b3420bf86988------2</link>
            <guid isPermaLink="false">https://medium.com/p/9aec6a770b6c</guid>
            <category><![CDATA[ai-agent]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[software-engineering]]></category>
            <category><![CDATA[mcp-server]]></category>
            <category><![CDATA[model-context-protocol]]></category>
            <dc:creator><![CDATA[Sanjai Kumar]]></dc:creator>
            <pubDate>Sat, 01 Aug 2026 15:05:28 GMT</pubDate>
            <atom:updated>2026-08-05T16:34:15.373Z</atom:updated>
            <content:encoded><![CDATA[<h4>We have been hearing this word whenever we speak about AI driven development</h4><p>If we speak about modern Software development. First thing you hear is whether have you connected your AI Agent with necessary software tools. That’s it!. It’s where MCP starts and hold it’s place.</p><blockquote>Every second tool now says “MCP support available.” Figma has it, Notion has it, Higgsfield has it. Even every company started providing it to let customers interact with their knowledge base.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*nPuUw2R8bGqNjmgNjIxIaQ.png" /></figure><p><strong><em>Let’s dive on</em> 🏊</strong></p><h3>What is MCP &amp; Who Actually built it</h3><p>MCP stands for Model Context protocol. It enables AI models to interact with external tools and thus helping users with what they needed.</p><p>It was built by Anthropic and open sourced it on November 25, 2024.</p><p>Here’s the interesting bit: MCP isn’t “Anthropic’s protocol” anymore. In December 2025, Anthropic donated it to a newly formed body called the <strong>AAIF (Agentic AI Foundation)</strong>, under the Linux Foundation, alongside founding contributions from Block and OpenAI.</p><blockquote>Okay, But What Problem Does MCP Actually Solve?</blockquote><p>Before MCP, if you wanted an AI assistant to talk to Slack, Postgres, GitHub, and Notion, you had to build <strong>four separate custom integrations</strong>. Every AI app (Claude, ChatGPT, your own agent) that wanted to use those same four tools had to build its <em>own</em> four integrations too.</p><p>That’s the classic <strong>M × N problem</strong> — M applications, N tools, and you’re writing M×N integration code.</p><p>MCP solves this with M + N. You build one MCP server for a tool and let all other MCP compatible AI assistants use it.</p><p>Anthropic calls it <strong>“USB-C port for AI applications”. </strong>Before USB-C, every device had its own charger shape. Now one port, one cable, works across devices. MCP is doing the same thing for “AI talking to tools.”</p><h3>The Three Players in MCP</h3><p>There are only three roles you need to understand in this concept:</p><ul><li><strong>Host</strong> — the actual AI application you’re using. Cursor, Claude Desktop, Claude Code.</li><li><strong>Client</strong> — lives inside the host, manages one dedicated connection to one server.</li><li><strong>Server — </strong>the thing <em>you</em> build. It exposes capabilities the AI can use.</li></ul><p>A server can expose three kinds of things:</p><ul><li><strong>Tools</strong> — functions the model can call to <em>do</em> something (run a query, send a message, create a ticket)</li><li><strong>Resources</strong> — data the host can read, more “GET this data” shaped than “do this action” shaped</li><li><strong>Prompts</strong> — reusable prompt templates the server can hand to the client</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/900/1*rdLTT93the_NQECqGoos5Q.gif" /></figure><h3>What MCP Enables</h3><ul><li>It let AI Agents interact with wide range of tools such as Figma, Notion, Jira and thus can help in complete end to end Software Development.</li><li>It connects with wide range of backend store e.g Postgres and helps answering to the questions of a non technical person e.g Enterprise Chatbots</li><li>It helps AI Agents connect with your day to day apps like Google Mail, Calendar and can help summarise/prioritise your work.</li></ul><h3>Where does MCP sits</h3><ul><li><strong>Developers — </strong>It helps developers in completed SDLC process right from analysing product requirements to connecting with Figma and Coding.</li><li><strong>AI Agents — </strong>It helps AI Agents to interact with many resources and provide clear picture of the data and it’s representation to an end-user</li><li><strong>End Users — </strong>It enhances the capabilities of AI Agents and takes action on behalf of en end-user with their approval.</li></ul><h3>Broad ecosystem support</h3><p>MCP is an open protocol supported across a wide range of clients and servers. AI assistants like <a href="https://claude.com/docs/connectors/building">Claude</a> and <a href="https://developers.openai.com/api/docs/mcp/">ChatGPT</a>, development tools like <a href="https://code.visualstudio.com/docs/copilot/chat/mcp-servers">Visual Studio Code</a>, <a href="https://cursor.com/docs/context/mcp">Cursor</a>, and many others are supporting MCP, making it easy to build once and integrate everywhere.</p><p>If you liked the content. Feel free to give your comments and thoughts on this 🤔.</p><p>If you find it more interesting, You can go through my next article “<em>How can we create a Postgres MCP Server</em>” on <a href="https://medium.com/@sanjaikumar.ask/postgres-mcp-server-dd56488028aa">here</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9aec6a770b6c" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[SQL vs NoSQL: How to Choose the Right Database for Your Application]]></title>
            <link>https://medium.com/@sanjaikumar.ask/sql-vs-nosql-how-to-choose-the-right-database-for-your-application-b5ae4e324593?source=rss-b3420bf86988------2</link>
            <guid isPermaLink="false">https://medium.com/p/b5ae4e324593</guid>
            <category><![CDATA[postgresql]]></category>
            <category><![CDATA[sql]]></category>
            <category><![CDATA[nosql]]></category>
            <category><![CDATA[software-engineering]]></category>
            <category><![CDATA[mongodb]]></category>
            <dc:creator><![CDATA[Sanjai Kumar]]></dc:creator>
            <pubDate>Sun, 26 Jul 2026 15:09:21 GMT</pubDate>
            <atom:updated>2026-07-26T15:14:03.225Z</atom:updated>
            <content:encoded><![CDATA[<p>A practical guide to picking the right database without the hype.</p><h3>Why Should Developers Care?</h3><p>Imagine you’re building an e-commerce application. You need to store users, products, orders, and reviews. Someone on your team suggests MySQL. Someone else says MongoDB. Both sound reasonable. How do you decide?</p><p>So you open your laptop and ask the one question every backend engineer eventually asks: <strong>should this go in MySQL, or should this go in MongoDB?</strong></p><p>It sounds like a small decision. It isn’t.</p><p>Pick the wrong database early on, and you don’t feel the pain immediately. You feel it six months later, when your user base has grown, your queries have gotten more complex, and suddenly every schema change feels like defusing a bomb. Migrating from one database paradigm to another mid-project is one of the more painful things a team can go through — not impossible, just expensive in time, bugs, and trust.</p><p>So before writing a single line of code, it helps to actually understand what SQL and NoSQL are solving for. Not as buzzwords as tools with real trade-offs.</p><h3>What is SQL?</h3><p>SQL databases are relational databases. They store data in tables — think of a spreadsheet with rows and columns. Each table has a fixed structure called a schema, and you define that schema before inserting any data.</p><p>Here’s what that means in practice:</p><p>A users table might have columns for id, name, email, and created_at. Every row in that table follows the same structure. You can&#39;t suddenly store a user with an address field unless you modify the schema first.</p><p>The power of relational databases comes from relationships. Your orders table can reference your users table through a foreign key. This means you can ask questions like &quot;show me all orders placed by user #42&quot; with a simple JOIN query.</p><p>SQL databases also guarantee ACID properties:</p><ul><li>Atomicity — A transaction either fully completes or fully rolls back. No half-finished operations.</li><li>Consistency — Data always moves from one valid state to another.</li><li>Isolation — Concurrent transactions don’t interfere with each other.</li><li>Durability — Once committed, data survives crashes.</li></ul><p>This matters when you’re dealing with money, inventory, or anything where partial updates would be catastrophic.</p><p>Popular SQL databases: MySQL, PostgreSQL, SQL Server.</p><p><strong>Real-world analogy:</strong> A SQL database is like a well-organized filing cabinet. Every folder has labeled sections, every document goes in the same spot, and if a form is missing a required field, the clerk simply won’t accept it.</p><h4>What is NoSQL?</h4><p>NoSQL databases take a different approach. Instead of forcing data into rigid tables, they let you store information in flexible formats, most commonly as documents (think JSON objects).</p><p>Why does this exist? Because not all data fits neatly into rows and columns.</p><p>Consider a product catalog. A laptop has specs like RAM, storage, and screen size. A t-shirt has size, colour, and material. In a SQL database, you’d either create separate tables for each product type or have a bloated table full of nullable columns. In a document database, each product is just a self-contained document with whatever fields it needs.</p><p>NoSQL databases come in several flavours:</p><ul><li>Document databases (MongoDB, Firestore) — Store data as flexible JSON-like documents.</li><li>Key-value stores (Redis, DynamoDB) — Simple lookup by key. Extremely fast for caching and session data.</li><li>Wide-column stores (Cassandra) — Optimised for writing and reading large volumes of data across distributed clusters.</li></ul><p>NoSQL databases typically trade strict consistency for flexibility and horizontal scalability. Many follow the <strong>BASE</strong> model (Basically Available, Soft state, Eventually consistent) — meaning your data will be consistent eventually, but not necessarily at the exact moment you read it.</p><p>Popular NoSQL databases: MongoDB, DynamoDB, Redis, Cassandra.</p><p><strong>Real-world analogy:</strong> A NoSQL database is like a big storage room where each box can be packed differently depending on what’s inside. Books in one box, clothes folded in another, electronics wrapped in bubble wrap in a third. No fixed template, just “whatever fits this item best.”</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*TYnOHhyJHYpupqCb-gQP1A.png" /></figure><h3>SQL vs NoSQL: A Side-by-Side Look</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*CIiK-pjaZgCvJgPMMT5_ZA.png" /></figure><h3>How Do They Scale?</h3><p>There are two ways to handle more traffic:</p><p>Vertical Scaling means upgrading your existing server, more CPU, more RAM, faster disks. It’s straightforward but has a ceiling. At some point, you can’t buy a bigger machine.</p><p>Horizontal Scaling means adding more servers and distributing the load across them. There’s no hard ceiling, but it introduces complexity , how do you split the data? How do you keep it consistent?</p><p>SQL databases have traditionally leaned on vertical scaling, because relationships and transactions across a distributed set of machines are genuinely hard to guarantee correctly. That said, modern SQL systems have gotten much better at horizontal scaling too (via sharding and distributed variants).</p><p>NoSQL databases were largely designed from day one with horizontal scaling in mind, since their data models don’t lean as heavily on cross-table relationships.</p><p>Two concepts worth knowing:</p><ul><li>Replication — Copying data to multiple servers for redundancy and read performance. Both SQL and NoSQL support this.</li><li>Sharding — Splitting data across servers so each one holds only a portion. NoSQL databases generally handle this more gracefully because of their self-contained data model.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2YmB2KP4k2bWjBxojpeyTw.png" /></figure><h3>Real-World Examples</h3><ul><li><strong>Banking systems → SQL.</strong> Money needs strict consistency. You cannot afford a “maybe” when it comes to a balance transfer.</li><li><strong>E-commerce orders → SQL.</strong> Orders, inventory, and payments involve relationships and transactions that benefit heavily from ACID guarantees.</li><li><strong>Social media feed → NoSQL.</strong> Posts vary wildly in structure, and the priority is fast reads at massive scale, not strict relational integrity.</li><li><strong>Chat applications → NoSQL.</strong> Messages are simple, high-volume, and need to be written and fetched quickly, often with flexible metadata (reactions, attachments, read receipts).</li></ul><p>Interestingly, most large scale companies don’t pick one and stop there. <strong>Netflix, Uber, and Amazon</strong> all use SQL and NoSQL side by side, a SQL database for billing and transactional data, NoSQL for recommendation feeds, session data, or activity logs. Different parts of the same system, different requirements.</p><h3>Why NoSQL Uses BASE</h3><p>Distributed NoSQL databases like Apache Cassandra, Amazon DynamoDB, and MongoDB often adopt the BASE model to handle web-scale applications. According to the CAP Theorem, a distributed system can only guarantee two out of three attributes: Consistency, Availability, and Partition Tolerance. Because distributed databases must tolerate network partitions, they must choose between consistency and availability. BASE prioritises <strong>Availability.</strong></p><p><strong>Real-world analogy: </strong>Consider an <strong>Instagram</strong> feed or a <strong>Netflix</strong> profile. When you post a photo or change your profile preference, the change is written to a local data hub immediately. Because of the <strong>Soft State</strong>, a friend accessing the app from another region might not see the update for a few seconds. However, the app remains <strong>Basically Available</strong> and operational rather than locking out users while waiting for every server worldwide to sync. Within a short period, <strong>Eventual Consistency</strong> is achieved, and everyone sees the identical profile state</p><h3>Final Thoughts</h3><p>SQL and NoSQL aren’t competing technologies — they’re complementary tools that solve different problems.</p><p>Choose SQL when your data is structured, relationships matter, and you need strong consistency guarantees. Choose NoSQL when your data varies in shape, you need to scale horizontally, or your access patterns are simple but high-volume.</p><p>The engineers who make the best database decisions aren’t the ones who know the most about any single technology. They’re the ones who start by understanding their data, their access patterns, and their consistency requirements and then pick the tool that fits.</p><p>Don’t choose a database because it’s popular. Choose it because it solves your problem.</p><p>If you find it interesting, please feel free to share your comments and thoughts 🚀.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b5ae4e324593" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Unit Tests vs Integration Tests]]></title>
            <link>https://medium.com/@sanjaikumar.ask/unit-tests-vs-integration-tests-2317941dc8ca?source=rss-b3420bf86988------2</link>
            <guid isPermaLink="false">https://medium.com/p/2317941dc8ca</guid>
            <category><![CDATA[software-engineering]]></category>
            <category><![CDATA[integration-testing]]></category>
            <category><![CDATA[software-testing]]></category>
            <category><![CDATA[unit-testing]]></category>
            <category><![CDATA[ci-cd-pipeline]]></category>
            <dc:creator><![CDATA[Sanjai Kumar]]></dc:creator>
            <pubDate>Wed, 22 Jul 2026 18:48:55 GMT</pubDate>
            <atom:updated>2026-07-22T18:52:42.612Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Ez2srlS6rKciGFXYQRFasw.png" /></figure><p>Every codebase has that one PR comment “Is this a unit test or an integration test”. Half the time, nobody’s fully sure, even the senior developers.</p><h4>What is Unit Test and Integration Test</h4><p>Unit test is verifies small unit of code whether it behaves correctly or not. It can be a <strong>method</strong> or sometimes a <strong>class </strong>without depending on the external systems such as Database, Redis, Files, Cloud etc…</p><blockquote>“I’m testing only my code with everything around it faked in an isolation”</blockquote><p>Integration Test verifies whether multiple components work together with realtime interactions with such external systems.</p><blockquote>“I’m testing multiple components wired together whether actually works or not with real pieces of data”</blockquote><p>It is <strong><em>“isolation”</em></strong> vs <strong><em>“reality”</em></strong></p><h3>Where They Sit on the Test Pyramid</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/720/1*VsRNnNiappW-WeW7XB9Shg.png" /></figure><p>Unit tests are cheaper to write and can run in milliseconds, so we can write many tests. Integration tests are somewhat slower to write and run as tests spins up real dependencies(DB queries, third-party APIs, message queues).</p><p>If your pyramid is upside down, mostly integration and E2E tests and few unit tests, your CI pipeline is probably slow, and failures are probably hard to localise.</p><h3>A Practical Example</h3><p>Imagine you’re implementing a new feature: <strong>Create Bank Account</strong>.</p><h4>Unit Test</h4><p>You test the business logic in isolation:</p><ul><li>Is the account number generated correctly?</li><li>Is the minimum deposit validation working?</li><li>Is the interest rate calculated correctly?</li></ul><h4>Integration Test</h4><p>You verify the complete backend flow:</p><ul><li>API receives the request.</li><li>Service processes it.</li><li>Data is saved to the database.</li><li>A confirmation event is published to Kafka.</li></ul><h3>Same Feature, Two Tests: A Node.js Example</h3><p>Say you’re validating a JWT in an Express auth middleware.</p><h4>Unit test — mock the dependency (Jest)</h4><pre>const { validateToken } = require(&#39;../auth&#39;);<br><br>test(&#39;expired token returns an error&#39;, () =&gt; {<br>  const fakeClock = () =&gt; new Date(&#39;2026-01-01T00:00:00Z&#39;);<br>  const expiredToken = generateTestToken({ exp: &#39;2025-01-01T00:00:00Z&#39; });<br><br>  expect(() =&gt; validateToken(expiredToken, fakeClock))<br>    .toThrow(&#39;TokenExpiredError&#39;);<br>});</pre><p>No database, no network, no real clock. One input, one behaviour, one reason to fail</p><h4>Integration test — real dependencies (Jest + Supertest + a real test DB)</h4><pre>const request = require(&#39;supertest&#39;);<br>const { app } = require(&#39;../app&#39;);<br>const { setupTestDb, teardownTestDb } = require(&#39;./helpers/db&#39;);<br><br>let db;<br><br>beforeAll(async () =&gt; { db = await setupTestDb(); });<br>afterAll(async () =&gt; { await teardownTestDb(db); });<br><br>test(&#39;valid token allows request through to /users/me&#39;, async () =&gt; {<br>  const token = await loginAndGetRealToken(app, &#39;sanjai@example.com&#39;);<br><br>  const res = await request(app)<br>    .get(&#39;/users/me&#39;)<br>    .set(&#39;Authorization&#39;, `Bearer ${token}`);<br><br>  expect(res.status).toBe(200);<br>});</pre><p>Real DB, real middleware chain, real HTTP round trip. It’s slower than the unit test, but it catches things the unit test structurally <em>cannot</em>, a bad SQL query, a misconfigured middleware order, a serialisation mismatch.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/760/1*afzABxBHyfbagkIk1XE8ww.png" /></figure><h3>How This Fits Into CI/CD</h3><p>The pyramid isn’t just a diagram — it should shape your pipeline stages:</p><ol><li><strong>On every commit / PR: </strong>Run the full unit test suite. It’s fast (seconds), so there’s no excuse to skip it. Fail the build immediately on any red test.</li><li><strong>Pre-merge or on merge to main: </strong>Run integration tests against ephemeral, real dependencies (a fresh Postgres container, a sandboxed third-party API). These take longer, so many teams gate them at the PR level rather than on every single push.</li><li><strong>Pre-deploy / staging: </strong>Run a smaller set of E2E tests against a staging environment that mirrors production.</li><li><strong>Production: </strong>Smoke tests only, checking that critical paths are alive after deploy.</li></ol><h3>Where People Actually Go Wrong</h3><p><strong>Junior developers</strong> tend to:</p><ul><li>Write “unit tests” that spin up a real DB, technically an integration test wearing in disguise, and it’s why the test suite feels slow for no clear reason.</li><li>Mock so aggressively that the test only verifies the mock was called and not that the logic is correct.</li><li>Skip integration tests entirely because they’re annoying to set up, then discover the DB query bug in staging.</li></ul><p><strong>Senior developers</strong> tend to:</p><ul><li>Over invest in integration tests for logic that’s actually pure and should be unit-tested in isolation, bloating CI time.</li><li>Under estimate the test<em>, </em>the exact point where your code hands off to Postgres, an external API, or a queue because that code is harder to reach with a fast unit test.</li><li>Forget to revisit test boundaries after a refactor, leaving unit tests that mock a dependency which no longer exists in that form.</li></ul><h4>Important Caveat as Integration Tests Touch Real Systems</h4><p>This is the part that bites teams in production, not just in CI. Because integration tests talk to real components, they inherit real world risk.</p><ul><li><strong>Never point integration tests at production infrastructure.</strong> Use ephemeral, disposable instances — Docker containers (test containers), an in-memory equivalent service, or a dedicated test project on a third-party service (Razorpay, for instance, has a sandbox mode — use it, never live keys).</li><li><strong>Isolate test data.</strong> Every test run should create its own data and clean up after itself. Shared test databases across parallel CI jobs are a classic source of flaky, order dependent failures.</li><li><strong>Watch for flakiness from real timing and network behavior.</strong> A real DB call or external API can be slow, rate limited, or briefly unavailable, none of which reflects a bug in your code. Build in retries and reasonable timeouts for the test harness itself, but don’t mask genuine failures by retrying blindly.</li><li><strong>Never let integration tests send real side effects</strong> such as real emails, real payment charges, real SMS. Always run against sandbox/test-mode credentials for third-party integrations.</li></ul><h3>The One Rule of Thumb</h3><blockquote><em>If your test would still pass after deleting the database, it’s a unit test. If deleting the database would break it, it’s an integration test.</em></blockquote><p>If you’re starting your career, a useful mindset is:</p><ul><li><strong>Unit tests answer:</strong> <em>“Did I write the logic correctly?”</em></li><li><strong>Integration tests answer:</strong> <em>“Did I connect all the pieces correctly?”</em></li><li><strong>End-to-end tests answer:</strong> <em>“Does the entire application work the way a user expects?”</em></li></ul><p>Building both kinds well isn’t about ratios or rules enforced by a linter. It’s about knowing, for each test you write, exactly what question you’re asking and picking the cheapest test that can honestly answer it.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2317941dc8ca" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Things to know about Elasticsearch]]></title>
            <link>https://medium.com/@sanjaikumar.ask/things-to-know-about-elasticsearch-e2276fdbcd44?source=rss-b3420bf86988------2</link>
            <guid isPermaLink="false">https://medium.com/p/e2276fdbcd44</guid>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[software-engineering]]></category>
            <category><![CDATA[distributed-systems]]></category>
            <category><![CDATA[system-design-interview]]></category>
            <category><![CDATA[elasticsearch]]></category>
            <dc:creator><![CDATA[Sanjai Kumar]]></dc:creator>
            <pubDate>Tue, 21 Jul 2026 18:03:55 GMT</pubDate>
            <atom:updated>2026-07-21T18:07:53.363Z</atom:updated>
            <content:encoded><![CDATA[<p>I’ve been working with Elasticsearch for a while now, and I remember how disorienting it felt initially, especially coming from a relational database background. No joins, no foreign keys, and the thing everyone casually calls a “<strong><em>table”</em></strong> doesn’t really behave like one. This post is my attempt to lay out the core ideas and what makes ES fast, how its data model is different from a traditional DB, and the query types you’ll end up using in pretty much every project.</p><figure><img alt="Elastic Search Cover image" src="https://cdn-images-1.medium.com/max/1024/1*uGup0EIlpo_vKzz_hhrclw.png" /><figcaption>Elastic Search</figcaption></figure><h3>What is Elasticsearch, really?</h3><p>At its core, Elasticsearch is a distributed search and analytics engine, built on top of Apache Lucene. Lucene handles the actual indexing and searching of text. Elasticsearch wraps around it with a REST API, horizontal scaling, and distributed coordination across a cluster of machines. Think of it as a database that’s optimised for finding things fast rather than for complex transactions.</p><p>Traditional databases (PostgreSQL, MySQL) are great at:</p><ul><li>Relationships between tables (JOINs)</li><li>Transactions (ACID guarantees)</li><li>Single record lookups by primary key</li></ul><p>Elasticsearch is great at:</p><ul><li>Full-text search (“find documents containing these words”)</li><li>Filtering across millions of records in milliseconds</li><li>Aggregations (counts, averages, histograms across your data)</li><li>Fuzzy/partial matching</li></ul><blockquote>Every record in Elasticsearch is a <strong>document</strong> — basically a JSON object, a key/value pair structure</blockquote><h3>Core concepts, mapped to relational DB terms</h3><p>The quickest way to get your bearings is to map each ES concept to its closest relational equivalent. The mappings isn’t perfect but it gives you something familiar to correlate.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jyy40YMvCIP-JE3JRtniMw.png" /><figcaption>ES Concepts</figcaption></figure><h3>What actually makes ES fast</h3><p>The answer is “<strong><em>inverted index</em></strong>”. In traditional relational database, when searching for rows where description contains the word &quot;quick&quot; typically ends up scanning through rows (unless there&#39;s a full-text index bolted on, which is often an afterthought). Elasticsearch flips this around entirely. Instead of asking &quot;what words are in this document,&quot; it precomputes the answer to &quot;which documents contain this word&quot; — for every single word, ahead of time.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qyXY0IPkmidcvp_nVuyO1A.png" /></figure><p>Here’s the picture: two small documents get tokenized (broken down into individual words), and each token becomes a key in an index that points back to the document IDs containing it.</p><p>Once that index exists, looking up “which documents contain ‘quick’ ” becomes a direct key lookup, not a scan. That’s the O(1) behaviour — the cost of a term lookup doesn’t grow with the number of documents in the index, because you’re not iterating over them at all. You’re jumping straight to the postings list for that term.</p><h3>How Data Lives in ES</h3><p>Complete example by “Employee Index”</p><p>Index Name: employees</p><p>Mapping:</p><pre>PUT /employees<br>{<br>  &quot;settings&quot;: {<br>    &quot;analysis&quot;: {<br>      &quot;normalizer&quot;: {<br>        &quot;lowercase&quot;: {<br>          &quot;type&quot;: &quot;custom&quot;,<br>          &quot;filter&quot;: [&quot;lowercase&quot;]<br>        }<br>      }<br>    }<br>  },<br>  &quot;mappings&quot;: {<br>    &quot;dynamic&quot;: &quot;false&quot;,<br>    &quot;properties&quot;: {<br>      &quot;emp_id&quot;: {<br>        &quot;type&quot;: &quot;keyword&quot;,<br>        &quot;normalizer&quot;: &quot;lowercase&quot;<br>      },<br>      &quot;first_name&quot;: {<br>        &quot;type&quot;: &quot;keyword&quot;,<br>        &quot;normalizer&quot;: &quot;lowercase&quot;,<br>        &quot;fields&quot;: {<br>          &quot;search&quot;: {<br>            &quot;type&quot;: &quot;text&quot;<br>          }<br>        }<br>      },<br>      &quot;last_name&quot;: {<br>        &quot;type&quot;: &quot;keyword&quot;,<br>        &quot;normalizer&quot;: &quot;lowercase&quot;,<br>        &quot;fields&quot;: {<br>          &quot;search&quot;: {<br>            &quot;type&quot;: &quot;text&quot;<br>          }<br>        }<br>      },<br>      &quot;email&quot;: {<br>        &quot;type&quot;: &quot;keyword&quot;,<br>        &quot;normalizer&quot;: &quot;lowercase&quot;<br>      },<br>      &quot;projects&quot;: {<br>        &quot;type&quot;: &quot;nested&quot;,<br>        &quot;properties&quot;: {<br>          &quot;project_name&quot;: {<br>            &quot;type&quot;: &quot;keyword&quot;,<br>            &quot;normalizer&quot;: &quot;lowercase&quot;,<br>            &quot;fields&quot;: {<br>              &quot;search&quot;: {<br>                &quot;type&quot;: &quot;text&quot;<br>              }<br>            }<br>          },<br>          &quot;start_date&quot;: {<br>            &quot;type&quot;: &quot;date&quot;,<br>            &quot;format&quot;: &quot;strict_date_optional_time||yyyy-MM-dd||epoch_millis&quot;<br>          },<br>          &quot;end_date&quot;: {<br>            &quot;type&quot;: &quot;date&quot;,<br>            &quot;format&quot;: &quot;strict_date_optional_time||yyyy-MM-dd||epoch_millis&quot;<br>          }<br>        }<br>      }<br>    }<br>  }<br>}</pre><figure><img alt="What each type defined in mapping does" src="https://cdn-images-1.medium.com/max/1024/1*RISMeJBhwau8c4KLTm7AFg.png" /><figcaption>Why each type was chosen</figcaption></figure><figure><img alt="Elastis search query types." src="https://cdn-images-1.medium.com/max/1024/1*m_IL7Nt_JUfYqi4LdTmMVw.png" /><figcaption>Query Types</figcaption></figure><p>Document Example</p><pre>POST /employees/_doc/EMP001<br>{<br>  &quot;emp_id&quot;: &quot;EMP001&quot;,<br>  &quot;first_name&quot;: &quot;Sanjai&quot;,<br>  &quot;last_name&quot;: &quot;A&quot;,<br>  &quot;email&quot;: &quot;sanjai@gmail.com&quot;,<br>  &quot;projects&quot;: [<br>    {<br>      &quot;project_name&quot;: &quot;Case Management Revamp&quot;,<br>      &quot;start_date&quot;: &quot;2026-03-01&quot;,<br>      &quot;end_date&quot;: &quot;2026-07-15&quot;<br>    },<br>    {<br>      &quot;project_name&quot;: &quot;ES Indexer Pipeline&quot;,<br>      &quot;start_date&quot;: &quot;2026-06-01&quot;,<br>      &quot;end_date&quot;: null<br>    }<br>  ]<br>}</pre><p>What ES stores internally (Lucene documents):</p><pre>Lucene Doc 0 (parent):<br>  emp_id = &quot;emp001&quot;          ← normalized to lowercase<br>  first_name = &quot;sanjai&quot;<br>  last_name = &quot;a&quot;<br>  email = &quot;sanjai@gmail.com&quot;<br><br>Lucene Doc 1 (nested child — projects[0]):<br>  projects.project_name = &quot;case management revamp&quot;<br>  projects.start_date = 1740787200000<br>  projects.end_date = 1752537600000<br><br>Lucene Doc 2 (nested child — projects[1]):<br>  projects.project_name = &quot;es indexer pipeline&quot;<br>  projects.start_date = 1748736000000<br>  projects.end_date = null</pre><p>Find employee by exact email</p><pre>GET /employees/_search<br>{<br>  &quot;query&quot;: {<br>    &quot;match&quot;: {<br>      &quot;first_name.search&quot;: &quot;san&quot;<br>    }<br>  }<br>}</pre><p>Search employees whose first name contains “san”</p><pre>GET /employees/_search<br>{<br>  &quot;query&quot;: {<br>    &quot;match&quot;: {<br>      &quot;first_name.search&quot;: &quot;san&quot;<br>    }<br>  }<br>}</pre><p>Find employees working on a specific project (nested query)</p><pre>GET /employees/_search<br>{<br>  &quot;query&quot;: {<br>    &quot;nested&quot;: {<br>      &quot;path&quot;: &quot;projects&quot;,<br>      &quot;query&quot;: {<br>        &quot;term&quot;: {<br>          &quot;projects.project_name&quot;: &quot;es indexer pipeline&quot;<br>        }<br>      }<br>    }<br>  }<br>}</pre><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e2276fdbcd44" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[What Really Happens When You Type a URL and Hit Enter]]></title>
            <link>https://medium.com/@sanjaikumar.ask/what-really-happens-when-you-type-a-url-and-hit-enter-65ec3b8794ff?source=rss-b3420bf86988------2</link>
            <guid isPermaLink="false">https://medium.com/p/65ec3b8794ff</guid>
            <category><![CDATA[networking]]></category>
            <category><![CDATA[internet]]></category>
            <category><![CDATA[sofrware-engineering]]></category>
            <category><![CDATA[web-development]]></category>
            <category><![CDATA[dns]]></category>
            <dc:creator><![CDATA[Sanjai Kumar]]></dc:creator>
            <pubDate>Wed, 15 Jul 2026 14:43:22 GMT</pubDate>
            <atom:updated>2026-07-21T13:53:55.179Z</atom:updated>
            <content:encoded><![CDATA[<p><em>A guided tour of the journey from your keyboard to a rendered page — DNS, IP, routing, TCP, TLS, and the HTTP call that ties everything together.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Kt8AtFxdmdUEpYoXCAezXw.png" /></figure><p>We do this hundreds of times a day, no? You type google.com, hit Enter, and the page just opens up — mostly in under a second. Feels like magic, or at least something too fast to have any real machinery behind it.</p><p>But in that split second, your computer is actually coordinating with a dozen-odd systems spread across the globe. It first figures out what you typed, then asks a global directory for an address, then negotiates a private conversation with a total stranger, then confirms that the stranger really is who it claims to be, and only after all that, it asks for the actual page.</p><p>This also happens to be one of the most commonly asked system design interview questions, and for good reason — it touches almost every layer of how the internet actually works. So let’s go through it properly, one step at a time, without skipping the boring bits.</p><h3>Step 1: The Browser Parses the URL</h3><p>Before anything even reaches the network, your browser first has to make sense of what you typed. A URL is not just some random string of text — it’s actually a structured address with clearly named parts.</p><p>Take this example:</p><pre>https://blog.example.com:443/articles/networking?ref=medium#intro</pre><p>The browser breaks this down as follows:</p><p>Part – Value – Meaning</p><p>Scheme – https – Which protocol to use</p><p>Host – blog.example.com – The domain to be located</p><p>Port – 443 – The door to knock on (443 being the default for HTTPS)</p><p>Path – /articles/networking – The exact resource being asked for</p><p>Query – ?ref=medium – Extra parameters tagged along</p><p>Fragment – #intro – A location within the page itself (this never goes to the server)</p><p>A few things happen here that most people don’t even realise:</p><ul><li><strong>Autocomplete and search fallback</strong> — If you type example.com without any scheme, the browser makes an educated guess. And if what you typed isn&#39;t a valid host at all (say, &quot;best biryani near me&quot;), it simply hands the whole thing over to your default search engine.</li><li><strong>HSTS check</strong> — The browser maintains a list of sites that have insisted on HTTPS-only access. If your site happens to be on that list, an http:// request gets silently upgraded to https:// even before it leaves your machine, which closes off a window that attackers could otherwise misuse.</li><li><strong>Punycode conversion</strong> — Domains with non-ASCII characters (like café.com) get converted into an ASCII-safe version (xn--caf-dma.com) that the network is actually able to route.</li></ul><p>Only once the browser has figured out the host does the real networking begin, because at this point it only has a <em>name</em> — and the internet, unfortunately, doesn’t route by names. It routes purely by numbers.</p><h3>Step 2: DNS — Turning a Name Into an Address</h3><p>Computers don’t really talk to blog.example.com. What they talk to is IP addresses, something like 93.184.216.34. <strong>DNS, or the Domain Name System, is essentially the internet&#39;s phone directory</strong> — it translates the names we understand into addresses that machines understand.</p><p>Your computer, quite sensibly, doesn’t want to ask the entire world every single time. So it checks a chain of caches first, starting from the closest one:</p><ol><li><strong>Browser cache</strong> — did I already look this up a few seconds back?</li><li><strong>OS cache</strong> — has any other app on this machine resolved it recently?</li><li><strong>Router/local resolver cache</strong> — has anyone else on my network already asked this?</li><li><strong>ISP’s recursive resolver</strong> — the first place that actually does the real work.</li></ol><p>If none of these have an answer ready, the recursive resolver sets off on a proper hunt through the DNS hierarchy:</p><ul><li>It asks a <strong>root nameserver</strong>: “Who is handling .com?&quot;</li><li>The root server points it to the <strong>TLD nameserver</strong> for .com.</li><li>The TLD server, in turn, points it to the <strong>authoritative nameserver</strong> for example.com.</li><li>The authoritative server finally gives the answer: “blog.example.com is 93.184.216.34.&quot;</li></ul><p>This answer arrives with a <strong>TTL (time to live)</strong> attached — basically the number of seconds it’s allowed to sit in cache — so that the next lookup is almost instant.</p><blockquote><strong><em>Analogy:</em></strong><em> Think of calling directory assistance for a phone number. They don’t know it off the top of their head, so they check with a national operator, who knows which regional operator handles that particular area code, who finally has the actual number. DNS works in exactly the same fashion — a chain of referrals, moving from general to specific.</em></blockquote><p>This entire process usually wraps up in a matter of tens of milliseconds. Your browser now has an IP address in hand. But there’s a small catch — that server could physically be anywhere on the planet. So how does your request actually reach it?</p><h3>Step 3: Routing — Finding a Path Across the Internet</h3><p>Here’s the part that’s genuinely a bit of a marvel: <strong>there is no single wire that connects you directly to that server.</strong> The internet is really a mesh of independent networks — your ISP, backbone providers, undersea cables, data-centre networks — all working together somehow.</p><p>Your request gets broken down into <strong>packets</strong>, and each one is stamped with the destination IP. From there, it begins a relay race across <strong>routers</strong>, whose entire job is to look at where a packet needs to go and decide, “what’s the best next hop to get it a little closer?”</p><p>A couple of ideas make this whole thing work at such a massive scale:</p><ul><li><strong>IP addresses are hierarchical</strong>, much like our postal PIN codes. Routers don’t need to know the exact location of every single address on Earth — just the general direction, the same way a mail sorter only needs to know “anything headed south goes on this particular truck.”</li><li><strong>BGP (Border Gateway Protocol)</strong> is how the large networks announce to each other which address ranges they’re able to reach. It’s the negotiation layer that stitches together thousands of independent networks into what we call one internet.</li><li><strong>Packets can take completely different paths</strong> and even arrive out of order. That’s fine, actually — putting them back together is a job for the next layer up (which we’ll get to shortly).</li></ul><p>You can watch this relay race with your own eyes. Just run traceroute google.com (on macOS/Linux) or tracert google.com (on Windows), and you&#39;ll see every hop your packets take, along with the time taken to reach each one.</p><p>So packets can find their way, alright. But finding a way isn’t the same thing as having a dependable conversation. For that, we need a proper handshake.</p><h3>Step 4: The TCP Handshake — Establishing a Reliable Connection</h3><p>IP gets packets to the right destination, but it doesn’t guarantee anything — packets can get lost, duplicated, or arrive all jumbled up. <strong>TCP (Transmission Control Protocol) sits on top of IP to add reliability</strong> — it makes sure the bytes you send reach the other side complete and in the right order.</p><p>Before any actual data moves, both machines go through the well-known <strong>three-way handshake</strong>:</p><ol><li><strong>SYN</strong> → Your computer: “I’d like to talk. Here’s my starting sequence number.”</li><li><strong>SYN-ACK</strong> ← Server: “Noted. Acknowledging yours, and here’s mine as well.”</li><li><strong>ACK</strong> → Your computer: “Acknowledged. Let’s proceed.”</li></ol><pre>Client                         Server<br>  |------------ SYN -----------&gt;|<br>  |&lt;-------- SYN-ACK -----------|<br>  |------------ ACK -----------&gt;|<br>  |                             |<br>  |====== connection open ======|</pre><p>Once these three messages are exchanged, both sides are in agreement on sequence numbers and are ready to exchange data reliably. This handshake alone costs <strong>one full round trip</strong>, even before a single byte of your actual request goes out — which is exactly why physical distance and latency end up mattering so much for performance.</p><blockquote><strong><em>Analogy:</em></strong><em> It’s like a phone call, not a letter. You don’t start speaking the moment you dial — you wait for the other person to pick up, say “hello”, and hear them say “hi” back. Only then do both of you know the line is open and it’s safe to talk.</em></blockquote><p>If you had typed http://, the browser could go ahead and make its request right now itself. But you typed https:// — and that little s means there&#39;s one more, rather important step still left.</p><h3>Step 5: The TLS Handshake — Making It Private and Trusted</h3><p>The s in HTTPS stands for <strong>secure</strong>, and this is delivered by <strong>TLS (Transport Layer Security)</strong>. Without it, everything you send would travel as plain text, readable or even alterable by any network sitting along the path. TLS handles three problems in one go:</p><ul><li><strong>Encryption</strong> — nobody in between can read your data.</li><li><strong>Authentication</strong> — you can actually confirm that the server is truly example.com, and not some impostor.</li><li><strong>Integrity</strong> — nobody can quietly tamper with the data while it’s in transit.</li></ul><p>Once the TCP handshake is done, a second handshake kicks off:</p><ol><li><strong>Client Hello</strong> — your browser states which TLS versions and cipher suites it supports, along with a random number.</li><li><strong>Server Hello + Certificate</strong> — the server picks the encryption settings and sends across its <strong>TLS certificate</strong>, which carries its public key.</li><li><strong>Certificate verification</strong> — your browser checks this certificate against a chain of trusted <strong>Certificate Authorities (CAs)</strong> that are already baked into your OS/browser. Is it signed by someone trustworthy? Is it meant for the right domain? Has it expired, or been revoked?</li><li><strong>Key exchange</strong> — both sides use public-key cryptography to agree upon a shared <strong>session key</strong>, without ever actually sending it out in the open.</li><li><strong>Finished</strong> — from this point on, everything is encrypted using that fast, shared symmetric key.</li></ol><blockquote><strong><em>Analogy:</em></strong><em> Before discussing anything sensitive with a stranger who claims to be from your bank, you’d first check their ID against a trusted authority (the CA), and then the two of you agree on a secret code language that only you both understand. Everyone else just hears gibberish.</em></blockquote><p>Modern <strong>TLS 1.3</strong> has simplified all this down to a single round trip (and sometimes even zero, for repeat visits). This certificate check is what powers that small padlock icon in your address bar — and its absence is exactly what triggers those alarming “Your connection is not private” warnings.</p><p>Now — and only now — is the channel actually secure. Your browser can finally ask for what it originally came for.</p><h3>Step 6: The HTTP(S) Request — Finally Asking for the Page</h3><p>With a reliable, encrypted, authenticated connection now open, your browser sends across an <strong>HTTP request</strong>. Stripped of its encryption, it would look something like this:</p><pre>GET /articles/networking HTTP/2<br>Host: blog.example.com<br>User-Agent: Mozilla/5.0 ...<br>Accept: text/html<br>Accept-Encoding: gzip, br<br>Cookie: session_id=abc123</pre><p>A few key ingredients here:</p><ul><li><strong>Method</strong> (GET, POST, PUT, DELETE, and so on) — what exactly you want to do.</li><li><strong>Path</strong> — which resource you’re after.</li><li><strong>Headers</strong> — metadata, such as which formats you accept, your cookies, caching hints, and who you are.</li></ul><h3>Step 7: The Backend Does Its Work</h3><p>Your request usually doesn’t land on just one server, but on an entire pipeline:</p><ol><li><strong>Load balancer</strong> — spreads incoming requests across several servers so that no single machine gets overloaded. It may also handle the TLS termination from the previous step.</li><li><strong>Web/application server</strong> — routes the request to whichever code handles that particular path.</li><li><strong>Business logic</strong> — the application actually runs: checking your session cookie, applying its rules, and deciding what data you’re allowed to see.</li><li><strong>Database &amp; caches</strong> — the app queries a database (or, quicker still, an in-memory cache like Redis) to fetch the content.</li><li><strong>Response assembly</strong> — the server puts together a response: HTML, JSON, an image, whatever was asked for.</li></ol><p>Along the way, a <strong>CDN (Content Delivery Network)</strong> may have already answered part or all of the request from a server physically closer to you — which is exactly why global websites feel fast no matter which part of the world you’re browsing from.</p><p>The server then sends back an <strong>HTTP response</strong>:</p><pre>HTTP/2 200 OK<br>Content-Type: text/html; charset=utf-8<br>Content-Encoding: gzip<br>Cache-Control: max-age=3600</pre><pre>&lt;!DOCTYPE html&gt;<br>&lt;html&gt; ... &lt;/html&gt;</pre><p>The <strong>status code</strong> tells you the story at a glance: 200 (success), 301 (moved), 404 (not found), 500 (server broke down). The <strong>headers</strong> guide the browser on how to interpret and cache what follows. And the <strong>body</strong> is the actual content itself.</p><h3>Step 8: The Browser Renders the Page</h3><p>Getting the HTML back is really just the beginning of what you actually <em>see</em> on screen:</p><ol><li>The browser <strong>parses the HTML</strong> into a <strong>DOM (Document Object Model)</strong> tree.</li><li>It spots references to more resources — CSS, JavaScript, images, fonts — and <strong>fires off fresh requests for each one</strong> (many of which repeat steps 2–7 above, though the connection and DNS lookup are often reused).</li><li><strong>CSS</strong> gets parsed into style rules; <strong>JavaScript</strong> is downloaded and run, which may go on to change the page further.</li><li>The browser then works out the layout (where everything should sit), and finally <strong>paints</strong> the pixels onto your screen.</li></ol><p>This is exactly why a page often appears in stages — text first, then images, and interactivity somewhere after. The browser renders whatever it can, as soon as it possibly can.</p><h3>The Whole Journey, at a Glance</h3><pre>You type a URL<br>      │<br>      ▼<br>[1] Browser parses URL  ──►  extracts host, checks HSTS<br>      │<br>      ▼<br>[2] DNS lookup          ──►  name  →  IP address<br>      │<br>      ▼<br>[3] Routing             ──►  packets hop across networks<br>      │<br>      ▼<br>[4] TCP handshake       ──►  reliable connection (SYN / SYN-ACK / ACK)<br>      │<br>      ▼<br>[5] TLS handshake       ──►  encrypted + authenticated channel<br>      │<br>      ▼<br>[6] HTTP request        ──►  &quot;GET /the-page&quot;<br>      │<br>      ▼<br>[7] Backend processing  ──►  load balancer → app → database → response<br>      │<br>      ▼<br>[8] Browser renders     ──►  HTML → DOM → paint → the page you see</pre><h3>Why This Matters (Even If Nobody Ever Asks You This in an Interview)</h3><p>Understanding this entire pipeline genuinely changes how you debug and build things:</p><ul><li><strong>Page loading slow?</strong> Now you know exactly where to look — DNS resolution, TLS negotiation, backend query time, or rendering are all separately measurable in your browser’s DevTools Network tab.</li><li><strong>Getting a security warning?</strong> You now understand it’s the TLS certificate check failing, and not just some random glitch.</li><li><strong>Designing a system of your own?</strong> You understand why CDNs, caching, and keeping connections alive make such a big difference — every round trip you manage to eliminate is real, physical time saved for the user.</li></ul><p>The next time a page loads in under a second, you’ll know it wasn’t magic at all. It was a name resolved, a path found, a connection opened, an identity verified, and a conversation held — all quicker than you could blink.</p><p><em>If this helped make the web feel a little less mysterious, do follow along — this is part of a series where I break down the systems that software engineers use daily but rarely stop to actually examine.</em></p><p><em>Which part of this journey surprised you the most? Do drop a comment — It will be more helpful for me to understand break it further down.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=65ec3b8794ff" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>