<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Satyam Choudhary on Medium]]></title>
        <description><![CDATA[Stories by Satyam Choudhary on Medium]]></description>
        <link>https://medium.com/@chsatyam?source=rss-3c0b02a60007------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*l6kW2Hgo_yoCfjob</url>
            <title>Stories by Satyam Choudhary on Medium</title>
            <link>https://medium.com/@chsatyam?source=rss-3c0b02a60007------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 15:41:42 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@chsatyam/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[MCP Explained for Beginners: Why Does It Exist If We Already Have APIs?]]></title>
            <link>https://chsatyam.medium.com/mcp-explained-for-beginners-why-does-it-exist-if-we-already-have-apis-c8e10c4f2f36?source=rss-3c0b02a60007------2</link>
            <guid isPermaLink="false">https://medium.com/p/c8e10c4f2f36</guid>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[mcp-server]]></category>
            <category><![CDATA[mcp-client]]></category>
            <category><![CDATA[mcp-protocol]]></category>
            <category><![CDATA[mcps]]></category>
            <dc:creator><![CDATA[Satyam Choudhary]]></dc:creator>
            <pubDate>Mon, 13 Jul 2026 19:17:46 GMT</pubDate>
            <atom:updated>2026-07-13T19:17:46.117Z</atom:updated>
            <content:encoded><![CDATA[<p>If you’ve recently started exploring AI agents, Claude Desktop, Cursor, Windsurf, or AI coding assistants, you’ve probably heard about <strong>MCP (Model Context Protocol)</strong>.</p><p>And chances are your first reaction was:</p><blockquote><em>“Wait… if my AI app can already call APIs directly, then why do we need MCP?”</em></blockquote><p>Honestly, that’s exactly the right question.</p><p>This article explains MCP from a software engineer’s perspective and answers almost every beginner doubt around it.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-_p6vph7ZumUrXGu-H5srw.png" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c8e10c4f2f36" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Understanding RAG (Retrieval-Augmented Generation) — A Beginner-Friendly Guide for Software…]]></title>
            <link>https://chsatyam.medium.com/understanding-rag-retrieval-augmented-generation-a-beginner-friendly-guide-for-software-eb4e0d4a442a?source=rss-3c0b02a60007------2</link>
            <guid isPermaLink="false">https://medium.com/p/eb4e0d4a442a</guid>
            <category><![CDATA[generative-ai]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[retrieval-augmented-gen]]></category>
            <category><![CDATA[software-engineering]]></category>
            <category><![CDATA[rags]]></category>
            <dc:creator><![CDATA[Satyam Choudhary]]></dc:creator>
            <pubDate>Mon, 13 Jul 2026 19:02:30 GMT</pubDate>
            <atom:updated>2026-07-13T19:02:30.523Z</atom:updated>
            <content:encoded><![CDATA[<h3>Understanding RAG (Retrieval-Augmented Generation) — A Beginner-Friendly Guide for Software Engineers</h3><p>If you’ve recently started exploring AI, you’ve probably heard terms like <strong>RAG</strong>, <strong>Fine-tuning</strong>, <strong>Vector Databases</strong>, and <strong>Embeddings</strong>.</p><p>For many beginners, RAG sounds much more complicated than it actually is.</p><p>This article explains RAG from a software engineer’s perspective using simple examples and without unnecessary AI jargon.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*52kld0c4yfcgoVZZpvuXxA.png" /></figure><h3>What is RAG ?</h3><p><strong>RAG stands for Retrieval-Augmented Generation.</strong></p><p>In simple words:</p><blockquote>It is used to help AI models provide more accurate and up-to-date answers.</blockquote><p>Think of an AI model like a student:</p><ul><li><strong>Normal AI model</strong> → Gives answers only based on what it learned during training. If you ask a question about something it did not learn during training, it may not know the correct answer. In such cases, there is a higher chance that the model may hallucinate and provide incorrect information.</li><li><strong>RAG-based AI model</strong> → Gives answers based on both its training knowledge and additional information retrieved from an external database that was added after training. If the model does not know the answer from its training data, it searches the relevant information database first. If it finds useful information there, it uses that information to generate a more accurate answer instead of relying only on its memory.</li></ul><p>Before answering, a RAG-based model first searches for relevant information in the database and then uses that information to generate its response.</p><p>Think of it like this:</p><ul><li>A normal LLM answers using its memory (training data).</li><li>A RAG-enabled LLM first looks for relevant information in your documents or database and then generates an answer using that information.</li></ul><h3>Normal LLM vs RAG</h3><h3>Normal LLM</h3><p>Imagine GPT as a student in an exam hall.</p><p>Teacher asks:</p><blockquote><em>What is the capital of India ?</em></blockquote><p>Student answers:</p><blockquote><em>Delhi.</em></blockquote><p>The answer came from the student’s memory.</p><h3>RAG-based LLM</h3><p>Teacher asks:</p><blockquote><em>What is employee 123’s salary ?</em></blockquote><p>The student doesn’t know the answer.</p><p>So someone hands him a paper:</p><pre>Employee 123 → Salary: 15 LPA</pre><blockquote>The student reads the paper and answers:</blockquote><blockquote><em>Employee 123’s salary is 15 LPA.</em></blockquote><p>The student didn’t learn the information permanently.</p><p>He simply used the information that was provided at answer time.</p><p>That’s RAG.</p><h3>Why Do We Need RAG?</h3><p>Suppose you already have a trained LLM like:</p><ul><li>GPT</li><li>Llama</li><li>Gemini</li><li>Claude</li></ul><p>Now you want this model to answer questions using your company’s internal data:</p><ul><li>HR policies</li><li>PDFs</li><li>Product documentation</li><li>Employee information</li><li>Internal wiki pages</li><li>Customer support articles</li></ul><p>You have two options.</p><h3>Option 1: Fine-tune the model</h3><p>You retrain the model using your company’s data.</p><h3>Problems:</h3><ul><li>Expensive</li><li>Time-consuming</li><li>Requires GPUs</li><li>Difficult to maintain</li><li>Every time your data changes, retraining may be required</li></ul><p>Imagine retraining GPT every time HR changes leave policy.</p><p>Clearly not practical.</p><h3>Option 2: Use RAG</h3><p>This is what almost everyone does in production.</p><p>Instead of retraining:</p><ul><li>Store company data separately.</li><li>Search relevant information when a question arrives.</li><li>Send that information to the LLM.</li><li>Let the LLM generate the final response.</li></ul><p>No retraining required.</p><h3>The Biggest Misconception About RAG (Easy Explanation)</h3><p>Many beginners think:</p><blockquote><em>“We give our company data to the AI once, and now it remembers it forever.”</em></blockquote><p>This is not true.</p><p>With <strong>RAG</strong>, the AI does <strong>not</strong> permanently store or learn your company data.</p><p>Think of the AI as a student:</p><ul><li>A <strong>normal student</strong> answers questions only from memory.</li><li>A <strong>RAG student</strong> is allowed to open a book and look up information before answering.</li></ul><p>So when a question comes:</p><pre>Question arrives<br>      ↓<br>The system searches the company documents or database<br>      ↓<br>The required information is found<br>      ↓<br>That information is given to the AI<br>      ↓<br>The AI uses it to generate the answer</pre><p>After answering, the AI does <strong>not</strong> keep that information in its memory forever.</p><h3>Simple one-line definition:</h3><blockquote><em>RAG does not teach new information to the AI permanently; it simply gives the AI the right information at the right time.</em></blockquote><h3>Example</h3><p>Suppose your company has a document:</p><pre>leave_policy.txt</pre><pre>WFH allowed: 3 days per week<br>Annual leaves: 24 per year</pre><p>A user asks:</p><blockquote><em>How many WFH days are allowed?</em></blockquote><p>The system searches the document and finds:</p><pre>WFH allowed: 3 days per week</pre><p>Then the LLM receives something like:</p><pre>Context:<br>WFH allowed: 3 days per week</pre><p>Question:<br>How many WFH days are allowed?</p><pre>Answer only using the context above.</pre><p>The model responds:</p><pre>Employees are allowed to work from home for 3 days per week.</pre><p>Notice something important:</p><p>The model never learned your leave policy.</p><p>It simply used the information it was given.</p><h3>Wait… Is This Prompt Engineering?</h3><p>Yes.<br>Let’s Understand How -</p><p>Suppose you ask an AI:</p><pre>&quot;How many WFH days are allowed?&quot;</pre><p>The AI by itself does not know your company’s WFH policy because it was not trained on your company’s internal data.</p><p>So before sending your question to the AI, the application does something behind the scenes:</p><ol><li>It searches the company documents.</li><li>It finds:</li></ol><pre>&quot;WFH allowed: 3 days per week&quot;</pre><p>3. It secretly sends both the information and your question to the AI:</p><pre>Context:<br>WFH allowed: 3 days per week<br><br>Question:<br>How many WFH days are allowed?</pre><p>The AI then answers:</p><blockquote><strong><em>“Employees are allowed to work from home for 3 days per week.”</em></strong></blockquote><h3>Why is this called Prompt Engineering?</h3><p>Because the <strong>original user prompt</strong> was only:</p><blockquote><em>“How many WFH days are allowed?”</em></blockquote><p>But the application <strong>modified the prompt</strong> by adding extra information (<strong>context</strong>) before sending it to the AI.</p><p>So the actual prompt that reached the AI was:</p><pre>WFH allowed: 3 days per week<br><br>Question:<br>How many WFH days are allowed?</pre><p>This process of adding extra information to the prompt is called:</p><ul><li><strong>Context Injection</strong></li><li><strong>Prompt Augmentation</strong></li><li><strong>Prompt Enrichment</strong></li></ul><p>These are different names for the same idea:</p><blockquote><strong><em>Adding relevant information to the prompt before giving it to the LLM.</em></strong></blockquote><h3>Simple analogy</h3><p>Imagine you ask a student:</p><blockquote><em>“What is our company’s WFH policy?”</em></blockquote><p>The student doesn’t know.</p><p>So you hand them a paper that says:</p><blockquote><em>“WFH allowed: 3 days per week”</em></blockquote><p>and then ask the question again.</p><p>The student reads the paper and answers:</p><blockquote><em>“3 days per week.”</em></blockquote><p>You didn’t teach the student permanently. You just gave them the information they needed for that one question.</p><p>That is essentially how RAG works.</p><h3>So Where Does Retrieval Happen?</h3><p>Suppose we have:</p><pre>Employee ID 123 → Salary 15 LPA<br>Employee ID 124 → Salary 20 LPA<br>Employee ID 125 → Salary 10 LPA</pre><h3>So Where Does Retrieval Happen?</h3><p>Suppose we have:</p><pre>Employee ID 123 → Salary 15 LPA<br>Employee ID 124 → Salary 20 LPA<br>Employee ID 125 → Salary 10 LPA</pre><p>User asks :</p><blockquote><em>What is the salary of employee 123?</em></blockquote><p>The system first retrieves:</p><pre>Employee ID 123 → Salary 15 LPA</pre><p>This step is called <strong>Retrieval</strong>.</p><h3>What is Augmentation?</h3><p>The retrieved data is added to the prompt:</p><pre>Employee Data:<br>Employee ID 123 → Salary 15 LPA<br><br>Question:<br>What is employee 123&#39;s salary?</pre><p>This step is called <strong>Augmentation</strong>.</p><h3>What is Generation?</h3><p>The LLM reads the prompt and generates:</p><pre>Employee 123&#39;s salary is 15 LPA.</pre><p>This is <strong>Generation</strong>.</p><p>Complete RAG Pipeline</p><pre>User Question<br>      ↓<br>Retrieve relevant information<br>      ↓<br>Add information to prompt<br>      ↓<br>Send prompt to LLM<br>      ↓<br>Generate answer</pre><p>Or visually:</p><pre>User Query<br>    ↓<br>Retriever<br>    ↓<br>Relevant Documents<br>    ↓<br>Prompt Augmentation<br>    ↓<br>LLM<br>    ↓<br>Final Response</pre><h3>Does RAG Require API Calls?</h3><p>This is another common beginner question.</p><p>The answer is:</p><pre>No. RAG and API calls are different concepts.</pre><h3>Real-World RAG Applications</h3><p>RAG powers:</p><ul><li>Company chatbots</li><li>Customer support assistants</li><li>Internal knowledge bases</li><li>Legal document search</li><li>Medical document assistants</li><li>Code assistants</li><li>Documentation Q&amp;A systems</li></ul><p>Whenever you see:</p><blockquote><em>“Chat with your PDFs”</em></blockquote><p>You are almost certainly looking at a RAG application.</p><h3>One-Line Definition of RAG</h3><p>If you remember only one sentence from this article, remember this:</p><blockquote><em>RAG does not teach new information to the LLM. It simply gives the LLM the relevant information at answer time.</em></blockquote><p>That single sentence explains almost everything about RAG</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=eb4e0d4a442a" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How AI Engineers Use Hugging Face, OpenHands, and Ollama to Build the Future]]></title>
            <link>https://chsatyam.medium.com/how-ai-engineers-use-hugging-face-openhands-and-ollama-to-build-the-future-1b170e6be939?source=rss-3c0b02a60007------2</link>
            <guid isPermaLink="false">https://medium.com/p/1b170e6be939</guid>
            <category><![CDATA[ollama]]></category>
            <category><![CDATA[open-hand]]></category>
            <category><![CDATA[ai-engineering]]></category>
            <category><![CDATA[software-engineering]]></category>
            <category><![CDATA[hugging-face]]></category>
            <dc:creator><![CDATA[Satyam Choudhary]]></dc:creator>
            <pubDate>Sun, 12 Jul 2026 09:41:00 GMT</pubDate>
            <atom:updated>2026-07-12T09:41:00.221Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*MaDbXMRDYKeEFZ5GFlRWiQ.png" /><figcaption>From Models to Agents: The Open-Source AI Tools Changing Software Engineering</figcaption></figure><p>Artificial Intelligence is no longer limited to chatbots and recommendation systems.</p><p>Today’s AI can write code, debug applications, analyze documents, generate images, conduct research, and even collaborate with humans as a software engineering partner.</p><p>What makes this revolution different from previous technological shifts is that much of it is happening in the open.</p><p>A few years ago, cutting-edge AI technologies were available only to large corporations with enormous budgets and massive computing resources.</p><p>Today, students, startups, independent developers, and researchers can build powerful AI applications using open-source tools that are freely available to everyone.</p><p>This article explores some of the most important open-source AI projects and explains how they fit together to create the modern AI ecosystem.</p><h3>1. OpenHands — The AI Software Engineer</h3><p>OpenHands represents one of the biggest shifts in software development.</p><p>Traditional coding assistants generate code snippets or autocomplete suggestions.</p><p>OpenHands goes much further.</p><p>It acts as an AI software engineering agent capable of:</p><ul><li>Understanding an entire repository</li><li>Reading project documentation</li><li>Identifying bugs</li><li>Modifying files</li><li>Running tests</li><li>Executing terminal commands</li><li>Refactoring code</li><li>Implementing features</li></ul><p>Instead of asking:</p><blockquote><em>“How do I write this function?”</em></blockquote><p>developers can now ask:</p><blockquote><em>“Fix the authentication bug and make sure all tests pass.”</em></blockquote><p>OpenHands attempts to solve the entire problem rather than simply generating code suggestions.</p><p>This changes the role of AI from being a coding assistant into becoming a collaborative software engineer.</p><p>Potential use cases include:</p><ul><li>Bug fixing</li><li>Legacy code maintenance</li><li>Feature development</li><li>Automated testing</li><li>Documentation generation</li><li>Repository onboarding</li></ul><p>Many developers describe OpenHands as the closest thing we currently have to an “AI junior developer.”</p><h3>2. Hugging Face — The GitHub of AI</h3><p>If OpenHands is an AI engineer, Hugging Face is the place where AI engineers get their tools.</p><p>Hugging Face is the world’s largest platform for:</p><ul><li>AI models</li><li>Datasets</li><li>Research papers</li><li>Model evaluation</li><li>AI demonstrations</li><li>Inference APIs</li></ul><p>Millions of developers use Hugging Face every day to discover and share models.</p><p>Whether you want to build:</p><ul><li>Chatbots</li><li>Translation systems</li><li>Voice assistants</li><li>Search engines</li><li>Recommendation systems</li><li>Image generation applications</li></ul><p>there is a good chance that someone has already published a model you can use immediately.</p><p>Some of the most popular open-source models available through Hugging Face include:</p><ul><li>Llama</li><li>Mistral</li><li>DeepSeek</li><li>Qwen</li><li>Gemma</li></ul><p>Hugging Face democratized access to AI in the same way GitHub democratized access to software development.</p><h3>3. Ollama — Running AI Locally</h3><p>One of the biggest concerns around AI is cost and privacy.</p><p>Cloud APIs require sending your data to external servers and paying for every request.</p><p>Ollama offers a different approach.</p><p>It allows developers to run large language models directly on their own machines.</p><p>With a single command:</p><pre>ollama run llama3</pre><p>a fully functional language model can run locally on your laptop.</p><p>Advantages include:</p><h3>Privacy</h3><p>Sensitive company data never leaves your machine.</p><h3>Cost Reduction</h3><p>No API charges.</p><h3>Offline Usage</h3><p>Models continue working without internet access.</p><h3>Customization</h3><p>Developers can fine-tune and customize local deployments.</p><p>Ollama has become one of the foundations of the self-hosted AI movement.</p><h3>Multi-SWE-Bench — The Exam for AI Software Engineers</h3><p>How do we know whether an AI coding agent is actually good?</p><p>Benchmarks provide the answer.</p><p>Multi-SWE-Bench evaluates AI systems using real-world software engineering tasks.</p><p>Instead of simple coding questions, it tests models on:</p><ul><li>Real repositories</li><li>Real GitHub issues</li><li>Real failing tests</li><li>Real codebases</li></ul><p>The benchmark measures:</p><ul><li>Problem understanding</li><li>File selection</li><li>Code quality</li><li>Test success rate</li><li>Production readiness</li></ul><p>As coding agents become more powerful, benchmarks like Multi-SWE-Bench will become increasingly important.</p><h3>Now Let’s Understand How You Can Use Multi-SWE-Bench, OpenHands, and Hugging Face as an AI Engineer</h3><p>As an AI Engineer, your work is not limited to building models anymore.</p><p>Modern AI engineering involves selecting the right models, integrating them into applications, evaluating their performance, and continuously improving them.</p><p>This is where tools like <strong>Hugging Face</strong>, <strong>OpenHands</strong>, and <strong>Multi-SWE-Bench</strong> become incredibly useful.</p><ul><li><strong>Hugging Face</strong> → Find and deploy AI models.</li><li><strong>OpenHands</strong> → Use AI agents to automate software engineering tasks.</li><li><strong>Multi-SWE-Bench</strong> → Measure and benchmark AI engineering performance.</li></ul><p>Let’s understand each of them with real-world examples.</p><h3>1. Hugging Face — Your AI Model Marketplace</h3><p>Imagine your company wants to build a customer support chatbot.</p><p>Ten years ago, your team would have needed:</p><ul><li>Massive datasets</li><li>Large GPU clusters</li><li>Months of model training</li><li>A dedicated machine learning team</li></ul><p>Today, an AI Engineer can simply visit Hugging Face and choose from thousands of pre-trained models.</p><p>For example:</p><p>You need:</p><ul><li>A chatbot.</li><li>Document summarization.</li><li>Text classification.</li><li>Image generation.</li><li>Speech recognition.</li></ul><p>Instead of building everything from scratch, you can simply use existing models.</p><h3>Example</h3><p>Suppose you want to create an AI-powered resume screening system.</p><p>You can use:</p><ul><li>A language model for understanding resumes.</li><li>An embedding model for semantic search.</li><li>A reranking model for candidate matching.</li></ul><p>Most likely, all three models already exist on Hugging Face.</p><p>Your work as an AI Engineer becomes:</p><blockquote><em>Selecting the right model instead of training a new one.</em></blockquote><p>This dramatically reduces development time.</p><p>What previously took months can now be done in days.</p><h3>In simple words:</h3><p>Hugging Face is where AI Engineers go shopping for models.</p><h3>2. OpenHands — Your AI Software Engineer</h3><p>Building AI applications involves a lot more than machine learning.</p><p>You still need to:</p><ul><li>Write APIs</li><li>Fix bugs</li><li>Update documentation</li><li>Refactor code</li><li>Write tests</li><li>Maintain repositories</li></ul><p>OpenHands acts as an AI software engineer that can perform many of these tasks autonomously.</p><h3>Example</h3><p>Suppose your Retrieval-Augmented Generation application suddenly starts failing.</p><p>Users report:</p><blockquote><em>“Uploaded PDFs are not appearing in search results.”</em></blockquote><p>Instead of manually inspecting the entire repository, you can ask OpenHands:</p><blockquote><em>Investigate why uploaded documents are missing from the vector database and fix the issue.</em></blockquote><p>OpenHands can:</p><ul><li>Explore the repository.</li><li>Read the ingestion pipeline.</li><li>Identify the bug.</li><li>Modify files.</li><li>Run tests.</li><li>Validate the solution.</li></ul><p>This changes your role from:</p><blockquote><em>Writing every line of code manually</em></blockquote><p>to:</p><blockquote><em>Managing and supervising AI agents.</em></blockquote><h3>Another Example</h3><p>Imagine your startup wants to add authentication to an AI application.</p><p>You can ask OpenHands:</p><blockquote><em>Add Google OAuth login support and create the necessary API endpoints.</em></blockquote><p>The agent may:</p><ul><li>Create backend routes.</li><li>Update frontend pages.</li><li>Install dependencies.</li><li>Write tests.</li><li>Update documentation.</li></ul><p>This is similar to assigning work to a junior developer.</p><h3>3. Multi-SWE-Bench — Your AI Evaluation Framework</h3><p>One of the biggest problems in AI engineering is evaluation.</p><p>Suppose you have two coding agents:</p><ul><li>Agent A</li><li>Agent B</li></ul><p>Which one is better?</p><p>A few demo examples are not enough.</p><p>You need standardized evaluation.</p><p>This is exactly what Multi-SWE-Bench provides.</p><h3>Example</h3><p>You want to compare:</p><ul><li>OpenHands using Claude.</li><li>OpenHands using DeepSeek.</li><li>OpenHands using Qwen.</li></ul><p>You run all three systems against Multi-SWE-Bench.</p><p>The benchmark provides:</p><ul><li>Real GitHub repositories.</li><li>Real issues.</li><li>Real failing tests.</li></ul><p>Now you can measure:</p><pre>Metric                        Claude   DeepSeek  Qwen<br>Issues solved                   71%     64%      58%<br>Tests passed                   92%      85%      79%<br>Average completion time       12min     15 min   18 min</pre><p>Now your decision becomes data-driven rather than opinion-driven.</p><h3>Another Example</h3><p>Suppose your company builds an internal AI coding assistant.</p><p>Before deploying it to developers, you evaluate it using Multi-SWE-Bench.</p><p>This helps answer questions such as:</p><ul><li>Can it understand large repositories?</li><li>Can it modify multiple files correctly?</li><li>Does it break existing functionality?</li><li>Can it solve production-level issues?</li></ul><p>Without benchmarks, it is impossible to answer these questions objectively.</p><h3>How an AI Engineer Uses All Three Together</h3><p>Now let’s connect everything.</p><p>Imagine you are building an AI-powered coding assistant.</p><h3>Step 1: Select Models</h3><p>You visit Hugging Face and choose:</p><ul><li>A coding model.</li><li>An embedding model.</li><li>A reranking model.</li></ul><h3>Step 2: Build the Product</h3><p>You integrate those models into your application.</p><p>OpenHands helps you:</p><ul><li>Write APIs.</li><li>Fix bugs.</li><li>Generate tests.</li><li>Improve documentation.</li></ul><h3>Step 3: Measure Performance</h3><p>You use Multi-SWE-Bench to evaluate:</p><ul><li>Accuracy</li><li>Repository understanding</li><li>Code quality</li><li>Bug-fixing capabilities</li></ul><h3>Final Workflow</h3><pre>Hugging Face<br>      ↓<br>Choose Models<br>      ↓<br>OpenHands<br>      ↓<br>Build Applications<br>      ↓<br>Multi-SWE-Bench<br>      ↓<br>Measure Performance</pre><h3>The New Role of an AI Engineer</h3><p>Five years ago, AI Engineers spent most of their time training models.</p><p>Today, the role is changing.</p><p>Modern AI Engineers spend more time:</p><ul><li>Selecting models</li><li>Building workflows</li><li>Orchestrating AI systems</li><li>Evaluating performance</li><li>Improving reliability</li></ul><p>The future AI Engineer is not simply someone who trains models.</p><p>The future AI Engineer is someone who knows how to combine tools like Hugging Face, OpenHands, and Multi-SWE-Bench into complete AI systems.</p><p>And that shift has already begun.</p><h3>The Future of AI</h3><p>The future will not be defined by humans competing against AI.</p><p>It will be defined by humans working with AI.</p><p>Developers who understand how to collaborate with AI systems will build products faster, solve problems more efficiently, and innovate at a scale that was previously impossible.</p><p>Just as calculators did not replace mathematicians, AI will not replace engineers.</p><p>Instead, it will amplify what engineers are capable of building.</p><p>OpenHands, Hugging Face, Ollama, LangGraph, CrewAI, Open Interpreter, vLLM, Ray, and Multi-SWE-Bench are early examples of this future.</p><p>And perhaps for the first time in history, the technologies shaping that future are open for everyone to explore, modify, and improve.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1b170e6be939" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Transformers Explained ( Attention is all you need ) : “ How Modern AI Actually Understands…]]></title>
            <link>https://chsatyam.medium.com/transformers-explained-attention-is-all-you-need-how-modern-ai-actually-understands-9f80799ccad7?source=rss-3c0b02a60007------2</link>
            <guid isPermaLink="false">https://medium.com/p/9f80799ccad7</guid>
            <category><![CDATA[transformer-architecture]]></category>
            <category><![CDATA[chatgpt]]></category>
            <category><![CDATA[transformers]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[ai-agent]]></category>
            <dc:creator><![CDATA[Satyam Choudhary]]></dc:creator>
            <pubDate>Thu, 09 Jul 2026 17:36:57 GMT</pubDate>
            <atom:updated>2026-07-09T17:36:57.873Z</atom:updated>
            <content:encoded><![CDATA[<h3>Transformers Explained ( Attention is all you need ) : “ How Modern AI Actually Understands Language ” With and Example :</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zp-kWPOy1mz7SlZzzK6oKg.png" /></figure><p>The image you are seeing is the original <strong>Transformer architecture</strong> from the 2017 paper <strong>“Attention Is All You Need”</strong>. It introduced the model family that eventually led to <strong>BERT, GPT, T5, Llama, Claude, ChatGPT</strong>, and almost every modern LLM.</p><p>This technical blog article, include :</p><ol><li>Input processing</li><li>Tokenization</li><li>Embeddings</li><li>Positional Encoding</li><li>Encoder</li><li>Decoder</li><li>Attention calculations with actual numbers</li><li>Multi-head attention</li><li>Feed-forward networks</li><li>Residual connections and LayerNorm</li><li>Training</li><li>Inference (how GPT generates text)</li><li>How GPT differs from the original Transformer</li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9f80799ccad7" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Transformers Explained ( Attention is all you need ) : “ How Modern AI Actually Understands…]]></title>
            <link>https://chsatyam.medium.com/transformers-explained-attention-is-all-you-need-how-modern-ai-actually-understands-7b1537c01a03?source=rss-3c0b02a60007------2</link>
            <guid isPermaLink="false">https://medium.com/p/7b1537c01a03</guid>
            <category><![CDATA[encoder-decoder]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[transformers]]></category>
            <category><![CDATA[transformer-architecture]]></category>
            <dc:creator><![CDATA[Satyam Choudhary]]></dc:creator>
            <pubDate>Thu, 09 Jul 2026 08:05:15 GMT</pubDate>
            <atom:updated>2026-07-09T14:50:52.293Z</atom:updated>
            <content:encoded><![CDATA[<h3>Transformers Explained ( Attention is all you need ) : “ How Modern AI Actually Understands Language ”</h3><p>Part -2 <br>In the first part we covered all about inputs — tokenixation embdding , vector , postional encoding in detail , <br>now we goona covered Encode and Decoreder in part -2</p><p>In the first part, we covered everything about inputs in detail — tokenization, embeddings, vectors, and positional encoding.</p><p>Now, in Part 2, we are going to cover the Encoder and Decoder architecture.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zp-kWPOy1mz7SlZzzK6oKg.png" /></figure><h3>Encoder: This is the Encoder part</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/332/1*sJKJWTeIQU1jlSbmwptcEw.png" /></figure><p>The primary function of the encoder is to create a high-dimensional representation of the input sequence that the decoder can use to generate the output. Encoder consists of multiple layers and each layer is composed of two main sub-layers:</p><ol><li><strong>Self-Attention Mechanism</strong>: This sub-layer allows the encoder to weigh the importance of different parts of the input sequence differently to capture dependencies regardless of their distance within the sequence.</li><li><strong>Feed-Forward Neural Network</strong>: This sub-layer consists of two linear transformations with a<a href="https://www.geeksforgeeks.org/deep-learning/relu-activation-function-in-deep-learning/"> ReLU activation </a>in between. It processes the output of the self-attention mechanism to generate a refined representation.</li></ol><p>Layer normalization and residual connections are used around each of these sub-layers to ensure stability and improve convergence during training.</p><h3>What is the Encoder?</h3><p>The <strong>Encoder</strong> is the part of the Transformer that reads the input sentence, understands the relationships between words, and creates a <strong>context-aware representation</strong> of every word.</p><h3>Simple definition:</h3><blockquote><em>The encoder’s job is to understand the meaning of each word </em><strong><em>in the context of the entire sentence</em></strong><em>.</em></blockquote><p>For example:</p><pre>The animal didn&#39;t cross the street because it was tired.</pre><p>A human immediately understands that “it” = “animal”.</p><p>The encoder learns this automatically.</p><h3>Structure of One Encoder Layer</h3><p>Each encoder layer contains:</p><p>Multi-Head Self Attention<br> ↓<br>Add &amp; Norm<br> ↓<br>Feed Forward Network<br> ↓<br>Add &amp; Norm</p><p>This entire block is repeated <strong>N times</strong>.</p><p>In the original Transformer paper:</p><pre>N = 6</pre><h3>Step 1: Multi-Head Self Attention</h3><p>This is the most important part of the encoder.</p><h3>Definition</h3><blockquote><em>Self-attention allows each word to look at all other words in the sentence and decide which ones are important for understanding its meaning.</em></blockquote><h3>Real Example</h3><p>Sentence:</p><pre>The animal didn&#39;t cross the street because it was tired.</pre><p>Suppose we are processing the word:</p><pre>it</pre><p>The model asks:</p><blockquote><em>Which words help me understand what “it” refers to?</em></blockquote><p>It checks all words:</p><ul><li>animal</li><li>street</li><li>cross</li><li>tired</li><li>because</li></ul><p>The attention scores may look like this:</p><pre>| Word    | Importance for &quot;it&quot; |<br>| ------- | ------------------- |<br>| animal  | 0.80                |<br>| street  | 0.05                |<br>| cross   | 0.05                |<br>| tired   | 0.08                |<br>| because | 0.02                |</pre><p>The highest score is for <strong>animal</strong>.</p><p>Therefore the model learns:</p><pre>it → animal</pre><h3>Real Life Analogy</h3><p>Imagine you hear:</p><blockquote><em>“John gave the book to Mike because he was leaving.”</em></blockquote><p>Who is “he”?</p><p>Your brain automatically checks:</p><ul><li>John?</li><li>Mike?</li><li>Book?</li></ul><p>That is exactly what self-attention does.</p><h3>How does Self-Attention work internally?</h3><p>For every word, the model creates three vectors:</p><h3>1. Query (Q)</h3><p>Question:</p><blockquote><em>What information am I looking for?</em></blockquote><h3>2. Key (K)</h3><p>Question:</p><blockquote><em>What information do I contain?</em></blockquote><h3>3. Value (V)</h3><p>Question:</p><blockquote><em>What information should I provide if someone pays attention to me?</em></blockquote><p>Suppose we are processing:</p><pre>it</pre><p>The model creates:</p><pre>Q_it</pre><p>For all words it creates keys:</p><pre>K_animal<br>K_street<br>K_cross<br>K_tired</pre><p>The model compares:</p><pre>Q_it with K_animal<br>Q_it with K_street<br>Q_it with K_cross</pre><p>The more similar they are, the higher the attention score.</p><p>Example:</p><pre>Q_it · K_animal = 12<br>Q_it · K_street = 1<br>Q_it · K_cross = 2</pre><p>After applying softmax:</p><pre>animal = 0.80<br>street = 0.05<br>cross = 0.15</pre><h3>Final Attention Output</h3><p>The model combines information from important words:</p><pre>Output_it =<br>0.80 × V_animal<br>+<br>0.05 × V_street<br>+<br>0.15 × V_cross</pre><p>Now the representation of <strong>“it”</strong> contains information about <strong>animal</strong>.</p><p>This is why we call it a <strong>context-aware representation</strong>.</p><h3>Why Multi-Head Attention?</h3><p>One attention mechanism is not enough because a sentence contains multiple relationships.</p><p>Different heads learn different types of relationships.</p><pre>Example<br><br>Sentence:<br><br>The animal didn&#39;t cross the street because it was tired.<br><br><br>Head 1<br><br>Learns:<br><br>it → animal<br><br>Pronoun relationship.<br><br>Head 2<br><br>Learns:<br><br>animal → cross<br><br>Action relationship.<br><br>Head 3<br><br>Learns:<br><br>tired → didn&#39;t cross<br><br>Cause and effect relationship.<br><br>Head 4<br><br>Learns:<br><br>cross → street<br><br>Location relationship.<br><br>All heads work in parallel.<br><br>Then their outputs are combined.<br><br>This is called:<br><br>Multi-Head Attention</pre><h3>Step 2: Add &amp; Norm</h3><p>After attention produces an output, the encoder does:</p><pre>Output = Input + Attention_Output</pre><p>This is called a <strong>Residual Connection</strong>.</p><h3>Why add the original input again?</h3><p>Because we don’t want to lose the original information.</p><p>Think of it like this:</p><pre>Original knowledge + New knowledge</pre><p>Example:</p><p>Before attention:</p><pre>animal</pre><p>After attention:</p><pre>animal + tired + cross information</pre><p>Final representation:</p><pre>original animal meaning<br>+<br>context information</pre><h3>Real Life Example</h3><p>Imagine reading notes from a book.</p><p>Then a teacher gives additional explanation.</p><p>You don’t throw away your original notes.</p><p>You do:</p><pre>Old Notes + Teacher&#39;s Explanation</pre><p>Residual connections work exactly like that.</p><h3>Layer Normalization</h3><p>Neural networks can produce values that are extremely large or extremely small.</p><p>Example:</p><pre>5000<br>-3000<br>0.0001<br>2000</pre><p>Training becomes unstable.</p><p>Layer normalization scales values into a manageable range.</p><p>Example:</p><pre>0.5<br>-0.3<br>0.8<br>0.1</pre><p>This makes training faster and more stable.</p><h3>Step 3: Feed Forward Network (FFN)</h3><p>After understanding relationships between words, each word is processed individually by a small neural network.</p><p>Formula:</p><pre>FFN(x) = W2(ReLU(W1x + b1)) + b2</pre><h3>Simple Definition</h3><blockquote><em>FFN improves the representation of each token independently.</em></blockquote><h3>Important Point</h3><p>The FFN does <strong>not</strong> allow words to communicate with each other.</p><p>Communication happens only in the attention layer.</p><p>Example:</p><pre>animal → FFN<br>street → FFN<br>cross → FFN</pre><p>Each token is processed separately.</p><h3>Real Life Example</h3><p>Suppose attention helped you gather information from multiple books.</p><p>Now you sit alone and think about what you learned.</p><p>That thinking process is similar to FFN.</p><h3>Step 4: Add &amp; Norm Again</h3><p>After FFN:</p><pre>Output = FFN_Output + Input</pre><p>Then normalization is applied again.</p><p>Same reason:</p><ul><li>preserve original information</li><li>improve gradient flow</li><li>stabilize training</li></ul><h3>Why Repeat the Encoder Layer N Times?</h3><p>One layer understands simple relationships.</p><p>More layers understand deeper meaning.</p><pre>Example<br>Layer 1<br><br>Learns:<br><br>it → animal<br>Layer 2<br><br>Learns:<br><br>animal is tired<br>Layer 3<br><br>Learns:<br><br>animal did not cross because it was tired<br>Layer 4<br><br>Learns the complete meaning:<br><br>The animal was too tired to cross the street.<br><br>Each layer builds on top of the previous one.</pre><h3><strong>Complete Encoder Flow</strong></h3><p>Input Sentence<br> ↓<br>Tokenization<br> ↓<br>Embeddings<br> ↓<br>Positional Encoding<br> ↓<br>Multi-Head Attention<br> ↓<br>Add &amp; Norm<br> ↓<br>Feed Forward Network<br> ↓<br>Add &amp; Norm<br> ↓<br>Repeat N Times<br> ↓<br>Final Context-Aware Representations</p><h3>Final One-Line Definition</h3><blockquote><em>The Transformer Encoder converts raw input words into rich contextual representations by allowing each word to attend to all other words, refining those representations through multiple layers of attention and neural processing.</em></blockquote><h3>Now we will Move to DECODER working -</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/322/1*JPbuYwkGAqCli985bjhChQ.png" /></figure><h3>What is the Decoder?</h3><h3>Definition</h3><blockquote><em>The decoder takes the information produced by the encoder and generates the final output sequence token by token.</em></blockquote><p>For example, in machine translation:</p><p>Input (Encoder)</p><pre>I love machine learning.</pre><p>Output (Decoder)</p><pre>J&#39;aime l&#39;apprentissage automatique.</pre><p>The encoder understands the English sentence.</p><p>The decoder generates the French sentence one word at a time.</p><h3>Structure of One Decoder Layer</h3><p>Unlike the encoder, the decoder contains <strong>three major blocks</strong>:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/322/1*Pq_VL1TALAPSu2wfVyuy9A.png" /></figure><pre>Masked Multi-Head Attention<br>        ↓<br>Add &amp; Norm<br>        ↓<br>Encoder-Decoder Multi-Head Attention<br>        ↓<br>Add &amp; Norm<br>        ↓<br>Feed Forward Network<br>        ↓<br>Add &amp; Norm<br><br></pre><p>This entire decoder layer is repeated <strong>N times</strong>.</p><p>In the original Transformer paper:</p><pre>N = 6</pre><p>Decoder in transformer also consists of multiple identical layers. Its primary function is to generate the output sequence based on the representations provided by the encoder and the previously generated tokens of the output.</p><p>Each decoder layer consists of three main sub-layers:</p><ol><li><strong>Masked Self-Attention Mechanism</strong>: Similar to the encoder’s self-attention mechanism but its main purpose is to prevent attending to future tokens to maintain the autoregressive property (no cheating during generation).</li><li><strong>Encoder-Decoder Attention Mechanism</strong>: This sub-layer allows the decoder to focus on relevant parts of the encoder’s output representation. This allows the decoder to focus on relevant parts of the input, essential for tasks like translation.</li><li><strong>Feed-Forward Neural Network</strong>: This sub-layer processes the combined output of the masked self-attention and encoder-decoder attention mechanisms.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zp-kWPOy1mz7SlZzzK6oKg.png" /></figure><h3>Summary</h3><h3>How Transformers Work in Short —</h3><h3>1. Input Representation</h3><p>The first step in processing input data involves converting raw text into a format that the transformer model can understand. This involves tokenization and embedding.</p><ul><li><a href="https://www.geeksforgeeks.org/nlp/what-is-tokenization/"><strong>Tokenization</strong></a>: The input text is split into smaller units called tokens, which can be words, sub words or characters. Tokenization ensures that the text is broken down into manageable pieces.</li><li><a href="https://www.geeksforgeeks.org/nlp/word-embeddings-in-nlp/"><strong>Embedding</strong></a>: Each token is then converted into a fixed-size vector using an embedding layer. This layer maps each token to a dense vector representation that captures its semantic meaning.</li><li><a href="https://www.geeksforgeeks.org/nlp/positional-encoding-in-transformers/"><strong>Positional encodings</strong></a><strong> </strong>are added to these embeddings to provide information about the token positions within the sequence.</li></ul><h3>2. Encoder Process in Transformers</h3><ul><li><strong>Input Embedding</strong>: The input sequence is tokenized and converted into embeddings with positional encodings added.</li><li><strong>Self-Attention Mechanism</strong>: Each token in the input sequence attends to every other token to capture dependencies and contextual information.</li><li><strong>Feed-Forward Network</strong>: The output from the self-attention mechanism is passed through a position-wise feed-forward network.</li><li><strong>Layer Normalization and Residual Connections</strong>: Layer normalization and residual connections are applied.</li></ul><h3>3. Decoder Process</h3><ul><li><strong>Input Embedding and Positional Encoding</strong>: The partially generated output sequence is tokenized and embedded with positional encodings added.</li><li><strong>Masked Self-Attention Mechanism</strong>: The decoder uses masked self-attention to prevent attending to future tokens ensuring that the model generates the sequence step-by-step.</li><li><strong>Encoder-Decoder Attention Mechanism</strong>: The decoder attends to the encoder’s output allowing it to focus on relevant parts of the input sequence.</li><li><strong>Feed-Forward Network</strong>: Similar to the encoder the output from the attention mechanisms is passed through a position-wise feed-forward network.</li><li><strong>Layer Normalization and Residual Connections</strong>: Similar to the encoder Layer normalization and residual connections are applied.</li></ul><h3>4. Training and Inference</h3><ul><li>Transformers are trained with teacher forcing, where the correct previous tokens are provided during training to predict the next token. Their encoder-decoder architecture combined with multi-head attention and feed-forward networks enables highly effective handling of sequential data.</li><li>Transformers have transformed deep learning by using self-attention mechanisms to efficiently process and generate sequences capturing long-range dependencies and contextual relationships. Their encoder-decoder architecture combined with multi-head attention and feed-forward networks enables highly effective handling of sequential data.</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7b1537c01a03" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How Neural Network works ?]]></title>
            <link>https://chsatyam.medium.com/how-neural-network-works-402b5d168018?source=rss-3c0b02a60007------2</link>
            <guid isPermaLink="false">https://medium.com/p/402b5d168018</guid>
            <category><![CDATA[artificial-neural-network]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[neural-networks]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[deep-learning]]></category>
            <dc:creator><![CDATA[Satyam Choudhary]]></dc:creator>
            <pubDate>Sun, 21 Jun 2026 11:50:35 GMT</pubDate>
            <atom:updated>2026-06-21T16:19:08.912Z</atom:updated>
            <content:encoded><![CDATA[<p><strong>Artificial Intelligence (AI)</strong> → <strong>Machine Learning (ML)</strong> →<br>→ <strong>Neural Networks / Deep Learning</strong></p><p><strong>What are Neural Networks ?</strong></p><blockquote>Neural networks are the fundamental building blocks of deep learning algorithms. A neural network is a type of machine learning algorithm that is designed to simulate the behavior of the human brain. It is made up of interconnected nodes, also known as artificial neurons, which are organized into layers.<br>( “Didn’t understand anything ? , No worries — I’ll explain everything in simple language. Just read the complete blog and then come back here again.”)</blockquote><p>Before understanding the <strong>Neural Network directly let’s understand How Neural Network differs from Machine Learning ?</strong><br> <br> A <strong>neural network</strong> is actually a <strong>type of machine learning</strong> model.<br> So the relationship is :</p><p>Think of it like this:</p><blockquote>Artificial Intelligence (AI)</blockquote><blockquote>↓</blockquote><blockquote>Machine Learning (ML)</blockquote><blockquote>↓</blockquote><blockquote>Deep Learning (DL)</blockquote><blockquote>↓</blockquote><blockquote>Neural Networks (NN)</blockquote><p>More precisely:</p><ul><li><strong>AI</strong> → The broad goal of making machines perform intelligent tasks.</li><li><strong>Machine Learning (ML)</strong> → A subset of AI where systems learn patterns from data.</li><li><strong>Neural Networks (NN)</strong> → One family of ML algorithms.</li><li><strong>Deep Learning (DL)</strong> → Neural networks with <strong>many layers</strong> (“deep” neural networks).</li></ul><p>So your corrected statement:</p><p><strong>AI is the big field → ML is a subset/building block of AI → Neural Networks are a subset of ML → Deep Learning is neural networks with multiple layers.</strong></p><blockquote>IMP: <strong>Deep Learning and Neural Networks are not separate levels in parallel. Deep Learning is built using neural networks.</strong></blockquote><h3>Now coming back to the topic — How do Neural Networks differ from Machine Learning ?</h3><p>Neural Networks can automatically learn patterns and improve from data with minimal manual feature engineering. Unlike traditional Machine Learning, they learn important features directly from the data instead of relying heavily on humans to define them.</p><p><strong>Machine Learning : </strong>Broad field of algorithms that learn from data</p><p><strong>Neural Networks : </strong>A specific ML technique inspired by the human brain</p><p><strong>Example</strong></p><p>Suppose you want to detect spam emails:</p><ul><li>A traditional ML algorithm might use: Keywords, sender reputation, frequency patterns.</li><li>A neural network can automatically learn complex patterns from millions of emails without manual feature design.</li></ul><p><strong>Key Point : </strong>Not all machine learning uses neural networks, but all neural networks are part of machine learning.</p><p>Neural networks became especially popular because they power modern systems like:</p><ul><li>OpenAI ChatGPT</li><li>self-driving cars</li><li>voice assistants</li><li>image recognition</li><li>translation systems</li></ul><blockquote>Q. What neural networks do differently is <strong>learn the useful patterns (features) automatically from data</strong>, instead of requiring humans to manually define those features.</blockquote><p><strong>Why people say “learns on its own” : </strong>Because humans don’t manually specify features anymore.</p><p><strong>Example </strong>: <br>Think of a Neural Network like teaching a child to recognize animals by showing many pictures.</p><h3>Example: Cat vs Dog Images</h3><p>You want a computer to tell whether a picture is a <strong>cat</strong> or a <strong>dog</strong>.</p><h3>Traditional Machine Learning (Old way)</h3><p>In traditional Machine Learning, humans must first tell the computer <strong>what to look at</strong>.</p><p>A person decides important features such as:</p><ul><li>Ear shape</li><li>Whisker length</li><li>Fur color</li><li>Eye distance</li></ul><p>Then the computer uses these features to make a decision.</p><p>Flow:</p><p><strong>Images → Human chooses features → Machine Learning → Prediction</strong></p><p>Example:</p><p>Picture → ears are pointy + long whiskers → Computer says <strong>Cat</strong></p><p>The problem:<br> If humans forget an important feature, the computer may make mistakes.</p><h3>Neural Network (New way)</h3><p>Neural Networks work differently.</p><p>You give the computer <strong>raw images directly</strong>.</p><p>You do <strong>not</strong> tell it:</p><ul><li>“Look at whiskers”</li><li>“Check ear shape”</li><li>“Check fur”</li></ul><p>Instead, the Neural Network learns by itself.</p><p>Flow:</p><p><strong>Images → Neural Network finds patterns → Prediction</strong></p><p>Example:</p><p>You show:</p><p>Cat image → Cat<br> Dog image → Dog<br> Cat image → Cat<br> Dog image → Dog</p><p>After seeing thousands of examples, it starts noticing patterns.</p><h3>What it learns inside:</h3><p><strong>Early layers:</strong><br> Learn simple things → lines, edges, colors</p><p>Example:<br> “Here is a curved line.”</p><p>↓</p><p><strong>Middle layers:</strong><br> Combine lines into shapes</p><p>Example:<br> “These lines look like ears.”</p><p>↓</p><p><strong>Deep layers:</strong><br> Recognize complete objects</p><p>Example:<br> “This shape looks like a cat face.”</p><p>Finally:</p><p><strong>Prediction → This is a Cat</strong></p><h3>How does it actually learn?</h3><p>Suppose you show this image:</p><p>🐱 (Actual answer = Cat)</p><p>But the network predicts:</p><p><strong>80% Dog</strong></p><p>It made a mistake.</p><p>Now learning starts.</p><h3>Step 1: Calculate the error</h3><p>Computer compares:</p><p>Prediction = Dog ❌<br> Correct answer = Cat ✅</p><p>It asks:</p><p><strong>“How wrong was I?”</strong></p><h3>Step 2: Send the mistake backward (Backpropagation)</h3><p>The network checks all its internal decisions.</p><p>It asks:</p><p><strong>“Which parts caused the wrong answer?”</strong></p><h3>Step 3: Adjust internal numbers (Weights)</h3><p>Inside the network are millions of small numbers called <strong>weights</strong>.</p><p>Weights decide what the network pays attention to.</p><p>Example:</p><p>Before:</p><ul><li>Ear importance = Low</li><li>Fur importance = High</li></ul><p>After adjustment:</p><ul><li>Ear importance = Higher</li><li>Fur importance = Lower</li></ul><h3>Step 4: Repeat many times</h3><p>Show another image.</p><p>Predict → Check error → Adjust</p><p>Again and again.</p><p>Like this:</p><p>Wrong → adjust<br> Less wrong → adjust<br> Better → adjust<br> Accurate → learned</p><p>This repeated learning process is called <strong>training</strong>.</p><h3>Why people say Neural Networks “learn by themselves”</h3><p>Because humans do <strong>not manually tell the computer what features matter</strong>.</p><p>Humans only provide:</p><ul><li>Data (images)</li><li>Correct answers</li></ul><p>The Neural Network discovers the useful patterns on its own.</p><h3><strong>Q. Why to use Deep Learning or Neural networks ?</strong></h3><p><strong>Deep Learning</strong> is used when the problem is too complex for traditional machine learning to solve well using manually engineered features.</p><p>Deep learning uses <strong>deep neural networks</strong> (many layers) to automatically learn increasingly complex patterns.</p><p><strong>Why use Deep Learning?</strong></p><p><strong>1. Automatic feature learning</strong></p><p>You don’t need to manually tell the system what to look for.</p><p>Example:</p><ul><li>Traditional ML → engineer creates features like edges, color, texture.</li><li>Deep Learning → learns those features directly from raw data.</li></ul><p><strong>2. Handles unstructured data extremely well</strong></p><p>Deep learning shines on:</p><ul><li><strong>Images</strong></li><li><strong>Audio / speech</strong></li><li><strong>Text</strong></li><li><strong>Video</strong></li></ul><p>Examples:</p><ul><li>Face recognition</li><li>Speech assistants</li><li>Translation</li><li>Medical image analysis</li></ul><p><strong>3. Better performance with large datasets</strong></p><p>As data increases, deep learning often improves more than many traditional approaches.</p><p>Example:</p><ul><li>Millions of images → better object recognition.</li><li>Billions of words → stronger language understanding.</li></ul><p><strong>4. Learns hierarchical representations</strong></p><p>Different layers learn different abstraction levels.</p><p>For an image:</p><p>Pixels</p><p>↓</p><p>Edges</p><p>↓</p><p>Shapes</p><p>↓</p><p>Objects</p><p>↓</p><p>Meaning</p><p>For language:</p><p>Letters</p><p>↓</p><p>Words</p><p>↓</p><p>Phrases</p><p>↓</p><p>Sentences</p><p>↓</p><p>Context</p><p><strong>5. Enables modern AI systems</strong></p><p>Deep learning powers:</p><ul><li>Chatbots</li><li>Self-driving systems</li><li>Image generation</li><li>Recommendation systems</li><li>Voice assistants</li></ul><p><strong>But Deep Learning is NOT always better</strong></p><p><strong>Use Traditional ML</strong></p><p><strong>Use Deep Learning</strong></p><p>Small dataset</p><p>Huge dataset</p><p>Need explainability</p><p>Need highest accuracy</p><p>Structured tables</p><p>Images/audio/text</p><p>Limited compute</p><p>Powerful GPUs</p><p>Example:</p><ul><li>Predicting house prices → traditional ML may be enough.</li><li>Recognizing objects in video → deep learning is usually better.</li></ul><p>So the short answer:</p><p>We use Deep Learning because it can automatically learn complex patterns from large amounts of data and solve problems that are difficult to program manually.</p><p>Now let’s understand technically —</p><p><strong>Q. How Neural Network works?</strong></p><p>A neural network is a mathematical system that learns a mapping: <br> <br> Input → Patterns → Output<br> <br> A neural network tries to answer:</p><p>“What combination of patterns best predicts the correct output?”</p><p>It does this by adjusting millions (or billions) of numbers called <strong>weights</strong>.</p><h3>Neural Networks Explained Like You’re Teaching a Child to Recognize Numbers</h3><p>Imagine you want to build a system that can look at this image:</p><p>Image → “4”</p><p>and answer:</p><p><strong>“This is digit 4.”</strong></p><p>How does the computer actually do that?</p><p>Not magic.</p><p>Just <strong>math + practice + correction</strong></p><h3>Big Picture</h3><p>A Neural Network learns:</p><pre>Input → Patterns → Output</pre><p>Example:</p><pre>Image → detect lines → detect shapes → predict &quot;4&quot;</pre><p>Like humans:</p><pre>eyes → brain → recognize object</pre><h3>Step 1 — Input Layer</h3><h3>Where does data come from?</h3><p>Look at the image.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5FXNlfpPq254Ad79aSP-gQ.png" /></figure><p>The handwritten number is stored as pixels.</p><p>Example:</p><pre>28 × 28 image<br><br>That means:<br><br>28 rows × 28 columns<br>= 784 pixels</pre><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*eGOWDra8CoIE5hCVk7i02g.png" /></figure><p>Each pixel has brightness:</p><pre>0 → black<br>1 → white<br>0.5 → gray</pre><p>Example image:</p><pre>0 0 0 0<br>0 1 1 0<br>0 1 1 0<br>0 0 0 0</pre><p>Computer does NOT see:</p><p>❌ “digit 4”</p><p>Computer sees:</p><pre>[0.0,0.1,0.3,0.9,... 784]</pre><p>784 numbers.</p><p>These become:</p><h3>Input neurons</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/656/1*eGOWDra8CoIE5hCVk7i02g.png" /></figure><pre>Input Layer<br><br>○<br>○<br>○<br>○<br>○<br>(784 neurons)</pre><p>Each neuron stores ONE pixel.</p><h3>Step 2 — What is a Neuron?</h3><p>A neuron is a tiny calculator.</p><p>Each neuron does:</p><pre>Output =<br>Activation( Input × Weight + Bias )</pre><p>Looks scary.</p><p>Actually simple.</p><p>Example:</p><pre>Input: 2 <br>Weight: 3<br>Bias:+1<br><br>Calculate: (2×3)+1 = 7</pre><p>Activation decides final output.</p><h3>Step 3 — Weight</h3><h3>Who decides the weights?</h3><p>Weights are NOT manually set.</p><p>The network learns them.</p><p>Weight means:</p><h3>How important is this input?</h3><p>Example:</p><p>Detecting digit 4:</p><pre>Top vertical line → important<br>Bottom left → less important</pre><p>Example:</p><pre>pixel × weight<br><br>Pixel: 0.8<br>Weight: 5<br><br>Output: 4</pre><p>Large weight = pay attention.<br>Small weight = ignore.</p><p>Initially:</p><pre>random weights</pre><p>After training:</p><pre>smart weights</pre><h3>Step 4 — Bias</h3><h3>Why do we need Bias?</h3><p>Bias gives flexibility.</p><pre>Formula:<br>Output = (Input×Weight)+Bias</pre><p>Example:</p><pre>Without bias: 3×2=6</pre><pre>With bias: 3×2+5=11</pre><p>Bias helps move decisions.</p><p>Think:</p><p>Weight → importance</p><p>Bias → adjustment</p><p>Like:</p><p>Teacher adds bonus marks.</p><h3>Step 5 — Activation Function</h3><h3>Why needed?</h3><p>Without activation:</p><p>Neural network becomes:</p><pre>big calculator</pre><p>Activation adds decision-making.</p><p>It answers:</p><blockquote><em>“Should this neuron activate?”</em></blockquote><p>Example :</p><pre><br>Input:<br><br>20<br><br>Activation:<br><br>Strong signal<br><br>Input:<br><br>−3<br><br>Activation:<br><br>Ignore<br><br>Common activations:<br><br>ReLU<br>Negative → 0<br><br>Positive → keep<br><br>Example:<br><br>ReLU(7)=7<br><br>ReLU(-5)=0</pre><h3>Step 6 — Hidden Layers</h3><h3>What actually happens?</h3><p>This is the thinking area.</p><p>Your uploaded diagram:</p><pre>Input<br>↓<br><br>Hidden Layer<br><br>↓<br><br>Output<br><br>Each hidden layer learns more complex patterns.<br><br>Layer 1<br><br>Find:<br><br>edges<br>lines<br><br>Example:<br><br>|<br>—<br>/<br>Layer 2<br><br>Combine edges:<br><br>corners<br>curves<br>Layer 3<br><br>Combine curves:<br><br>parts of numbers<br>Deep layer<br><br>Recognize:<br><br>&quot;looks like 4&quot;<br><br>Flow:<br><br>pixels<br><br>↓<br><br>edges<br><br>↓<br><br>curves<br><br>↓<br><br>digit parts<br><br>↓<br><br>digit</pre><h3>Step 7 — Forward Propagation</h3><h3>How prediction happens</h3><p>This means:</p><p>Move information:</p><pre>left → right<br><br>Example:<br><br>Image<br><br>↓<br><br>Input<br><br>↓<br><br>Hidden Layers<br><br>↓<br><br>Output<br><br>No learning yet.<br><br>Just prediction.<br><br>Example output:<br><br>0 → 0.01<br><br>1 → 0.02<br><br>2 → 0.05<br><br>3 → 0.80<br><br>4 → 0.07<br><br>Prediction:<br><br>3</pre><h3>Step 8 — Softmax</h3><h3>Why do we need it?</h3><p>Output values may be random.</p><p>Example:</p><pre>5<br>8<br>2<br>1<br><br>Softmax converts to probabilities:<br><br>0 → 1%<br><br>1 → 2%<br><br>2 → 4%<br><br>3 → 88%<br><br>4 → 5%<br><br>Total:<br><br>100%<br><br>Choose highest.<br><br>Prediction:<br><br>3</pre><h3>Step 9 — Loss Function</h3><h3>How wrong are we?</h3><p>Suppose:</p><pre>mage:<br><br>Actual = 4<br><br>Prediction:<br><br>3<br><br>Loss computes:<br><br>How wrong?<br><br>Small loss:<br><br>✅ Good<br><br>Large loss:<br><br>❌ Bad<br><br>Example:<br><br>Prediction:<br><br>4 → 90%<br><br>Loss:<br><br>small<br><br>Prediction:<br><br>8 → 90%<br><br>Loss:<br><br>large<br></pre><h3>Step 10 — Backpropagation</h3><h3>How does network improve?</h3><p>This is the learning step.</p><pre>Suppose:<br><br>Predicted = 3<br>Actual = 4<br><br>Backpropagation says:<br><br>“Which weights caused the mistake?”<br><br>Then updates:<br><br>weight1<br>weight2<br>weight3<br><br>Makes tiny corrections.<br><br>Think:<br><br>Wrong<br><br>↓<br><br>adjust<br><br>↓<br><br>try again<br><br>↓<br><br>adjust<br><br>Millions of times.</pre><h3>Step 11 — Training Loop</h3><h3>Entire learning process</h3><p>Repeat:</p><pre>1. Take image<br><br>↓<br><br>2. Predict<br><br>↓<br><br>3. Calculate loss<br><br>↓<br><br>4. Backpropagation<br><br>↓<br><br>5. Update weights<br><br>↓<br><br>6. Next image<br><br>Thousands of iterations.<br><br>Eventually:<br><br>Before:<br><br>Accuracy = 20%<br><br>After:<br><br>Accuracy = 98%</pre><p>Full Neural Network Flow</p><pre>Image (28×28)<br><br>↓<br><br>Input Layer<br>(784 pixels)<br><br>↓<br><br>Weights + Bias<br><br>↓<br><br>Activation<br><br>↓<br><br>Hidden Layers<br><br>↓<br><br>Output Scores<br><br>↓<br><br>Softmax<br><br>↓<br><br>Prediction<br><br>↓<br><br>Loss<br><br>↓<br><br>Backpropagation<br><br>↓<br><br>Update Weights<br><br>↓<br><br>Repeat</pre><p>That entire repeat cycle is called:</p><h3>TRAINING</h3><p>And after training:</p><p>You only do:</p><pre>Image<br><br>↓<br><br>Prediction</pre><p>No more learning.</p><p>That stage is called:</p><h3>INFERENCE</h3><p>This is the complete beginner → intermediate mental model of how your uploaded neural network diagrams work.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=402b5d168018" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Rubrics:]]></title>
            <link>https://chsatyam.medium.com/rubrics-a0b7515d93a1?source=rss-3c0b02a60007------2</link>
            <guid isPermaLink="false">https://medium.com/p/a0b7515d93a1</guid>
            <category><![CDATA[rubrics]]></category>
            <category><![CDATA[rubrics-creation]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[rubrics-in-ai]]></category>
            <category><![CDATA[chatgpt]]></category>
            <dc:creator><![CDATA[Satyam Choudhary]]></dc:creator>
            <pubDate>Tue, 16 Jun 2026 08:29:36 GMT</pubDate>
            <atom:updated>2026-06-16T08:29:36.597Z</atom:updated>
            <content:encoded><![CDATA[<p>Rubrics: What They Are, Why They Matter, How to Create Them, and How AI Uses Them</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*NHVucuuMYEc8Fu6G.png" /></figure><p>If you’ve ever submitted an assignment and wondered:</p><ul><li><em>“Why did I get 7/10 instead of 9/10?”</em></li><li><em>“What exactly makes a good answer?”</em></li><li><em>“How do evaluators judge fairly?”</em></li></ul><p>The answer is often: <strong>a rubric</strong>.</p><p>Rubrics are one of the most useful tools for <strong>evaluation, feedback, consistency, and decision-making</strong> — in education, hiring, business, and increasingly in AI systems.</p><p>Let’s break it down in a practical, medium-blog style.</p><h3>📘<strong> What is a Rubric?</strong></h3><p>A <strong>rubric</strong> is a <strong>structured scoring guide</strong> used to evaluate performance, work, or responses based on predefined criteria.</p><p>Instead of saying:</p><p>❌ “This answer feels good.”</p><p>A rubric says:</p><p>✅ “This answer scores high because it is accurate, complete, clear, and well-structured.”</p><p>Think of it as a <strong>grading blueprint</strong>.</p><p>For example, in an essay rubric:</p><ul><li>Clarity → 5 marks</li><li>Grammar → 5 marks</li><li>Argument quality → 10 marks</li></ul><p>Each criterion has levels (excellent, good, poor), so evaluation is consistent — not random.</p><h3>Why Do We Need Rubrics?</h3><p>Rubrics solve several evaluation problems.</p><h3>1. Consistency</h3><p>Without rubric:</p><p>Evaluator A → 9/10<br> Evaluator B → 6/10</p><p>With rubric:</p><p>Both judge using the same rules.</p><h3>2. Fairness</h3><p>Rubrics reduce:</p><ul><li>personal bias</li><li>favoritism</li><li>inconsistent standards</li><li>emotional decision-making</li></ul><p>People know <strong>how they are being judged</strong>.</p><h3>3. Clarity of Expectations</h3><p>Rubrics answer:</p><ul><li>What is expected?</li><li>What counts as excellent?</li><li>What mistakes reduce score?</li></ul><p>Students, employees, and AI systems all perform better when expectations are explicit.</p><h3>4. Better Feedback</h3><p>Instead of:</p><p>❌ “Needs improvement”</p><p>You get:</p><p>✅ “Strong ideas, but evidence and organization need work.”</p><h3>🎯<strong> Why Do We Write Rubrics?</strong></h3><p>Rubrics exist to remove guesswork and subjectivity.</p><p><strong>Key reasons:</strong></p><ul><li><strong>Consistency</strong> → Everyone is judged by the same standards</li><li><strong>Transparency</strong> → Students/users know what’s expected</li><li><strong>Efficiency</strong> → Faster and more structured evaluation</li><li><strong>Feedback quality</strong> → Shows <em>where</em> someone did well or poorly</li></ul><p>Without a rubric, evaluation becomes vague like: <em>“This looks okay.”</em></p><p>With a rubric: <em>“Strong argument, but weak structure and examples.”</em></p><h3>A rubric usually contains :</h3><pre>| Criterion    | What is being measured    |<br>| ------------ | ------------------------- |<br>| Accuracy     | Is the content correct?   |<br>| Completeness | Did it answer everything? |<br>| Clarity      | Is it understandable?     |<br>| Creativity   | Is it original?           |<br>| Evidence     | Is reasoning supported?   |</pre><h3>⚙️<strong> What Does a Rubric Actually Do ?</strong></h3><p>A rubric acts like a <strong>decision-making framework</strong>.</p><p>It:</p><ul><li>Breaks a task into <strong>measurable components</strong></li><li>Defines <strong>quality levels</strong> for each component</li><li>Helps assign <strong>scores or judgments</strong></li><li>Reduces bias in evaluation</li></ul><p>In short:<br> 👉 It turns subjective judgment into <strong>structured evaluation</strong></p><h3>Simple Example of a Rubric</h3><p>Imagine evaluating an essay.</p><p>Without rubric:</p><p>Teacher:</p><pre>&quot;I think this deserves 75.&quot;</pre><p>With rubric:</p><pre>| Criteria      | Weight | Score |<br>| ------------- | -----: | ----: |<br>| Understanding |    30% |    25 |<br>| Structure     |    20% |    15 |<br>| Evidence      |    30% |    20 |<br>| Grammar       |    20% |    18 |</pre><p>Total = <strong>78/100</strong></p><p>Now evaluation becomes transparent.</p><h3>Components of a Good Rubric</h3><p>Most rubrics have 4 parts.</p><h3>1. Criteria</h3><p>What matters?</p><p>Example:</p><pre>Accuracy<br>Depth<br>Presentation</pre><h3>2. Performance Levels</h3><p>How good?</p><p>Example:</p><pre>| Level | Meaning   |<br>| ----- | --------- |<br>| 4     | Excellent |<br>| 3     | Good      |<br>| 2     | Fair      |<br>| 1     | Poor      |</pre><h3>3. Descriptors</h3><p>Define each level.</p><p>Example:</p><pre>| Score | Description             |<br>| ----- | ----------------------- |<br>| 4     | Complete and error-free |<br>| 3     | Mostly complete         |<br>| 2     | Partial                 |<br>| 1     | Incorrect               |</pre><h3>4. Weight</h3><p>Importance.</p><p>Example:</p><pre>Accuracy → 50%<br> Presentation → 10%</pre><h3>Types of Rubrics</h3><h3>1. Holistic Rubric</h3><p>One overall score.</p><p>Example:</p><pre>| Score | Description |<br>| ----- | ----------- |<br>| 5     | Outstanding |<br>| 3     | Average     |<br>| 1     | Poor        |</pre><p>Good for:</p><ul><li>quick grading</li></ul><p>Bad for:</p><ul><li>detailed feedback</li></ul><h3>2. Analytic Rubric</h3><p>Multiple dimensions.</p><p>Example:</p><pre>| Criterion | Score |<br>| --------- | ----- |<br>| Accuracy  | 4     |<br>| Logic     | 3     |<br>| Clarity   | 5     |</pre><p>Good for:</p><ul><li>detailed evaluation</li></ul><p>Most AI evaluation uses this.</p><h3>3. Developmental Rubric</h3><p>Tracks progress.</p><p>Example:</p><p>Beginner → Intermediate → Advanced</p><p>Useful for:</p><ul><li>learning and coaching.</li></ul><h3>How to Create a Rubric (Step-by-Step)</h3><p>Let’s build one.</p><p>Suppose we want to evaluate a blog article.</p><h3>Step 1: Define Objective</h3><p>Question:</p><p>“What are we evaluating?”</p><p>Goal:<br> Evaluate article quality.</p><h3>Step 2: Choose Criteria</h3><p>Example:</p><ul><li>Accuracy</li><li>Clarity</li><li>Structure</li><li>Engagement</li></ul><h3>Step 3: Define Levels</h3><p>Example:</p><pre>| Score | Meaning   |<br>| ----- | --------- |<br>| 4     | Excellent |<br>| 3     | Good      |<br>| 2     | Fair      |<br>| 1     | Poor      |</pre><h3>Step 4: Write Observable Descriptions</h3><p>Bad:</p><p>❌ “Pretty good”</p><p>Good:</p><p>✅ “Explains concepts clearly with examples.”</p><h3>Step 5: Add Weight</h3><pre>| Criterion  | Weight |<br>| ---------- | -----: |<br>| Accuracy   |    40% |<br>| Clarity    |    25% |<br>| Structure  |    20% |<br>| Engagement |    15% |</pre><h3>Final Rubric</h3><pre>| Criteria  | 1            | 2              | 3              | 4               |<br>| --------- | ------------ | -------------- | -------------- | --------------- |<br>| Accuracy  | Many errors  | Some errors    | Mostly correct | Fully correct   |<br>| Clarity   | Confusing    | Understandable | Clear          | Extremely clear |<br>| Structure | Disorganized | Average        | Logical        | Excellent flow  |</pre><h3>Rubrics in AI (Very Important)</h3><p>Rubrics became extremely important with modern AI.</p><p>AI systems are evaluated using rubrics constantly.</p><h3>Example: Evaluating AI Answers</h3><p>Question:</p><blockquote><em>Explain photosynthesis.</em></blockquote><p>Rubric:</p><pre>| Criterion    | Weight |<br>| ------------ | -----: |<br>| Correctness  |     40 |<br>| Completeness |     30 |<br>| Clarity      |     20 |<br>| Safety       |     10 |</pre><p>AI output:</p><pre>| Criterion    | Score |<br>| ------------ | ----: |<br>| Correctness  |    38 |<br>| Completeness |    25 |<br>| Clarity      |    18 |<br>| Safety       |    10 |</pre><p>Total:<br> 91/100</p><h3>AI Training Uses Rubrics</h3><p>Humans label outputs using rubrics.</p><p>Example evaluation:</p><p>Prompt:</p><blockquote><em>Write email.</em></blockquote><p>Rubric:</p><ul><li>Grammar</li><li>Tone</li><li>Completeness</li><li>Helpfulness</li></ul><p>Human scores examples → AI learns.</p><h3>LLM Evaluation Rubrics</h3><p>Modern language models are often evaluated on:</p><pre>| Dimension             | Meaning               |<br>| --------------------- | --------------------- |<br>| Accuracy              | Factually correct     |<br>| Helpfulness           | Useful                |<br>| Safety                | No harmful output     |<br>| Instruction Following | Did it follow prompt? |<br>| Reasoning             | Logical thinking      |<br>| Tone                  | Appropriate style     |</pre><h3>Common Mistakes When Creating Rubrics</h3><h3>Too many criteria</h3><p>Bad:<br> 15 categories.</p><p>Good:<br> 4–7 categories.</p><h3>Vague descriptions</h3><p>Bad:<br> “Good work.”</p><p>Better:<br> “Uses at least 3 supporting examples.”</p><h3>Equal weights for everything</h3><p>Not everything matters equally.</p><h3>Measuring impossible things</h3><p>Bad:<br> “Looks intelligent.”</p><p>Good:<br> “Provides evidence.”</p><h3>Rubric vs Checklist</h3><p>People confuse these.</p><pre>| Checklist       | Rubric        |<br>| --------------- | ------------- |<br>| Done / Not Done | Quality scale |<br>| Binary          | Graduated     |<br>| Completion      | Evaluation    |</pre><p>Example:</p><p>Checklist:</p><ul><li>Submitted?</li><li>References included?</li></ul><p>Rubric:</p><ul><li>How strong were references?</li></ul><h3>A Formula to Remember</h3><p><strong>Rubric = Criteria + Levels + Descriptions + Weights</strong></p><p>Or:</p><blockquote><em>“What matters” + “How good” + “How much it counts”</em></blockquote><h3>Final Thought</h3><p>Rubrics are not just grading tools.</p><p>They are <strong>decision systems</strong>.</p><p>Whenever humans or AI need to answer:</p><blockquote>“How do we judge quality fairly and consistently?”</blockquote><p>rubrics become the language of evaluation.</p><p>That’s why they appear in:</p><ul><li>Education</li><li>Hiring</li><li>Product reviews</li><li>Performance management</li><li>Research</li><li>Machine Learning</li><li>Generative AI evaluation</li></ul><p>Once you learn to design good rubrics, you stop relying on opinions and start making decisions with structure.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a0b7515d93a1" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Agentic AI v/s AI Agents v/s Agentic AI]]></title>
            <link>https://chsatyam.medium.com/agentic-ai-v-s-ai-agents-v-s-agentic-ai-953bf737210b?source=rss-3c0b02a60007------2</link>
            <guid isPermaLink="false">https://medium.com/p/953bf737210b</guid>
            <category><![CDATA[generative-ai-tools]]></category>
            <category><![CDATA[chatgpt]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[ai-agent]]></category>
            <category><![CDATA[agentic-ai]]></category>
            <dc:creator><![CDATA[Satyam Choudhary]]></dc:creator>
            <pubDate>Sat, 13 Jun 2026 19:17:14 GMT</pubDate>
            <atom:updated>2026-06-14T06:09:46.528Z</atom:updated>
            <content:encoded><![CDATA[<p>Understanding the Real Difference with Examples, Architecture, and Practical Use Cases</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*7zJqG3NY4B3BLIMS.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*PuNaPC8F9pBG-65Y" /></figure><p>Artificial Intelligence is evolving fast — and with that evolution, new terms are appearing everywhere:</p><ul><li>Generative AI</li><li>AI Agents</li><li>Agentic AI</li></ul><p>People often use these words interchangeably.</p><p>But they are <strong>not the same thing</strong>.</p><p>If you understand the difference, you’ll better understand:</p><ul><li>What tools like ChatGPT actually do</li><li>Why AI agents are becoming popular</li><li>Why many believe Agentic AI is the next major shift after Generative AI</li></ul><p>Let’s break this down deeply.</p><h3>Introduction: Why Everyone Is Confused</h3><h3>A few years ago, most AI systems were designed for prediction:</h3><h3>1. Traditional AI = Predicts</h3><p>You give input → AI gives one answer.</p><p>Examples:</p><ul><li>“Will this customer leave?” → <strong>Yes / No</strong></li><li>“Is this image a cat?” → <strong>Cat / Not Cat</strong></li><li>“Will it rain tomorrow?”</li></ul><p>This type of AI answers a <strong>specific question</strong>.</p><p>Think of it like a <strong>student taking a multiple-choice test</strong>.</p><p>Then Generative AI arrived.</p><p>Instead of predicting labels —</p><p>AI started creating.</p><h3>2. Generative AI = Creates</h3><p>Instead of choosing an answer, AI starts <strong>making new content</strong>.</p><p>Examples:</p><ul><li>Write an email</li><li>Generate an image</li><li>Create code</li><li>Summarize documents</li></ul><p>You ask → AI creates.</p><p>Think of it like a <strong>writer or artist</strong>.</p><p>Example:</p><blockquote><em>Prompt: “Write a story about space”</em></blockquote><blockquote><em>AI: Creates a brand-new story.</em></blockquote><p>Soon after, developers realized:</p><blockquote><em>What if AI could not only create…<br>but also use tools?</em></blockquote><p>That led to AI Agents.</p><h3>3. AI Agents = Uses Tools to Complete Tasks</h3><p>Now AI can <strong>do things</strong>, not just talk.</p><p>It can:</p><ul><li>Search the web</li><li>Open files</li><li>Use software</li><li>Run code</li><li>Send messages</li></ul><p>Example:</p><blockquote><em>“Book me the cheapest flight.”</em></blockquote><p>Agent:</p><ol><li>Searches flights</li><li>Compares prices</li><li>Selects options</li><li>Helps complete booking</li></ol><p>Think of it like a <strong>personal assistant</strong>.</p><p>Then researchers pushed further:</p><blockquote><em>What if AI could plan, execute, monitor, adapt, and improve continuously?</em></blockquote><p>That is where Agentic AI begins.</p><h3>4. Agentic AI = Works Toward Goals Independently</h3><p>This is the next level.</p><p>AI doesn’t wait for every instruction.</p><p>It can:</p><ul><li>Plan</li><li>Break goals into steps</li><li>Execute tasks</li><li>Check progress</li><li>Adjust if something fails</li><li>Continue improving</li></ul><p>Example:</p><blockquote><em>Goal: “Grow my online store sales.”</em></blockquote><p>Agentic AI might:</p><ol><li>Analyze traffic</li><li>Find weak pages</li><li>Create ads</li><li>Monitor results</li><li>Change strategy automatically</li></ol><p>Think of it like a <strong>project manager or employee</strong>.</p><h4>Think of it as evolution:</h4><blockquote><strong>Traditional AI → Generative AI → AI Agents → Agentic AI</strong></blockquote><h3>Evolution in one line:</h3><p><strong>Traditional AI</strong> → <em>Answers</em><br> ↓<br> <strong>Generative AI</strong> → <em>Creates</em><br> ↓<br> <strong>AI Agents</strong> → <em>Acts</em><br> ↓<br> <strong>Agentic AI</strong> → <em>Plans + Acts + Adapts</em></p><h3>Deep Understanding of each :</h3><h3>1. Generative AI — The Creator</h3><h3>Definition</h3><p>Generative AI is AI designed to generate new content.</p><p>That content may include:</p><ul><li>Text</li><li>Images</li><li>Code</li><li>Audio</li><li>Video</li><li>Designs</li></ul><p>Its primary purpose:</p><blockquote><em>“Create something for me.”</em></blockquote><h3>Popular Generative AI Examples</h3><h3>ChatGPT</h3><p>Generates:</p><ul><li>text</li><li>code</li><li>explanations</li><li>ideas</li></ul><h3>DALL·E</h3><p>Generates:</p><ul><li>images</li></ul><h3>GitHub Copilot</h3><p>Generates:</p><ul><li>code suggestions</li></ul><h3>Midjourney</h3><p>Generates:</p><ul><li>visual artwork</li></ul><h3>Real Example</h3><p>Prompt:</p><blockquote><em>“Write a social media campaign for a coffee brand.”</em></blockquote><p>Generative AI may produce:</p><ul><li>Instagram caption</li><li>ad copy</li><li>hashtags</li><li>campaign ideas</li></ul><p>But it stops there.</p><p>It does not:</p><ul><li>publish posts</li><li>track engagement</li><li>optimize performance</li></ul><h3>Strengths of Generative AI</h3><p>✅ Fast content creation<br>✅ Creative ideation<br>✅ Knowledge assistance<br>✅ Coding support</p><h3>Limitations</h3><p>❌ No memory of goals<br>❌ No autonomous action<br>❌ No execution capability<br>❌ No long-term planning</p><h3>Is ChatGPT Agentic?</h3><p>Short answer:</p><p>No. ChatGPT by itself is primarily Generative AI.</p><p>It generates outputs.</p><p>However —</p><p>when connected with tools, memory, workflows, or automation layers — it can participate inside an AI agent system.</p><h3>2. AI Agents — The Doers</h3><h3>Definition</h3><p>AI Agents are systems that combine:</p><ul><li>AI reasoning</li><li>Tool usage</li><li>Multi-step execution</li></ul><p>Their purpose:</p><blockquote><em>“Do this task for me.”</em></blockquote><p>Unlike Generative AI —</p><p>agents can interact with the outside world.</p><h3>AI Agent Architecture</h3><p>User Goal<br>↓<br>Reason<br>↓<br>Choose Tool<br>↓<br>Execute<br>↓<br>Observe Result<br>↓<br>Continue</p><h3>Example: Travel Booking Agent</h3><p>User says:</p><blockquote><em>“Book the cheapest flight to Mumbai tomorrow.”</em></blockquote><p>The AI Agent:</p><p>Step 1 → Search flight APIs<br>Step 2 → Compare prices<br>Step 3 → Check baggage<br>Step 4 → Select option<br>Step 5 → Book ticket<br>Step 6 → Confirm reservation</p><p>That is not content generation.</p><p>That is execution.</p><h3>Tools AI Agents Commonly Use</h3><ul><li>Web search</li><li>Databases</li><li>APIs</li><li>File systems</li><li>Terminal commands</li><li>Browser automation</li><li>Email systems</li><li>CRMs</li></ul><h3>Example: Developer AI Agent</h3><p>User says:</p><blockquote><em>“Convert this JavaScript project to TypeScript.”</em></blockquote><p>Agent workflow:</p><ol><li>Read repository</li><li>Analyze structure</li><li>Convert files</li><li>Update imports</li><li>Run tests</li><li>Fix errors</li><li>Commit changes</li></ol><p>This goes beyond writing code.</p><h3>Cursor as an Example</h3><p>Cursor behaves like an AI Agent because it can:</p><ul><li>inspect code</li><li>edit files</li><li>run commands</li><li>refactor projects</li></ul><p>You provide objectives.</p><p>It executes.</p><h3>Strengths of AI Agents</h3><p>✅ Task completion<br>✅ Tool integration<br>✅ Workflow automation<br>✅ Reduced manual work</p><h3>Limitations</h3><p>❌ Usually bounded to one workflow<br>❌ Needs clear instructions<br>❌ Limited autonomy</p><h3>3. Agentic AI — The Strategist</h3><h3>Definition</h3><p>Agentic AI is a broader concept where AI:</p><ul><li>understands goals</li><li>creates plans</li><li>executes actions</li><li>evaluates outcomes</li><li>adapts automatically</li></ul><p>Its objective:</p><blockquote><em>“Achieve this result.”</em></blockquote><h3>Core Agentic Loop</h3><p>Think<br>↓<br>Plan<br>↓<br>Act<br>↓<br>Observe<br>↓<br>Learn<br>↓<br>Improve</p><p>Repeat.</p><h3>Example: Growing an E-Commerce Business</h3><p>User Goal:</p><blockquote><em>“Increase store revenue by 20%.”</em></blockquote><p>Agentic AI may:</p><p>Step 1 → Analyze sales data</p><p>Step 2 → Identify weak categories</p><p>Step 3 → Generate ads</p><p>Step 4 → Launch campaigns</p><p>Step 5 → Monitor results</p><p>Step 6 → Adjust targeting</p><p>Step 7 → Reallocate budget</p><p>Step 8 → Continue optimizing</p><p>Notice:</p><p>You gave the goal once.</p><p>The system handles execution.</p><h3>Another Example: Recruiting</h3><p>Goal:</p><blockquote><em>“Hire 5 engineers.”</em></blockquote><p>Agentic system could:</p><ul><li>analyze hiring funnel</li><li>generate job descriptions</li><li>post listings</li><li>schedule interviews</li><li>rank candidates</li><li>improve sourcing strategy</li></ul><p>That is continuous optimization.</p><h3>Agentic AI Architecture</h3><p>Goal<br>↓<br>Planner<br>↓<br>Task Decomposition<br>↓<br>Multiple Agents<br>↓<br>Execution<br>↓<br>Monitoring<br>↓<br>Feedback Loop<br>↓<br>Goal Achievement</p><h3>Why Agentic AI Feels Different</h3><p>Generative AI says:</p><blockquote><em>“Here’s content.”</em></blockquote><p>AI Agent says:</p><blockquote><em>“I completed your task.”</em></blockquote><p>Agentic AI says:</p><blockquote><em>“I achieved your objective.”</em></blockquote><h3>Comparison Table</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/988/1*pQU5IEW727vqQOe1QmwG4A.png" /></figure><h3>Real-World Scenario Comparison</h3><h3>Goal:</h3><p>Launch a new mobile app.</p><h3>Generative AI</h3><p>You ask:</p><blockquote><em>“Create launch copy.”</em></blockquote><p>Output:</p><ul><li>ads</li><li>emails</li><li>content</li></ul><p>Stops.</p><h3>AI Agent</h3><p>You ask:</p><blockquote><em>“Launch my app campaign.”</em></blockquote><p>Agent:</p><ul><li>generates content</li><li>uploads ads</li><li>schedules posts</li></ul><p>Stops after execution.</p><h3>Agentic AI</h3><p>You ask:</p><blockquote><em>“Get 50,000 installs.”</em></blockquote><p>System:</p><ul><li>launches campaigns</li><li>tracks metrics</li><li>reallocates budget</li><li>iterates strategy</li><li>improves continuously</li></ul><h3>Where the Industry Is Heading</h3><p>Today:</p><p>Most products → Generative AI</p><p>Emerging:</p><p>Workflow automation → AI Agents</p><p>Future direction:</p><p>Autonomous business operations → Agentic AI</p><p>This doesn’t mean Generative AI disappears.</p><p>Instead:</p><p>Agentic systems often use Generative AI internally.</p><p>Relationship:</p><p>Agentic AI<br>→ Uses AI Agents<br>→ Which often use Generative AI</p><h3>Final Mental Model</h3><p>Imagine building a house.</p><p>Generative AI:</p><blockquote><em>Gives you blueprints.</em></blockquote><p>AI Agent:</p><blockquote><em>Builds one room.</em></blockquote><p>Agentic AI:</p><blockquote><em>Manages the entire construction project until completion.</em></blockquote><p>That is the difference.</p><h3>Conclusion</h3><p>Generative AI changed how we create.</p><p>AI Agents changed how we execute.</p><p>Agentic AI is changing how goals are achieved.</p><p>The future is likely not one replacing another —</p><p>but all three working together.</p><p>Create → Execute → Optimize.</p><p>That is the next era of AI.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=953bf737210b" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Semantic Search (Vector Databases) Lexical Search v/s Semantic Search]]></title>
            <link>https://chsatyam.medium.com/semantic-search-vector-databases-lexical-search-v-s-semantic-search-92b948c98aed?source=rss-3c0b02a60007------2</link>
            <guid isPermaLink="false">https://medium.com/p/92b948c98aed</guid>
            <category><![CDATA[vector-database]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[lexical-search]]></category>
            <category><![CDATA[semantic-search]]></category>
            <category><![CDATA[searchbar]]></category>
            <dc:creator><![CDATA[Satyam Choudhary]]></dc:creator>
            <pubDate>Sat, 13 Jun 2026 18:31:31 GMT</pubDate>
            <atom:updated>2026-06-14T07:40:36.801Z</atom:updated>
            <content:encoded><![CDATA[<p>Before Understanding the Semantic Search (Vector Databases) and Lexical Search , First understand the basic Normal Search, and see —</p><p>How Search Bars Actually Work: <br>From Typing “S” to Smart Suggestions (And How It Connects to Vector Databases) .</p><p>Every day we use search bars.</p><p>On Android apps.<br>On ecommerce websites.<br>On Google.<br>On streaming apps.<br>On dashboards.</p><p>But have you ever stopped and thought:</p><blockquote><em>How does the search bar know what I want before I finish typing?</em></blockquote><p>If I type:</p><pre>s</pre><p>why does it show:</p><ul><li>Shoes</li><li>Shorts</li><li>Scissors</li></ul><p>Why not:</p><ul><li>Television</li><li>Banana</li><li>Random products</li></ul><p>And where are these suggestions coming from?</p><p>Are they stored in the database?<br>In local code?<br>In API responses?<br>How does the entire system work?</p><p>This article explains the complete journey — from simple autocomplete to modern semantic search and vector databases.</p><h3>Level 1 — Basic Search Suggestion (Autocomplete)</h3><p>Suppose you have an ecommerce website.</p><p>Database:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/856/1*NONfQopnneBAGAGjYM7jVA.png" /></figure><p>User types:</p><pre>s</pre><pre>Frontend sends request:</pre><pre>GET /search?q=s</pre><p>Backend:</p><pre>&lt;SQL&gt;<br><br>SELECT product_name<br>FROM products<br>WHERE product_name LIKE &#39;s%&#39;<br>LIMIT 10;</pre><p>Result:</p><pre>&lt;JSON&gt;<br><br>[<br> &quot;Shoes&quot;,<br> &quot;Shorts&quot;,<br> &quot;Scissors&quot;<br>]</pre><p>This is the simplest version.</p><pre>Flow:<br><br>(Keyboard)User types: s <br> ↓<br>Search bar<br> ↓<br>API call<br> ↓<br>Database query<br> ↓<br>Suggestions<br> ↓<br>UI updates</pre><h3>Level 2 — Why it doesn’t search on every keystroke immediately</h3><p>If user types:</p><pre>s<br>sh<br>sho<br>shoe<br>shoes<br></pre><p>You don’t want:</p><pre>5 API calls</pre><p>So frontend uses <strong>debouncing</strong>.</p><p>Example:</p><pre>debounce(search, 300)</pre><p>Meaning:</p><p>Wait 300 ms after user stops typing.</p><p>Timeline:</p><pre>s<br>sh<br>sho<br>(wait)<br>→ API call</pre><p>This reduces server load.</p><h3>Level 3 — Prefix Index ( Fast search with indexes )</h3><p>Imagine 10 million products.</p><p>This query becomes slow:</p><pre>&lt;SQL&gt;<br><br>SELECT product_name<br>FROM products<br>WHERE product_name LIKE &#39;s%&#39;<br>LIMIT 10;</pre><p>So databases build an <strong>index</strong>.</p><p>Think of index like dictionary pages.</p><p>Without index:</p><pre>Search every row<br>O(N)</pre><p>With index:</p><pre>Jump directly<br>O(log N)</pre><p>Example:</p><p>Index:</p><pre>a → ...<br>b → ...<br>s →<br>   scissors<br>   shirt<br>   shoes<br>   shampoo</pre><p>Then searching &quot;sh&quot; becomes:</p><pre>sh →<br>  shoes<br>  shorts<br>  shampoo</pre><p>Then searching “sha” becomes:</p><pre>sha → <br> shampoo</pre><p>s → show broad result <br>sh → narrow results<br>sha → more narrow results</p><h3>Phase 3 — Popularity + Analytics</h3><p>Formula:</p><pre>score =<br>(prefix_match)<br>+<br>(click_rate)<br>+<br>(order_count)<br>+<br>(recency)<br><br>Or <br><br>score =<br>popularity<br>+ recent searches<br>+ purchase history<br>+ category relevance<br>+ availability</pre><p>Example:</p><p>User types:</p><pre>s</pre><pre>Data:<br>| Query    | Searches |<br>| -------- | -------- |<br>| shoes    | 2M       |<br>| scissors | 10K      |<br>| sofa     | 800K     |</pre><p>Ranking:</p><pre>shoes<br>sofa<br>scissors</pre><p>Not alphabetical.</p><p>Now , Let’s understand the Hot topic which is Semantic Search (Vector Databases)</p><h3>Semantic Search</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DFPJZb3b_ZqakaeY9x3u0w.png" /></figure><h3>Semantic Search + Embeddings (Vector Databases)</h3><p>Now user types:</p><pre>running shoes</pre><p>Database:</p><pre>Nike Sports Sneakers</pre><p>No exact word match</p><p>System converts text into vectors.</p><p>Example:</p><pre>&quot;running shoes&quot;<br>→ [0.23, -0.81, ... ]<br><br>&quot;Nike Sports Sneakers&quot;<br>→ [0.21, -0.79, ... ]</pre><p>Nearby vectors → similar meaning.</p><p>Flow:</p><pre>Keyboard<br> ↓<br>Search API<br> ↓<br>Embedding Model<br> ↓<br>Vector Search<br> ↓<br>Ranking Service<br> ↓<br>Suggestions</pre><h3>Semantic Search Version</h3><p>Suppose user types:</p><pre>s</pre><p>Frontend still sends:</p><pre>GET /search?q=s</pre><p>But backend changes.</p><h3>Step 1 — Convert query into vector</h3><p>Model converts:</p><pre>&quot;s&quot;</pre><p>into:</p><pre>[0.21, -0.43, 0.87, ...]</pre><p>(embedding)</p><p>Every product already has vectors stored.</p><p>Database:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1016/1*_hQpKmKkJH4OBGPilCvxQg.png" /></figure><h3>Step 2 — Similarity search</h3><p>Instead of:</p><pre>&lt;SQL&gt;<br><br>LIKE &#39;s%&#39;</pre><p>System does:</p><pre>SELECT nearest_vectors(query_vector)<br>LIMIT 10</pre><p>Conceptually:</p><pre>Similarity(<br> user_vector,<br> product_vector<br>)</pre><p>Result:</p><pre>Shoes              0.97<br>Running Sneakers   0.95<br>Sports Socks       0.93<br>Television         0.11</pre><h3>Step 3 — Return ranked suggestions</h3><p>API:</p><pre>&lt;JSON&gt;<br>[<br> &quot;Shoes&quot;,<br> &quot;Running Sneakers&quot;,<br> &quot;Sports Socks&quot;<br>]</pre><p>Notice:</p><p>Product doesn’t need to start with &quot;s&quot;.</p><p>User typed:</p><pre>sport shoe</pre><p>Can still return:</p><pre>&lt;JSON&gt; <br><br>[<br> &quot;Nike Sneakers&quot;,<br> &quot;Running Shoes&quot;<br>]</pre><p>because meaning matches.</p><h4>Real-world flow</h4><pre>Keyboard<br> ↓<br>Search Bar<br> ↓<br>GET /search?q=s<br> ↓<br>Embedding Model<br> ↓<br>Convert text → vector<br> ↓<br>Vector Database<br> ↓<br>Nearest Neighbour Search<br> ↓<br>Ranking Engine<br> ↓<br>Suggestions<br> ↓<br>UI updates</pre><p>One important detail:</p><p>For <strong>very short inputs like </strong><strong>&quot;s&quot;</strong>, most ecommerce apps <strong>do NOT use pure semantic search</strong>.</p><p>Usually they combine:</p><pre>Prefix Match<br>+<br>Popularity<br>+<br>User History<br>+<br>Semantic Search</pre><p>Because &quot;s&quot; alone has almost no meaning.</p><p>That hybrid approach is what modern search bars usually do.</p><h3>One More Example of Semantic Search :</h3><p>Imagine you open Flipkart and type:</p><blockquote><strong><em>football shoes on Flipkart Application</em></strong></blockquote><p>and the system generates and show’s:</p><ul><li>Football</li><li>Football kit</li><li>Football pump</li><li>Football socks</li><li>Sports accessories</li></ul><blockquote>You expect only football shoes.</blockquote><blockquote>Why searching “football shoes” also shows football, football kit, and football pump , Sports accessories</blockquote><p>This is where <strong>semantic search</strong> works.</p><p>Semantic search does not only look at the exact words you typed.</p><p>It tries to understand the <strong>meaning behind the search</strong>.</p><h3>1. Step 1: Convert your search into meaning (Embedding)</h3><p>When you type:</p><pre>football shoes</pre><p>The system does not keep it as plain text.</p><p>Instead, an AI model converts it into numbers called an <strong>embedding vector</strong>.</p><p>Example:</p><pre>Embedding model converts it into a vector:<br><br>football shoes <br>↓ <br><br>[0.82, -0.14, 0.33, 0.71 ...]</pre><p>Those numbers simply represent the meaning of your search.</p><p>This vector may internally capture ideas like:</p><pre>football <br>equipment <br>footwear <br>training</pre><p>Think of it like converting language into coordinates.</p><h3>2. Vector Search retrieves similar products</h3><p>Every product already has its own embedding</p><p>Before users search anything, Flipkart has already processed millions of products.</p><p>Each product gets converted into vectors.</p><pre>Example:<br><br>Product                   Vector<br>Football Shoes         [0.81, -0.12...]<br>Football               [0.79, -0.16...]<br>Football Kit           [0.77, -0.11...]<br>Cricket Bat            [0.15, 0.93...]</pre><p>Again, numbers are not important.</p><p>What matters is:</p><p>Products with similar meaning end up close together.</p><h3>Step 3: Search inside Vector Database</h3><p>Now your query vector enters a <strong>Vector Database</strong>.</p><p>Instead of asking:</p><pre>Which products contain &quot;football shoes&quot;?</pre><p>the system asks:</p><pre>Which products are closest in meaning?</pre><p>This search process is called:</p><pre>ANN (Approximate Nearest Neighbor Search)</pre><p>The system quickly finds nearby vectors.</p><p>Result:</p><pre>1 Football Shoes<br>2 Football<br>3 Football Kit<br>4 Football Pump</pre><p>Notice something interesting.</p><p>Football Pump does not contain the exact phrase:</p><pre>football shoes</pre><p>But semantically it belongs to the same sports context.</p><h3>Visualize it like a map</h3><p>Imagine products are placed on a map.</p><pre>          Football Kit<br><br>                       ●<br><br><br>Football Pump ●                ● Football Shoes<br><br><br>                 ● football (your search)<br><br><br><br><br><br>                               ● Cricket Bat<br></pre><p>Semantic search finds nearby points.</p><p>Keyword search only matches exact words.</p><h3>Traditional Search vs Semantic Search</h3><p>Traditional Search:</p><pre>Search → football shoes</pre><pre>Output:<br>Football Shoes<br>Football Shoes for Men<br>Football Shoes Size 9</pre><p>Semantic Search:</p><pre>Search → football shoes</pre><pre>Output:<br>Football Shoes<br>Football<br>Football Kit<br>Football Pump<br>Football Socks</pre><p>Traditional search understands words.</p><p>Semantic search understands intent.</p><h3>But this is only retrieval</h3><p>One important thing:</p><p>Semantic search does <strong>not decide the final order</strong>.</p><p>It only finds candidate products.</p><p>After retrieval, another ranking system decides:</p><ul><li>What users usually click</li><li>What users buy together</li><li>Product popularity</li><li>Inventory</li><li>Personalization</li></ul><p>That is why your final suggestions feel intelligent.</p><p>This is the core idea behind modern e-commerce search:</p><p><strong>Text → Embedding → Vector Search → Similar Products → Ranking → Final Suggestions</strong></p><p>That is how searching one product often reveals an entire shopping ecosystem.</p><h3>Lexical search</h3><p>Lexical search means:</p><blockquote><em>Search based on </em><strong><em>actual words/characters typed</em></strong><em>, not meaning.</em></blockquote><p>It tries to match <strong>text tokens literally</strong></p><p>Example:</p><p>Database:</p><pre>1. Shoes<br>2. Running Sneakers<br>3. Sports Shorts<br>4. Television</pre><p>User searches:</p><pre>shoe</pre><p>Lexical search asks:</p><blockquote><em>“Which documents contain the word </em><em>shoe or something textually close?</em></blockquote><p>Result:</p><pre>✓ Shoes<br>✗ Running Sneakers<br>✗ Television</pre><p>Because it matches <strong>words</strong>, not concepts.</p><h3>How lexical search works internally</h3><h3>1. Tokenization</h3><p>Product:</p><pre>Running Sports Shoes</pre><p>becomes:</p><pre>[&quot;running&quot;, &quot;sports&quot;, &quot;shoes&quot;]</pre><h3>2. Build an inverted index</h3><p>Instead of scanning all rows.</p><p>Index:</p><pre>running → [Product 1]<br>sports  → [Product 1]<br>shoes   → [Product 1]</pre><p>Now search becomes very fast.</p><h3>3. Ranking</h3><p>Search:</p><pre>shoe</pre><p>System scores:</p><pre>TF-IDF<br>BM25<br>Exact match<br>Position<br>Popularity</pre><p>Output:</p><pre>Running Shoes<br>Nike Shoes<br>Sports Shoes</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/767/0*_i1rrZDJJaUqjjMj.png" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=92b948c98aed" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[What is a Vector Database ?]]></title>
            <link>https://chsatyam.medium.com/what-is-a-vector-database-97d558e339c8?source=rss-3c0b02a60007------2</link>
            <guid isPermaLink="false">https://medium.com/p/97d558e339c8</guid>
            <category><![CDATA[vector]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[vector-embeddings]]></category>
            <category><![CDATA[vector-search]]></category>
            <category><![CDATA[vector-database]]></category>
            <dc:creator><![CDATA[Satyam Choudhary]]></dc:creator>
            <pubDate>Fri, 05 Jun 2026 06:01:40 GMT</pubDate>
            <atom:updated>2026-06-10T10:41:32.518Z</atom:updated>
            <content:encoded><![CDATA[<h3>Vector Databases: The Essential Guide for Modern AI Applications</h3><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/0*xOp9hyyqErOFvEnY.png" /></figure><p>Before Understanding the <strong>vector database , let’s understand the Vector’s First —</strong></p><h3>What is Vector ?</h3><p><strong>Vector (Basic Math Meaning)<br> </strong>In mathematics, a <strong>vector</strong> is:</p><p>A quantity that has:</p><ul><li><strong>Magnitude</strong> (length)</li><li><strong>Direction</strong></li></ul><p>Example:<br> (3,4) → a vector in 2D space</p><ul><li>It means: move <strong>3 units right, 4 units up</strong></li></ul><p>You can imagine it as an <strong>arrow from the origin (0,0)</strong> pointing somewhere.</p><h3>In AI, a vector is:</h3><p>A <strong>list of numbers that represents meaning.</strong></p><p>Example:</p><p>“dog” → [0.21, -0.44, 0.78, …]</p><p>“cat” → [0.19, -0.40, 0.75, …]</p><p>These numbers are created by models and capture <strong>semantic meaning</strong>.</p><ul><li>Similar words → similar vectors</li><li>Different meanings → far apart vectors</li></ul><p>“dog” → [0.21, -0.44, 0.78, …] These number’s are called Vector Embedding .<br><em>If you want a detailed explanation of Vector Embedding as well,<br></em><a href="https://chsatyam.medium.com/embedding-in-llm-b89f5217d352?postPublishedType=repub"><em>Click Here</em></a></p><h3><strong>Why Vectors Matter ?</strong></h3><p>Vectors let computers:</p><ul><li>Compare meaning (not just exact words)</li><li>Find similarity</li></ul><p>One important detail: Embeddings capture <strong>usage context</strong>, not dictionary definitions.<br> For example:</p><ul><li><em>bank</em> (money) and <em>loan</em> → close</li><li><em>bank</em> (river edge) and <em>water</em> → close</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/912/0*rXGLkW7ywaAFTWy5.png" /><figcaption>Vector Database</figcaption></figure><h3>Now , Let’s understand the Vector database —</h3><p><strong>A vector database stores data as embeddings (vectors) — numerical representations of text, images, audio, etc.</strong></p><p>A <strong>vector database</strong> is a system that <strong>stores vector embeddings and helps you quickly find similar ones</strong>.</p><p>👉 Simple words: <strong>Vector DB = storage + smart search for embeddings</strong></p><p>A <strong>vector database</strong> is a special database that stores <strong>embeddings</strong> (numerical representations of meaning). But it does more than just save them. <br>Its real job is to <strong>quickly find and use similar embeddings</strong> so applications can work with meaning, not just exact words.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wdYaYEVjmqPwqYbSIbpjZA.png" /></figure><h3><strong>Real Problem Without Vector DB</strong></h3><p><strong>Example: E-commerce Search</strong></p><p>User:</p><p>“cheap shoes”</p><p>Database:</p><ul><li>“budget sneakers”</li><li>“formal leather shoes”</li></ul><h4>Without vector DB:</h4><ul><li>Only exact word match</li><li>Misses “budget sneakers”</li></ul><h4>With vector DB:</h4><ul><li>Understands meaning</li><li>Returns correct result</li></ul><h3>How a Vector Database Works (Step by Step)</h3><blockquote>🔄 Workflow</blockquote><blockquote><strong>Data (text, images, GIFs, etc.)</strong><br> → <strong>Convert into embeddings</strong><br> → <strong>Store embeddings in the vector database</strong></blockquote><blockquote><strong>User query</strong><br> → <strong>Convert query into an embedding</strong><br> → <strong>Perform similarity search</strong><br> → <strong>Retrieve top matching results</strong><br> → <strong>Return relevant responses</strong></blockquote><h3>End-to-End Flow</h3><ol><li><strong>Input Data</strong> — Text, images, GIFs, or other content</li><li><strong>Generate Embeddings</strong> — Convert the content into numerical vectors</li><li><strong>Store in Vector DB</strong> — Save embeddings for efficient retrieval</li><li><strong>User Sends a Query</strong> — The query is also converted into an embedding</li><li><strong>Similarity Search</strong> — Compare the query vector with stored vectors</li><li><strong>Return Top Matches</strong> — Retrieve the most relevant results</li></ol><blockquote>Data(Text , image , etc..) → Embedding → Store in DB</blockquote><blockquote>User Query → Embedding → Search → Top matches → Return result</blockquote><h3>Now , Lets understand in deep , How <strong>Similarity or semantic Search Work’s —</strong></h3><h3>1. Perform Similarity Search ( How the DB finds similar vectors )</h3><p>After converting the user query into an embedding, the query becomes something like:</p><pre>User Query:<br>&quot;Best laptop for coding&quot;<br>↓<br>Query Vector:<br>[0.23, -0.81, 0.45, 0.91, ...]</pre><pre>Inside the Vector DB, every stored document already has vectors:</pre><pre>Document 1:<br>&quot;MacBook for developers&quot;<br>[0.25, -0.79, 0.41, 0.88]<br><br>Document 2:<br>&quot;Gaming mouse&quot;<br>[-0.42, 0.67, 0.12, -0.22]<br><br>Document 3:<br>&quot;Laptop for software engineers&quot;<br>[0.20, -0.84, 0.47, 0.93]</pre><p>The vector database compares the <strong>query vector</strong> with all stored vectors.</p><p>But not by exact text.</p><p>It calculates <strong>distance/similarity mathematically</strong>.</p><h3>Let’s Understand the Common Similarity Algorithms</h3><h3>A. Cosine Similarity (Most Common)</h3><p><strong>cosine similarity is the core idea behind vector search and RAG</strong>.<br>Measures the angle between vectors.</p><p>Think of vectors as <strong>arrows starting from the origin</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/268/1*6WAZd4uizYB6bfjJbD7vjg.png" /><figcaption>Most explanations stop at the formula.</figcaption></figure><p>Where:</p><ul><li><strong>A · B</strong> → dot product (how much the vectors point together)</li><li><strong>‖A‖ and ‖B‖</strong> → lengths (magnitudes) of the arrows</li><li><strong>θ (theta)</strong> → angle between them</li></ul><p>Because we divide by the lengths, the final value depends mainly on <strong>angle</strong>, not size.</p><p>The real understanding comes when you see <strong>what vectors represent geometrically</strong> and <strong>why angle matters more than distance</strong>.</p><h3>What Cosine Similarity Actually Means</h3><p>Imagine embeddings are arrows in space.<br>Think of vectors as <strong>arrows starting from the origin</strong>.</p><p>Each vector points in some direction.</p><p>Example:</p><pre>A = [1, 1]<br>B = [10, 10]</pre><p>Visual:</p><pre>y<br>↑<br>10 |                              ● B(10,10)<br> 9 |                           ↗<br> 8 |                        ↗<br> 7 |                     ↗<br> 6 |                  ↗<br> 5 |               ↗<br> 4 |            ↗<br> 3 |         ↗<br> 2 |      ↗<br> 1 |  ● A(1,1)<br> 0 +--------------------------------------------→ x<br>    0   1   2   3   4   5   6   7   8   9   10</pre><h3>Intuition: Ignore arrow length, care about direction</h3><p>Imagine two arrows:</p><ul><li>Arrow A → points northeast , A = [1, 1]</li><li>Arrow B → also points northeast but is much longer , B = [10, 10]</li></ul><p>Cosine similarity says:</p><blockquote><em>“ </em>Even though B is longer — they point in <strong>the same direction</strong>. <em>so they’re highly similar.”</em></blockquote><p>So:</p><p>Lengths are very different.</p><p>But angle = <strong>0°</strong></p><p>Cosine similarity:</p><pre>Cosine Similarity = 1 / cos(0∘)=1</pre><p>Result → <strong>perfect match</strong></p><p>because they mean almost the same thing.</p><h4>Q. How cos(0∘)= 1 , let’s see the actual calculation —</h4><p>Cosine similarity only measures <strong>direction</strong>, not <strong>length (magnitude)</strong>.</p><p>You gave:</p><pre>A=(1,1)<br>B=(10,10)</pre><p>These vectors point in exactly the same direction, so the angle between them is 0∘.</p><p>Let’s calculate it step by step.</p><p>Cosine similarity formula:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/248/1*tEJUKl7pVRBB4Kub1XNXbQ.png" /></figure><p>Step 1: Compute the dot product A⋅B</p><pre>A⋅B=(1)(10)+(1)(10)<br>=10+10=20<br></pre><p>IMP- <br>Dot product (also called scalar product) has a <strong>definition</strong>:<br> A⋅B=A1B1 + A2B2 + ⋯+ AnBn.</p><p>Step 2: Compute the lengths (magnitudes)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/862/1*KRUGLPUx50hv45lxg54Dgw.png" /></figure><p>Step 3: Now , Plug into formula​<br> ​</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/754/1*ILC7v8VUXtxQ6PibEfB0FA.png" /></figure><p>Let’s Take one More Example —</p><p>Suppose:</p><pre>A=(10,0) <br>B=(10,4)</pre><p>Visually:</p><pre>y<br>↑<br>5 |<br>4 |                    B(10,4) ●<br>3 |<br>2 |<br>1 |<br>0 |------------------------------●------→ x<br>                          A(10,0)<br>   0   2   4   6   8   10  12</pre><p>B points mostly to the right, but slightly upward.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/764/1*yYxrla_eQYKDu47Ml2k97g.png" /></figure><h3>Interpretation</h3><p>Cosine similarity = <strong>0.93</strong> means:</p><ul><li>vectors are pointing <strong>almost the same direction</strong></li><li>not perfectly aligned</li><li>only a small difference</li></ul><p>Think of recommendation systems:</p><ul><li>User A likes → Movies, Cricket, Music</li><li>User B likes → Movies, Cricket, Music, Travel</li></ul><p>Their interest vectors point in nearly the same direction → similarity close to <strong>1</strong>.</p><p>Another intuition:</p><ul><li><strong>1.0</strong> → same direction ✅</li><li><strong>0.9</strong> → very similar 👍</li><li><strong>0.5</strong> → somewhat related 🤔</li><li><strong>0</strong> → unrelated ↔️</li><li><strong>−1</strong> → opposite directions ❌</li></ul><h4>IMPORTANT things to remember :</h4><ul><li><strong>Similar words → similar vectors</strong><br> Words (or sentences) with related meanings are represented as points that end up <strong>close together</strong> in a high-dimensional space.</li><li>Example:</li><li><em>king</em> → [0.21, -0.84, ...]</li><li><em>queen</em> → [0.24, -0.79, ...]<br> These vectors are close because the meanings are related.</li><li><strong>Different meanings → far apart vectors</strong><br> Words with unrelated meanings get vectors that are farther apart.</li><li>Example:</li></ul><blockquote><em>banana</em> and <em>quantum mechanics</em> → large distance</blockquote><blockquote><em>cat</em> and <em>dog</em> → smaller distance</blockquote><p>This is the intuition behind <strong>semantic embeddings</strong> in NLP.</p><p>A common way to measure similarity is:</p><ul><li><strong>Cosine similarity ≈ 1</strong> → very similar meaning</li><li><strong>Cosine similarity ≈ 0</strong> → unrelated</li><li><strong>Cosine similarity &lt; 0</strong> → opposite direction / very different (depending on embedding model)</li></ul><p>One important detail: embeddings capture <strong>usage context</strong>, not dictionary definitions.<br> For example:</p><ul><li><em>bank</em> (money) and <em>loan</em> → close</li><li><em>bank</em> (river edge) and <em>water</em> → close</li></ul><p>Modern models create <strong>contextual embeddings</strong>, so the vector for <em>bank</em> changes depending on the sentence.</p><h3>Why Vector Databases Use Angle Instead of Distance —</h3><h3>Example: Text embeddings</h3><p>Suppose an embedding model converts sentences into vectors.</p><p>Sentence 1:</p><blockquote><em>“I love dogs”</em></blockquote><pre>Vector: (0.8,0.2)</pre><p>Sentence 2:</p><blockquote><em>“I really really love dogs”</em></blockquote><pre>Vector: (8,2)</pre><p>Distance says:</p><ul><li>far apart ❌</li></ul><p>Cosine similarity says:</p><ul><li>same direction → highly similar ✅</li></ul><p>Because the meaning is nearly the same.</p><h3>Another Example —</h3><p>The key idea is: <strong>vector databases care more about direction than physical distance.</strong></p><p>Let’s use one real world example.</p><p>Query vector: <br>Q=[1,2]</p><p>Document A: <br>A=[100,200]</p><p>Document B: <br>B=[−1,−2]</p><p>Imagine vectors drawn from the origin:</p><pre>y<br>↑<br>220 |                                              ● A(100,200)<br>200 |                                           ↗<br>180 |                                        ↗<br>160 |                                     ↗<br>140 |                                  ↗<br>120 |                               ↗<br>100 |                            ↗<br> 80 |<br> 60 |<br> 40 |<br> 20 |      ↗ Q(1,2)<br>  0 +------●----------------------------------------------→ x<br>-20 |    ↙<br>-40 |<br>     ● B(-1,-2)<br><br>     -20     0      20      40      60      80      100</pre><p>Notice:</p><ul><li><strong>Query</strong> and <strong>Doc A</strong> point to the <strong>same direction</strong></li><li><strong>Doc B</strong> points in the <strong>opposite direction</strong></li></ul><p>Where:</p><ul><li>Query vector <strong>Q = [1,2]</strong></li><li>Document A = <strong>[100,200]</strong></li><li>Document B = <strong>[-1,-2]</strong></li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/778/1*cnwYfCa9Q27w78ZQgbrymA.png" /></figure><p>Where:</p><ul><li>Query vector <strong>Q = [1,2]</strong></li><li>Document A = <strong>[100,200]</strong></li><li>Document B = <strong>[-1,-2]</strong></li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/778/1*d0Hq6mgs3BpDkYQsesmJWQ.png" /></figure><h3>B. Euclidean Distance</h3><p>Measures physical distance between vectors.</p><p>Euclidean Distance = “How far apart are the points?”<br>Instead of comparing <strong>angles</strong>, Euclidean distance compares <strong>actual physical distance</strong>.<br>Smaller distance = more similar.</p><p>Imagine vectors as <strong>locations in space</strong>, not arrows.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/588/1*FFUBbahGaB_b0rUkyzp2Mw.png" /></figure><p>then distance is:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/648/1*rR7hjJIOqvGMIbgqZoHH4w.png" /></figure><p>This is just the <strong>Pythagorean theorem</strong>.</p><pre>You have two points:<br>A=(x1​,y1​), B=(x2​,y2​)</pre><p>These are <strong>locations in space</strong>, not arrows.</p><p>Your goal:</p><blockquote><em>Find the straight-line distance from A to B.</em></blockquote><h3>Step 1: Draw the right triangle</h3><p>From the image</p><pre>          B(x₂,y₂)<br>            ●<br>           /|<br>          / |<br>       d /  | BC<br>        /   |<br>       /____|<br>     A   AC<br>   (x₁,y₁)</pre><p>AB = actual distance (what we want) , AB is also called the d .</p><p>AC = horizontal movement</p><p>BC = vertical movement</p><h3>Step 2: Compute horizontal side AC</h3><p>Look at x-values:</p><pre>A starts at: x1<br> ​B ends at: x2<br> ​<br><br><br>So horizontal distance:<br><br>AC=x2−x1<br> ​</pre><p>Example:</p><pre>If:<br><br>A=(2,3)<br>B=(8,7)<br><br>            B(x₂,y₂)<br>            ●<br>           /|<br>          / |<br>       d /  | BC<br>        /   |<br>       /____|<br>     A   AC<br>  A(x₁,y₁)<br><br><br><br>then:<br><br>AC=8−2=6<br><br>Meaning:<br><br>Move 6 units right.</pre><h3>Step 3: Compute vertical side BC</h3><pre>             B(x₂,y₂)<br>            ●<br>           /|<br>          / |<br>       d /  | BC<br>        /   |<br>       /____|<br>     A   AC<br>  A(x₁,y₁)<br><br>A=(2,3)<br>B=(8,7)<br><br><br>Look at y-values:<br><br>BC=y2 ​−y1<br> ​<br><br>Example:<br><br>BC=7−3=4<br><br>Meaning:<br><br>Move 4 units up</pre><p>Now triangle becomes:</p><pre>      B<br>      ●<br>     /|<br>    / |<br>   /  | 4<br>  /   |<br> /____|<br>A   6</pre><h3>Step 4: Apply Pythagorean theorem</h3><p>You already know:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/250/1*XStDUJRYG2qUi15iJyh6pQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/648/1*u7maK4bBz4Re1jQRlfShzA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/654/1*vLEY6wcwihHTLKCONajzcw.png" /></figure><h3>Example from embeddings :</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/702/1*Gyer2kmDjt220AiGC8GiNw.png" /></figure><p>Interpretation:</p><p>The document is <strong>5 units away in vector space</strong>.</p><p>Smaller distance → more similar.</p><blockquote>Now compare this with cosine:</blockquote><blockquote><strong>Euclidean → “How far are the points?”</strong></blockquote><blockquote><strong>Cosine → “Are they pointing in the same direction?”</strong></blockquote><h3>Example — Query vs Documents</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/742/1*cCsZZpBinCUbi4lMZfDlBw.png" /></figure><h3>Real Systems Don’t Compare Every Vector -</h3><p>If you have <strong>1 billion vectors</strong>, comparing against every vector is impossible in real-time.</p><p>If DB has:</p><pre>1 Billion vectors</pre><p>Comparing one-by-one would be slow.</p><p>So vector databases use:</p><h3>ANN (Approximate Nearest Neighbor)</h3><p>Algorithms:</p><pre>HNSW<br>IVF<br>PQ<br>ScaNN</pre><pre>Flow:<br><br>Query Vector<br>     ↓<br>Navigate Graph / Index<br>     ↓<br>Skip irrelevant vectors<br>     ↓<br>Find nearest candidates</pre><pre>Time becomes:<br><br>Milliseconds<br>instead of hours</pre><h3>2. Retrieve Top Matching Results</h3><p>After similarity calculation:</p><p>DB creates ranking.</p><p>Example:</p><pre>Query:<br>&quot;Best coding laptop&quot;<br><br>Results:<br><br>1. MacBook Air      → 0.95<br>2. Dell XPS         → 0.93<br>3. ThinkPad         → 0.89<br>4. Gaming Laptop    → 0.44</pre><p>Top-K retrieval:</p><pre>K = 3</pre><pre>Return:<br><br>[<br> MacBook Air,<br> Dell XPS,<br> ThinkPad<br>]</pre><p>This is called:</p><pre>Top-K Nearest Neighbor Searc</pre><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=97d558e339c8" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>