<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by praveenreddy_c on Medium]]></title>
        <description><![CDATA[Stories by praveenreddy_c on Medium]]></description>
        <link>https://medium.com/@mailpraveenreddy.c?source=rss-2d3bb2dcf96a------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*ezg8rSdlmauz2CDidsXbaw.jpeg</url>
            <title>Stories by praveenreddy_c on Medium</title>
            <link>https://medium.com/@mailpraveenreddy.c?source=rss-2d3bb2dcf96a------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sun, 22 Nov 2026 03:36:38 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@mailpraveenreddy.c/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[My Understanding of Indexes in Vector Databases: Flat Index, ANN, HNSW, HFresh]]></title>
            <link>https://medium.com/@mailpraveenreddy.c/my-understanding-of-indexes-in-vector-databases-flat-index-ann-hnsw-hfresh-20064f5f2a56?source=rss-2d3bb2dcf96a------2</link>
            <guid isPermaLink="false">https://medium.com/p/20064f5f2a56</guid>
            <category><![CDATA[indexing]]></category>
            <category><![CDATA[ivf]]></category>
            <category><![CDATA[vector-database]]></category>
            <category><![CDATA[hfresh]]></category>
            <category><![CDATA[hnsw]]></category>
            <dc:creator><![CDATA[praveenreddy_c]]></dc:creator>
            <pubDate>Mon, 01 Jun 2026 12:49:24 GMT</pubDate>
            <atom:updated>2026-06-01T12:49:24.815Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*INbacS5-opb09u_8i24ugw.png" /><figcaption>ChatGPT</figcaption></figure><p>While creating my RAG playlist on my YouTube “<strong>Praveen Reddy Learnings”</strong>, I came to a topic: <strong>Indexing in Vector Databases</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/536/1*ownPWGj6hyWgOdMoIO-4Mg.png" /></figure><p>As I started preparing for it, I realised that I had never actually wrote a blog dedicated to vector indexes, which is why I decided to write this blog.</p><p>We need to understand, <strong>how does a Vector Database search through millions of vectors so quickly?</strong> That’s where indexes come in. So in this blog, let’s start from the basics, understand why indexes are needed in Vector Databases, explore the different indexing techniques available, and then dive deep into the most popular one used today: <strong>HNSW</strong>.</p><h3>Indexes</h3><blockquote>How does a Vector Database find the most similar vectors so quickly?</blockquote><p>Imagine we have only 100 vectors stored in your database. When a user asks a question, the user query is first converted into an embedding, and the database can simply compare the user query vector against all 100 vectors to find the most similar matches(<em>using cosine similarity or so…</em>).</p><p>Here it is a small dataset so, it is fine but, what happens when the dataset grows? What if we have 10 Million vectors ?</p><p>For every user query, the database would need to compare the user query vector against 10 Million vectors stored in the vector DB. This approach is known as <strong>Brute Force Search</strong>.</p><blockquote>The major drawback here is: <strong>Speed.</strong></blockquote><p>As the number of vectors increases, the number of similarity calculations also increases. What works well for hundreds of vectors quickly becomes impractical when dealing with millions or even billions of vectors. The search latency starts increasing, computational costs go up, and the user experience suffers. <strong>This is exactly the problem that indexes are designed to solve.</strong></p><p>Instead of checking every vector in the database, an index helps narrow down the search space and quickly guide the search toward regions where similar vectors are most likely to be found.</p><p>As we all know that, the concept of indexing is not new. We have been using indexes in traditional relational databases for decades. Imagine an employees table containing millions of records, and a user executes a query like:</p><pre>SELECT *<br>FROM employees<br>WHERE employee_id = 1001;</pre><p>Without an index, the database would have to scan every row in the table until it finds the matching employee. This process, known as a <strong>full table scan</strong>, becomes increasingly expensive as the table grows. To solve this problem, databases create indexes on frequently searched columns such as employee_id. Once an index is available, the database can directly navigate to the required records instead of searching through all rows, significantly reducing query execution time. Vector Databases face a very similar challenge. Instead of searching through millions of vector rows. Just as traditional databases use indexes to avoid scanning entire tables, <strong>Vector Databases use specialised vector indexes to avoid comparing a query against every stored vector, making similarity search fast and scalable.</strong></p><p>Before we dive into the different types of vector indexes, below are the three key factors that every indexing algorithm is trying to optimise:</p><h4>1. Search Speed</h4><p>How quickly can the database return results?</p><h4>2. Accuracy (Recall)</h4><p>How close are the returned results to the actual nearest neighbours?</p><h4>3. Memory Consumption</h4><p>How much memory is required to store and maintain the index?</p><p>Now, let’s dive into some of the most commonly used indexing techniques in Vector Databases.</p><h3>1. Flat Index (Brute Force)</h3><p>The simplest indexing technique available in Vector Databases is the <strong>Flat Index</strong>, often referred to as <strong>Brute Force Search</strong>. Although it is called an index, it doesn’t perform any optimization. The vectors are simply stored as they are, and whenever a query arrives, the database compares the query vector against every single vector in the dataset. It calculates the similarity score for each vector, sorts the results, and returns the top matching vectors. In other words, nothing is skipped — every vector is examined before producing the final result.</p><h3>2. Approximate Nearest Neighbour (ANN)</h3><p>As datasets grew larger, researchers realised that most AI applications do not need the mathematically perfect nearest neighbour. For example, if the best match has a similarity score of <strong>0.95</strong> and another result scores <strong>0.94</strong>, the difference is often negligible from a user’s perspective. This led to the concept of <strong>Approximate Nearest Neighbour (ANN)</strong>, where the goal is to find results that are very close to the actual nearest neighbours while dramatically improving search speed.</p><h4>IVF (Inverted File Index)</h4><blockquote>One of the earliest ANN techniques is <strong>IVF (Inverted File Index)</strong>. Instead of searching every vector, first group similar vectors into clusters and then search only within the most relevant cluster.</blockquote><h4><strong>IVF Working:</strong></h4><p>During index creation, vectors are grouped into multiple clusters, and a centroid is created for each cluster. When a query arrives, the database first finds the nearest centroid and then searches only the vectors belonging to that cluster.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*nGqxoPquvNV3ljtO9XUrQg.png" /><figcaption>ChatGPT</figcaption></figure><p>This significantly reduces the search space. Instead of comparing against one million vectors, the database may only need to examine a few thousand vectors within the selected cluster, resulting in much faster searches.</p><h4>IVF Limitations:</h4><p>The main limitation of IVF is that the actual nearest neighbor may exist in a different cluster than the one selected for the search. If that cluster is never examined, the best result can be missed, reducing recall, which is being handled in the next technique HNSW.</p><h3>HNSW (Hierarchical Navigable Small World)</h3><p>Now comes the most popular indexing technique used in modern Vector Databases: <strong>HNSW (Hierarchical Navigable Small World)</strong>. The reason for its popularity is simple — it provides an excellent balance between <strong>search speed, recall, and scalability</strong>. Because of this balance, HNSW has become the default indexing choice for many production-grade vector databases.</p><p>In HNSW, every vector is represented as a node, and each node is connected to a set of nearby nodes. Together, these connections form a graph. When a query arrives, instead of comparing against every vector in the dataset, the algorithm navigates through this graph, hopping from node to node until it reaches the vectors most similar to the query.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*hv48xW7EHJwMTMcG4mHN0g.png" /><figcaption>ChatGPT</figcaption></figure><p>Instead of maintaining a single graph, HNSW organises the graph into multiple layers.</p><ul><li><strong>Top Layer:</strong> Very sparse, containing only a few nodes that provide a rough starting point.</li><li><strong>Middle Layers:</strong> More nodes and connections, helping refine the search.</li><li><strong>Bottom Layer:</strong> Contains almost all vectors and performs the final fine-grained search.</li></ul><blockquote><em>A useful analogy is navigating with maps: </em>You might first look at a country map to find the right region, then zoom into a state map, then a city map, and finally a street map. HNSW follows a similar approach, progressively narrowing down the search as it moves through the layers.</blockquote><p>Because only a small portion of the graph is explored, HNSW can find highly relevant vectors without examining the entire dataset, making searches extremely efficient.</p><p>HNSW has become the industry standard because it offers an attractive combination of benefits:</p><ul><li><strong>Extremely Fast Search:</strong> Only a small fraction of the dataset is explored during retrieval.</li><li><strong>High Recall:</strong> Often achieves recall rates above 95%, meaning it finds results very close to the true nearest neighbours.</li><li><strong>Excellent Scalability:</strong> Works efficiently even when the dataset contains millions of vectors.</li></ul><h4>HNSW Parameters</h4><h4>M:</h4><p>The <strong>M</strong> parameter controls how many neighbours each node maintains in the graph.</p><p>Higher <strong>M</strong></p><ul><li>Better graph connectivity</li><li>Higher recall</li><li>Increased memory usage</li></ul><p>Lower <strong>M</strong></p><ul><li>Lower memory consumption</li><li>Faster index creation</li><li>Reduced recall</li></ul><h4>efConstruction</h4><p>The <strong>efConstruction</strong> parameter controls how much effort HNSW spends while building the graph. During index creation, HNSW evaluates multiple candidate nodes before deciding which neighbours should be connected.</p><p>Higher efConstruction:</p><ul><li>Better graph quality</li><li>Higher recall</li><li>Longer indexing time</li></ul><p>Lower efConstruction:</p><ul><li>Faster index creation</li><li>Lower graph quality</li><li>Potentially lower recall</li></ul><h4>efSearch</h4><p>The <strong>efSearch</strong> parameter is used during query time and determines how many candidate nodes are explored during the search.</p><p>Higher efSearch:</p><ul><li>Explores more nodes</li><li>Better recall</li><li>Slower search</li></ul><p>Lower efSearch:</p><ul><li>Faster search</li><li>Lower recall</li></ul><p>A simple <strong>efSearch</strong> definition is:</p><blockquote>How much effort should the database spend looking for the best possible answer?</blockquote><p>The higher the value, the more effort it spends and the better the chances of finding the true nearest neighbors.</p><h4>Dynamic Indexes</h4><p>While HNSW works exceptionally well for large datasets, maintaining a graph structure for very small datasets may be unnecessary. To address this, some Vector Databases such as Weaviate introduced <strong>Dynamic Indexes</strong>.</p><p>The idea is simple:</p><blockquote>For small datasets, the database uses a Flat Index because brute-force search is already fast enough. As the dataset grows, it automatically switches to HNSW to maintain high search performance. This approach avoids the overhead of maintaining an HNSW graph for tiny datasets while still providing scalability when the amount of data increases.</blockquote><h3>HFresh</h3><p>While HNSW has become the most popular indexing technique, it comes with a tradeoff: <em>the graph structure requires additional memory</em>. As the number of vectors grows into the hundreds of millions or billions, maintaining a large graph can become expensive. To address this challenge, Weaviate introduced a newer indexing approach called <strong>HFresh</strong>.</p><p>Instead of building an HNSW graph for every vector in the dataset, HFresh first groups similar vectors into clusters. Rather than navigating through millions of individual vectors, it creates an HNSW graph only on the cluster centroids. During a search, the system first identifies the most relevant clusters and then searches within those clusters to find the final results.</p><p>This approach significantly reduces memory consumption because the graph is maintained on a much smaller set of centroids rather than on every vector. As a result, HFresh can scale more efficiently for extremely large datasets while still delivering good search performance.</p><p>For most RAG applications and medium-sized datasets, HNSW remains the most commonly used choice. However, for very large deployments where memory efficiency becomes a major concern, HFresh can be an attractive alternative.</p><p>So,</p><ul><li>Flat Index gives perfect accuracy but doesn’t scale.</li><li>IVF introduced the idea of reducing the search space through clustering.</li><li>HNSW took things further by organizing vectors into a hierarchical graph structure that allows extremely fast navigation while maintaining high recall.</li></ul><p>That’s it all about Indexing in vector stores.</p><h3>Thank You !!</h3><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=20064f5f2a56" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[My Understanding of vLLM and PagedAttention]]></title>
            <link>https://medium.com/@mailpraveenreddy.c/my-understanding-of-vllm-and-pagedattention-58cfafa30f3d?source=rss-2d3bb2dcf96a------2</link>
            <guid isPermaLink="false">https://medium.com/p/58cfafa30f3d</guid>
            <category><![CDATA[vllm]]></category>
            <category><![CDATA[llm-inference]]></category>
            <category><![CDATA[pagedattention]]></category>
            <dc:creator><![CDATA[praveenreddy_c]]></dc:creator>
            <pubDate>Sun, 24 May 2026 15:09:36 GMT</pubDate>
            <atom:updated>2026-05-24T15:09:36.821Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ddOTiT70FM4XkGFFV0s2rw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/863/1*f4lEtmYIIYP_AoqkXcEj0Q.png" /></figure><p>I wish this is the last major topic I need to deeply understand in the LLM and fine-tuning space for this year, before I move back to creating YouTube videos on RAG and Fine-Tuning in my YouTube Channel “<strong><em>Praveen Reddy Learnings</em></strong>”.</p><p>For the past few months, I have been stuck trying to properly understand some core LLM inference concepts. One of those topics recently led me to the <strong>PagedAttention paper </strong>while exploring how KV-Cache actually works internally.</p><p>Before understanding this paper, I strongly recommend to check out my KV-Cache blog first, because this entire paper is built on top of the KV-Cache bottleneck problem.</p><p><a href="https://medium.com/@mailpraveenreddy.c/understanding-kv-cache-in-llms-bfc2656242df">Understanding KV Cache in LLM’s</a></p><p>As we already discussed in the KV-Cache blog above, KV-Cache helps LLMs avoid recomputing attention for previously generated tokens again and again. Since autoregressive models take the entire sequence generated so far as input to predict the next token, the model would normally have to recompute the Keys and Values for all previous tokens at every generation step.</p><p>To avoid this repeated computation, KV-Cache stores the previously computed Keys and Values in memory and reuses them during next-token generation, significantly improving inference speed and efficiency.</p><p>But they mentioned in the paper that, the moment LLMs starts serving multiple users simultaneously, KV-Cache itself became one of the biggest bottlenecks in production systems.</p><h4>Challenges with Traditional KV-Cache Systems</h4><h4>1. KV-Cache Consumes Huge GPU Memory</h4><p>For every generated token:</p><ul><li>Key vectors and Value vectors are stored across every transformer layer and across every attention head present inside those across all transformer layers and attention heads</li></ul><blockquote>Since each transformer layer contains multiple attention heads, every head computes its own separate Query, Key, and Value vectors <strong>for every token.</strong></blockquote><p>For example, if a single transformer layer has 10 attention heads, then for a single word/token:</p><ul><li>10 different Query vectors, 10 different Key vectors and 10 different Value vectors are generated inside that layer itself, because each head learns different relationships and attention patterns from the same token.</li></ul><p>This means KV-Cache grows continuously as tokens increase.</p><h4>2. Sequence Length is Unpredictable</h4><p>We never know how long a user will generate text. User A may stop conversation with LLM after 30 tokens, User B may continue conversation for 4000 tokens. So inference systems cannot accurately preallocate memory.</p><p>To stay safe: <em>systems reserve much larger memory regions for storing KV Cache than actually needed. </em>This creates memory waste.</p><h4>3. Internal Fragmentation</h4><p>Let’s assume, A system allocates memory for: <em>2048 tokens. </em>But if the user generates only: <em>300 tokens. </em>Remaining memory stays unused. This unused reserved memory is called: <strong>Internal Fragmentation</strong></p><h4>4. External Fragmentation</h4><p>In the realtime, many requests will be arriving and finishing dynamically. GPU memory starts looking like:</p><pre>[A][free][B][free][C]</pre><p>Here, even though the total free memory available might actually be enough for a new request, the problem is that the free memory is split into multiple smaller chunks instead of one large continuous block. Traditional memory allocation systems often require one large contiguous memory region to allocate KV-Cache for a new sequence. Since such a continuous block may not exist anymore, the allocation fails even though enough total memory is technically available. This problem is called <strong>External Fragmentation.</strong></p><p>They mentioned in the paper that, This is exactly the same memory management problem the traditional operating systems faced before the concept of <strong>Virtual Memory Paging</strong>.</p><p>As LLM serving systems(like ChatGPT) have started handling:</p><ul><li>Longer conversations</li><li>More concurrent users</li><li>Larger KV-Caches</li><li>Dynamic batching</li><li>Advanced decoding strategies</li></ul><p>Traditional contiguous memory allocation started becoming highly inefficient.</p><blockquote>Why are we managing KV-Cache like rigid continuous tensors instead of managing them like operating system memory?</blockquote><p>and that single idea eventually led to the introduction of:</p><ul><li><strong>PagedAttention</strong></li><li>and the<strong> vLLM serving architecture.</strong></li></ul><blockquote>Instead of allocating one large continuous memory region for every token sequence, vLLM treats KV-Cache memory similarly to virtual memory paging in operating systems by <strong>dividing KV-Cache into smaller fixed-size blocks that can be allocated dynamically across GPU memory.</strong> This reduces memory fragmentation, improves GPU utilisation, and enables efficient large-scale LLM serving.</blockquote><p>To solve these KV-Cache memory challenges, the paper introduces a new memory management mechanism called <strong>PagedAttention</strong>.</p><h4>Core Idea Behind PagedAttention</h4><p>In operating systems:</p><ul><li>Memory is divided into pages, pages are mapped logically &amp; physical memory does not need to be contiguous.</li></ul><p>Similarly:</p><p>vLLM divides KV-Cache into:</p><ul><li>Fixed-size blocks/pages. Instead of allocating one huge continuous memory chunk, KV-Cache grows block-by-block dynamically.</li></ul><h4>Traditional KV-Cache Layout</h4><p>Older systems used like:</p><pre>[Sequence A Continuous Memory]</pre><p>If sequence grows:</p><ul><li>Larger memory allocated, old cache copied, previous memory freed.</li></ul><p>This creates:</p><ul><li>Fragmentation</li><li>Copying overhead</li><li>Allocation inefficiency.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/504/1*dDo75oc9pyh-FK3G9AaKNA.png" /></figure><h4>PagedAttention Layout</h4><p>Each block can exist anywhere in GPU memory. No need for contiguous allocation. This completely changes memory management.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*05ptz9QpLnpxdgrDvG0MEA.png" /></figure><h4>Here we will see on how Paged Attention is implemented:</h4><h4>1. Logical Blocks</h4><p>Instead of treating the entire KV-Cache as one huge continuous memory region, PagedAttention divides the sequence into smaller fixed-size chunks called logical blocks. These blocks are created based on the token order in the sequence.</p><p>Example:</p><pre>Tokens 1–16  -&gt; Belongs to Logical Block 0<br>Tokens 17–32 -&gt; Belongs to Logical Block 1</pre><h4>2. Physical Blocks</h4><p>Physical Blocks represent the actual locations where KV-Cache data is stored inside GPU memory. While Logical Blocks maintain the sequence order of tokens, Physical Blocks are the real memory blocks allocated on the GPU hardware.</p><p>Example:</p><pre>Logical Block 0 -&gt; Physical Block 7<br>Logical Block 1 -&gt; Physical Block 21</pre><h4>3. Block Table</h4><p>In PagedAttention, the Block Table is responsible for tracking where each part of a sequence is actually stored inside GPU memory. Since KV-Cache is divided into smaller blocks and these blocks can be placed anywhere in memory, the system needs a way to identify which physical memory block belongs to which part of the sequence.</p><p>The Block Table maintains this mapping information. For every Logical Block in a sequence, it stores the corresponding Physical Block location where the actual KV data exists.</p><p>The block table connect: <strong><em>Logical Block to the Physical Block</em></strong></p><p>Example:</p><pre>Logical Block 0 may point to Physical Block 7<br>Logical Block 1 may point to Physical Block 21<br>Logical Block 2 may point to Physical Block 3</pre><p>Because of this block-based memory design, KV-Cache can now grow dynamically as new tokens are generated, instead of requiring one huge preallocated continuous memory region from the beginning. As blocks can now be allocated independently anywhere in GPU memory, the system no longer depends on finding large contiguous free spaces for every sequence. This dramatically reduces both internal and external memory fragmentation.</p><p>Now, let’s understand the overall vLLM System Architecture and how different components work together to efficiently serve Large Language Models at scale.</p><h4>vLLM System Architecture</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*naCtE3slpTSeGf0-WIjjCQ.png" /></figure><h4>1. Request Scheduler</h4><p>The Request Scheduler is responsible for managing incoming user requests and efficiently organising them for GPU execution. In real-world LLM serving systems, users continuously send prompts at different times, and every request may have completely different sequence lengths and generation speeds.</p><p>The request scheduler handles:</p><ul><li>Accepting new requests</li><li>Batching multiple requests together</li><li>Deciding the execution order</li><li>Dynamically inserting new sequences and removing completed sequences</li></ul><p>One of the most important capabilities enabled by the request scheduler is <strong>Continuous Batching.</strong></p><blockquote><strong>Batch means: </strong>Group of multiple user requests processed together on the GPU at the same time, because GPUs are designed for parallel processing and can handle many computations simultaneously much more efficiently than processing requests one-by-one sequentially.</blockquote><p>Instead of waiting for an entire batch to finish before processing new requests, vLLM continuously updates active batches by inserting new requests whenever space becomes available. This keeps the GPU busy almost continuously and significantly improves GPU utilisation and serving throughput.</p><h4>2. KV-Cache Manager</h4><p>The KV-Cache Manager acts as the core memory management engine behind PagedAttention. Since KV-Cache is now divided into smaller blocks that can dynamically grow and move across GPU memory, the system needs a dedicated component to manage all these operations efficiently, which is done by KV Cache manager.</p><p>The KV-Cache Manager is responsible for:</p><ul><li>Allocating new blocks</li><li>Freeing unused blocks</li><li>Maintaining <strong>block tables</strong></li></ul><p>This component is one of the main reasons why vLLM can handle large numbers of concurrent requests efficiently without excessive memory fragmentation.</p><p>So overall, the KV-Cache Manager is the component that intelligently handles all these memory operations efficiently behind the scenes.</p><h4>3. PagedAttention CUDA Kernel</h4><blockquote>A kernel in GPU programming is a small highly optimized program that runs directly on the GPU to perform a specific computation in parallel across thousands of GPU cores.</blockquote><p>Traditional attention kernels are usually designed with the assumption that KV-Cache exists in one continuous memory region. However, in PagedAttention, KV blocks can be scattered across different physical memory locations on the GPU.</p><p>Because of this non-contiguous memory layout, traditional attention kernels have become inefficient. To solve this problem, vLLM introduces custom PagedAttention CUDA kernels specifically optimized for block-based KV access.</p><p>These kernels are responsible for:</p><ul><li>Gathering scattered KV blocks</li><li>Reconstructing the correct logical token order</li><li>Computing attention efficiently and maintaining high GPU throughput.</li></ul><h4>4. GPU Workers</h4><p>GPU Workers are responsible for performing the <strong>actual transformer computations</strong> during inference. Once requests are scheduled and KV blocks are managed properly, GPU workers execute the model operations required for token generation.</p><p>Their responsibilities include:</p><ul><li>Transformer layer execution</li><li>Attention computation</li><li>Feed-forward network computation and next-token generation.</li></ul><p>These workers continuously process active sequences during inference while interacting with the PagedAttention kernels to fetch the required KV blocks dynamically.</p><h4>5. CPU Coordination Layer</h4><p>While GPUs perform the heavy tensor computations, the CPU manages the control flow and coordination across different system components.</p><p>This layer handles:</p><ul><li>Request management</li><li>Scheduling decisions</li><li>Worker coordination</li><li>Memory management coordination and communication across different execution components.</li></ul><p>It acts as the central controller that ensures all parts of the vLLM serving pipeline work together efficiently during large-scale LLM inference.</p><h4>Now, Let’s understand How Decoding Works in vLLM step by step:</h4><h4>Step 1 — User Request Arrives</h4><p>Example:</p><pre>&quot;Explain transformers&quot;</pre><p>When a user sends a prompt, the Request Scheduler first accepts the request and creates a sequence for it. <em>A sequence simply represents the complete token flow associated with that request</em>, including:</p><ul><li>The input prompt tokens, and all future generated output tokens.</li></ul><p>At this stage, the sequence currently contains only the input prompt tokens: <em>&quot;Explain transformers&quot;. </em>As the model starts generating new tokens one-by-one, those generated tokens also become part of the same sequence.</p><h4>Step 2 — KV Blocks Allocated Dynamically</h4><p>Once the input prompt tokens are processed, the model generates the corresponding Keys and Values for those tokens. Instead of allocating one huge continuous memory region for storing the entire KV-Cache, vLLM allocates smaller fixed-size KV blocks dynamically and incrementally as needed.</p><p>For smaller prompts, only a small number of KV blocks may be allocated initially. <em>For example, if the prompt contains only a few tokens and fits within the configured block size, a single KV block may be sufficient.</em></p><p>Example:</p><pre>Block 1<br>Block 2</pre><p>As more tokens are generated during inference, additional KV blocks are allocated dynamically whenever required.</p><h4>Step 3 — Tokens Generated Sequentially</h4><p>As the model starts generating new tokens one-by-one, new Key and Value entries are continuously added to the KV-Cache. Additional KV blocks are allocated dynamically only when the existing blocks become full. This incremental block allocation helps avoid unnecessary memory reservation and reduces memory over-allocation.</p><h4>Step 4 — Attention Computation</h4><p>During token generation, the PagedAttention CUDA kernel fetches the required KV blocks from GPU memory, gathers the corresponding Keys and Values, and performs the attention computation needed to predict the next token. Even though the KV blocks may be physically scattered across different GPU memory locations, the Block Table ensures that the logical token ordering of the sequence remains correct during computation.</p><h4>Step 5 — Sequence Completion</h4><p>Once the request generation finishes, the KV blocks associated with that sequence are immediately freed and become available for future requests. This dynamic allocation and deallocation mechanism helps vLLM utilize GPU memory very efficiently while supporting large numbers of concurrent requests.</p><p>Now that we understood how a single request gets processed internally in vLLM using dynamically allocated KV blocks, the next important question is: <em>How does vLLM efficiently handles thousands of user requests simultaneously while keeping GPUs continuously utilized?</em></p><p>This is where one of the most important optimizations in modern LLM serving systems comes into the picture:</p><h4>Continuous Batching in vLLM</h4><p>vLLM introduces Continuous Batching. Instead of waiting for the entire batch to finish, vLLM continuously updates the active batch dynamically. As soon as one request finishes, a new incoming request is immediately inserted into the batch if GPU space becomes available.</p><p>This keeps the GPU busy almost continuously and significantly improves:</p><ul><li>Throughput</li><li>GPU utilization and request latency.</li></ul><h4>Why PagedAttention is Critical for Continuous Batching</h4><p>PagedAttention solves this problem by allowing KV-Cache blocks to be allocated dynamically and independently across GPU memory instead of requiring one large continuous memory region for every sequence. This makes dynamic insertion and removal of requests much more efficient and enables continuous batching to scale effectively.</p><h4>Performance Impact of vLLM and PagedAttention</h4><p>The paper demonstrated that PagedAttention significantly improves GPU memory utilization and overall LLM serving throughput compared to traditional inference systems. Since memory fragmentation becomes extremely small, vLLM can fit a much larger number of active sequences inside GPU memory simultaneously.</p><p>This directly improves:</p><ul><li>Concurrent request handling</li><li>Batching efficiency</li><li>Long-context inference and overall GPU utilization.</li></ul><p>Since vLLM can dynamically insert and remove requests efficiently without expensive memory reallocations, GPUs remain busy almost continuously instead of waiting for entire batches to finish.</p><p>Because of all these optimizations together, vLLM achieved significantly higher serving throughput while drastically reducing KV-Cache memory waste, which is one of the main reasons why vLLM became one of the most widely adopted LLM serving systems today.</p><blockquote>One important thing to understand here is that vLLM is an inference and serving engine that sits between the model and the GPU to make LLM inference much more efficient.</blockquote><p>Normally, during inference, people directly load models using frameworks like HuggingFace Transformers and generate outputs. But as the number of concurrent users increases, KV-Cache memory usage, fragmentation, and GPU inefficiencies start becoming major bottlenecks.</p><p><strong>This is where vLLM comes into the picture.</strong></p><p>As discussed, <em>Instead of manually handling KV-Cache and batching logic, vLLM automatically manages: PagedAttention, Dynamic KV-Cache allocation, Continuous batching, GPU memory optimization and high-throughput request serving internally.</em></p><p>vLLM is installed on the inference server or GPU machine and acts as the runtime responsible for serving the model efficiently.</p><p>Another important point is that vLLM works perfectly even with fine-tuned models such as LoRA-based models. Helps the fine-tuned model to benefit from all the optimizations provided by vLLM, including efficient KV-Cache management, PagedAttention, continuous batching, and better GPU utilization.</p><p>Below is a simple code comparison between traditional HuggingFace inference and vLLM inference.</p><h4>Traditional HuggingFace inference:</h4><pre>from transformers import AutoTokenizer, AutoModelForCausalLM<br>import torch<br>model_name = &quot;meta-llama/Llama-2-7b-chat-hf&quot;<br>tokenizer = AutoTokenizer.from_pretrained(model_name)<br>model = AutoModelForCausalLM.from_pretrained(<br>    model_name,<br>    torch_dtype=torch.float16,<br>    device_map=&quot;auto&quot;<br>)<br>prompt = &quot;Explain transformers&quot;<br>inputs = tokenizer(prompt, return_tensors=&quot;pt&quot;).to(&quot;cuda&quot;)<br>outputs = model.generate(<br>    **inputs,<br>    max_new_tokens=100<br>)<br>print(tokenizer.decode(outputs[0]))</pre><p>In this approach:</p><ul><li>HuggingFace directly handles inference</li><li>KV-Cache management is traditional</li><li>Batching optimizations are limited</li><li>GPU utilization may become inefficient at scale</li></ul><h4>Now the same idea using vLLM:</h4><pre>from vllm import LLM, SamplingParams<br>llm = LLM(<br>    model=&quot;meta-llama/Llama-2-7b-chat-hf&quot;<br>)<br>sampling_params = SamplingParams(<br>    temperature=0.7,<br>    max_tokens=100<br>)<br>#SamplingParams is used to define how the model should generate text during inference, such as controlling randomness, output length, and decoding behavior.<br>outputs = llm.generate(<br>    &quot;Explain transformers&quot;,<br>    sampling_params<br>)<br>print(outputs[0].outputs[0].text)<br></pre><p>Here, vLLM automatically handles:</p><ul><li>PagedAttention</li><li>KV-Cache block allocation</li><li>Continuous Batching</li><li>GPU memory optimization</li><li>Efficient concurrent request servin</li></ul><p>without anyone manually implementing any of those optimizations.</p><p>Similarly, even LoRA fine-tuned models can be served through vLLM while still benefiting from all these inference optimizations.</p><p>That’s it….</p><h4>Thank You !! Happy Learning !!</h4><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=58cfafa30f3d" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Understanding KV Cache in LLM’s]]></title>
            <link>https://medium.com/@mailpraveenreddy.c/understanding-kv-cache-in-llms-bfc2656242df?source=rss-2d3bb2dcf96a------2</link>
            <guid isPermaLink="false">https://medium.com/p/bfc2656242df</guid>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[cache]]></category>
            <category><![CDATA[kv-cache]]></category>
            <dc:creator><![CDATA[praveenreddy_c]]></dc:creator>
            <pubDate>Sat, 23 May 2026 11:38:51 GMT</pubDate>
            <atom:updated>2026-05-23T11:38:51.877Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kyslg4AyYiQTyCqgNle0Mg.png" /></figure><p>We all know that Large Language Models (LLMs) are autoregressive models.</p><blockquote>Autoregressive means the model generates one token at a time by looking at all the previous tokens.</blockquote><p>For example, if the input to the model is:<strong> “<em>I love learning about”. </em></strong>The model predicts the next token: “<strong><em>AI”. </em></strong>Now the sentence becomes: “<strong><em>I love learning about AI” </em></strong>and the input to the model to predict the next word will be the whole sentence: “<strong><em>I love learning about AI”</em></strong></p><p>Now, the model looks at the entire updated sentence and predicts the next token: “<strong><em>models”</em></strong></p><p>Now the input to the model becomes: “<strong><em>I love learning about AI models”.</em></strong></p><p>And this process keeps repeating token by token.</p><p>So,</p><blockquote>The model repeatedly processes the growing context &amp; e<em>very new token a model gonna generate depends on the input given to the LLM + previously generated tokens by the LLM</em>.</blockquote><p>This is called as autoregressive generation and this is what happens behind every LLM. As we can understand, This is computationally heavy to reprocess the same token every time.</p><p>Let’s say if the model already processed: “<strong><em>I love learning about AI”</em></strong></p><p>Now while generating the next token, it should again recompute the understanding of: “<strong><em>I love learning about AI”</em></strong></p><p>This becomes extremely costly when:</p><ul><li>Context length becomes large</li><li>Conversations become longer</li></ul><p>This repeated computation would waste:</p><ul><li>GPU compute</li><li>Memory bandwidth</li><li>Latency</li><li>Inference cost</li></ul><p>And this is exactly where KV Cache comes into the picture.</p><h4>What is KV Cache?</h4><p>KV in KV Cache means:</p><ul><li>K → Key</li><li>V → Value</li></ul><p>If you want to get a clear understanding of Q, K, V. You can check out my blog where I have clearly explained what Query (Q), Key (K), and Value (V) are in Multi-Head Attention.</p><p><a href="https://medium.com/@mailpraveenreddy.c/explanation-of-q-k-v-and-attention-in-transformers-without-complex-math-5583698e374e">Explanation of Q, K, V and Attention in Transformers Without Complex Math</a></p><p>Whenever a token passes through the transformer, the model creates:</p><ul><li>Query (Q)</li><li>Key (K)</li><li>Value (V)</li></ul><p>The attention mechanism works roughly like this:</p><ul><li>Current token’s Query checks all previous Keys</li><li>Based on similarity, it combines the corresponding Values</li><li>This helps the model understand which previous tokens are important</li></ul><p>Now here is the important observation is:</p><p>For previously generated tokens:</p><ul><li>Their Keys do not change</li><li>Their Values do not change</li></ul><p>So instead of recomputing them again and again, why not simply store them?</p><h4>That stored memory is called the <strong>KV Cache.</strong></h4><h4>Without KV Cache</h4><p>For every generated token:</p><ul><li>Recompute attention for all previous tokens</li><li>Recalculate Keys and Values repeatedly</li><li>Higher latency</li><li>Very expensive inference</li></ul><p><strong>Example:</strong></p><blockquote>Generating the 100th token means recomputing Q, K, V calculations for previous 99 tokens again.</blockquote><h4>With KV Cache</h4><p>For every generated token:</p><ul><li>Previous Keys and Values are already stored</li><li>Only compute K, V for the new token</li><li>Reuse old cached K, V tensors</li></ul><p>This dramatically improves generation speed.</p><blockquote>So while generating the 100th token, Tokens 1 → 99 already have cached K, V. Only token 100 calculations are newly added</blockquote><p>This is why LLM inference becomes much faster.</p><h4>One Important Thing to Understand is:</h4><p>Even though KV Cache reduces computation. It increases memory usage….</p><p>Across all transformer layers, the model now stores:</p><ul><li>Keys for all previous tokens</li><li>Values for all previous tokens</li></ul><p>And for long contexts, this cache becomes huge. We call this challenge as: <em>KV Cache memory bottleneck</em></p><p>For scenarios where a model supports:</p><ul><li>32K context</li><li>128K context</li></ul><p>KV cache must store tensors for all those tokens. So memory grows rapidly. This is why serving long-context LLMs becomes difficult.</p><h4>Important Concepts Related to KV Cache</h4><h4>1. Pre-fill Phase</h4><p>This is the initial stage where:</p><ul><li>Entire input prompt is processed</li><li>KV cache is built for entire input prompt tokens</li></ul><p><strong>Example: </strong>If the user input is:</p><blockquote>Explain transformers clearly</blockquote><p>The model first processes the whole prompt and creates initial KV cache.</p><p>This stage is usually compute-heavy.</p><h4><strong>2. Decode Phase</strong></h4><p>After pre-fill:</p><ul><li>Tokens are generated one by one</li><li>Cached KV tensors are reused</li></ul><p>This stage is where KV Cache gives massive speed improvements.</p><h4>Problems with Traditional KV Cache</h4><p>Even though KV Cache is powerful, it has limitations.</p><h4>1. Huge Memory Consumption</h4><ul><li>Longer context = larger KV cache.</li><li>Memory becomes the bottleneck quickly.</li></ul><h4>2. GPU Fragmentation</h4><ul><li>Dynamic allocation and deallocation of K, V tensors can fragment GPU memory.</li></ul><h4>3. Multi-user Scaling Challenges</h4><ul><li>Serving thousands of users simultaneously means storing thousands of KV caches.</li></ul><p>This becomes expensive.</p><h4>What New Techniques Are Replacing or Improving KV Cache?</h4><p>Traditional KV Cache is becoming expensive for extremely long contexts. So researchers and companies are building newer optimizations.</p><ol><li>PagedAttention</li><li>Sliding Window Attention</li><li>Grouped Query Attention (GQA)</li><li>Multi-Query Attention (MQA)</li><li>KV Cache Quantization</li><li>Retrieval-Based Memory Systems</li></ol><p>I will explain the few major important techniques mentioned above in separate blogs each.</p><p>Without KV Cache:</p><ul><li>ChatGPT-like systems would become much slower</li><li>Costs would increase heavily</li><li>Real-time generation would not feel smooth</li></ul><p>But now, as context windows grow larger and models serve millions of users, traditional KV Cache itself is becoming a bottleneck. To overcome those limitations industry has come up with above mentioned techniques.</p><p>That’s it all about KV Cache….</p><p>Thank You !! Happy Learning !!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=bfc2656242df" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Understanding Classification Metrics in Machine Learning — Accuracy, Precision, Recall, F1 Score…]]></title>
            <link>https://medium.com/@mailpraveenreddy.c/understanding-classification-metrics-in-machine-learning-accuracy-precision-recall-f1-score-a381cd41cc0b?source=rss-2d3bb2dcf96a------2</link>
            <guid isPermaLink="false">https://medium.com/p/a381cd41cc0b</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[classification]]></category>
            <category><![CDATA[accuracy]]></category>
            <category><![CDATA[precision]]></category>
            <dc:creator><![CDATA[praveenreddy_c]]></dc:creator>
            <pubDate>Sun, 17 May 2026 09:17:58 GMT</pubDate>
            <atom:updated>2026-05-17T09:17:58.254Z</atom:updated>
            <content:encoded><![CDATA[<h3>Understanding Classification Metrics in Machine Learning — Accuracy, Precision, Recall, F1 Score, Specificity, ROC-AUC.</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ZZzFvmqMPlhKX3-e2kX-vg.png" /></figure><p>Recently, I was working on a small fine tuning task using GCP Document AI. While evaluating the outputs, I suddenly realised I had forgotten some of the core ML evaluation metrics like Precision, Recall, F1 Score.</p><p>These are concepts most of us learn during ML basics. There are already many great blogs and explanations available on these topics. But I still wanted to write this blog mainly for myself — so that I can refer to my own stuff whenever I want to recall. So in this blog, I will try to explain these metrics in simple terms, using practical examples.</p><p>When we build a Machine Learning classification model, the first thing most people look at is:<strong><em> accuracy</em></strong></p><p>But in real-world AI systems, accuracy alone is often not enough. This is why Machine Learning uses several evaluation metrics below:</p><ul><li>Accuracy</li><li>Precision</li><li>Recall</li><li>F1 Score</li><li>Specificity</li><li>ROC-AUC</li></ul><blockquote>Each metric tells a <strong>different story</strong> about the model.</blockquote><h4>Before Understanding Metrics — We need to understand Positive and Negative Classes</h4><p>In ML classification problems, the <strong>positive class is usually the thing we are trying to detect</strong>. Everything else becomes the negative class. Below image explains the same on <em>what is +ve class &amp; -ve class mean</em>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*hm_rKx348UDtxjEKkFG64w.png" /></figure><p>Then we need to understand 4 terms:</p><ul><li><strong>True Positive (TP):</strong> If the model predicted an email as spam (positive) and it was actually spam, then it is called True Positive.</li><li><strong>True Negative (TN):</strong> If the model predicted an email as normal (negative) and it was actually normal, then it is called True Negative.</li><li><strong>False Positive (FP):</strong> If the model predicted an email as spam (positive) but it was actually a normal email, then it is called False Positive.</li><li><strong>False Negative (FN):</strong> If the model predicted an email as normal (negative) but it was actually spam, then it is called False Negative.</li></ul><p>Now, Let’s go ahead and understand all the terms we mentioned above:</p><h4>1. Accuracy</h4><p>Accuracy is nothing but:</p><blockquote>Out of all predictions, how many predictions were correct</blockquote><pre>Accuracy = (TP + TN) / (TP + TN + FP + FN)</pre><p>As you can see from the above formula, we are dividing the total number of correct predictions (<em>True Positives + True Negatives</em>) by the total number of model predictions. Since we only have 4 possible prediction outcomes — TP, TN, FP, and FN — accuracy basically tells us how many predictions the model got right out of all the predictions it made.</p><p>But, we need to know that, <em>Accuracy can sometimes be misleading, especially when the dataset is imbalanced.</em></p><p>Let’s say we have a dataset with:</p><ul><li>99,000 transactions are genuine</li><li>1,000 transactions are fraud</li></ul><p>Now imagine if the model predicts: <em>Everything is genuine(All 1 lakh rows). </em>Even though the model completely failed to detect fraud, the accuracy still becomes:</p><pre>Accuracy = (TP + TN) / (TP + TN + FP + FN)<br><br>Accuracy = (0 + 99,000) / (0 + 99,000 + 0 + 1,000)<br><br>Accuracy = 99,000 / 100,000<br><br>Accuracy = 99%</pre><p>Here the model looks good based on accuracy, but in reality, it is not solving the actual problem. Which is why we need to understand the next technique: <strong>Precision</strong>.</p><h4>2. Precision</h4><p>Here, we mainly concentrate on how correctly the model is predicting the positive class. As we saw above, accuracy may not properly reflect the positive cases in imbalanced datasets. In real-world scenarios, datasets are usually imbalanced(<em>only a few transactions are actually fraud</em>). But our main goal is to correctly identify these few important positive cases.</p><p>Precision is nothing but:</p><blockquote>Out of all the positive predictions made by the model, how many were actually positive (the class we are most interested in detecting).</blockquote><pre>Precision = TP / (TP + FP)</pre><p>High Precision Means</p><ul><li>Fewer false alarms</li><li>Positive predictions are reliable</li></ul><p>Let’s say:</p><ul><li>Total fraud transactions = 1,000</li><li>The model predicts only 10 transactions as fraud</li><li>Out of those 10 predictions, all 10 are actually fraud</li></ul><p>Then:</p><pre>Precision = TP / (TP + FP)<br><br>Precision = 10 / (10 + 0)<br>Precision = 10 / 10<br>Precision = 100%</pre><p>Now the model looks excellent based on Precision because whenever it predicted fraud, it was correct but there is a problem here.</p><p>Even though the model achieved 100% Precision, it detected only 10 fraud transactions out of the actual 1,000 fraud cases. The remaining 990 fraud transactions were completely missed by the model.</p><p>So Precision mainly tells us:</p><blockquote>How correct are the positive predictions made by the model?</blockquote><p>But it does not tell us:</p><blockquote>How many actual positive cases the model failed to detect.</blockquote><p>Which is why we need to understand the next technique: <strong>Recall</strong>.</p><h4>3. Recall</h4><p>Recall mainly focuses on how many actual positive cases the model was able to detect successfully.</p><blockquote>Out of all the actual positive cases, how many did the model successfully find?</blockquote><pre>Recall = TP / (TP + FN)</pre><p>High Recall Means:</p><ul><li>Very few missed positive cases</li><li>The model captures most important events</li></ul><p>Let’s say:</p><ul><li>Total fraud transactions = 1,000</li><li>The model detected 900 fraud transactions</li><li>Missed 100 fraud transactions</li></ul><p>So,</p><pre>Recall = TP / (TP + FN)<br><br>Recall = 900 / (900 + 100)<br>Recall = 900 / 1000<br>Recall = 90%</pre><p>Now the model looks good based on Recall because it successfully detected most of the actual fraud transactions.</p><p>This is extremely important in real-world systems like:</p><ul><li>Fraud detection</li><li>Disease detection</li><li>Cybersecurity attacks</li></ul><p>because missing actual positive cases can become very costly or dangerous.</p><h4>Precision vs Recall — The Real Tradeoff</h4><p>Most ML systems cannot maximise both perfectly.</p><p>Usually,</p><ul><li>Increasing recall may reduce precision</li><li>Increasing precision may reduce recall</li></ul><p>Let’s look at an example: <strong>Airport Security</strong></p><p>Suppose airport scanners are made extremely sensitive so that they do not miss any dangerous objects.</p><p>Because of this:</p><ul><li>Recall increases → <em>Because the scanner detects more actual dangerous objects and misses fewer positive cases (False Negatives decrease).</em></li><li>Almost every dangerous object gets detected</li></ul><p>But the scanner may also start flagging many harmless objects as dangerous items. This increases False Positives.</p><p>So even though the system is catching almost all dangerous objects (high Recall), many normal passengers are also getting stopped unnecessarily, which reduces Precision → <em>Because many harmless passengers are wrongly flagged as dangerous, increasing False Positives.</em></p><p>That is why there is usually a tradeoff:</p><ul><li>Increasing Recall may decrease Precision</li><li>Increasing Precision may decrease Recall — This happens because Precision only focuses on how correct the detected positive predictions are, and not on the actual positive cases that the model failed to detect..</li></ul><h4>4. F1 Score — Balancing Precision and Recall</h4><p>As we saw above:</p><ul><li>Precision focuses on how correct the positive predictions are</li><li>Recall focuses on how many actual positive cases were detected</li></ul><h4>But in many real-world problems, we need both. This is where F1 Score becomes useful.</h4><p>F1 Score combines both Precision &amp; Recall into a single metric.</p><pre>F1 Score = 2 * (Precision * Recall) / (Precision + Recall)</pre><blockquote>F1 Score mainly helps us understand: Is the model maintaining a good balance between Precision and Recall?</blockquote><p>Let’s say:</p><ul><li>Total fraud transactions = 1,000</li><li>The model detected only 10 fraud transactions</li><li>All 10 detected frauds were actually correct</li></ul><p>Then:</p><pre>Precision = 10 / (10 + 0)<br>Precision = 100%</pre><p>But,</p><pre>Recall = 10 / (10 + 990)<br>Recall = 1%</pre><p>Now let’s calculate the F1 Score using the above Precision and Recall values:</p><pre>F1 Score = 2 * (Precision * Recall) / (Precision + Recall)<br><br>F1 Score = 2 * (1.0 * 0.01) / (1.0 + 0.01)<br><br>F1 Score = 0.0198<br><br>F1 Score ≈ 1.98%</pre><p>So, even though precision was 100%. The F1 Score became very low because Recall was extremely poor. Which says that,</p><blockquote>The model is not actually performing well overall because it is missing most of the important positive cases.</blockquote><p>So F1 Score helps expose situations where:</p><ul><li>One metric looks excellent</li><li>But the overall model performance is still poor.</li></ul><blockquote>F1 Score exposes the imbalance between Precision and Recall and gives a better overall understanding of the model performance.</blockquote><h4>F1 score matters ‘cos:</h4><p>A model can sometimes:</p><ul><li>Achieve high Precision by predicting very few positive cases, <em>because the few positive predictions it makes are mostly correct.</em></li><li>Or achieve high Recall by predicting too many positive cases, because the model captures most actual positives but also increases false alarms.</li></ul><p>But F1 Score helps us understand whether the model is balancing both properly.</p><h4>Example: Job Recommendation System</h4><p>Suppose LinkedIn’s recommendation engine:</p><ul><li>Recommends only a few jobs</li><li>But those jobs are highly relevant</li></ul><p>Here, Precision becomes high but many relevant jobs are never shown to the user So, Recall becomes low.</p><p>F1 Score helps measure the balance between:</p><ul><li>Showing relevant recommendations</li><li>Not missing many good opportunities</li></ul><h4>5. Specificity</h4><blockquote>Out of all the actual negative cases, how many were correctly identified as negative?</blockquote><pre>Specificity = TN / (TN + FP)</pre><h4>Example — Loan Approval System</h4><p>(Here, the positive class is risky applicants and the negative class is safe applicants)</p><p>Suppose a bank uses AI to detect risky applicants. If the system wrongly flags too many safe customers as risky:</p><ul><li>Good customers get rejected</li><li>The bank may lose genuine customers and business</li></ul><p>Specificity becomes important because:</p><blockquote>We want to correctly identify safe customers as safe.</blockquote><pre>Let&#39;s say:<br><br>Total safe applicants = 10,000<br>The model wrongly flagged 500 safe applicants as risky<br><br>Then:<br><br>TN = 9,500<br>FP = 500<br><br>Specificity = TN / (TN + FP)<br><br>Specificity = 9,500 / (9,500 + 500)<br><br>Specificity = 9,500 / 10,000<br><br>Specificity = 95%</pre><p>This means the model correctly identified 95% of the safe applicants as safe. Only 5% of the safe customers were wrongly flagged as risky, which helps the bank avoid rejecting too many genuine customers.</p><h4>High Specificity Means</h4><ul><li>Fewer false positives</li><li>Better identification of negative cases</li></ul><h4>6. ROC-AUC</h4><p>We know that, most classification models do not directly output: <em>Positive OR Negative.</em> Instead, they usually output a probability score.</p><p>For example, in fraud detection system can output like below:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/924/1*5ybMBr3X79DUhAJtl65e6Q.png" /></figure><p>We decide something like(<em>threshold = 0.5)</em>, anything more than 0.5 is called <strong>Fraud</strong> and anything less than 0.5 is called <strong>Not a Fraud</strong>.</p><h4>What ROC-AUC Measures is:</h4><blockquote><em>How well can the model distinguish between positive and negative classes across different thresholds?</em></blockquote><p><strong>Instead of evaluating the model at only one threshold, ROC checks model performance across multiple thresholds.</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ssQvdeajxBOBK7hFZ6N2QA.png" /></figure><p>Do not worry on how the TPR, FPR is calculated, the explanation is below.</p><h4>Understanding ROC Curve</h4><p>ROC stands for: <strong><em>Receiver Operating Characteristic</em></strong></p><pre><strong>ROC Curve plots:</strong><br><br>True Positive Rate (TPR)  vs  False Positive Rate (FPR)</pre><h4>1. True Positive Rate (TPR)</h4><p>TPR is also called Recall.</p><blockquote>Out of all actual positive cases, how many did the model correctly detect?</blockquote><pre>TPR = TP / (TP + FN)</pre><p>Higher TPR means:</p><ul><li>More actual positives are detected</li><li>Fewer important cases are missed</li></ul><pre>Let&#39;s say:<br><br>Total fraud transactions = 1,000<br>The model correctly detected 900 fraud transactions<br>The model missed 100 fraud transactions<br><br>TPR = TP / (TP + FN)<br><br>TPR = 900 / (900 + 100)<br><br>TPR = 900 / 1000<br><br>TPR = 90%</pre><h4>2. False Positive Rate (FPR)</h4><blockquote>“Out of all actual negative cases, how many were wrongly predicted as positive?”</blockquote><pre>FPR = FP / (FP + TN)</pre><p>Higher FPR means:</p><ul><li>More false alarms</li><li>More negative cases wrongly flagged as positive</li></ul><pre>Let&#39;s say:<br><br>Total genuine transactions = 10,000<br>The model wrongly flagged 2,000 genuine transactions as fraud<br><br>FPR = FP / (FP + TN)<br><br>FPR = 2,000 / (2,000 + 8,000)<br><br>FPR = 2,000 / 10,000<br><br>FPR = 20%</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/684/1*WDbw4LxSouxWINVpWRgtyw.png" /></figure><p>Which means:</p><ul><li>The model is correctly detecting 90% of fraud transactions</li><li>But it is also wrongly flagging 20% of genuine transactions as fraud</li></ul><p>ROC Curve helps visualise this tradeoff between:</p><ul><li>Detecting more actual positive cases (higher TPR)</li><li>Avoiding unnecessary false alarms (lower FPR)</li></ul><blockquote>A better model usually tries to: Increase TPR &amp; keep FPR as low as possible</blockquote><p>That’s it — these are some of the most important metrics we commonly use in Machine Learning classification problems.</p><p>The main thing to remember is:</p><blockquote>A model is not considered good just because it has high accuracy. We also need to understand how well it detects important positive cases, how many cases it misses, and how reliable its predictions are.</blockquote><h3>Thank You !! Happy Learning !!</h3><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a381cd41cc0b" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Vision Transformer (ViT) — The Architecture Behind Modern Multimodal LLM’s]]></title>
            <link>https://medium.com/@mailpraveenreddy.c/vision-transformer-vit-the-architecture-behind-modern-multimodal-llms-2032969d8dc9?source=rss-2d3bb2dcf96a------2</link>
            <guid isPermaLink="false">https://medium.com/p/2032969d8dc9</guid>
            <category><![CDATA[vit]]></category>
            <category><![CDATA[computer-vision]]></category>
            <category><![CDATA[vision-transformer]]></category>
            <dc:creator><![CDATA[praveenreddy_c]]></dc:creator>
            <pubDate>Sun, 10 May 2026 14:43:27 GMT</pubDate>
            <atom:updated>2026-05-10T14:43:27.922Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/987/1*coGJJlG-KIkM0eBgwDnc8A.png" /></figure><p>We all know how powerful the Transformer architecture has become in the NLP world. Models like GPT, BERT, and many other LLMs are all built on top of Transformers. But have you ever wondered how modern multimodal systems are able to understand images so accurately and generate meaningful responses from them? <strong>That is where the Vision Transformer (ViT) comes into the picture.</strong></p><blockquote><strong>Vision Transformer(ViT) is the architecture that brought the Transformer idea from NLP into Computer Vision. </strong>ViT processes image patches as tokens and learns relationships between them using self-attention.</blockquote><p>In this blog, we are not going to focus on the basic Transformer concepts like <em>Query, Key, Value, self-attention, encoders, decoders,</em> etc. Instead, we will mainly focus on the Vision Transformer (ViT) architecture, the additional components introduced specifically for handling images, and the non-Transformer concepts that make ViT capable of processing images.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qDaFwCZkSCnZA38owiAGgw.png" /></figure><h4>The Biggest Challenge in Using Transformers for Image Understanding → Images Are Not Sentences</h4><p>A Transformer expects input in this format:</p><pre>[token1, token2, token3, ...]<br><br>In NLP:<br>- we can assume, each token = a word</pre><p>But images are not sequences(<em>like a sentence of words</em>). An image can be represented as:</p><pre>Height × Width × Channels<br><br>Where <br> - Channels = R G B<br><br>if we take an Example image of size 224 x 224 resolution then, the image size can be represented as:<br><br>--&gt; 224 × 224 × 3</pre><p>Now the next question is: <em>How can we convert this 2D image(of size/resolution = </em>224 × 224 × 3<em>) into a sequence of tokens?</em></p><p>This is the core idea of <strong>ViT</strong>. We can achieve that using the following steps.</p><h4>1. Splitting the Image into Patches</h4><p>Instead of processing the full image directly, ViT splits the image into small fixed-size <strong>Patches</strong>(<em>aka parts or splits</em>). The <strong>patch size</strong> can be 16 × 16, 32 × 32, 14 × 14 etc., and it is a hyper parameter that is decided before training the model.</p><pre>Suppose:<br>image size = H × W<br>patch size = P × P<br><br>Then the number of patches becomes:<br><br>N = (H × W) / P²<br><br>Where:<br>- H = image height<br>- W = image width<br>- P = patch size<br><br>N = total number of patches or splits the image gonna be broken into<br><br>Example:<br>Image Size = 224 × 224<br>Patch Size = 16 × 16<br><br>N = (224 × 224) / (16 × 16)<br>N = 196 patches<br></pre><p><strong>So the image is now converted into: 196 patches. </strong>Each patch behaves like a word token in NLP.</p><p>Now, even though the image is split into patches, these patches still only contain raw pixel values. Transformers cannot directly understand meaningful information from raw pixels, so the next step is to convert these patches into a format that the Transformer can understand.</p><h4>2. Flattening the Patches</h4><p>Now, each patch has dimensions:</p><pre>P × P × C<br><br>Where:<br>- C = number of image channels. For R G B images → C = 3<br><br>Let&#39;s say the patch size is 16 x 16 <br><br>then each patch will be flattened to: 16 × 16 × 3 = 768</pre><p><strong>So every patch becomes a 1D vector of size 768. </strong>After flattening, each patch becomes a vector containing 768 numbers, where these numbers represent the pixel values of that small region/part/patch of the image.</p><p>So after flattening, every image patch is now converted into a fixed-size vector. But these vectors still only represent raw pixel intensities and do not yet contain semantic meaning.</p><h4>3. Patch Embedding</h4><p>Now the flattened patch contains a 768 pixel values is still just raw pixel information, and the model cannot directly understand meaningful patterns from those raw numbers. Similar to how Transformers in NLP convert words into embeddings, <strong>Vision Transformers(ViT) also convert these flattened pixel dimensions into embeddings that capture meaningful information about that specific image patch.</strong></p><blockquote>For example, in NLP, the word “bank” beside the word “river” will have a different embedding compared to the word “bank” beside the word “financial” because embeddings capture contextual meaning.</blockquote><p>In the same way, these 768 pixel values are passed through<strong> a learnable projection layer</strong> so that the model can generate embeddings representing the semantic meaning of that image patch, such as edges, textures, shapes, or object-related information that the Transformer can understand and process effectively.</p><pre>So each flattened patch is projected into a trainable embedding space and that is represented using the below formula in the paper.<br><br>E ∈ R(P²·C) × D<br><br>Where:<br> - P²·C = flattened patch dimension --&gt; see section 2 above, we have the same formula<br> - D = Transformer hidden dimension<br><br><strong>E = trainable projection matrix</strong></pre><p>So this is how the image patches are finally converted into trainable embedding vectors that the Transformer can understand. Once these embeddings are generated, <strong>the remaining process is very similar to the standard Transformer architecture</strong>, <em>where these patch embeddings are passed as a sequence of tokens into the Transformer encoder.</em></p><p>Along with the image patch embeddings, ViT also adds some additional information(xclass) before passing everything into the Transformer encoder. This helps the model understand the overall image better and also preserves positional information of the patches.</p><p>The paper mathematically represents this complete input sequence formation in ViT using the below formula:</p><pre>z₀ = [xclass; xₚ¹E; xₚ²E; ... ; xₚᴺE] + Epos<br><br>Now let us understand every single parameter carefully.<br><br>xₚ¹ = First image patch.<br>xₚ² = Second image patch.<br>xₚᴺ = Nth image patch.<br>E = Trainable embedding projection matrix. Used to convert flattened patches into Transformer embeddings.<br><br>So, <br><br>xₚ¹E = Embedded representation of patch 1. Equivalent to word embeddings in NLP.<br>Now the image patches are finally converted into meaningful embedding vectors that can be processed by the Transformer architecture, very similar to how word embeddings are processed in NLP Transformers.<br><br>Epos = Positional embeddings.<br> - Transformers do not understand order naturally.<br> - Without positional embeddings: patch positions are lost<br><br>z₀ = Final input sequence given to Transformer encoder.</pre><p>One important component in the above formula that needs special attention is the class token (xclass).</p><p>We did not discuss about the class token (xclass) above.</p><blockquote>In Vision Transformer, It is a special learnable embedding added along with all the image patch embeddings before giving them to the Transformer.</blockquote><p><strong>Do not assume it like the original full(unpatched) image embeddings, but instead acts like a summary vector for the complete original image.</strong></p><h4>Transformer Encoder in ViT</h4><p>After all the above steps, the data is passed into the standard Transformer Encoder.</p><p>The encoder contains repeated layers of:</p><ul><li>Multi-Head Self Attention (MSA)</li><li>MLP Block (Feed Forward Network / FFN)</li><li>Layer Normalisation</li><li>Residual Connections</li></ul><p>The architecture is almost similar to the original Transformer paper.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*E80aLHu95OagjLn8XXiXjg.png" /></figure><h4>Attention Block Formula</h4><p>The paper defines the attention block using the below formula:</p><pre>z&#39;ₗ = MSA(LN(zₗ₋₁)) + zₗ₋₁<br><br>Where <br> - zₗ₋₁ = Input from previous layer.<br> - LN = Layer Normalization.<br> - MSA = Multi-Head Self Attention.<br> - + zₗ₋₁ = Residual connection.<br></pre><h4>As the image patch embeddings pass through multiple Transformer layers, The class token gradually gathers information from all the image patches through the self-attention mechanism.</h4><h4>Final Output Representation</h4><p>The paper defines final output as:</p><pre>y = LN(zᴸ₀)<br><br>Where:<br> - zᴸ₀ = represents the final output embedding after the last Transformer layer<br> - LN  = represents Layer Normalization</pre><p>So the workflow as per the paper looks like below:</p><pre>Input Image<br>   ↓<br>Split into patches<br>   ↓<br>Convert patches into embeddings<br>   ↓<br>Pass through Transformer Encoder<br>   ↓<br><strong>Final class token </strong>becomes summarized image representation<br>   ↓<br><strong>Final class token</strong> representation passes through the Classification Head (Final Layer)<br>   ↓<br>Predict image class</pre><blockquote>One important point to remember is that: Although all patch embeddings are processed inside the Transformer, only the final class token embedding (xclass) is used for the final image classification because it acts as the summarized representation of the complete image. The other patch embeddings are not directly used in the final classification stage of the Vision Transformer because the class token gradually gathers information from all the image patches as it passes through multiple Transformer layers using the self-attention mechanism.</blockquote><p>That is all about the architecture of ViT…..</p><p>That is the core architecture behind Vision Transformers (ViT), which later became one of the foundational building blocks for modern multimodal AI systems.</p><p>Let’s now look at some other useful concepts mentioned in the paper:</p><h4>Inductive Bias</h4><pre>Inductive bias means:<br> - assumptions built into the architecture</pre><p>They mentioned in the paper that, CNNs contain strong image-specific <strong><em>inductive biases</em></strong>. But Vision Transformers do not.</p><h4>Biases Present in CNNs are:</h4><h4>1. Locality</h4><p>CNNs assume nearby pixels are related like edges, textures &amp; patterns. This assumption is naturally useful for images.</p><h4>2. Translation Equivariance</h4><p>If an object shifts slightly:</p><ul><li>CNN can still recognize it</li></ul><p>Because the same convolution filter slides everywhere.</p><p>For example:</p><ul><li>cat on left side</li><li>cat on right side</li></ul><p>CNN still recognizes it as a cat.</p><h4>3. 2D Spatial Structure</h4><p>CNNs naturally understand:</p><ul><li>neighboring pixels</li><li>spatial geometry</li><li>local patterns</li></ul><p>Because convolution operates spatially.</p><h4>So, Let’s understand why ViT Does NOT Have These Biases</h4><p>Vision Transformers process patches globally using self-attention, which means: Every patch can attend to every other patch</p><p>There is no built-in assumption that:</p><ul><li>nearby patches matter more</li></ul><p>Which means:</p><blockquote>ViT has much weaker image inductive bias</blockquote><p>The paper specifically says:</p><blockquote>spatial relationships in ViT must be learned from scratch</blockquote><p>This is extremely important because CNNs already contain many built-in assumptions about images, such as locality, neighbouring pixel relationships, and translation equivariance. These inductive biases help CNNs generalise well even when trained on smaller datasets.</p><p>On the other hand, Vision Transformers(ViT) do not naturally contain these strong image-specific biases. Instead, ViT learns all these spatial relationships directly from data through self-attention. Because of this, ViT initially could not outperform CNNs when trained on smaller datasets.</p><p>However, when trained on extremely large datasets, ViT was able to learn these relationships effectively from the data itself and eventually matched or even outperformed CNN-based architectures. This led to one of the biggest conclusions of the paper:</p><blockquote>Large-scale training can compensate for the lack of strong inductive biases in Vision Transformers.</blockquote><h4>The paper also does Comparison to State-of-the-Art CNN Models</h4><p>The paper compares ViT against powerful CNN architectures like ResNet, BiT &amp; Hybrid CNN + ViT</p><p>They mentioned that results showed:</p><ul><li>with small datasets → CNNs still strong</li><li>with huge datasets → ViT becomes superior</li></ul><p>ViT achieved state-of-the-art performance on:</p><ul><li>ImageNet</li><li>CIFAR-100</li><li>VTAB benchmarks</li></ul><p>This also proved that Transformers can dominate Computer Vision too.</p><h4>Pre-Training Data Requirements</h4><blockquote>ViT is extremely data hungry.</blockquote><p>CNNs work well even with smaller datasets because:</p><ul><li>Inductive bias already exists</li></ul><p>But ViT must learn:</p><ul><li>Edges</li><li>Textures</li><li>Spatial structure</li><li>Object relationships</li></ul><p>from scratch.</p><p>So ViT requires:</p><ul><li><strong>Massive pre-training datasets</strong></li></ul><blockquote>This is one of the biggest practical limitations of ViT.</blockquote><h4>That’s it….</h4><p>Vision Transformer completely changed the direction of Computer Vision. Instead of handcrafted image-specific architectures like CNNs, ViT showed that:</p><blockquote>A simple Transformer with enough data and scale can learn vision tasks extremely well.</blockquote><p>The biggest breakthroughs were:</p><ul><li><em>Treating image patches as tokens</em></li><li><em>Using self-attention for global understanding</em></li><li><em>Removing heavy image-specific assumptions</em></li><li><em>Scaling training massively</em></li></ul><p>CNNs relied heavily on inductive biases but, ViT relied more on:</p><ul><li>data</li><li>scale</li><li>learning capability</li></ul><p>And that has eventually became the foundation for modern multimodal AI systems. That’s it….</p><p>Hope, I gave the clear understanding on ViT</p><h3>Thank You !! Happy Learning !!</h3><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2032969d8dc9" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Explanation of Q, K, V and Attention in Transformers Without Complex Math]]></title>
            <link>https://medium.com/@mailpraveenreddy.c/explanation-of-q-k-v-and-attention-in-transformers-without-complex-math-5583698e374e?source=rss-2d3bb2dcf96a------2</link>
            <guid isPermaLink="false">https://medium.com/p/5583698e374e</guid>
            <category><![CDATA[transformers]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[attention]]></category>
            <category><![CDATA[ai]]></category>
            <dc:creator><![CDATA[praveenreddy_c]]></dc:creator>
            <pubDate>Sat, 09 May 2026 15:10:42 GMT</pubDate>
            <atom:updated>2026-05-09T15:10:42.668Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Wb609ChPJkHbvm1n9dR6tQ.png" /></figure><p>We all know that Transformers are the reasons behind today’s powerful LLM’s. I had a little confusion around the Q, K, V concept in attention. Before understanding the Vision Transformer (ViT), I have decided to understand the Q, K, V first so, I sat down and understood it properly.</p><p>We know that a Transformer model contains multiple layers. Inside every layer, we mainly have two important blocks:</p><ul><li>Multi-Head Attention</li><li>Feed Forward Neural Network (FFN)</li></ul><p>And around these blocks, we also have:</p><ul><li>Residual connections</li><li>Layer normalisation</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/786/1*eaMz1Jd_uEVUmLX4vDhYDQ.png" /></figure><p>This entire structure gets repeated multiple times across layers(we can see in the above image from the ‘<em>Attention is all you need</em>’ paper.).</p><p>But in this blog, let us focus only on the Multi-Head Attention block because this is where the actual magic happens. Before understanding Multi-Head Attention, let us first understand a single attention head.</p><p>Inside every layer, during training, the model learns 3 very important matrices. These are learnable parameters of the model. During training, they continuously get updated using back-propagation. Once training is completed, these matrices become fixed for that trained model during inference (unless we fine-tune the model again).</p><ul><li>WQ → Weight matrix for Query</li><li>WK → Weight matrix for Key</li><li>WV → Weight matrix for Value</li></ul><p>We all know when we give an input sentence to the model, the words are first converted into embeddings along with positional encodings.</p><p>For example, Let’s assume the input is: “I love AI”</p><p>This input gets converted into vectors like:</p><ul><li>“I” → into one embedding vector</li><li>“love” → into one embedding vector</li><li>“AI” → into one embedding vector</li></ul><p>Now each embedding vector gets multiplied with the learned matrices the model already has(WQ, WK, WV).</p><p>So for every word:</p><ul><li>Embedding × WQ → Query vector (Q)</li><li>Embedding × WK → Key vector (K)</li><li>Embedding × WV → Value vector (V)</li></ul><h3>This is how Q, K and V are generated for every single word.</h3><p>Now comes the real attention mechanism. Let us assume the model is currently processing the word:</p><p>“<em>love</em>” in the sentence “<em>I love AI</em>”</p><p>The Query vector of “love” basically asks:</p><ul><li>What other words are important for me?</li></ul><p>Now this Query vector is compared with the Key vectors of all other words(I, love, AI) in the sentence.</p><p>So internally:</p><ul><li>Q(love) gets compared with K(I)</li><li>Q(love) gets compared with K(love)</li><li>Q(love) gets compared with K(AI)</li></ul><p>This comparison is done using dot product.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/154/1*6isPb9yuKREnhUtqMjsBxg.png" /></figure><p>If the score is high, it means the words are highly related. If the score is low, it means the relation is weak.</p><p>Let’s say we get scores like this:</p><ul><li>I → 1.2</li><li>love → 3.5</li><li>AI → 2.8</li></ul><p>Now in real Transformer models, embeddings usually have very high dimensions like 512, 768, 1024, and even larger. Because of these high dimensions, the dot product values can grow very large. Large values can make the softmax function extremely sharp, which can lead to unstable training and very small gradients.</p><p>To stabilize this, the Transformer scales the scores by dividing them with:</p><p>√dk</p><pre>where dk is the dimension size of the Key vector for a single attention head.<br>For example, if the model embedding dimension is 512 and there are 8 attention heads, then each head usually works with vectors of size:<br>512 / 8 = 64<br>So in this case:<br>dk = 64</pre><p>This is why the mechanism is called Scaled Dot-Product Attention.</p><p>After scaling, softmax is applied.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/814/1*kIKjM_DZ-AYqJLxXpgvddg.png" /></figure><p>Softmax converts these scores into probabilities.</p><p>For example:</p><ul><li>I → 10%</li><li>love → 50%</li><li>AI → 40%</li></ul><p>These percentages are called attention weights.</p><p>This basically means:</p><p>While understanding the word “love”, the model should focus:</p><ul><li>50% on itself</li><li>40% on AI</li><li>10% on I</li></ul><p>Now finally, these attention weights are multiplied with the Value vectors. This allows the model to collect and combine information from the most relevant words in the sentence.</p><p>Now an important question here is:</p><blockquote>Why are these attention weights multiplied with the Value (V) vector?</blockquote><p>The reason is:</p><ul><li>Query vectors are mainly used for asking: “<em>What information am I looking for?</em>”</li><li>Key vectors are mainly used for matching: “<em>Am I relevant to this query?</em>”</li></ul><h3>But the actual information carried by a word is stored inside the Value vector.</h3><p>So once the model figures out which words are important using Q and K, it finally collects the actual information from the corresponding Value vectors.</p><p>For example, while processing the word “love”:</p><ul><li>The model first identifies that “AI” is important using Q and K.</li><li>Then it takes the actual contextual information from V(AI).</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/814/1*kIKjM_DZ-AYqJLxXpgvddg.png" /></figure><p>So attention weights basically decide:</p><p><em>How much information should I take from each word?</em></p><p>And the Value vectors are the actual information being transferred and combined. This weighted combination of Value vectors becomes the new context-aware representation of the word. This is how the model builds contextual understanding.</p><h3>That is the real power of attention.</h3><p>And this entire process happens in parallel for all words in the sentence.</p><p>Now imagine not just one attention head, but multiple attention heads running in parallel. Different heads learn different relationships like grammar, semantics, long-term dependencies, subject-object relationships, etc.</p><p>That is why it is called Multi-Head Attention. That’s all about Q, K, V in multiple head attention…..</p><h3>Thank You !! Happy Learning !!</h3><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5583698e374e" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Direct Preference Optimization (DPO): A Simpler Alternative to RLHF]]></title>
            <link>https://medium.com/@mailpraveenreddy.c/direct-preference-optimization-dpo-a-simpler-alternative-to-rlhf-b59cb60e593e?source=rss-2d3bb2dcf96a------2</link>
            <guid isPermaLink="false">https://medium.com/p/b59cb60e593e</guid>
            <category><![CDATA[dpos]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[fine-tuning]]></category>
            <dc:creator><![CDATA[praveenreddy_c]]></dc:creator>
            <pubDate>Sun, 03 May 2026 04:45:07 GMT</pubDate>
            <atom:updated>2026-05-03T04:45:07.150Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Tj8QWcX5LbT-Sz-NIHJNpw.png" /></figure><p>I have written a blog on RLHF, which explains how we can fine-tune a pre-trained model. If you are not familiar with it, you can check it out at the link below.</p><p><a href="https://medium.com/@mailpraveenreddy.c/rlhf-explained-the-reason-chatgpt-responds-like-a-human-b83be3c98239">RLHF Explained: The Reason ChatGPT Responds Like a Human</a></p><p>There is another fine-tuning technique called DPO <strong>(Direct Preference Optimization).</strong></p><blockquote>The DPO paper says that RLHF introduces a lot of complexity, since we need to train a separate reward model and then use reinforcement learning (like PPO) to update the language model so that it maximises the reward. While this process does help fine-tune the final LLM, it involves multiple steps and added overhead.</blockquote><p>DPO avoids all this complexity. Instead of training a reward model and applying reinforcement learning, it directly uses human feedback (or human preferences) to train the language model. This makes the overall process much simpler and more efficient.</p><h4>Idea Behind DPO</h4><p>The key idea in DPO is that we don’t actually need a separate reward model. <strong>During training the model is given a prompt along with two responses: one chosen (preferred) and one rejected.</strong> If a human prefers response A over response B, then the model should assign a higher probability to A than B. That’s it.</p><p>So instead of:</p><ul><li>Training a separate reward model that takes a prompt and response, and gives a score based on human preferences (preferred responses get higher scores).</li><li>Using a reinforcement learning (like PPO) to update the language model so that it generates responses that get higher scores from the reward model.</li></ul><p>DPO directly does:</p><ul><li>Increase probability of preferred response</li><li>Decrease probability of rejected response</li></ul><p>This is super easy to train.</p><h4>How DPO Defines the Loss</h4><blockquote>DPO works with preference data in the form of: prompt, chosen response, rejected response.</blockquote><p>The loss function is designed to:</p><ul><li>Increase the log probability of the chosen response</li><li>Decrease the log probability of the rejected response</li></ul><p>But this is not done as two independent steps. Instead of treating them separately, DPO directly compares the two responses and learns to make the chosen response more likely than the rejected one.</p><blockquote>DPO is not just saying make good answers more likely, but rather: <strong>make good answers more likely than bad ones, while staying close to a stable baseline model.</strong></blockquote><h4>Another important part of DPO is the use of a reference model.</h4><p>(<em>The reference model ensures that, while updating the parameters, the main model stays close to its original (base) behaviour and does not deviate too much</em>).</p><p>In the DPO process, we keep two models: <strong><em>a main model and a reference model</em>.</strong></p><ul><li>The main model is the one that gets updated based on human preferences.</li><li>The reference model, however, remains fixed and is only used for comparison.</li></ul><p>The reason we have two models is to ensure that, as we fine-tune the main model, it doesn’t stray too far from the original behaviour when the updates are happened while updating the parameters as per the preference. This way, the main model learns to follow human preferences while avoiding extreme or unstable parameter updates, maintaining overall quality and fluency.</p><p>This makes training efficient because the model focuses more on fixing its mistakes.</p><p>Now, let’s move to the formula part. I won’t go into a detailed explanation here, as I came across a video that explains it very clearly and in depth. I highly recommend going through that, it should be more than enough to understand the concept.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2Fk2pD3k1485A%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3Dk2pD3k1485A&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2Fk2pD3k1485A%2Fhqdefault.jpg&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/f4cc8f35d060ed9e5d378c64df7f7e64/href">https://medium.com/media/f4cc8f35d060ed9e5d378c64df7f7e64/href</a></iframe><p>That’s all about DPO — Direct Preference Optimization.</p><h3>Thank You, Happy Learning</h3><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b59cb60e593e" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How LLMs Learn to Think: Inside DeepSeek’s GRPO Technique]]></title>
            <link>https://medium.com/@mailpraveenreddy.c/how-llms-learn-to-think-inside-deepseeks-grpo-technique-c2acf34aa6e1?source=rss-2d3bb2dcf96a------2</link>
            <guid isPermaLink="false">https://medium.com/p/c2acf34aa6e1</guid>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[grpo]]></category>
            <category><![CDATA[reasoning]]></category>
            <category><![CDATA[fine-tuning]]></category>
            <dc:creator><![CDATA[praveenreddy_c]]></dc:creator>
            <pubDate>Sat, 02 May 2026 07:03:42 GMT</pubDate>
            <atom:updated>2026-05-02T07:03:42.931Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*yWcluVScWAJmCDx7LhvoGQ.png" /></figure><p>I first came across <strong>GRPO</strong> on LinkedIn and got a high-level understanding through ChatGPT. But later, I felt like diving deeper into the actual paper as I got curious to understand how this technique helped build a<strong> <em>reasoning model</em></strong>. Instead of simply generating responses, the model learns to think through problems, which makes a huge difference when handling complex tasks.</p><p>We all know the techniques like <em>chain-of-thought (CoT)</em> prompting, where models are guided to generate intermediate reasoning steps. However, these methods depend heavily on human-designed examples, which can limit scalability and reduce performance if the CoT prompt is not well-designed. <em>Additionally, they constrain models to mimic human thought processes, preventing them from discovering potentially better, non-human reasoning strategies.</em></p><p>In this blog, we explore how reinforcement learning (RL), specifically <em>Group Relative Policy Optimization (GRPO)</em>, enables models to develop reasoning capabilities through self-evolution.</p><p>This technique was developed by DeepSeek, and their models leverage it to handle complex reasoning tasks effectively.</p><p>Everything discussed below is applied on top of a pre-trained model. In the paper, they use the DeepSeek base model as the starting point and then apply these techniques to enable more advanced reasoning and thinking capabilities in the model and named the final model as <strong>DeepSeek-R1</strong>.</p><h4>So, Let’s go ahead and understand GRPO….</h4><blockquote>GRPO on high level is nothing but a reinforcement learning method where the model generates multiple answers, compares them within the group and learns to prefer the better ones</blockquote><p>Now, Let’s understand it step by step:</p><p>It starts with a <strong>question (input prompt)</strong>. For each question, instead of generating just one answer, the model generates <strong>N different outputs </strong>(<em>In the paper, they mentioned they have asked DeepSeek pre-trained model to generate 16 outputs for each question</em>).</p><blockquote>The outputs are generated by the model itself and the dataset we use while training only contains: Question, Correct final answer (ground truth)</blockquote><p>For each generated output, we compute a <strong>reward</strong>:</p><ul><li>If the answer is correct → <strong>reward = 1</strong></li><li>If the answer is wrong → <strong>reward = 0</strong></li></ul><p>Now, for each output, we compute something called <strong>advantage (Ai)</strong>:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/193/1*Qbdoxw_CeBSFsQts5EFVVQ.png" /></figure><pre>Ai = (ri - mean(r1, r2, ..., rG)) / std(r1, r2, ..., rG)<br><br>ri = reward of the current output<br>mean(r1, r2, ..., rG) = average reward of all outputs in the group<br>std(r1, r2, ..., rG) = standard deviation of rewards<br>G = total number of outputs (e.g., 16)</pre><p>This <strong>Ai</strong> can also be called as <strong>relative advantage</strong>.</p><ul><li>If an output is <strong>better than average</strong> → Ai will be <strong>positive</strong></li><li>If an output is <strong>worse than average</strong> → Ai will be <strong>negative</strong></li></ul><blockquote>The important point is: Instead of judging answers independently, we compare them <strong>within a group</strong>.</blockquote><p>That’s why it’s called <strong>Group Relative Policy Optimization (GRPO)</strong>.</p><p>The <em>Policy Optimization</em> part refers to updating the model parameters using this advantage signal. Outputs with higher ( Ai ) are reinforced, while those with lower ( Ai ) are discouraged. During this update, <strong>KL Divergence(Dkl) </strong>is used as a regularisation term to ensure the model does not deviate too much from a reference model, helping maintain stable training.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/150/1*mjmAK4FOuRvyiuDV5iyFVQ.png" /></figure><pre>Dkl​(πθ​∣∣πref​)<br><br>Which Says,<br>- Don&#39;t let the model change too drastically<br>- Stay somewhat close to original behavior</pre><p>Also the paper has mentioned they have come up with 2 types of rewards and each one is used as per the task type:</p><ul><li>Accuracy Reward</li><li>Format Reward</li></ul><p><strong>Accuracy Reward </strong>checks if the final answer is correct or not:</p><ul><li>For Math tasks, It compares the final number</li><li>For Coding tasks, It run code against test cases</li></ul><p>So:</p><ul><li>Correct answer = 1</li><li>Wrong answer = 0</li></ul><p><strong>Format Reward</strong> Checks if the model has followed the required structure? In this reward type we force the model to write like this:</p><pre>&lt;think&gt; reasoning steps &lt;/think&gt;<br>&lt;answer&gt; final answer &lt;/answer&gt;</pre><p>So,</p><ul><li>we can clearly separate <strong>thinking vs final answer</strong></li><li>which makes outputs easier to analyse</li></ul><p>I hope I gave a clear understanding of GRPO and this all used to train <strong>DeepSeek-R1-Zero</strong> to develop reasoning capabilities on top of a pre-trained model.</p><p>Even though <strong>DeepSeek-R1-Zero</strong> was very strong in reasoning, it had below problems it seems:</p><ul><li>The output is often messy, difficult for humans to read</li><li>In the output it has mixed languages (English + Chinese in same answer)</li><li>Not good at general tasks (writing, normal Q&amp;A)</li></ul><p>Which is why they have made some tweaks in the process and trained a new reasoning model called <strong>DeepSeek-R1</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1019/1*m69gyi1Rg2odV5vdRlwcPw.png" /></figure><p>Instead of relying only on reinforcement learning with GRPO like in DeepSeek-R1-Zero, DeepSeek has tweaked the training process for <strong>DeepSeek-R1 </strong>and it has adopted a <strong>multi-stage training pipeline</strong> that combines different techniques to improve both reasoning and usability.</p><p>The first stage involves collecting <strong>cold-start data(</strong><em>data to kick start</em><strong>)</strong>, which consists of human like conversational reasoning examples. This data is more structured and readable.</p><blockquote>The goal of this step is simple: guide the model toward producing responses that feel natural and aligned with how humans explain their thinking.</blockquote><p>Next comes the <strong>first reinforcement learning (RL) stage</strong>, where methods like GRPO are applied. In this phase, the model improves its reasoning ability while also learning to maintain language consistency as we got language mixed up issue in DeepSeek-R1-Zero.</p><blockquote>To address issues like language mixing, an additional <strong>language reward (</strong>Hope you remember the 2 rewards used in the DeepSeek-R1-Zero → <em>Format Reward, Accuracy Reward</em><strong>)</strong> is introduced, which measures how much of the response is written in the target language. This encourages the model to produce cleaner and more consistent outputs.</blockquote><p>After this, the pipeline moves to <strong>Rejection sampling + Supervised fine-tuning (SFT)</strong>. Here, the model generates multiple responses same like DeepSeek-R1-Zero at GRPO step, for a a single input, only the high-quality ones are selected. These selected outputs are then used to fine-tune(Supervised Fine Tuning) the model.</p><blockquote>The Important thing at this stage is to include both <strong>reasoning data</strong> and <strong>non-reasoning data</strong> (such as writing tasks and general Q&amp;A). Because of which the model not only retain its reasoning strength but also perform well in everyday use cases.</blockquote><p>Finally, a <strong>second RL stage</strong> is applied, often referred to as the alignment stage. In this phase, the model is trained using a <strong>combination of reward signals.</strong></p><blockquote>The rewards here include <strong>reasoning rewards</strong> (<em>correctness on reasoning tasks</em>), <strong>general rewards</strong> (<em>overall helpfulness and quality on general tasks</em>), and <strong>language rewards</strong> (<em>for language consistency</em>). By combining these 3 signals, the model becomes more balanced — capable of strong reasoning while also being helpful, safe.</blockquote><p>This is how they have updated the DeepSeek-R1 training pipeline and avoided the issues they faced in DeepSeek-R1-Zero</p><p>That’s all on how they have achieved the reasoning capabilities in DeepSeek-R1.</p><h4>Thank You, Happy Learning !!</h4><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c2acf34aa6e1" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[RLHF Explained: The Reason ChatGPT Responds Like a Human]]></title>
            <link>https://medium.com/@mailpraveenreddy.c/rlhf-explained-the-reason-chatgpt-responds-like-a-human-b83be3c98239?source=rss-2d3bb2dcf96a------2</link>
            <guid isPermaLink="false">https://medium.com/p/b83be3c98239</guid>
            <category><![CDATA[rlhf]]></category>
            <category><![CDATA[fine-tuning]]></category>
            <category><![CDATA[llm]]></category>
            <dc:creator><![CDATA[praveenreddy_c]]></dc:creator>
            <pubDate>Fri, 01 May 2026 05:01:38 GMT</pubDate>
            <atom:updated>2026-05-01T05:01:38.675Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*khQl_3SeWntWHDNUwH6I-w.png" /></figure><p>I first came across RLHF(<strong>Reinforcement Learning with Human Feedback</strong>) on LinkedIn. When I went through the paper, one thing became very clear to me — this is actually the reason why responses from ChatGPT feel so human.</p><blockquote>LLMs are not just being trained on data anymore. We are now trying to make them behave in a way that humans actually like &amp; do.</blockquote><h4>RLHF is trying to solve one main problem:</h4><p>Because the problem with traditional fine-tuning is, It doesn’t really help the model respond in a way that feels like a human conversation. It learns what is <em>correct</em>, but not what feels <em>natural</em> or <em>preferred</em> by humans.</p><h4>Now, Let’s understand how RLHF is helping the LLM’s to respond like human.</h4><p>RLHF process mainly happens in three steps, but with a couple of important details in between.</p><h4>1. Start with a PreTrained Model</h4><p>First, we take a pre-trained LLM. Before applying RLHF, the model goes through a step called <strong>Supervised Fine-Tuning (SFT)</strong>. In this step, the model is trained on conversation data (like real conversational datasets → where we have input(query) &amp; the expected response)</p><ul><li>The model learns how to generate proper replies with this conversational dataset.</li></ul><p>This helps the model understand:</p><ul><li>How conversations flow</li><li>How to respond in a structured way</li></ul><p>So at this stage:</p><ul><li>The model can generate meaningful responses, but it is still not aligned with how real humans actually have conversations</li><li>Because SFT only teaches the model to <strong>imitate the training data</strong>.</li></ul><h4>2. Define a Reward Model (Using User Behavior)</h4><p>Next, we train a new model called <strong>reward model</strong>. This is where things become different from traditional training. Instead of giving direct labels while training this reward model, we use <strong>implicit user feedback</strong>, like:</p><ul><li>Engagement (did the user continue?)</li><li>Sentiment (did their tone improve?)</li><li>Relevance (was the response on-topic?)</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/882/1*JvcrBRwt1eL46zQlpGDK1g.png" /></figure><p>Since this feedback is not perfect and comes from real user behaviour, it is often <strong>noisy and indirect</strong>. But still, it gives a strong signal about user satisfaction.</p><p>Once this reward model is trained, it can take any new response and assign a <strong>reward score</strong>. So now, every response generated by the LLM is evaluated with this score by, which judges the LLM response as :</p><blockquote>How satisfied is the user likely to be for this LLM response?</blockquote><h4>3. Optimize using PPO</h4><p>Now we improve the model that was trained using SFT in the first step, using <strong>PPO (Proximal Policy Optimization)</strong>. At a high level, PPO helps the model learn this:</p><ul><li>If the model response gets a <strong>high reward </strong>from reward model → generate more responses like this</li><li>If the model response gets a <strong>low reward</strong> from reward model→ avoid these kinds of responses</li></ul><p>So instead of learning from labels, the model is now learning from <strong>rewards</strong>.</p><p>The important point here is, We don’t want the model to change too drastically in one step. If we update it too aggressively:</p><ul><li>It might forget what it already learned</li><li>Or start generating unstable responses</li></ul><p>So PPO makes <strong>small, controlled updates</strong>. That’s why it’s called <em>“Proximal”. </em>it keeps the new model close to the old one while improving it.</p><h4>4. Maintaining Conversation Context</h4><p>Since this is a conversational system, the model doesn’t just look at one message. It keeps track of <strong>recent conversation history</strong> using a sliding window(typical Chatbot).</p><p>So every response is generated based on:</p><ul><li>What the user said before</li><li>What the model replied earlier</li></ul><p>This helps maintain <strong>context and consistency</strong> in multi-turn conversations.</p><h4>This is what makes ChatGPT feel Human</h4><p>Now if we connect all these steps together, we can understand why ChatGPT feels more human compared to traditional models.</p><p>In traditional fine-tuning:</p><ul><li>The model learns from static data</li><li>It tries to predict the most likely correct answer</li></ul><p>But in RLHF:</p><ul><li>The model learns from how humans react</li><li>It adapts based on what humans actually prefer</li></ul><p>Instead of asking:</p><blockquote>Is this answer correct?</blockquote><p>The system is now asking:</p><blockquote>Did the user like this response?</blockquote><p>Because of this training:</p><ul><li>The model becomes better at continuing conversations naturally</li><li>It learns to avoid responses that feel robotic or irrelevant</li><li>It improves at tone, clarity, and helpfulness</li></ul><p>So over time, the model doesn’t just give answers.<strong> <em>It starts behaving like a conversation partner.</em></strong></p><p>So, RLHF in simple terms:</p><ul><li><strong>SFT teaches the model how to talk</strong></li><li><strong>Reward model tells what humans like</strong></li><li><strong>PPO makes the model adapt to that</strong></li></ul><p>And when you combine all three, we get a model like ChatGPT.</p><p>Hope I explained RLHF clearly…</p><h4>Thank You !! Happy Learning !!</h4><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b83be3c98239" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[IA³ Fine-Tuning Explained — Infused Adapter by Inhibiting and Amplifying Inner Activations]]></title>
            <link>https://medium.com/@mailpraveenreddy.c/ia%C2%B3-fine-tuning-explained-infused-adapter-by-inhibiting-and-amplifying-inner-activations-ab080daa2f56?source=rss-2d3bb2dcf96a------2</link>
            <guid isPermaLink="false">https://medium.com/p/ab080daa2f56</guid>
            <category><![CDATA[ia3]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[fine-tuning]]></category>
            <category><![CDATA[lora]]></category>
            <dc:creator><![CDATA[praveenreddy_c]]></dc:creator>
            <pubDate>Mon, 13 Apr 2026 01:14:16 GMT</pubDate>
            <atom:updated>2026-04-13T01:14:16.838Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*A5U1JZnisOvEzAIBk0NKSw.png" /></figure><p>Hey, In this blog, we will discuss <strong>IA³ fine-tuning</strong>, a parameter-efficient fine tuning method that the paper says <strong>IA³</strong> performance is better than <strong>in-context learning (ICL), prefix tuning, prompt tuning </strong>in many scenarios.(I have shared the prefix, prompt tuning blogs of mine below.)</p><p><strong>In-Context Learning (ICL)</strong> is a method where a model learns a task just by seeing examples in the prompt(<em>like a few-shot example prompting</em>), without updating its weights. As we provide a few examples in the prompt, the model follows the same pattern to answer new/fresh queries.</p><h4>The paper first says that instead of just relying on prompting (like ICL), slightly updating the model with below 2 techniques can give more stable and better results.</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/602/1*YH3gLSGKNYZwg2dwbDBolg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/616/1*57SVmHQbefJWH8ViW--1NQ.png" /></figure><p>Those 2 techniques are called as: <strong>Unlikelihood Training</strong> and <strong>Length Normalization</strong>.</p><p>We know LLM’s while training uses <strong>cross-entropy loss </strong>for calculating the loss, where the goal is simple: For each token, the model learns to assign a higher probability to the correct next word.</p><pre>LLM = - (1/T) * Σ log p(yt | x, y&lt;t)</pre><p>Here, the model is just learning to “<strong><em>make the correct answer more likely.</em></strong>”</p><p>But the problem is — this only focuses on the correct answer. It <em>doesn’t actively penalize wrong answers</em>. This becomes a big issue in <strong>tasks like rank classification</strong>. In transformers we do <strong>rank classification, </strong>which means — There are multiple possible next words when Transformer has to choose the next word/token, but only <strong>one token is finally chosen based on the highest probability</strong>. <em>So the model must not only score the correct answer high, but also ensure that incorrect options get lower scores.</em> If it doesn’t do that, a wrong answer might still get a high probability and be ranked incorrectly.</p><p>To fix this, the paper introduces <strong>Unlikelihood Loss (LUL)</strong>. The idea behind is nothing but: <em>explicitly push down the probability of incorrect answers</em>. Below is the formula mentioned in the paper to achieve that.</p><pre>L_UL = - ( Σ_{n=1}^{N} Σ_{t=1}^{T(n)} log(1 - p(ŷ_t^(n) | x, ŷ_&lt;t^(n))) ) <br>       / ( Σ_{n=1}^{N} T(n) )<br><br>Let&#39;s understand how this penalizes wrong predictions:<br><br>ŷ^(n) = wrong answer sequences --&gt; which means:<br><br>For the input: Capital of France?<br><br>Correct sequence → y = &quot;Paris&quot;<br><br>Wrong sequences →<br>ŷ^(1) = &quot;Berlin&quot;<br>ŷ^(2) = &quot;Rome&quot;<br><br>For each wrong token, we compute its probability:<br>p(ŷ_t^(n) | x, ŷ_&lt;t^(n))<br><br>Then we apply: log(1 - p)<br><br>Case 1: Model assigns HIGH probability to a wrong token<br>------------------------------------------------------<br>p ≈ 0.9<br>1 - p ≈ 0.1<br>log(0.1) → very negative  → strong penalty<br>After outer minus → large positive loss<br><br>=&gt; Big penalty (incorrect behavior - which we ca change using Unlikelihood Loss)<br><br><br>Case 2: Model assigns LOW probability to a wrong token<br>-----------------------------------------------------<br>p ≈ 0.1<br>1 - p ≈ 0.9<br>log(0.9) → slightly negative → good behavior<br>After outer minus → small loss<br><br>=&gt; Small penalty (desired behavior)</pre><p>So now the model learns two things:</p><ul><li>Increase probability of correct answers</li><li>Decrease probability of wrong answers</li></ul><p>This makes the model much better at selecting the next word/token, because it creates a clearer gap between correct and incorrect choices.</p><p>Next comes another very important issue: <strong>length bias.</strong> We know LLM computes the probability token by token and then multiplies them together to get the total probability of the full sequence(total sentence).</p><p>So for the answer to the question: “<strong>What is the capital city of France?</strong>”</p><ul><li><strong>Option 1: </strong><em>Paris</em> → maybe 1 token</li><li><strong>Option 2: </strong><em>The capital of France is Paris </em>→ many tokens</li></ul><p>When we calculate the total probability:</p><ul><li>For Option 1 → p₁</li><li>For Option 2 → p₁ × p₂ × p₃ × … × pₙ</li></ul><p>The issue here is: We know <strong>each token probability is ≤ 1. </strong>When we multiply many numbers less than 1, the result becomes smaller. Look at the example below:</p><pre>For Option 1:<br>p = 0.8<br><br>For Option 2:<br>p = 0.8 × 0.8 × 0.8 × 0.8 ≈ 0.41<br><br>Even if the model is equally confident about both, <br>the longer answer ends up with a lower total probability just because, it has more tokens.</pre><p>Because of this, the model may:</p><ul><li>Prefer <strong>short answers</strong></li><li>Even when a <strong>longer answer is actually correct &amp; better</strong></li></ul><p>So we call this as <strong>length bias</strong> — the model is biased toward shorter answers, not because they’re better, but because of how probabilities are computed.</p><p>To fix this, they use <strong>Length Normalization</strong>, where instead of total probability, better option is to <strong>average it over the sequence length(</strong><em>sequence length is nothing but the total no of tokens present in the answer/output</em>).</p><pre>β(x, y) = (1/T) * Σ log p(yt | x, y&lt;t)</pre><p>Here, <strong>T is the total number of tokens in the answer</strong>, by dividing the total log probability by T, we are essentially measuring the <strong>per-token confidence</strong> of the model, rather than the total accumulated probability.</p><p>This changes the comparison significantly and a longer answer will no longer be unfairly penalized just because it has more tokens. <em>If the model is confident about each token in a longer sequence, its average score can still be high.</em></p><p>So, length normalization ensures that the model prefers answers based on <strong>quality (confidence per token)</strong> rather than <strong>quantity (number of tokens)</strong>.</p><p>Finally in the paper they introduce another loss called <strong>Length-Normalized Loss (LLN)</strong>, which says “make the correct answer score higher than all incorrect ones, <em>after normalizing for length</em>.”</p><pre>LLN = -log ( exp(β(x,y)) / (exp(β(x,y)) + Σ exp(β(x, ŷ(n))) ) )</pre><p>So in the paper they have mentioned that, even before IA³ <strong>just fixing how we train and evaluate the model already gives a solid boost</strong>.</p><p>Now, Let’s move ahead to IA³ immediately!!</p><p>The paper focuses on designing a fine-tuning method that satisfies three key requirements. First, the method should avoid updating the entire model and instead modify only a small number of parameters. Second, it should achieve strong accuracy even when only a limited amount of training data is available. Third, the method should support mixed-task batches, where a single training batch can contain examples from different tasks.</p><blockquote>Instead of modifying the model weights, the paper proposes to <strong><em>modify the activations</em> </strong>inside the model.</blockquote><p><strong>Activations</strong> is nothing but the output produced inside a neural network at each layer. When an input passes through the model, every layer transforms it and passes the result forward — these results are called activations. <em>They represent what the model is “thinking” at each step before producing the final output.</em></p><pre>Example: <br>x = input_vector to the Model</pre><pre># First layer<br>h1 = W1 * x          # activation from hidden layer 1</pre><pre># Apply non-linearity<br>h1 = relu(h1)        # still activation</pre><pre># Second layer<br>h2 = W2 * h1         # activation from next hidden layer 2</pre><pre># Final output<br>y = softmax(h2)      # Final activations<br></pre><p>The paper explains that the key reason for modifying activations is that activations are computed separately for each input. This means every example in a batch can have its own modification, making the approach flexible and computationally efficient. Because of this, the model can naturally support mixed-task batches(<em>where different examples in the same batch belong to different tasks and can be processed together by the model at the same time</em>).</p><p>Earlier activation-based methods like prompt tuning and prefix tuning also follow a similar idea. They work by adding learned vectors to the inputs or intermediate activations. The paper says they do not achieve strong accuracy in practice, especially in their experimental setup.</p><p>If you want to check out those, I have written the blogs on them already.</p><h4>Prompt Tuning:</h4><p><a href="https://medium.com/@mailpraveenreddy.c/p-tuning-v2-fine-tuning-technique-explained-66e801f31758">P-Tuning v2 — Fine Tuning Technique Explained</a></p><h4>Prefix Tuning:</h4><p><a href="https://medium.com/@mailpraveenreddy.c/prefix-tuning-a-simple-and-clear-explanation-ab864ce99af1">Prefix Tuning: A Simple and Clear Explanation</a></p><p>The paper says, the Limitations of Prefix, Prompt Tuning are covered by <strong>(IA)³</strong>, which is based on <strong>element-wise scaling (multiplication)</strong>. The idea is simple: <em>We multiply each feature by a learned value(l), which controls its importance.</em></p><pre>l ⊙ x<br><br>⊙ means multiplying each feature of a model vector independently (feature-wise scaling), rather than performing full matrix multiplication.</pre><p>Here, the scaling is done per feature dimension. This means each dimension in the activation can be adjusted independently — <strong>some features can be amplified (increased), while others can be suppressed (reduced).</strong></p><pre>We can represent the same with below equation:<br><br>(l ⊙ x)_{i,j} = l_j * x_{i,j}</pre><blockquote>This gives the model fine-grained control over which features matter more for a given task.</blockquote><p>The paper suggests to apply the scaling in two important components: <em>the attention mechanism and the feed-forward layers.</em></p><blockquote>In the attention mechanism, scaling is applied to the <strong>keys (K)</strong> and <strong>values (V)</strong>. Scaling the keys affects how attention scores are computed, while scaling the values affects the final output of the attention layer.</blockquote><pre>softmax( Q (l_k ⊙ K^T) / sqrt(d_k) ) (l_v ⊙ V)<br><br>Scale the Keys, Values (K,V)<br>- l_k is a learned vector that scales each feature of the Keys<br>- l_v is a learned vector that scales each feature of the Values<br><br>K_scaled = l_k ⊙ K^T<br><br>V_scaled = l_v ⊙ V</pre><blockquote>In the feed-forward layer, scaling is applied to the intermediate representation <strong>after the first linear transformation and activation function</strong>. This again allows the model to control which features should be emphasized.</blockquote><pre>(l_ff ⊙ γ(W1 x)) W2<br><br>- l_ff is a learned vector that scales each dimension</pre><p>To enable this behavior, the paper introduces a small set of new parameters: <strong>scaling vectors for keys, values, and feed-forward layers</strong>. Each layer gets its own set of these vectors, but overall the number of added parameters is still very small compared to full fine-tuning.</p><pre>Here we will understand the total no of parameters gonna be added<br><strong><br># For Encoder</strong> -- The formula for the total no of learned vectors added is: <br><strong>L (d_k + d_v + d_ff)</strong><br><br><strong># For Decoder</strong> -- The formula for the total no of learned vectors added is:<br><strong>L (2d_k + 2d_v + d_ff)</strong><br><br><br>For the encoder, each layer introduces three learned scaling vectors:<br>l_k --&gt; for keys (size = d_k)<br>l_v --&gt; for values (size = d_v)<br>l_ff --&gt; for feed-forward layer (size = d_ff)<br>So, in one layer, total parameters = d_k + d_v + d_ff.<br>Since there are L layers, total becomes: L (d_k + d_v + d_ff)<br><br>For the decoder, it is slightly larger because the decoder has two attention blocks:<br>- Self-attention<br>- Cross-attention (attention on encoder outputs)<br><br>Each attention block has its own:<br>- keys (d_k)<br>- values (d_v)<br>So for each layer:<br>Keys: 2 × d_k<br>Values: 2 × d_v<br>Feed-forward: d_ff<br>Total per layer = 2d_k + 2d_v + d_ff<br>Across L layers: L (2d_k + 2d_v + d_ff)</pre><p>The paper initialize all scaling vectors to 1 in the starting. <strong><em>This ensures that, at the start of training, the model behaves exactly like the original pretrained model because multiplying by 1 does not change anything.</em></strong></p><pre>l_k = l_v = l_ff = 1</pre><p>We already know that, the name <strong>IA³</strong> stands for <em>Infused Adapter by Inhibiting and Amplifying Inner Activations</em>. <strong>This reflects the core idea of the method: some features are inhibited (reduced), while others are amplified (increased), depending on their importance for the task.</strong></p><p>Because the scaling is applied independently for each example, the method naturally supports mixed-task batches for fine-tuning. Different inputs in the same batch can use different scaling vectors, allowing the same model to handle multiple tasks in a single forward pass.</p><p>The paper also says that<strong> scaling the activation is mathematically equivalent to scaling the weights. This allows the scaling parameters to be merged into the model weights during inference, resulting in no additional runtime cost.</strong></p><pre>l ⊙ (W x) = (l ⊙ W) x</pre><p>Finally, The paper introduces something called <strong>T-Few recipe</strong>, which combines all their ideas into a simple and consistent training approach.</p><ul><li>They use the T0 model as the base and enhance it with (IA)³.</li><li>For training, they combine three types of loss functions: the standard language modeling loss, unlikelihood loss to penalize incorrect predictions, and length-normalized loss to handle bias toward shorter outputs.</li><li>The training setup is kept minimal and efficient, using 1000 steps, a batch size of 8, a learning rate of 3 × 10⁻³, and a short warmup of 60 steps.</li></ul><p>An important point in the paper is that the exact same setup is used across all datasets, without any hyper parameter tuning or dataset-specific modifications and it worked better than prompt tuning, prefix tuning &amp; ICL.</p><p>That’s it. That’s all about <strong>IA³. </strong>Thank you for checking out my work. :)</p><h3>Thank You !! Happy Learning !!</h3><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ab080daa2f56" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>