<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Sagar Patil on Medium]]></title>
        <description><![CDATA[Stories by Sagar Patil on Medium]]></description>
        <link>https://medium.com/@sagarpatiler?source=rss-ee89211c0d51------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*yox7_1yENt3Ao_QV</url>
            <title>Stories by Sagar Patil on Medium</title>
            <link>https://medium.com/@sagarpatiler?source=rss-ee89211c0d51------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 04:59:08 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@sagarpatiler/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[AGENTS.md]]></title>
            <link>https://medium.com/@sagarpatiler/agents-md-dbc1f69f689d?source=rss-ee89211c0d51------2</link>
            <guid isPermaLink="false">https://medium.com/p/dbc1f69f689d</guid>
            <category><![CDATA[claude]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[agents]]></category>
            <dc:creator><![CDATA[Sagar Patil]]></dc:creator>
            <pubDate>Sat, 07 Feb 2026 09:13:13 GMT</pubDate>
            <atom:updated>2026-02-07T09:13:58.181Z</atom:updated>
            <content:encoded><![CDATA[<blockquote>AGENTS.md is a single Markdown file that lives at your repository root and tells AI coding tools exactly how to work on your project. Over 60,000 repositories already use it, reporting 35–55% fewer AI-generated bugs.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/874/1*Jv20I012hsHDpXE8D3XtUw.png" /></figure><h3>What Is AGENTS.md?</h3><p>AGENTS.md solves a fundamental problem in AI-assisted development. While README.md files are written for humans with project descriptions, badges, and contribution guidelines, AI coding tools need different information to work effectively on your codebase.</p><h4><strong>The Problem Before AGENTS.md</strong></h4><p>Before AGENTS.md became the standard, developers faced several challenges:</p><ul><li>README.md was optimized for human readers, not AI tools</li><li>AI tools would run wrong commands, ignore conventions, and break development workflows</li><li>Each tool demanded its own configuration file like .cursorrules, CLAUDE.md, .gemini, or .github/copilot-instructions.md</li><li>Teams maintained four to six overlapping configuration files, leading to duplication and version drift</li><li>New contributors and new tools had to hunt for the correct rules across multiple files</li></ul><h4><strong>The AGENTS.md Solution</strong></h4><p>AGENTS.md provides a standardized approach:</p><ul><li>A single AGENTS.md file placed at your repository root</li><li>Written in plain Markdown with no special schema, YAML, or JSON required</li><li>All major AI coding tools read it automatically without additional configuration</li><li>One source of truth that eliminates duplication and confusion</li></ul><h4>Evolution</h4><p>AGENTS.md was released in <strong>August 2025 by OpenAI </strong>and is now stewarded by the Linux Foundation’s Agentic AI Foundation. The adoption and impact have been significant:</p><ul><li>Over 60,000 repositories on GitHub have adopted AGENTS.md</li><li>Projects report 35–55% fewer agent-generated bugs after implementation</li><li>Setup time for new AI coding sessions drops from 20–40 minutes to under 2 minutes</li><li>The OpenAI repository itself contains 88 AGENTS.md files across its monorepo</li></ul><h3>Supported Tools and Platforms</h3><p>AGENTS.md has broad support across the AI coding tool ecosystem.</p><p><strong>Tools with Native Support</strong></p><p>The following tools read AGENTS.md automatically without any configuration:</p><ul><li>Cursor</li><li>GitHub Copilot agent mode</li><li>Claude Code</li><li>Aider, OpenHands, and Windsurf all have built-in support</li><li>Amp, Jules, Factory, Codex, RooCode, and Zed</li></ul><p><strong>Tools Requiring Configuration</strong></p><p>For Gemini CLI, add this to your .gemini/settings.json file:</p><pre>{<br>  &quot;contextFileName&quot;: &quot;AGENTS.md&quot;<br>}</pre><h3>Basic AGENTS.md Template</h3><p>The simplest effective AGENTS.md includes four key sections. Here is a basic template you can adapt to your project:</p><pre># AGENTS.md<br><br>## Setup Commands<br>- Install dependencies with `pnpm install`<br>- Start development server with `pnpm dev`<br>- Run tests with `pnpm test`<br>- Build for production with `pnpm build`<br><br>## Code Style<br>- Use TypeScript strict mode for all new code<br>- Prefer single quotes over double quotes<br>- Do not use semicolons<br>- Use functional patterns where possible<br><br>## Testing<br>- Run tests before committing any changes<br>- Add tests for all new features<br>- Maintain minimum 80% code coverage<br><br>## Security<br>- Never commit secrets or API keys to the repository<br>- Use environment variables for all sensitive configuration<br>- Sanitize all user inputs before processing</pre><p>This basic structure provides AI tools with the essential information they need to work effectively on your project.</p><h3>Working with Monorepos</h3><p>AGENTS.md supports nested files for monorepos where different packages have different requirements. This is one of the most powerful features of the standard.</p><p><strong>Directory Structure:</strong></p><pre>my-monorepo/<br>├── AGENTS.md              (root-level defaults)<br>├── packages/<br>│   ├── api/<br>│   │   └── AGENTS.md      (API-specific instructions)<br>│   ├── web/<br>│   │   └── AGENTS.md      (web app instructions)<br>│   └── shared/<br>│       └── AGENTS.md      (shared library instructions)</pre><p>When an AI tool works on a file, it reads the nearest AGENTS.md file in the directory tree. The closest file takes precedence, allowing you to set sensible defaults at the root and override them for specific packages.</p><p><strong>Conflict Resolution Hierarchy:</strong></p><p>The precedence order when instructions conflict is clearly defined:</p><ol><li>Explicit user chat prompts always override everything</li><li>The nearest AGENTS.md file to the edited file takes precedence</li><li>The root AGENTS.md provides fallback defaults</li></ol><p>This hierarchy ensures that users always have final control while still providing intelligent defaults.</p><h4>Example AGENTS.md for Fastapi</h4><pre># AGENTS.md<br><br>## Setup Commands<br>- Install dependencies: `pip install -r requirements.txt`<br>- Run development server: `uvicorn main:app --reload`<br>- Run tests: `pytest`<br>- Format code: `black .`<br>- Lint code: `ruff check .`<br><br>## Code Style<br>- Use Python 3.10+ with modern type hints<br>- All functions must have type annotations<br>- Use `async def` for all route handlers<br>- Import order: standard library, third-party, local application<br>- Use descriptive variable names, avoid single letters except in loops<br><br>## FastAPI Conventions<br>- Use Pydantic models for all request and response bodies<br>- Return Pydantic models from endpoints, not dictionaries<br>- Use dependency injection for database sessions<br>- Apply appropriate HTTP status codes from `fastapi.status`<br>- Add response_model to all route decorators<br><br>## Testing<br>- Run tests before committing any changes<br>- Add tests for new endpoints, even if not explicitly requested<br>- Use pytest fixtures for test client and database<br>- All tests must pass before code is complete<br><br>## Security<br>- Never commit API keys, secrets, or credentials<br>- Use environment variables for sensitive configuration<br>- Always validate input with Pydantic models<br>- Use authentication dependencies for protected routes<br><br>## Common Gotchas<br>- FastAPI routes are async - always use `async def` for handlers<br>- Remember to `await` database operations<br>- Pydantic v2 uses `model_validate()` not `parse_obj()`<br>- Don&#39;t forget to add new routes to the router</pre><h3>Conclusion</h3><p>AGENTS.md represents a standardization effort that benefits the entire AI-assisted development ecosystem. By adopting this standard, you contribute to better interoperability across tools while making your own development workflows more efficient and reliable.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=dbc1f69f689d" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Prompt/Tokens Optimization -TOON]]></title>
            <link>https://ai.plainenglish.io/prompt-tokens-optimization-toon-87999f1944c8?source=rss-ee89211c0d51------2</link>
            <guid isPermaLink="false">https://medium.com/p/87999f1944c8</guid>
            <category><![CDATA[agentic-rag]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[generative-ai-tools]]></category>
            <category><![CDATA[optimization]]></category>
            <dc:creator><![CDATA[Sagar Patil]]></dc:creator>
            <pubDate>Fri, 14 Nov 2025 12:43:19 GMT</pubDate>
            <atom:updated>2025-11-25T07:07:56.269Z</atom:updated>
            <content:encoded><![CDATA[<p><strong>TOON (Token-Oriented Object Notation)</strong>- compact, human-readable serialization format designed for passing structured data to Large Language Models with significantly reduced token usage up to 60%</p><h3>The Hidden Cost of JSON in AI Applications</h3><p>If you’re building applications with Large Language Models, you’ve likely experienced this moment: you craft a perfect prompt, add your structured data in clean JSON format, hit send, and watch your token count skyrocket. That beautiful, human-readable JSON? It’s quietly draining your budget with every curly brace, quote mark, and repeated key.</p><p>TOON (Token-Oriented Object Notation) promises to reduce token usage and API costs by over 50% by replacing JSON with a compact format built for LLM efficiency. This isn’t just theoretical optimization — it’s a practical solution to one of the most expensive aspects of modern AI development.</p><h3>Understanding the Token Economics Problem</h3><p>Large language models don’t see text as we do — they see it as a stream of tokens, which are common sequences of characters. Every model has a limited context window (such as 128,000 tokens), and everything costs tokens for input and output.</p><p>Consider a simple product catalog sent to an LLM:</p><pre>{<br>  &quot;products&quot;: [<br>    {<br>      &quot;sku&quot;: &quot;LPT-1234&quot;,<br>      &quot;productName&quot;: &quot;ProLaptop 15&quot;,<br>      &quot;inStock&quot;: true,<br>      &quot;price&quot;: 1299.99<br>    },<br>    {<br>      &quot;sku&quot;: &quot;MSC-5678&quot;,<br>      &quot;productName&quot;: &quot;Wireless Mouse&quot;,<br>      &quot;inStock&quot;: false,<br>      &quot;price&quot;: 25.50<br>    }<br>  ]<br>}</pre><p>The actual data is minimal, but the JSON braces, quotes, and repeated keys had somehow tripled the size. You’re literally paying for punctuation.</p><h3>What is TOON?</h3><p>TOON relies on a token-efficient tabular style, which is much closer to how LLMs naturally understand structured data. Instead of repeating field names for every object, TOON declares the structure once and then streams the data.</p><p>Here’s the same product data in TOON format:</p><pre>products[2]{sku,productName,inStock,price}:<br>LPT-1234,ProLaptop 15,true,1299.99<br>MSC-5678,Wireless Mouse,false,25.50</pre><p>That’s roughly a 56% reduction in prompt tokens — same information, half the cost.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Z58LXbmTi0edQsaG.png" /></figure><h3>How TOON Works: The Core Syntax</h3><p>TOON’s design philosophy is elegant: It borrows YAML’s indentation-based structure for nested objects and CSV’s tabular format for uniform data rows, then optimizes both for token efficiency in LLM contexts.</p><h4>Key Features</h4><p><strong>Array Declaration with Length</strong> TOON includes the array length in brackets, which helps LLMs understand and validate the structure:</p><pre>users[3]{id,name,role}:</pre><p><strong>Field Headers Declared Once</strong> Unlike JSON where each object repeats all keys, TOON declares the structure once in the header and then streams the values in tabular rows, eliminating the token cost of repeating keys and punctuation.</p><p><strong>Smart Quoting</strong> TOON uses deterministic quoting rules — only adding quotes when necessary (strings with commas, leading spaces, or values that look like numbers/booleans).</p><p><strong>Indentation for Nesting</strong> Complex nested structures use indentation similar to YAML:</p><pre>metadata:<br>  version: 1<br>  author: test<br>items[2]{id,name}:<br>1,Item1<br>2,Item2<br>tags[3]: alpha,beta,gamma</pre><h3>Real-World Performance: The Numbers</h3><p>The TOON project ran comprehensive benchmarks across different data types and LLM models, testing 4 different models (GPT-5 Nano, Claude Haiku, Gemini Flash, Grok) on 154 data retrieval questions.</p><p><strong>Token Efficiency Results:</strong></p><ul><li>Token savings of 30–60% for uniform tabular data</li><li>A 56% reduction in prompt tokens with noticeable 5-second speed improvement</li></ul><p><strong>Accuracy Improvements:</strong> Surprisingly, TOON achieves 73.9% accuracy versus JSON’s 69.7% while using 39.6% fewer tokens. The explicit structure (array lengths, field declarations) helps models parse and validate data more reliably.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*B-_YCQ1TXRlhl7_XZ56Tmw.png" /><figcaption>credits -<a href="http://www.curiouslychase.com/">www.curiouslychase.com</a></figcaption></figure><h3>When TOON Shines: Ideal Use Cases</h3><h4>1. AI Agent Communication</h4><p>A sales AI agent that exchanges JSON payloads with CRMs could switch to TOON to halve its data usage. When agents need to pass structured data between systems, token efficiency directly impacts response times and costs.</p><h4>2. RAG Systems</h4><p>RAG (Retrieval-Augmented Generation) systems query knowledge bases and send results to LLMs, where TOON makes this efficient:</p><p>toon</p><pre>documents[100]{id,title,content,relevance}:<br>doc-1,Authentication Best Practices,&quot;Use bcrypt for...&quot;,0.98<br>doc-2,Session Management,&quot;HTTP-only cookies...&quot;,0.95<br>doc-3,OAUTH Implementation,&quot;Third-party auth...&quot;,0.92</pre><h4>3. Database Query Results</h4><p>Consider a scenario where there is a knowledge base and the LLM can call a tool to access a database — the database usually returns this data in JSON, which is then passed back into the LLM so it can reason over it. Converting these results to TOON before sending to the LLM can cut costs significantly.</p><h4>4. Multi-Turn Conversations</h4><p>When building conversational AI that maintains context across turns, every token saved in the conversation history allows for longer, more meaningful exchanges within the context window.</p><p><strong>Use TOON when:</strong></p><ul><li>Working with language models, building AI agents, or developing systems where token costs directly impact your budget</li><li>Sending uniform arrays of objects to LLMs</li><li>Token efficiency is a significant concern</li><li>Making frequent LLM API calls with structured data</li></ul><h3>Implementation: Getting Started</h3><p>TOON has a growing ecosystem across multiple languages. Following is python implementation.</p><pre>pip install toon-format</pre><pre>from toon_format import estimate_savings, compare_formats, count_tokens<br><br># Measure savings<br>data = {&quot;users&quot;: [{&quot;id&quot;: 1, &quot;name&quot;: &quot;Alice&quot;}, {&quot;id&quot;: 2, &quot;name&quot;: &quot;Bob&quot;}]}<br>result = estimate_savings(data)<br>print(f&quot;Saves {result[&#39;savings_percent&#39;]:.1f}% tokens&quot;)  # Saves 42.3% tokens<br><br># Visual comparison<br>print(compare_formats(data))<br># Format Comparison<br># ────────────────────────────────────────────────<br># Format      Tokens    Size (chars)<br># JSON            45             123<br># TOON            28              85<br># ────────────────────────────────────────────────<br># Savings: 17 tokens (37.8%)<br><br># Count tokens directly<br>toon_str = encode(data)<br>tokens = count_tokens(toon_str)  # Uses tiktoken (gpt5/gpt5-mini)print(decoded_data)  # Returns list of dictionaries</pre><h3>The Hybrid Approach: Best of Both Worlds</h3><p>The core philosophy is simple: use JSON in your application code, convert to TOON when sending data to LLMs, then work with JSON results.</p><p>This approach gives you:</p><ul><li><strong>Development clarity</strong> — JSON’s familiarity and tooling during development</li><li><strong>Production efficiency</strong> — TOON’s token savings where it matters</li><li><strong>Maintainability</strong> — Standard JSON for APIs, databases, and logging</li><li><strong>Cost optimization</strong> — Reduced LLM API expenses</li></ul><h3>Accuracy Comparison by Model</h3><p>The benchmarks tested how well each LLM could extract information from data in different formats:</p><p><strong>GPT-5 Nano</strong> (Best Overall)</p><pre>TOON:          ███████████████████░ 96.1% (148/154)<br>CSV:           ██████████████████░░ 91.6% (141/154)<br>YAML:          ██████████████████░░ 91.6% (141/154)<br>JSON Compact:  ██████████████████░░ 91.6% (141/154)<br>XML:           █████████████████░░░ 87.0% (134/154)<br>JSON:          █████████████████░░░ 86.4% (133/154)</pre><p><strong>Claude Haiku 4.5</strong> (Most Consistent)</p><pre>JSON:          ██████████░░░░░░░░░░ 50.0% (77/154)<br>YAML:          ██████████░░░░░░░░░░ 49.4% (76/154)<br>TOON:          ██████████░░░░░░░░░░ 48.7% (75/154)<br>XML:           ██████████░░░░░░░░░░ 48.1% (74/154)</pre><p><strong>Gemini 2.5 Flash</strong> (CSV-Optimized)</p><pre>CSV:           ██████████████████░░ 87.7% (135/154)<br>XML:           ██████████████████░░ 87.7% (135/154)<br>TOON:          █████████████████░░░ 86.4% (133/154)</pre><h3>Key Insights from the Exploration</h3><ol><li><strong>TOON’s Sweet Spot</strong>: The benchmarks test datasets that favor TOON’s strengths (uniform tabular data). Real-world performance depends heavily on your data structure</li><li><strong>CSV Limitations</strong>: While CSV is highly token-efficient, it only supports flat tabular data and failed on 95 out of 154 questions that required nested structures</li><li><strong>Compact JSON vs Pretty JSON</strong>: Minifying JSON saves tokens but TOON still outperforms it by 10–15% while being more readable</li><li><strong>YAML Performance</strong>: YAML is less verbose than JSON but still uses more tokens than TOON due to its structural indicators</li></ol><h3>Real-World Cost Impact</h3><p>Let’s translate token savings into actual dollars:</p><p><strong>Scenario</strong>: AI analytics platform processing employee data</p><ul><li>Dataset: 100 employee records, 10 fields each</li><li>Requests: 10,000 API calls per month</li><li>Model: GPT-4 at $0.03 per 1K input tokens</li></ul><p><strong>Token Usage Per Request</strong>:</p><ul><li>JSON: 4,545 tokens</li><li>TOON: 2,744 tokens</li><li>Savings: 1,801 tokens (39.6%)</li></ul><p><strong>Monthly Cost</strong>:</p><ul><li>JSON: (10,000 × 4,545 tokens) / 1000 × $0.03 = $1,363.50</li><li>TOON: (10,000 × 2,744tokens) / 1000 × $0.03 = <strong>$823.20</strong></li><li><strong>Monthly Savings: $540.30</strong></li><li><strong>Annual Savings: $6,483.60</strong></li></ul><h3>The Future of AI-Native Data Formats</h3><p>As AI models process and reason over text, a new challenge emerged: token efficiency, where every character counts in LLMs, directly affecting cost and performance. TOON represents the first wave of formats designed specifically for this challenge.</p><p>TOON isn’t another transient data format — it’s a token-aware communication layer for the AI era that complements JSON rather than replaces it.</p><p>As context windows grow and models become more sophisticated, the importance of token efficiency won’t diminish — it will evolve. Just as JSON has been a standard for the Web’s data exchange, TOON may soon be standardized for AI data interchange.</p><h3>Getting Started Today</h3><p>The TOON ecosystem is production-ready and actively maintained:</p><ul><li><strong>Official Specification</strong>: <a href="https://github.com/toon-format/toon">github.com/toon-format/toon</a></li><li><strong>JavaScript/TypeScript</strong>: npm install @toon-format/toon</li><li><strong>Python</strong>: pip install toon-format</li><li><strong>CLI Tools</strong>: Available for batch conversions and benchmarking</li></ul><p>Use the official TOON repository and the interactive tokenization playground to see exactly how much you’d save with your data.</p><h3>Conclusion: A Format for the Token Economy</h3><p>The format aligns perfectly with the trend toward token-aware engineering — building AI pipelines that are not only accurate, but efficient.</p><p>TOON doesn’t replace JSON everywhere, nor should it. But for the specific use case of feeding structured data to LLMs — particularly uniform, tabular data — it offers compelling advantages in cost, speed, and even accuracy.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=87999f1944c8" width="1" height="1" alt=""><hr><p><a href="https://ai.plainenglish.io/prompt-tokens-optimization-toon-87999f1944c8">Prompt/Tokens Optimization -TOON</a> was originally published in <a href="https://ai.plainenglish.io/">Artificial Intelligence in Plain English</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Vector Quantization]]></title>
            <link>https://ai.plainenglish.io/vector-quantization-2eef9db8a61f?source=rss-ee89211c0d51------2</link>
            <guid isPermaLink="false">https://medium.com/p/2eef9db8a61f</guid>
            <category><![CDATA[generative-ai-tools]]></category>
            <category><![CDATA[vector-database]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Sagar Patil]]></dc:creator>
            <pubDate>Mon, 06 Oct 2025 08:08:05 GMT</pubDate>
            <atom:updated>2025-10-07T16:06:05.190Z</atom:updated>
            <content:encoded><![CDATA[<h4>Why …..What…..How</h4><blockquote>Generative AI has brought embeddings into the spotlight. Whether it’s a chatbot retrieving knowledge through <strong>Retrieval-Augmented Generation (RAG)</strong> or a recommendation engine surfacing the right product, embeddings power the semantic understanding that makes these systems intelligent.</blockquote><p>But there’s a catch — <strong>embeddings are large</strong>. Vector databases today often need to store <strong>millions or even billions</strong> of high-dimensional vectors. Storing, transmitting, and retrieving them in <strong>full float32 precision</strong> is memory-intensive, slow, and costly. For instance, a single <strong>1536-dimensional embedding/vector</strong> (like OpenAI’s text-embedding-ada-002) consumes about <strong>6 KB of memory</strong>. Multiply that by 100 million vectors, and you’re looking at nearly 600 gigabytes <strong>of RAM</strong> — far beyond what’s practical or affordable to operate at scale.</p><p>As systems grow from handling hundreds of queries to <strong>thousands per second</strong>, this overhead becomes a major bottleneck. The challenges are clear:</p><ul><li><strong>High memory usage</strong> from large float32 embeddings</li><li><strong>Increased latency</strong> when computing similarity across huge datasets</li><li><strong>Rising infrastructure costs</strong> as hardware scales to keep embeddings in RAM</li><li><strong>Reduced throughput</strong> due to the computational overhead of high-precision data</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*E-tTeZK_myi96gHmVCMOGw.png" /><figcaption>Memory requirement for 32 FP vector embedding</figcaption></figure><p>This is where <strong>vector quantization</strong> becomes essential — a technique that compresses embeddings while preserving their meaning. By reducing precision of vectors (for example, from float32 to float16, int8, or even binary), organizations can:</p><ul><li>Cut memory usage by <strong>70–90%</strong> with minimal loss in accuracy</li><li>Achieve <strong>faster retrieval speeds</strong> and <strong>lower latency</strong></li><li>Scale semantic search and generative AI systems <strong>efficiently and affordably</strong></li></ul><p>In essence, vector quantization turns large-scale vector search from a costly experiment into a <strong>production-ready, scalable, and high-performance solution</strong>.</p><h3>What is vector quantization?</h3><p>Before diving into vector quantization, let’s first understand <strong>quantization</strong> in general.</p><h4>Quantization</h4><p>In simple terms, <strong>quantization</strong> is a process that reduces the precision of numbers to make computations faster and storage more efficient. It’s commonly used in neural networks to shrink large models without significantly hurting their accuracy. For example, model parameters (weights and activations) originally stored as <strong>32-bit floating-point numbers (float32)</strong> — each taking 4 bytes — can be converted into <strong>8-bit integers (int8)</strong>, which use just 1 byte. This results in a <strong>75% reduction in memory usage</strong> while keeping most of the model’s performance intact.</p><p>Quantization works much like rounding numbers in everyday math — you trade a little precision for a big gain in efficiency. Modern hardware like GPUs and TPUs are optimized for low-precision arithmetic, making quantized models run faster and consume less power.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5PptEc7BAcADHuQBOVph2w.png" /><figcaption>Quantization</figcaption></figure><h4>Vector Quantization</h4><p>Now, when we apply this idea to <strong>embeddings</strong> — the numerical representations used for text, images, or audio — it becomes <strong>vector quantization</strong>.</p><p>Vector embeddings have been typically represented as 32-bit floating-point numbers. <strong>Vector quantization techniques reduce the size of the vector embeddings by representing them as smaller numbers, such as 8-bit integers or binary numbers. </strong>Some quantization techniques also reduce the number of dimensions in the vector embeddings.</p><h4>Floating Point and memory requirement</h4><p>To understand <strong>vector quantization (VQ) basics</strong>, it’s important to first grasp how different <strong>numeric data types of</strong> store values and how reducing precision saves memory. In computing, <strong>floating-point numbers</strong> represent real values with three key components:</p><ul><li><strong>Sign</strong>: A single bit that indicates whether the number is positive (0) or negative (1).</li><li><strong>Exponent</strong>: Determines the scale of the number by shifting the decimal point, allowing very large and very small values to be represented.</li><li><strong>Mantissa (significand)</strong>: Holds the significant digits of the number, providing precision.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*h0VxwPxDiBZgWTCy4B8p0g.png" /></figure><p>For example, <strong>Float32 (FP32)</strong> — also known as <em>single-precision floating point</em> — uses <strong>32 bits (4 bytes)</strong> per value.</p><h4>Calculating Memory Requirements</h4><p>The memory needed for storing an embedding depends on two factors:</p><ul><li><strong>Bit-width of the data type</strong> (e.g., 32 for FP32, 16 for FP16, 8 for INT8).</li><li><strong>Number of dimensions</strong> in the vector.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/481/1*_kdJlUS2GlGPyU6IUV9VvA.png" /></figure><p>Now let&#39;s calculate memory required for vector of dimensions of 512:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*LajIABz9dAZjvuiuA07HaQ.png" /><figcaption>Vector</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*szSwHrl6nDNFhuVZmvSFEg.png" /><figcaption>Memory requirement for vector with 512 dimensions</figcaption></figure><h3>Key Techniques in Vector Quantization</h3><p>Vector quantization is not a one-size-fits-all technique. Depending on the trade-offs between <strong>accuracy, compression ratio, and speed</strong>, several quantization methods are commonly applied in vector databases and generative AI systems. Let’s look at the most widely used ones. There are three main approaches:</p><ol><li><strong>Scalar quantization</strong>: Converts individual 32-bit floating-point values to 8-bit integers, reducing the memory usage of vector values by 75% while maintaining reasonable precision.</li><li><strong>Binary quantization</strong>: This transforms vectors into binary (0/1) representations, achieving maximum compression but with more significant information loss.</li><li><strong>Product quantization</strong>: Compresses entire vectors at once by mapping them to a codebook of representative vectors, offering better compression than scalar quantization at the cost of more complex encoding/decoding.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Xr8tDhmwFcdAcehI.png" /><figcaption>Credit :Qdrant</figcaption></figure><h4><strong>Scalar Quantization</strong></h4><p>Scalar quantization is the simplest approach: each dimension of a vector is quantized independently. This means converting every <strong>float32</strong> value into a smaller format <strong>int8</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-Y025IfDJoZnurBxJjEDLg.png" /><figcaption>Scalar Quantization</figcaption></figure><p>Scalar quantization maps each floating-point number into a small integer value by first scaling the float into a limited range, then rounding it to the nearest integer. For example, you take the minimum and maximum float values, use them to linearly scale (Normalization) every float into the range –128 to 127, then round. The result is an <strong>int8 value</strong> instead of a float — saving memory while preserving the approximate relationships between vectors.</p><h4><strong>Binary Quantization (1-bit Embeddings)</strong></h4><p>Binary quantization pushes compression to the extreme: each dimension (each floating-point number) is reduced to just <strong>one bit</strong> (0/1). Values greater than zero are converted to 1. Values less than or equal to zero are converted to 0.</p><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*b3CxDXj1g9J2HOiGupJ7Dw.png" /><figcaption>Binary Quantization</figcaption></figure><p><strong>Memory Reduction —</strong> Binary quantization reduces memory requirement by 32x. eg. a 512-dimensional vector from 2 KB to just 64 B (32× memory reduction)</p><p><strong>Speed Gains — </strong>Binary Quantization offers the <strong>largest processing speed gains</strong> compared to Scalar and Product Quantization. It can speed up search operations by <a href="https://qdrant.tech/articles/binary-quantization/"><strong>up to 40x</strong></a>, depending on the dataset and hardware. The binary format leverages optimized CPU instructions (e.g., <strong>XOR</strong> and <strong>Pop count</strong>) for extremely fast distance computations.</p><p><strong>Model compatibility matters — </strong>Not all models handle Binary Quantization equally well — some may suffer higher accuracy loss. Qdrant recommends applying it to embeddings with <strong>at least 1024 dimensions</strong>. Models like <strong>OpenAI’s text-embedding-ada-002 (1536 dimensions)</strong> and <strong>Cohere’s embed-english-v2.0 (4096 dimensions)</strong> have shown the <strong>best compatibility</strong>, delivering substantial speed and memory gains with only minimal accuracy loss.</p><p><strong>Accuracy Loss and mitigation —</strong> In practice, recall often drops. To mitigate this, many systems adopt a <strong>two-stage retrieval strategy</strong>:</p><ol><li><strong>Fast coarse search</strong> — Perform the initial search using binary embeddings. This step is extremely fast because similarity can be computed with simple bitwise operations like XOR and Popcount.</li><li><strong>Re-ranking with higher precision</strong> — Once a small set of top candidates is retrieved (e.g., top 200), their original float32 or float16 embeddings are used to re-rank them. This restores accuracy while still benefiting from the massive speed and memory efficiency of binary search.</li></ol><h4>Product Quantization (PQ)</h4><p>Product Quantization is a smarter way to compress high-dimensional vectors by breaking them into parts and encoding each part separately. Rather than compressing each dimension independently, PQ divides a high-dimensional vector into several <strong>smaller, equal-length subvectors</strong>. This decomposition allows the algorithm to handle smaller chunks of data, making compression both more effective and scalable in high-dimensional spaces. you:</p><ol><li><strong>Split the vector into sub-vectors</strong> e.g. a 1536-dim vector might be split into 8 subvectors of 192 dims (values)each.</li><li><strong>Train a small codebook for each sub-vector space</strong> using clustering (like k-means). Each subspace gets its own centroids.</li><li><strong>Replace each sub-vector with the index of its nearest centroid.</strong> So each original sub-vector becomes a small integer (centroid ID).</li><li><strong>Concatenate those centroid indices</strong> to form a compact “code” for the original vector.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qIOEPgsw3hOFzpf7VeZL3A.png" /><figcaption>Product Quantization</figcaption></figure><p>At query time, you can approximate distances by comparing the query’s sub-vectors against those centroids, summing up the partial distances. This lets you do similarity search without decompressing the entire vector.</p><p>This approach achieves <strong>significant compression</strong> while largely preserving the geometric structure of the data, enabling <strong>fast similarity searches</strong> and <strong>efficient storage</strong> in vector databases. However, PQ introduces more approximation than scalar quantization — the <strong>higher the compression</strong>, the greater the <strong>potential loss in precision</strong>.</p><h4>Conclusion</h4><p><strong>Binary Quantization</strong> is the fastest and most memory-efficient option, delivering up to <strong>40× faster searches</strong> and <strong>32× memory reduction</strong>. It’s best used with models proven to work well in binary form, such as <strong>OpenAI’s text-embedding-ada-002</strong>, especially when <strong>speed and efficiency are critical</strong>. <strong>Scalar Quantization</strong> provides up to <strong>4× memory savings</strong> with minimal accuracy loss, making it a solid choice when you need a <strong>balanced trade-off between accuracy, speed, and compression</strong>. <strong>Product Quantization (PQ)</strong> offers the <strong>highest compression ratio</strong> — up to <strong>64× memory reduction</strong> — and is most suitable when <strong>minimizing memory usage is the top priority</strong>, provided some accuracy loss and slower indexing can be tolerated</p><p>Related article : <a href="https://medium.com/ai-in-plain-english/vector-databases-514b94ca5a0a">Vector database for dummies</a></p><h3>A message from our Founder</h3><p><strong>Hey, </strong><a href="https://linkedin.com/in/sunilsandhu"><strong>Sunil</strong></a><strong> here.</strong> I wanted to take a moment to thank you for reading until the end and for being a part of this community.</p><p>Did you know that our team run these publications as a volunteer effort to over 3.5m monthly readers? <strong>We don’t receive any funding, we do this to support the community. ❤️</strong></p><p>If you want to show some love, please take a moment to <strong>follow me on </strong><a href="https://linkedin.com/in/sunilsandhu"><strong>LinkedIn</strong></a><strong>, </strong><a href="https://tiktok.com/@messyfounder"><strong>TikTok</strong></a>, <a href="https://instagram.com/sunilsandhu"><strong>Instagram</strong></a>. You can also subscribe to our <a href="https://newsletter.plainenglish.io/"><strong>weekly newsletter</strong></a>.</p><p>And before you go, don’t forget to <strong>clap</strong> and <strong>follow</strong> the writer️!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2eef9db8a61f" width="1" height="1" alt=""><hr><p><a href="https://ai.plainenglish.io/vector-quantization-2eef9db8a61f">Vector Quantization</a> was originally published in <a href="https://ai.plainenglish.io/">Artificial Intelligence in Plain English</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Why Language Models Hallucinate ?(Research)]]></title>
            <link>https://ai.plainenglish.io/why-language-models-hallucinate-research-ca9062771bd9?source=rss-ee89211c0d51------2</link>
            <guid isPermaLink="false">https://medium.com/p/ca9062771bd9</guid>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[genai]]></category>
            <category><![CDATA[ai]]></category>
            <dc:creator><![CDATA[Sagar Patil]]></dc:creator>
            <pubDate>Fri, 12 Sep 2025 12:38:16 GMT</pubDate>
            <atom:updated>2025-09-17T08:44:25.375Z</atom:updated>
            <content:encoded><![CDATA[<h3>Why Language Models Hallucinate?(Research)</h3><p>Language models (LMs), especially large ones (LLMs) like ChatGPT or GPT-5, are very good at generating fluent text. They can answer questions, write essays, translate, etc. But a persistent problem is <strong>hallucination</strong>: when these models present false or made-up statements <em>confidently</em>, as if they were true.</p><p>Recently, researchers at OpenAI published a paper titled <strong><em>“Why Language Models Hallucinate”</em> (Kalai, Nachum, Vempala, Zhang, 2025</strong>) which tries to explain <strong><em>structurally</em></strong> why hallucinations occur — not just as bugs or bad data, but as outcomes of how models are trained and evaluated. This article walks through what hallucination means, what the paper claims are its main causes, with examples, and what this means in practice.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/973/1*PIjQGHdsndg_qq6qoYpXEg.png" /></figure><h3>What is Hallucination?</h3><p>Hallucination refers to instances where a language model generates statements that:</p><ul><li>are <strong>plausible</strong> (they sound right, grammatical, coherent),</li><li><strong>presented confidently</strong>,</li><li>but are <strong>false</strong> or ungrounded in reality.</li></ul><p>For example, in the OpenAI paper, when asked: <em>What is the title of Adam Tauman Kalai’s PhD dissertation?</em> the model gave three different but incorrect answers. Asked for his birthday, it similarly gave various wrong dates. These are hallucinations because although the model acts confident, none of the answers is correct.</p><p>Hallucinations are serious because they reduce trust, mislead users, and in high-stakes settings (medicine, law, etc.) can be harmful.</p><h3>What Causes Hallucination?</h3><p>The paper argues that hallucinations come from both <strong>pre-training</strong> and <strong>post-training / evaluation</strong> processes. Below are the major causes, with simplified examples.</p><h4>Pre-Training factors</h4><p>(Errors caused by pre-training)</p><blockquote><strong>Next-Word Prediction Without Truth Labels</strong></blockquote><p>During pre-training, the model learns to predict the next word (token) given preceding text, using large corpora of text. The training data doesn’t come with labels that explicitly say, “this fact is true” or “this is false”. Models learn statistical patterns of language, co-occurrence, style, grammar, etc., but are not told which statements are factually correct.</p><p>Because of this, when asked questions about things not well represented in training data (rare facts), the model must “invent” plausible continuations. The inventing can lead to false facts.</p><blockquote><strong>Rare Facts and the “Singleton” Problem</strong></blockquote><p>Many facts or pieces of information appear only once (or very few times) in the training data. If a fact is rare, there may not be enough signal to reliably learn it. The paper draws from statistical learning theory (e.g., calibration, error bounds) to show that even under clean data, some hallucinations are statistically inevitable for “arbitrary facts” (facts that are not systematically repeated). For these one-off facts, the model doesn’t have enough examples to truly learn them, so when asked later it can only make an educated guess. Statistical learning theory shows that there’s a lower bound on errors in such cases, meaning some hallucinations will always occur no matter how advanced the model becomes.</p><p>Examples include obscure dates, unusual names, or unique personal details. If none of the examples in training gives enough evidence, the model may produce a plausible but incorrect answer.</p><p><strong>Analogy</strong></p><p>Think of it like studying for an exam:</p><ul><li>If the teacher repeats a fact 50 times in class, you’ll likely remember it correctly. If the teacher mentions a fact once, in passing, you might forget or confuse it. No matter how smart you are, if many exam questions are based on “facts you only heard once,” you will inevitably get some wrong.</li></ul><p>That’s what the paper means: hallucinations are baked in for rare, one-off facts.</p><p><strong>Additional Sources of Errors Highlighted in the Paper</strong></p><blockquote><strong>Computational Hardness</strong></blockquote><ul><li>Some problems are inherently too hard for any algorithm on a classical computer (because of complexity theory limits).</li><li>Example: decrypting an arbitrary ciphertext without the key. Even a “superhuman AI” can’t do this.</li><li>For such queries, <em>“I don’t know”</em> is the only valid answer, but models sometimes still try to guess.</li><li><em>Analogy:</em> Asking an AI to instantly solve an unsolved math problem like the Riemann Hypothesis. If it gives an answer, it’s likely wrong, because the task is beyond current computational limits.</li></ul><blockquote><strong>Distribution Shift</strong></blockquote><ul><li>Errors often happen when test-time prompts differ from what the model saw in training (out-of-distribution inputs).</li><li>Example: the classic riddle <em>“What’s heavier, a pound of feathers or a pound of lead?”</em> Such unusual or adversarial phrasing might not appear in training data, and some models answer incorrectly.</li><li><em>Analogy:</em> If you studied only standard exam problems but get trick questions with unusual wording, you might slip up even if you know the underlying concepts.</li></ul><blockquote><strong>GIGO: Garbage In, Garbage Out</strong></blockquote><ul><li>Large text corpora inevitably contain factual errors, conspiracy theories, or misconceptions. Models can and do reproduce them.</li><li>Example: If the internet says <em>“humans only use 10% of their brain”</em> often enough, a model may repeat that myth.</li><li>The paper notes that <strong>post-training</strong> (like RLHF) helps filter out some of these errors — but it doesn’t eliminate them, and sometimes new hallucinations can be introduced.</li><li><em>Analogy:</em> If you learn history from a textbook with mistakes, you may confidently repeat those mistakes later, unless a teacher corrects you.</li></ul><h4>Evaluation benchmarks factor</h4><p>(Why hallucinations survive post-training?)</p><p>After pretraining, models are post-trained and evaluated on benchmarks. The problem is that <strong>these benchmarks mainly reward accuracy</strong>: correct answers increase the score, while abstentions or uncertainty get no credit. As a result, models are <strong>incentivized to guess</strong>, since even a wrong guess sometimes leads to a correct hit, whereas saying “I don’t know” never helps.</p><p>Like a test where only correct answers count, and unanswered questions or “I don’t know” get zero. A student may prefer guessing something wrong rather than leaving blank, because wrong still gives a chance. Models act similarly under standard metrics.</p><p>In the same way, suppose a model is asked: <em>“What is the birthday of Dr. Peter Parker’s pet parrot?”</em> — something extremely rare in training data.</p><ul><li>If it says, “I don’t know,” it gets zero credit.</li><li>If it guesses “July 4,” there’s a tiny chance it’s correct.</li><li>On leaderboards, across thousands of questions, guessing may yield slightly more correct answers than abstaining — so the model is <em>rewarded for guessing</em>.</li></ul><p>The outcome? More hallucinations, because models are not incentivized to say, “I don’t know.”</p><p>The following system card data on SimpleQA eval benchmark highlights the trade-off between abstaining, accuracy, and hallucinations:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/787/1*QHewpHZvUsFrU7geol1Ong.png" /></figure><ul><li><em>gpt-5-thinking-mini</em> abstains a lot (52%). That lowers its accuracy score (22%) but also keeps error rate much lower (26%).</li><li><em>OpenAI o4-mini</em> almost never abstains (1%), so it scores slightly better on accuracy (24%) but suffers a huge error rate (75%).</li></ul><p>This perfectly illustrates the paper’s point: models trained/evaluated under <strong>accuracy-only incentives</strong> are pushed to guess, improving their accuracy score but <strong>dramatically increasing hallucinations</strong>.</p><p><strong>Accuracy-only scoreboards dominate leaderboards and model cards, motivating developers to build models that guess rather than hold back.</strong> This explains why even advanced models continue to hallucinate — they are trained and selected to sound confident, not cautious.</p><p>Moreover, while models may internally sense uncertainty, evaluation regimes rarely reward expressing it. Without changes to metrics, the safest strategy for a model remains: answer confidently, even if wrong.</p><h3>Conclusion</h3><p>Hallucinations are not random bugs but arise structurally from:</p><ul><li><strong>Pretraining limitations</strong> (no truth labels,rare facts), and</li><li><strong>Post-training incentives</strong> (accuracy-only benchmarks that reward guessing).</li></ul><p>The OpenAI paper makes clear that even as models become more advanced, hallucinations persist because evaluation systems push them toward confident answers rather than cautious honesty.</p><p>For practitioners, the key lesson is to rethink how we train, evaluate, and reward models — moving from accuracy-only metrics toward reliability, calibration, and humility.</p><h3>A message from our Founder</h3><p><strong>Hey, </strong><a href="https://linkedin.com/in/sunilsandhu"><strong>Sunil</strong></a><strong> here.</strong> I wanted to take a moment to thank you for reading until the end and for being a part of this community.</p><p>Did you know that our team run these publications as a volunteer effort to over 3.5m monthly readers? <strong>We don’t receive any funding, we do this to support the community. ❤️</strong></p><p>If you want to show some love, please take a moment to <strong>follow me on </strong><a href="https://linkedin.com/in/sunilsandhu"><strong>LinkedIn</strong></a><strong>, </strong><a href="https://tiktok.com/@messyfounder"><strong>TikTok</strong></a>, <a href="https://instagram.com/sunilsandhu"><strong>Instagram</strong></a>. You can also subscribe to our <a href="https://newsletter.plainenglish.io/"><strong>weekly newsletter</strong></a>.</p><p>And before you go, don’t forget to <strong>clap</strong> and <strong>follow</strong> the writer️!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ca9062771bd9" width="1" height="1" alt=""><hr><p><a href="https://ai.plainenglish.io/why-language-models-hallucinate-research-ca9062771bd9">Why Language Models Hallucinate ?(Research)</a> was originally published in <a href="https://ai.plainenglish.io/">Artificial Intelligence in Plain English</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Chain of Draft (CoD)]]></title>
            <link>https://blog.cubed.run/chain-of-draft-cod-f9c70f2d2f65?source=rss-ee89211c0d51------2</link>
            <guid isPermaLink="false">https://medium.com/p/f9c70f2d2f65</guid>
            <category><![CDATA[prompt]]></category>
            <category><![CDATA[generative-ai-tools]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <dc:creator><![CDATA[Sagar Patil]]></dc:creator>
            <pubDate>Sat, 08 Mar 2025 11:07:43 GMT</pubDate>
            <atom:updated>2025-03-09T22:46:53.971Z</atom:updated>
            <content:encoded><![CDATA[<h4>A Smarter Way for LLMs to Think Faster and Cheaper</h4><h4>Introduction</h4><p>Large Language Models (LLMs) have revolutionized reasoning and problem-solving, excelling in tasks ranging from mathematical problem-solving to commonsense reasoning. One of the most widely adopted techniques for improving LLM reasoning is <strong>Chain of Thought (CoT) prompting</strong>, which breaks complex problems into step-by-step solutions, mimicking human-like reasoning.</p><p>However, CoT is <strong>expensive and slow</strong>. It generates <strong>verbose intermediate steps</strong>, consuming excessive computational resources and increasing latency. This verbosity makes LLM deployment <strong>less practical in cost-sensitive environments</strong>.</p><p>To solve this issue, researchers from Zoom Communications ( <a href="https://arxiv.org/html/2502.18600v1">https://arxiv.org/html/2502.18600v1</a> ) introduced <strong>Chain of Draft (CoD)</strong> — a new reasoning paradigm that <strong>mimics how humans take notes concisely</strong> while solving problems. CoD achieves <strong>accuracy comparable to CoT while reducing computational cost and latency by up to 92.4%</strong>.</p><p>Let’s explore the <strong>problems with CoT</strong> and how <strong>CoD is a game-changer</strong> for efficient LLM reasoning.</p><h4>Chain of Thought (CoT) and Its Issues</h4><p>CoT prompting enables LLMs to <strong>think step by step</strong> by explicitly breaking problems into intermediate steps. This improves model reasoning, especially for complex tasks like <strong>math word problems, logic puzzles, and symbolic reasoning</strong>.</p><p>While <strong>accurate and interpretable</strong>, CoT <strong>has major drawbacks</strong>:</p><p><strong>Excessive Token Usage</strong></p><ul><li>CoT responses are long and detailed, consuming <strong>5–10× more tokens than necessary</strong>.</li><li>This increases the <strong>cost of inference</strong>, making it expensive to deploy in production.</li></ul><p><strong>Higher Latency</strong></p><ul><li>More tokens mean longer response times.</li><li>In real-time applications like AI assistants or coding copilots, <strong>speed is crucial</strong>.</li></ul><p><strong>Unnecessary Details</strong></p><ul><li>Humans do not <strong>elaborate on obvious steps</strong> when solving problems.</li><li>CoT forces LLMs to <strong>explain everything</strong>, even when concise reasoning suffices.</li></ul><h4>Chain of Draft (CoD): What Is It?</h4><p><strong>Chain of Draft (CoD)</strong> is a <strong>more efficient reasoning approach</strong> where LLMs generate <strong>concise intermediate steps</strong> instead of verbose reasoning chains.</p><ul><li><strong>Key Idea:</strong> Instead of full explanations, CoD <strong>drafts minimal reasoning steps</strong>, focusing only on essential information.</li><li><strong>Inspired by Human Note-Taking:</strong> Humans <strong>jot down</strong> short, abstract notes rather than writing <strong>full-length explanations</strong>. CoD <strong>trains LLMs to do the same</strong>.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/615/1*7FTfKIHTxyYnrS90hkNQbA.png" /><figcaption>Prompt for CoT and CoD</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/444/1*rVbdtWLz8lIXI15mPPa9iw.png" /><figcaption>Comparison of CoT and CoD</figcaption></figure><h4>Benefits of Chain of Draft (CoD)</h4><p><strong>1. 90% Reduction in Token Usage</strong></p><ul><li><strong>CoD uses as little as 7.6% of the tokens compared to CoT.</strong></li><li>This makes <strong>LLM inference cheaper</strong> — critical for large-scale AI applications.</li></ul><p><strong>2. 50–75% Faster Inference</strong></p><ul><li><strong>CoD speeds up response generation by cutting down token count.</strong></li><li>Less reasoning overhead means <strong>faster answers</strong> for real-time applications.</li></ul><p><strong>3. Comparable or Better Accuracy</strong></p><ul><li><strong>In benchmarks like GSM8K (math reasoning), CoD achieves 91% accuracy vs. CoT’s 95%, but with 80% fewer tokens.</strong></li><li>On <strong>symbolic reasoning tasks, CoD reaches 100% accuracy with 68–86% fewer tokens.</strong></li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/498/1*_MaJChr9iMCGqJyYYM49HQ.png" /></figure><p><strong>4. Better for Cost-Sensitive Applications</strong></p><ul><li>Cloud-based AI services <strong>charge per token</strong> (e.g., OpenAI, Anthropic, Google Gemini).</li><li>CoD significantly <strong>lowers costs</strong> while maintaining high-quality responses.</li></ul><p><strong>5. Scales Well for Real-World LLM Use Cases</strong></p><ul><li><strong>AI Assistants</strong> (faster, cost-effective responses).</li><li><strong>Code Generation</strong> (minimal explanations, faster execution).</li><li><strong>Mathematical &amp; Logical Reasoning</strong> (concise, correct results).</li></ul><h4>Limitations of Chain of Draft (CoD)</h4><p><strong>1. Less Effective for Small Language Models (&lt; 3 B Parameters)</strong></p><ul><li>Small models struggle with CoD due to lack of <strong>concise reasoning patterns in training data</strong>.</li><li><strong>Example:</strong> Qwen2.5–1.5B shows a <strong>significant accuracy drop compared to CoT</strong>.</li><li><strong>Solution:</strong> Fine-tune small models on compact reasoning datasets.</li></ul><p><strong>2. Lower Performance Without Few-Shot Examples</strong></p><ul><li>CoD relies on <strong>few-shot prompting</strong> for structured, minimal reasoning.</li><li><strong>Zero-shot CoD</strong> leads to lower accuracy and less concise responses.</li><li><strong>Example:</strong> GSM8K accuracy drops from <strong>91% (few-shot) to 84.4% (zero-shot)</strong>.</li><li><strong>Solution:</strong> Fine-tune models with CoD-style reasoning data.</li></ul><p><strong>3. May Oversimplify Complex Reasoning Tasks</strong></p><ul><li><strong>Risk:</strong> Losing critical insights in tasks requiring deep explanations (e.g., legal, medical).</li><li><strong>Example:</strong> CoD might <strong>skip important context</strong> in argumentation or subjective tasks.</li><li><strong>Solution:</strong> Use <strong>hybrid approaches</strong> combining CoD with self-reflection methods.</li></ul><p><strong>4. Requires Careful Prompt Engineering</strong></p><ul><li><strong>Risk:</strong> Poorly structured CoD prompts may lead to <strong>overly concise or ambiguous responses</strong>.</li><li><strong>Example:</strong> <strong>“France → Capital? → Paris.”</strong> (too direct) instead of <strong>“France’s capital is Paris.”</strong></li><li><strong>Solution:</strong> Use <strong>standardized CoD templates</strong> to maintain clarity and accuracy.</li></ul><h4>Conclusion</h4><p>The <strong>Chain of Draft (CoD)</strong> technique is a <strong>powerful, cost-efficient alternative to Chain of Thought (CoT)</strong>. It enables <strong>faster, cheaper, and equally accurate reasoning</strong> for LLMs, making it ideal for <strong>real-world AI applications</strong>.</p><p>By focusing on <strong>concise, essential reasoning</strong>, CoD reduces <strong>token usage, latency, and inference costs</strong>, paving the way for <strong>more efficient AI systems</strong>.</p><p>CoD could be combined with <strong>adaptive parallel reasoning or knowledge graphs</strong> to further <strong>optimize AI decision-making</strong>.</p><h3>Thank you for being a part of the community</h3><p><em>Before you go:</em></p><ul><li>Be sure to <strong>clap</strong> and <strong>follow</strong> the writer ️👏<strong>️️</strong></li><li>Follow us: <a href="https://x.com/inPlainEngHQ"><strong>X</strong></a> | <a href="https://www.linkedin.com/company/inplainenglish/"><strong>LinkedIn</strong></a> | <a href="https://www.youtube.com/channel/UCtipWUghju290NWcn8jhyAw"><strong>YouTube</strong></a> | <a href="https://newsletter.plainenglish.io/"><strong>Newsletter</strong></a> | <a href="https://open.spotify.com/show/7qxylRWKhvZwMz2WuEoua0"><strong>Podcast</strong></a> | <a href="https://differ.blog/inplainenglish"><strong>Differ</strong></a></li><li><a href="https://cofeed.app/"><strong>Check out CoFeed, the smart way to stay up-to-date with the latest in tech</strong></a> <strong>🧪</strong></li><li><a href="https://differ.blog/"><strong>Start your own free AI-powered blog on Differ</strong></a> 🚀</li><li><a href="https://discord.gg/in-plain-english-709094664682340443"><strong>Join our content creators community on Discord</strong></a> 🧑🏻‍💻</li><li>For more content, visit <a href="https://plainenglish.io/"><strong>plainenglish.io</strong></a> + <a href="https://stackademic.com/"><strong>stackademic.com</strong></a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f9c70f2d2f65" width="1" height="1" alt=""><hr><p><a href="https://blog.cubed.run/chain-of-draft-cod-f9c70f2d2f65">Chain of Draft (CoD)</a> was originally published in <a href="https://blog.cubed.run/">Cubed</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Anthropic’s Model Context Protocol (MCP)]]></title>
            <link>https://ai.plainenglish.io/anthropics-model-context-protocol-mcp-9e76a9fb9ff4?source=rss-ee89211c0d51------2</link>
            <guid isPermaLink="false">https://medium.com/p/9e76a9fb9ff4</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[generative-ai-tools]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[llm]]></category>
            <dc:creator><![CDATA[Sagar Patil]]></dc:creator>
            <pubDate>Sun, 08 Dec 2024 10:09:26 GMT</pubDate>
            <atom:updated>2025-04-22T09:33:50.836Z</atom:updated>
            <content:encoded><![CDATA[<p>Anthropic has unveiled the <strong>Model Context Protocol (MCP)</strong>, open-source standard that transforms how AI tools and chatbots interact with diverse data systems. By bridging the gap between AI systems and their data sources, MCP significantly enhances usability and response quality, paving the way for smarter and more efficient AI applications. It will also boost the security in agentic systems.</p><h4>What is MCP</h4><p>MCP is a <strong>way to connect AI systems/agents/assistants to a system where data is stored.</strong> It enables secure, controlled interactions between AI applications and local or remote resources. You can connect agents to data present in GitHub repo or databases etc.</p><p>MCP serves as a <strong>universal connector</strong>, linking AI systems with various data sources. It seamlessly integrates:</p><ul><li><strong>Local Data</strong>: Accessing databases, files, and services directly.</li><li><strong>Remote APIs</strong>: Connecting to platforms like Slack, GitHub, and more.</li></ul><h4>Why MCP Matters</h4><ul><li>AI systems often struggle when isolated from essential data sources or when constrained by rigid integration requirements. Also, giving direct access of data to AI agent is big security issue. MCP addresses this challenge by providing a flexible, universal framework that simplifies AI integration across systems, enhancing performance and security.</li><li>MCP eliminates the complexity of creating specialized setups for each new system. Acting as a “<strong>universal translator</strong>,” it enables AI tools to seamlessly interact with diverse environments, removing barriers to functionality and adaptability.</li></ul><blockquote>💡 For example, with MCP, developers can streamline tasks such as <a href="https://notify.cx/mcp">sending emails</a>, enabling efficient communication within AI workflows.</blockquote><ul><li>Robust access controls ensure secure integration with tools and data sources which important for <strong>Enterprise-Grade Security</strong></li></ul><h4>How it works</h4><ul><li>At its core, MCP represents an <strong>enhanced form of function calling</strong>, taking this concept to the next level.</li><li>With MCP, developers can expose a curated list of endpoints within an API that enable specific actions or data retrieval tasks. For example, a GitHub MCP server can interact with repositories, handling tasks such as committing changes or retrieving repository details. Similarly, a Google Drive MCP server can conduct searches within a drive or perform file management actions.</li><li>Anthropic has also integrated MCP into its <strong>Claude Desktop</strong> environment, allowing LLMs to query MCP servers whenever relevant. This setup empowers the LLM to fetch data or perform actions dynamically, enhancing its utility and efficiency in real-world applications.</li><li>At its core, MCP follows a client-server architecture where a host application can connect to multiple servers:</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/936/1*Sa0JkZPtlfD3oZyxprcV1A.png" /></figure><ul><li>MCP Hosts: Programs like Claude Desktop, IDEs, or AI tools that want to access resources through MCP</li><li>MCP Clients: Protocol clients that maintain 1:1 connections with servers</li><li>MCP Servers: Lightweight programs that each expose specific capabilities through the standardized Model Context Protocol</li><li>Local Resources: Your computer’s resources (databases, files, services) that MCP servers can securely access</li><li>Remote Resources: Resources available over the internet (e.g., through APIs) that MCP servers can connect to</li></ul><p>for example- MCP on SQLite Database</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/939/1*AmXUNW0-GUlG8XM395LGYg.png" /></figure><ol><li>Claude Desktop acts as our MCP client</li><li>A SQLite MCP Server provides secure database access</li><li>Your local SQLite database stores the actual data</li></ol><p>The communication between the SQLite MCP server and your local SQLite database happens entirely on your machine — your SQLite database is not exposed to the internet.</p><p>The Model Context Protocol ensures that Claude Desktop can only perform approved database operations through well-defined interfaces. This gives you a secure way to let Claude analyze and interact with your local data while maintaining complete control over what it can access.</p><h3>In Plain English 🚀</h3><p><em>Thank you for being a part of the </em><a href="https://plainenglish.io/"><strong><em>In Plain English</em></strong></a><em> community! Before you go:</em></p><ul><li>Be sure to <strong>clap</strong> and <strong>follow</strong> the writer ️👏<strong>️️</strong></li><li>Follow us: <a href="https://x.com/inPlainEngHQ"><strong>X</strong></a> | <a href="https://www.linkedin.com/company/inplainenglish/"><strong>LinkedIn</strong></a> | <a href="https://www.youtube.com/channel/UCtipWUghju290NWcn8jhyAw"><strong>YouTube</strong></a> | <a href="https://discord.gg/in-plain-english-709094664682340443"><strong>Discord</strong></a> | <a href="https://newsletter.plainenglish.io/"><strong>Newsletter</strong></a> | <a href="https://open.spotify.com/show/7qxylRWKhvZwMz2WuEoua0"><strong>Podcast</strong></a></li><li><a href="https://differ.blog/"><strong>Create a free AI-powered blog on Differ.</strong></a></li><li>More content at <a href="https://plainenglish.io/"><strong>PlainEnglish.io</strong></a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9e76a9fb9ff4" width="1" height="1" alt=""><hr><p><a href="https://ai.plainenglish.io/anthropics-model-context-protocol-mcp-9e76a9fb9ff4">Anthropic’s Model Context Protocol (MCP)</a> was originally published in <a href="https://ai.plainenglish.io/">Artificial Intelligence in Plain English</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Superposition Prompting for RAG systems]]></title>
            <link>https://medium.com/@sagarpatiler/superposition-prompting-for-rag-systems-74b7332a3005?source=rss-ee89211c0d51------2</link>
            <guid isPermaLink="false">https://medium.com/p/74b7332a3005</guid>
            <category><![CDATA[generative-ai-tools]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[deep-learning]]></category>
            <dc:creator><![CDATA[Sagar Patil]]></dc:creator>
            <pubDate>Wed, 25 Sep 2024 02:58:14 GMT</pubDate>
            <atom:updated>2024-09-25T02:58:14.788Z</atom:updated>
            <content:encoded><![CDATA[<p>Apple has introduced the Superposition Prompting framework for Retrieval-Augmented Generation (RAG) systems, delivering a remarkable 93x reduction in compute time and a 43% increase in accuracy compared to naive RAG prompt approach.</p><h4><strong>Naive RAG</strong></h4><p>In a naive RAG setup, the process starts when a user submits a query. The system first analyzes the query and searches a knowledge base to retrieve relevant documents by using vector similarity. Once these documents are retrieved, they are fed into the Large Language Model (LLM). The LLM then processes both the original query and the retrieved documents simultaneously and gives answer. However, traditional RAG has some drawbacks which impact an outcome from LLMs.</p><h4><strong>Drawbacks of Naive RAG system</strong></h4><p>While Retrieval-Augmented Generation (RAG) has proven to be a powerful approach for enhancing the capabilities of Large Language Models (LLMs), the traditional RAG prompting methods come with notable drawbacks that impact their performance, efficiency, and accuracy. Below are some of the key challenges associated with traditional RAG prompting:</p><p><strong>1. Processing Long Contexts: Speed and Cost Challenges</strong></p><p>In traditional RAG prompting, a large number of retrieved documents are bundled together with the system prompt and the user’s query. While this enriches the input context, LLMs face significant issues when processing long contexts. This is primarily due to the <strong>quadratic compute cost of self-attention</strong>, a core mechanism within LLMs. As the length of the input sequence increases, the inference cost scales quadratically, making the process computationally expensive, especially for deployment in real-world scenarios. Longer contexts also introduce added latency, further impacting the speed of generating responses.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/821/1*4lBV3kvS4QaEKAVRtDE3nA.png" /><figcaption>Theoretical Maximum Speedup vs. Accuracy (Best EM Subspan) on NaturalQuestions-Open using the mpt-7b instruct model (Muennighoff et al., 2023).</figcaption></figure><p><strong>2. Distraction Phenomenon and Accuracy Degradation</strong></p><p>Another major drawback of traditional RAG prompting is the “<strong>distraction phenomenon</strong>.” This occurs when irrelevant or redundant information is included in the retrieved documents that are fed into the LLM. LLMs struggle to differentiate between essential and non-essential content, leading to degradation in output quality. An irrelevant context can distract the model from focusing on the most relevant parts of the prompt, resulting in less accurate responses.</p><p><strong>3. Lost in the Middle Effect</strong></p><p>Recent research by Liu et al. (2023) has highlighted the “Lost in the Middle” effect, a common issue in RAG systems when retrieved documents are processed sequentially. LLMs often <strong>tend to ignore the context presented in the middle of long inputs</strong>. This can be problematic, especially when the crucial information or the <strong>“golden document”</strong> containing the answer is buried within the middle of a long sequence. The location of the key document significantly affects the model’s ability to provide accurate responses, further limiting the effectiveness of traditional RAG prompting approaches.</p><p><strong>4. Hallucinations and Divergent Responses</strong></p><p>Long input contexts in traditional RAG systems can also exacerbate the issue of hallucinations, where the LLM generates inaccurate or fictitious responses. When large volumes of retrieved documents are processed sequentially, the model may struggle to integrate the information correctly, leading to hallucinations or divergent responses. This unpredictability undermines the trustworthiness of RAG applications, particularly in domains where accuracy and reliability are critical, such as healthcare or legal fields.</p><h4>Superposition Prompting — How?</h4><p>In quantum mechanics, superposition means a particle can exist in multiple states at the same time until it is observed. Similarly, in Superposition Prompting, the LLM explores multiple potential responses based on different query-document pairs simultaneously. This method helps the model better capture the subtle details of the information found in the retrieved documents, improving its accuracy and response quality.</p><p>Superposition Prompting optimizes the traditional RAG process by treating different text segments separately, enabling faster and more accurate results.</p><blockquote>classical approach is a “linked-list” style DAG, superposition prompting arranges token dependencies such that all documents are processed independently. Due to this dependency structure, we can easily leverage the LLM logits to prune irrelevant context, improving long context reasoning. The dependency structure also allows for faster prompt processing, due to the new opportunities for caching and parallelism of the KV cache and logit computations. — Authors</blockquote><p>Let’s simplify this. Here’s a step-by-step breakdown of how this approach works:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*m6xD1CUO8XrsiwqeEt1gMg.png" /></figure><p><strong>1.Identify Text Segments</strong>: In the retrieval-augmented generation task, the input consists of three distinct parts:</p><ul><li>The <strong>preamble</strong> (also known as the system prompt).</li><li>A <strong>corpus of documents</strong> (static and pre-retrieved).</li><li>A <strong>query</strong> (dynamic, user-supplied). Traditionally, these segments would be concatenated into one long text sequence, but Superposition Prompting avoids this.</li></ul><p><strong>2. Grouping into Batches</strong>: Instead of concatenating, Superposition Prompting groups the preamble, documents, and different queries into separate “batches.” Each batch contains these components but avoids lumping them into a single sequence. These batches are processed individually by the LLM, using the previously stored key-value (KV) caches from earlier parts of the input. This avoids redundant re-computation of token interactions.</p><p><strong>3. Independent Processing of Document-Query Pairs</strong>: For each query, Superposition Prompting independently processes different document-query pairs while keeping the preamble in context. This step resembles a branching structure, where multiple document combinations are paired with the query and processed in parallel, each conditioned by the preamble.</p><p><strong>4.Path Pruning for Efficiency</strong>: Once the query and its paired documents are processed, the system employs <strong>path pruning</strong>. This means it discards unnecessary or irrelevant attention dependencies using an importance metric. Documents that are distracting or unrelated to the query are pruned away, improving both the accuracy and efficiency of the generation process.</p><p><strong>5. Token Sharing and Path Caching</strong>: The added structure of Superposition Prompting enables further optimizations. One such technique is <strong>token sharing across prompts</strong>, which allows for <strong>prompt path caching</strong>. This means that shared components of different prompts can be reused, avoiding redundant calculations and accelerating inference.</p><p><strong>6. Parallelization for Faster Processing</strong>: Finally, the approach introduces <strong>prompt path parallelization</strong>, which leverages the independence between segments (preamble, document, and query). This allows the system to process different paths concurrently, further speeding up the overall inference process.</p><p>In essence, Superposition Prompting improves the traditional RAG system by breaking down the task into more manageable, parallelizable parts. By pruning irrelevant information and reusing cached computations, it achieves a significant boost in both speed and accuracy.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=74b7332a3005" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Evaluating RAG systems using RAGAS framework]]></title>
            <link>https://ai.plainenglish.io/evaluating-rag-systems-using-ragas-framework-d4b8df74d027?source=rss-ee89211c0d51------2</link>
            <guid isPermaLink="false">https://medium.com/p/d4b8df74d027</guid>
            <category><![CDATA[generative-ai-tools]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Sagar Patil]]></dc:creator>
            <pubDate>Fri, 23 Aug 2024 17:00:09 GMT</pubDate>
            <atom:updated>2024-08-26T08:10:40.813Z</atom:updated>
            <content:encoded><![CDATA[<p>Retrieval-Augmented Generation (RAG) has become a critical technique in the landscape of large language models (LLMs), combining the strengths of both retrieval and generation to provide more accurate and contextually relevant responses. Any RAG implementation comprises two essential aspects: <strong>Retrieval</strong> and <strong>Generation</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*oKrOMcOuulijXB-gj1VArg.png" /></figure><blockquote>The retrieval process is responsible for establishing the context by pulling in relevant information from an external database, while the generation process is where the LLM steps in to produce an answer based on the retrieved content.</blockquote><p>When evaluating a RAG pipeline, it is crucial to assess both these components independently and in tandem. This dual evaluation helps in pinpointing specific areas that require improvement, offering a clearer understanding of the overall performance of the RAG model.</p><p>To measure the effectiveness of your RAG application, quantitative evaluation is necessary. This evaluation involves two primary ingredients: an <strong>evaluation metric</strong> and an <strong>evaluation dataset</strong>.</p><p>The metric provides a means to quantify the performance, while the dataset serves as the benchmark against which the RAG pipeline is assessed. Both these needs are incorporated by RAGAS. This article delves into the methodology of evaluating a RAG pipeline, with a particular focus on the RAGAS framework — a structured approach to systematically measure and enhance the effectiveness of RAG implementations.</p><h4>What is RAGAS?</h4><ul><li><strong>RAGAS (RAG Assessment)</strong> is a framework designed to evaluate Retrieval-Augmented Generation (RAG) pipelines. RAG refers to a category of LLM applications that enhance the model’s context using external data sources.</li><li>While various tools and frameworks exist to help build these pipelines, assessing and quantifying their performance can be challenging — this is where RAGAS proves invaluable.</li></ul><p><strong>RAGAS</strong> offers tools rooted in the latest research for evaluating LLM-generated content, providing valuable insights into your RAG pipeline’s effectiveness. Moreover, RAGAS can be seamlessly integrated into your CI/CD pipeline, enabling continuous performance monitoring and ensuring that your RAG applications maintain optimal quality.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*89kDiIIVHrGKlg-i2z_qeA.png" /></figure><h4>A. Evaluation Metrics</h4><p>RAGAS offers a set of metrics to evaluate a RAG pipeline <strong>both component-wise</strong> and <strong>end-to-end evaluation. RAGAS</strong> started out as a framework for “reference-free” evaluation. That means, instead of having to rely on human-annotated ground truth labels in the evaluation dataset, RAGAS leverages LLMs under the hood to conduct the evaluations. While using as little human-annotated data as possible makes it a cheaper and faster evaluation method</p><p><strong>a. Component-Level Evaluation</strong>:</p><p><strong>Retrieval Component</strong>:</p><p><strong>Context Precision</strong>:</p><ul><li>Measures the signal-to-noise ratio of the retrieved context, calculated using the question and the retrieved contexts.</li><li>It focuses on measuring how accurately the retrieved context aligns with the information needed to answer a question, providing a numerical representation of this precision. ( Plaban Nayak)</li><li>Helps to optimize chunk size.</li></ul><p><strong>Context Recall</strong>:</p><ul><li>Assesses whether all relevant information required to answer the question was retrieved, using ground truth labels and contexts. This is the only metric in the framework that relies on human-annotated ground truth.</li></ul><p><strong>Generative Component</strong>:</p><p><strong>Faithfulness</strong>:</p><ul><li>Evaluates the factual accuracy of the generated answer by dividing the number of correct statements by the total number of statements in the answer. This metric uses the question, contexts, and the generated answer.</li></ul><p><strong>Answer Relevancy</strong>:</p><ul><li>Measures how relevant is the generated answer to the question. For example, if the question is “Where is France and what is its capital?” and the answer is “France is in western Europe,” the answer relevancy would be low because only part of the question is answered.</li></ul><p>All metrics are scaled between [0, 1], with higher values indicating better performance.</p><h4><strong>b . End-to-End Evaluation</strong>:</h4><ul><li>The harmonic mean of these 4 aspects gives you the <strong>Ragas score</strong> which is a single measure of the performance of your QA system across all the important aspects.</li><li>RAGAS also provides metrics like <strong>answer semantic similarity</strong> and <strong>answer correctness</strong> to assess the pipeline as a whole.</li></ul><h3>RAGAS dataset:</h3><p>Ragas references the following data.</p><ul><li><strong>Question</strong>: (User query/Input) <em>These are the questions your RAG pipeline will be evaluated on.</em></li><li><strong>Answer</strong>: (Output) The<em> answer generated from the RAG pipeline and presented to the user.</em></li><li><strong>Contexts</strong>: (Retrieved Chunks) <em>The contexts </em>retrieved from the external knowledge source and <em>passed into the LLM to answer the question.</em></li><li><strong>Ground Truths</strong>: <em>The answers to the questions considered to be truth. </em>This is the only human-annotated information.</li></ul><h4>Implementation</h4><p>You need to create dataset in following manner</p><pre>data = {<br> &quot;question&quot;: questions,<br> &quot;answer&quot;: answers,<br> &quot;contexts&quot;: contexts,<br> &quot;ground_truths&quot;: ground_truths<br>}<br># Convert dict to dataset<br>dataset = Dataset.from_dict(data)</pre><p>And then by using evaluate () we can get metrics</p><pre>from ragas import evaluate<br>from ragas.metrics import (<br> faithfulness,<br> answer_relevancy,<br> context_recall,<br> context_precision,<br>)<br>result = evaluate(<br> dataset = dataset, <br> metrics=[<br> context_precision,<br> context_recall,<br> faithfulness,<br> answer_relevancy,<br> ],<br>)<br>df = result.to_pandas()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*XV-YYCRwAyCSC0Wz.png" /><figcaption>RAGAS dataset and metrics</figcaption></figure><p><strong>Conclusion:</strong><br>Thus, implementing the RAGAS evaluation framework helps in objectively assessing the performance of retrieval-augmented generation (RAG) pipelines. By providing a structured approach to evaluation, RAGAS helps organizations reduce costs, ensure quality before deployment, and safeguard the integrity of their AI systems. This ensures that RAG systems are both effective and reliable, driving better outcomes in real-world applications.</p><h3>In Plain English 🚀</h3><p><em>Thank you for being a part of the </em><a href="https://plainenglish.io/"><strong><em>In Plain English</em></strong></a><em> community! Before you go:</em></p><ul><li>Be sure to <strong>clap</strong> and <strong>follow</strong> the writer ️👏<strong>️️</strong></li><li>Follow us: <a href="https://twitter.com/inPlainEngHQ"><strong>X</strong></a> | <a href="https://www.linkedin.com/company/inplainenglish/"><strong>LinkedIn</strong></a> | <a href="https://www.youtube.com/channel/UCtipWUghju290NWcn8jhyAw"><strong>YouTube</strong></a> | <a href="https://discord.gg/in-plain-english-709094664682340443"><strong>Discord</strong></a> | <a href="https://newsletter.plainenglish.io/"><strong>Newsletter</strong></a></li><li>Visit our other platforms: <a href="https://cofeed.app/"><strong>CoFeed</strong></a> | <a href="https://differ.blog/"><strong>Differ</strong></a></li><li>More content at <a href="https://plainenglish.io/"><strong>PlainEnglish.io</strong></a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d4b8df74d027" width="1" height="1" alt=""><hr><p><a href="https://ai.plainenglish.io/evaluating-rag-systems-using-ragas-framework-d4b8df74d027">Evaluating RAG systems using RAGAS framework</a> was originally published in <a href="https://ai.plainenglish.io/">Artificial Intelligence in Plain English</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Prompt Caching in Generative AI applications]]></title>
            <link>https://ai.plainenglish.io/prompt-caching-in-generative-ai-applications-3b9b6cf2b334?source=rss-ee89211c0d51------2</link>
            <guid isPermaLink="false">https://medium.com/p/3b9b6cf2b334</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[generative-ai-tools]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[ai]]></category>
            <dc:creator><![CDATA[Sagar Patil]]></dc:creator>
            <pubDate>Mon, 19 Aug 2024 06:05:16 GMT</pubDate>
            <atom:updated>2024-08-19T17:45:57.148Z</atom:updated>
            <content:encoded><![CDATA[<h3>What is Prompt caching in LLMs?</h3><ul><li>Prompt caching enables developers to cache frequently used context between API calls. With prompt caching, users can provide LLM with more background knowledge and example outputs one time and use that in subsequent calls.</li><li>Many prompts, especially in enterprise grade applications, contain a long recurring pattern or shared elements. By leveraging this commonality, prompt caching aims<strong> to eliminate redundant computations</strong> and<strong> significantly speed up the inference process.</strong></li></ul><blockquote>Prompt caching is an innovative technique designed to optimize the inference process of Large Language Models by strategically storing and reusing precomputed attention states.</blockquote><ul><li>When an LLM processes a prompt, it generates internal representations called <strong>attention states</strong>, which capture the model’s understanding of the relationships between different parts of the input text. Traditionally, these attention states are computed anew for each inference request, even if parts of the prompt have been processed before. That was time taking and used to cost more. This problem is solved by prompt caching.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/0*cIh65Zydr7-_cbH7" /><figcaption>credit- Paper (Prompt Cache: Modular Attention Reuse for Low-Latency Inference)</figcaption></figure><h3>Benefits of using Prompt Caching —</h3><ol><li><strong>Performance Optimization:</strong> Caching reduces the need for repeated computations in LLMs, leading to faster response times and more efficient resource use and cost saving.</li></ol><ul><li><strong>Cost Reduction:</strong> By minimizing redundant computations, prompt caching lowers the high operational costs of running LLMs in production.</li><li><strong>Latency Reduction:</strong> Caching decreases the time needed to deliver responses, critical for real-time applications like chatbots.</li></ul><p>For example-</p><blockquote>For example, as claimed by Anthropic for Claude 3.5 Sonnet and Claude 3 Haiku, in a use case like “Chat with a book” where a 100,000-token prompt is cached, the latency without caching (time to first token) was 11.5 seconds, but with caching,<strong><em> it reduced to 2.4 seconds (-79%)</em></strong>, resulting in a <strong><em>90% cost reduction</em></strong>.</blockquote><p><strong>2. Improved User Experience:</strong> It ensures consistent and quick responses for repeated or similar queries, enhancing user satisfaction.</p><p><strong>3. Scalability:</strong> As user numbers grow, caching helps scale the system efficiently without increasing the computational load.</p><p><strong>4. Consistency:</strong> It guarantees that identical prompts produce consistent results, crucial for applications like customer support.</p><h3>How Prompt Caching works —</h3><p>When you send a request with Prompt Caching enabled:</p><ol><li>The system checks if the prompt prefix is already cached from a recent query.</li><li>If found, it uses the cached version, reducing processing time and costs.</li><li>Otherwise, it processes the full prompt and caches the prefix for future use.</li></ol><h3>When to use prompt caching —</h3><p>Prompt caching can be effective in situations where you want <strong><em>to send a large amount of prompt context once and then refer to that information repeatedly in subsequent requests</em></strong>, including:</p><ul><li><strong>Conversational agents:</strong></li></ul><p>Reduce cost and latency for extended conversations, especially those with long instructions or uploaded documents.</p><ul><li><strong>Coding assistants:</strong></li></ul><p>Improve autocomplete and codebase Q&amp;A by keeping a summarized version of the codebase in the prompt.</p><ul><li><strong>Large document processing:</strong></li></ul><p>Incorporate complete long-form material including images in your prompt without increasing response latency.</p><ul><li><strong>Detailed instruction sets:</strong></li></ul><p>Share extensive lists of instructions, procedures, and examples to fine-tune Claude’s responses. Developers often include a few examples in their prompt, but with prompt caching you can get even better performance by including dozens of diverse examples of high-quality outputs.</p><ul><li><strong>Agentic search and tool use:</strong></li></ul><p>Enhance performance for scenarios involving multiple rounds of tool calls and iterative changes, where each step typically requires a new API call.</p><ul><li><strong>Talk to books, papers, documentation, podcast transcripts, and other long-form content:</strong></li></ul><p>Bring any knowledge base alive by embedding the entire document(s) into the prompt, and letting users ask it questions.</p><p>for example -</p><ul><li>In legal domain- large volumes of case law, statutes, and legal documents</li><li>In financial domain-company reports, historical market data and financial regulations and compliances</li><li>In medical domain- patient histories and conversations, medical literature, and treatment protocols, Regulations</li><li>In big companies-product catalogs, company policies, and customer histories and feedback</li></ul><h3>prompt caching in Anthropic’s Claude (announced on 14 august) —</h3><p>Prompt caching is available today in public beta for Claude 3.5 Sonnet and Claude 3 Haiku, with support for Claude 3 Opus coming soon.</p><h4>Overall Performance Analysis —</h4><p>Prompt caching will reduce costs by up to 90% and latency by up to 85% for long prompts.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Z_MwcQ2wE3BO5DXxmd1j5Q.png" /><figcaption>credit-Anthropic</figcaption></figure><h4>Cost Analysis —</h4><p>for Sonnet model which more accurate but little slow than haiku -</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*u48yJX6xCXloyxYpzu9cbg.png" /></figure><p>For Haiku -</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*c9lC2Xi7TzJy9-zfe8FYlg.png" /></figure><p>In summary, Cached prompts are <strong><em>priced based on the number of input tokens</em></strong> you cache and how frequently you use that content. Writing to the cache <strong><em>costs 25% more than our base input token price</em> </strong>for any given model, while using cached content is significantly<strong><em> cheaper, costing only 10% of the base input token price</em></strong>.</p><p>i.e</p><ul><li>Cache write tokens are 25% more expensive than base input tokens</li><li>Cache read tokens are 90% cheaper than base input tokens</li><li>Regular input and output tokens are priced at standard rates.</li></ul><h4>Implementation —</h4><p>Just by using <strong>cache control block</strong> as following:</p><pre>import anthropic<br><br>client = anthropic.Anthropic()<br><br>response = client.beta.prompt_caching.messages.create(<br>    model=&quot;claude-3-5-sonnet-20240620&quot;,<br>    max_tokens=1024,<br>    system=[<br>      {<br>        &quot;type&quot;: &quot;text&quot;, <br>        &quot;text&quot;: &quot;You are an AI assistant tasked with analyzing literary works. Your goal is to provide insightful commentary on themes, characters, and writing style.\n&quot;,<br>      },<br>      {<br>        &quot;type&quot;: &quot;text&quot;, <br>        &quot;text&quot;: &quot;&lt;the entire contents of &#39;Pride and Prejudice&#39;&gt;&quot;,<br>        &quot;cache_control&quot;: {&quot;type&quot;: &quot;ephemeral&quot;}<br>      }<br>    ],<br>    messages=[{&quot;role&quot;: &quot;user&quot;, &quot;content&quot;: &quot;Analyze the major themes in &#39;Pride and Prejudice&#39;.&quot;}],<br>)<br>print(response)</pre><h3>Comparison of Anthropic’s Claude prompt caching vs Gemini context caching —</h3><p>Following breakdown allows you to see the subtle distinctions between the two caching solutions, helping to determine which might be more appropriate for specific use cases.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/843/1*KXIZnS85vs9OFFCqjJ4jhg.png" /><figcaption>credits- Mervin Praison</figcaption></figure><p>According to <strong>Sriram Parthasarathy,</strong> Claude and Gemini offer distinct approaches to cache lifetime and cost models, each tailored to different usage patterns. <strong><em>Claude employs a dynamic caching system</em></strong> where the cache lasts for<strong> 5 minutes but refreshes with each use</strong>, making it ideal for scenarios with frequent queries. For a 100,000-word manual (approximately 133,333 tokens), Claude initially caches at a cost of $0.50, with subsequent uses costing $0.04 each. This translates to rates of $3.75 per million tokens for initial caching and $0.30 per million tokens for each use.</p><p>In contrast, <strong>Gemini operates on an hourly charging model</strong>, providing two options: Pro at $0.60 per hour ($4.50 per million tokens per hour) and Flash at $0.13 per hour ($1 per million tokens per hour). Gemini’s cache remains available for the full hour, regardless of usage frequency, making it well-suited for longer-term, less frequent access needs.</p><p>Claude’s model can potentially save up to 90% on input costs for frequently accessed data, while Gemini offers a consistent 25% discount on input tokens.</p><p>Thus, according to him, the optimal choice depends on your specific use case:</p><ul><li>For applications with burst-like, <strong>frequent queries, Claude’s dynamic caching</strong> may be more cost-effective.</li><li>For consistent,<strong> longer-term caching needs, Gemini’s hourly model</strong> could provide better value.</li></ul><h3>Conclusion</h3><p>Prompt caching is poised to be a game-changing feature in the rapidly evolving field of generative AI. By reducing both operational costs and latency, this technology enhances the efficiency and effectiveness of AI applications. Despite being in its early stages, prompt caching has already demonstrated its potential to revolutionize how we interact with AI systems. As the technology advances, we can anticipate the development of even more sophisticated caching strategies, further transforming the landscape of generative AI.</p><p>In future articles, I plan to delve deeper into the topic of prompt caching, if time allows.</p><h3>In Plain English 🚀</h3><p><em>Thank you for being a part of the </em><a href="https://plainenglish.io/"><strong><em>In Plain English</em></strong></a><em> community! Before you go:</em></p><ul><li>Be sure to <strong>clap</strong> and <strong>follow</strong> the writer ️👏<strong>️️</strong></li><li>Follow us: <a href="https://twitter.com/inPlainEngHQ"><strong>X</strong></a> | <a href="https://www.linkedin.com/company/inplainenglish/"><strong>LinkedIn</strong></a> | <a href="https://www.youtube.com/channel/UCtipWUghju290NWcn8jhyAw"><strong>YouTube</strong></a> | <a href="https://discord.gg/in-plain-english-709094664682340443"><strong>Discord</strong></a> | <a href="https://newsletter.plainenglish.io/"><strong>Newsletter</strong></a></li><li>Visit our other platforms: <a href="https://cofeed.app/"><strong>CoFeed</strong></a> | <a href="https://differ.blog/"><strong>Differ</strong></a></li><li>More content at <a href="https://plainenglish.io/"><strong>PlainEnglish.io</strong></a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3b9b6cf2b334" width="1" height="1" alt=""><hr><p><a href="https://ai.plainenglish.io/prompt-caching-in-generative-ai-applications-3b9b6cf2b334">Prompt Caching in Generative AI applications</a> was originally published in <a href="https://ai.plainenglish.io/">Artificial Intelligence in Plain English</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Federated Learning]]></title>
            <link>https://blog.cubed.run/federated-learning-8173ad1481ac?source=rss-ee89211c0d51------2</link>
            <guid isPermaLink="false">https://medium.com/p/8173ad1481ac</guid>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Sagar Patil]]></dc:creator>
            <pubDate>Sun, 11 Aug 2024 11:58:22 GMT</pubDate>
            <atom:updated>2024-08-16T08:13:01.415Z</atom:updated>
            <content:encoded><![CDATA[<blockquote>Federated learning is a way to develop and validate AI models from diverse data sources while mitigating the risk of compromising data security or privacy, as the data never leaves individual sites.</blockquote><h4>What is Federated Learning?</h4><ul><li>Federated learning is a way to train a machine learning model without needing to collect all the data in one place. Instead of sending all your data to a central server (one place), your devices (like your smartphones, hospital labs) trains the model using its own data and then sends only the updates (not the data itself) back to a central place where all the updates from different devices are combined.</li><li>This stands in contrast to machine learning settings in which data is centrally stored. One of the primary characteristics of federated learning is data heterogeneity.</li><li>The general principle consists in <strong>training local models on local data samples and exchanging parameters </strong>(e.g. the weights and biases of a deep neural network) between these local nodes at some frequency to generate a global model shared by all nodes.</li><li>Data is naturally distributed. It is distributed across organizations (hospitals, governments, finance , manufacturing ) and across devices (phones, laptops, automobiles, IoT devices). Also, collection of data involves many issues such as data volume, data sensitivity, data regulations, practicality)</li><li>In the words of Andrew Ng, federated learning is <strong>taking training to the data.</strong></li><li>Frameworks like <strong>Flower, Nvidia Flare</strong> have made federated learning simple.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*fUCQW3xCrzdgl0pZ.png" /><figcaption>credit Wikipedia</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*JGMaNZrGtHCFUBOmH7sLlQ.png" /><figcaption>credit Nvidia</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/879/1*NNVAwdYnSgjUqzh7Y8VI4Q.png" /><figcaption>credits- Federated Learning of Gboard Language Models with Differential Privacy (xu,zhang)</figcaption></figure><h4>Why federated learning?</h4><p><strong>Privacy and Security</strong></p><ul><li><strong>Data Stays Local</strong>: Federated learning allows data to remain on local devices such as smartphones or IoT devices or local data centers, rather than being sent to a central server. This <strong>reduces the risk of data breaches</strong> and protects sensitive information.</li><li><strong>Compliance with Regulations</strong>: By keeping data on local environment where it was generated, federated learning can help organizations comply with strict data protection regulations like <strong>GDPR, HIPAA</strong>, and others.</li><li><strong>Minimized Risk</strong>: Since raw data isn’t transmitted across networks to a central server, federated learning reduces the attack surface for potential hackers. This makes it more secure, as only model updates are shared.</li></ul><p><strong>Reduced Communication Costs</strong></p><ul><li><strong>Efficient Data Transfer</strong>: Instead of sending entire datasets to a central server, federated learning only transmits model updates (like weights and gradients). This significantly <strong>reduces the amount of data that needs to be communicated, saving bandwidth and reducing costs.</strong></li><li><strong>Suitable for Low-Bandwidth Environments</strong>: Federated learning is ideal for environments with limited or expensive connectivity, as it minimizes the amount of data that needs to be transferred.</li></ul><p><strong>Ethical and Responsible AI</strong></p><ul><li><strong>Respecting User Privacy</strong>: Federated learning aligns with the principles of ethical AI by prioritizing user privacy and security. It allows organizations to develop AI models that are both powerful and respectful of individual rights.</li><li><strong>Transparency and Trust</strong>: By not requiring users to share their raw data, federated learning can increase trust in AI systems, making users more comfortable with the technology.</li></ul><p><strong>Scalability</strong></p><ul><li><strong>Distributed Computing Power</strong>: Federated learning leverages the computational resources of many devices, allowing for the training of large-scale models without requiring centralized high-performance computing resources.</li><li><strong>Handling Large-Scale Data</strong>: By processing data on local devices, federated learning can efficiently manage and analyze data from millions of devices without the need to centralize it.</li></ul><p><strong>Personalization</strong></p><ul><li><strong>Localized Model Training</strong>: Federated learning allows for the creation of models that can be personalized to individual users based on the data on their own devices. This can improve the relevance and accuracy of AI applications like predictive text or personalized recommendations without compromising privacy.</li><li><strong>Context-Aware Models</strong>: Since training happens on local devices, models can be more aware of the specific context in which they will be used, leading to more accurate and tailored predictions.</li></ul><p><strong>Cross-Device Learning</strong></p><ul><li><strong>Collaborative Model Improvement</strong>: Federated learning allows models to be trained collaboratively across many devices, which can lead to better generalization and robustness of the model.</li><li><strong>Consistency Across Devices</strong>: It enables consistent performance and improvements across different devices, as all devices contribute to the model’s training.</li></ul><h4>Difference between traditional and federated learning :</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/826/1*R-50-lnJ0QKclWr6OqUuYQ.png" /></figure><h4>How does federated learning work?</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*JlIP0l3odOPuTY6ICMLARw.png" /></figure><p><strong>1. Initialization:</strong></p><ul><li>The process begins with the server setting up a global model, which is like a starting point or a basic version of the model that needs to be improved.</li></ul><p><strong>2. Communication Round:</strong></p><ul><li>The server sends this global model to all the participating devices (clients), like your smartphone or computer.</li><li>Each device receives this model and prepares to work on it.</li></ul><p><strong>3. Client Training and Model Update:</strong></p><ul><li>Each device uses its own data to train the model. For example, your phone might use your text messages to improve a predictive text model.</li><li>After training the model locally, each device sends its updated model back to the server. The actual data never leaves the device, only the model updates.</li></ul><p><strong>4. Model Aggregation:</strong></p><ul><li>The server collects all the updated models from different devices.</li><li>It combines these updates using a special method (like averaging) to create a new, improved global model.</li></ul><p><strong>5. Convergence Check:</strong></p><ul><li>The server checks if the model is good enough (has “converged”). If it is, the process stops.</li><li>If not, the server repeats the process by sending the updated global model back to the devices for further improvement.</li></ul><p>In summary, federated learning is particularly useful when privacy, security, scalability, and efficient communication are critical. It allows for powerful, personalized, and ethical AI models that can be trained across distributed data sources without compromising user privacy. In the next article, I will try to explore the relevance of federated learning in generative ai era.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8173ad1481ac" width="1" height="1" alt=""><hr><p><a href="https://blog.cubed.run/federated-learning-8173ad1481ac">Federated Learning</a> was originally published in <a href="https://blog.cubed.run/">Cubed</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>