<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Anil on Medium]]></title>
        <description><![CDATA[Stories by Anil on Medium]]></description>
        <link>https://medium.com/@infinityin-minds?source=rss-ecb47cee9377------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*JJy_H_st8rvWcJHjuuUCTQ.jpeg</url>
            <title>Stories by Anil on Medium</title>
            <link>https://medium.com/@infinityin-minds?source=rss-ecb47cee9377------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Tue, 17 Nov 2026 21:14:52 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@infinityin-minds/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Harness Engineering: The Harness Around the Brain]]></title>
            <link>https://infinityin-minds.medium.com/harness-engineering-the-harness-around-the-brain-fc7d91737e64?source=rss-ecb47cee9377------2</link>
            <guid isPermaLink="false">https://medium.com/p/fc7d91737e64</guid>
            <category><![CDATA[harness-engineering]]></category>
            <category><![CDATA[ai-engineering]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[agent-harness]]></category>
            <dc:creator><![CDATA[Anil]]></dc:creator>
            <pubDate>Mon, 21 Sep 2026 07:26:01 GMT</pubDate>
            <atom:updated>2026-09-21T07:26:01.310Z</atom:updated>
            <content:encoded><![CDATA[<h3>Demystifying the Illusion</h3><p>We just covered the loop — the while-loop that makes an LLM behave like an agent in one of the previous articles. But the loop is only the skeleton. A loop by itself, run naked against a raw API, is a liability. It hallucinates tool names, leaks context, crashes on bad JSON, and burns your budget on a prompt you can&#39;t see.</p><p>So what wraps the loop to make it survivable?<em> </em><strong><em>The harness</em></strong><em>…..</em></p><p><strong>Think like this</strong>: the LLM is a brilliant, amnesiac intern with no hands. The loop is the job description (<strong>keep going until you’re done</strong>). The harness is the entire workstation — the tools on the desk, the permission system on the file cabinet, the notes pinned to the wall, the manager who steps in when the intern starts doing something dumb, and the timesheet that stops the shift from running forever.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*SpVYGsvihBbafX770KQZww.png" /></figure><p>The harness is <strong><em>all the code that isn’t the LLM</em></strong>. And here’s the uncomfortable truth: the quality of your agent is determined almost entirely by the harness, not the model. Two teams using the same GPT-4.5-class model can build systems that are night and day different, purely because of how they built the harness around it.</p><p>The brain (loop) decides <em>what to do next</em>. The harness decides <em>what it’s allowed to do, what it can see, and when it gets shut down</em>.</p><h3>The Pillars:</h3><p>A production harness is five subsystems bolted around the loop.</p><ol><li><strong>Tool registry &amp; invocation -</strong> A typed catalog of every action the model can take — each with a name, a JSON schema, a permission level, and a real executor. The harness is what turns the model’s “call search_db&quot; intent into an actual, sandboxed, permission-checked function call. If the model invents search_databse, the registry rejects it <em>before</em> anything runs.</li><li><strong>Context manager -</strong> This is the RAM from the last article, but now it’s a real component with logic. It decides what goes into the prompt each iteration, what gets summarized, what gets externalized to a cache. It’s the difference between a brain that forgets and a brain with a filing system.</li><li><strong>State store -</strong> The loop is stateless; the harness isn’t. It persists the conversation, the tool-call log, and the current step to a database so the agent can be paused, resumed, replayed, or audited. If your process dies on iteration 7, a good harness restarts at 7, not at 1.</li><li><strong>Guardrails &amp; validators -</strong> Input and output filters that run <em>around</em> every model call. Schema validation on tool args, PII redaction on the way in, a sanity check on the way out. This is where you stop the model from DROP TABLEing your production database because it misread a prompt.</li><li><strong>Evaluators &amp; termination -</strong> The circuit breakers, plus a scoring layer. Is the model actually making progress, or spinning? Did the final answer pass a check? The harness watches the loop the way a supervisor watches a junior dev — and pulls the plug when the work stops being real.</li></ol><pre># The harness wrapping a single loop iteration<br>def step(state):<br>    prompt   = ctx_manager.build(state)          # 2. context<br>    resp     = llm.call(prompt, registry.tools)  # 1. tools exposed<br>    resp     = validator.check_output(resp)      # 4. guardrail<br>    if resp.tool_call:<br>        result = registry.execute(resp.tool_call) # 1. sandboxed run<br>        result = guardrail.sanitize(result)       # 4. redact/shape<br>    state_store.save(state, result)               # 3. persist<br>    if evaluator.should_stop(state):              # 5. termination<br>        return finish(state)<br>    return step(state)</pre><blockquote>Same loop as before. Now it’s wearing armor.</blockquote><h3>Where the Harness Breaks</h3><p>This is where teams release an agent that functions in the demo but fails in production. The tool surface is too broad. Every registered tool adds a potential error for the model. Fifty tools create fifty chances for a bad call. The solution is not smarter prompting — it is narrowing the surface. Show only tools relevant to the current step. A research step does not require the write-to-production tool.</p><p>Guardrails become the bottleneck. Every validator is a network hop or a CPU cycle. Layer twelve of them and your “fast” agent takes forty seconds per turn. You have to triage: which checks are safety-critical (keep) and which are nice-to-have (drop or batch)?</p><p>State and context drift apart. The state store says the agent is on step 5. The context window is stuffed with step-3 noise the model no longer needs. When the two disagree, the agent makes decisions on stale evidence. You need the context manager and the state store to be <em>two views of the same source of truth</em>, not two separate logs.</p><p>The harness is where the security lives — and where it gets skipped. The model is not a trusted component. It reads untrusted input (user prompts, web pages, file contents) and it <em>will</em> be nudged. The harness is the only thing standing between a prompt injection and a real side effect. If your tool executor runs with the same credentials as the rest of your system, one injected instruction and an attacker is reading your database.</p><h3>Why You Build One</h3><p>You don’t build a harness for every LLM call. You build it when the model is doing something you can’t afford to get wrong. The trigger list is short:</p><ul><li><strong>System Interaction Risks:</strong> The model interacts with real systems such as databases, APIs, file writes, email, and financial operations; hallucinated calls may mutate state, requiring a registry, sandbox, and guardrails.</li><li><strong>Scale and Control:</strong> Loops exceeding two iterations and systems reaching around twenty calls demand state management, termination, and observability.</li><li><strong>Boundary Protection:</strong> Untrusted inputs like user prompts, scraped content, files, and emails may carry prompt injection; the harness is the sole barrier to side effects.</li><li><strong>Reproducibility and Auditing:</strong> A state store is needed to explain actions such as email sending, supporting regulators, incident reviews, and future analysis.</li><li><strong>Efficiency at Scale:</strong> A harness that trims context, parallelizes tool calls, and terminates early helps manage cost and latency.</li></ul><p>If two or more of those apply, the harness isn’t optional. It’s the difference between a demo and a product.</p><h3>When to Skip It</h3><p>The other direction matters just as much. A harness on a task that doesn’t need one is overengineering, and it adds latency, code, and failure surface for zero return.</p><ul><li><strong>Simple Implementation:</strong> Use single-shot, no-tool approaches and basic if/else logic instead of complex setups when the solution is deterministic.</li><li><strong>Avoid Overengineering:</strong> Prefer a short function over a large harness; use regex instead of looping a probabilistic model for simple tasks.</li><li><strong>Early Exploration:</strong> Build loops first during initial project phases; add harnesses only after task shape and tool relevance are clear.</li><li><strong>Prevent Premature Locking:</strong> Avoid early harnessing because it restricts flexibility before learning occurs.</li><li><strong>Low Cost Justification:</strong> Harnesses incur engineering time and runtime overhead that may not be justified for low-volume, simple tasks.</li></ul><p>The rule of thumb: build the loop first, add harness components one at a time as a specific failure or requirement forces you to. A harness that grows from actual pain beats a harness designed up front for pain that never arrived.</p><h3>The Cost of the Harness</h3><p>Counterintuitively, a <em>better</em> harness is often <em>cheaper</em> at runtime. A tight tool surface means fewer, smaller prompts. Good context management means you don’t re-send 80K tokens every turn. Proper termination means you stop at step 6 instead of step 14. The harness is where you claw back the cost the raw loop wastes.</p><p>The engineering cost is real, though. A good harness is 5–10× the code of the loop itself. Teams under budget this and then blame the model.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=fc7d91737e64" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Loop Engineering: The Brain Behind the Agent]]></title>
            <link>https://infinityin-minds.medium.com/loop-engineering-the-brain-behind-the-agent-4b5b5d77fbdb?source=rss-ecb47cee9377------2</link>
            <guid isPermaLink="false">https://medium.com/p/4b5b5d77fbdb</guid>
            <category><![CDATA[ai-agents-in-action]]></category>
            <category><![CDATA[llm-agent]]></category>
            <category><![CDATA[loop-engineering]]></category>
            <category><![CDATA[ai-engineering]]></category>
            <dc:creator><![CDATA[Anil]]></dc:creator>
            <pubDate>Sat, 19 Sep 2026 15:36:08 GMT</pubDate>
            <atom:updated>2026-09-19T15:36:08.528Z</atom:updated>
            <content:encoded><![CDATA[<h4><strong>Demystifying the Illusion</strong></h4><p>Everyone talks about <em>AI agents</em> like they’re little digital minds solving problems. They’re not. An LLM is a stateless text predictor. You hand it a string of tokens, it spits back a probability distribution over the next one. No memory between calls. <em>No intent. No thinking.</em></p><p>So where’s the brain? <strong><em>It’s the loop you build around it.</em></strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*IpZaFgfYb0oiqPgfcezmKw.png" /></figure><p>Strip away every framework — LangGraph, CrewAI, OpenAI Assistants, whatever — and you get this:</p><pre>state = init(goal, system_prompt, tools)<br>while not done:<br> prompt = build_context(state)<br> resp = llm.call(prompt, tools)<br>if resp.tool_call:<br> state.append(execute(resp.tool_call))<br> elif resp.final_answer:<br> break<br>state.iteration += 1</pre><blockquote>That’s the entire <strong>Brain</strong>. A while-loop with a probabilistic decision function in the middle. The LLM is one node. The tools are the hands. The context window is working memory. And you — the engineer — are the one deciding when the brain stops thinking.</blockquote><h3>The Loop, Step by Step</h3><p>Here’s what actually happens in one pass:</p><ol><li><strong>Build the prompt -</strong> You assemble the system prompt, tool schemas, conversation history, and the user’s goal into one big string. Every iteration you <em>rebuild</em> it, because you get to choose what the model sees this time.</li><li><strong>Call the model -</strong> It reads the prompt and outputs either a tool call (structured JSON: “run this query”) or a final answer (“here’s the result, I’m done”).</li><li><strong>Execute or stop -</strong> Tool call? Run it — database query, API hit, file read. Final answer? The loop ends.</li><li><strong>Inject the result back -</strong> The tool output gets appended to the message history. Next iteration, the model “sees” it and decides what to do.</li></ol><p>No hidden state. No magic memory. Just a list of messages that grows one entry at a time, fed into a model that has no idea what happened three iterations ago unless it’s still in the context window.</p><p>The key mental model: the context window is RAM, not a hard drive. Everything in it, the model can use. Everything not in it, is gone. You didn’t save it. You didn’t log it. The model simply doesn’t know it happened.</p><h3>Where the Brain Breaks</h3><p>This is where loop engineering stops being a tutorial exercise and becomes an actual engineering problem.</p><p>The context fills up. A single read_file call can dump 10,000 tokens of code into the history. Twenty iterations later, the window is stuffed. The model re-calls the same API it already ran. It drops evidence it used three iterations ago. It looks &quot;confused.&quot; It&#39;s not.. A bigger context window helps, but it&#39;s not the real fix — the problem is unbounded growth, not a small bucket. So you add compression and externalization: summarize old messages, push bulky results into a cache. The context becomes a cache, not the source of truth.</p><p>The loop won’t stop. The model calls the same tool with the same args on iterations 11, 12, and 13. It’s spinning. You need circuit breakers: a max-iteration cap, a token budget, a wall-clock timeout, and a repetition detector that kills the loop when nothing new is happening.</p><p>Tool calls fail. The API times out. The model’s JSON is malformed. The tool doesn’t exist. Your loop catches it, formats a clean error message, and feeds it back so the model can retry or pivot. Crash on a bad tool call and you’ve built a brain that dies at the first stubbed toe.</p><p>Latency stacks. Each iteration is 2–5 seconds of inference plus tool execution. Ten iterations is a minute-plus of spinner-watching. You need streaming responses, parallel tool calls where possible, and aggressive caching.</p><h3>The Cost of Thinking</h3><p>A loop that runs 12 iterations when 6 would do costs you roughly 50% more in inference (<em>example, not a measurement</em>). Loop engineering is a cost discipline. Every unnecessary iteration is a token paid for a thought the model didn’t need. Tight system prompts with clear stopping criteria, smaller tool payloads, and aggressive early termination — that’s the difference between a lean system and one burning cash for no reason.</p><h3>What to Actually Read</h3><ul><li>Anthropic, “Building Effective Agents” (Dec 2024) — the clearest writeup on when a loop beats a single call. Short, practical, no fluff.</li><li>LangGraph state-machine docs — see the loop formalized as explicit nodes and edges instead of a while-loop.</li><li>OpenAI function-calling spec — the actual JSON contract between your loop and the model. Shorter than you think.</li></ul><p>The brain isn’t the model. The brain is the loop, the memory management, the termination logic, and the error handling you build around it. The model is just the next-token function inside your while-loop. And the loop is what you actually own.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4b5b5d77fbdb" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Axios Supply Chain Attack: Technical Insights from the Front Lines]]></title>
            <link>https://infinityin-minds.medium.com/axios-supply-chain-attack-technical-insights-from-the-front-lines-8a2e5d83d78c?source=rss-ecb47cee9377------2</link>
            <guid isPermaLink="false">https://medium.com/p/8a2e5d83d78c</guid>
            <category><![CDATA[supply-chain-attack]]></category>
            <category><![CDATA[npm-supply-chain-attack]]></category>
            <category><![CDATA[axios]]></category>
            <dc:creator><![CDATA[Anil]]></dc:creator>
            <pubDate>Thu, 02 Apr 2026 09:14:58 GMT</pubDate>
            <atom:updated>2026-04-02T09:14:58.587Z</atom:updated>
            <content:encoded><![CDATA[<h4>Quick Summary</h4><p><strong>Incident Date</strong>: March 30–31, 2026<br><strong>Affected Library</strong>: axios (JavaScript HTTP client)<br><strong>Attack Method</strong>: Lead maintainer credential hijack<br><strong>Malicious Releases</strong>: axios@1.14.1 and axios@0.30.4<br><strong>Backdoor Package</strong>: plain-crypto-js (phantom dependency)<br><strong>C2 Server Target</strong>: sfrclak[.]com<br><strong>Safe Version</strong>: axios@1.14.0 or earlier</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*fNL1i-ESDflIapo_3s1Cdw.png" /></figure><h4>The Incident Timeline</h4><p>The incident began subtly, as most supply chain attacks do. A routine dependency review on March 30, 2026 revealed an update to Axios version 1.14.1. Soon after, builds began failing with abnormal network timeouts. Security tools later detected an unauthorized binary linked to a phantom dependency, <strong>plain-crypto-js</strong>. This discovery triggered a multi-team investigation that exposed a coordinated supply chain compromise.</p><h4>What Actually Happened</h4><p>Attackers did not compromise your code repository directly. They compromised the lead maintainer’s account credentials on either the <strong>npm registry or GitHub</strong> organization. Once they had <strong>write access</strong>, they published two malicious releases designed to be invisible at the application level:</p><p>- <strong>axios@1.14.1 (latest minor version)<br>- axios@0.30.4 (older legacy version)</strong></p><p>Both versions functioned identically to legitimate <strong>axios</strong> from an API perspective. That is the beauty of this attack — it was designed to be invisible at the app level while silently compromising your developer environment.</p><h4>The Phantom Dependency Mechanism</h4><p>Instead of modifying actual Axios source code, attackers added a secondary package named <strong>plain-crypto-js</strong> to the package manifest <strong>(package.json, yarn.lock, or pnpm-lock.yaml)</strong>. This secondary package contains a postinstall script that triggers every time you run npm install.</p><p>This script performs OS detection and downloads platform-specific binaries:</p><p><strong>- Windows: Drops executable to C:\ProgramData\<br>- macOS: Drops binary to /Library/Caches/<br>- Linux: Deploys Python-based installer to /tmp/ld.py</strong></p><p>The result is a Remote Access Trojan component that establishes persistence on your machine.</p><h4>What Can Be Compromised</h4><p>The attack targets the developer environment, not your application code directly. Where the damage becomes personal includes:</p><p><strong>Environment Variables:</strong><br>- Database connection strings (AWS RDS)<br>- API keys for external services<br>- Database passwords stored in .env files</p><p><strong>Provider Secrets:</strong><br>- AWS CLI tokens from AWS_PROFILE<br>- Azure Cloud Shell session tokens<br>- Google Cloud Service Account keys</p><p><strong>Local Data:</strong><br>- Browser authentication cookies<br>- Saved passwords in browser extensions<br>- SSH keys for private repositories<br>- Local git credentials</p><p><strong>Code Integrity:</strong><br>- Lateral movement into private repositories<br>- Injection of malicious code into production<br>- Committing malware to internal repositories</p><p>The most dangerous outcome is lateral movement. Once the <strong>RAT</strong> establishes persistence on your machine, it can access any other repository you are authenticated against. This is why AWS credentials are particularly concerning — they can be used to create IAM users and pivot deeper into cloud infrastructure.</p><h4><strong>How to Detect If Your System Is Compromised</strong></h4><p>The malware is designed to be stealthy, but it leaves specific footprints depending on your operating system.</p><p><strong>Step 1: Check Your Project Lockfiles (All Platforms)</strong></p><p>Run this in your project root to verify if axios was compromised:</p><pre>grep -E &quot;axios@(1.14.1|0.30.4)&quot; package-lock.json yarn.lock pnpm-lock.yaml</pre><p>If you see either version, that is a red flag immediately. You might not even know it because of the phantom dependency:</p><pre>grep -E &quot;plain-crypto-js&quot; package-lock.json yarn.lock pnpm-lock.yaml</pre><p>This check reveals if plain-crypto-js has been injected into your lockfile.</p><p><strong>Step 2: Check for RAT Binary (OS Specific)</strong></p><p>The RAT drops binaries in unexpected locations. Here is how to spot them on each platform:</p><p><strong>Windows: </strong>Open PowerShell and run this command:</p><pre>dir C:\ProgramData\wt.exe</pre><p>If wt.exe appears (or any other suspicious .exe in that directory), you have a candidate payload. The malware specifically targets this location because it is typically ignored by developers.</p><p><strong>macOS: </strong>Open Terminal and run this command:</p><pre>ls -la /Library/Caches/com.apple.act.mond</pre><p>Look for any file with unexpected permissions or ownership. If you see anything that does not match your system cache, investigate immediately.</p><p><strong>Linux: </strong>Check the temporary directory:</p><pre>ls -la /tmp/ld.py</pre><p>The presence of a Python script in /tmp named ld.py is a major indicator of compromise. The malware uses this location because it is ephemeral by default, making cleanup easier for attackers.</p><p><strong>Step 3: Check Network Connections</strong></p><p>The RAT attempts to communicate with a command-and-control server. Here is how you can detect that connection:</p><p><strong>macOS and Linux Users:</strong></p><pre>lsof -i -nP | grep sfrclak</pre><p>This will show any process connected to the C2 domain. If you see output, immediately disconnect from the network.</p><p><strong>Windows Users (PowerShell)</strong>:</p><pre>Get-NetTCPConnection | Where-Object { $_.RemoteAddress -like &quot;*sfrclak*&quot; }</pre><p>If this returns any connections, your system has established contact with the attacker’s server.</p><h4>Remediation Steps</h4><p>Here is what we did to clean up our systems:</p><p><strong>Phase 1: Immediate Isolation</strong></p><p>The moment we detected suspicious activity:</p><p>- Disconnect the machine from the network immediately (unplug Ethernet, disable Wi-Fi)<br>- Do not use any cloud management consoles — assume they may be compromised<br>- Notify your security operations team</p><p><strong>Phase 2: Dependency Rollback</strong></p><p>Force downgrade axios to a safe version:</p><pre>npm install axios@1.14.0 - save-exact<br># OR for yarn:<br>yarn add axios@1.14.0 - exact<br># OR for pnpm:<br>pnpm add axios@1.14.0 - lockfile-only</pre><p>Then purge the phantom package completely:</p><pre>rm -rf node_modules/plain-crypto-js # macOS/Linux<br>Remove-Item -Recurse -Force node_modules\plain-crypto-js # Windows<br># Then regenerate lockfile:<br>npm install</pre><p><strong>Phase 3: Credential Rotation (Crucial)</strong></p><p>This is where most people get it wrong. You must assume ALL secrets on the machine are compromised:</p><p><strong>Rotate these immediately:</strong><br>- NPM Personal Access Tokens<br>- AWS CLI Profile Credentials<br>- Azure Cloud Shell Sessions<br>- GCP Service Account Keys<br>- SSH Keys for all repositories<br>- Database Connection Strings in .env files<br>- Browser Cookies and Session Data</p><p><strong>Phase 4: System Rebuild</strong></p><p>In a professional environment, the safest path is to restore from backup created before March 30. Do not rollback on a compromised disk.</p><p>1. Restore from clean backup before incident date<br>2. Reboot machine fresh<br>3. Install dependencies cleanly<br>4. Reconnect and validate security posture</p><h4><strong>Prevention: Building Your Defense</strong></h4><p>Here is what we are implementing going forward:</p><p><strong>Registry Security Best Practices</strong></p><p>Enable audit before every major npm install using npm audit. For continuous monitoring, enable Socket or similar tools to watch for supply chain anomalies in real-time.</p><p><strong>Two-Factor Authentication Enforcement: </strong>All registry accounts (GitHub, npm, Azure DevOps) must have:<br>- Two-Factor Authentication enabled<br>- Emergency contact set up<br>- MFA token regeneration every 90 days</p><p><strong>Dependency Policy Configuration: </strong>Add to your organization CI/CD pipeline configuration:</p><pre>dependency-policy:<br> allow-prerelease: false<br> max-dependency-levels: 2<br> scan-on-install: true<br> require-peer-deps: true</pre><p><strong>Lockfile Integrity Checks: </strong>Run these checks in your CI/CD pipeline to verify lockfile matches package.json exactly before deployment.</p><h4>Stay Vigilant</h4><p>Supply chain attacks do not discriminate. A compromised maintainer account, an overlooked dependency update, or an unmonitored install script can all become entry points into your developer environment.</p><p>The axios incident shows us that modern development tools can be turned against you. That is why security must be built into every layer of our workflow — not just treated as a separate concern to check occasionally.</p><h4>Summary</h4><p>- Check your lockfiles for axios@1.14.1 or 0.30.4<br>- Look for plain-crypto-js in dependencies<br>- Run OS-specific binary checks for suspicious files<br>- Disconnect from network immediately if compromised<br>- Rotate all credentials before rebuilding systems<br>- Enable 2FA on all registry accounts</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8a2e5d83d78c" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Fine-Tune Language Models on Your Laptop Like a Pro]]></title>
            <link>https://infinityin-minds.medium.com/fine-tune-language-models-on-your-laptop-like-a-pro-582d02ee399b?source=rss-ecb47cee9377------2</link>
            <guid isPermaLink="false">https://medium.com/p/582d02ee399b</guid>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[fine-tuning]]></category>
            <category><![CDATA[ai]]></category>
            <dc:creator><![CDATA[Anil]]></dc:creator>
            <pubDate>Sun, 22 Mar 2026 20:26:15 GMT</pubDate>
            <atom:updated>2026-03-23T11:18:24.594Z</atom:updated>
            <content:encoded><![CDATA[<h3>Introduction</h3><p>Fine-tuning a large language model used to mean renting a cluster of A100s and setting aside a weekend. That’s no longer true. With modern parameter-efficient techniques and the hardware in a recent MacBook, you can meaningfully adapt an open-source model to a specific domain — locally, in minutes, without writing a single cloud provider a check.</p><p>This project is a small <strong>proof-of-concept</strong> built to test exactly that. The idea was simple: take a pre-trained model, feed it a focused document corpus, and measure whether the resulting model gives noticeably better answers on that domain compared to the base model out of the box.</p><p>The approach used here is <strong>LoRA (Low-Rank Adaptation)</strong> — a technique that freezes the base model’s weights and introduces a small set of trainable matrices alongside the existing layers. Instead of updating billions of parameters on every training step, you’re only updating a fraction of them. The base model stays intact; the adapter learns the delta. It’s fast, memory-efficient, and surprisingly effective for domain adaptation tasks.</p><p>The document corpus chosen for this POC is the <strong>GDPR</strong> — the EU’s General Data Protection Regulation. It’s a natural fit: dense, technical, domain-specific, and the kind of text that generic language models handle poorly. Most models have seen summaries and commentary about GDPR, not the regulation itself. Fine-tuning directly on the source gives the model something closer to first-hand knowledge.</p><p>The full pipeline — from PDF ingestion to a queryable web interface — runs on an Apple M1 Max with no external dependencies. One document, one epoch, one focused experiment. The goal was to validate the method, not ship a product.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*skz3kdl40VX90vtxn-9Jzg.png" /></figure><h3>The Problem with Generic LLMs</h3><p>Out-of-the-box language models are trained on the internet. They’ve seen <em>summaries</em> of GDPR. They’ve seen blog posts about it. But they haven’t deeply internalized the actual text — the specific articles, recitals, and obligations that matter when you’re trying to answer a real compliance question.</p><p>The fix isn’t to write a better prompt. The fix is to teach the model the source material directly.</p><p>That’s fine-tuning, learn more about LLM <a href="https://medium.com/@infinityin-minds/build-llm-vocab-tokens-embedding-and-context-05ee1de4df6e">jargons</a></p><h3>The Stack (Before We Dive In)</h3><ul><li><strong>Model</strong>: Microsoft Phi-2 (2.7B parameters)</li><li><strong>Fine-tuning method</strong>: LoRA (Low-Rank Adaptation)</li><li><strong>Hardware</strong>: Apple M1 Max, 32GB unified memory</li><li><strong>Interface</strong>: Streamlit web app</li><li><strong>PDF processing</strong>: PyPDF2 + LangChain text splitters</li><li><strong>Training</strong>: HuggingFace Transformers + PEFT</li></ul><p>Nothing exotic. Everything open source.</p><h3>Step 1: Turning a PDF into Training Data</h3><p>The first hurdle is getting text out of a regulatory PDF in a format a model can actually learn from.</p><p>The pipeline uses PyPDF2 to extract raw text, then LangChain’s RecursiveCharacterTextSplitter to cut it into digestible chunks:</p><ul><li><strong>Chunk size</strong>: 128 tokens</li><li><strong>Overlap</strong>: 25 tokens</li></ul><p>The overlap is important. You don’t want a sentence about data subject rights split across two chunks with no shared context — the model would learn two orphaned half-thoughts instead of one coherent idea.</p><p>One GDPR PDF produces about <strong>1,500 usable chunks</strong>. That’s your training dataset.</p><h3>Step 2: Why LoRA Instead of Full Fine-Tuning</h3><p>Full fine-tuning a 2.7B parameter model means updating <em>all</em> 2.7 billion weights on every training step. On a laptop, that’s not just slow — it’s a non-starter.</p><p>LoRA is the practical alternative. Instead of touching every weight, you freeze the base model and insert small trainable matrices alongside the attention and feed-forward layers. The result: you’re only updating <strong>~0.8% of total parameters</strong>, while still achieving meaningful domain adaptation.</p><p>The target layers in this project are q_proj, v_proj, fc1, and fc2 — the attention projections and feed-forward components where domain knowledge tends to live. The LoRA config looks like this:</p><pre>LoraConfig(<br>    r=4,           # rank<br>    lora_alpha=8,<br>    lora_dropout=0.1,<br>    target_modules=[&quot;q_proj&quot;, &quot;v_proj&quot;, &quot;fc1&quot;, &quot;fc2&quot;]<br>)</pre><p>Low rank. Lightweight. Effective enough for a proof-of-concept.</p><h3>Step 3: Training on Apple Silicon</h3><p>Apple’s M1/M2 chips support PyTorch’s MPS (Metal Performance Shaders) backend, which lets you use the GPU cores in the chip for training — no NVIDIA required.</p><p>The training loop is a custom implementation (bypassing HuggingFace’s Trainer) to give tighter control over memory usage and real-time progress feedback. Configuration is straightforward:</p><pre>model_name: &quot;microsoft/phi-2&quot;<br>learning_rate: 0.0005<br>batch_size: 1<br>num_epochs: 1<br>max_length: 2048</pre><p>Training 500 samples for one epoch took <strong>about 6.5 minutes</strong> on an M1 Max. Loss dropped from <strong>0.4566 → 0.1959</strong> — a solid convergence for a POC run.</p><h3>Step 4: Merging the Model for Clean Deployment</h3><p>After training, you have two things: the original base model and the LoRA adapter weights. At inference time you can load both together — but that means your deployment requires the PEFT library and the overhead of composing adapters on the fly.</p><p>The cleaner production path is to <em>merge</em> them:</p><pre>model = model.merge_and_unload()  # fuses LoRA weights into base model<br>model.save_pretrained(&quot;output/merged_model/&quot;)</pre><p>The result is a single standalone model (~5.5 GB) that loads and runs like any normal HuggingFace model — no PEFT, no adapters, no extra dependencies.</p><h3>Step 5: The Interface</h3><p>The Streamlit app wraps everything into a four-page UI: Home, Preprocessing, Fine-tuning, and Inference.</p><p>At query time, the prompt is formatted as:</p><pre>Question: What are the rights of a data subject under GDPR?</pre><p>The app checks for a merged model first, falls back to the LoRA adapter if none exists, and displays the generated response along with token counts so you can see exactly how much of the context window you’re consuming.</p><h3>Does It Actually Work?</h3><p>Short answer: yes, noticeably.</p><p>Side-by-side comparisons between the base Phi-2 and the fine-tuned version show a clear difference in GDPR-specific density. The fine-tuned model references specific obligations, data subject rights, and regulatory terminology that the base model glosses over or hallucinates.</p><p>It’s not perfect — one epoch on 500 chunks is a POC, not a production system. But it validates the pipeline. The model learns from the source material.</p><h3>The Takeaway</h3><p>You don’t need a data center to do meaningful fine-tuning. With LoRA and a modern Apple Silicon machine, you can take a compact open-source model, teach it a specific document corpus in under ten minutes, and build a working query interface on top of it — all locally, all for free.</p><p>Regulatory AI assistants don’t have to be generic. They can be grounded in the actual text that matters. The code is modular, the pipeline is extensible, and the approach scales. That’s the point of a good proof-of-concept.</p><p><strong>Additional articles for LLM</strong>:<br><a href="https://medium.com/gopenai/building-a-fully-local-rag-system-with-lm-studio-and-chromadb-197b025de54f">https://medium.com/gopenai/building-a-fully-local-rag-system-with-lm-studio-and-chromadb-197b025de54</a>f</p><ul><li><a href="https://infinityin-minds.medium.com/why-quantized-small-models-will-dominate-ais-future-honest-take-7a2e54a0fad0">Why Quantized Small Models will Dominate AI’s Future (Honest Take)</a></li><li><a href="https://blog.gopenai.com/llms-approach-to-extended-dialogues-understanding-memory-and-context-windows-ae6c51fb3bb5">LLM’s Approach to Extended Dialogues: Understanding Memory and Context Windows</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=582d02ee399b" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Building a Fully Local RAG System with LM Studio and ChromaDB]]></title>
            <link>https://blog.gopenai.com/building-a-fully-local-rag-system-with-lm-studio-and-chromadb-197b025de54f?source=rss-ecb47cee9377------2</link>
            <guid isPermaLink="false">https://medium.com/p/197b025de54f</guid>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[llm-training]]></category>
            <category><![CDATA[retrieval-augmented-gen]]></category>
            <dc:creator><![CDATA[Anil]]></dc:creator>
            <pubDate>Mon, 09 Feb 2026 19:05:45 GMT</pubDate>
            <atom:updated>2026-02-10T08:51:11.328Z</atom:updated>
            <content:encoded><![CDATA[<p>Retrieval‑Augmented Generation (RAG) has quickly become one of the most effective ways to build intelligent applications that can reason over private documents. While many RAG tutorials rely on cloud‑hosted LLMs and managed vector databases, it’s entirely possible to build a fully local, privacy‑preserving RAG pipeline on a Mac — even on consumer hardware like the M1 Max with 32 GB of unified memory. Same can be mimic on Windows machine with supported hardware.</p><p>In this article, we’ll walk through a complete, working RAG setup built around:</p><ul><li><strong>LM Studio</strong> running a local 20B‑parameter model</li><li><strong>ChromaDB</strong> as a lightweight, persistent vector database</li><li><strong>SentenceTransformers</strong> for local embeddings</li><li><strong>Python</strong> for ingestion, chunking, retrieval, and orchestration</li><li><strong>PDF ingestion</strong> using pypdf</li><li><strong>A simple text splitter</strong></li></ul><p>The final system can ingest PDFs from a folder, embed them, store them in a vector DB, and answer questions using a local LLM — all without sending data to the cloud.</p><p><strong>Why Build RAG Locally?</strong></p><p>Running RAG locally has several advantages:</p><ol><li><strong>Privacy: </strong>Your documents never leave your machine. This is essential for legal, financial, medical, or corporate data.</li></ol><p><strong>2. Cost: </strong>No API calls, no cloud GPUs, no hosting fees.</p><p><strong>3. Control: </strong>You choose the model, the embeddings, the chunking strategy, and the retrieval logic.</p><p><strong>4. Experimentation: </strong>Local RAG setups are perfect for prototyping before deploying to production. With tools like LM Studio and ChromaDB, local RAG is more accessible than ever.</p><p><strong>System Architecture Overview:</strong></p><p>Your RAG pipeline follows a clean, modular flow:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gPuxJfBqwmFfmJSuVqq-gQ.png" /></figure><p>Let’s break down each component.</p><p><strong>1. PDF Ingestion</strong></p><p>PDFs are notoriously inconsistent, but pypdf provides a reliable baseline for extracting text. The ingestion function loops through each page and concatenates the extracted text.</p><pre>def extract_text_from_pdf(path):<br>    reader = PdfReader(path)<br>    text = &quot;&quot;<br>    for page in reader.pages:<br>        page_text = page.extract_text()<br>        if page_text:<br>            text += page_text + &quot;\n&quot;<br>    return text</pre><p>This keeps the pipeline dependency‑light and avoids the overhead of OCR unless needed.</p><p><strong>2. Chunking</strong></p><p>Chunking is essential for RAG because LLMs cannot process entire documents at once. Instead of relying on LangChain’s text splitters, your implementation uses a simple sliding‑window approach:</p><pre>def split_text(text, chunk_size=500, chunk_overlap=100):<br>    chunks = []<br>    start = 0<br>    step = chunk_size - chunk_overlap<br><br>    while start &lt; len(text):<br>        end = start + chunk_size<br>        chunks.append(text[start:end])<br>        start += step<br><br>    return chunks</pre><p>This gives you: Predictable chunk sizes, Overlap to preserve context and Zero external dependencies</p><p><strong>3. Local Embeddings with SentenceTransformers</strong></p><p>Embeddings convert text into numerical vectors that can be searched efficiently. You chose:</p><pre>intfloat/e5-small-v2</pre><p>This model is:</p><ul><li>Fast on CPU/MPS</li><li>High‑quality for retrieval tasks</li><li>Lightweight enough for repeated queries</li></ul><pre>embedder = SentenceTransformer(EMBED_MODEL)</pre><p>This keeps your pipeline fully local and performant.</p><p><strong>4. Vector Storage with ChromaDB</strong></p><p>ChromaDB is ideal for local RAG because:</p><ul><li>It runs entirely on your machine</li><li>It supports persistence</li><li>It has a simple Python API</li><li>It integrates cleanly with embedding models</li></ul><p>Your setup:</p><pre>client = chromadb.PersistentClient(path=DB_FOLDER)<br>collection = client.get_or_create_collection(&quot;documents&quot;)</pre><p>Each chunk is stored with:</p><ul><li>An ID</li><li>The embedding vector</li><li>The raw text</li></ul><p>This makes retrieval straightforward.</p><p><strong>5. Embedding and Storing Chunks</strong></p><p>Each chunk is embedded and inserted into ChromaDB:</p><pre>def embed_and_store(chunks, source_name):<br>    for i, chunk in enumerate(chunks):<br>        embedding = embedder.encode(chunk).tolist()<br>        collection.add(<br>            ids=[f&quot;{source_name}_{i}&quot;],<br>            embeddings=[embedding],<br>            documents=[chunk]<br>        )</pre><p>This creates a searchable knowledge base from your PDFs.</p><p><strong>6. Semantic Search</strong></p><p>When the user asks a question, the system embeds the query and retrieves the top‑k most relevant chunks:</p><pre>def search(query, k=3):<br>    q_emb = embedder.encode(query).tolist()<br>    results = collection.query(<br>        query_embeddings=[q_emb],<br>        n_results=k<br>    )<br>    return results[&quot;documents&quot;][0]</pre><p>This is the “retrieval” part of RAG.</p><p><strong>7. Querying LM Studio</strong></p><p>LM Studio exposes a local API that mimics OpenAI’s interface. Your implementation sends a structured prompt containing:</p><ul><li>The retrieved context</li><li>The user’s question</li><li>Instructions to avoid hallucination</li></ul><pre>prompt = f&quot;&quot;&quot;<br>You are a helpful assistant. Use ONLY the context below to answer the question.<br><br>Context:<br>{context}<br><br>Question:<br>{query}<br><br>Answer:<br>&quot;&quot;&quot;</pre><p>The request is sent to LM Studio’s /chat endpoint:</p><pre>response = requests.post(url, json=payload, timeout=60)</pre><p>Your code also handles multiple possible response formats, which is smart because LM Studio’s output schema varies by model.</p><p><strong>8. The RAG Pipeline</strong></p><p>The full retrieval‑and‑generation flow is wrapped in a single function:</p><pre>def rag_query(query):<br>    context_chunks = search(query)<br>    context = &quot;\n\n&quot;.join(context_chunks)<br>    answer = ask_llm(query, context)<br>    return answer</pre><p>This makes the system easy to call from a CLI, API, or UI.</p><p><strong>9. Running the System</strong></p><p>When executed, the script:</p><ol><li>Ingests all PDFs in the ./doc folder</li><li>Builds the vector database</li><li>Enters an interactive loop where the user can ask questions</li></ol><p>This makes it a perfect POC for local document intelligence.</p><h4>For best results:</h4><ul><li>Keep chunk size around 400–600 characters</li><li>Use small embedding models</li><li>Limit retrieval to top‑3 or top‑5 chunks</li><li>Use temperature 0.0–0.3 for factual answers</li></ul><p><strong>Why This Setup Works So Well</strong></p><p>This architecture is:</p><p><strong>Simple: </strong>No LangChain, no cloud services, no complex orchestration.</p><p><strong>Local: </strong>Every component runs on your machine.</p><p><strong>Modular: </strong>You can swap out:</p><ul><li>Embedding models</li><li>LLMs</li><li>Vector DBs</li><li>Chunking strategies</li></ul><h3>Conclusion:</h3><p>Building a fully local RAG system on macOS is far more achievable than most developers expect. With LM Studio handling LLM inference, ChromaDB providing fast vector search, and lightweight Python components for ingestion and embedding, you can create a private, high‑performance retrieval pipeline that never leaves your machine. This setup is ideal for prototyping knowledge assistants, internal search tools, or domain‑specific chatbots — especially when working with sensitive documents.</p><p>What makes this approach powerful is its simplicity. There’s no dependency on cloud APIs, no vendor lock‑in, and no complex orchestration layers. Each component is modular, so you can easily swap models, upgrade your vector database, or extend the system with a UI or metadata filtering. As local LLM tooling continues to improve, this kind of architecture will only become more capable and more relevant for developers who value privacy, control, and experimentation.</p><h4>Further Reading:</h4><ul><li><a href="https://infinityin-minds.medium.com/why-quantized-small-models-will-dominate-ais-future-honest-take-7a2e54a0fad0">Why Quantized Small Models will Dominate AI’s Future (Honest Take)</a></li><li><a href="https://blog.gopenai.com/llms-approach-to-extended-dialogues-understanding-memory-and-context-windows-ae6c51fb3bb5">LLM’s Approach to Extended Dialogues: Understanding Memory and Context Windows</a></li><li><a href="https://infinityin-minds.medium.com/why-llm-models-hallucinates-ac038ec14640">Why LLM models Hallucinates</a></li><li><a href="https://blog.gopenai.com/fine-tuning-vs-retrieval-augmented-generation-rag-d54ed6f11920">Fine-Tuning vs Retrieval Augmented Generation (RAG)</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=197b025de54f" width="1" height="1" alt=""><hr><p><a href="https://blog.gopenai.com/building-a-fully-local-rag-system-with-lm-studio-and-chromadb-197b025de54f">Building a Fully Local RAG System with LM Studio and ChromaDB</a> was originally published in <a href="https://blog.gopenai.com/">GoPenAI</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Why Quantized Small Models will Dominate AI’s Future (Honest Take)]]></title>
            <link>https://infinityin-minds.medium.com/why-quantized-small-models-will-dominate-ais-future-honest-take-7a2e54a0fad0?source=rss-ecb47cee9377------2</link>
            <guid isPermaLink="false">https://medium.com/p/7a2e54a0fad0</guid>
            <category><![CDATA[quantization]]></category>
            <category><![CDATA[llm-quantization]]></category>
            <category><![CDATA[llm-mobile]]></category>
            <category><![CDATA[ai-and-machine-learning]]></category>
            <dc:creator><![CDATA[Anil]]></dc:creator>
            <pubDate>Mon, 05 Jan 2026 08:02:40 GMT</pubDate>
            <atom:updated>2026-01-05T08:02:40.730Z</atom:updated>
            <content:encoded><![CDATA[<p><em>The shift from giant generalists to tiny specialists</em></p><p>For the last few years, the AI world has been obsessed with one thing: <strong><em>Scale: Big and Large models</em></strong>, more parameters, more GPUs, more data. Every new breakthrough, new LLM models seemed to come from adding another billion parameters to an already massive architecture.</p><p>But something interesting is happening beneath the surface — a quiet counter‑revolution. A growing number of researchers, developers, and businesses are realizing that <strong>small, quantized models might actually be the future of AI</strong>.</p><p>And honestly, they might be right, here is my opinion on why it’s more important to focus on smaller quantized models for specific business use cases rather than relying on larger models on each use.</p><h3>Quantization</h3><p>What actually makes LLM model size smaller, it’s <strong>Quantization</strong>. It’s a technique used to shrink the size of a LLM model by reducing the precision of its numerical weights. Instead of storing values in high‑precision formats like 32‑bit floating point, a quantized model uses lower‑precision formats such as INT8, INT4, or even INT2. This dramatically cuts memory usage, speeds up inference, and allows models to run efficiently on mobile devices, edge hardware, and low‑power environments — often with only a small drop in accuracy. Quantization is one of the key enablers of practical, on‑device AI, making advanced models lightweight, fast, and accessible without relying on heavy cloud infrastructure.</p><h3>The Rise of Small, Quantized Models</h3><p>Small models are not just “mini versions” of big LLMs, they represent a shift in LLM philosophy: <strong>Easy to deploy on edge devices,</strong> <strong>Make AI efficient, specialized with concentrated knowledge.</strong></p><p><strong>Quantization — reducing model precision from FP32 to INT8, INT4, or even INT2</strong> which shrinks models dramatically. Suddenly, models doesn’t require massive data centers, expensive GPUs and lot of resources, instead it can run on:</p><ul><li>smartphones and laptops</li><li>edge devices</li><li>IoT hardware</li><li>offline environments</li></ul><p>This shift is huge. It means AI is no longer locked behind cloud APIs or expensive GPU clusters. It becomes <strong>personal, local, and accessible</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rKjAJFZU2rSF0RbQuc4TBg.png" /></figure><h3>Why Specialization Beats General Knowledge</h3><p>Large general-purpose LLMs are impressive, but they’re also unfocused. They know a little bit about everything — and sometimes that’s the problem. It’s like Jake of all trades and master of none. A small model trained specifically for:</p><ul><li>finance</li><li>cybersecurity</li><li>medical diagnosis</li><li>legal reasoning</li><li>customer support</li></ul><p>…will often outperform a giant general model in that domain.</p><p>Why? Because <strong>specialization reduces noise</strong>. It narrows the model’s knowledge to what actually matters. It cuts down hallucinations. It improves reliability.</p><p>In the real world, accuracy beats size every time.</p><h3>Why Businesses Should Pay Attention</h3><p>Companies are starting to realize that they don’t need a 70B or 120B parameter model to answer internal HR questions or summarize policy documents. They need:</p><ul><li>a <strong>5B model</strong> fine‑tuned on their domain</li><li>quantized to <strong>INT4</strong></li><li>running locally or on a small server</li><li>connected to their internal knowledge base</li></ul><p>This approach is: cheaper, faster, more secure and more accurate for their use case. It’s not just a technical shift — it’s a business advantage.</p><h3>On‑Device AI is the Next Big Leap</h3><p>Your phone becomes your AI assistant. Your laptop becomes your private analyst. Your smartwatch becomes your personal coach.</p><p>This is the future people actually want — AI that lives <em>with</em> them, not somewhere in a distant server farm. Imagine AI that:</p><ul><li>works offline</li><li>respects your privacy</li><li>responds instantly</li><li>doesn’t send your data to the cloud</li><li>doesn’t cost a fortune to run</li></ul><p>That’s exactly what small quantized models enable.</p><h3>The Future:</h3><p>Instead of one giant model doing everything, imagine a system made of: a small finance model, legal model, a small HR model. Each one is an expert. Each one is efficient. Each one is easy to update. Together, they form a powerful, modular AI ecosystem.</p><p>This is where the industry is heading. The future is a network of small, specialized AI experts — not one giant model. Those who are early adopters will lead in this space.</p><blockquote>Loved this article? Follow for more insights and subscribe to stay updated on AI + Cyber Security breakthroughs! 🚀</blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7a2e54a0fad0" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[LLM’s Approach to Extended Dialogues: Understanding Memory and Context Windows]]></title>
            <link>https://blog.gopenai.com/llms-approach-to-extended-dialogues-understanding-memory-and-context-windows-ae6c51fb3bb5?source=rss-ecb47cee9377------2</link>
            <guid isPermaLink="false">https://medium.com/p/ae6c51fb3bb5</guid>
            <category><![CDATA[ai-and-language]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[llm-memory-management]]></category>
            <category><![CDATA[llm-memory-and-context]]></category>
            <dc:creator><![CDATA[Anil]]></dc:creator>
            <pubDate>Sat, 03 Jan 2026 18:22:56 GMT</pubDate>
            <atom:updated>2026-01-08T07:35:27.198Z</atom:updated>
            <cc:license>https://creativecommons.org/licenses/by-nc-sa/4.0/</cc:license>
            <content:encoded><![CDATA[<h3>Context Window</h3><p>Imagine you’re telling a story to a friend, but your friend can only remember the last <strong>10 sentences</strong> at a particular moment. If you talk too much and provide too much information, the initial part of your story disappears from their memory. That’s what happens with LLMs — they have a “context window,” which is like a box that can only hold a certain number of <a href="https://medium.com/@infinityin-minds/build-llm-vocab-tokens-embedding-and-context-05ee1de4df6e">words or tokens</a>.</p><p><strong>Example:</strong><br>If the model’s box holds 4K tokens, it’s like saying:</p><blockquote><em>“I can remember everything you said… as long as it fits in this box. If you bring a bigger box then sorry, I’ll forget what you said earlier!”</em></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*SPUrYiwkAgcPoPODXU4hFg.jpeg" /></figure><h3>Limited Context Windows Problem</h3><p>LLMs process text in chunks called <strong>tokens</strong> (pieces of words). The <strong>context window</strong> is the maximum number of tokens the model can process at once.</p><p><strong>Example:</strong> If an LLM has 2K token limit, it can only consider the last 2K tokens of your latest conversation. Anything before is gone out of context.</p><p><strong>Why it’s a problem?</strong></p><ul><li><strong>Cost and speed:</strong> Increasing the window size makes <strong>computation expensive</strong> because attention mechanisms scale roughly quadratically with the number of tokens.</li><li><strong>Loss of coherence:</strong> In long conversations or documents, the model forgets earlier details (e.g., character names, instructions).</li><li><strong>Truncation:</strong> If you feed a 100-page document, only the last few pages fits.</li></ul><h3>Context Windows of different LLMs:</h3><ul><li>Claude 3.5 Sonnet / Opus class models: widely reported with up to ~200k tokens</li><li>GPT‑4.1 (API): up to about 1M tokens of context</li><li>Gemini 1.5 Pro: advertised with up to 1M tokens</li></ul><h3>How latest LLM models handles context window better:</h3><p>Since LLM models cannot have unlimited context window size, so it’s much more important to make the context window larger enough to hold the context without loosing too much information. There are many ways to improve and handle this issue, here are several strategies to overcome these limitations:</p><h4>Bigger Context Windows</h4><ul><li>Models like <strong>GPT-5.2 (~400K tokens)</strong> and <strong>Claude 4.5 (200K tokens)</strong> massively increase the memory box.</li><li>Trade-off: Larger windows = more compute and cost, but better for long documents.</li></ul><h4>Retrieval-Augmented Generation (RAG)</h4><ul><li>Rather stuffing of information into the LLM model while training, <a href="https://medium.com/gopenai/fine-tuning-vs-retrieval-augmented-generation-rag-d54ed6f11920"><strong><em>RAG</em></strong></a><strong><em> </em></strong>fetches relevant chunks from an external database.</li><li>Think of it as the model saying: “I don’t need to memorize the whole Wikipedia, I’ll just look up for the right page when needed”</li></ul><h4>Persistent Memory</h4><ul><li>Experimental approaches store long-term context outside the model like writing in a notebook.</li><li>The model can refer back to this memory when needed.</li></ul><h4>Efficient Attention Mechanisms</h4><ul><li>Techniques like <strong>FlashAttention</strong> and <strong>Sparse Attention</strong> reduce the cost of handling large windows.</li><li>Instead of comparing every token with every other token, they use smart shortcuts.</li></ul><h3>Conclusion</h3><p>Large Language Models are powerful, but their ability to “remember” is limited by the size of their <strong>context window</strong> — like a friend who can only recall the last few pages of a book. This constraint leads to challenges in maintaining coherence during long conversations or processing large documents. Newer models tackle this by:</p><ul><li>Expanding context windows (up to hundreds of thousands of tokens).</li><li>Using smarter attention mechanisms to reduce computational cost.</li><li>Employing techniques like <strong>summarization</strong>, <strong>retrieval-augmented generation (RAG)</strong>, and <strong>persistent memory</strong> to keep important information accessible without overwhelming the model.</li></ul><p>As research advances, and advanced of newer architecture and design, LLMs can handle <strong>massive contexts efficiently</strong>, blending short-term and long-term memory — making them more like a well-organized librarian than a forgetful friend.</p><p><em>If you are interested in fine tuning and building RAG system then learn more here: </em><a href="https://medium.com/gopenai/fine-tuning-vs-retrieval-augmented-generation-rag-d54ed6f11920"><em>https://medium.com/gopenai/fine-tuning-vs-retrieval-augmented-generation-rag-d54ed6f11920</em></a></p><p><em>Want to talk like AI engineer: </em><a href="https://infinityin-minds.medium.com/build-llm-vocab-tokens-embedding-and-context-05ee1de4df6e"><em>https://infinityin-minds.medium.com/build-llm-vocab-tokens-embedding-and-context-05ee1de4df6e</em></a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ae6c51fb3bb5" width="1" height="1" alt=""><hr><p><a href="https://blog.gopenai.com/llms-approach-to-extended-dialogues-understanding-memory-and-context-windows-ae6c51fb3bb5">LLM’s Approach to Extended Dialogues: Understanding Memory and Context Windows</a> was originally published in <a href="https://blog.gopenai.com/">GoPenAI</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Why LLM models Hallucinates]]></title>
            <link>https://infinityin-minds.medium.com/why-llm-models-hallucinates-ac038ec14640?source=rss-ecb47cee9377------2</link>
            <guid isPermaLink="false">https://medium.com/p/ac038ec14640</guid>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[llm-hallucinations]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[llm-security]]></category>
            <dc:creator><![CDATA[Anil]]></dc:creator>
            <pubDate>Thu, 20 Nov 2025 04:08:45 GMT</pubDate>
            <atom:updated>2025-11-20T04:25:31.403Z</atom:updated>
            <content:encoded><![CDATA[<h3><strong>AI Hallucinations</strong></h3><p>In the context of <strong>Large Language Models (LLMs)</strong> and other generative AI systems, <em>hallucination</em> occurs when the model produces output that is <strong>factually incorrect, logically inconsistent, or entirely fabricated</strong>, yet presents it with high confidence and fluency.</p><p>Examples:</p><ul><li>Creating a research paper or citation that isn’t actually there.</li><li>Inaccurate historical facts.</li><li>Producing code that looks legit but fails when executed.</li><li>Creating physically impossible objects.</li></ul><p>Hallucinations pose significant challenges in high-stakes fields such as law, medicine, finance, and scientific research, where misinformation can have serious consequences.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/576/1*zryPclbNjCFbey6Pnxl5qQ.jpeg" /></figure><h3><strong>Why Do AI Models Hallucinate?</strong></h3><p>Hallucinations arise from a combination of <strong>limitations in knowledge, model architecture, and probabilistic nature of text generation</strong>.</p><p><strong>Key Causes:</strong></p><ul><li><strong>Limitations of Training Data: </strong>Models are trained on vast internet-scale datasets containing both accurate and inaccurate information. Outdated, biased, or incomplete data can lead to flawed outputs.</li><li><strong>Lack of Grounding in Reality: </strong>LLMs don’t have a live connection to factual databases unless explicitly integrated. They generate text by predicting the most likely next token, not by verifying truth.</li><li><strong>Overgeneralization of Patterns: </strong>The model may apply learned patterns to contexts where they don’t fit, producing plausible but wrong answers.</li><li><strong>Prompt Ambiguity: </strong>Vague or misleading prompts can cause the model to “fill in the blanks” with fabricated details.</li><li><strong>Inherent Probabilistic Nature: </strong>LLMs are statistical systems — they optimize for <em>linguistic plausibility</em>, not. factual accuracy.</li></ul><h3><strong>How Hallucinations Happen</strong></h3><p>Hallucinations are not random — they follow predictable patterns:</p><ul><li><strong>Intrinsic Hallucinations:</strong> These happen when the output doesn’t match the original source, like misremembering something from a document.</li><li><strong>Extrinsic Hallucinations:</strong> These occur when the output includes information that wasn’t actually in the source.</li><li><strong>Prompt Contradictions: </strong>The output doesn’t quite match the original question.</li><li><strong>Sentence Contradictions:</strong> There are inconsistencies within the same response.</li></ul><p><strong>Technical Pathway:</strong></p><ul><li>Input text is tokenized.</li><li>The model predicts the next token based on learned probabilities.</li><li>Without a fact-checking step, the model may choose a token sequence that is linguistically likely but factually false.</li><li>The output is delivered with the same confidence as correct answers.</li></ul><h3><strong>How to Reduce Hallucinations</strong></h3><p>Mitigation requires a <strong>multi-layered approach</strong>:</p><p><strong>Data-Level Strategies</strong></p><ul><li><strong>Improve Training Data Quality</strong>: Curate datasets to remove errors and bias.</li><li><strong>Domain-Specific Fine-Tuning</strong>: Train on verified, high-quality domain data.</li></ul><p><strong>Model-Level Strategies</strong></p><ul><li><strong>Retrieval-Augmented Generation (RAG)</strong>: Combine the LLM with a search or database retrieval step to ground answers in real data.</li><li><strong>RLHF (Reinforcement Learning from Human Feedback)</strong>: Teach the model to prefer accurate, helpful responses.</li><li><strong>Multi-Model Cross-Checking</strong>: Use multiple models to verify each other’s outputs.</li></ul><p><strong>Prompt-Level Strategies</strong></p><ul><li><strong>Prompt Engineering</strong>: Ask the model to cite sources, explain reasoning, or state uncertainty.</li><li><strong>Contextual Framing</strong>: Provide relevant background to reduce guesswork.</li></ul><p><strong>Post-Processing Strategies</strong></p><ul><li><strong>Automated Fact-Checking</strong>: Run outputs through verification tools.</li><li><strong>Human-in-the-Loop Review</strong>: Experts validate outputs before use in critical applications.</li></ul><h3><strong>How to Detect Hallucinations</strong></h3><p>Detecting hallucinations can be tricky because they often sound real, but here are some ways to help:</p><ul><li>Source Verification: Make sure the model’s claims come from a trustworthy source.</li><li>Cross-Referencing: Check if the information matches up with well-known databases, APIs, or other AI models.</li><li>Consistency Checks: Ask the model the same question in different ways — hallucinations often give different answers.</li><li>Domain Expert Review: Have a human expert look over the content in areas where they have special knowledge.</li><li>Confidence Indicators: Some systems can show how confident they are or point out parts that are a bit unclear.</li></ul><h4>Conclusion:</h4><p>AI hallucinations occur when generative AI systems produce incorrect or fabricated outputs with high confidence. This issue arises from limitations in training data, lack of reality facts, and the probabilistic nature of text generation. Mitigation strategies include improving training data quality, using retrieval-augmented generation, and implementing post-processing techniques like fact-checking and expert review.</p><p>If you are interested in fine tuning and building RAG system then learn more here: <a href="https://medium.com/gopenai/fine-tuning-vs-retrieval-augmented-generation-rag-d54ed6f11920">https://medium.com/gopenai/fine-tuning-vs-retrieval-augmented-generation-rag-d54ed6f11920</a></p><p>Want to talk like AI engineer:</p><p><a href="https://infinityin-minds.medium.com/build-llm-vocab-tokens-embedding-and-context-05ee1de4df6e">https://infinityin-minds.medium.com/build-llm-vocab-tokens-embedding-and-context-05ee1de4df6e</a></p><p>If you find this interesting, then do share your thoughts.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ac038ec14640" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Build LLM vocab: Tokens, Embedding, and Context:]]></title>
            <link>https://infinityin-minds.medium.com/build-llm-vocab-tokens-embedding-and-context-05ee1de4df6e?source=rss-ecb47cee9377------2</link>
            <guid isPermaLink="false">https://medium.com/p/05ee1de4df6e</guid>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[ai-learning]]></category>
            <category><![CDATA[context-window]]></category>
            <category><![CDATA[llm]]></category>
            <dc:creator><![CDATA[Anil]]></dc:creator>
            <pubDate>Sat, 06 Sep 2025 21:34:30 GMT</pubDate>
            <atom:updated>2025-11-20T04:24:00.246Z</atom:updated>
            <content:encoded><![CDATA[<h3><strong>Build LLM vocab: Tokens, Embedding and Context Windows</strong></h3><p>In the world of Large Language Models (LLMs), data is barely a raw text — it needs to be carefully transformed into a form that machines can understand and manipulate. This transformation process is at the heart of how LLMs operate, and it relies on a few key concepts: <strong>tokens</strong>, <strong>tokenization</strong>, <strong>embeddings</strong>, <strong>vocabulary size</strong>, and the <strong>context window</strong>. Together, these elements form the bridge between human language and machine computation.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4DEdfVP18xycMB3vy5q4mQ.png" /></figure><h3><strong>Tokens</strong></h3><blockquote><strong>The Lego Bricks of Machine Language</strong></blockquote><p>Think of tokens as the tiny Lego bricks that LLMs use to build up meaning. Without them, the machine is basically staring into the void<em>.</em></p><p>Tokens are the atoms of machine language. They’re the base unit, the “stuff” that everything else relies on, you can’t explain “oh this article looks interesting” to an LLM unless you break it down into the right token bits first. It’s the bridge between our lovely chaotic human language and the machine’s world of numbers. Without tokens, an LLM is basically like you are reading German or French without knowing the words</p><p>For ex: <em>“This article looks interesting for LLM”</em></p><p>Depending on the tokenizer (each model has its own style), it might split like this:</p><p>[“This”, “article”, “looks”, “interesting”, “for”, “LL”, “M”]</p><p>Funny yup, notice the twist: many tokenizers can’t keep acronyms as a single word. “LLM” could come out as <strong>“LL”</strong> + <strong>“M”</strong>, because of tokenizer type.</p><h3><strong>Tokenization:</strong></h3><blockquote><strong>Turning Text into Tokens</strong></blockquote><p>Before a LLM can understand language, it needs to chop text into small, manageable chunks called tokens and the process to do that is called <em>tokenization</em>, and it’s the very first step in turning human words into something a machine can work with.</p><p>There are 2 main cutting styles:</p><ul><li>Word-level tokenization — Splits text by spaces or punctuation.<br> <em>Like cutting a pizza into slices: each word = one big slice.</em><br> Example: “Dogs are lovely” → [“Dogs”, “are”, “lovely]</li><li>Subword tokenization — Breaks words into smaller, reusable chunks.</li></ul><p><em>This is like slicing your pizza into bite-sized squares, so you can reuse leftovers anywhere.</em><br> For eg: “unbelievable” → [“un”, “believ”, “able”]</p><h3><strong>Embeddings</strong></h3><blockquote><strong>Convert Tokens into Number</strong></blockquote><p>After we slice text into tokens, the model still needs to <em>understand</em> what those tokens actually mean. This is where embeddings come in.</p><p>An embedding is simply a list of numbers (a vector) that represents a token’s meaning in a <em>multi-dimensional array</em>. Think of it as giving each token an “address” on a vast map of language.</p><p>Here’s why embeddings matter:</p><ul><li>Similar meanings live close together<br> Tokens like <em>“Stockholm”</em> and <em>“Delhi”</em> end up with vectors that sit near each other on the map, because in a context both are capitals of a country.</li><li>Context shapes meaning<br> The word <em>“bank”</em> doesn’t always mean the same thing. In <em>“river bank”</em>, its embedding will be different than in <em>“money bank”</em>. Contextual models know how to shift the “address” depending on the neighborhood the word appears in.</li><li>• The model thinks in vectors, not words<br> Once tokens are converted to embeddings, the LLM can do math with them — like measuring distances or directions — to capture relationships between words.</li></ul><h3><strong>Vocabulary Size</strong></h3><blockquote><strong>The Model’s Lexicon</strong></blockquote><p>When we say a model has a “vocabulary,” it means the total number of unique tokens (pieces of text) that the model has been trained to recognize, and not about memorizing the Oxford English Dictionary.</p><p>Think of it as the menu of tokens the model is allowed to work with. If a token is on the menu, the model already knows it. If not, it has to creatively chop the word into smaller sub-tokens it does know.</p><p>Model with Small Vocabulary</p><ul><li>The model knows fewer tokens, so it has to break words down more often (using subwords or even single characters).</li><li>Pros: Super flexible — it can handle new words, slang, or languages it wasn’t explicitly trained on by just remixing known chunks.</li><li>Cons: Sentence processing takes more steps because simple words get split into many tokens.</li></ul><p>Model with Large Vocabulary</p><ul><li>The model stores more whole tokens, so it can recognize complex or rare words all at once.</li><li>Pros: Faster processing for known words, since less splitting is needed.</li><li>Cons: Bigger vocabulary = more memory and compute requirements, because each token needs its own spot (embedding vector) in the model’s brain</li></ul><p>Real-World Balance</p><p>Modern LLMs (like GPT-style models) usually aim for a middle ground.</p><ul><li>Big enough to capture lots of words directly.</li><li>Small enough to stay efficient without consuming too much memory.</li></ul><p>A blend of both small and big is the core reason why new models can handle everyday tasks.</p><h3><strong>Context Window</strong></h3><blockquote><strong>The Model’s Cache Memory</strong></blockquote><p>Think context window as maximum number of words you can remember while active listening, in LLM terms its the number of token the model can remember at once.</p><ul><li>A model with 2K token can remember around 1500 words during a particular conversation</li><li>Longer context windows allow the model to handle longer conversations, bigger documents without losing track of earlier details.</li></ul><p>Once the limit is reached, older tokens are either replaced or dropped completely, which means model tend to “forget” earlier conversations.</p><blockquote><strong>So, a natural question is why all above topics matter in LLM</strong></blockquote><p>Understanding above concepts is essential because they directly affect:</p><ul><li><strong>Performance</strong>: Tokenization impacts speed and accuracy.</li><li><strong>Cost</strong>: Many LLM APIs charge per token processed.</li><li><strong>Capabilities</strong>: Vocabulary size and context window determine how well a model can handle complex or lengthy inputs.</li><li><strong>Interpretability</strong>: Embeddings reveal how the model organizes and relates concepts internally.</li></ul><h3><strong>Summary</strong></h3><p>LLMs are like language robots that understand and generate human language. They do this by breaking down language into smaller pieces called tokens. Each token has a special meaning, and a vocabulary size determines how many different tokens the model can understand. The context window is like the robot’s memory, which helps it understand the context of a sentence. All these pieces work together to make LLMs capable of answering questions, writing poetry, and more.</p><p>Interested in learning Fine tuning vs RAG, read more <a href="https://medium.com/gopenai/fine-tuning-vs-retrieval-augmented-generation-rag-d54ed6f11920">https://medium.com/gopenai/fine-tuning-vs-retrieval-augmented-generation-rag-d54ed6f11920</a></p><p>Find this article interesting? Support and drop a comment.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=05ee1de4df6e" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Fine-Tuning vs Retrieval Augmented Generation (RAG)]]></title>
            <link>https://blog.gopenai.com/fine-tuning-vs-retrieval-augmented-generation-rag-d54ed6f11920?source=rss-ecb47cee9377------2</link>
            <guid isPermaLink="false">https://medium.com/p/d54ed6f11920</guid>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[machine-learning-ai]]></category>
            <category><![CDATA[llm-evaluation]]></category>
            <category><![CDATA[rag-vs-fine-tuning]]></category>
            <category><![CDATA[llm]]></category>
            <dc:creator><![CDATA[Anil]]></dc:creator>
            <pubDate>Mon, 11 Aug 2025 20:11:16 GMT</pubDate>
            <atom:updated>2025-08-15T14:59:13.325Z</atom:updated>
            <content:encoded><![CDATA[<blockquote>Choosing the right approach for build custom LLMs</blockquote><h3>Intro:</h3><p>Large language models (LLMs) are becoming a huge part of how AI tools are build these days. Did you wonder what’s the tricky part — if you want one to work really well for your specific needs, what’s the best way to do it? Right now, two main approaches which really stand out are <strong><em>Fine-Tuning and Retrieval-Augmented Generation (RAG)</em></strong>. Each has its own pros and cons, and which one works best really depends on what you’re trying to achieve.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0CJ7isZEIPFpNA8xm-GnCA.jpeg" /></figure><h3>Definition:</h3><ul><li><strong>Fine Tuning</strong>: Retrains an existing LLM on domain-specific data, adding that knowledge directly into the model’s parameters, think like it’s updating the model’s brain.</li><li><strong>Retrieval-Augmented Generation (RAG)</strong>: Allows an LLM with a retrieval system to fetch and use external, up-to-date information at query time without changing its parameters, think like model refers to external library (like PDF or Database) to get more clarity on a specific topic.</li></ul><p><em>Fine tuning changes the model’s brain and RAG gives the model a set of data stores to read from.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/960/1*AD1BBMj0T_RPzJteMIQkFQ.png" /></figure><h3>How to decide:</h3><ul><li>Fine Tuning is better when you want to train based on point in time data and data updates are less frequent. More relevant when priority is domain-specific reasoning.</li><li>RAG is better when you priority is updated data. Smaller storage and lower cost of training</li></ul><h3>Can we choose Hybrid</h3><p>Absolutely, this is a perfect mix of both the worlds, if costs is not a primary concern and you want updated data from various tools or data sources then go with hybrid approach. First, the base LLM model is taken and fine-trained on specific key areas or domain. Once trained, RAG features are added that will provide real world data when processing requests.</p><h3>Cost &amp; Performance</h3><p>Fine Tuning: High upfront cost due to intensive compute resource requirements and data labeling for domain-specific retraining. Once done, inference cost is standard and no external retrieval needed.Strong domain adaptation with high accuracy and low latency during inference since no external lookup is needed.</p><p>RAG: Incur moderate initial cost, mainly from setting up and maintaining retrieval infrastructure (embedding models, vector databases). Ongoing costs incur with each query retrieval. Excels at providing up-to-date responses however, retrieval adds latency, making it slower than fine-tuned models.</p><h3>Common mistakes to avoid:</h3><p>Fine-Tuning Pitfalls:</p><ul><li>Training on small datasets, this will model more rigid.</li><li>Forget to test or train the model on fresh information.</li><li>Ignoring key details like tuning settings and prompt formats can mess with model’s stability and output quality.</li></ul><p>RAG Pitfalls:</p><ul><li>Thinking RAG gives the same pinpoint accuracy as fine-tuning is risky, it depends heavily on how good your retrieval data is.</li><li>Assuming retrieval won’t slow things down adds complexity and can affect response times.</li></ul><h3>Conclusion:</h3><p>Choosing between Fine-Tuning and RAG depends on your specific and business use case:</p><ul><li><strong>Fine-Tuning</strong> offers precision and speed for static and domains specific knowledge.</li><li><strong>RAG</strong> provides flexibility and freshness for updated environments.</li><li><strong>Hybrids </strong>unlocks best of both worlds.</li></ul><p>Fine-Tuning vs. RAG isn’t just a technical debate — it’s shaping the future of how we build smart, responsive and modern systems. Which approach do you think wins in real-world scenarios?</p><p>Let’s hear your take! Drop your thoughts in the comments — your insight could help others make smarter choices too.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d54ed6f11920" width="1" height="1" alt=""><hr><p><a href="https://blog.gopenai.com/fine-tuning-vs-retrieval-augmented-generation-rag-d54ed6f11920">Fine-Tuning vs Retrieval Augmented Generation (RAG)</a> was originally published in <a href="https://blog.gopenai.com/">GoPenAI</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>