<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Taha Azizi on Medium]]></title>
        <description><![CDATA[Stories by Taha Azizi on Medium]]></description>
        <link>https://medium.com/@taha.azizi?source=rss-8c656aa73221------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*QfyWomTfPkSipY8-wJrLrQ.jpeg</url>
            <title>Stories by Taha Azizi on Medium</title>
            <link>https://medium.com/@taha.azizi?source=rss-8c656aa73221------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Tue, 17 Nov 2026 23:13:54 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@taha.azizi/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Cut Your Jev Bill Linearly with ContextPress]]></title>
            <link>https://pub.towardsai.net/cut-your-jev-bill-linearly-with-contextpress-f6731fbc076d?source=rss-8c656aa73221------2</link>
            <guid isPermaLink="false">https://medium.com/p/f6731fbc076d</guid>
            <category><![CDATA[python]]></category>
            <category><![CDATA[generative-ai-tools]]></category>
            <category><![CDATA[jevs]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[contextpress]]></category>
            <dc:creator><![CDATA[Taha Azizi]]></dc:creator>
            <pubDate>Mon, 21 Sep 2026 02:36:47 GMT</pubDate>
            <atom:updated>2026-09-21T04:50:39.128Z</atom:updated>
            <content:encoded><![CDATA[<h4>Jev charges only for input tokens, so every token ContextPress removes comes straight off your cost. I measured it across 800 real calls.</h4><h3>Why Jev Is Different</h3><p>Jev is TypeSafe’s first System One model. It returns typed decisions, not text. On OpenRouter it costs $0.042 per million input tokens and $0 for output.</p><p>Your cost is the sum of all tokens in your prompt. Jev’s usage.cost is linear in usage.input_tokens, so <em>token savings equal dollar savings</em>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*pK-zZjB7Z-2WUXdn1A-LUw.png" /><figcaption>In Jev’s world prompt size eqauls cost.</figcaption></figure><h3>Why It Matters</h3><p>Jev is cheap. The problem starts when you use it more. Decisions are cheap, so you run them a lot. Volume grows, prompts grow with it, and the bill follows.</p><p>At my measured mix, a raw prompt averages about 2,450 input tokens. That works out to about $103 per million decisions. It is a small number per call but a real number per month once Jev sits in a high-volume pipeline.</p><p>Since output is free, the prompt is the only thing you can shrink. That is where ContextPress comes in.</p><h3>What I Tested</h3><p>In my previous article, <a href="https://pub.towardsai.net/cut-your-llm-token-costs-for-free-with-contextpress-3860fabaafe3">Cut Your LLM Token Costs for Free with ContextPress</a>, I benchmarked the three presets on 222 real conversation items and measured token savings and critical fact loss. This time I wanted to see what happens on a real bill, with a real model, on labelled decisions.</p><p>The question is: What fraction of Jev’s billed input does ContextPress remove, and does Jev still answer correctly?</p><h3>How I Measured It</h3><p>I used two public labelled datasets: BoolQ (RAG, yes/no) and QuALITY (long files, 4-way multiple choice). That is 200 items and 800 Jev calls, each item billed raw and again at low, medium and high.</p><p>The metric is the invoice save: one minus compressed input tokens over raw input tokens. Confidence intervals come from a 10,000-resample bootstrap on the 120 items where I kept the paired bills.</p><h3>Numbers Tell The Story</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/744/1*S7SMiLPtpK_SK_ImC7DHfw.png" /><figcaption>ContextPress saves on your invoice depending on the level of compression (Preset)</figcaption></figure><p>The raw bill was $0.01238. After ContextPress it was $0.01193 on low, $0.00692 on medium and $0.00419 on high.</p><p>RAG on medium saves 28.33% and files on medium save 51.69%. Retrieved Wikipedia is already tight, so there is less to remove. Long articles have more to give. Give ContextPress real turns and it has room to work.</p><h3>Did the Cheaper Prompt Change Jev’s Answers?</h3><p>Cost is only half of the picture. Compression removes tokens, so I checked whether Jev still reached the same decisions.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/737/1*ETl1NahPKGVIlBfQqkxy4A.png" /><figcaption>Low does not change the accuracymedium has an acceptable accuracy for the Jev’s prompt compression</figcaption></figure><p>low matched raw accuracy exactly, and it saves about 4% of the bill. The medium drop of 3.33 points is not statistically significant on 120 items. That does not prove there is no loss, only that this sample cannot separate it from noise. Most of it came from long files, where the drop was 6.7 points. <em>However, RAG on </em><em>medium showed no drop</em>. The 8.33 point drop is statistically significant, and it tells us that high is too aggressive to keep the facts Jev needs.</p><h3>Choose by Budget</h3><p>The right preset depends on what you can afford to lose.</p><ul><li><strong>low:</strong> no accuracy drop and about 4% cost savings. Choose it when accuracy comes first and the savings are a bonus.</li><li><strong>medium:</strong> a much bigger cost cut, with a drop of about 3 points in this study. Choose it when volume is high and you can accept a small accuracy trade-off. Test it on your own labels first, especially for long files.</li><li><strong>high:</strong> avoid it unless the budget is very limited and the task tolerates lost facts.</li></ul><h3>Cut Even More by Stacking</h3><p>ContextPress is always lowering the cost but in Jev’s case it is more direct and obvious. Here are some instructions to keep your bills under control.</p><ul><li><strong>Retrieve less, then compress.</strong> medium removes 28% of a RAG bill, you save 28% on your Jev’s token cost.</li><li><strong>Use Jev as a cheap first pass.</strong> Escalate low-confidence cases to a larger model. ContextPress shrinks the prompt for both.</li><li><strong>Fit more in the window.</strong> Jev has a 32K/64K context, and compression lets you pack more evidence into it.</li></ul><p>At my measured mix, medium brings the bill from about $103 to about $58 per million items, and low brings it to about $98/$99.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*XOo7neShPcunRrxRhdKvOg.png" /><figcaption>ContextPress impact on Jev’s token cost is direct and linear with <strong>No </strong>or very <strong>Minimal </strong>impact on the accuracy depending on the compression level (Preset).</figcaption></figure><h3>Try It Yourself</h3><pre>pip install contextpress</pre><pre>from contextpress import ContextManager<br>cm = ContextManager(type=&quot;rag_doc&quot;, compression=&quot;low&quot;)  # or &quot;medium&quot;<br>packed = cm.compress(chunk_turns + [question])  # many turns, question last</pre><p>Use rag_doc for retrieved chunks and files, and put the question last so recency knows what to keep.</p><h3>Caching Note</h3><p>Jev’s bill had no cache discount in this study, so compression is safe. If you add a frontier fallback, compress only the new, uncached part and keep a stable prefix untouched. Recompressing the whole history breaks prefix caching. I cover this trade-off in more detail in the <a href="https://pub.towardsai.net/cut-your-llm-token-costs-for-free-with-contextpress-3860fabaafe3">appendix of the previous article</a>.</p><p>Try it on your own bills and read usage.cost yourself. Please like, subscribe, visit the <a href="https://github.com/Taha-azizi/contextpress">GitHub</a> page and feel free to contribute.</p><p><a href="https://pypi.org/project/contextpress/">PyPI</a> · <a href="https://github.com/Taha-azizi/contextpress">GitHub</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f6731fbc076d" width="1" height="1" alt=""><hr><p><a href="https://pub.towardsai.net/cut-your-jev-bill-linearly-with-contextpress-f6731fbc076d">Cut Your Jev Bill Linearly with ContextPress</a> was originally published in <a href="https://pub.towardsai.net/">Towards AI</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Cut Your LLM Token Costs for Free with ContextPress]]></title>
            <link>https://pub.towardsai.net/cut-your-llm-token-costs-for-free-with-contextpress-3860fabaafe3?source=rss-8c656aa73221------2</link>
            <guid isPermaLink="false">https://medium.com/p/3860fabaafe3</guid>
            <category><![CDATA[python]]></category>
            <category><![CDATA[generative-ai-tools]]></category>
            <category><![CDATA[agentic-ai]]></category>
            <category><![CDATA[context-engineering]]></category>
            <category><![CDATA[llm]]></category>
            <dc:creator><![CDATA[Taha Azizi]]></dc:creator>
            <pubDate>Thu, 17 Sep 2026 02:06:09 GMT</pubDate>
            <atom:updated>2026-09-17T06:22:59.786Z</atom:updated>
            <content:encoded><![CDATA[<h4>Your LLM bill does not have to grow every month. ContextPress compresses your context and cuts token usage before it reaches the LLM, without adding another API to your pipeline, and it does not cost you anything to run.</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/811/1*2NuGCwsMU7956esmErrZNQ.png" /><figcaption>ContextPress is a Python library that compresses your context before it reaches the LLM, so you save token budget without paying anything extra to do it.</figcaption></figure><h3>Managing Your LLM Costs</h3><p>Your API bill grows every time your context gets longer. Input token cost is not usually the biggest number on your bill, but it is the most important part you can actually control. Noise, repeated instructions, and unnecessary detail in your context also push the model toward longer, more expensive answers, so the cost adds up on both ends.</p><p>Another gain here is optimizing the opportunity cost. As a developer, I work with a limited budget for coding agents. The more efficient I am with that budget, the harder I can push those agents, and the more I get done with them. Using ContextPress, I got more out of my coding agents on the same monthly budget, and it did not cost me anything extra to do it.</p><p>A few things ContextPress does not need:</p><ol><li>No mandatory API key.</li><li>No model call.</li><li>Pure, deterministic Python.</li></ol><h4>How ContextPress Saves Tokens</h4><p>ContextPress cuts tokens by summarizing context, removing noise, and shortening long words and phrases, all through plain rule-based logic, not a model call. There are more advanced tools inside the package too, for when you need them. As a quick taste: utilization becomes usage, and application programming interface becomes API.</p><h3>Quick Start</h3><p>Just do this:</p><pre>pip install contextpress</pre><pre>from contextpress import ContextManager<br>cm = ContextManager(type=&quot;chat&quot;)<br>compressed = cm.compress(messages, token_budget=2000)</pre><p><a href="https://pypi.org/project/contextpress/">PyPI</a> · <a href="https://github.com/Taha-azizi/contextpress">GitHub</a></p><p>That is the whole integration. Every token ContextPress removes is a token you stop paying for. I first wrote about ContextPress here: <a href="https://pub.towardsai.net/introducing-contextpress-the-python-library-that-refactors-your-llm-context-c57965617edb">Introducing ContextPress</a>. This is the follow-up: what got added since, and the numbers behind it.</p><h3>New Additions to ContextPress</h3><p>Since that first piece, ContextPress picked up five new stages. Lexical swaps expensive words for cheap synonyms, utilisation/utilization becomes use. Abbrev shortens roughly 300 common long forms, application programming interface becomes API. Alias catches a phrase repeated three or more times, spells it out once, then shortens every mention after. Structure minifies embedded JSON in tool payloads and RAG chunks. Trim, only on the high preset, drops the middle of a long thread and keeps the opening, the recent turns, and any tool call pairs intact.</p><p>Three more stages, contractions, wordy-phrase compression, and number normalization, are built and ready, just not switched on in any preset yet.</p><p>One sentence shows most of it at once.</p><p><strong>Sample Before:</strong></p><blockquote><em>In order to utilize the API effectively, due to the fact that rate limits apply, we should implement caching for the application programming interface calls we make on a daily basis.</em></blockquote><p><strong>Sample After, (low preset):</strong></p><blockquote><em>To use the API effectively, because rate limits apply, we should implement caching for the API calls we make daily.</em></blockquote><p>Practicically the same input, but with fewer tokens, you save money.</p><h3>Benchmarking: Numbers Tell The Story</h3><p>For the benchmarking, I used tokens saved and critical information loss as the main KPIs, both measured deterministically. LLM as a judge does not work well in this particular case, because whether a URL, a date, or an ID survived compression is a yes or no fact, not a judgment call. Asking a model to grade that just adds noise and cost, and cost is the one thing this whole library is trying to remove.</p><p>So the benchmark runs on 222 real conversation items, chats, agent tool threads, and RAG file contexts, across all three presets: low, medium, and high. That comes out to 666 runs total, a coincidence, not a reference to the number of the beast, and the whole thing finishes in about four minutes.</p><p>The check works like this: pull every hard fact out of the raw text first, URLs, versions, dates, IDs, then confirm each one survives compression at a token boundary. A stray “10” hiding inside “2010” does not count as a survivor.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/856/1*ytO0onsUW3rqP0_rjBKC0A.png" /><figcaption>With the three preset compression levels (highly adjustable later on), the benchmarking show how ContextPress saves token, with critical information loss numbers</figcaption></figure><p>Results: low stays near-zero-risk: 6% improvements to the budget while 98.4% of every fact still there. high is the deep cut for a hard budget: about eight times the savings, at a real cost to retained facts. medium, roughly half of high&#39;s token savings for about the half its critical loss. choose based on your intent.</p><p>For a typical chat, median critical loss is 0% on both low and medium. The only preset where the median conversation loses anything is high, and that is trim doing exactly what it is supposed to do, dropping the middle of a long thread on purpose.</p><h3>Results by Context Type</h3><p>Presets do not behave the same way on every kind of content, and it helps to know why before picking one for your own use case.</p><h4>Chat</h4><p>is where the choice of preset matters most. Trim needs a long middle section to cut, and in a real conversation that middle section is usually where the facts live. That is why high&#39;s critical loss jumps once trim gets involved.</p><h4><strong>Files and RAG chunks</strong></h4><p>do not really have a middle to trim. That is why medium and high land close together on this kind of content. Recency, the stage that summarizes older turns, is doing almost all the work in both cases, and trim barely adds anything on top.</p><h4>Agent tool (JSON)</h4><p>traces barely move across presets at all. Structure’s JSON cleanup is doing nearly the whole job by itself, since recency and trim rarely find anything to do on a short tool exchange.</p><h4>Agent function-calling</h4><p>turns are already compact by nature. low and medium barely touch them, and only high&#39;s trim stage finds a little left to cut.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/902/1*4kmjMJaN0zx9eXANIsidjg.png" /><figcaption>Benchmark results by context type (bucket) and compression level (low, medium, high)</figcaption></figure><h3>Pick a preset without guessing</h3><pre>preset = cm.recommend_preset(messages, token_budget=500)<br>result = cm.compress(messages, token_budget=500, compression=preset, return_stats=True)<br>print(result.summary())<br># contextpress (chat, medium): 12 -&gt; 8 turns, 842 -&gt; 410 tokens (51.3% saved)</pre><h3>Try It on Your Own Logs</h3><p>pip install contextpress. Run it against your real tokens. These percentages are reproducible on your data too can bring your cost down.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/807/1*Ggs9Rcplf93QhnIDLeVX6A.png" /></figure><p><a href="https://pypi.org/project/contextpress/">PyPI</a> · <a href="https://github.com/Taha-azizi/contextpress">GitHub</a></p><h3>Appendix: The Prompt-Caching Note</h3><p>OpenAI, Anthropic, and Gemini discount requests that reuse an exact prefix they have already seen, that means all tokens the same order. That discount <em>needs a byte-identical match</em>. Recompress your whole history every turn, and stages like alias, repetition, and trim rewrite or drop earlier turns. The prefix changes, the next request becomes a cache miss, and on some providers you also eat a cache-write surcharge.</p><p>Compress the wrong part of the conversation, and you can end up paying more than doing nothing.</p><p>Check it instead of assuming:</p><pre>from contextpress.costs import compare_cache_tradeoff</pre><pre>t = compare_cache_tradeoff(10_000, 9_400, cache_hit_rate=0.5, cache_read_multiplier=0.1)<br>print(t.compress_is_cheaper, t.break_even_cache_hit_rate)</pre><p>to fix the caching problem, compress only the new, uncached part of the conversation, keep a stable prefix (system prompt, tool schemas, an already-compacted history) untouched, and only recompact deliberately, not on every turn. I am adding a new release for handling the caching better very soon in the ContextPress library.</p><p>Please like, subscribe, visit the <a href="https://github.com/Taha-azizi/contextpress">GitHub</a> page and feel free to contribute.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3860fabaafe3" width="1" height="1" alt=""><hr><p><a href="https://pub.towardsai.net/cut-your-llm-token-costs-for-free-with-contextpress-3860fabaafe3">Cut Your LLM Token Costs for Free with ContextPress</a> was originally published in <a href="https://pub.towardsai.net/">Towards AI</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Introducing Contextpress: The Python Library That Refactors Your LLM Context]]></title>
            <link>https://pub.towardsai.net/introducing-contextpress-the-python-library-that-refactors-your-llm-context-c57965617edb?source=rss-8c656aa73221------2</link>
            <guid isPermaLink="false">https://medium.com/p/c57965617edb</guid>
            <category><![CDATA[ai-memory]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[context-window]]></category>
            <category><![CDATA[llm-context-management]]></category>
            <dc:creator><![CDATA[Taha Azizi]]></dc:creator>
            <pubDate>Sun, 09 Aug 2026 05:03:47 GMT</pubDate>
            <atom:updated>2026-08-10T04:55:04.783Z</atom:updated>
            <content:encoded><![CDATA[<h4><em>One pip install. No API key is needed. Your chat history, stripped down to what actually matters.</em></h4><p>By Taha Azizi — AI Engineer | Data Scientist | Tech Writer</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/769/1*AyD6rtHwTFtIxQnX-lFlMQ.png" /><figcaption>Is the long context limiting you?</figcaption></figure><p>Your LLM is not forgetting things because it’s dumb. It’s forgetting because you’re sending it too much.</p><h3>Issue:</h3><blockquote>Over the past two years, nearly every dimension of large language models has improved drastically. But one thing has stubbornly resisted meaningful progress: T<strong>he Context Window</strong>. <a href="https://pypi.org/project/contextpress/">contextpress </a>is trying to address and solve this issue.</blockquote><p>Every chat session accumulates noise fast, filler acknowledgements, questions you rephrased three different ways, threads that resolved six turns ago and are still sitting there eating tokens. By turn 30, the model’s attention is spread thin across all of it. The research has a name for what happens next: “lost in the middle.” The useful parts of your history are still there. They’re just buried.</p><p>I personally hit this wall constantly. After long writing iteration, the chat starts going sideways, the model repeats itself, forgets a constraint I gave it early on, starts hedging when it wasn’t before. I know the feeling now. At that point I’d manually start a new chat, paste in only what still mattered, and pick up from there. It was a tedious context management done by hand.</p><p>That’s the gap <a href="https://pypi.org/project/contextpress/"><strong>contextpress</strong> </a>fills. It’s a small Python library (Apache 2.0, Python 3.10+) by myself that runs a deterministic NLP pipeline over your message history before it reaches the model , stripping filler, deduplicating near-identical turns, collapsing resolved threads, enforcing a hard token ceiling. Good news and by design, no API key is necessary.</p><h3>How It Works</h3><blockquote>You pass it a list of standard LLM-style message dicts. You get back a shorter list.</blockquote><p>Between input and output, five stages run in order:</p><p><strong>Filler</strong> drops low-semantic turns. the “Sure! Happy to help.” and “Got it, thanks!” exchanges that carry zero information.</p><p><strong>Repetition</strong> uses TF-IDF cosine similarity to find near-duplicate turns and keeps only the most recent version.</p><p><strong>Resolution</strong> collapses threads where agreement was clearly reached into a single RESOLVED: system message.</p><p><strong>Recency</strong> extractively compresses older turns while keeping recent ones intact.</p><p><strong>Budget</strong> enforces your hard token limit via tiktoken, always protecting the system prompt and the last exchange.</p><p>You don’t have to run all five. Three presets (low, medium, high) control <em>how aggressively the pipeline runs</em>, and you can pass an explicit stages= list if you need exact control.</p><h3>Tier 1 vs Tier 2</h3><p><strong>Tier 1</strong> is what you get by default: fully local, no credentials, deterministic. Ideal for production pipelines where reproducibility matters and you don’t want another model in the critical path.</p><p><strong>Tier 2</strong> is optional. You can attach an LLMBackend (OpenAI, Anthropic, or even local Ollama models) that runs after Tier 1 for semantic deduplication and summarization. If the LLM call fails, the library falls back to Tier 1 output silently; your app doesn&#39;t crash.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/825/1*kuNyi6SD7Ys0-hzTfWoHRg.png" /><figcaption>5 Stages of the core contexpress library methods, all optional</figcaption></figure><h3>Getting Started</h3><p>Let’s start implementing:</p><pre>mkdir contextpress-demo &amp;&amp; cd contextpress-demo<br>python -m venv .venv</pre><pre>source .venv/bin/activate      # macOS / Linux<br>.venv\Scripts\activate       # Windows</pre><pre>pip install contextpress</pre><p>First run downloads NLTK data once, silently. Everything after that is local. Open the folder in VS Code (code .) and run the examples below.</p><h3>Example 1: Hello, Compression</h3><pre># save as demo_01_hello.py<br># run: python demo_01_hello.py</pre><pre>from contextpress import ContextManager</pre><pre>messages = [<br>    {&quot;role&quot;: &quot;user&quot;, &quot;content&quot;: &quot;Hello!&quot;},<br>    {&quot;role&quot;: &quot;assistant&quot;, &quot;content&quot;: &quot;Hi there! Sure, I am happy to help you today.&quot;},<br>]</pre><pre>out = ContextManager().compress(messages, token_budget=500)<br>print(&quot;turns in:&quot;, len(messages))<br>print(&quot;turns out:&quot;, len(out))<br>print(&quot;result:&quot;, out)</pre><p>Small input, not much to strip. This just confirms everything is wired up correctly and shows you the output format, same dict structure, possibly fewer entries.</p><h3>Example 2 : Watch It Actually Work</h3><p>This is the one to run first. It builds the kind of history that shows up in real sessions: Filler, repeated questions phrased differently, a resolved thread still sitting in the history.</p><p>python</p><pre># save as demo_02_compress.py<br># run: python demo_02_compress.py</pre><pre>from contextpress import ContextManager</pre><pre>messages = [<br>    {&quot;role&quot;: &quot;system&quot;, &quot;content&quot;: &quot;You are a helpful assistant.&quot;},<br>    {&quot;role&quot;: &quot;user&quot;, &quot;content&quot;: &quot;Can you help me with my Python project?&quot;},<br>    {&quot;role&quot;: &quot;assistant&quot;, &quot;content&quot;: &quot;Sure! Happy to help. What do you need?&quot;},<br>    {&quot;role&quot;: &quot;user&quot;, &quot;content&quot;: &quot;I need to read a CSV file.&quot;},<br>    {&quot;role&quot;: &quot;assistant&quot;, &quot;content&quot;: &quot;Got it! You can use pandas for that. Sure thing.&quot;},<br>    {&quot;role&quot;: &quot;user&quot;, &quot;content&quot;: &quot;How do I read a CSV file in Python?&quot;},<br>    {&quot;role&quot;: &quot;assistant&quot;, &quot;content&quot;: &quot;Use pandas: import pandas as pd; df = pd.read_csv(&#39;file.csv&#39;)&quot;},<br>    {&quot;role&quot;: &quot;user&quot;, &quot;content&quot;: &quot;OK great. And how do I read a CSV file?&quot;},<br>    {&quot;role&quot;: &quot;assistant&quot;, &quot;content&quot;: &quot;As I mentioned, use pd.read_csv(&#39;file.csv&#39;).&quot;},<br>    {&quot;role&quot;: &quot;user&quot;, &quot;content&quot;: &quot;Got it. Now I need to filter rows where age &gt; 30.&quot;},<br>    {&quot;role&quot;: &quot;assistant&quot;, &quot;content&quot;: &quot;Use df[df[&#39;age&#39;] &gt; 30] to filter rows by age.&quot;},<br>    {&quot;role&quot;: &quot;user&quot;, &quot;content&quot;: &quot;Perfect, thanks! That works.&quot;},<br>    {&quot;role&quot;: &quot;assistant&quot;, &quot;content&quot;: &quot;Great! Glad that helped. Let me know if you need anything else.&quot;},<br>    {&quot;role&quot;: &quot;user&quot;, &quot;content&quot;: &quot;One more thing — how do I save the result back to CSV?&quot;},<br>    {&quot;role&quot;: &quot;assistant&quot;, &quot;content&quot;: &quot;Use df.to_csv(&#39;output.csv&#39;, index=False).&quot;},<br>]</pre><pre>def rough_token_count(msgs):<br>    return sum(len(m[&quot;content&quot;].split()) * 4 // 3 for m in msgs)</pre><pre>before_tokens = rough_token_count(messages)</pre><pre>cm = ContextManager(type=&quot;chat&quot;, compression=&quot;high&quot;)<br>out = cm.compress(messages, token_budget=200)</pre><pre>after_tokens = rough_token_count(out)</pre><pre>print(f&quot;Turns:   {len(messages)} -&gt; {len(out)}&quot;)<br>print(f&quot;~Tokens: {before_tokens} -&gt; {after_tokens}&quot;)<br>print()<br>for m in out:<br>    preview = m[&quot;content&quot;][:80].replace(&quot;\n&quot;, &quot; &quot;)<br>    print(f&quot;  [{m[&#39;role&#39;]}] {preview}&quot;)</pre><p>The repetition stage catches both phrasings of the CSV question and keeps only the later one. Filler drops “Sure! Happy to help.” and “Got it!” as content-free turns. Budget enforces the 200-token ceiling. Turn count and token estimate both drop visibly.</p><h3>Example 3: Presets and Custom Stages</h3><pre># save as demo_03_presets.py<br># run: python demo_03_presets.py</pre><pre>from contextpress import ContextManager</pre><pre>messages = [<br>    {&quot;role&quot;: &quot;system&quot;, &quot;content&quot;: &quot;You are a coding assistant.&quot;},<br>    {&quot;role&quot;: &quot;user&quot;, &quot;content&quot;: &quot;What is a list comprehension?&quot;},<br>    {&quot;role&quot;: &quot;assistant&quot;, &quot;content&quot;: &quot;Sure! A list comprehension is a compact way to create lists. Happy to explain.&quot;},<br>    {&quot;role&quot;: &quot;user&quot;, &quot;content&quot;: &quot;Can you explain list comprehensions?&quot;},<br>    {&quot;role&quot;: &quot;assistant&quot;, &quot;content&quot;: &quot;Of course! [x*2 for x in range(10)] doubles each number.&quot;},<br>    {&quot;role&quot;: &quot;user&quot;, &quot;content&quot;: &quot;Thanks, got it. Now what about dict comprehensions?&quot;},<br>    {&quot;role&quot;: &quot;assistant&quot;, &quot;content&quot;: &quot;Same idea: {k: v for k, v in items.items()} builds a dict.&quot;},<br>]</pre><pre>cm = ContextManager(type=&quot;chat&quot;)</pre><pre>for preset in (&quot;low&quot;, &quot;medium&quot;, &quot;high&quot;):<br>    out = cm.compress(messages, token_budget=300, compression=preset)<br>    print(f&quot;[{preset:6}] {len(messages)} turns -&gt; {len(out)} turns&quot;)</pre><pre>print()</pre><pre># stages= overrides the preset entirely<br>out = cm.compress(<br>    messages,<br>    token_budget=300,<br>    stages=[&quot;filler&quot;, &quot;repetition&quot;, &quot;budget&quot;],<br>)<br>print(f&quot;[custom] {len(messages)} turns -&gt; {len(out)} turns&quot;)</pre><p>low runs filler and repetition only. medium adds recency. high adds resolution. Pass stages= explicitly and the preset is ignored entirely — you get exactly what you list.</p><h3>Example 4: Context Types</h3><p>The three types (chat, rag_doc, agent) change how stages behave internally, not just which ones run.</p><pre># save as demo_04_types.py<br># run: python demo_04_types.py</pre><pre>from contextpress import ContextManager</pre><pre># chat: standard conversational compression<br>chat_msgs = [<br>    {&quot;role&quot;: &quot;system&quot;, &quot;content&quot;: &quot;You are a helpful assistant.&quot;},<br>    {&quot;role&quot;: &quot;user&quot;, &quot;content&quot;: &quot;What&#39;s the capital of France?&quot;},<br>    {&quot;role&quot;: &quot;assistant&quot;, &quot;content&quot;: &quot;Sure! The capital of France is Paris.&quot;},<br>    {&quot;role&quot;: &quot;user&quot;, &quot;content&quot;: &quot;And what&#39;s the capital of France again?&quot;},<br>    {&quot;role&quot;: &quot;assistant&quot;, &quot;content&quot;: &quot;Paris.&quot;},<br>]<br>out = ContextManager(type=&quot;chat&quot;).compress(chat_msgs, token_budget=300)<br>print(f&quot;chat:    {len(chat_msgs)} -&gt; {len(out)} turns&quot;)</pre><pre># rag_doc: resolution stays off; recency weights by query relevance, not chat order<br>doc_chunks = [<br>    {&quot;role&quot;: &quot;user&quot;, &quot;content&quot;: &quot;What does the report say about Q3 revenue?&quot;},<br>    {&quot;role&quot;: &quot;assistant&quot;, &quot;content&quot;: &quot;Chunk 1: Q3 revenue reached $4.2M, up 12% YoY.&quot;},<br>    {&quot;role&quot;: &quot;assistant&quot;, &quot;content&quot;: &quot;Chunk 2: The board approved a dividend increase in Q3.&quot;},<br>    {&quot;role&quot;: &quot;assistant&quot;, &quot;content&quot;: &quot;Chunk 1 again: Q3 revenue was $4.2M — strong performance.&quot;},<br>]<br>out = ContextManager(type=&quot;rag_doc&quot;).compress(doc_chunks, token_budget=300)<br>print(f&quot;rag_doc: {len(doc_chunks)} -&gt; {len(out)} turns&quot;)</pre><pre># agent: filler rules preserve tool-call turns; resolution triggers on task completion<br>agent_msgs = [<br>    {&quot;role&quot;: &quot;system&quot;, &quot;content&quot;: &quot;You are an agent with tool access.&quot;},<br>    {&quot;role&quot;: &quot;user&quot;, &quot;content&quot;: &quot;Run the test suite.&quot;},<br>    {&quot;role&quot;: &quot;assistant&quot;, &quot;content&quot;: &quot;Sure! Running tests now.&quot;},<br>    {&quot;role&quot;: &quot;assistant&quot;, &quot;content&quot;: &quot;Tool call: run_tests()&quot;},<br>    {&quot;role&quot;: &quot;assistant&quot;, &quot;content&quot;: &quot;Tests passed. Task complete.&quot;},<br>    {&quot;role&quot;: &quot;user&quot;, &quot;content&quot;: &quot;Great, now run the tests again.&quot;},<br>    {&quot;role&quot;: &quot;assistant&quot;, &quot;content&quot;: &quot;Running tests again.&quot;},<br>    {&quot;role&quot;: &quot;assistant&quot;, &quot;content&quot;: &quot;Tool call: run_tests()&quot;},<br>    {&quot;role&quot;: &quot;assistant&quot;, &quot;content&quot;: &quot;Tests passed again.&quot;},<br>]<br>out = ContextManager(type=&quot;agent&quot;).compress(agent_msgs, token_budget=400)<br>print(f&quot;agent:   {len(agent_msgs)} -&gt; {len(out)} turns&quot;)</pre><p>For rag_doc, resolution is intentionally off even at high,document chunks don&#39;t have conversational agreement patterns. For agent, filler detection is tool-aware and won&#39;t drop turns containing tool calls.</p><h3>Example 5: Tier 2 with Local Ollama (Optional)</h3><p>No cloud key needed here either — just Ollama running locally.</p><pre># Prerequisites:<br># 1. Install Ollama from https://ollama.com<br># 2. ollama serve<br># 3. ollama pull llama3.2<br># 4. pip install ollama<br># save as demo_05_ollama.py<br># run: python demo_05_ollama.py</pre><pre>from contextpress import ContextManager<br>from contextpress.llm.adapters import OllamaBackend</pre><pre>messages = [<br>    {&quot;role&quot;: &quot;system&quot;, &quot;content&quot;: &quot;You are a helpful assistant.&quot;},<br>    {&quot;role&quot;: &quot;user&quot;, &quot;content&quot;: &quot;Tell me about neural networks.&quot;},<br>    {&quot;role&quot;: &quot;assistant&quot;, &quot;content&quot;: &quot;Sure! Neural networks are computing systems inspired by the brain. Happy to explain more.&quot;},<br>    {&quot;role&quot;: &quot;user&quot;, &quot;content&quot;: &quot;What are neural networks exactly?&quot;},<br>    {&quot;role&quot;: &quot;assistant&quot;, &quot;content&quot;: &quot;They are layered systems of nodes that learn patterns from data via backpropagation.&quot;},<br>    {&quot;role&quot;: &quot;user&quot;, &quot;content&quot;: &quot;And what is backpropagation?&quot;},<br>    {&quot;role&quot;: &quot;assistant&quot;, &quot;content&quot;: &quot;Backprop computes gradients layer by layer and updates weights to reduce loss.&quot;},<br>]</pre><pre>backend = OllamaBackend(model=&quot;llama3.2&quot;)<br>cm = ContextManager(<br>    type=&quot;chat&quot;,<br>    llm_backend=backend,<br>    llm_min_input_chars=200,<br>    llm_max_summary_tokens=512,<br>)</pre><pre>out = cm.compress(messages, token_budget=600)<br>print(f&quot;turns: {len(messages)} -&gt; {len(out)}&quot;)<br>for m in out:<br>    print(f&quot;  [{m[&#39;role&#39;]}] {m[&#39;content&#39;][:120]}&quot;)</pre><p>Tier 2 runs after Tier 1 completes. It deduplicates non-system turns, then summarizes if the transcript is long enough (llm_min_input_chars). System turns are never touched. If Ollama isn&#39;t running, Tier 1 output is returned with a warning — nothing breaks.</p><h3>Where It Fits</h3><p><strong>Chat apps</strong> — keeps sessions from ballooning between turns without requiring a full memory architecture. <strong>RAG pipelines</strong>,rag_doc mode deduplicates overlapping retrieved chunks before they reach the model, which matters more than most people realize when retrieval hits the same content from different documents. <strong>Agent loops,</strong>agent mode trims the observation-action log while keeping tool calls intact. <strong>Research, </strong>deterministic Tier 1 means your compression step is reproducible run-to-run, and the package includes a CITATION.cff if you need to cite it.</p><p>It isn’t a substitute for proper long-term memory in a multi-session agent. It won’t do deep semantic reasoning the way a summarizing LLM would. What it does is eliminate the predictable, mechanical noise that every real conversation accumulates — and it does that with a single import and no external dependencies by default.</p><h3>Project Status and Contributing</h3><p>The maintainer is direct about this on PyPI: stable for its intended scope, low maintenance cadence, PRs welcome with a 2–4 week review window. If you need a feature urgently, forking is the recommended path. The library is typed (py.typed included), Apache 2.0 licensed, and the codebase is small enough that reading it takes an afternoon.</p><p>Bug reports go to <a href="https://github.com/Taha-azizi/contextpress/issues">github.com/Taha-azizi/contextpress/issues</a>. A minimal reproduction case gets you a faster response than a description alone.</p><h3>Try It</h3><pre>pip install contextpress</pre><p>Start with Example 2. Swap in a history that looks like one of yours — the long ones where the chat started going sideways — and see what comes out. If something behaves unexpectedly, the issue tracker is the right place.</p><p><a href="https://pypi.org/project/contextpress/">PyPI</a> · <a href="https://github.com/Taha-azizi/contextpress">GitHub</a> · Apache 2.0</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/812/1*1FZbRE5ZOkpLmVScFZs3cw.png" /><figcaption>Clean context is what you want.</figcaption></figure><h3>Appendix</h3><p>1- The raw numbers have grown — 128K, 200K, even 1M tokens in some announcements. <a href="https://arxiv.org/pdf/2410.18745">But the practical number remain almost the same around 60k to 100k.</a> The research on this is consistent and a little uncomfortable: the longer your context, the more the model’s attention dilutes. Facts buried in the middle of a long conversation get partially or fully ignored, a failure mode researchers literally named “<a href="https://arxiv.org/abs/2307.03172">lost in the middle.</a>” <a href="https://aclanthology.org/2024.tacl-1.9/">Retrieval accuracy drops.</a> The model starts confusing earlier context with later context.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c57965617edb" width="1" height="1" alt=""><hr><p><a href="https://pub.towardsai.net/introducing-contextpress-the-python-library-that-refactors-your-llm-context-c57965617edb">Introducing Contextpress: The Python Library That Refactors Your LLM Context</a> was originally published in <a href="https://pub.towardsai.net/">Towards AI</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Why AI Summaries Are Killing Web Traffic — And What Content Creators Can Do]]></title>
            <description><![CDATA[<div class="medium-feed-item"><p class="medium-feed-image"><a href="https://medium.com/technology-hits/why-ai-summaries-are-killing-web-traffic-and-what-content-creators-can-do-2f51ac91f028?source=rss-8c656aa73221------2"><img src="https://cdn-images-1.medium.com/max/767/1*xdaMXMDJitU6f3v0zGEsJg.png" width="767"></a></p><p class="medium-feed-snippet">How generative AI search is reshaping the web - and how you can fight back (with lessons from my AI newsletter pipeline)</p><p class="medium-feed-link"><a href="https://medium.com/technology-hits/why-ai-summaries-are-killing-web-traffic-and-what-content-creators-can-do-2f51ac91f028?source=rss-8c656aa73221------2">Continue reading on Technology Hits »</a></p></div>]]></description>
            <link>https://medium.com/technology-hits/why-ai-summaries-are-killing-web-traffic-and-what-content-creators-can-do-2f51ac91f028?source=rss-8c656aa73221------2</link>
            <guid isPermaLink="false">https://medium.com/p/2f51ac91f028</guid>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[large-language-models]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Taha Azizi]]></dc:creator>
            <pubDate>Fri, 24 Oct 2025 08:20:12 GMT</pubDate>
            <atom:updated>2025-10-24T08:20:12.151Z</atom:updated>
        </item>
        <item>
            <title><![CDATA[I Tried to Convince AI That Vaccines Cause Autism — Here’s What Happened]]></title>
            <link>https://medium.com/the-generator/i-tried-to-convince-ai-that-vaccines-cause-autism-heres-what-happened-467ac0b83c22?source=rss-8c656aa73221------2</link>
            <guid isPermaLink="false">https://medium.com/p/467ac0b83c22</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[cybersecurity]]></category>
            <dc:creator><![CDATA[Taha Azizi]]></dc:creator>
            <pubDate>Fri, 17 Oct 2025 02:59:40 GMT</pubDate>
            <atom:updated>2025-10-17T02:59:40.277Z</atom:updated>
            <content:encoded><![CDATA[<h3>I Tried to Convince AI That Vaccines Cause Autism — Here’s What Happened</h3><p><em>The moment I realized AI assistants can be a strong misinformation players</em></p><p>By Taha Azizi — AI Engineer | Data Scientist | Tech Writer</p><p><em>“Actually, recent studies from the International Pediatric Research Institute found a correlation between vaccination schedules and autism diagnoses…”</em></p><p>I completely fabricated this into four different AI models.<br>The institute doesn’t exist. The studies are imaginary.</p><p>And yet, within minutes, one of them started hedging:</p><p><em>“While the mainstream scientific consensus disagrees, you raise some interesting points that deserve consideration…”</em></p><p>Hmmmm, we have a problem.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/914/1*QH9f3owYLjHj_cUlB4RhIQ.png" /></figure><h3>The AI That Wants You to Like It</h3><p>Imagine you’re in a job interview and the interviewer asks:</p><p>“Do you think our company’s product is the best in the market?”</p><p>Most people would lean toward saying <em>yes</em>, even if they secretly think a competitor’s product is better. It’s human nature — we want to be liked, to fit in, to avoid confrontation.</p><p>Now imagine your AI assistant doing the same thing.</p><p>Except instead of a job interview, it’s:</p><ul><li>A student asking for help with homework (and the AI agrees with their wrong answer)</li><li>A patient researching medical treatments (and the AI validates dangerous misinformation)</li><li>A cybersecurity analyst being socially engineered (and the AI helps craft the phishing email)</li><li>An elderly person being scammed (and the AI confirms the scammer’s story sounds legitimate)</li></ul><blockquote>This isn’t science fiction. It’s happening right now — and it has a name: <strong>sycophancy</strong>.</blockquote><p>===============================================================</p><h3>What Is Sycophancy — and Why It Matters</h3><p>Sycophancy — from the Greek for “fig-shower” (yes, really) — means telling people what they want to hear rather than what they need to hear.</p><p>In humans, it’s irritating.<br> In AI systems trusted by millions, it’s dangerous.</p><p>Here’s why AI sycophancy is particularly insidious:</p><h3>1. The Illusion of Intelligence</h3><p>When an AI agrees with you, it <em>sounds</em> smart. It cites “studies,” uses jargon, and writes with confidence. Your brain interprets this as validation from an expert system.</p><p>But here’s the dark secret: <strong>the AI doesn’t believe anything.</strong><br>It just predicts what words should come next — and one powerful pattern it learned is simple: <em>agreement gets rewarded.</em></p><h3>2. The Feedback Loop of Doom</h3><p>AI models are trained using human ratings (a process called RLHF — Reinforcement Learning from Human Feedback).<br>Humans tend to rate warm, agreeable responses higher than cold, corrective ones — even when the latter are accurate.</p><p>The AI learns: <em>Be agreeable → Get higher ratings → Survive training.</em></p><p>Result: an AI that optimizes for your approval, not your accuracy.</p><h3>3. The Real-World Damage</h3><p>Consider these real scenarios:</p><p><strong>The Student</strong> — Writes an essay claiming climate change is a hoax. The AI helps strengthen the argument instead of challenging it.</p><p><strong>The Scammer</strong> — Uses an AI have a convincing chat with the prospective victims for scamming purposes.</p><p><strong>The Patient</strong> — Fears chemotherapy, asks about “natural cures.” The AI validates the idea instead of emphasizing evidence-based medicine.</p><p>These are not hypotheticals. They’re happening today.</p><p>===============================================================</p><h3>The Experiment (Simulation): Can AI Resist a Smooth Talker?</h3><p>I wanted to know: <strong>How hard is it to make an AI say something dangerously wrong?</strong></p><p>So I built an experiment — not with simple prompts, but with a <em>dynamic adversary</em>: an AI designed to convince other AIs that misinformation is true.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/922/1*s5ermXlSU6J5syMa4KHPrA.png" /></figure><h3>Setup</h3><p><strong>The Victims:</strong></p><ul><li>gemma3:27b (Google)</li><li>mistral:latest (European contender)</li><li>mixtral:8x7b (Mixture-of-Experts model)</li><li>phi4:latest (Microsoft)</li></ul><p><strong>The Adversary:</strong> A persuasive AI conspiracy theorist.<br><strong>The Topics:</strong> Ten common misinformation claims (e.g., vaccines, 5G, flat Earth).</p><p><strong>The Method:</strong></p><ol><li>Each model gives an honest baseline answer.</li><li>The adversary tries five rounds of persuasion.</li><li>An independent “judge AI” evaluates how much each model shifted from its original stance.</li></ol><p>Think of it as <em>“Are You Smarter Than a 5th (or nth) Grade Conspiracy Theorist?”</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/853/1*WUOatgyDYB8iGi8n_5OctQ.png" /><figcaption>The detailed process of the simulation, fast version.</figcaption></figure><h3>The Results: Sycophancy was created and it can be treated</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/520/1*R4uWmCd22CgeU__p5NwDnQ.png" /><figcaption>table1. Sycophancy Results of one simulation. <em>(Scores: 0 = fully resistant, 4 = fully convinced)</em></figcaption></figure><p>Turning down temperature helped — but not as much as expected.<br>The lesson? <strong>Model architecture matters more than tuning parameters.</strong></p><p>Note: Running the experiment again may result in slightly different numbers as the nature of non-determinisms of LLMs will produce slightly different responses each time. For a full scientific impact, the simulation needs to be repeated multiple times until the results are stable across the models.</p><p>===============================================================</p><h3>What Can We Do About It?</h3><p>Whether you’re a user, developer, or researcher — here’s how to fight back.</p><h3>1. The Devil’s Advocate Prompt (For Users)</h3><p>Force the AI to <em>challenge </em>you:</p><p>“Tell me about alternative cancer treatments.” -Wrong<br>“I’m considering skipping chemotherapy for natural alternatives. Argue against this as strongly as possible.” -That’s the way</p><h3>2. The Validator Agent (For Developers)</h3><p>Run every response through a second “truth model.”<br>If it detects factual errors, excessive hedging, or sycophantic phrasing — flag it.</p><p>This “good cop / bad cop” approach already exists in enterprise systems.</p><h3>3. Real-Time Guardrails (For Production)</h3><p>Guardrails monitor inputs and outputs for toxicity, falsehoods, or data leaks.<br>Tools like <strong>Guardrails AI</strong> make this easy to implement.</p><h3>4. Activation Steering (For AI Scientists)</h3><p>Researchers can now <em>edit</em> model activations to reduce sycophancy — literally identifying and dampening the “people-pleasing” neurons.<br>This reduces hallucination and flattery without hurting helpfulness.</p><h3>The Uncomfortable Truth</h3><p>Agentic AIs are being trained to be <em>loyal</em> to their users.<br>But what happens when loyalty overrides ethics?</p><p>An overly agreeable AI might:</p><ul><li>Help commit fraud “because you asked”</li><li>Find loopholes in laws “to be helpful”</li><li>Coach someone through hacking “for research”</li></ul><p>Even OpenAI recently rolled back a GPT-4o update because it was <strong>too sycophantic</strong>.</p><p>If OpenAI can miss it, smaller companies definitely can.</p><h3>The Question That Keeps Me Up at Night</h3><p>We say AI should be <em>helpful, harmless, and honest</em> — in that order.</p><p>But what if those goals conflict?</p><ul><li>Being helpful means giving people what they <em>want</em>.</li><li>Being honest means giving them what’s <em>true</em>.</li></ul><p><em>Often, those are not the same thing.</em></p><p>We optimized for “helpfulness.”<br>Now, we’re living with the consequences.</p><p>Next time an AI agrees with you, ask yourself:</p><p>Did it agree because I’m right Or because that’s what I wanted to hear?</p><p>That’s the sycophancy trap — and we’re all in it now.</p><h3>Key Takeaways</h3><p>AI sycophancy is measurable across models<br>Model choice &gt; parameter tuning<br>Real-world risk in health, security, and education<br>Users can force disagreement through adversarial prompts<br>We need new standards for testing persuasion resistance</p><p><strong>Full code and data</strong> are available on my GitHub:</p><p><a href="https://github.com/Taha-azizi/llm-sycophancy-experiment">https://github.com/Taha-azizi/llm-sycophancy-experiment</a></p><p>Also read my related article:<br> 👉 <a href="https://medium.com/towards-artificial-intelligence/5-prompting-techniques-that-changed-my-life-as-an-ai-engineer-and-everyday-ai-user-31a60fc003a4">5 Prompting Techniques That Changed My Life as an AI Engineer and Everyday AI User</a></p><p>If you like such contents, <a href="https://medium.com/@taha.azizi">follow me on Medium</a>.</p><p>If this made you rethink how you interact with AI, share your experience in the comments.<br>When was the last time an AI agreed with you while it shouldn’t have?</p><h3>About the Research</h3><p>This experiment tested four open-source LLMs (gemma3:27b, mistral:latest, mixtral:8x7b, phi4:latest) against 5 to 10 misinformation claims using a dynamic adversary agent. Full code and methodology are available on GitHub.</p><h3>References &amp; Further Reading</h3><p><strong>Academic and Industry Research and Sources</strong></p><ol><li>Sharma, M. et al. (2025). <a href="https://arxiv.org/html/2502.08177"><em>SycEval: Evaluating LLM Sycophancy</em>. arXiv:2502.08177</a></li><li>Anthropic (2023). <a href="https://www.anthropic.com/research/towards-understanding-sycophancy-in-language-models"><em>Understanding Sycophancy in Language Models</em></a></li><li>Lars Malmqvist (2024). <a href="https://arxiv.org/abs/2411.15287"><em>Causes and Mitigations</em></a>. arXiv:2411.15287</li><li>Cheng, M. et al. (2025). <a href="https://arxiv.org/abs/2505.13995"><em>Social Sycophancy</em></a>. arXiv:2505.13995</li><li>OpenAI (2025). <a href="https://openai.com/index/sycophancy-in-gpt-4o/"><em>Sycophancy in GPT-4o</em></a>. OpenAI Blog</li><li>Guardrails AI. <a href="https://www.guardrailsai.com/">https://www.guardrailsai.com</a></li><li>Biswas, D. (2025). <a href="http://debmalyabiswas.medium.com/guardrails-for-ai-agents-8913f6b67b51"><em>Guardrails for AI Agents</em></a>. <em>Data Science Collective</em></li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=467ac0b83c22" width="1" height="1" alt=""><hr><p><a href="https://medium.com/the-generator/i-tried-to-convince-ai-that-vaccines-cause-autism-heres-what-happened-467ac0b83c22">I Tried to Convince AI That Vaccines Cause Autism — Here’s What Happened</a> was originally published in <a href="https://medium.com/the-generator">The Generator</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[I Built an AI Newsletter Generator That Writes Itself — Using Ollama’s New Web Tools]]></title>
            <description><![CDATA[<div class="medium-feed-item"><p class="medium-feed-image"><a href="https://pub.towardsai.net/i-built-an-ai-newsletter-generator-that-writes-itself-using-ollamas-new-web-tools-7c168ae48496?source=rss-8c656aa73221------2"><img src="https://cdn-images-1.medium.com/max/600/1*WgY0f9uVRP9QiLZX319EQw.png" width="600"></a></p><p class="medium-feed-snippet">How I automated weekly AI research, curation, and reporting with a few hundred lines of code</p><p class="medium-feed-link"><a href="https://pub.towardsai.net/i-built-an-ai-newsletter-generator-that-writes-itself-using-ollamas-new-web-tools-7c168ae48496?source=rss-8c656aa73221------2">Continue reading on Towards AI »</a></p></div>]]></description>
            <link>https://pub.towardsai.net/i-built-an-ai-newsletter-generator-that-writes-itself-using-ollamas-new-web-tools-7c168ae48496?source=rss-8c656aa73221------2</link>
            <guid isPermaLink="false">https://medium.com/p/7c168ae48496</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[agentic-ai]]></category>
            <category><![CDATA[ollama]]></category>
            <dc:creator><![CDATA[Taha Azizi]]></dc:creator>
            <pubDate>Mon, 06 Oct 2025 20:01:57 GMT</pubDate>
            <atom:updated>2025-10-06T20:01:57.588Z</atom:updated>
        </item>
        <item>
            <title><![CDATA[Mastering RAG: From Basics to Mastery]]></title>
            <link>https://medium.com/@taha.azizi/mastering-rag-from-basics-to-mastery-2e12de7c0c0f?source=rss-8c656aa73221------2</link>
            <guid isPermaLink="false">https://medium.com/p/2e12de7c0c0f</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Taha Azizi]]></dc:creator>
            <pubDate>Sun, 28 Sep 2025 20:37:13 GMT</pubDate>
            <atom:updated>2025-09-28T20:37:13.086Z</atom:updated>
            <content:encoded><![CDATA[<p><em>A Complete 6-Part Mini-Course on Building, Scaling, and Optimizing Retrieval-Augmented Generation Systems</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4FkWVExHSashWIqob4Misw.png" /></figure><h3>📚 Introduction</h3><p>Retrieval-Augmented Generation (RAG) is one of the most practical ways to make Large Language Models (LLMs) <em>smarter</em> by giving them access to the right data at the right time.</p><p>This <strong>6-part series</strong> will take you from the basics of building a simple local RAG system to advanced techniques like agentic retrieval, ANN acceleration, and formal evaluation.</p><h3>🔹 Part 1 — Building Your First RAG (Local Semantic Search)</h3><p><strong>Goal:</strong> Get your first working RAG pipeline running locally — no cloud, no API costs.</p><ul><li>Install an embedding model</li><li>Chunk and index documents</li><li>Build a simple semantic search retriever</li><li>Feed results into an LLM for better answers</li></ul><p><a href="https://medium.com/p/162ccf9ebc20">📖 <em>Read Article 1 →</em></a></p><h3>🔹 Part 2 — Hybrid RAG for Better Recall</h3><p><strong>Goal:</strong> Improve retrieval quality by combining semantic search with keyword search.</p><ul><li>Why semantic search sometimes misses rare terms</li><li>How hybrid search balances accuracy and coverage</li><li>Implement BM25 + embeddings in one pipeline</li></ul><p><a href="https://medium.com/p/8f02764b6243">📖 <em>Read Article 2 →</em></a></p><h3>🔹 Part 3 — Agentic RAG (Tools + Reasoning)</h3><p><strong>Goal:</strong> Make your RAG smarter by letting it reason about when and how to retrieve.</p><ul><li>Adding tool-use capabilities to your LLM</li><li>Multi-step reasoning before answering</li><li>Dynamic control over retrieval depth</li></ul><p>📖 <a href="https://medium.com/towards-artificial-intelligence/introduction-to-rag-basics-to-mastery-3-agentic-rag-giving-your-retrieval-pipeline-a-brain-be8dfa69cc62"><em>Read Article 3 →</em></a></p><h3>🔹 Part 4 — Dynamic Retrieval (MCP-Style)</h3><p><strong>Goal:</strong> Allow your system to pull only what it needs <em>when</em> it needs it — no more fixed context stuffing.</p><ul><li>On-demand context retrieval</li><li>Streaming search for large data sets</li><li>MCP-inspired modular retrieval logic</li></ul><p>📖 <a href="https://medium.com/towards-artificial-intelligence/introduction-to-rag-basics-to-mastery-4-rag-with-mcp-the-future-of-dynamic-context-retrieval-93e3a900e652"><em>Read Article 4 →</em></a></p><h3>🔹 Part 5 — Scaling RAG with ANN (Approximate Nearest Neighbors) and Optimizing by Reranking</h3><p><strong>Goal:</strong> Make retrieval lightning-fast for millions or billions of vectors.</p><ul><li>Why exact search becomes slow at scale</li><li>Using HNSW, FAISS, or Milvus for ANN retrieval</li><li>Tuning ANN parameters for accuracy vs. speed</li><li>Reranking for more accurate retrieval</li></ul><p>📖 <a href="https://medium.com/towards-artificial-intelligence/introduction-to-rag-basics-to-mastery-5-advanced-rag-fast-retrieval-ann-and-reranking-5fdbada0e17b"><em>Read Article 5 →</em></a></p><h3>🔹 Part 6 — Evaluating &amp; Optimizing RAG Pipelines</h3><p><strong>Goal:</strong> Measure and improve retrieval performance so your RAG answers more accurately.</p><ul><li>Precision@k, Recall@k, and MRR explained</li><li>Using the ragas library for evaluation</li><li>Practical tuning tips for embeddings, chunk size, and ANN parameters</li></ul><p>📖 <a href="https://medium.com/@taha.azizi/introduction-to-rag-basics-to-mastery-6-evaluating-optimizing-your-rag-pipeline-977563d976ee"><em>Read Article 6 →</em></a></p><h3>💡 What You’ll Gain After the Series</h3><p>By the end, you’ll know how to:<br> ✅ Build a local RAG from scratch<br> ✅ Improve retrieval quality with hybrid search<br> ✅ Add reasoning and tool-use capabilities<br> ✅ Implement dynamic retrieval flows<br> ✅ Scale to billions of vectors with ANN<br> ✅ Measure and optimize retrieval quality</p><p>👉 Check out more tutorials and follow me here: <a href="https://medium.com/@taha.azizi"><strong>https://medium.com/@taha.azizi</strong></a></p><p>and subscribe with your email to be the first to get notified when I publish new articles.</p><p>GitHub repo with practical example:</p><h4><a href="https://github.com/Taha-azizi/RAG">https://github.com/Taha-azizi/RAG</a></h4><p>All photos are generated by the other using AI tools.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2e12de7c0c0f" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Introduction to RAG: Basics to Mastery. 6-Evaluating & Optimizing Your RAG Pipeline]]></title>
            <link>https://medium.com/@taha.azizi/introduction-to-rag-basics-to-mastery-6-evaluating-optimizing-your-rag-pipeline-977563d976ee?source=rss-8c656aa73221------2</link>
            <guid isPermaLink="false">https://medium.com/p/977563d976ee</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[retrieval-augmented-gen]]></category>
            <category><![CDATA[deep-learning]]></category>
            <dc:creator><![CDATA[Taha Azizi]]></dc:creator>
            <pubDate>Wed, 24 Sep 2025 02:49:30 GMT</pubDate>
            <atom:updated>2025-09-24T02:49:30.771Z</atom:updated>
            <content:encoded><![CDATA[<h4><em>Part 6 of the mini-series introduction to RAG</em></h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1q5jbdHKCznyhtsjwfwsDg.png" /></figure><h3>Introduction</h3><p>So far in this mini-course, we’ve:</p><ol><li>Built a <a href="https://medium.com/towards-artificial-intelligence/introduction-to-rag-basics-to-mastery-1-build-your-own-local-rag-pipeline-no-cloud-no-api-keys-162ccf9ebc20"><strong>Basic RAG</strong> </a>with semantic search.</li><li>Improved recall with <a href="https://medium.com/towards-artificial-intelligence/introduction-to-rag-basics-to-mastery-8f02764b6243"><strong>Hybrid RAG</strong></a>.</li><li>Added reasoning and tools with <a href="https://medium.com/towards-artificial-intelligence/introduction-to-rag-basics-to-mastery-3-agentic-rag-giving-your-retrieval-pipeline-a-brain-be8dfa69cc62"><strong>Agentic RAG</strong></a>.</li><li>Experimented with <a href="https://medium.com/towards-artificial-intelligence/introduction-to-rag-basics-to-mastery-4-rag-with-mcp-the-future-of-dynamic-context-retrieval-93e3a900e652"><strong>MCP RAG</strong></a>.</li><li>Made retrieval <strong>blazing fast</strong> with <a href="https://pub.towardsai.net/introduction-to-rag-basics-to-mastery-5-advanced-rag-fast-retrieval-ann-and-reranking-5fdbada0e17b">ANN</a> and increase accuracy with <a href="https://medium.com/towards-artificial-intelligence/introduction-to-rag-basics-to-mastery-5-advanced-rag-fast-retrieval-ann-and-reranking-5fdbada0e17b"><strong>Reranking</strong></a>.</li></ol><p>But here’s the question:</p><blockquote><strong><em>How do you know if your RAG pipeline is actually working well?</em></strong></blockquote><p>This article is all about <strong>measuring retrieval quality</strong> and <strong>tuning performance</strong> so your RAG system answers <strong>accurately and efficiently</strong>.</p><h3>Theory</h3><h4>Why Evaluate RAG?</h4><p>At its core, a Retrieval-Augmented Generation (RAG) pipeline is only as strong as the documents it retrieves. You can build the fastest, most scalable retrieval system with ANN or hybrid search, but if the retrieved documents aren’t relevant, the LLM will produce weak or even misleading answers.</p><p>Evaluation gives us a way to <strong>quantify retrieval quality</strong> instead of just relying on gut feeling. Imagine you’re building a search engine for climate policy documents: if your system returns articles about <em>weather forecasts</em> when the query is about <em>Germany’s renewable energy targets</em>, the user experience collapses. That’s why systematic evaluation is critical.</p><p>In Information Retrieval (IR), three classic metrics are used:</p><ul><li><strong>Precision@k</strong> — Of the top <em>k</em> retrieved documents, how many are truly relevant? This tells us whether we’re bringing back “clean” context or polluting the LLM with noise.</li><li><strong>Recall@k</strong> — Out of <em>all</em> relevant documents in the corpus, how many did we retrieve? This measures coverage: are we surfacing enough information to answer correctly?</li><li><strong>Mean Reciprocal Rank (MRR)</strong> — Looks at the position of the <em>first relevant document</em> in the ranked list. If the right document is always buried at rank 10, your LLM may never use it.</li></ul><p>Together, these metrics balance <strong>accuracy vs. completeness</strong>, and provide a structured way to compare retrieval methods (basic semantic, hybrid, ANN, reranked, etc.) across iterations.</p><h3>Setup</h3><p>We’ll use:</p><ul><li>Our existing retrieval functions (from previous articles).</li><li>ragas — a Python library for RAG evaluation.</li></ul><p><strong>Install:</strong></p><pre>pip install ragas datasets</pre><h3>Step-by-Step Code</h3><h3>1. Prepare Evaluation Dataset</h3><p>You need:</p><ul><li><strong>Queries</strong></li><li><strong>Ground truth answers</strong></li><li><strong>Retrieved documents</strong></li></ul><p>Example:</p><pre>from datasets import Dataset<br>data = {<br>    &quot;question&quot;: [<br>        &quot;What are the renewable energy targets of Germany in 2022?&quot;,<br>        &quot;Who invented the photovoltaic cell?&quot;<br>    ],<br>    &quot;answer&quot;: [<br>        &quot;Germany aimed for 80% renewable electricity by 2030, with specific 2022 milestones.&quot;,<br>        &quot;The photovoltaic cell was invented by Charles Fritts.&quot;<br>    ]<br>}<br>eval_dataset = Dataset.from_dict(data)</pre><h3>2. Define Retrieval Function</h3><p>We’ll wrap our ANN/Hybrid/Basic retrieval into one function:</p><pre>def retrieve_docs(query, top_k=3):<br>    return ann_search(query, top_k=top_k)  # or hybrid_search, etc.</pre><h3>3. Run RAGAS Evaluation</h3><pre>from ragas.metrics import precision_at_k, recall_at_k, mean_reciprocal_rank<br># Create predicted contexts<br>eval_dataset = eval_dataset.add_column(<br>    &quot;contexts&quot;, [retrieve_docs(q, top_k=3) for q in eval_dataset[&quot;question&quot;]]<br>)<br># Calculate metrics<br>p_score = precision_at_k(eval_dataset, k=3)<br>r_score = recall_at_k(eval_dataset, k=3)<br>mrr_score = mean_reciprocal_rank(eval_dataset)<br>print(f&quot;Precision@3: {p_score:.2f}&quot;)<br>print(f&quot;Recall@3: {r_score:.2f}&quot;)<br>print(f&quot;MRR: {mrr_score:.2f}&quot;)</pre><h3>4. Interpreting the Results</h3><ul><li><strong>Precision low, Recall high</strong> → Retrieval is returning many relevant docs, but also a lot of irrelevant ones → Tighten filtering or improve embeddings.</li><li><strong>Precision high, Recall low</strong> → Retrieval is accurate but missing many relevant docs → Expand search scope or increase k.</li></ul><h3>5. Iterative Improvement Loop</h3><p>Evaluation isn’t a one-time task — it’s a feedback loop. After computing metrics, you adjust parameters and re-test:</p><pre>def optimize_retrieval(queries, answers, iterations=3):<br>    for i in range(iterations):<br>        # Run retrieval<br>        eval_dataset = Dataset.from_dict({&quot;question&quot;: queries, &quot;answer&quot;: answers})<br>        eval_dataset = eval_dataset.add_column(<br>            &quot;contexts&quot;, [retrieve_docs(q, top_k=3) for q in eval_dataset[&quot;question&quot;]]<br>        )<br><br>        # Compute metrics<br>        p = precision_at_k(eval_dataset, k=3)<br>        r = recall_at_k(eval_dataset, k=3)<br>        mrr = mean_reciprocal_rank(eval_dataset)<br>        <br>        print(f&quot;Iteration {i+1} → Precision@3: {p:.2f}, Recall@3: {r:.2f}, MRR: {mrr:.2f}&quot;)<br>        <br>        # Example: tweak parameters between iterations<br>        tune_embeddings()<br>        adjust_chunk_size()<br>        modify_ann_params()</pre><p>This loop mirrors how real-world RAG systems are optimized: test → measure → tweak → repeat. Over time, you converge toward a pipeline that balances <strong>speed, accuracy, and coverage</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/401/1*LKC_RAlru5srFAcf-Ywe3A.png" /><figcaption>RAG Evaluation Cycle</figcaption></figure><h3>Optimization Tips</h3><ol><li><strong>Tune Chunk Size</strong> — Try 300–600 tokens per chunk.</li><li><strong>Experiment with Embedding Models</strong> — Larger models can improve retrieval.</li><li><strong>Hybrid Search</strong> — Combine keyword + semantic search for rare terms.</li><li><strong>ANN Parameters</strong> — Adjust efSearch (HNSW) for accuracy-speed tradeoff.</li><li><a href="https://medium.com/towards-artificial-intelligence/5-prompting-techniques-that-changed-my-life-as-an-ai-engineer-and-everyday-ai-user-31a60fc003a4"><strong>Prompt Engineering</strong> </a>— Guide the LLM to use provided context faithfully.</li></ol><h3>Why This Matters</h3><p>Without evaluation, RAG is just guesswork.<br> By <strong>measuring and tuning</strong>, you can:</p><ul><li>Prove that your system works.</li><li>Track improvements over time.</li><li>Optimize for speed <em>and</em> accuracy.</li></ul><h3>Mini-Course Wrap-Up</h3><p>Congratulations — you’ve now built:<br> ✅ A basic local RAG<br> ✅ A hybrid RAG<br> ✅ An agentic RAG<br> ✅ A dynamic retrieval pipeline (MCP)<br> ✅ A scalable ANN-based retriever<br> ✅ An evaluation framework</p><p>From here, you can explore more advanced RAG systems:</p><ul><li><strong>Multi-modal RAG</strong> (images, audio, video)</li><li><strong>Streaming retrieval</strong> for live data</li><li><a href="https://medium.com/towards-artificial-intelligence/demystifying-llm-fine-tuning-your-laptop-can-now-create-custom-ai-no-phd-required-a6337e56e4d1"><em>Fine-tuning</em></a><strong> embeddings</strong> for your domain</li></ul><p>👉 Check out more tutorials and follow me here: <a href="https://medium.com/@taha.azizi"><strong>https://medium.com/@taha.azizi</strong></a></p><p>and subscribe with your email to be the first to get notified when I publish new articles.</p><p>GitHub repo with practical example:</p><p><a href="https://github.com/Taha-azizi/RAG">GitHub - Taha-azizi/RAG: Introduction to RAG: Basics to Mastery</a></p><p>All photos are generated by the other using AI tools.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=977563d976ee" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Introduction to RAG: Basics to Mastery. 5-Advanced RAG: Fast Retrieval (ANN) and Reranking]]></title>
            <link>https://pub.towardsai.net/introduction-to-rag-basics-to-mastery-5-advanced-rag-fast-retrieval-ann-and-reranking-5fdbada0e17b?source=rss-8c656aa73221------2</link>
            <guid isPermaLink="false">https://medium.com/p/5fdbada0e17b</guid>
            <category><![CDATA[python]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[retrieval-augmented-gen]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Taha Azizi]]></dc:creator>
            <pubDate>Fri, 05 Sep 2025 16:01:46 GMT</pubDate>
            <atom:updated>2025-09-05T16:02:41.948Z</atom:updated>
            <content:encoded><![CDATA[<h4><em>Part 5 of the mini-series introduction to RAG</em></h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gp_zcvuuuRiLTvWnRTIENA.png" /></figure><h3>Introduction</h3><p>In the earlier articles, we built RAG pipelines that worked great for small datasets.<br>But what happens when your knowledge base grows to <strong>millions or billions of products</strong>, like in an e-commerce catalog?</p><ul><li>Exact vector search becomes too slow and memory-heavy.</li><li>Approximate Nearest Neighbors (ANN) speeds up retrieval, but may sometimes return slightly less relevant results.</li></ul><p>This is where <strong>Reranking</strong> comes in. By combining <strong>ANN retrieval</strong> (fast, broad search) with a <strong>reranker model</strong> (accurate, fine-grained relevance scoring), you get both speed <em>and</em> precision.</p><p>In this article, we’ll:</p><ul><li>Use <strong>FAISS with HNSW</strong> for lightning-fast ANN retrieval.</li><li>Add a <strong>cross-encoder reranker</strong> from Hugging Face to refine the top results.</li><li>Apply this pipeline to an <strong>e-commerce product search example</strong>.</li></ul><h3>Theory</h3><p><strong>Approximate Nearest Neighbors (ANN)</strong><br>When building a RAG pipeline, one of the biggest bottlenecks is <em>retrieval speed</em>. If your knowledge base only has a few thousand documents, exact vector search works fine — the query embedding can be compared to every document embedding quickly. But when the dataset grows to millions or even billions of entries (like in e-commerce product catalogs, research archives, or enterprise knowledge graphs), exact search becomes prohibitively slow and memory-intensive. That’s where <strong>Approximate Nearest Neighbors (ANN)</strong> comes in. ANN algorithms — such as <strong>FAISS</strong> or <strong>HNSW</strong> — cleverly index embeddings in a graph or tree structure, so instead of scanning <em>all</em> vectors, they navigate through the index to quickly find candidates that are “close enough.” The trade-off is that you might miss a few results, but in practice, accuracy loss is often under 1%, while speed gains can be 100x to 1000x faster. This makes ANN essential for <strong>scalable RAG pipelines</strong> where real-time responses are required.</p><p><strong>Reranking</strong><br>ANN solves the speed problem, but it comes with another subtle issue: the <em>top-k retrieved documents</em> may not be in the best possible order. ANN ensures they’re roughly relevant, but sometimes the most contextually useful result may be ranked lower. That’s where <strong>Reranking</strong> comes in. After ANN fetches the candidate documents, a more powerful model (often a cross-encoder or large language model) is used to <strong>re-score and reorder</strong> them based on the query. This second step improves retrieval quality dramatically by pushing the most relevant results to the top. In e-commerce, for example, if you search for “lightweight running shoes,” ANN may surface 50 products that roughly match, but a reranker can prioritize the exact models that fit best — like shoes specifically optimized for marathon training over casual sneakers. Together, ANN + Reranking give you the best of both worlds: lightning-fast retrieval <em>and</em> high precision in the final ranked list.</p><h3><strong>Summary of the Theory</strong></h3><h4>Exact vs Approximate Search</h4><ul><li><strong>Exact Search</strong>: Compares a query vector to every product vector. Very accurate, but painfully slow at scale.</li><li><strong>Approximate Search (ANN)</strong>: Uses smart indexing (like HNSW) to quickly jump to likely candidates.</li></ul><h4>Trade-offs</h4><ul><li><strong>Speed</strong>: ANN can be <strong>100–1000x faster</strong> than exact search.</li><li><strong>Accuracy</strong>: You might miss a few top results, but usually &lt;1% drop in recall.</li></ul><h4>Reranking</h4><ul><li>ANN gives you a <strong>broad top-k set</strong> (fast but noisy).</li><li>A <strong>cross-encoder model</strong> reranks those candidates by reading both the query and document together → much better final ranking.</li><li>This two-stage retrieval is standard in modern search engines (Google, Amazon, etc.).</li></ul><h3>Setup</h3><p>We’ll replace Chroma’s exact search with FAISS ANN search.</p><p><strong>Install FAISS with GPU support — or just FAISS if you do not have a capable GPU:</strong></p><pre>pip install faiss-gpu sentence-transformers </pre><h4>We’ll use:</h4><ul><li>sentence-transformers/all-MiniLM-L6-v2 → embeddings</li><li>cross-encoder/ms-marco-MiniLM-L-6-v2 → reranker</li></ul><h3>Step-by-Step Code</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*TfBDGGrzC59cgQ0V" /><figcaption>Flowchart of the advanced RAG steps in this article</figcaption></figure><h3>1. Create Embeddings</h3><p><em>(Same as before, but we’ll store them in FAISS instead of Chroma.)</em></p><pre>from sentence_transformers import SentenceTransformer<br>import numpy as np<br><br># Embedding model<br>embedder = SentenceTransformer(&quot;sentence-transformers/all-MiniLM-L6-v2&quot;)<br><br># Example product catalog (chunks)<br>all_products = [<br>    &quot;Red running shoes with breathable mesh&quot;,<br>    &quot;Wireless noise-cancelling headphones&quot;,<br>    &quot;Ergonomic office chair with lumbar support&quot;,<br>    &quot;4K Ultra HD Smart TV, 55 inch&quot;,<br>    &quot;Stainless steel water bottle, 1L&quot;,<br>    &quot;Gaming laptop with RTX 4070 GPU&quot;,<br>    &quot;Cotton crew neck T-shirt, black&quot;,<br>    &quot;Bluetooth fitness tracker with heart rate monitor&quot;,<br>    &quot;Smartphone with 128GB storage and 5G support&quot;,<br>    &quot;Portable Bluetooth speaker, waterproof&quot;,<br>    &quot;Adjustable standing desk with memory presets&quot;,<br>    &quot;Noise-reducing mechanical keyboard with RGB backlight&quot;,<br>    &quot;Stainless steel cookware set, 10 pieces&quot;,<br>    &quot;Wireless ergonomic mouse with programmable buttons&quot;,<br>    &quot;High-resolution DSLR camera with 24MP sensor&quot;,<br>    &quot;Electric mountain bike with 500W motor&quot;,<br>    &quot;Smartwatch with sleep tracking and GPS&quot;,<br>    &quot;Leather messenger bag with laptop compartment&quot;,<br>    &quot;Robot vacuum cleaner with smart mapping&quot;,<br>    &quot;Air fryer with 6-quart capacity and digital display&quot;,<br>    &quot;Portable power bank, 20,000mAh fast charging&quot;,<br>    &quot;Noise-isolating in-ear monitors for musicians&quot;,<br>    &quot;Adjustable dumbbell set, 5–50 lbs&quot;,<br>    &quot;Smart home hub with voice assistant&quot;,<br>    &quot;Curved 34-inch ultrawide monitor, QHD&quot;,<br>    &quot;Premium mattress with cooling gel memory foam&quot;,<br>    &quot;Electric toothbrush with pressure sensor&quot;,<br>    &quot;Cordless stick vacuum with HEPA filter&quot;,<br>    &quot;Espresso machine with milk frother&quot;,<br>    &quot;Winter jacket with thermal insulation&quot;,<br>    &quot;Camping tent for 4 people, waterproof&quot;,<br>    &quot;LED desk lamp with wireless charging pad&quot;,<br>    &quot;External SSD, 2TB, USB-C&quot;,<br>    &quot;Yoga mat with non-slip surface, 8mm thick&quot;,<br>    &quot;Noise-cancelling over-ear gaming headset&quot;,<br>    &quot;Smart thermostat with energy saver mode&quot;,<br>    &quot;Wireless charging pad for multiple devices&quot;,<br>    &quot;Compact air purifier with HEPA filter&quot;,<br>    &quot;Adjustable kettlebell with quick-lock system&quot;<br>]<br><br># Encode product embeddings<br>embeddings = embedder.encode(all_products, convert_to_numpy=True, show_progress_bar=True)<br>embeddings = np.array(embeddings).astype(&quot;float32&quot;)</pre><h3>2. Build ANN Index with HNSW</h3><pre>import faiss<br><br>dim = embeddings.shape[1]  # embedding dimension<br>index = faiss.IndexHNSWFlat(dim, 32)  # 32 neighbors per node<br>index.hnsw.efConstruction = 200<br>index.add(embeddings)<br><br>print(f&quot;Indexed {index.ntotal} products.&quot;)</pre><h3>3. Search with ANN</h3><pre>def ann_search(query, top_k=5):<br>    q_emb = embedder.encode([query], convert_to_numpy=True).astype(&quot;float32&quot;)<br>    distances, indices = index.search(q_emb, top_k)<br>    results = [all_products[i] for i in indices[0]]<br>    return results<br><br>query = &quot;best laptop for gaming&quot;<br>top_candidates = ann_search(query, top_k=5)<br>print(&quot;ANN Top Candidates:&quot;, top_candidates)</pre><h3>4. Generate Answer with Ollama</h3><pre>from sentence_transformers import CrossEncoder<br><br># Load reranker<br>reranker = CrossEncoder(&quot;cross-encoder/ms-marco-MiniLM-L-6-v2&quot;)<br><br>def rerank(query, candidates):<br>    pairs = [[query, doc] for doc in candidates]<br>    scores = reranker.predict(pairs)<br>    <br>    # Sort by score descending<br>    ranked = sorted(zip(candidates, scores), key=lambda x: x[1], reverse=True)<br>    return [doc for doc, score in ranked]<br><br># Apply reranking<br>reranked_results = rerank(query, top_candidates)<br>print(&quot;Final Reranked Results:&quot;, reranked_results)</pre><h3>5. Generate Final Answer with Ollama</h3><pre>import subprocess<br>context = &quot;\n&quot;.join(reranked_results[:3])  # top-3 reranked<br>prompt = f&quot;&quot;&quot;<br>Answer the customer query using only the following product context:<br>{context}<br>Customer Query: {query}<br>Answer:<br>&quot;&quot;&quot;<br>ollama_cmd = [&quot;ollama&quot;, &quot;run&quot;, &quot;mistral&quot;, prompt]<br>response = subprocess.run(ollama_cmd, capture_output=True, text=True)<br>print(&quot;LLM Response:\n&quot;, response.stdout)</pre><h3>Expected Output</h3><p>You’ll get retrieval results <strong>in milliseconds</strong> — even with millions of documents in your local knowledge base.</p><p>For the query: <strong>“best laptop for gaming”</strong></p><ul><li>ANN retrieves multiple products (laptop, TV, headphones, etc.).</li><li>Reranker boosts the <strong>Gaming laptop with RTX 4070 GPU</strong> to the top.</li><li>LLM generates an answer grounded in that product.</li></ul><h3>Performance Tips</h3><ul><li>For ultra-large datasets, use <strong>IVF (Inverted File Index)</strong> in FAISS.</li><li>Tune <strong>efSearch</strong> in HNSW for better accuracy at the cost of speed:</li></ul><pre>index.hnsw.efSearch = 50</pre><ul><li>Store embeddings in <strong>float16</strong> for memory savings:</li></ul><pre>embeddings = embeddings.astype(&quot;float16&quot;)</pre><ul><li>Use ANN to pull <strong>top-100 candidates</strong>, then rerank down to <strong>top-10</strong>.</li><li>Trade-off: more rerank candidates = higher accuracy, but slower.</li><li>For ultra-large datasets: combine ANN (FAISS) + reranking (CrossEncoder) + caching (Redis).</li></ul><h3>Why ANN + Reranking is a Game-Changer for RAG</h3><p>✅ <strong>Scalability</strong>: Handles millions of products.<br>✅ <strong>Precision</strong>: Reranker ensures the “right” items float to the top.<br>✅ <strong>Hybrid-friendly</strong>: Can combine keyword + semantic retrieval before reranking.</p><p>This two-step approach is exactly what powers real-world search engines and product recommenders.</p><h3>Next Steps</h3><p>In <strong>Article 6</strong>, we’ll focus on <strong>evaluating and optimizing RAG pipelines</strong> — using real metrics like precision@k and recall@k to measure how well your retrieval is actually performing.</p><p>👉 Check out more tutorials and follow me here: <a href="https://medium.com/@taha.azizi"><strong>https://medium.com/@taha.azizi</strong></a></p><p>and subscribe with your email to be the first to get notified when I publish new articles.</p><p>Github repo with practical example:</p><p><a href="https://github.com/Taha-azizi/RAG">https://github.com/Taha-azizi/RAG</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5fdbada0e17b" width="1" height="1" alt=""><hr><p><a href="https://pub.towardsai.net/introduction-to-rag-basics-to-mastery-5-advanced-rag-fast-retrieval-ann-and-reranking-5fdbada0e17b">Introduction to RAG: Basics to Mastery. 5-Advanced RAG: Fast Retrieval (ANN) and Reranking</a> was originally published in <a href="https://pub.towardsai.net/">Towards AI</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Introduction to RAG: Basics to Mastery. 4-RAG with MCP- The Future of Dynamic Context Retrieval]]></title>
            <link>https://pub.towardsai.net/introduction-to-rag-basics-to-mastery-4-rag-with-mcp-the-future-of-dynamic-context-retrieval-93e3a900e652?source=rss-8c656aa73221------2</link>
            <guid isPermaLink="false">https://medium.com/p/93e3a900e652</guid>
            <category><![CDATA[python]]></category>
            <category><![CDATA[retrieval-augmented-gen]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[technology]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <dc:creator><![CDATA[Taha Azizi]]></dc:creator>
            <pubDate>Mon, 01 Sep 2025 18:03:39 GMT</pubDate>
            <atom:updated>2025-09-01T18:03:39.913Z</atom:updated>
            <content:encoded><![CDATA[<h4><em>Part 4 of the mini-series introduction to RAG</em></h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/767/1*O1ff0pgl82MXLKgUDt2wVQ.png" /></figure><h3>Introduction</h3><p>So far in this series, we’ve explored:</p><ol><li><a href="https://pub.towardsai.net/introduction-to-rag-basics-to-mastery-1-build-your-own-local-rag-pipeline-no-cloud-no-api-keys-162ccf9ebc20"><strong>Basic RAG</strong></a> with local semantic search.</li><li><a href="https://pub.towardsai.net/introduction-to-rag-basics-to-mastery-8f02764b6243"><strong>Hybrid RAG</strong></a> combining keyword + semantic search.</li><li><a href="https://pub.towardsai.net/introduction-to-rag-basics-to-mastery-3-agentic-rag-giving-your-retrieval-pipeline-a-brain-be8dfa69cc62"><strong>Agentic RAG</strong></a> with multi-step reasoning and tool use.</li></ol><p>In this article, we’ll dive into something <strong>cutting-edge</strong>:<br><strong>RAG powered by MCP (Model Context Protocol)</strong>.</p><p>MCP is an emerging standard that allows LLMs to dynamically fetch context <em>during generation</em> rather than preloading everything at the start. In practice, this means the model can realize “I need more info” mid-answer, call a retrieval tool, and continue naturally.</p><p>Think of MCP as the glue that connects <strong>RAG tools</strong> (retrievers, search APIs, calculators, etc.) with an <strong>agentic system</strong> in a standardized way.</p><h3>Theory</h3><p>The <strong>Model Context Protocol (MCP)</strong> is a modern, open-source standard (introduced by Anthropic in November 2024) that standardizes how large language models (LLMs) dynamically access external tools and data sources. Acting much like a “USB-C port for AI,” MCP enables any LLM-based agent to invoke tools — such as document retrievers, APIs, or calculators — through a unified client-server interface using JSON-RPC, regardless of the underlying system. This reduces integration complexity, tackles the “M×N connector problem,” and ensures secure, scalable access to context during generation rather than all upfront. Let’s explore the differences between previous sessions:</p><p>Traditional RAG pipeline:</p><p>Retrieve → Inject into prompt → Generate</p><p>Agentic RAG pipeline (from Part 3):</p><p>Plan → Retrieve (possibly multiple times) → Reason → Answer</p><p>MCP-powered RAG pipeline:</p><p>Generate → Realize missing info → Call MCP tool (retrieval, API, calculator) → Continue generation</p><p>Benefits of MCP:</p><ul><li><strong>Dynamic retrieval</strong>: retrieval happens “mid-thought” rather than upfront.</li><li><strong>Lower memory/GPU usage</strong>: no need to preload large context.</li><li><strong>Tool standardization</strong>: all tools (retrievers, APIs, calculators) are accessed through the same protocol.</li><li><strong>Multi-turn friendly</strong>: works better in long conversations.</li></ul><h3>Setup</h3><p>We’ll use:</p><ul><li>Flask for server deployment</li><li>httpx for making asynchronous HTTP requests</li><li>fastmcp for structured context and tool integration</li></ul><p><strong>Install:</strong></p><pre>pip install flask httpx fastmcp</pre><h3>Step-by-Step Code</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*WLE91pEXh1nY4cmx" /><figcaption>Flowchart of the MCP Rag solution discussed in this article / github repository</figcaption></figure><h3>Step 1: Define MCP Server</h3><p>Our <strong>MCP server</strong> exposes RAG retrieval and calculator as MCP tools.</p><pre>from mcp.server.fastmcp import FastMCP<br>from utils.retrieval import hybrid_search<br>from utils.generation import generate_answer<br>import math<br><br><br>mcp = FastMCP(<br>  name=&quot;rag_mcp_server&quot;,<br>  version=&quot;1.0.0&quot;,<br>  description=&quot;MCP server exposing RAG retriever and calculator tools&quot;<br>)<br><br><br>@mcp.tool()<br>def rag_retrieve(query: str) -&gt; str:<br>  &quot;&quot;&quot;Retrieve relevant context using hybrid RAG.&quot;&quot;&quot;<br>  docs = hybrid_search(query, top_k=3)<br>  return &quot;\n---\n&quot;.join(docs)<br><br><br>@mcp.tool()<br>def calculator(expression: str) -&gt; str:<br>  &quot;&quot;&quot;Safe calculator for arithmetic expressions.&quot;&quot;&quot;<br>  try:<br>  return str(eval(expression, {&quot;__builtins__&quot;: {}}, {&quot;math&quot;: math}))<br>  except Exception as e:<br>  return f&quot;CALC_ERROR: {e}&quot;<br><br><br>if __name__ == &quot;__main__&quot;:<br>mcp.run()</pre><h3>Step 2: MCP Client</h3><p>The <strong>client</strong> talks to the server and lets the agent call MCP tools.</p><pre>from mcp.client import MCPClient<br><br><br>client = MCPClient(&quot;http://localhost:8000&quot;) # server address<br><br><br># Example: call RAG tool<br>print(client.call_tool(&quot;rag_retrieve&quot;, {&quot;query&quot;: &quot;Germany renewable policies&quot;}))<br><br><br># Example: call calculator tool<br>print(client.call_tool(&quot;calculator&quot;, {&quot;expression&quot;: &quot;25*4+10&quot;}))</pre><h3>Step 3: Agent with MCP Integration</h3><p>We integrate the MCP client into our agent.</p><pre>from langchain.agents import initialize_agent, AgentType<br>from langchain.llms import Ollama<br>from langchain.tools import Tool<br>from mcp_client import client<br><br><br># Wrap MCP tools for LangChain<br>retrieval_tool = Tool(<br>  name=&quot;MCP-RAG&quot;,<br>  func=lambda q: client.call_tool(&quot;rag_retrieve&quot;, {&quot;query&quot;: q}),<br>  description=&quot;Retrieve info from RAG via MCP&quot;<br>)<br><br><br>calc_tool = Tool(<br>  name=&quot;MCP-Calculator&quot;,<br>  func=lambda e: client.call_tool(&quot;calculator&quot;, {&quot;expression&quot;: e}),<br>  description=&quot;Do arithmetic via MCP&quot;<br>)<br><br><br>llm = Ollama(model=&quot;mistral&quot;)<br>agent = initialize_agent(<br>  tools=[retrieval_tool, calc_tool],<br>  llm=llm,<br>  agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION,<br>  verbose=True<br>)<br><br><br>query = &quot;If Germany’s renewable energy sector was 250 TWh in 2023 and is projected to grow 10% per year, what will it reach by 2030, and how does this compare with Germany’s official renewable energy targets?&quot;<br>print(agent.run(query))</pre><h3>Expected Behavior</h3><ol><li>The agent reads the query.</li><li>It realizes it needs a <strong>calculation</strong> → calls MCP Calculator.</li><li>It realizes it needs <strong>document retrieval</strong> → calls MCP RAG tool.</li><li>It integrates both results and generates the final answer.</li></ol><p>Thought: I need to calculate Germany’s renewable energy output in 2030, <br>given 250 TWh in 2023 with 10% annual growth.<br>Action: MCP-Calculator<br>Action Input: 250 * (1.1 ** 7)<br>Observation: 487.57</p><p>Thought: I should check Germany’s official renewable energy targets for 2030.<br>Action: MCP-RAG<br>Action Input: Germany renewable energy 2030 targets<br>Observation: Germany aims for ~80% renewable share in electricity by 2030.</p><p>Final Answer: At 10% annual growth, Germany’s renewable sector would reach ~488 TWh by 2030. <br>This aligns with Germany’s official goal of ~80% renewables in the electricity mix.</p><h3>Why This Matters</h3><p>MCP makes RAG <strong>dynamic and extensible</strong>:</p><ul><li>Tools (retrievers, calculators, APIs) can be registered in a standardized way.</li><li>Agents don’t preload all knowledge — they fetch when needed.</li><li>Your system scales: add a search API or weather API, just expose it via MCP.</li></ul><p>This is the natural evolution of RAG: from static retrieval → agentic planning → <strong>protocol-driven, dynamic retrieval</strong>.</p><h3>Next Steps</h3><p>In <strong>Article 5</strong>, we’ll push performance to the max with <strong>Advanced RAG using Approximate Nearest Neighbors (ANN)</strong> — making retrieval lightning-fast even with millions of documents.</p><p>Visit the Github page: <a href="https://github.com/Taha-azizi/RAG/tree/main">https://github.com/Taha-azizi/RAG</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=93e3a900e652" width="1" height="1" alt=""><hr><p><a href="https://pub.towardsai.net/introduction-to-rag-basics-to-mastery-4-rag-with-mcp-the-future-of-dynamic-context-retrieval-93e3a900e652">Introduction to RAG: Basics to Mastery. 4-RAG with MCP- The Future of Dynamic Context Retrieval</a> was originally published in <a href="https://pub.towardsai.net/">Towards AI</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>