<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Michael Reid on Medium]]></title>
        <description><![CDATA[Stories by Michael Reid on Medium]]></description>
        <link>https://medium.com/@mikeyreid38?source=rss-faed8a9af795------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*zjCXLtu3U4HvRj53.</url>
            <title>Stories by Michael Reid on Medium</title>
            <link>https://medium.com/@mikeyreid38?source=rss-faed8a9af795------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 12:34:00 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@mikeyreid38/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Babysitting Claude Code’s 5-minute cache TTL is the most expensive Tamagotchi ever built]]></title>
            <link>https://medium.com/@mikeyreid38/babysitting-claude-codes-5-minute-cache-ttl-is-the-most-expensive-tamagotchi-ever-built-346da300ba59?source=rss-faed8a9af795------2</link>
            <guid isPermaLink="false">https://medium.com/p/346da300ba59</guid>
            <category><![CDATA[cursor-ai]]></category>
            <category><![CDATA[claude-cowork]]></category>
            <category><![CDATA[openai-codex]]></category>
            <category><![CDATA[tokenomics]]></category>
            <category><![CDATA[claude-code]]></category>
            <dc:creator><![CDATA[Michael Reid]]></dc:creator>
            <pubDate>Fri, 04 Sep 2026 03:38:05 GMT</pubDate>
            <atom:updated>2026-09-04T03:38:05.666Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5KceQls0Fz9BVNW-t76yig.jpeg" /><figcaption>image generated by gemini-flash 3.8</figcaption></figure><p>There is no operational anxiety quite like watching Claude Code work while trying not to let the 5-minute cache TTL expire.</p><p>You’re stuck in this loop where you have to babysit the CLI like a hawk. Every 3 minutes it pauses to ask for confirmation to run a command. If you look away, take a phone call, or go to the bathroom, you cross minute 5.</p><p>Boom. Cache evicted.</p><p>You press “Y” to approve a one-line directory listing, and suddenly your terminal hangs while the API charges you a full-price cache-write across a 100k token context window.</p><p>And what’s inside that 100k payload?</p><ul><li>40k tokens of JSON schemas painstakingly explaining what bash is, how to use grep, and the exotic concept of reading a file.</li><li>50k tokens of conversation history you didn’t even need.</li><li>10k actual code diff.</li></ul><p>The cherry on top? The moment that massive 100k prefill finally finishes re-caching in the provider’s VRAM, the agent’s internal compaction threshold triggers: <em>“Context window getting full, compacting history…”</em></p><p>You literally just paid full retail price to write the cache, only for the agent to immediately truncate the context and invalidate the cache you bought 4 seconds ago.</p><p>We don’t need faster tokens/sec. We need providers to stop pretending that 40,000 tokens of first-party CLI tool documentation is private, volatile user data that must be purged from GPU memory every 300 seconds.</p><p>— the fundamental premise was re-worded by Gemini 3.8 Flash</p><p>It all started when I conjectured that Tools like Claude Code, could have global cache for their bloated tool definitions, and system prompts.. put a version mark on those prompts and quite pretending like the cache needs refreshed. it should be a header, no different than the .Net SDK version2.x.x.x for Windows XP file menu options..</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=346da300ba59" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Context Isn’t Optional: Teach AI to Know the Moment]]></title>
            <link>https://medium.com/@mikeyreid38/context-isnt-optional-teach-ai-to-know-the-moment-e091df638265?source=rss-faed8a9af795------2</link>
            <guid isPermaLink="false">https://medium.com/p/e091df638265</guid>
            <dc:creator><![CDATA[Michael Reid]]></dc:creator>
            <pubDate>Fri, 22 Aug 2025 13:50:38 GMT</pubDate>
            <atom:updated>2025-08-22T13:50:38.013Z</atom:updated>
            <content:encoded><![CDATA[<h4>Most LLMs lack <strong>situational awareness</strong>. They see text. They don’t see <em>context</em>.</h4><p>I bounced between two windows this morning: the Vertex AI console (every token billed) and a premium chat tab with daily caps. The model treated both the same. Same greeting, same gentle preamble, same “let me think.” One of those replies costs real money per character. The other burns a scarce “best model” attempt. The model didn’t know. And it showed.</p><p>That’s the problem: most LLMs lack <strong>situational awareness</strong>. They see text. They don’t see <em>context</em>.</p><h3>Three moments, three different jobs</h3><h3>1) API / production (cost-aware)</h3><p>Reality: every token is a line item.<br> What’s needed: terse, deterministic output that hits a schema and gets out of the way.<br> Bad example: “Hello! As a large language model…” (then the JSON).<br> Good example: just the JSON. No chitchat. No side quests.</p><h3>2) Premium / pro tiers (value-aware)</h3><p>Reality: the expensive resource is the <em>attempt</em> on the “big brain.”<br> What’s needed: first-shot success. Use your best reasoning up front. Remove doubt so I don’t have to re-prompt and burn another attempt.</p><h3>3) Free / exploratory (engagement-aware)</h3><p>Reality: low stakes.<br> What’s needed: show range, suggest paths, teach a bit. This is the storefront.</p><p>One model trying to “pretend” it’s in each mode via a prompt is exactly how we get bloated answers in APIs and undercooked ones in premium slots.</p><h3>Emulation vs. knowing</h3><p>Prompting is emulation. You say “be concise,” the model spends tokens <em>becoming</em> concise, suppressing default habits, and sometimes wobbling even at temperature 0. That warm-up costs money and predictability.</p><p>Knowing is different. You train the behavior in. The shortest path through the network already <em>is</em> “concise JSON mode,” or “first-shot premium mode.” No warm-up. Fewer butterflies. More consistent outcomes.</p><h3>Make the context explicit (and real)</h3><p>Expose a small, boring flag that the runtime sets and the model actually respects through training:</p><pre>{ &quot;execution_context&quot;: &quot;api_low_cost&quot; }</pre><p>Other values: &quot;premium_first_shot&quot;, &quot;free_explore&quot;.</p><p>Don’t just inject the flag into a system prompt. Back it with fine-tuning or adapters so it flips a real policy, not a mood.</p><h3>It’s not just jargon; it’s operating policy</h3><p>Specialization isn’t only vocabulary. It’s <strong>how the work should be done</strong> under different risks and goals.</p><ul><li><strong>Legal/finance:</strong> source, quote precisely, mark unknowns, avoid speculation.</li><li><strong>Creative/marketing:</strong> produce divergent options, vary tone, accept stylistic risk.</li><li><strong>Software/DevOps:</strong> emit runnable code first, minimal commentary, stable interfaces.</li><li><strong>Education/tutoring:</strong> scaffold, ask guiding questions, adapt to the learner.</li></ul><p>Same knowledge base, different operating rules.</p><h3>A practical MoE: How, What, Who</h3><p>A Mixture-of-Experts fits here, but tune the experts by <strong>delivery</strong> as much as by <strong>domain</strong>.</p><p><strong>HOW (situational experts)</strong></p><ul><li>api_low_cost: terse, schema-first, low variance.</li><li>premium_first_shot: deep reasoning, citations, decisive answer.</li><li>free_explore: examples, alternatives, gentle guidance.</li></ul><p><strong>WHAT (domain experts)</strong><br> Security, law, medicine, software, finance, etc., each with their own guardrails.</p><p><strong>WHO (audience experts)</strong><br> Patient vs. clinician. Exec vs. engineer. Beginner vs. expert. Same facts, different voice.</p><p>A router blends HOW + WHAT + WHO using the execution flag, task hints, and audience signals. The “bathwater” (raw data) is fine. The <strong>centrifuge</strong> — how we stratify and route behavior — needs a rethink.</p><h3>Build it without boiling the ocean</h3><ul><li><strong>Define the contract:</strong> standardize three contexts to start: api_low_cost, premium_first_shot, free_explore.</li><li><strong>Collect traces per context:</strong> API logs that reward brevity and schema validity; premium sessions scored on one-shot success; free sessions on exploration that doesn’t hallucinate.</li><li><strong>Train adapters:</strong> LoRA or similar so you don’t retrain the base.</li><li><strong>Route intentionally:</strong> flag + task type + audience → experts.</li><li><strong>Measure the right things:</strong></li><li>API: tokens/output, schema-valid %, latency, determinism</li><li>Premium: single-turn solve rate, citation fidelity, re-prompt rate.</li><li>Free: session depth, idea diversity, error rate.</li><li><strong>Keep guardrails intact:</strong> safety stays constant; delivery changes, not the bounds.</li></ul><h3>Reality check</h3><p>Routers misroute. Adapters drift. Teams will argue over “too terse” vs. “too helpful.” That’s normal. The point is to stop paying for preambles in places they don’t belong and stop short-changing premium moments that deserve the model’s full attention.</p><p>Teach the model to see the moment it’s in. Then hold it to that standard.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e091df638265" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Navigating the Hype and Reality of MedPrompt+ and MMLU Benchmarks: A Researcher’s Nightlong Journey]]></title>
            <link>https://medium.com/@mikeyreid38/navigating-the-hype-and-reality-of-medprompt-and-mmlu-benchmarks-a-researchers-nightlong-journey-606adc177555?source=rss-faed8a9af795------2</link>
            <guid isPermaLink="false">https://medium.com/p/606adc177555</guid>
            <category><![CDATA[chatgpt]]></category>
            <category><![CDATA[prompt-engineering]]></category>
            <category><![CDATA[google-gemini-ai]]></category>
            <dc:creator><![CDATA[Michael Reid]]></dc:creator>
            <pubDate>Fri, 15 Dec 2023 14:24:45 GMT</pubDate>
            <atom:updated>2024-01-06T17:02:34.929Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*RcOBmuNM5gnUBdbOyk6uBQ.png" /></figure><p>As an AI researcher and recent blogger, I’ve always been on the lookout for groundbreaking advancements in the field. The recent buzz around Google’s new AI model, ‘Gemini Ultra,’ caught my attention, especially with claims of it surpassing OpenAI’s GPT-4. This claim, prominently featured in a press release from PC World, “<a href="https://www.pcworld.com/article/2170884/ai-battle-continues-google-launches-gemini-to-beat-openai-and-microsofts-gpt-4.html">Google’s Gemini AI takes aim at OpenAI and Microsoft’s GPT-4</a>”, led me to dive deeper into the technological marvels of these AI giants.</p><p>With high expectations, I turned my focus to the <a href="https://github.com/microsoft/promptbase/">promptbase repository</a> that held the current GOAT, MedPrompt+ that was referenced when touting these impressive results on the MMLU tests. The excitement was palpable, as the AI community was abuzz with how MedPrompt+ had outperformed Gemini Ultra. This was significant news, considering the intense competition in AI development.</p><p>However, my experience with the repository was far from straightforward. The README, filled with high praises for the results, set a high bar. Eager to witness these achievements firsthand, I embarked on a journey to replicate the MMLU tests. What I expected to be a seamless exploration turned into a challenging endeavor that consumed my entire night.</p><p>As the hours ticked by, my initial enthusiasm waned, replaced by skepticism. The term “Fake News” echoed in my mind, reminding me of past instances where hype overshadowed reality. Despite this, I persisted, navigating through the repository, making necessary adjustments, and fixing imports.</p><p>Finally, after hours of troubleshooting and coding, I managed to get the first half of the process running. But at 2:47 AM, I hit another roadblock — an error indicating that ‘torch’ wasn’t compiled with CUDA (I’ll take the blame for this one). This was a stark contrast to the README’s implication that running MMLU would be as simple as downloading the dataset and executing a command.</p><p>As I reflect on this experience, it brings forth an important reminder for teams like Microsoft: the rush to outdo competitors should not compromise the accessibility and usability of their tools. While a 0.06% improvement over a competitor is commendable, it loses its luster if the process to achieve or replicate it is fraught with obstacles.</p><p>The good news, the first half of the results look promising, but they were viewed through the eyes of exhaustion.</p><p>Microsoft PromptBase Team, I urge you to do better. It’s crucial to ensure that when you extend an invitation to the community to leverage your work, it should be accompanied by clear, accurate, and user-friendly documentation. This not only fosters trust and credibility but also encourages more researchers and enthusiasts to engage with your technology.</p><p>As we continue to explore the vast potential of AI, let’s remember that the journey is as important as the destination. Let’s strive for transparency, ease of use, and accurate representation in our shared quest for innovation.</p><p>*This blog post was generated with the assistance of GPT-4 (A really short prompt, and an outline of my own experience).</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=606adc177555" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>