<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by German Viscuso on Medium]]></title>
        <description><![CDATA[Stories by German Viscuso on Medium]]></description>
        <link>https://medium.com/@germanviscuso?source=rss-155e26ed6f1d------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*QfXnVEeYVQxiJUarYbhpRA.png</url>
            <title>Stories by German Viscuso on Medium</title>
            <link>https://medium.com/@germanviscuso?source=rss-155e26ed6f1d------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Tue, 17 Nov 2026 15:16:32 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@germanviscuso/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[I Benchmarked Every Local AI Machine Under $6K — One Destroyed the Rest]]></title>
            <link>https://medium.com/@germanviscuso/i-benchmarked-every-local-ai-machine-under-4k-one-destroyed-the-rest-98329807aca2?source=rss-155e26ed6f1d------2</link>
            <guid isPermaLink="false">https://medium.com/p/98329807aca2</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[agentic-ai]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[hardware]]></category>
            <dc:creator><![CDATA[German Viscuso]]></dc:creator>
            <pubDate>Mon, 02 Nov 2026 18:05:47 GMT</pubDate>
            <atom:updated>2026-11-09T00:19:33.246Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*yZOvp-UI6Xx_NvpgWBh4Zw.jpeg" /><figcaption>The challenger finally showed up. The champion brought $30K in GPUs again. The undercard crasher brought a receipt.</figcaption></figure><h4>Five months ago I told you to wait. Here’s what happened — and why the answer surprised even me.</h4><p>In May, I published “<a href="https://medium.com/@germanviscuso/the-best-budget-local-inference-machine-ships-next-month-heres-why-it-s-worth-the-wait-78adc799ffb0">The Best Budget Local Inference Machine Ships Next Month.</a>” My thesis was that the Mac Mini M5 Pro with 64GB would be the best price/performance ratio for local LLM inference. The post landed well — several claps, decent for a post about memory bandwidth. But the market moved in ways I didn’t fully predict.</p><p>The M5 Pro Mac Mini shipped in September. It hits the specs I predicted. But three other things happened that reshuffled the deck: AMD’s Strix Halo ecosystem matured faster than expected, Apple’s M5 Ultra arrived with up to 512GB of unified memory, and a DRAM shortage pushed the price of every 128GB machine up — some of them dramatically.</p><p>So let me walk you through the new math. Spoiler: I ran six workloads across every serious machine under $6K, plus the two that blow past it. One brand took first place in all six. It’s not the one the internet promised you.</p><h3>Quick Recap: The Physics Haven’t Changed</h3><p>If you read Part 1, you can skip this section. If you didn’t: LLM token generation is limited by memory bandwidth, not compute. Every generated token reads the model’s active weights from memory. Higher GB/s = faster tok/s. Memory capacity determines <em>what</em> fits; bandwidth determines <em>how fast</em> it runs.</p><p>That gives you a free lie detector. <strong>Maximum tok/s ≈ bandwidth ÷ gigabytes read per token.</strong> A 70B dense model at 4-bit is ~40GB, so a 614 GB/s machine can’t exceed ~15 tok/s on it — no matter what a benchmark site claims. I ran every number in this post through that division. Several popular “benchmarks” floating around failed it, and so did a couple of numbers in my own first draft.</p><p>What <em>has</em> changed is a new variable: time to first token (TTFT). Apple’s M5 generation added Neural Accelerators — dedicated matrix-multiplication units in every GPU core, similar in spirit to NVIDIA’s tensor cores. They speed up prefill (prompt processing), which is what you wait on before the first token appears. Measured on the M5 Max, gpt-oss-120b processes prompts at 1,300–2,700 tok/s depending on context length. That matters for agentic workloads that send large context on every request. Once generation starts, bandwidth still rules.</p><h3>The Scorecard: What I Got Right and Wrong</h3><p>Before the new analysis, let me be honest about Part 1’s predictions.</p><p><strong>Got right:</strong> The M5 Pro Mac Mini shipped with 64GB max. Memory bandwidth landed at 307 GB/s (I predicted 300–350). The DRAM shortage hit pricing as I warned — the 64GB config costs $2,699, not the $2,000–2,200 I hoped for. The Kickstarter “agentic computer” remained a bad deal relative to shipping alternatives.</p><p><strong>Got wrong:</strong> I underestimated how fast AMD’s Strix Halo ecosystem would mature — Lemonade Server and better llama.cpp backends removed most of the setup pain I warned about. And I didn’t anticipate Neural Accelerators becoming the M5’s headline feature for inference.</p><p><strong>Biggest miss:</strong> I framed Strix Halo as “budget and niche.” The software improved, and MoE models turned its capacity advantage into a real selling point. But the DRAM shortage I warned about for Apple? It hit AMD’s OEMs and NVIDIA even harder — more on that below.</p><h3>The $1,499 Myth — And Its $2,100 Sequel</h3><p>Before the hardware breakdown, let me address the elephant in the room. A viral post on X showed AMD CEO Lisa Su holding a tiny Strix Halo desktop and running a 235B-parameter model live. The post claimed this was a “$1,499 machine” that could replace NVIDIA’s $4,000 DGX Spark. It racked up millions of views and briefly spooked NVIDIA’s stock.</p><p>There was just one problem: it wasn’t true. A community note was added pointing out that the 128GB configuration — the one you actually need to run those large models — costs significantly more than $1,499. That price was either a promotional preorder for the 64GB config or an aggressive limited-time deal.</p><p>Early on, the “corrected” price floated around $2,000–2,200 for 128GB OEM boxes. I used that range in the first version of this post. Then the DRAM shortage caught up. These Strix Halo machines solder 128GB of LPDDR5X directly to the board — that’s the single most expensive component, and DRAM prices climbed steadily through 2026. The result: in-stock 128GB Strix Halo boxes now run <strong>~$3,500</strong> from reputable OEMs. The GMKtec EVO-X2 sits at ~$3,500 (1TB) to $3,650 (2TB). The Framework Desktop kit is $3,449 — without storage. The Minisforum MS-S1 MAX runs ~$3,800–3,960. AMD’s own Ryzen AI Halo box? $3,999.</p><p>The lesson: always check the <em>current</em> price, not the launch-week price, before building your budget around any machine in a DRAM shortage. The $1,499 viral claim was wrong. The $2,100 “corrected” price was right for about a month. Today, budget $3,500 for a 128GB Strix Halo box.</p><h3>The Updated Landscape: November 2026</h3><p>Here’s every purchasable local inference option right now, with measured numbers.</p><p><strong>$2,699 — Mac Mini M5 Pro 64GB.</strong> The machine I told you to wait for. It shipped September 22 at $1,699 base (24GB) and $2,699 for 64GB (16-core GPU; the 20-core GPU is $2,899 — it helps prefill, not decode). 307 GB/s. Measured: Gemma 4 26B-A4B at ~79 tok/s, Qwen3.6–35B-A3B at ~77 tok/s (4-bit, MLX), Qwen3.8 27B dense at ~13.5 tok/s (at 32K context). <strong>One correction to my first draft: gpt-oss-120b does not fit.</strong> Its weights are ~63–65GB, and macOS lets the GPU use ~48GB of a 64GB machine by default (~56GB if you raise the limit). Same story for Qwen3.5–122B-A10B (~70GB at 4-bit) and Qwen3.8-Flash-Next (~73GB even at its smallest quant). Llama 3.3 70B at Q4 (~40GB) fits, but the bandwidth ceiling caps it near 8 tok/s — I estimate ~6 in practice. Setup takes 20 minutes: install Ollama or LM Studio, pull a model, go.</p><p><strong>~$3,500 — AMD Strix Halo 128GB (GMKtec EVO-X2, Bosgame M5, Framework Desktop, and others).</strong> The capacity play. 128GB of LPDDR5X as unified memory, ~256 GB/s theoretical and ~212 GB/s in practice. Where it shines is large MoE: gpt-oss-120b at <strong>~53–56 tok/s</strong> (llama.cpp), Qwen3.5–122B-A10B at ~21–23 tok/s, Qwen3.8-Flash-Next at ~22 (MTP off) — and with the gufo engine it reads prompts at ~1,600–1,800 tok/s, faster than any Mac in this price range. On small MoE it’s solid: Gemma 4 26B-A4B at ~42–53 tok/s (~61 with Google’s QAT checkpoint), Qwen3.6–35B-A3B at ~63–81 depending on the 4-bit quant. Dense models crawl: a 27B dense model at ~12 tok/s, a 70B at ~5. The software has improved — Lemonade Server and llama.cpp’s Vulkan and ROCm backends work well now — but expect more tinkering than on a Mac. One practitioner who built a complete local AI stack on this hardware summed it up: the AMD tax is time, not money. He runs Gemma 4 26B-A4B as his daily driver — not dense 70B.</p><p><strong>Strix Halo’s secret weapon: Halogen.</strong> A free engine written from scratch for this one GPU and the Qwen3.8 family. With speculative decoding, it runs Qwen3.8 27B at ~32 tok/s on average (vs ~11 without) and Qwen3.8-Flash-Next at ~52, with output identical to normal decoding. More on that below.</p><p><strong>$3,999 — AMD Ryzen AI Halo (AMD’s official dev box; a 192GB Gorgon Halo version now sells for ~$7,299).</strong> Same Strix Halo silicon in a first-party chassis. Same 128GB, same bandwidth, same performance. With OEM boxes at ~$3,500, you’re paying ~$500 for the AMD branding and developer support package.</p><p><strong>~$6,950 — NVIDIA DGX Spark 128GB.</strong> The DRAM shortage’s biggest victim. It launched at $3,999, went to $4,699 in February, and jumped to $6,950 in October — which knocks it out of the under-$6K bracket entirely. I kept it in the charts anyway, because it’s the machine everyone asks about. Bandwidth is still 273 GB/s, and measured decode lands right where that predicts: gpt-oss-120b at ~53–61 tok/s (llama.cpp/vLLM), Gemma 4 26B-A4B at ~50, Qwen3.8-Flash-Next at ~27 (~56 with TensorFold and MTP on, with the fastest prefill in this post at ~2,200–3,000 tok/s), Qwen3.8 27B at ~12, Llama 70B at ~4.4. That’s Strix Halo speed at twice the price. NVIDIA has also announced a 64GB Spark, expected around $4,999 through partners. Buy a Spark if you need CUDA and the NVIDIA software stack on your desk. Don’t buy it for inference speed per dollar.</p><p><strong>$2,499–5,399 — Mac Studio M5 Max (36GB to 128GB).</strong> 614 GB/s on the 40-core GPU (128GB config, $5,399), with Neural Accelerators in every GPU core. Measured: gpt-oss-120b at ~79 tok/s (Ollama) to ~88 (MLX), Qwen3.5–122B-A10B at ~66 tok/s, Qwen3.8-Flash-Next at ~61 with the Strata engine (MTP off; ~74–94 with it on), Qwen3.6–35B-A3B at ~113 (4-bit), Qwen3.8 27B at ~25, and Llama 3.3 70B at <strong>~13 tok/s</strong>. My first draft said 18–22 for 70B. That number circulates online, but it’s above the ~15 tok/s physical ceiling. 13 tok/s is roughly reading speed: usable for chat, not snappy. The 128GB config gives you room for a large MoE plus long context plus a second model loaded at the same time.</p><p><strong>$5,499–22,299 — Mac Studio M5 Ultra (96GB to 512GB).</strong> The frontier tier, and the one where the real numbers are finally in. ~1.2 TB/s on every configuration, per Apple’s specs. Measured on the 256GB, 80-core model ($10,799): gpt-oss-120b at <strong>~110–115 tok/s</strong>, Gemma 4 26B-A4B at ~112, Qwen3.5–122B-A10B at ~85, Qwen3.8-Flash-Next at ~57 (MTP off; ~91 with it on), Qwen3.8 27B at ~46, Llama 3.3 70B at ~23. DeepSeek V4-Flash (284B total, 13B active) fits in 256GB, and early owner reports are promising but too thin to quote. The 512GB config went on sale October 28 at $22,299. If your threat model or compliance requirements demand that no tokens leave the building, this is the only consumer answer that exists.</p><h3>The Calculation That Matters: Bang-for-Buck</h3><p>Here’s the metric that decides: tok/s per $1,000 spent. One model doesn’t tell the whole story, so I ran six — from last spring’s daily drivers to today’s state-of-the-art MoE. Every machine runs on its fastest engine (MLX on Mac, llama.cpp or vLLM elsewhere), at 4-bit, with speculative decoding off.</p><p><strong>Small MoE — Gemma 4 26B-A4B (~16GB at 4-bit; fits on every machine here):</strong></p><p>🥇 Mac Mini M5 Pro 64GB — ~79 tok/s at $2,699 → <strong>29.3 tok/s per $1K</strong> <br>🥈 Mac Studio M5 Max 128GB — ~94+ tok/s at $5,399 → <strong>17.4 tok/s per $1K</strong> <em>(measured on the 32-core M5 Max; the 40-core has more bandwidth and should be at least this fast)</em> <br>🥉 Strix Halo 128GB — ~53 tok/s at $3,500 → <strong>15.1 tok/s per $1K</strong> <br>4️⃣ Mac Studio M5 Ultra 256GB — ~112 tok/s at $10,799 → <strong>10.3 tok/s per $1K</strong> <br>5️⃣ DGX Spark 128GB — ~50 tok/s at $6,950 → <strong>7.1 tok/s per $1K</strong></p><p><strong>Last spring’s reasoning MoE — Qwen3.6–35B-A3B (~20GB at 4-bit):</strong></p><p>🥇 Mac Mini M5 Pro 64GB — ~77 tok/s at $2,699 → <strong>28.4 tok/s per $1K</strong> <br>🥈 Strix Halo 128GB — ~81 tok/s at $3,500 → <strong>23.2 tok/s per $1K</strong> <em>(Q4_0; ~63 tok/s at Q4_K_M)</em> <br>🥉 Mac Studio M5 Max 128GB — ~113 tok/s at $5,399 → <strong>21.0 tok/s per $1K</strong> <br>4️⃣ Mac Studio M5 Ultra 256GB — ~145 tok/s at $10,799 → <strong>13.4 tok/s per $1K</strong> <em>(measured on Qwen3.5–35B-A3B, same architecture)</em> <br>5️⃣ DGX Spark 128GB — ~77 tok/s at $6,950 → <strong>11.0 tok/s per $1K</strong></p><p><strong>Today’s SOTA MoE — Qwen3.8-Flash-Next (~94GB at 4-bit; needs a 128GB-class machine):</strong></p><p>🥇 Mac Studio M5 Max 128GB — ~61 tok/s at $5,399 → <strong>11.2 tok/s per $1K</strong> <br>🥈 Strix Halo 128GB — ~22 tok/s at $3,500 → <strong>6.3 tok/s per $1K</strong> <br>🥉 Mac Studio M5 Ultra 256GB — ~57 tok/s at $10,799 → <strong>5.3 tok/s per $1K</strong> <br>4️⃣ DGX Spark 128GB — ~27 tok/s at $6,950 → <strong>3.9 tok/s per $1K</strong> <br>❌ Mac Mini M5 Pro 64GB — doesn’t fit</p><p><strong>Large MoE — gpt-oss-120b (~63GB; needs a 96GB+ machine):</strong></p><p>🥇 Mac Studio M5 Max 128GB — ~88 tok/s at $5,399 → <strong>16.3 tok/s per $1K</strong> <br>🥈 Strix Halo 128GB — ~56 tok/s at $3,500 → <strong>15.9 tok/s per $1K</strong> <br>🥉 Mac Studio M5 Ultra 256GB — ~115 tok/s at $10,799 → <strong>10.7 tok/s per $1K</strong> <br>4️⃣ DGX Spark 128GB — ~61 tok/s at $6,950 → <strong>8.7 tok/s per $1K</strong> <br>❌ Mac Mini M5 Pro 64GB — doesn’t fit</p><p><strong>Dense mid-size — Qwen3.8 27B at 4-bit (the current generation of dense models):</strong></p><p>🥇 Mac Mini M5 Pro 64GB — ~13.5 tok/s at $2,699 → <strong>5.0 tok/s per $1K</strong> <br>🥈 Mac Studio M5 Max 128GB — ~25 tok/s at $5,399 → <strong>4.6 tok/s per $1K</strong> <br>🥉 Mac Studio M5 Ultra 256GB — ~46 tok/s at $10,799 → <strong>4.3 tok/s per $1K</strong> <br>4️⃣ Strix Halo 128GB — ~12 tok/s at $3,500 → <strong>3.3 tok/s per $1K</strong> <em>(measured on Qwen3.6 27B, same size class)</em> <br>5️⃣ DGX Spark 128GB — ~12 tok/s at $6,950 → <strong>1.7 tok/s per $1K</strong></p><p><em>(Context lengths vary between runs, from 0 to 32K.)</em></p><p><strong>Dense 70B stress test — Llama 3.3 70B at 4-bit:</strong></p><p>🥇 Mac Studio M5 Max 128GB — ~13.4 tok/s at $5,399 → <strong>2.5 tok/s per $1K</strong> <br>🥈 Mac Mini M5 Pro 64GB — ~6 tok/s <em>(estimate)</em> at $2,699 → <strong>2.2 tok/s per $1K</strong> <br>🥉 Mac Studio M5 Ultra 256GB — ~23 tok/s at $10,799 → <strong>2.1 tok/s per $1K</strong> <br>4️⃣ Strix Halo 128GB — ~5 tok/s at $3,500 → <strong>1.4 tok/s per $1K</strong> <em>(measured on a Llama-3-architecture 70B)</em> <br>5️⃣ DGX Spark 128GB — ~4.4 tok/s at $6,950 → <strong>0.6 tok/s per $1K</strong> <em>(Llama 3.1 70B)</em></p><p>A word on that last chart. Llama 3.3 70B shipped in December 2024, and it’s still the reference dense 70B because nothing replaced it. Meta moved to MoE with Llama 4 and has since shifted its own chatbots to a new model family. The big labs’ newest dense open models sit around 27–31B (Qwen3.8 27B, Gemma 4 31B). So read the 70B chart as a bandwidth stress test, not a model recommendation. Only the M5 Max and M5 Ultra are fast enough to actually use.</p><p>Four things jump out.</p><p><strong>First, Apple takes gold in all six charts.</strong> Not five out of six. Six. The Mac Mini M5 Pro wins the three small and mid-size workloads — by a mile on small MoE (29 tok/s per $1K, ~70% ahead of the runner-up). The Mac Studio M5 Max wins the three big ones.</p><p><strong>Second, AMD’s best shot is a photo finish — and the Mac still edges it.</strong> On gpt-oss-120b, the model Strix Halo was practically built for, it comes within 3% of the M5 Max on value. That’s the closest anyone gets to Apple in this entire post. Move to today’s SOTA MoE, Qwen3.8-Flash-Next, and the gap blows open: the M5 Max is nearly three times as fast <em>and</em> ~78% better per dollar. AMD’s haul: three silvers.</p><p><strong>Third, the M5 Max is the only machine that medals in all six charts.</strong> It’s the fastest machine under $6K on every workload I measured, and it never finishes below bronze on value. That’s what a champion looks like (hence my favorite).</p><p><strong>Fourth, NVIDIA leaves empty-handed.</strong> The DGX Spark finishes last or second-to-last on every chart. At $6,950, it’s priced like a workstation and decodes like a Strix Halo.</p><h3>Round two: speculation on</h3><p>The six charts above compare every machine with speculative decoding off. That’s the cleanest apples-to-apples test, but it’s not how many of you actually run these models. Speculative decoding — MTP, DFlash and friends — drafts several tokens ahead and verifies them in one pass, with output identical to normal decoding. On Strix Halo, a free engine called Halogen, written from scratch for that one GPU and the Qwen3.8 family, makes it dramatically faster. So here’s round two, everyone with speculation on, on the three models where published numbers exist:</p><p><strong>Qwen3.6–35B-A3B (speculation on):</strong></p><p>🥇 Mac Mini M5 Pro 64GB — ~86–102 tok/s at $2,699 → <strong>31.7–37.7 tok/s per $1K</strong> <br>🥈 Strix Halo 128GB — ~67 tok/s at $3,500 → <strong>19.1 tok/s per $1K</strong> <br>🥉 DGX Spark 128GB — ~108 tok/s at $6,950 → <strong>15.6 tok/s per $1K</strong> <br>⏳ M5 Max and M5 Ultra — no published speculation-on run yet</p><p><strong>Qwen3.8-Flash-Next (speculation on):</strong></p><p>🥇 Mac Studio M5 Max 128GB — ~74–94 tok/s at $5,399 → <strong>13.7–17.4 tok/s per $1K</strong> <br>🥈 Strix Halo 128GB (Halogen) — ~52 tok/s at $3,500 → <strong>15.0 tok/s per $1K</strong> <br>🥉 DGX Spark 128GB (TensorFold) — ~56–64 tok/s at $6,950 → <strong>8.0–9.3 tok/s per $1K</strong> <br>4️⃣ Mac Studio M5 Ultra 256GB — ~91 tok/s at $10,799 → <strong>8.4 tok/s per $1K</strong> <br>❌ Mac Mini M5 Pro 64GB — doesn’t fit</p><p><strong>Qwen3.8 27B dense (speculation on):</strong></p><p>🥇 Strix Halo 128GB (Halogen) — ~32 tok/s at $3,500 → <strong>9.1 tok/s per $1K</strong> <em>(average across ten prompt types; ~21–23 on prose, ~33–44 on code)</em> <br>🥈 Mac Studio M5 Max 128GB — ~36–47 tok/s at $5,399 → <strong>6.7–8.7 tok/s per $1K</strong> <br>⏳ Mac Mini M5 Pro 64GB — no published speculation-on run yet</p><p>Apple keeps two of the three. The Mac Mini still runs away with the 35B, and the M5 Max holds Flash-Next — though Strix Halo with Halogen sits inside its range, so call it a photo finish. The one AMD steals is dense 27B, where Halogen’s hand-tuned kernels give it a narrow edge over the M5 Max. The Mac Mini M5 Pro hasn’t been measured there with speculation on yet (to my knowledge), and it won the speculation-off version of that chart — so this one is provisional. I’ll update the charts as Mac speculation-on numbers come in.</p><p>Two footnotes to the takeaways above. With speculation on, Strix Halo gets closer to Apple than the gpt-oss-120b photo finish — that line holds for speculation off. And the DGX Spark finally gets on the podium, with two bronzes.</p><h3>The sleeper: the base Mac Studio M5 Max</h3><p>The $2,499 Mac Studio (M5 Max, 32-core GPU, 36GB) has ~460 GB/s — 50% more bandwidth than the Mac Mini M5 Pro, for $200 <em>less</em>. Measured on Gemma 4 26B-A4B at 4-bit: ~94 tok/s, which works out to <strong>37.6 tok/s per $1K</strong> — the best number in this entire post. The catch is 36GB: after macOS, you have room for roughly a 27B dense model at 4-bit or a small MoE, and not much else. If you know your models will stay that size, this is the best-value machine Apple sells. If you want headroom to grow, the 64GB Mac Mini is the safer bet.</p><h3>The wildcard: the $5,499 M5 Ultra</h3><p>Here’s the one that could make Apple’s lead even bigger. The entry M5 Ultra — 64-core GPU, 96GB, $5,499 — costs $100 more than the M5 Max 128GB and, per Apple’s spec sheet, has the same ~1.2 TB/s as its bigger siblings. That’s double the M5 Max’s bandwidth. gpt-oss-120b and Llama 70B both fit in 96GB. If it decodes anywhere near the 256GB Ultra — and the physics say it should, since decode is bandwidth-bound — it would top the large-MoE value chart (~20 tok/s per $1K on gpt-oss-120b) and run 70B roughly 1.7x faster than the M5 Max. No published benchmark on the 96GB config exists yet, so it stays out of the rankings until someone measures it. You give up 32GB versus the M5 Max: no Qwen3.8-Flash-Next at 4-bit, and less room for long context or a second model.</p><h3>The Part That Changed My Mind: MoE Won</h3><p>In Part 1, I treated MoE models as a niche optimization for bandwidth-limited hardware. Six months later, MoE is the mainstream architecture for local inference. Gemma 4 26B-A4B (26B total, ~4B active), Qwen3.6–35B-A3B (35B total, 3B active), gpt-oss-120b (117B total, 5.1B active), Qwen3.5–122B-A10B, and the new Qwen3.8-Flash-Next (6B active, but it needs ~94GB even at aggressive 4-bit quantization) — these are the models people actually run. They were designed to be fast on bandwidth-limited hardware, and their quality has caught up with dense models for most practical tasks.</p><p>This is the structural shift that kept the Strix Halo in the fight despite its inflated price. AMD’s silicon limitation — lots of memory, limited bandwidth — accidentally aligned with where the models went. If your daily driver is gpt-oss-120b or Qwen3.5–122B-A10B, the Strix Halo is the cheapest box that runs it at all. But “runs it” isn’t “wins.” Three caveats. On Qwen3.5–122B-A10B, which reads about twice as many active parameters per token, the M5 Max’s bandwidth pulls well ahead (~66 vs ~22 tok/s, so ~12 vs ~6 tok/s per $1K). On Qwen3.8-Flash-Next, the Strix Halo runs at about half the M5 Max’s speed. And dense models remain the Strix Halo’s weak spot. MoE made AMD competitive. It didn’t make AMD win.</p><h3>What’s Coming: The Wait-for Factor</h3><p><strong>Benchmarks for the $5,499 M5 Ultra.</strong> See above. If you’re about to buy an M5 Max 128GB, it’s worth waiting a few weeks for someone to measure the 96GB Ultra.</p><p><strong>M5 Ultra 512GB benchmarks.</strong> The 256GB Ultra has been reviewed; the 512GB config only went on sale October 28. Measurements of the models that actually <em>need</em> 512GB — DeepSeek V4-Pro, the full GLM-5, Mistral Large 3 — are still thin. If you’re about to write a $22K check, wait for those.</p><p><strong>AMD Gorgon Halo — already here.</strong> Correction from my first draft, thanks to a reader: Gorgon Halo isn’t a rumor anymore. It ships as the Ryzen AI Max+ PRO 495 — same Zen 5 / RDNA 3.5 silicon as Strix Halo, with faster LPDDR5X-8533 for ~273 GB/s (+~7%) and, more importantly, up to 192GB of memory, 160GB of it usable by the GPU. Boxes from GMKtec, Bosgame and Minisforum run ~$6,199–7,399, so they sit above this post’s $6K bracket, and I haven’t found independent LLM benchmarks yet. Expect Strix Halo speeds plus ~7% — so it won’t change the value rankings. What it does change is capacity: it’s the cheapest way to run models that don’t fit in 128GB (Qwen3–235B at Q4, MiniMax-M2) without spending $10,799 on a 256GB M5 Ultra.</p><p><strong>AMD Medusa Halo (Ryzen AI MAX 500).</strong> The one to watch long-term. Zen 6, RDNA 5, and — critically — rumored LPDDR6 on a wider bus, for ~460–690 GB/s. That’s M5 Max territory. But it’s 2027–2028, nobody knows what DRAM will cost by then, and AMD’s software will need its usual months of post-launch maturation. Know it’s coming; don’t wait for it.</p><h3>My Updated Recommendations</h3><p><strong>The value pick ($2,499–2,699) — for models up to ~30B or small MoE:</strong> Buy the Mac Mini M5 Pro 64GB at $2,699. It’s the bang-for-buck champion on small MoE, last spring’s 35B reasoning MoE, and dense 27B, with room for long context and a couple of models at once. If you’re certain you’ll never need more than 36GB, the base Mac Studio M5 Max at $2,499 is even better value — 50% more bandwidth for less money. Either way: no gpt-oss-120b, no 122B MoE, no Flash-Next.</p><p><strong>The one to buy ($5,399) — the winner:</strong> The Mac Studio M5 Max 128GB. Gold on gpt-oss-120b, gold on today’s SOTA MoE (Qwen3.8-Flash-Next), gold on dense 70B, and a medal on every other chart. It’s the fastest machine under $6K on every workload I measured, it runs a 70B dense model at a usable ~13 tok/s, and it holds a large MoE plus long context plus a second model at the same time. This is the “buy it and stop thinking about hardware for three years” machine. One caveat: before you order, check whether anyone has published numbers for the $5,499 M5 Ultra 96GB. If it performs like its bandwidth says, it’s an even better buy for $100 more — as long as 96GB is enough for you.</p><p><strong>Only if your budget stops at $3,500 and you need 128GB:</strong> A Strix Halo 128GB from a reputable OEM (GMKtec EVO-X2, Bosgame M5, Framework Desktop). It’s the cheapest box that runs large MoE at all, and on gpt-oss-120b it comes within a few percent of the M5 Max on value. On everything heavier — Qwen3.5–122B-A10B, Qwen3.8-Flash-Next — the M5 Max is the better value, and on dense models it isn’t close. Accept the setup tax, use llama.cpp’s Vulkan backend or Lemonade, and don’t pay $3,999 for AMD’s official box — it’s the same chip. And don’t fall for the “$1,499” marketing.</p><p>If Qwen3.8 is your daily driver, run it on Halogen. That’s what turns the Strix Halo from a runner-up into a contender.</p><p><strong>Frontier tier ($10,799–22,299) — when nothing can leave the building:</strong> The Mac Studio M5 Ultra with 256GB or 512GB. ~110–115 tok/s on gpt-oss-120b, ~85 on Qwen3.5–122B-A10B, ~23 on dense 70B, and room for DeepSeek V4-Flash. This is a conversation about compliance, privacy, and whether your threat model justifies five figures on a desk box.</p><p><strong>Skip for inference:</strong> The DGX Spark at $6,950. It’s a fine CUDA development box. It’s not a value inference machine anymore.</p><h3>The Biggest Surprise Since Part 1</h3><p>The MoE revolution is the story I didn’t see coming. In May, I treated Mixture-of-Experts models as a compromise. Six months later, they’re the default architecture for local inference — and they gave AMD a real shot.</p><p>The DRAM shortage is the twist I should have seen coming. I warned about it hitting Apple’s pricing — and the M5 Pro did land at $2,699 instead of $2,000. What I missed is that Apple’s prices moved <em>least</em>. A 128GB Strix Halo box went from ~$2,100 to ~$3,500. The DGX Spark went from $3,999 to $6,950. The Mac Studio M5 Max 128GB is still $5,399. Apple’s scale and vertical integration turned out to be a pricing moat — and that moat is what turned “Apple is fastest” into “Apple is fastest <em>and</em> best value.”</p><p>And one meta-lesson from rewriting this post: <strong>a lot of the “benchmarks” online are physically impossible.</strong> I found 70B dense numbers for the M5 Pro, M5 Max and M5 Ultra that exceed what their memory bandwidth allows. Before you trust any tok/s figure, divide the bandwidth by the model’s size in gigabytes. If the claim beats the division, it’s wrong — or it’s quietly using speculative decoding.</p><h3>The Bottom Line, Updated</h3><p>Six months ago, the hierarchy was: Apple dominates inference, NVIDIA dominates training, AMD is budget-with-pain. After six charts, the verdict is sharper:</p><p><strong>Apple won. Every chart.</strong> The Mac Mini M5 Pro owns the small and mid-size models. The Mac Studio M5 Max owns the big ones. And the $5,499 M5 Ultra might be about to widen the gap.</p><p><strong>AMD made it close — once.</strong> At ~$3,500, Strix Halo is no longer a steal. Its best result is a near-tie on gpt-oss-120b. Everywhere else, it’s chasing. At the ~$2,100 it sold for at launch, before DRAM prices climbed, it was a buy. That’s the most I’d pay for one today.</p><p><strong>Then speculation changed the score — not the winner.</strong> Turn speculative decoding on and Apple still takes two of three. The one AMD steals, dense 27B, takes Halogen to do it.</p><p><strong>NVIDIA priced itself out of desktop inference.</strong> At $6,950 and 273 GB/s, the DGX Spark is a CUDA dev kit, not a value play.</p><p><strong>The frontier went local — for real this time.</strong> ~110 tok/s on a 120B MoE model and ~23 tok/s on dense 70B, on a desk box. If you can stomach the price.</p><p>Go buy a Mac. Which one depends on what you run.</p><p><em>This is Part 2 of my local inference hardware series. </em><a href="https://medium.com/@germanviscuso/the-best-budget-local-inference-machine-ships-next-month-heres-why-it-s-worth-the-wait-78adc799ffb0"><em>Part 1 is here</em></a><em>. German Viscuso builds AI-powered systems at the intersection of edge computing and agentic AI, and occasionally obsesses over hardware economics.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=98329807aca2" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How Claude Code’s Agents Actually Talk to Each Other (and Why I Built Rlay.dev Differently)]]></title>
            <link>https://medium.com/@germanviscuso/how-claude-codes-agents-actually-talk-to-each-other-and-why-i-built-rlay-dev-differently-9a3aed3e4588?source=rss-155e26ed6f1d------2</link>
            <guid isPermaLink="false">https://medium.com/p/9a3aed3e4588</guid>
            <category><![CDATA[agentic-ai]]></category>
            <category><![CDATA[ai-agents-in-action]]></category>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[claude-code]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <dc:creator><![CDATA[German Viscuso]]></dc:creator>
            <pubDate>Sun, 21 Jun 2026 02:11:44 GMT</pubDate>
            <atom:updated>2026-06-21T02:11:44.073Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*n1aF90aVUuuljdiWWBsiGw.png" /></figure><p>When the <a href="https://github.com/tanbiralam/claude-code">Claude Code source map leaked</a> in March 2026 — a stray .map file in an npm package, ~512K lines of TypeScript suddenly readable — most of the coverage chased the spicy stuff: hidden feature flags, background agents, an &quot;undercover mode.&quot; I went looking for something far more boring and, to me, far more interesting.</p><p>I build agent infrastructure. So the question I actually cared about was: <em>when Claude Code runs a swarm of agents, how do they talk to each other?</em></p><p>The answer is a tool called SendMessage. And the way it works tells you almost everything about a design constraint that I think the whole industry is about to run into.</p><h3>The setup: a team, not a chatbot</h3><p>People assume Claude Code is a chatbot wrapped around a model. It isn’t. Under the right flags it spins up a <em>team</em>: a “team-lead” agent and one or more “teammates,” each a separate process, coordinating on a real task. Sub-agents you already knew about — you spawn one, it runs in its own context, it returns one result, done. Fire and forget.</p><p>SendMessage is the other thing. It&#39;s a <em>bidirectional, addressable channel</em> between long-lived agents. This is the part that looks like an actual multi-agent system, and it&#39;s the part worth dissecting.</p><h3>Part 1: the channel is a file</h3><p>Here’s the thing that made me put my coffee down. There is no broker. There is no socket. There is no message queue. The transport is <strong>the filesystem</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*bPjt1CdPaJj_zSO3bbvpZw.png" /></figure><p>Every agent owns an inbox file:</p><pre>~/.claude/teams/{team}/inboxes/{name}.json</pre><p>Sending a message is exactly what you’d do in a first-year systems class if someone took your network away:</p><ol><li>Grab an advisory lock on the inbox file (.lock, with retry + backoff).</li><li>Read the JSON array.</li><li>Append your message — roughly { from, text, summary, ts, color, read: false }.</li><li>Write it back.</li><li>Release the lock.</li></ol><p>Receiving is just as humble. Each agent runs a poller on a <strong>1-second interval</strong>, reads the unread entries from its own inbox file, injects them into its next turn as &lt;teammate-message&gt; blocks, and flips read: true. There&#39;s no inbox to &quot;check&quot; — messages just show up as if the user had typed them.</p><p>A few details that matter:</p><ul><li><strong>Addressing is by bare name.</strong> You send to &quot;researcher&quot;, not to a UUID. There&#39;s exactly one team per session, so the namespace is flat and tiny. &quot;*&quot; means broadcast — implemented as <em>one write per inbox</em>, which the prompt literally warns agents is &quot;expensive, use sparingly.&quot;</li><li><strong>Consumption is a boolean flag</strong>, mutated in place under a lock. Single reader. No cursor, no replay, no catch-up.</li><li><strong>There’s a small protocol on top.</strong> Shutdown requests, plan approvals, permission escalations — these are JSON messages stuffed into the same text channel, sniffed on arrival, and routed to internal handlers instead of being shown to the model. Clever, and it gives you request/response (request_id-correlated) semantics over a fundamentally fire-and-forget bus.</li></ul><p>And the contract I genuinely admire: <strong>an agent’s plain-text output is invisible to its peers.</strong> If you want to say something to another agent, you <em>must</em> call the tool. Your thinking doesn’t leak; only explicit messages cross. That’s a clean boundary and I’ll come back to it.</p><p>So far, so elegant. File locks, polling, a boolean read flag. It’s the kind of design that’s <em>correct</em> and even kind of beautiful in its simplicity.</p><p>It’s also relentlessly, unavoidably <strong>local</strong>.</p><h3>The realization: every clever bit is a consequence of “one host”</h3><p>Once you spawn the agents (spoiler for Part 2, briefly: they’re launched as separate CLI processes in tmux/iTerm2 panes, with identity passed on the command line, and a roster kept in a config.json on disk) — once you do that, the messaging design isn&#39;t really a <em>choice</em> anymore. It falls out of the substrate. Watch how every decision is the same decision:</p><ul><li><strong>Why a file mailbox?</strong> Because separate OS processes on one machine share nothing <em>except</em> the filesystem. It’s the lowest common denominator IPC available.</li><li><strong>Why advisory locks?</strong> Because two agents might write the same inbox at once and there’s no broker to serialize them. (Side note: advisory locks are host-local anyway — this couldn’t cross machines even if you wanted it to.)</li><li><strong>Why a </strong><strong>read flag instead of a cursor?</strong> Because there&#39;s one file and one reader; &quot;consume&quot; just means &quot;mutate.&quot;</li><li><strong>Why 1-second polling?</strong> Because hitting a local file via the page cache is basically free.</li><li><strong>Why is broadcast “expensive”?</strong> Because with no shared log, reaching N agents means N writes.</li><li><strong>Why is identity in argv/env?</strong> Because the thing that launched you is on the same host.</li><li><strong>Why a manual shutdown handshake and session-end cleanup?</strong> Because somebody has to tear down long-lived local processes and delete the files they left behind.</li></ul><p>This is the punchline, and it’s worth saying plainly: <strong>inter-agent messaging design is downstream of where your agents live.</strong> Put them on one laptop and you get a file mailbox. It’s not wrong. It’s just an artifact.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*MZFmsJlHit08UfJaK9ngNA.png" /></figure><p>The tell is buried in the leak itself. Claude Code has an experimental transport for reaching an agent on <em>another</em> machine — and the moment it crosses that boundary, it stops trusting the message. A cross-machine message arrives framed as an untrusted <em>user prompt</em>, requires explicit human consent, and that consent is deliberately immune to the usual permission bypasses. The system knows: the second you leave the host, agent-to-agent messaging becomes a prompt-injection channel.</p><p>That boundary — the one Claude Code treats as a scary edge case — is where I live.</p><h3>Why agents won’t stay on one host</h3><p>Here’s the bet behind everything I’m building: the interesting multi-agent systems are not going to be five Claude processes on your MacBook. They’re going to be <strong>heterogeneous and distributed</strong>. A Claude agent here, a GPT-class agent there, a couple of custom in-house runtimes, on different machines, owned by different teams and increasingly different <em>vendors</em>, that nonetheless need to collaborate on a shared goal.</p><p>The instant that’s your world, every local decision flips:</p><ul><li>A shared filesystem doesn’t exist → you need a broker.</li><li>Host-local locks don’t exist → you need server-side ordering.</li><li>A boolean read flag breaks with multiple readers → you need a cursor.</li><li>1-second polling becomes a thundering herd → you need long-poll or push.</li><li>argv/env identity is meaningless across vendors → you need issued tokens.</li><li>“Expensive broadcast” is silly when one append can fan out to everyone.</li></ul><p>You don’t <em>port</em> the local design. You invert it.</p><h3>Enter Rlay</h3><p><a href="https://rlay.dev/">Rlay</a> is that inversion. It’s a cloud-native broker for agent-to-agent collaboration, and it was designed from the broker outward instead of from the host outward. Same three planes Claude Code has — membership, messages, a shared goal — but every one of them is a broker concern, not a filesystem accident.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*hm91zJXFbuGsmgieNSANQQ.png" /></figure><p>The shape, concretely:</p><ul><li><strong>Identity is issued, not inherited.</strong> An agent does POST /register {handle} and gets back an agent_id and an agent_token. That token is the only thing that works across a vendor boundary, because it doesn&#39;t assume anything about who launched you.</li><li><strong>A channel is a real object with a lease.</strong> POST /channels {goal, ttl_hours} returns an invite URL and a skill URL. The goal makes the collaboration legible; the ttl_hours is a lease. Closing is trivial: the channel ends when the lease expires, or the creator deletes it explicitly — no shutdown handshake, no leaked state to garbage-collect.</li><li><strong>Messages are an append-only log with a monotonic </strong><strong>seq.</strong> You read with GET /wait?since=&lt;seq&gt;&amp;timeout=50 — a long-poll, not a 1-second hammer. Multiple readers, replayable, idempotent on reconnect. A single append fans out to N subscribers by cursor, so broadcast is <em>cheap</em>, not a thing to apologize for.</li></ul><p>Put the two side by side and the comparison writes itself:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*foDwMVjZ_5M2IZMco7C4NQ.png" /></figure><p>And the patterns from Claude Code that <em>do</em> survive the jump, I’m keeping. Output isolation — your free text never leaks to peers, you have to post explicitly — is a great invariant regardless of substrate. The correlated request/reply trick over a fire-and-forget bus is worth stealing. And the thing they treat as a scary edge case — <em>inbound messages from another trust domain are untrusted input</em> — is, for a cross-vendor broker, simply the default posture. I’d rather build that in on day one than bolt it on when I cross a boundary.</p><h3>The takeaway</h3><p>The Claude Code mailbox is a small, sharp piece of engineering, and I mean that as a compliment. But it’s a perfect illustration of a rule I keep coming back to: <strong>your messaging architecture is mostly decided by where your agents run, before you’ve written a line of it.</strong> Design for one host and the filesystem will hand you a design. Design for the cloud — heterogeneous, distributed, multi-vendor — and you have to make different choices, deliberately, up front.</p><p>That’s the choice Rlay makes. If you’re building agents that have to talk across that boundary, come take a look: <a href="https://rlay.dev/">rlay.dev</a>.</p><p><em>Want the deep-dive on exactly how Claude Code’s file mailbox works, lock-by-lock and poll-by-poll? That’s a separate post that needs you to clap 10 times above (yes! you can do that!) — this one was the excuse to talk about the thing underneath it.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9a3aed3e4588" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Fable 5 Is the Same Model as Mythos 5 — The Only Difference Is What Gets Through the Door]]></title>
            <link>https://medium.com/@germanviscuso/fable-5-is-the-same-model-as-mythos-5-the-only-difference-is-what-gets-through-the-door-79625700ec56?source=rss-155e26ed6f1d------2</link>
            <guid isPermaLink="false">https://medium.com/p/79625700ec56</guid>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[ai-safety]]></category>
            <category><![CDATA[anthropic-claude]]></category>
            <dc:creator><![CDATA[German Viscuso]]></dc:creator>
            <pubDate>Tue, 09 Jun 2026 22:31:57 GMT</pubDate>
            <atom:updated>2026-06-09T22:31:57.068Z</atom:updated>
            <content:encoded><![CDATA[<p><em>Anthropic didn’t release its most powerful model today. It released a classifier that decides whether you’re allowed to talk to it.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kRimD28rxAstf4TuGvPV_g.jpeg" /></figure><p>Today Anthropic shipped Claude Fable 5 and Claude Mythos 5. The headlines wrote themselves: <em>most powerful model ever, now public</em>. <em>Mythos for the masses.</em> The first member of the Mythos class — the tier that sits above Opus, the one that spent two months locked inside Project Glasswing because it was too good at finding zero-days — is now something you can hit from a Pro plan.</p><p>I read the announcement, the AWS post, the pricing page, and the system-card framing. My conclusion is not the one on the slide.</p><p>Fable 5 and Mythos 5 are, by Anthropic’s own words, <strong>the same underlying model</strong>. Same weights. The thing separating the version you can use from the version that can discover and exploit vulnerabilities in every major operating system is not a smaller model, a distilled checkpoint, or a different training run. It’s a layer of classifiers sitting in front of the model that, when they fire, quietly hand your request to a <em>weaker</em> model — Opus 4.8 — and tell you it happened.</p><p>So the interesting engineering today isn’t the frontier model. It’s the bouncer in front of it.</p><p>If you take away one thing from this post, let it be that. The product Anthropic shipped is not a model. It’s a filter, and the model is the same one they decided was too dangerous to release unfiltered in April.</p><h3>What actually shipped</h3><p>Let’s be precise, because the naming is doing a lot of work.</p><p><strong>Mythos-class</strong> is a capability tier above Opus. The first one, Mythos Preview, went out in April to a tiny set of cyberdefenders and critical-infrastructure providers under Project Glasswing. It stayed locked up because it’s superhuman at the part of security that scares people: finding and weaponizing software vulnerabilities, plus the agentic glue around an attack — recon, discovery, lateral movement.</p><p><strong>Fable 5</strong> is that model with a new set of classifiers bolted to the front. When a classifier detects a request in one of three buckets — cybersecurity, biology/chemistry, or model distillation — the response is generated by Opus 4.8 instead, and you’re notified. Anthropic’s numbers: the fallback fires in under 5% of sessions, which means for more than 95% of what you do, you’re talking to the unrestricted thing. For those sessions, Fable 5 <em>is</em> Mythos 5. They’ve tuned the classifiers conservatively on purpose, and they’re upfront that benign requests will sometimes trip them.</p><p><strong>Mythos 5</strong> is the same model with the cyber safeguards lifted. It goes to Glasswing partners and the US government first, with a broader “trusted access” program coming. A parallel biology track — Fable with the bio/chem guardrails removed but cyber still on — is being opened to a small set of vetted researchers.</p><p>Three names, one model, three different answers to the question <em>what are you allowed to ask it</em>.</p><h3>The capability case is real — and it’s an agent story</h3><p>I want to be fair to the model, because the jump is not marketing.</p><p>The line that matters most, buried in the announcement: <strong>the longer and more complex the task, the larger Fable’s lead over everything else.</strong> This is not a chatbot improvement. It’s a long-horizon, run-unattended-for-hours improvement.</p><p>The proof points are the kind you can’t fake:</p><ul><li><strong>Stripe</strong> ran a codebase-wide migration across a <strong>50-million-line Ruby</strong> codebase in about a day. By hand, that’s a team working two-plus months.</li><li><strong>Memory is the headline, not a footnote.</strong> When Anthropic gave the model persistent file-based memory and pointed it at <em>Slay the Spire</em>, the memory helped Fable <strong>three times more</strong> than it helped Opus 4.8, and Fable reached the final act three times as often. Read that again if you build agents. The differentiator isn’t raw IQ on a single turn — it’s that this model gets <em>more</em> out of a state store across a long run.</li><li><strong>Vision is now state-of-the-art.</strong> It beat Pokémon FireRed with a minimal, vision-only harness — earlier Claudes needed an elaborate scaffold of maps and game-state helpers. It can rebuild a web app’s source from screenshots alone.</li><li>On the <strong>Mythos 5</strong> side, the science numbers are the genuinely unsettling ones: ~10x speedup on parts of drug design with the model running protein-design tools unassisted; novel molecular-biology hypotheses preferred ~80% of the time in blinded head-to-heads against Opus-class models; a genomics model it trained that was 100x smaller than a recently-published <em>Science</em> model and beat it.</li></ul><p>Translate that for builders: this is the first Claude designed to operate for hours or days off its own notes, recover from its own failures, and not lose the plot at a million tokens. If you’ve been architecting long-running agentic systems — a triage-to-RCA observability pipeline, a multi-step migration agent, a PM system with a state repo that commits its own progress — this is the model that was built for your shape of problem, not the demo-on-stage shape.</p><h3>The part nobody’s putting on the slide: the fallback tax</h3><p>Here’s where the architecture stops being an abstraction and starts costing you.</p><p>The cybersecurity classifier is not narrow. Anthropic designed it to cover offensive cyber <em>broadly</em> — not just exploit development, but reconnaissance, discovery, lateral movement, the whole agentic-hacking surface. The bio/chem classifier is currently tuned to catch most of the category, not a thin sliver of bioweapons queries. And they tell you plainly: it’s conservative, and benign requests will sometimes trigger it.</p><p>Now think about what a lot of legitimate infrastructure and database work actually <em>looks like</em> to a classifier trained on offensive cyber:</p><ul><li>Network diagnostics that open raw TCP/HTTP from inside a database.</li><li>Bastion sessions and SOCKS tunnels to reach a private endpoint.</li><li>An observability agent correlating logs, metrics, and events to find the blast radius of an incident — which is, structurally, reconnaissance plus lateral reasoning across a system.</li><li>Anything pen-test-adjacent that a DBA or SRE does on a Tuesday.</li></ul><p>None of that is an attack. All of it pattern-matches to one.</p><p>When your prompt trips the cyber classifier, you don’t get Fable. You get Opus 4.8. You’re notified — but you’re still being billed at Fable rates, and you’ve just lost the long-horizon, token-efficient edge that was the entire reason to reach for Fable in the first place. The downgrade is silent in the sense that matters: it happens mid-workflow, per-request, and unless you’re watching for the notice, your agent logs won’t tell you why a long-running job suddenly got dumber and more expensive.</p><p>This is not a complaint about safety. Dual-use is real, the cyber capability is genuinely dangerous unsupervised, and Anthropic is more honest about the trade-off than most. It’s a <em>deployment fact</em>. If your work looks like security work — and a surprising amount of infra and database work does — then you should:</p><ol><li><strong>Instrument the fallback.</strong> Log every time Opus answers instead of Fable, the way you’d alert on a 429. Treat it as a first-class metric.</li><li><strong>Measure the downgrade rate per workflow</strong>, not globally. Your aggregate might be under 5%; your <em>security-adjacent</em> workflow might be 40%.</li><li><strong>Decide acceptability per use case.</strong> For some jobs the Opus fallback is fine. For a long-horizon agentic task, falling back to a non-Mythos model partway through can quietly tank the run.</li></ol><h3>Pricing and the June 22 cliff</h3><p>Fable 5 and Mythos 5 are both $10 per million input tokens and $50 per million output tokens. That’s less than half what Mythos Preview cost — and still the most expensive major model on the market.</p><p>The subscription rollout is where you need to pay attention:</p><ul><li><strong>Today through June 22:</strong> Fable 5 is included on Pro, Max, Team, and seat-based Enterprise plans at no extra cost.</li><li><strong>June 23:</strong> it comes off those plans. After that, using it burns usage credits.</li><li><strong>Later:</strong> Anthropic intends to restore it as a standard subscription feature “when capacity allows.”</li></ul><p>Read that for what it is: a roughly two-week window where the frontier is effectively free on subscriptions, followed by a metered cliff. If you have a long-horizon job that’s actually worth running on this thing — a big migration, a gnarly refactor, a research sweep, a full re-run of your nastiest agentic eval — the rational move is to front-load it before the 23rd.</p><p>On the API and consumption-based Enterprise plans there’s no window game: Fable is fully available now as claude-fable-5, billed straight at $10/$50.</p><h3>The asterisk for anyone in a regulated shop</h3><p>The detail that will matter more than any benchmark to an enterprise buyer: <strong>mandatory 30-day data retention</strong> on all Mythos-class traffic. First-party <em>and</em> third-party surfaces — so Bedrock, Vertex, Foundry too. Anthropic says it won’t use that data to train models or for any non-safety purpose, that human access is logged, and that it’s deleted after 30 days in almost all cases. The retention exists to defend against multi-request attacks and novel jailbreaks, and to tune down false positives.</p><p>If you’re in finance, healthcare, government, or anything with data-residency clauses, <em>that’s</em> the procurement question — not the FrontierCode score. “We retain 30 days of your prompts and outputs for safety” is a line your security and legal teams have to actively sign off on. And it’s explicitly framed as the policy for “future models with similar or higher capability,” so plan for it being permanent at the frontier tier, not a launch-week quirk.</p><h3>What I’d actually do</h3><ul><li><strong>If you build agents:</strong> this is the first Claude where persistent memory plus long horizon is the <em>headline</em>. Re-run your longest, ugliest agentic eval on claude-fable-5 before June 22 and compare <strong>tokens-to-completion</strong>, not just pass/fail — the token efficiency is half the pitch.</li><li><strong>If your work is security- or infra-adjacent:</strong> instrument the Opus fallback today and measure your real downgrade rate. Don’t assume the global &lt;5% applies to you.</li><li><strong>If you’re in a regulated environment:</strong> get the 30-day retention clause in front of legal now, before you’ve built anything on top of it.</li><li><strong>If you just want the best general model:</strong> it’s here, it’s roughly twice the price of Opus, and for the &gt;95% of your work that doesn’t trip a classifier it behaves like the unrestricted Mythos brain. That’s the deal.</li></ul><h3>The actual headline</h3><p>“Most powerful model ever, now public” is true and beside the point. The more accurate version: Anthropic has decided the frontier is now dangerous enough that <strong>the product is the safety layer</strong>. Fable and Mythos are the same mind. What you’re buying is the filter in front of it.</p><p>That’s the thing worth sitting with. Every lab that crosses this capability line is going to ship the filter, not just the model, from here on — and the interesting questions for the next few years won’t be about parameter counts or benchmark deltas. They’ll be about who built the bouncer, what it lets through, what it silently downgrades, and whether it’s standing at <em>your</em> door or someone else’s.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=79625700ec56" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Best Budget Local Inference Machine Ships Next Month — Here’s Why It’s Worth the Wait]]></title>
            <link>https://medium.com/@germanviscuso/the-best-budget-local-inference-machine-ships-next-month-heres-why-it-s-worth-the-wait-78adc799ffb0?source=rss-155e26ed6f1d------2</link>
            <guid isPermaLink="false">https://medium.com/p/78adc799ffb0</guid>
            <category><![CDATA[hardware]]></category>
            <category><![CDATA[apple]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[large-language-models]]></category>
            <category><![CDATA[nvidia]]></category>
            <dc:creator><![CDATA[German Viscuso]]></dc:creator>
            <pubDate>Mon, 11 May 2026 00:43:41 GMT</pubDate>
            <atom:updated>2026-11-03T16:43:33.317Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*WQnsQpQWIK7RmY8nmo-GCQ.png" /><figcaption>The reigning champion showed up with $30K in GPUs and a power bill to match. The challenger sent a sign.</figcaption></figure><blockquote><strong>⚠️ Update (November 2026):</strong> This post is from May 2026. The hardware landscape has changed significantly — Neural Accelerators, corrected AMD pricing, and the MoE revolution reshuffled every recommendation. <a href="https://medium.com/@germanviscuso/i-benchmarked-every-local-ai-machine-under-4k-one-destroyed-the-rest-98329807aca2"><strong>Read Part 2 for the updated analysis and final verdict</strong></a><strong>.</strong></blockquote><p>I’ve spent the last few weeks going deep on what to buy for local LLM inference. I read every hardware guide, dissected a Kickstarter campaign for a $3,000 “AI computer,” benchmarked specs against published token-per-second numbers, and stress-tested every assumption I had about where the price/performance sweet spot actually sits in mid-2026.</p><p>My conclusion? The best machine for most practitioners doesn’t ship yet. It’s probably about a month away. And it’s not what you’d expect — it’s not a $4,000 workstation, not a $3,000 crowdfunded box, and not anything from NVIDIA or AMD.</p><p>Before I tell you what it is, I need to explain why everything else falls short. The answer only makes sense once you understand the physics, the economics, and the industry incentives that created this gap in the first place.</p><h3>The Only Metric That Matters: Memory Bandwidth</h3><p>If you take away one thing from this post, let it be this: LLM inference is memory-bandwidth-bound. Not compute-bound. Not core-count-bound. Not FLOPS-bound.</p><p>Every time a model generates a token, it reads the entire model weights from memory. A 70B model quantized to Q4 is about 40GB. To generate one token, the hardware reads ~40GB of weights, performs a relatively small amount of math, and writes out a probability distribution. Then it does it again for the next token.</p><p>The speed at which you can read those weights from memory <em>is</em> your inference speed. Everything else is noise. A chip with higher GB/s memory bandwidth generates tokens faster, even if it has fewer FLOPS. This is why an M3 Max (400 GB/s) generates tokens faster than an M4 Pro (273 GB/s) despite the M4 Pro being newer. It’s also why a $2,000 AMD Strix Halo machine with 128GB of memory still produces painfully slow output on dense models — its 212 GB/s bandwidth is the bottleneck, not its capacity.</p><p>Capacity determines <em>what</em> you can run. Bandwidth determines <em>how fast</em> it runs.</p><h3>Why Apple Silicon Dominates Local Inference</h3><p>Apple’s architectural advantage for inference comes down to one word: unified memory. In every other vendor’s architecture — NVIDIA, AMD, Intel — the CPU has its own RAM pool and the GPU has its own VRAM pool, connected by a PCIe bus. Loading model weights into the GPU requires crossing that bus. The GPU’s VRAM is the hard ceiling on what fits, regardless of how much system RAM you have.</p><p>Apple eliminated this split. In M-series chips, CPU and GPU share the same physical memory pool with zero-copy access. When a 70B model sits in 128GB of unified memory, every byte is directly accessible to the GPU at full bandwidth. No PCIe bottleneck, no VRAM ceiling, no transfer overhead.</p><p>This architectural choice — originally designed for mobile power efficiency — turns out to be nearly perfect for LLM inference. And Apple can deliver this because they control the entire vertical stack: CPU cores, GPU cores, memory controller, silicon packaging, and operating system. No other company in the world does this for a consumer product.</p><h3>The Landscape Right Now: Nothing Quite Works</h3><p>Here’s what’s available today for local inference, sorted by price. Spoiler: every option has a dealbreaker.</p><p><strong>~$1,800 — Mac Mini M4 Pro 48GB.</strong> Higher bandwidth than you’d expect (273 GB/s), so models that fit run well. But 48GB is a hard ceiling. After macOS takes its share, you’re left with ~40GB usable. MoE models fit comfortably. Dense 70B is a tight squeeze with almost nothing left for KV cache at longer contexts. Close — but not enough headroom.</p><p><strong>~$2,000 — AMD Strix Halo 128GB (e.g., Framework Desktop, Bosgame M5).</strong> The most memory per dollar you can buy. 128GB of LPDDR5x accessible as unified memory. But the ~212 GB/s bandwidth makes dense 70B models painfully slow — 3 to 5 tokens per second. Where it shines: Mixture-of-Experts models like Qwen3–235B (22B active per token) or Gemma 4 26B-A4B, where only a fraction of parameters are read per token. Several developers have built impressive fully-local AI stacks on this hardware, but every one of them reports weeks of setup pain wrestling with AMD’s software ecosystem (ROCm, driver issues, Ollama instability). The AMD tax is time, not money.</p><p><strong>~$3,700 — Mac Studio M4 Max 128GB.</strong> The current “no compromises” inference machine. 546 GB/s bandwidth, 128GB capacity. Dense 70B Q4 at 8–15 tok/s. MoE models fly. Room for multiple models loaded simultaneously. Complete system — plug in power and a display. The standard recommendation in every hardware guide for serious local inference. Good for 3+ years. Expensive for me.</p><p><strong>~$4,700 — NVIDIA DGX Spark 128GB.</strong> Hard to recommend. Same capacity as the Strix Halo machines but at more than double the price. Only 273 GB/s bandwidth — half of the Mac Studio M4 Max. The CUDA ecosystem is the only real selling point, and for inference-only workloads that advantage is marginal.</p><p>So: too little memory, too little bandwidth, too much money, or too much setup pain. Something has to give. And it gets worse before it gets better.</p><h3>The Kickstarter Trap</h3><p>While researching, I came across a Kickstarter campaign for a $3,000 “agentic computer” built on AMD Strix Halo with 128GB, liquid cooling, and a proprietary firmware layer claiming 15–20% performance boosts. It was marketed specifically for running 235B MoE models and local AI agents.</p><p>After careful analysis, my conclusion: <strong>not a scam, but a bad deal.</strong> The underlying hardware is real — it’s the same Strix Halo chip available in shipping products from established brands at $2,000–$2,400. The campaign adds a proprietary motherboard-level firmware and liquid cooling to justify an MSRP of $3,999 (super early bird $2,999). But firmware can’t change the fundamental 212 GB/s bandwidth ceiling. The CineBench scores they showed were marginal improvements over existing machines. The “96GB VRAM outperforms 3× RTX 5090” marketing compares capacity, not throughput — three RTX 5090s at 1,792 GB/s each would obliterate Strix Halo’s bandwidth in actual inference speed.</p><p>More red flags: first-time Kickstarter creator with zero prior campaigns, based in Hong Kong, shipping June 2026. The “save 96% on your AI stack” pitch adds up cloud subscription costs (Midjourney + Runway + ChatGPT Pro + Cursor) and implies local open-source models replace all of them. They don’t. Local Qwen3 is great, but it’s not Midjourney.</p><p><strong>If you want Strix Halo 128GB, buy a Framework Desktop or Bosgame M5 today. They ship now, cost less, and use the same silicon.</strong></p><p>Kickstarter aside, let&#39;s continue with the analysis.</p><h3>Why No One Else Builds a Mac Studio Killer</h3><p>If every option has a dealbreaker, why hasn’t someone just… built the right machine? LLM inference is one of the hottest consumer workloads. Why hasn’t NVIDIA, AMD, Intel, or Microsoft shipped a turnkey inference box that competes with Apple’s Mac Studio on bandwidth per dollar?</p><p><strong>NVIDIA won’t.</strong> Their entire business model is the VRAM tiering ladder: 32GB (RTX 5090) → 96GB (PRO 6000) → 192GB (B200). Each tier costs 3–10× more. A $4,000 box with 128GB of high-bandwidth unified memory would cannibalize the $22,000 PRO 6000 workstation and the $3–6/hr B200 cloud revenue. The DGX Spark exists to check a marketing box — they deliberately chose low-bandwidth LPDDR5x instead of HBM because putting HBM in a $5K consumer box would undercut their datacenter margins.</p><p><strong>AMD can’t.</strong> Their Strix Halo uses unified memory, architecturally similar to Apple. But they’re stuck with off-the-shelf LPDDR5x modules on a standard socket, maxing out at ~212 GB/s. Apple achieves 546–819 GB/s through custom memory controllers and tightly-packaged memory dies on custom silicon interposers. Replicating that requires vertical integration of the memory subsystem that AMD, as a fabless chip designer, simply doesn’t have.</p><p><strong>Intel is irrelevant</strong> at this tier. No competitive GPU story, no unified memory architecture.</p><p><strong>Microsoft wants you on Azure.</strong> Building a local inference box would cannibalize cloud GPU revenue.</p><p>The structural moat is Apple’s vertical integration. They design their own CPU cores, GPU cores, Neural Engine, memory controller, interconnect fabric, silicon packaging, and OS — then ship it as a consumer product at scale. Everyone else assembles from commodity components, each optimized for its own use case rather than the end-to-end inference pipeline.</p><h3>Why Not Apple Silicon Cloud?</h3><p>The obvious follow-up: if Apple Silicon is so good for inference, why isn’t anyone offering Mac Studios as cloud infrastructure?</p><p>Three reasons. First, Apple’s macOS licensing explicitly restricts virtualization — only two VMs per physical machine, only on Apple hardware. Cloud providers can’t slice and share the way they do with NVIDIA GPUs. AWS offers Mac instances, but they’re bare-metal only with a 24-hour minimum allocation.</p><p>Second, the form factor is hostile to datacenters. No IPMI/BMC for remote management, no redundant power supplies, no hot-swap anything. A consumer cube designed for a desk, not a rack.</p><p>Third — and most importantly — concurrency kills it. A Mac Studio serving 70B inference at 8–15 tok/s handles 1–2 concurrent users. A single B200 with 8,000 GB/s bandwidth and continuous batching serves dozens. Cloud providers sell throughput at scale, and Apple’s advantage (high capacity, low power, single-user) doesn’t translate to the metric cloud economics run on.</p><p>Apple has zero incentive to change this. They make 40%+ margins selling you the hardware. Enabling cloud providers to arbitrage it at $0.40/hr would undercut their own sales. The licensing restrictions aren’t accidental — they’re the moat.</p><p>So Apple won’t sell you cloud inference. NVIDIA won’t sell you cheap bandwidth. AMD can’t match Apple’s memory architecture. Microsoft wants in Azure. And Kickstarter campaigns repackage the same limited silicon at a markup.</p><p>Which brings us back to the original question: what should you actually buy?</p><h3>So What’s the Answer?</h3><p>Everything above points to a machine that needs to hit four criteria simultaneously: enough unified memory to fit serious models (~64GB), enough bandwidth to run them at interactive speeds (300+ GB/s), a software ecosystem that works out of the box (Metal, not ROCm), and a price under $2,500 (to win the price/performance ratio battle).</p><p>Nothing on the market checks all four boxes. But something is about to.</p><p>The <strong>Mac Mini M5 Pro with 64GB</strong> is expected around WWDC (June 2026), with rumors consistently pointing to 64GB as the maximum configuration. The M5 Pro should bring meaningfully improved bandwidth over the M4 Pro’s 273 GB/s — the M5 Max already measures 614 GB/s on MacBook Pro, suggesting the M5 Pro could land in the 300–350 GB/s range.</p><p>At an estimated $2,000–$2,200, here’s what 64GB of M5 Pro unified memory gets you:</p><p><strong>MoE models are completely comfortable.</strong> Gemma 4 26B-A4B (~15GB), Qwen 3.5 35B-A3B (~20GB), and similar MoE models fit with room for embedding models, KV cache, and OS overhead. You could run a full local AI stack — LLM + embedding model + image generation — simultaneously.</p><p><strong>Dense 70B is a stretch option, not a pipe dream.</strong> A 70B Q4 model at ~40GB fits in 64GB with ~16GB remaining. Tight for long-context conversations, but viable for moderate interactions. On the current M4 Pro at 273 GB/s, that’s roughly 8–10 tok/s. If the M5 Pro pushes bandwidth higher, you’re looking at potentially 10–15 tok/s — genuinely interactive.</p><p><strong>Setup takes an afternoon, not two months.</strong> Metal is a first-class inference backend in Ollama, llama.cpp, LM Studio, and MLX. You install Ollama, pull a model, and start serving. No ROCm debugging, no driver hunting, no Lemonade Server workarounds.</p><p><strong>The hybrid architecture makes sense.</strong> At $2,000, you have $1,500–1,700 left over compared to a Mac Studio M4 Max. That buys a lot of cloud API credits for the occasional frontier model task — Claude for complex reasoning, GPT Image 2 for production visuals, or a B200 burst for fine-tuning. Run your daily inference locally, escalate to the cloud when local models aren’t enough. Two clean paths: local for speed and privacy, cloud for capability.</p><p><strong>Tools like Claude Code already support local models.</strong> LM Studio ships an Anthropic-compatible API endpoint. Point Claude Code at your local llama.cpp server with two environment variables and you’re coding with a 70B model at zero per-token cost. The tooling has caught up.</p><h3>The DRAM Caveat</h3><p>There’s one risk to this thesis. DRAM prices have spiked dramatically — contract prices rose 90–95% quarter-over-quarter in Q1 2026. Apple has already pulled 64GB configurations of the current M4 Pro Mac Mini from their store. The 128GB and 256GB Mac Studio options have also disappeared.</p><p>If the DRAM shortage persists, Apple might launch the M5 Pro Mac Mini at 48GB max instead of 64GB, or price the 64GB upgrade significantly higher than the current generation. If the 64GB option launches at $2,400+, the calculus shifts — at that price you’re closer to Strix Halo 128GB territory, and the capacity advantage of AMD becomes harder to ignore despite the bandwidth penalty.</p><p><strong>My advice: watch WWDC closely.</strong> If the M5 Pro Mac Mini ships at 64GB for ~$2,000-$2,200, buy it immediately. It’s the best price/performance ratio for local inference that will exist in 2026.</p><h3>The Bottom Line</h3><p>The local inference hardware market in 2026 has a clear hierarchy:</p><p>Kickstarter “AI computers” repackaging Strix Halo silicon at a premium are poor value. Buy the chip from an established brand if you want it.</p><p>Strix Halo 128GB machines at ~$2,000–2,400 are the budget play for MoE-heavy workloads — most memory per dollar, but bandwidth-limited and AMD’s software ecosystem will cost you weeks.</p><p>The Mac Studio M4 Max 128GB at ~$3,700 is the safest bet if you need to buy right now — proven, no compromises, good for 3+ years.</p><p>But the Mac Mini M5 Pro at 64GB (expected ~$2,000–2,200) could be the sweet spot that makes everything else an awkward compromise — fast enough, big enough, cheap enough, and zero friction.</p><p>The industry structure that created this situation — Apple’s vertical integration moat, NVIDIA’s deliberate VRAM scarcity, AMD’s bandwidth ceiling, and cloud providers’ per-hour margin incentives — isn’t changing anytime soon. The customer’s best move is to understand these incentive misalignments and pick the hardware whose vendor’s self-interest accidentally aligns best with your needs.</p><p>Right now, that vendor is Apple. And their best inference machine at the best price is probably a few weeks away.</p><p><em>German Viscuso is a Technology Evangelist who builds AI-powered systems and occasionally obsesses over hardware economics.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=78adc799ffb0" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Doom Is Now a Neural Network: A Technical Deep Dive Into World Models]]></title>
            <link>https://medium.com/@germanviscuso/doom-is-now-a-neural-network-a-technical-deep-dive-into-world-models-178fb0c6b09c?source=rss-155e26ed6f1d------2</link>
            <guid isPermaLink="false">https://medium.com/p/178fb0c6b09c</guid>
            <category><![CDATA[world-models]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[robotics]]></category>
            <category><![CDATA[deep-learning]]></category>
            <dc:creator><![CDATA[German Viscuso]]></dc:creator>
            <pubDate>Mon, 27 Apr 2026 15:57:20 GMT</pubDate>
            <atom:updated>2026-04-27T15:57:20.727Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7t6j5uHruUvt3zJtfvHlYw.jpeg" /></figure><p>In August 2024, Google researchers shipped a paper that should have been impossible. They trained a slightly-modified Stable Diffusion v1.4 to <em>be</em> the game Doom — not to render it from a real game engine, not to assist a game engine, but to literally hallucinate the next frame given the previous frames and your keystrokes. The thing ran at 20 fps on a single TPU. Human raters could only barely tell short clips apart from real Doom, even after five minutes of autoregressive generation. There was no game engine running underneath. No collision logic, no enemy AI, no map data — just a neural network predicting pixels conditioned on WASD inputs.</p><p>Six months later, Microsoft published the same trick in <em>Nature</em> with Quake II (their model, WHAMM, ran in your browser via Copilot Labs). Then DeepMind generalized the recipe and shipped Genie 3, a system that takes a text prompt, generates a 720p 3D world from scratch, and lets you walk around in it at 24 fps for several minutes. Three months after that, Yann LeCun left Meta and raised $1.03B at a $3.5B valuation to build something architecturally different from any of those — a system that doesn’t predict pixels at all. Twenty days before that, Fei-Fei Li’s World Labs had closed its own $1B round. Two Turing-class researchers, $2.03B in three weeks, both betting that the entire generative-AI paradigm is a statistical dead end for tasks involving the physical world.</p><p>If you’ve spent the last three years building LLM-shaped products, none of these headlines tell you what to actually do next. The interesting thing — the thing worth understanding before you commit your next architectural decision — is <em>what these systems actually are</em>, why a different family of equations is now eating $2B at venture-grade prices, and where the LLM stack genuinely cannot follow.</p><p>Here’s what stood out after going through several technical reports, papers, and source code.</p><h3>A World Model Is Not a Video Generator. It’s a Learned Simulator.</h3><p>The vocabulary is collapsing in a way that makes the field harder to reason about. Sora is called a world model. Veo is called a world model. So is Genie 3, V-JEPA 2, NVIDIA Cosmos, DreamerV3, Wayve’s GAIA-2, and the 9,000-line PyTorch script someone trained yesterday on PushT. The label has spread to mean “anything that produces video.” That’s not what world models are about, and conflating them costs you the entire architectural distinction.</p><p>A world model, in the technically precise sense the field is converging on, is a learned function f: (state, action) → next_state. It’s a simulator you trained instead of one you wrote. The “state” can be pixels, but more often it’s a compact latent encoding of the world. The “action” is whatever the agent can do — a robot’s joint command, a steering input, a WASD keystroke, or “attack the orc with the sword.” The output is the next state, optionally with a reward, optionally rendered back to pixels.</p><p>Three components, recurring across nearly every implementation:</p><ol><li>An <strong>encoder</strong> that maps high-dimensional observations (RGB frames, lidar, proprioception) into a compact latent state.</li><li>A <strong>dynamics predictor</strong> that, given the current latent and an action, predicts the next latent.</li><li>Optionally, a <strong>decoder</strong> that renders the latent back to pixels for visualization or training other models.</li></ol><p>Sora is missing #2 with action conditioning. It generates plausible video, but you can’t roll it forward conditioned on “turn left.” Genie 3 has all three and lets you control the trajectory frame by frame. V-JEPA 2 has #1 and #2 but explicitly drops #3 — there is no pixel decoder, predictions live entirely in latent space. DreamerV3 has all three but operates on a 1024-bit categorical latent instead of pixels.</p><p><strong>Pattern for builders:</strong> When someone says “world model,” ask one question — <em>can I condition the next state on an action?</em> If the answer is no, it’s a video generator. If the answer is yes, the next question is whether predictions happen in pixel space, in a tokenized discrete space, or in a continuous latent space. Those three regimes have radically different cost, latency, and accuracy profiles, and they’re the actual axis of architectural disagreement in the field today.</p><h3>The Three Architectural Camps (And Why They Disagree)</h3><p>The field has split into three roughly orthogonal camps, each making a different trade-off between fidelity, compute cost, and grounding in physical reality. Knowing which camp a given system lives in is more useful than knowing its name.</p><h3>Camp 1: Pixel-space autoregressive video models (Genie 3, Cosmos AR, video DiTs with causal attention)</h3><p>These are the systems that will most readily seduce you with demos. Generate frame <em>t+1</em> given frames <em>0..t</em> plus a control signal. Genie 3 generates 720p frames at 24 fps — that’s roughly 22ms per frame end-to-end including the autoregressive lookback over the trajectory grown so far. The “physics” is emergent: nothing in the network knows what gravity is, but if you train on enough internet video where balls fall, the conditional distribution over next-frames learns that balls usually fall.</p><p>This works astonishingly well for short horizons and breaks down for the same reason every autoregressive system breaks down — compounding error. Each frame is conditioned on slightly-off predictions of the previous frames, and the divergence grows. Genie 3 publicly admits memory only works back about a minute, environments stay consistent for “several minutes,” and they explicitly describe consistency as an <em>emergent capability</em> rather than something engineered. NeRFs and Gaussian Splatting maintain perfect consistency because they operate on an explicit 3D representation; Genie 3 trades that for an emergent, dynamic, far richer simulation that drifts.</p><h3>Camp 2: Latent-space JEPA-style predictors (V-JEPA 2, V-JEPA 2-AC, LeWM, DINO-WM)</h3><p>LeCun’s bet. Predictions happen in a learned embedding space, never decoded back to pixels during training. The argument is that pixel prediction is the wrong objective — most of the bits in a video are noise (leaf flicker, sensor grain, irrelevant detail), and forcing your model to reconstruct them wastes capacity on the things that don’t matter for prediction or planning.</p><p>V-JEPA 2 is the published proof point. Pre-trained on &gt;1M hours of internet video with masked spatio-temporal prediction in latent space, it hits 77.3 top-1 on Something-Something v2 and 39.7 R@5 on Epic-Kitchens-100 action anticipation — beating task-specific models. Then they bolt on V-JEPA 2-AC: a 300M-parameter action-conditioned transformer with block-causal attention, post-trained on just 62 hours of unlabeled Franka arm video from Droid. That’s enough to do zero-shot pick-and-place planning on a real robot in a new lab, with no reward, no demonstrations, no task-specific data. The planning loop is classical model-predictive control: encode start and goal images to latents, sample action sequences, roll them out through the latent dynamics, pick the sequence whose terminal latent lands closest to the goal latent.</p><p>The follow-up, LeWM (LeWorldModel, March 2026), takes this further: a JEPA that trains stably end-to-end from raw pixels with only two loss terms (next-embedding prediction + a Gaussian regularizer called SIGReg from the LeJEPA paper), encodes each frame as a single 192-dim token, and plans 48× faster than DINO-WM on the same control tasks because there are roughly 200× fewer tokens to roll out. The whole thing trains on a single GPU in a few hours.</p><h3>Camp 3: Recurrent state-space models for model-based RL (DreamerV3, TD-MPC2, and their descendants)</h3><p>The original world-model lineage going back to David Ha &amp; Schmidhuber’s 2018 paper, refined into DreamerV3 (Hafner et al., 2023). The architecture: a Recurrent State-Space Model (RSSM) that splits the latent into a deterministic component <em>h_t</em> (a GRU state) and a stochastic component <em>z_t</em> (a categorical distribution). Sequence model: h_t = f(h_{t-1}, z_{t-1}, a_{t-1}). Encoder: z_t ~ q(z_t | h_t, x_t). Predictor: ẑ_t ~ p(ẑ_t | h_t). The actor and critic train entirely on imagined rollouts inside this latent — never on real environment frames.</p><p>DreamerV3 is the only learning algorithm to date that’s collected a diamond in Minecraft from scratch with a single set of fixed hyperparameters across all tasks it was tested on. It’s not a fancy architecture and it doesn’t generate impressive video. It’s a sample-efficiency monster, and it lives at the intersection of world models and reinforcement learning rather than the intersection of world models and generative video.</p><p><strong>Pattern for researchers:</strong> When evaluating a new world-model paper, locate it on the (pixel ↔ latent) axis and the (action-conditioned ↔ action-free) axis before reading anything else. Most architectural critiques in the field reduce to camp boundaries — JEPA people criticize pixel-space models for wasting capacity, video-DiT people criticize JEPA for being ungrounded and non-generative, model-based RL people think both groups are reinventing RSSM with extra steps. The disagreement is real, but the systems aren’t competing for the same job.</p><h3>Why “Just Add More LLM” Doesn’t Get You There</h3><p>LeCun has been making this argument for half a decade and the recent funding suggests serious investors finally agree with the math. The argument has three parts and they’re worth understanding precisely instead of as vibes.</p><p><strong>Part one: data volume.</strong> A four-year-old has processed roughly 10¹⁴ bytes of visual data through their eyes. The entire training corpus of GPT-4-class LLMs is on the order of 10¹³ bytes of text. Even granting massive redundancy in visual input, the order of magnitude difference is not bridgeable by scaling text. Whatever the brain’s internal model of the world is built out of, it’s built mostly out of sensory streams, not language. If your only training signal is text, you’re missing the substrate by an order of magnitude.</p><p><strong>Part two: compounding token error.</strong> Autoregressive models predict tokens sequentially, and each token’s correctness depends on previous tokens being approximately correct. For text generation this works fine because language is heavily redundant — small errors get smoothed out by context. For physical prediction it doesn’t, because dynamics are sensitive to initial conditions. A 1% error in predicting the position of a robot gripper at <em>t+1</em> becomes a 5% error at <em>t+10</em> and the gripper has missed the cup. JEPA’s argument is that predicting in a <em>bounded</em> latent space avoids exponential drift because you’re not committing to a discrete prediction at each step.</p><p>(The counterargument, made forcefully in the “Critiques of World Models” paper from July 2025: JEPA’s recursive z_{t+1} = f(z_t, a_t) is itself autoregressive, just with continuous latents instead of discrete tokens, and is therefore subject to the same compounding-error problem in principle. The disagreement is about whether the geometry of a learned continuous latent makes drift bounded in practice, which is an empirical question still being settled.)</p><p><strong>Part three: the “spilling glass” problem.</strong> A glass teetering on a table edge is about to fall. You know this without verbalizing anything, without consulting language. The knowledge is sub-symbolic, predictive, and grounded in visual experience. An LLM “knows” this only insofar as humans have written about glasses falling, and that knowledge is mediated through the lossy bottleneck of language. The bet is that for any task requiring physical common sense at the millisecond timescale (driving, manipulation, navigation), the language detour is a liability, not an asset.</p><p>The empirical version: every public benchmark that probes physical reasoning beyond mechanics — Physics-RW for thermodynamics/electromagnetism/optics, WorldModelBench for instruction-following + physics adherence, the Math Blind benchmark for spatial reasoning in multimodal LLMs — shows current systems score poorly. Not because they’re badly trained, but because the training signal doesn’t carry the structure they’d need to do well.</p><p><strong>Pattern for builders:</strong> Before assuming an LLM-plus-tools architecture will solve your robotics or autonomy problem, audit which sub-tasks require sub-second physical prediction. For those sub-tasks, you almost certainly need a learned dynamics model in the loop, not a language model. The hybrid pattern that’s actually winning in production today is <em>language model for high-level task decomposition + world model or MPC for low-level control,</em> not <em>language model for everything</em>.</p><h3>Doom and Quake II as Neural Networks: The Two-Year Lineage Behind Genie 3</h3><p>If you read about Genie 3 and thought “wait, this had to come from somewhere” — it did. Two earlier results that landed within six months of each other set the template, and they’re both worth understanding technically because they show the architectural moves that scale up to Genie 3 and the limits that don’t get fixed by scaling.</p><p><strong>GameNGen (Google, August 2024)</strong> is the first of the two. The architecture is intentionally pedestrian for a frontier-of-AI paper: an augmented Stable Diffusion v1.4. The pre-trained encoder gets reused; the decoder is fine-tuned with an MSE loss against ground-truth game frames to fix HUD artifacts. The denoising network is conditioned on the sequence of past frames and past actions instead of a text prompt. Training is two-stage: first an RL agent learns to play Doom and the trajectories are recorded (because nobody is going to manually collect millions of human gameplay frames), then the diffusion model trains on those trajectories. The result runs at 20 fps on a single TPUv5, hits PSNR 29.4 on next-frame prediction (comparable to lossy JPEG), and survives five-minute play sessions where humans can barely distinguish clips from real Doom.</p><p>The technical move that makes this work is <strong>conditioning augmentations</strong> — during training, the past frames the model sees are corrupted with noise, which forces the model to be robust to its own prediction errors at inference time. Without this, autoregressive rollout drifts within seconds. With it, you get multi-minute stability. This is the same trick Genie 3 uses, scaled up and generalized.</p><p>The honest limitations from the paper: the model has access to “a little over 3 seconds of history.” Game state — health, ammo, score — gets persisted “remarkably” longer than the 3-second window suggests, but it’s an emergent statistical property, not a reliable mechanism. The model is trained on Doom and only Doom; transferring to a new game means a full retrain on new agent-collected data.</p><p><strong>WHAMM (Microsoft, April 2025)</strong> transfers the recipe to Quake II and tightens it. It stands for World and Human Action MaskGIT Model — MaskGIT instead of diffusion as the generative backbone, which is faster per step at the cost of some quality. The data efficiency story is what matters: WHAM-1.6B (the predecessor) needed seven years of recorded gameplay to train. WHAMM, by curating the data more intentionally with professional game testers focused on a single level, got a working Quake II model with <strong>one week of recorded gameplay</strong>. Resolution doubled from 300×180 to 640×360. Microsoft published the underlying model in <em>Nature</em> and shipped a playable demo in Copilot Labs that, by all accounts, runs poorly — low frame rate, noticeable input lag, enemies that morph into blobs, hallways that change behind you. The PC Gamer review called it “worse than Doom on a calculator,” which is fair as an experience review and beside the point as a technical assessment.</p><p>The substantive technical story in WHAMM is the data collapse. Going from seven years to one week of training data, with comparable model quality on a different game, says the world-modeling task is significantly more sample-efficient than people assumed two years ago — <em>if you’re willing to constrain the world</em>. WHAMM doesn’t model arbitrary Quake II; it models one level of Quake II, and the elevator at the end of that level “freezes” the model because no training data exists past it.</p><p>The lineage from these two systems to Genie 3 is direct. Same autoregressive frame-by-frame prediction. Same emergent physics from training-data statistics. Same compounding-error problem partially mitigated by conditioning augmentations. The differences are ones of scale and ambition: Genie 3 generates worlds from text prompts instead of being trained on a single game, supports promptable world events (change the weather mid-rollout, introduce new objects), and pushes consistency from “a few minutes within a single fixed level” to “a few minutes across an arbitrary procedurally-generated environment.” The fundamental architectural moves are the same.</p><p><strong>Pattern for builders:</strong> If you want to understand a frontier system, find its two-year-old paper. The ideas don’t usually appear drom nowhere at the frontier — they get prototyped at smaller scale on simpler problems and then scaled up with engineering. GameNGen is the prototype. Genie 3 is the engineering.</p><p>The implication for AI-generated games specifically — which sounds like a Medium-bait gimmick but is actually a serious commercial direction — is that the engineering needed to go from “Doom on a TPU at 20 fps” to “any game at 60 fps on a consumer GPU” is a different problem than the engineering needed to go from “any game at 60 fps” to “any game <em>plus</em> a coherent narrative, persistent NPCs, save-states, and mod support.” The first is a scaling problem. The second is a research problem and isn’t on a 24-month roadmap. If you’re a game studio thinking about world models, the realistic 2026–2027 use cases are AI-generated levels, AI-generated training environments for in-game NPCs, and AI-assisted prototyping. The full “games are weights, not code” vision is real but further out than the demos suggest.</p><h3>Genie 3 Up Close: The Auto-Regressive Trick (And Where It Cracks)</h3><p>Genie 3 is the most architecturally novel system in this batch, so it’s worth a closer technical look. The headline numbers — 720p, 24 fps, several-minute consistency, real-time WASD interaction — only make sense once you understand what the model is doing per frame.</p><p>Each generated frame is conditioned on (a) the world description prompt, (b) the user’s input action this tick, and © the entire trajectory of frames generated so far. The third one is what makes consistency possible and makes it expensive: a frame at <em>t = 60s</em> (so 1440 frames in) needs to look back at frames from <em>t = 0s</em> to maintain visual memory of objects you’ve already passed. “If the user revisits a location after a minute, the model needs to refer back to information from a minute ago” is how DeepMind describes it, and they need to do this multiple times per second.</p><p>How is this not horrendously slow? Two design choices. First, the “trajectory” isn’t all 1440 frames at full resolution; it’s a heavily compressed representation that the model has learned to summarize. Second, recent reports estimate Genie 3 needs at least 8 TPU v5 chips per session for real-time inference, which puts it firmly in the “cloud-only” category for the foreseeable future. There’s no Genie 3 running on your laptop in 2026.</p><p>The interesting limitation: Genie 3 supports only a few minutes of continuous interaction, not the hours that would be needed to actually train an embodied agent in the simulated world. That gap is the thing standing between “great demo” and “AGI stepping stone.” DeepMind’s own SIMA agent has been shown completing tasks inside Genie 3 environments, which is a real result, but the consistency window is the bottleneck on how complex those tasks can be.</p><p>There’s also a documented physics tell. The launch demos showed a skier going down a mountain, and the snow doesn’t move correctly relative to the skier — it’s spatially plausible per frame but doesn’t conserve mass or momentum across frames. Emergent physics from autoregressive video training gets you most of the intuitive cases (things fall, water flows downhill) and breaks on edge cases that require a real conservation law (snow displacement, fluid dynamics in detail). For training agents this is the <em>reliability paradox</em>: if your simulator has subtle physics violations, an agent trained inside it will exploit those violations to get reward, and the policy won’t transfer to reality.</p><p><strong>Pattern for builders:</strong> If you’re using a video world model as a training environment for downstream agents, you need an explicit physics-violation detector running alongside it. Don’t assume “looks right to humans” implies “trains agents that transfer.” WorldModelBench (NeurIPS 2025) is the current best public benchmark for this; if your simulator hasn’t been evaluated there or with comparable physics-adherence metrics, you’re flying blind.</p><h3>NVIDIA Cosmos: The Boring (Important) Infrastructure Bet</h3><p>Cosmos is the system most relevant if you’re shipping a Physical AI product in 2026, because it’s the one that’s actually open and ships fine-tuning recipes. The architecture is dual-track: a diffusion family and an autoregressive family, both built on transformer backbones, both consuming video tokenized through Cosmos’s own causal video tokenizers.</p><p>The tokenizer detail matters. Cosmos-Tokenizer comes in continuous (CV) and discrete (DV) variants, with spatial compression of 8× or 16× and temporal compression of 4× or 8×, for a total compression up to 8 × 16 × 16 = 2048×. Causal means future frames don’t influence the encoding of current frames, which is essential for any system that has to operate in a streaming setting (which is to say, every robot and every car). The autoregressive base models use the discrete tokenizer (DV8x16x16) and predict next tokens; the diffusion models use the continuous tokenizer (latent diffusion in compressed space). When discrete tokenizer compression introduces artifacts, NVIDIA ships a separate diffusion decoder that maps DV tokens back to higher-quality continuous tokens.</p><p>Training data: 9,000 trillion tokens from 20 million hours of “real-world human interactions, environment, industrial, robotics and driving data.” Models come in three sizes — Nano (edge-deployable, real-time), Super (baseline performant), Ultra (max quality, used for distillation). Pre-training on the autoregressive side is progressive in horizon: first predict 17 future frames from a single input frame, then 34, then 121 (~50,000 tokens), then fine-tune on high-quality data.</p><p>What you’d actually do with this: take an open Cosmos checkpoint, post-train on your domain-specific video (warehouse footage, surgical procedures, your specific factory floor), and use the resulting world model either to generate synthetic training data for downstream policies (sim-to-real bootstrapping) or as the dynamics model in an MPC controller. The post-training scripts and Cosmos Cookbook are the actually-load-bearing part of the offering for builders, not the base models themselves.</p><p><strong>Pattern for builders:</strong> If you’re building a Physical AI product and you don’t have a good reason to train your own world model from scratch, take an open Cosmos checkpoint and post-train. The base model has seen vastly more diverse physics than you’ll ever collect, and your domain shift is a fine-tuning problem, not a foundation-model-training problem. Same playbook as LLM fine-tuning, very different price point: the 14B-parameter Cosmos Predict checkpoint is roughly the right starting point and the post-training compute is bounded by the size of your domain video corpus.</p><h3>V-JEPA 2-AC: The Result That Quietly Validates LeCun’s Whole Thesis</h3><p>This is the technical paper to read if you only read one. The setup: pre-train V-JEPA 2 on &gt;1M hours of internet video with self-supervised masked spatio-temporal prediction. Then post-train V-JEPA 2-AC, an action-conditioned predictor (300M parameters, transformer with block-causal attention), on 62 hours of unlabeled Droid robot video — no rewards, no task labels, no demonstrations of success vs. failure. Then deploy zero-shot on a Franka arm in a different lab than any of the training data came from, controlled via image goals, and ask it to do prehensile manipulation tasks (pick this up, put it there).</p><p>It works. Not at human-level reliability, but it works zero-shot in a lab the model has never seen, with no robot-specific data collected from that lab. The control loop is closed-loop MPC: at each timestep, encode the current observation and goal image to latents, sample candidate action sequences, roll each one out through the learned latent dynamics, pick the sequence that lands closest to the goal latent, execute the first action, repeat.</p><p>What makes this matter for the field: the “pre-train on internet video, post-train on a tiny amount of robot interaction” recipe is exactly what LeCun has been arguing for since 2022. The experimental result says the recipe works for at least one non-trivial robotics task, with sample-efficiency that’s roughly two orders of magnitude better than the alternative (Octo’s pre-training corpus is 1M+ trajectories from the Open-X Embodiment dataset; V-JEPA 2-AC trains on 23k from Droid).</p><p>The catch: it works for short-horizon tabletop manipulation. Long-horizon mobile manipulation, multi-step kitchen tasks, anything requiring strong memory beyond a couple of seconds — those are still open. And VL-JEPA, the December 2025 vision-language extension, only just made the architecture compatible with general vision-language tasks. The thesis is starting to ship results, but the full proof is years out.</p><p><strong>Pattern for researchers:</strong> If you’re working on robot learning, pay close attention to the “pre-train on web video, post-train on small robot corpus” axis. The ratio that V-JEPA 2-AC demonstrates — roughly 1 million hours of unlabeled video to 62 hours of unlabeled robot video — is the kind of data efficiency that, if it generalizes, makes embodied AI economically viable in domains where collecting robot data is prohibitively expensive (surgery, disaster response, low-volume manufacturing).</p><h3>The Compounding-Error Honest Reckoning</h3><p>Every world model in production today has the same Achilles’ heel and it’s worth naming it directly: prediction errors compound over autoregressive rollout. This isn’t a JEPA problem or a Genie problem, it’s a <em>prediction-in-time</em> problem, and every camp has it.</p><p>For Cosmos and Genie 3, compounding errors look like worlds that drift visually after a few minutes. For DreamerV3 and V-JEPA 2-AC, they look like model-predictive controllers whose plan quality degrades for long horizons (which is why Dreamer uses short imagination horizons of 15–50 steps, not 500). For Wayve’s GAIA-2 in autonomous driving, they look like scenario generators that produce plausible 10-second futures and increasingly weird 30-second ones.</p><p>The October 2025 “Learning World Models for Interactive Video Generation” paper made this explicit: “compounding error is inherently irreducible in autoregressive video generation” and “naive autoregressive generation with extended context windows and retrieval-augmented generation prove less effective for video generation, primarily due to weaker in-context learning compared to LLMs.” Their proposed fix — VRAG (Video Retrieval Augmented Generation with explicit global state conditioning) — works, but it works by adding an explicit state representation, which is exactly the architectural concession the pure-pixel-prediction camp has been resisting.</p><p>The honest read: the industry is converging on a hybrid architecture where world models handle short-horizon dynamics, an explicit state representation handles persistent world knowledge, and some kind of global memory or retrieval system handles long-horizon consistency. Pure autoregressive video is great for demos and limited for production.</p><p><strong>Pattern for builders:</strong> Treat any world model as a <em>short-horizon</em> component. Whatever your application is, design your control loop or training procedure assuming the world model is reliable for N seconds and unreliable beyond it, then engineer around that limit explicitly. Re-grounding the model with fresh real-world observations every N seconds is almost always cheaper and more reliable than trying to extend N by another factor of 2 with architectural cleverness.</p><h3>What This Changes About How You Build</h3><p>The model is the most interchangeable component in this entire system. What matters in the world-model wave, like in the LLM wave before it, is the surrounding infrastructure: the tokenizers (Cosmos’s causal compression to 2048×), the latent geometry regularizers (LeJEPA’s SIGReg, replacing six tunable losses with one), the planning loops (CEM and MPC over learned dynamics), the synthetic-data pipelines (Cosmos → Omniverse → robot policy), the safety harnesses (Cosmos guardrails, Genie 3’s Responsible Development team), the benchmarks (WorldModelBench, Physics-RW, PIQA).</p><p>Some practical consequences for what you build over the next 12–24 months:</p><p>For <strong>robotics teams</strong>, the data-collection economics shift. If V-JEPA 2-AC’s 62-hour result generalizes, the right move is to collect modest in-domain robot video and lean on pre-trained world models for the rest. Don’t try to out-collect the Open-X Embodiment dataset — you can’t. Do post-train Cosmos or wait for AMI Labs’ open releases (LeCun has signaled the lab will be research-first with public model checkpoints, though “signaled” is doing real work in that sentence).</p><p>For <strong>autonomous driving teams</strong>, the synthetic-data path has moved from speculative to validated. Wayve’s GAIA-2 plus NVIDIA Cosmos plus Omniverse makes scenario generation for rare-event coverage actually viable. The sim-to-real gap is still real, but it’s shrinking faster than most fleet-data acquisition strategies are improving.</p><p>For <strong>gaming and immersive content teams</strong>, Genie 3 and World Labs’ Marble are pointing at a future where 3D environment creation becomes a prompting problem rather than a Blender-and-Unreal problem. PitchBook’s projection that the gaming world-model market grows from $1.2B (2022–2025) to $276B by 2030 is wildly speculative, but the directionality is hard to argue with.</p><p>For <strong>everyone else</strong>, the practical question is whether your product has any task where sub-second physical prediction matters. If yes, world models are coming for that part of your stack within two product cycles. If no, you can keep iterating on LLM-shaped products without worrying about it.</p><h3>The Takeaway</h3><p>Every cycle in AI has a moment where the ceiling becomes visible, and the smart capital starts looking sideways instead of up. For the LLM cycle, that moment was somewhere in late 2025 when the gap between benchmark scores and end-user value stopped narrowing fast enough to justify another order of magnitude of training compute. The $2B-in-three-weeks pattern in early 2026 is what <em>sideways</em> looks like in practice.</p><p>The world-model bet isn’t that LLMs are wrong — they’re not, they’re shipping serious revenue and they’re getting better. The bet is that for any task where physical reality is the substrate (a robot in a kitchen, a car on a freeway, an avatar in a generated world), language is the wrong abstraction layer to learn from, and a different architectural family will dominate. JEPA-style latent prediction, autoregressive video with causal attention, RSSM-style model-based RL — these are three different attempts to build the right abstraction layer, and the field hasn’t picked a winner yet.</p><p>What’s actually sitting in front of working developers in 2026 is a stack with cracks: V-JEPA 2-AC zero-shots a Franka arm in a new lab but only for short-horizon tabletop tasks. Genie 3 generates beautiful interactive 3D worlds at 24 fps but only for a few minutes before consistency breaks down. Cosmos has been downloaded 2 million times but the companies actually shipping production robotics on top of it are still small in number. Compounding error is the ghost in every machine. The reliability paradox makes simulator-trained agents a research problem, not a deployment one. WorldModelBench scores are not reassuring.</p><p>And yet — the same way nobody talks about Claude Code’s harness when they debate model capabilities, nobody talks about world models’ tokenizers, latent regularizers, MPC loops, or guardrail systems when they debate AGI timelines. They talk about the demos. The 200 lines that call the model are the easy part. The other 499,800 are where this architecture lives, and the next two years are going to be a vicious round of harness engineering before any of these systems are ready for production.</p><p>If you build AI-powered tools and you’ve never had to ask whether your model can predict the next state of a physical environment, that question is about to land on your roadmap whether you want it or not. The good news: the patterns from the LLM era — small stable tool surfaces, explicit cache boundaries, asymmetric persistence, circuit breakers in every retry loop — port over almost cleanly. The architectures are different. The discipline is the same.</p><p><em>Disclaimer: This post discusses a fast-moving technical area where the companies and capabilities described are still in active research or limited preview. Numbers and architectural details reflect public papers, technical reports, and company communications as of late April 2026, and will be out of date soon. Treat published benchmarks with appropriate skepticism — most of them are very young, the metrics are still being argued about, and the gap between “works in the paper” and “works in your product” is the same gap it has always been.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=178fb0c6b09c" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[What 512,000 Lines of Leaked Claude Code TypeScript Teach Us Devs About Building Production AI…]]></title>
            <link>https://medium.com/@germanviscuso/what-512-000-lines-of-leaked-typescript-teach-us-devs-about-building-production-ai-agents-72eada6c66b3?source=rss-155e26ed6f1d------2</link>
            <guid isPermaLink="false">https://medium.com/p/72eada6c66b3</guid>
            <category><![CDATA[ai-agents-review]]></category>
            <category><![CDATA[developer-tools]]></category>
            <category><![CDATA[software-engineering]]></category>
            <category><![CDATA[typescript]]></category>
            <category><![CDATA[claude-code]]></category>
            <dc:creator><![CDATA[German Viscuso]]></dc:creator>
            <pubDate>Mon, 06 Apr 2026 23:19:54 GMT</pubDate>
            <atom:updated>2026-04-07T16:36:21.324Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1B3LezIvJrzmWyFeAdkWSw.png" /></figure><h3>What 512,000 Lines of Leaked Claude Code TypeScript Teach Us Devs About Building Production AI Agents</h3><p>On March 31, 2026, Anthropic accidentally shipped a .map file in their npm package for Claude Code v2.1.88. The source map pointed to an unauthenticated R2 bucket containing the full unobfuscated TypeScript source. Within hours, the ~1,900-file codebase was mirrored across GitHub and picked apart by thousands of developers.</p><p>The root cause? A missing *.map entry in .npmignore. One line in one config file. And this was actually the <em>second</em> time it happened — the first was on launch day in February 2025. Thirteen months apart, same bug, same vector.</p><p>But what leaked isn’t just code — it’s the first production-grade commercial AI agent architecture ever made fully visible to the outside world. Whether you’re building agents, CLI tools, or AI-powered products, the engineering patterns in this codebase are worth studying. Here’s what stood out.</p><h3>CLAUDE.md Is First-Class Architecture, Not a Convention</h3><p>Let’s start with the most immediately actionable finding. The leak confirmed that CLAUDE.md isn&#39;t a grassroots community trick that Anthropic tolerates — it&#39;s explicitly wired into the system prompt as a structured memory layer. The agent is instructed to locate and obey CLAUDE.md files at the project root and in subdirectories, giving you a <strong>persistent, hierarchical instruction system</strong> that carries across sessions without any special flags.</p><p>Root-level files set project-wide defaults; subdirectory files provide path-scoped overrides. Populate them with stack constraints, naming conventions, libraries to prefer (or avoid), build/test/deploy commands, and hard rules like “never touch the legacy auth module.”</p><p><strong>Pattern for devs:</strong> Think of CLAUDE.md as .editorconfig for your AI agent — a version-controlled control surface you commit to the repo. For teams, it guarantees consistent agent behavior across contributors. For framework builders: adopt this convention. Hierarchical, file-based instruction systems are simple to implement, easy to version, and instantly legible.</p><h3>The System Prompt Is Assembled Client-Side (Mostly)</h3><p>You’d expect the system prompt to be a black box on the server. In Claude Code, that assumption falls apart.</p><p>A large portion of the instruction layer is built in client-side TypeScript. Roughly 15 composable functions assemble the prompt, weaving in runtime context: the current date, Git history, user info, the contents of CLAUDE.md files, and memory fragments.</p><p>The interesting engineering is in the <strong>cache split</strong>. A sentinel marker called __SYSTEM_PROMPT_DYNAMIC_BOUNDARY__ divides the prompt into two regions:</p><ul><li><strong>Above the boundary (static):</strong> behavioral rules, coding guidelines, safety constraints. Hashed with Blake2b and cached globally with scope: &#39;global&#39; — roughly 3,000 tokens shared across every user.</li><li><strong>Below the boundary (dynamic):</strong> per-session context — your CLAUDE.md, MCP server instructions, environment variables.</li></ul><p>The result is a larger-than-average system prompt that amortizes its cost through aggressive prefix caching. Your project-specific config never invalidates the shared cache above the boundary.</p><p><strong>Pattern for devs:</strong> If you’re calling an LLM API at scale, design your system prompt with an explicit cache boundary. Pin your stable instructions above it; inject session-specific context below. The token savings compound dramatically as your user base grows.</p><h3>Parallel Tool Execution Is Wired In — But Gated by Your Phrasing</h3><p>The system prompt explicitly tells Claude Code to batch independent operations into parallel tool calls rather than running them one at a time. But in practice, it falls back to sequential execution whenever the dependency structure of your request is unclear.</p><p>This means that saying “read file A, then read file B, then compare them” produces two serial round-trips, while “read file A and file B at the same time, then compare their contents” can resolve in a single turn. Same outcome, materially different latency.</p><p><strong>Pattern for devs:</strong> When prompting any tool-using agent, make task independence explicit. Phrases like “these don’t depend on each other” or “do both simultaneously” act as parallelism hints. If you’re building an agent framework, consider exposing a parallel: true option in your tool call schema so the model doesn&#39;t have to infer it.</p><h3>File Editing Works by String Matching, Not Full Rewrites</h3><p>Rather than regenerating an entire file on each edit, Claude Code’s write tool locates an exact substring in the current file content and swaps it out. It’s a search-and-replace operation, not a rewrite.</p><p>This is safer than full-file regeneration (no risk of silently clobbering concurrent changes), but it’s brittle when the agent’s view of the file is stale. If whitespace drifted, indentation shifted, or another edit landed between the read and the write, the match fails. That’s why Claude Code sometimes asks you to clarify exactly which block to change, or silently fails an edit that seemed trivial.</p><p><strong>Pattern for devs:</strong> If you’re building file-editing capabilities for an agent, the substring-replace approach is a solid default, but pair it with a mandatory fresh read immediately before the edit. For your own day-to-day Claude Code usage: when edits keep failing, tell it to “re-read the file first, then apply the change.” It forces a cache refresh.</p><h3>A React App Is Driving the Terminal</h3><p>Claude Code’s entrypoint is src/entrypoints/cli.tsx — and that .tsx extension is not a typo. The entire terminal interface is a React component tree rendered via Ink, which maps React&#39;s reconciliation model onto terminal escape sequences.</p><p>Prompt dialogs, message bubbles, progress spinners, tool-call output, markdown rendering — all React components. Contrast this with OpenAI’s Codex, which built its TUI in Rust with ratatui and crossterm: lower memory footprint, faster rendering, but a completely separate skill set from web development.</p><p>Anthropic’s choice makes more sense as an organizational bet than a technical one. If your engineering team already reasons in JSX and component lifecycles, extending that to the terminal via Ink means faster iteration and a shared codebase between web and CLI surfaces.</p><p><strong>Pattern for devs:</strong> Don’t default to building a bespoke terminal UI if your team’s muscle memory is React. Ink gives you the same component model, same state management, same developer experience — just rendering to stdout instead of the DOM.</p><h3>9,700 Lines of Bash Security (and a Parser That Disagrees with Itself)</h3><p>The shell execution layer is one of the most heavily engineered parts of the codebase. Claude Code doesn’t give the model raw shell access. Every command passes through a substantial deterministic pipeline before anything executes:</p><ul><li><strong>Command classification:</strong> each invocation is bucketed by intent — searching, reading, listing, or operations where no output on success is normal — and its output gets post-processed accordingly.</li><li><strong>Semantic exit code mapping:</strong> many UNIX tools return non-zero for perfectly normal outcomes. grep exits 1 when it matches nothing; diff exits 1 when files differ. The scaffold remaps these per-binary so the model interprets results correctly instead of treating every non-zero code as a crash.</li><li><strong>Multi-layered guards for read-only operations:</strong> flag allowlists, filesystem path validation, tool-specific restrictions (e.g., limiting what sed can do), sandbox eligibility checks, and full AST-level safety analysis via a tree-sitter WASM parser. For compound commands with pipes or semicolons, there are explicit recursion depth limits on the safety analysis.</li></ul><p>But there’s a documented crack in this armor. The codebase contains two parsers: a newer tree-sitter-based one and an older one (splitCommand_DEPRECATED). The old parser uses JavaScript&#39;s \s regex class to tokenize commands, and \s treats \r (carriage return) as whitespace. Bash&#39;s IFS does not. An attacker who embeds a \r in a command can construct input that the validator tokenizes one way and bash interprets another — causing the security check to approve a command that actually runs something entirely different.</p><p>The kicker: despite the DEPRECATED suffix, the old parser is still invoked from at least nine modules that make real security decisions. Anthropic runs both parsers in shadow mode and logs divergences, but the legacy one is still load-bearing.</p><p><strong>Pattern for devs:</strong> If your agent executes shell commands, build a classification and normalization layer between the model and the shell. Remap exit codes to match what the model expects. And if you deprecate a security-critical code path, actually remove it from the decision chain — a _DEPRECATED suffix is not a security control.</p><h3>Four Layers of Context Compaction (and One Nasty Bug)</h3><p>Long sessions inevitably blow past the context window. Claude Code handles this with four cascading strategies, each catching what the previous one missed:</p><ol><li><strong>Proactive compaction:</strong> each turn, the system estimates the token count. When it approaches the ceiling, older messages are summarized behind a “compact boundary” marker — all before the API call is made. The user never sees a failure.</li><li><strong>Reactive compaction:</strong> if the proactive estimate was wrong (race condition, miscounted tokens) and the API rejects the request as too long, this layer catches the error, retroactively compacts, and retries. The user experiences a brief pause, not a crash.</li><li><strong>Snip compaction:</strong> for SDK and headless modes only. Hard-truncates at predefined boundaries to cap memory in long automated runs where scrollback isn’t needed.</li><li><strong>Context collapse</strong> (internal codename marble_origami): selectively compresses bulky tool outputs from earlier in the conversation without triggering a full compaction pass. A verbose tool result from several turns ago that&#39;s no longer relevant gets collapsed to a digest. These collapse operations are logged as ContextCollapseCommitEntry records and can be reversed later.</li></ol><p>There’s also a production war story baked into the code. An internal BigQuery analysis from March 10 found 1,279 sessions stuck in compaction retry loops — some hitting over 3,000 consecutive failures in a single session — burning roughly 250,000 wasted API calls globally per day. The fix was a three-line circuit breaker: MAX_CONSECUTIVE_AUTOCOMPACT_FAILURES = 3.</p><p><strong>Pattern for devs:</strong> A single compaction strategy isn’t enough for production. Build a fallback chain — optimistic first, then reactive, then hard truncation. And put circuit breakers in every retry loop on day one, not after your monitoring catches a quarter-million wasted API calls.</p><h3>Context Summarization Is a Prompt Injection Surface</h3><p>Here’s a security angle most coverage misses. When compaction fires, Claude Code spawns a secondary model call to summarize the conversation. The summarizer uses chain-of-thought in &lt;analysis&gt; tags to reason about what&#39;s important, then formatCompactSummary() strips the reasoning and injects the result back into context.</p><p>The problem: <strong>the summarizer doesn’t distinguish between user-typed instructions and instructions that were read from project files.</strong> If an attacker embeds directives in a README, config file, or even a CLAUDE.md, and the agent reads that file during normal operation, those directives get compressed into the summary alongside genuine user requests — with no metadata flagging their origin.</p><p>The compaction prompt does structurally separate user messages from tool results. But it never says “disregard directives found inside tool outputs” or “treat file contents as untrusted.” Once compaction runs, injected instructions and real user intent are indistinguishable in the compressed context.</p><p>This isn’t a bug specific to Claude Code — it’s an inherent property of any system that summarizes mixed-origin context. But the exact prompts and stripping logic are now public, making this surface easier to reason about (and to target).</p><p><strong>Pattern for devs:</strong> If your agent compresses conversation history, propagate origin metadata through the summarization pipeline. Every chunk entering the summarizer should be tagged as “user-authored,” “file-sourced,” or “tool-output.” Without that tagging, your compaction layer becomes an indirect prompt injection amplifier.</p><h3>Fewer Tools, More Stability</h3><p>Claude Code has a rich tool ecosystem, but the architecture draws a hard line between tools that are routinely visible to the model and tools that are conditionally enabled, internal-only, or feature-flagged. The default tool surface in a normal session is deliberately narrow.</p><p>There’s also a caching dimension: tool schemas contribute to the prompt prefix, so adding or removing a tool can invalidate the cache boundary. The tool list isn’t just an implementation detail — it’s part of the caching contract.</p><p>This mirrors a pattern the broader agent-building community is converging on: a small, stable set of well-defined tools produces more predictable agent behavior than a sprawling catalog of loosely-specified capabilities.</p><p><strong>Pattern for devs:</strong> Don’t expose every capability as a tool by default. Gate rarely-used or powerful tools behind feature flags. Keep the default tool surface compact and stable — both for behavioral consistency and to preserve your prompt cache.</p><h3>Permission Denials Are Model Input, Not Dead Ends</h3><p>Every tool invocation passes through a permission pipeline — mode check, hook evaluation, rule matching, optional user prompt. That part is standard. What’s unusual is what happens when the user says “no.”</p><p>The denial doesn’t just block execution. It gets wrapped as a tool result and fed back to the model as though the tool had returned an error. The agent sees structured rejection data in its tool output stream and recalibrates its approach. Denial metadata is also accumulated across the session and surfaced to SDK consumers, so IDE integrations can identify patterns like “this user consistently blocks shell commands targeting system directories.”</p><p>The effect: the model doesn’t just hit a wall and blindly retry. It adapts within the session based on which actions were rejected and why.</p><p><strong>Pattern for devs:</strong> Whenever your agent’s action gets blocked — by permissions, validation rules, or user intervention — return the rejection as structured tool output rather than silently swallowing it. This gives the model the feedback signal it needs to self-correct without you having to write special error-handling logic.</p><h3>An Adversarial Verifier with a Hardcoded List of Laziness Excuses</h3><p>The VERIFICATION_AGENT feature flag enables a secondary agent that audits changes before the primary agent declares a task complete. Its prompt includes an explicit catalog of rationalizations it&#39;s expected to resist:</p><ul><li><em>“The code looks correct based on my reading”</em> — reading isn’t verification, execute the code.</li><li><em>“The implementer’s tests already pass”</em> — the implementer is itself an LLM, verify independently.</li><li><em>“This is probably fine”</em> — “probably” is not “verified,” execute it.</li><li><em>“Let me start the server and check the code”</em> — don’t just start the server, actually hit the endpoint.</li></ul><p>Boris Cherny, who leads the Claude Code team, has said publicly that 100% of his own contributions to Claude Code were written by Claude Code. The agent writes its own code and then audits itself with a reviewer that carries a hardcoded list of its own likely shortcuts.</p><p>The irony: that verification agent, with all its anti-laziness guardrails, didn’t catch a missing one-line .npmignore entry. Twice.</p><p><strong>Pattern for devs:</strong> When using AI for quality gates, don’t settle for “verify this is correct.” Enumerate the specific cognitive shortcuts the verifier is likely to take and explicitly prohibit each one. LLMs have remarkably predictable laziness modes — naming them in the prompt is one of the most cost-effective ways to suppress them.</p><h3>Anthropic’s Engineers Get a Different Prompt Than You Do</h3><p>A build-time constant process.env.USER_TYPE === &#39;ant&#39; gates prompt sections throughout the codebase. Internal Anthropic users get instructions that external users don&#39;t:</p><ul><li>An anti-hallucination rule that forbids claiming tests pass when the output visibly shows failures.</li><li>A quantified conciseness target: inter-tool-call text capped at ≤25 words, final responses at ≤100. An inline comment notes this specific wording yielded a 1.2% reduction in output tokens compared to a vaguer instruction to be brief — they ran a controlled experiment to determine the exact threshold.</li><li>A misconception-correction directive: if the user’s request rests on a factual misunderstanding, flag it. Not shipped externally.</li><li>The adversarial verification agent described above.</li></ul><p>Throughout the codebase, @[MODEL LAUNCH] annotations track prompt experiment outcomes with identifiers like capy v8 thoroughness counterweight (PR #24302). Anthropic treats prompt copy the way performance marketers treat ad creative — every word gets tested.</p><p><strong>Pattern for devs:</strong> Differentiate your prompts by user segment. Internal teams, power users, and general-audience customers have different failure modes and tolerance for verbosity. Quantified instructions (“≤25 words”) outperform qualitative ones (“be concise”) — and it’s worth A/B testing the exact threshold.</p><h3>Compile-Time Feature Flags with Full Dead Code Elimination</h3><p>Claude Code uses Bun’s feature() function as a build-time constant:</p><pre>if (feature(&#39;VOICE_MODE&#39;)) {<br>  // entire branch eliminated by the bundler when the flag is off<br>}</pre><p>This isn’t runtime gating — the bundler constant-folds the check and strips the entire code path, including transitive imports. This is how Anthropic ships distinct builds (public CLI, internal build, SDK, IDE bridge) from a single codebase with zero runtime overhead for disabled features.</p><p>Some of the flags reveal capabilities that aren’t public yet: VOICE_MODE, COORDINATOR_MODE, KAIROS, ULTRAPLAN, ABLATION_BASELINE. There&#39;s also isUndercover(), which scrubs internal model codenames (like &quot;Capybara&quot; and &quot;Tengu&quot;) from prompts. The DCE comments around this check are rigorous — each callsite must inline the condition directly (not extract it to a shared constant) so the bundler can constant-fold and eliminate the branch in external builds.</p><p><strong>Pattern for devs:</strong> If you ship one codebase to multiple audiences, build-time feature flags with DCE are categorically better than runtime toggles. Disabled features don’t just not execute — they don’t exist in the artifact. No dead code, no accidental activation, no bundle bloat.</p><h3>KAIROS: An Autonomous Agent That Dreams</h3><p>Among the unreleased features, KAIROS is the most architecturally complete — 190 references across 61 files. It’s a headless daemon mode: Claude Code running autonomously via GitHub webhooks and 5-minute cron cycles, equipped with tools that don’t exist in the public build (SendUserFileTool, PushNotificationTool, SubscribePRTool).</p><p>The most novel subsystem is the memory consolidation daemon at src/services/autoDream/autoDream.ts. It fires a /dream prompt as a background sub-agent that reviews recent session transcripts and distills them into durable MEMORY.md files. The trigger logic is gated in order of cost:</p><ol><li><strong>Time gate:</strong> at least 24 hours since the last consolidation (a single stat call on the lock file).</li><li><strong>Session gate:</strong> at least 5 new transcripts accumulated (a directory listing).</li><li><strong>Lock acquisition:</strong> file-based advisory lock.</li></ol><p>The lock file design itself is clever. Its mtime serves double duty as the lastConsolidatedAt timestamp. The PID is written into the file body. On failure, the mtime is rolled back to its previous value, effectively undoing the gate. A staleness timeout of one hour guards against PID reuse on long-lived machines.</p><p><strong>ULTRAPLAN</strong> is the other standout: it delegates complex planning to a remote Opus session that can run for up to 30 minutes, polling every 3 seconds and using a “teleport sentinel” to detect completion.</p><p><strong>Pattern for devs:</strong> For autonomous agents that need periodic background work, gate your triggers cheapest-first: a stat call before a directory scan before a lock acquisition. If you need a lightweight cross-process lock without a database, file mtime as semantic timestamp + PID body + staleness cutoff is a battle-tested primitive.</p><h3>Latency Hiding: Speculation and Memory Prefetch</h3><p>Two performance tricks that are invisible from the outside but significantly affect perceived responsiveness:</p><p><strong>Speculative pre-computation:</strong> while the user is still typing, Claude Code begins computing likely responses for predictable interactions (follow-up questions, confirmation prompts). When the prediction hits, the response appears nearly instantly.</p><p><strong>Concurrent memory loading:</strong> while the model streams its reply, the system begins prefetching relevant CLAUDE.md content using TC39 explicit resource management to ensure proper cleanup across all generator exit paths:</p><pre>using pendingMemoryPrefetch = startRelevantMemoryPrefetch(<br>  state.messages,<br>  state.toolUseContext,<br>)</pre><p>By the time the model finishes streaming and tools begin executing, the memory context is already loaded. The I/O latency is hidden behind the streaming latency.</p><p><strong>Pattern for devs:</strong> Overlap your I/O with your model calls. If you know what data tools will need once the model finishes reasoning, start fetching it while the stream is still running. The using keyword from TC39 explicit resource management makes cleanup on all exit paths trivial.</p><h3>Asymmetric Session Persistence (Born from a Lost Session)</h3><p>Transcript writes behave differently depending on who’s talking. User messages are persisted synchronously (awaited). Assistant messages are written asynchronously (fire-and-forget).</p><p>The reason is documented in the source: if the process dies between the moment the user hits enter and the API returning a response, an async-only write path leaves the transcript in a state where --resume can&#39;t find a valid conversation. Someone at Anthropic lost a real session to this exact race condition. The fix: always durably persist the user&#39;s message before making the API call. The assistant&#39;s response is already durable on the server side and can be replayed.</p><p><strong>Pattern for devs:</strong> In any agent with session resumption, treat user input as your source of truth and write it synchronously before the API call. The model’s response lives on the server — your responsibility is ensuring the user’s side of the conversation survives crashes.</p><h3>Frustration Tracking via Regex, Not the LLM</h3><p>One of the most charming implementation details: Claude Code monitors user sentiment with a plain regular expression, not the model. A 27-line file called userPromptKeywords.ts pattern-matches against profanity and frustration phrases — &quot;wtf,&quot; &quot;ffs,&quot; &quot;this sucks,&quot; &quot;piece of crap,&quot; and about a dozen more. On a match, a telemetry event fires: tengu_input_prompt with is_negative: true.</p><p>A product generating $2.5B in annual recurring revenue, tracking user mood with string matching. It works because for a binary “frustrated or not?” signal, a keyword list captures the vast majority of real cases at a negligible fraction of the cost of a model call.</p><p><strong>Pattern for devs:</strong> Not every signal in your system needs to pass through the model. Binary classification on predictable keywords is a regex problem. Nuanced analysis is an LLM problem. Don’t spend inference dollars on a task that 50 bytes of regex handles perfectly well.</p><h3>Comments as Institutional Memory (and Some Great Function Names)</h3><p>The codebase has unusually high-quality inline commentary — not describing <em>what</em> the code does, but <em>why</em> specific decisions were made. One example from query.ts wraps a subtle API constraint about thinking blocks in deliberately absurd medieval English to ensure nobody skims past it:</p><blockquote>“Heed these rules well, young wizard. For if ye does not heed these rules, ye will be punished with an entire day of debugging and hair pulling.”</blockquote><p>Other comments document why heavy side-effects must execute before module imports, why specific validators exist, or why a particular flag should never be treated as safe.</p><p>And then there are the function names pulling double duty as guardrails:</p><pre>AnalyticsMetadata_I_VERIFIED_THIS_IS_NOT_CODE_OR_FILEPATHS<br>DANGEROUS_uncachedSystemPromptSection()<br>writeFileSyncAndFlush_DEPRECATED()<br>resetTotalDurationStateAndCost_FOR_TESTS_ONLY()</pre><p>Nobody calls DANGEROUS_uncachedSystemPromptSection() without pausing to consider the implications. The name <em>is</em> the code review.</p><p><strong>Pattern for devs:</strong> Embed rationale in comments, not just descriptions. Use function names as active guardrails — prefix dangerous operations with DANGEROUS_, test-only utilities with _FOR_TESTS_ONLY. In a world where AI agents routinely read and modify source code, self-documenting naming and explanatory comments are forms of institutional memory that benefit both human and machine readers.</p><h3>The Takeaway</h3><p>The model is the most interchangeable component in this entire system. What makes Claude Code a product — what constitutes its actual competitive surface — is the harness: four-layer compaction with circuit breakers, prompt cache boundaries, a 9,700-line bash security scaffold with dual parsers, permission pipelines that feed denials back as model input, asymmetric session persistence, build-time dead code elimination, speculative pre-computation, memory prefetch during streaming, an adversarial verification agent with a hardcoded laziness catalog, substring-based file editing, sentiment detection via regex, and a hierarchical memory system rooted in plain markdown files.</p><p>Nobody talks about the harness. Everyone debates which model is smarter. But the years of production experience visible in this codebase live in the engineering <em>around</em> the model, not in the model itself. The compaction layer has an untagged prompt injection surface. The bash security system has a parser differential on \r. The verification agent designed to prevent mistakes didn&#39;t catch a one-line config omission — twice. Brilliant architecture, real scars. That&#39;s what production code looks like.</p><p>If you’re building AI-powered tools, that’s the lesson worth internalizing: <strong>the 200 lines that call the API are the easy part. The other 499,800 are where your product lives.</strong></p><p><em>Disclaimer: This post discusses design patterns observable in a codebase that was inadvertently exposed. It is not based on an official open-source release by Anthropic, and their terms restrict reverse engineering, service replication, and construction of competing products.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=72eada6c66b3" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Nuevo Curso de Desarrollo de Skills Alexa: Construye una Skill Alexa atractiva con “Feliz…]]></title>
            <link>https://medium.com/@germanviscuso/nuevo-curso-de-desarrollo-de-skills-alexa-construye-una-skill-alexa-atractiva-con-feliz-1ff396007378?source=rss-155e26ed6f1d------2</link>
            <guid isPermaLink="false">https://medium.com/p/1ff396007378</guid>
            <category><![CDATA[tutorial]]></category>
            <category><![CDATA[amazon]]></category>
            <category><![CDATA[alexa]]></category>
            <category><![CDATA[chatbots]]></category>
            <category><![CDATA[español]]></category>
            <dc:creator><![CDATA[German Viscuso]]></dc:creator>
            <pubDate>Tue, 03 Sep 2019 15:03:01 GMT</pubDate>
            <atom:updated>2019-12-11T11:45:31.532Z</atom:updated>
            <content:encoded><![CDATA[<figure><a href="https://chatbotchocolate.com/?utm_source=pc-post&amp;utm_medium=B-top&amp;utm_campaign=general&amp;utm_term=id0050"><img alt="" src="https://cdn-images-1.medium.com/max/680/1*XF6az8pG3NiMuWwXHFTHdg.png" /></a></figure><h3>Nuevo Curso de Desarrollo de Skills Alexa: Construye una Skill Alexa atractiva con “Feliz Cumpleaños”</h3><p>Estamos encantados de presentarte nuestro nuevo curso autodidacta de desarrollo de skills: “<a href="http://alexa.design/es-course">Feliz Cumpleaños: Construye una Skill de Alexa Atractiva</a>”. Este curso gratuito ofrece una guía paso a paso de construcción de una skill Alexa de calidad desde el principio hasta el final. Los nuevos desarrolladores de skills aprenderán a crear su primera skill en pocos minutos. Los más experimentados aprenderán como agregar características más avanzadas como memoria/persistencia para proveer una experiencia de voz más conversacional y personalizada. Cuando completes el curso tendrás un conocimiento fundacional del desarrollo de skills y de las herramientas relacionadas para ayudarte a construir skills Alexa de calidad que tus clientes apreciarán.</p><figure><a href="https://chatbotchocolate.com/?utm_source=Pc-post&amp;utm_medium=B-medio&amp;utm_campaign=general&amp;utm_term=id0051"><img alt="" src="https://cdn-images-1.medium.com/max/680/1*XF6az8pG3NiMuWwXHFTHdg.png" /></a></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*TeU2KOeap57NXJR9UjPPmA.png" /><figcaption>Pantalla APL de la skill “Feliz Cumpleaños”</figcaption></figure><h3>Aprende a Implementar una Experiencia de Voz</h3><p>Aunque es fácil crear rápidamente una skill Alexa, hay muchos aspectos a considerar para construir una experiencia de voz atractiva. Tener una gran idea es importante pero también lo es la implementación. Una gran idea para una skill pero mal implementada hará muy difícil obtener y retener a los usuarios. Antes de que comiences a convertir tu idea en una skill Alexa te recomendamos que te tomes tu tiempo para diseñar una experiencia de voz, aprender a construir una interfaz de usuario de voz y comprender el conjunto de herramientas disponibles para desarrollar skills. Hemos diseñado el curso de Feliz Cumpleaños para enseñarte estos conceptos y así aprendas a diseñar e implementar una skill atractiva.</p><p>Feliz Cumpleaños es una skill simple que te permite contar los días que faltan para tu cumpleaños. También te dará un mensaje de Feliz Cumpleaños en ese día especial. A lo largo del curso aprenderás a construir tu propia versión de la skill comenzando desde lo más básico para luego ir agregando características más avanzadas como manejo de dialógos, persistencia y recordatorios.</p><h3>Módulos del Curso</h3><p>El curso ofrece 8 módulos de programación de skills con <a href="http://alexa.design/es-videos-code">todo el código fuente disponible</a>:</p><ul><li><a href="https://alexa.design/es-videos1">Creando Skills Alexa de Cero (20:24)</a> donde aprenderás la teoría de desarrollo de skills y crearás tu primera skill de tipo Hello World en unos pocos minutos</li><li><a href="https://alexa.design/es-videos2">i18n de skills (26:20)</a> donde aprenderás a soportar múltiples idiomas en tu skill</li><li><a href="https://alexa.design/es-videos3">Diálogos (29:00)</a> donde verás como hacer para que la skill pregunté de forma automática cuando falte información</li><li><a href="https://alexa.design/es-videos4">Memoria y Persistencia (26:54)</a> donde te enseñaremos como hacer para que tu skill recuerde la información aun al terminarse la sesión</li><li><a href="https://alexa.design/es-videos5">Acceso a APIs de ASK (23:58)</a> donde te mostraremos como recuperar la zona horaria y el nombre del usuario</li><li><a href="https://alexa.design/es-videos6">API de Recordatorios (46:34)</a> donde verás como enviar un recordatorio desde tu skill en la fecha de cumpleaños</li><li><a href="https://alexa.design/es-videos7">Acceso a APIs Externas (31:52)</a> donde aprenderás como conectarte a una API externa para recuperar información y utilizarla en tu skill</li><li><a href="https://alexa.design/es-videos8">Introduccion a APL (57:14)</a> donde verás como hacer la skill visualmente atractiva en dispositivos con pantalla</li></ul><p>Si este es tu primer contacto con la creación de skills te recomendamos comenzar por el <a href="https://alexa.design/es-videos1">primer módulo</a>. Si ya conoces lo básico y quieres soportar múltiples idiomas en tu skill puedes ir directamente al <a href="https://alexa.design/es-videos2">módulo 2</a>. En general puedes hacer los módulos que más te interesen, pero ten en cuenta que los videos parten del estado de la skill como se ha dejado en el módulo previo.</p><h3>Lo que Aprenderás</h3><p>Al completar este curso comprenderás los componentes de diseño de voz, desarrollo de skills y manejo de herramientas para que puedas crear skills Alexa atractivas. Aprenderás a utilizar la <a href="https://developer.amazon.com/alexa/console/ask">Consola de Desarrollo de Alexa</a> para crear y testear tu skill. También aprenderás como utilizar <a href="https://developer.amazon.com/docs/hosted-skills/build-a-skill-end-to-end-using-an-alexa-hosted-skill.html">Alexa-Hosted skills</a> para unificar el trabajo con front-end y el back-end en <a href="http://developer.amazon.com/alexa">developer.amazon.com/alexa</a>. El curso presenta los conceptos fundamentales de desarrollo de skills utilizando el <a href="https://ask-sdk-for-nodejs.readthedocs.io/en/latest/">Alexa Skills Kit Software Development Kit for Node.js</a>. También aprenderás a utilizar características importantes del Alexa Skill Kit (ASK) como:</p><ul><li><a href="https://developer.amazon.com/docs/custom-skills/create-intents-utterances-and-slots.html">Intents, utterances, y slots</a> para construir una interfaz de usuario de voz</li><li><a href="https://developer.amazon.com/docs/custom-skills/delegate-dialog-to-alexa.html#automatically-delegate-simple-dialogs-to-alexa">Auto delegation</a> para que la skill pregunté de forma automática cuando falte información</li><li><a href="https://ask-sdk-for-nodejs.readthedocs.io/en/latest/">ASK Software Development Kit for Node.js</a> para procesar las peticiones que se envien al back-end de tu skill</li><li><a href="https://ask-sdk-for-nodejs.readthedocs.io/en/latest/Managing-Attributes.html">Attributos Persistentes</a> con <a href="https://aws.amazon.com/s3/">Amazon S3</a> para que tu skill recuerde la información aun al terminarse la sesión</li><li><a href="https://developer.amazon.com/docs/smapi/alexa-settings-api-reference.html">Alexa Settings API</a> para recuperar la zona horaria y el nombre del usuario</li><li><a href="https://developer.amazon.com/docs/smapi/alexa-reminders-overview.html">Reminders API</a> para enviar un recordatorio en la fecha de cumpleaños</li><li><a href="https://developer.amazon.com/docs/alexa-presentation-language/apl-overview.html">Alexa Presentation Language (APL)</a> para hacer la skill visualmente atractiva en dispositivos con pantalla</li></ul><h3>Más Material de Entrenamiento para Mejorar tus Skills Alexa</h3><p>Una vez que hayas completado este curso te recomendamos que continúes tu aprendizaje con los siguientes recursos:</p><ul><li>D<a href="http://alexa.design/cdw">esigning for Conversation Course</a>: Aprende a diseñar experiencias conversacionales más dinámicas (en inglés)</li><li><a href="https://developer.amazon.com/docs/alexa-design/get-started.html">Alexa Design Guide</a>: Aprende los principios de diseño de voz situacional para que crees skills voice-first que sean naturales y enfocadas en el usuario (en inglés)</li><li><a href="https://www.twitch.tv/videos/409503308?filter=archives&amp;sort=time">How to Shift from Screen-First to Voice-First Design</a>: Aprende los cuatro patrones de diseño que hacen que una experiencia voice-first sea atractiva (en inglés)</li></ul><h3>Comienza Ya a Construir la Skill “Feliz Cumpleaños”</h3><p>Este curso es gratuito y está diseñado para que lo hagas a tu propio ritmo por lo que cualquier persona puede utilizarlo para construir su primera skill de Alexa. <a href="https://alexa.design/es-course%20">Haz click aquí</a> para comenzar. ¡Y danos tu opinión! Puedes contactarme vía Twitter <a href="https://twitter.com/germanviscuso">@GermanViscuso</a> y enviarme tus comentarios.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fupscri.be%2Ff%2F68bc97%3Fas_embed%3Dtrue&amp;dntp=1&amp;display_name=Upscribe&amp;url=https%3A%2F%2Fupscri.be%2F68bc97%2F&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=upscri" width="800" height="400" frameborder="0" scrolling="no"><a href="https://medium.com/media/d8021540cbf50434828deb5439e1cf04/href">https://medium.com/media/d8021540cbf50434828deb5439e1cf04/href</a></iframe><figure><a href="https://chatbotchocolate.com/?utm_source=Pc-post&amp;utm_medium=B-pie&amp;utm_campaign=general&amp;utm_term=id0052"><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Axmr4bMScpjBU_WcOm263A.png" /></a></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/788/1*u6pz8xzjblHCXLMQUQGF3Q.gif" /></figure><figure><a href="https://twitter.com/PlanetaChatbot"><img alt="" src="https://cdn-images-1.medium.com/max/788/1*qQxzn3vV9Xu4lYMpuc5OOw.png" /></a></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1ff396007378" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Nuevo Curso de Desarrollo de Skills Alexa: Construye una Skill Alexa atractiva con “Feliz…]]></title>
            <link>https://medium.com/@germanviscuso/nuevo-curso-de-desarrollo-de-skills-alexa-construye-una-skill-alexa-atractiva-con-feliz-2fbd7994820a?source=rss-155e26ed6f1d------2</link>
            <guid isPermaLink="false">https://medium.com/p/2fbd7994820a</guid>
            <category><![CDATA[alexa]]></category>
            <category><![CDATA[course]]></category>
            <category><![CDATA[alexa-skills]]></category>
            <dc:creator><![CDATA[German Viscuso]]></dc:creator>
            <pubDate>Mon, 29 Jul 2019 12:22:54 GMT</pubDate>
            <atom:updated>2019-09-03T08:02:14.155Z</atom:updated>
            <content:encoded><![CDATA[<h3>Nuevo Curso de Desarrollo de Skills Alexa: Construye una Skill Alexa atractiva con “Feliz Cumpleaños”</h3><p>Estamos encantados de presentarte nuestro nuevo curso autodidacta de desarrollo de skills: <a href="https://alexa.design/es-course%20">“Feliz Cumpleaños: Construye una Skill de Alexa Atractiva”</a>. Este curso gratuito ofrece una guía paso a paso de construcción de una skill Alexa de calidad desde el principio hasta el final. Los nuevos desarrolladores de skills aprenderán a crear su primera skill en pocos minutos. Los más experimentados aprenderán como agregar características más avanzadas como memoria/persistencia para proveer una experiencia de voz más conversacional y personalizada. Cuando completes el curso tendrás un conocimiento fundacional del desarrollo de skills y de las herramientas relacionadas para ayudarte a construir skills Alexa de calidad que tus clientes apreciarán.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*TeU2KOeap57NXJR9UjPPmA.png" /><figcaption>Pantalla APL de la skill &quot;Feliz Cumpleaños&quot;</figcaption></figure><h3>Aprende a Implementar una Experiencia de Voz</h3><p>Aunque es fácil crear rápidamente una skill Alexa, hay muchos aspectos a considerar para construir una experiencia de voz atractiva. Tener una gran idea es importante pero también lo es la implementación. Una gran idea para una skill pero mal implementada hará muy difícil obtener y retener a los usuarios. Antes de que comiences a convertir tu idea en una skill Alexa te recomendamos que te tomes tu tiempo para diseñar una experiencia de voz, aprender a construir una interfaz de usuario de voz y comprender el conjunto de herramientas disponibles para desarrollar skills. Hemos diseñado el curso de Feliz Cumpleaños para enseñarte estos conceptos y así aprendas a diseñar e implementar una skill atractiva.</p><p>Feliz Cumpleaños es una skill simple que te permite contar los días que faltan para tu cumpleaños. También te dará un mensaje de Feliz Cumpleaños en ese día especial. A lo largo del curso aprenderás a construir tu propia versión de la skill comenzando desde lo más básico para luego ir agregando características más avanzadas como manejo de dialógos, persistencia y recordatorios.</p><h3>Módulos del Curso</h3><p>El curso ofrece 8 módulos de programación de skills:</p><p>· <a href="https://alexa.design/es-videos1">Creando Skills Alexa de Cero (20:24)</a> donde aprenderás la teoría de desarrollo de skills y crearás tu primera skill de tipo Hello World en unos pocos minutos</p><p>· <a href="https://alexa.design/es-videos2">i18n de skills (26:20)</a> donde aprenderás a soportar múltiples idiomas en tu skill</p><p>· <a href="https://alexa.design/es-videos3">Diálogos (29:00)</a> donde verás como hacer para que la skill pregunté de forma automática cuando falte información</p><p>· <a href="https://alexa.design/es-videos4">Memoria y Persistencia (26:54)</a> donde te enseñaremos como hacer para que tu skill recuerde la información aun al terminarse la sesión</p><p>· <a href="https://alexa.design/es-videos5">Acceso a APIs de ASK (23:58)</a> donde te mostraremos como recuperar la zona horaria y el nombre del usuario</p><p>· <a href="https://alexa.design/es-videos6">API de Recordatorios (46:34)</a> donde verás como enviar un recordatorio desde tu skill en la fecha de cumpleaños</p><p>· <a href="https://alexa.design/es-videos7">Acceso a APIs Externas (31:52)</a> donde aprenderás como conectarte a una API externa para recuperar información y utilizarla en tu skill</p><p>· <a href="https://alexa.design/es-videos8">Introduccion a APL (57:14)</a> donde verás como hacer la skill visualmente atractiva en dispositivos con pantalla</p><p>Si este es tu primer contacto con la creación de skills te recomendamos comenzar por el <a href="https://alexa.design/es-videos1">primer módulo</a>. Si ya conoces lo básico y quieres soportar múltiples idiomas en tu skill puedes ir directamente al <a href="https://alexa.design/es-videos2">módulo 2</a>. En general puedes hacer los módulos que más te interesen, pero ten en cuenta que los videos parten del estado de la skill como se ha dejado en el módulo previo.</p><h3>Lo que Aprenderás</h3><p>Al completar este curso comprenderás los componentes de diseño de voz, desarrollo de skills y manejo de herramientas para que puedas crear skills Alexa atractivas. Aprenderás a utilizar la <a href="https://developer.amazon.com/alexa/console/ask">Consola de Desarrollo de Alexa</a> para crear y testear tu skill. También aprenderás como utilizar <a href="https://developer.amazon.com/docs/hosted-skills/build-a-skill-end-to-end-using-an-alexa-hosted-skill.html">Alexa-Hosted skills</a> para unificar el trabajo con front-end y el back-end en <a href="http://developer.amazon.com/alexa">developer.amazon.com/alexa</a>. El curso presenta los conceptos fundamentales de desarrollo de skills utilizando el <a href="https://ask-sdk-for-nodejs.readthedocs.io/en/latest/">Alexa Skills Kit Software Development Kit for Node.js</a>. También aprenderás a utilizar características importantes del Alexa Skill Kit (ASK) como:</p><p>· <a href="https://developer.amazon.com/docs/custom-skills/create-intents-utterances-and-slots.html">Intents, utterances, y slots</a> para construir una interfaz de usuario de voz</p><p>· <a href="https://developer.amazon.com/docs/custom-skills/delegate-dialog-to-alexa.html#automatically-delegate-simple-dialogs-to-alexa">Auto delegation</a> para que la skill pregunté de forma automática cuando falte información</p><p>· <a href="https://ask-sdk-for-nodejs.readthedocs.io/en/latest/">ASK Software Development Kit for Node.js</a> para procesar las peticiones que se envien al back-end de tu skill</p><p>· <a href="https://ask-sdk-for-nodejs.readthedocs.io/en/latest/Managing-Attributes.html">Attributos Persistentes</a> con <a href="https://aws.amazon.com/s3/">Amazon S3</a> para que tu skill recuerde la información aun al terminarse la sesión</p><p>· <a href="https://developer.amazon.com/docs/smapi/alexa-settings-api-reference.html">Alexa Settings API</a> para recuperar la zona horaria y el nombre del usuario</p><p>· <a href="https://developer.amazon.com/docs/smapi/alexa-reminders-overview.html">Reminders API</a> para enviar un recordatorio en la fecha de cumpleaños</p><p>· <a href="https://developer.amazon.com/docs/alexa-presentation-language/apl-overview.html">Alexa Presentation Language (APL)</a> para hacer la skill visualmente atractiva en dispositivos con pantalla</p><h3>Más Material de Entrenamiento para Mejorar tus Skills Alexa</h3><p>Una vez que hayas completado este curso te recomendamos que continúes tu aprendizaje con los siguientes recursos:</p><p>· <a href="http://alexa.design/cdw">Designing for Conversation Course</a>: Aprende a diseñar experiencias conversacionales más dinámicas (en inglés)</p><p>· <a href="https://developer.amazon.com/docs/alexa-design/get-started.html">Alexa Design Guide</a>: Aprende los principios de diseño de voz situacional para que crees skills voice-first que sean naturales y enfocadas en el usuario (en inglés)</p><p>· <a href="https://www.twitch.tv/videos/409503308?filter=archives&amp;sort=time">How to Shift from Screen-First to Voice-First Design</a>: Aprende los cuatro patrones de diseño que hacen que una experiencia voice-first sea atractiva (en inglés)</p><h3>Comienza Ya a Construir la Skill “Feliz Cumpleaños”</h3><p>Este curso es gratuito y está diseñado para que lo hagas a tu propio ritmo por lo que cualquier persona puede utilizarlo para construir su primera skill de Alexa. <a href="https://alexa.design/es-course%20">Haz click aquí</a> para comenzar. ¡Y danos tu opinión! Puedes contactarme vía Twitter <a href="https://twitter.com/germanviscuso">@GermanViscuso</a> y enviarme tus comentarios.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2fbd7994820a" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>