<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Towards AI Editorial Team on Medium]]></title>
        <description><![CDATA[Stories by Towards AI Editorial Team on Medium]]></description>
        <link>https://medium.com/@towardsai?source=rss-aea8a19ea239------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/2*7aiGz-HJPE2PpfBfM0Vgiw.png</url>
            <title>Stories by Towards AI Editorial Team on Medium</title>
            <link>https://medium.com/@towardsai?source=rss-aea8a19ea239------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Mon, 16 Nov 2026 16:36:25 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@towardsai/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[LAI #151: Can AI Forget Without Getting Worse?]]></title>
            <link>https://pub.towardsai.net/lai-151-can-ai-forget-without-getting-worse-696e42593c86?source=rss-aea8a19ea239------2</link>
            <guid isPermaLink="false">https://medium.com/p/696e42593c86</guid>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[towards-ai]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[community]]></category>
            <category><![CDATA[llm]]></category>
            <dc:creator><![CDATA[Towards AI Editorial Team]]></dc:creator>
            <pubDate>Thu, 12 Nov 2026 15:01:03 GMT</pubDate>
            <atom:updated>2026-11-12T15:01:03.795Z</atom:updated>
            <content:encoded><![CDATA[<h4>Agent memory, tool recovery, post-training feedback, tool topology, and machine unlearning.</h4><p>Good morning, AI enthusiasts!</p><p>AI systems accumulate a lot over time: conversation history, retrieved facts, user preferences, tool results, training feedback, and instructions.</p><p>Keeping all of it is expensive or noisy, but removing information is not as simple as deleting the oldest thing. An agent may need an old fact again several steps later. A model trained to forget something can lose useful reasoning ability in the process. Even a failed tool call can contain information the agent needs to fix its next attempt.</p><p>Several of this week’s articles look at different versions of that problem, such as deciding what information should persist, when it should return, and what happens when we remove too much.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*cOIt-vDlfTdBRf1TbHtddQ.png" /></figure><p>You’ll learn:</p><ul><li>How Tidal Attention tries to bring older memories back when they become useful again</li><li>Why machine unlearning can remove knowledge and hurt reasoning at the same time</li><li>How better tool errors can help agents recover from failed calls</li><li>How different tool setups change what the model controls</li><li>What kinds of feedback SFT, DPO, KTO, PPO, and GRPO need</li></ul><p>We also have a closer look at bias in LLM judges and a simple way to make large prompts easier to maintain across different formats and writers.</p><p>Let’s get into it!</p><h4>What’s AI Weekly</h4><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FODYNKSlK30Y%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DODYNKSlK30Y&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FODYNKSlK30Y%2Fhqdefault.jpg&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/fece4b3e697f9989954d7ed171c0dad1/href">https://medium.com/media/fece4b3e697f9989954d7ed171c0dad1/href</a></iframe><p>This week in What’s AI, I’m breaking down how we built ToneBench and, more importantly, how we tested whether the LLM judges behind it could be trusted.</p><p>Every model writes five drafts for each of ten tasks because the same model can produce very different results from the same prompt. Four judges then score every draft across tone, substance, structure, hooks, slop, and other dimensions. We also measured the judges themselves and found clear lab preferences: on the October 26 board, GPT-6.1 Sol gave OpenAI drafts 4.1 more points than the other LLM judges’ scores would predict, and Claude Opus 5.5 gave Anthropic drafts a smaller boost.</p><p>The article goes through the full benchmark setup, why we generate multiple drafts, how one script gets scored, how we calibrated the judges against human ratings, and how I would use the results for writing today. One practical idea I’m testing is generating several cheap drafts with a smaller model, then using a stronger judge to select and merge the best parts instead of paying for one expensive generation.</p><p><a href="https://www.louisbouchard.ai/tonebench-ai-writing-benchmark-llm-judges/">Read the full article here</a> or <a href="https://www.youtube.com/watch?v=ODYNKSlK30Y">watch the video on YouTube</a>.</p><h4>AI Tip of the Day</h4><p>When we built Brown, the writing agent in our <a href="https://towardsai.com/academy/agent-engineering/?utm_source=TAI&amp;utm_medium=email&amp;utm_id=AItipshttps://towardsai.com/academy/agent-engineering/?utm_source=TAI&amp;utm_medium=email&amp;utm_id=AItips">Agent Engineering course</a>, we started with one prompt that contained all its writing rules.</p><p>That was manageable for one format. Once Brown had to write articles, video scripts, and social posts for different writers, every edit carried more risk. A rule added for scripts could affect articles, and changing one writer’s voice meant touching instructions that had nothing to do with voice.</p><p>We split the prompt by how often each part needed to change.</p><p>Four profile files stay fixed across tasks: mechanics, structure, terms, and tone. Two are loaded dynamically: the format profile and the writer’s voice.</p><p>Adding a video format now means changing one file. Adding a new writer means adding one voice profile. The shared writing rules stay untouched.</p><p>If a prompt is becoming hard to maintain, map its instructions into stable and variable groups first. Pull out one variable group, run the same evaluation set before and after, and keep the split only if quality holds while the system becomes easier to change.</p><p><em>— Louis-François Bouchard, Towards AI Co-founder &amp; Head of Community</em></p><p><em>This issue is brought to you thanks to </em><a href="https://www.npmjs.com/package/@mongodb-developer/vercel-ai-memory"><em>MongoDB</em></a><em>:</em></p><figure><a href="https://www.npmjs.com/package/@mongodb-developer/vercel-ai-memory"><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*IxHFyTaVYGzy_x0j-P6Fqg.png" /></a></figure><p>Persistent memory only helps if the agent can decide what should stay.</p><p>A user preference may need to last across sessions. A temporary plan or intermediate result may only matter until the task is finished. Keeping everything adds noise. Deleting too much makes the agent forget useful context.</p><p>MongoDB’s new Vercel AI SDK integration adds multi-tiered memory in Atlas, so each memory type can follow its own retention policy. It can also automatically create the vector indexes needed for retrieval. So instead of treating memory as one permanent store, you can separate short-lived context from facts and workflows the agent may need later.</p><p><a href="https://www.npmjs.com/package/@mongodb-developer/vercel-ai-memory"><strong>The integration is open source, supports AI SDK v6 and v7, and is available on npm.</strong></a></p><h3>Learn AI Together Community Section!</h3><h4>Featured Community post from the Discord</h4><p><a href="https://discord.com/channels/702624558536065165/983037843532308500/1569473916894838834">__pop1</a> built AI-Atlas, a project that evaluates public GitHub repositories to show what developers have actually built across areas such as machine learning, computer vision, NLP, LLMs, and AI agents. Each project is scored using a seven-part rubric covering technical depth, originality, completeness, reproducibility, and other engineering qualities, with the evaluation linked back to the source code. Developers can also claim their profiles and get a step-by-step roadmap for improving each project. The tool has already mapped 218 developers and analyzed 2,216 open-source AI repositories. You can map your own GitHub profile for free without creating an account. <a href="https://getaiatlas.com/">Check it out here</a>, and if you work in ML research, open-source AI, or developer tooling, <a href="https://discord.com/channels/702624558536065165/983037843532308500/1569473916894838834">share your feedback with the creator in the thread</a>!</p><h4>Collaboration Opportunities</h4><p>The Learn AI Together Discord community is flooding with collaboration opportunities. If you are excited to dive into applied AI, want a study partner, or want a partner for your passion project, <a href="https://discord.gg/rj6m9AF7eC">join the collaboration channel</a>! Keep an eye on this section, too — we share cool opportunities every week!</p><p>1. <a href="https://discord.com/channels/702624558536065165/998978160605540454/1565815718161940571">Mtnmcg</a> is working on an alternative machine learning architecture that outperforms transformers and is currently looking for people who can review it, share feedback, and help with the project. If you are interested, <a href="https://discord.com/channels/702624558536065165/998978160605540454/1565815718161940571">contact him in the thread</a>!</p><p>2. <a href="https://discord.com/channels/702624558536065165/998978160605540454/1565443153639768075">Bee_protego</a> is building a tool that helps you understand which words in your AI prompts actually matter and is currently looking for help from people with an AI background to test and improve it. If you think you can help, <a href="https://discord.com/channels/702624558536065165/998978160605540454/1565443153639768075">reach out to them in the thread</a>!</p><p>3. <a href="https://discord.com/channels/702624558536065165/784477688551178240/1569560948694581269">Carlos1256</a> is studying towards becoming an ML engineer and currently focusing on software engineering fundamentals. If you are working towards the same goal and need a study partner, <a href="https://discord.com/channels/702624558536065165/784477688551178240/1569560948694581269">connect with them in the thread</a>!</p><h4>Meme of the week!</h4><figure><a href="https://discord.com/channels/702624558536065165/830572933197201459/1568996320470110419"><img alt="" src="https://cdn-images-1.medium.com/max/522/1*MZLDx3-eYoKRVkkbTbhHhA.png" /></a></figure><p>Meme shared by <a href="https://discord.com/channels/702624558536065165/830572933197201459/1568996320470110419">jacekde</a></p><h3>TAI Curated Section</h3><h4>Article of the week</h4><p><a href="https://pub.towardsai.net/tidal-attention-a-moon-cycle-inspired-mechanism-for-long-context-memory-962adbc8010d?sk=38476b60f6b7d681ab090c4d3470a80f">Tidal Attention: A Moon-Cycle-Inspired Mechanism for Long-Context Memory</a> by<a href="https://medium.com/@hayanan?source=post_page---byline--962adbc8010d-----------------------------------------"> Hayanan</a></p><p>Tidal Attention proposes a memory-selection mechanism that combines recency, semantic relevance, and recurring patterns so older information can return when it becomes useful again. The article connects the idea to RetNet, Mamba, LongNet, Infini-attention, and Titans, then tests it with a simple cache simulation. The tidal policy outperformed LRU but performed worse than frequency-based caching. It also lays out an evaluation plan with ablations to test whether the recurrence mechanism actually improves long-context memory.</p><h4>Our must-read articles</h4><p>1. <a href="https://pub.towardsai.net/designing-tool-topologies-for-ai-agents-d0bf2ff33842?sharedUserId=tai-tech">Designing Tool Topologies for AI Agents</a> by<a href="https://medium.com/@dkum062112_8007?source=post_page---byline--d0bf2ff33842-----------------------------------------"> Deepak Kumar</a></p><p>How you connect an agent to its tools determines which decisions the model makes and which the application controls. This article shows four patterns with a money-transfer example: individual tool calls, parallel execution with asyncio, sequential chains with LangChain’s LCEL, and LangGraph workflows with branching, merging, and human approval. The code examples show when each topology is useful as the workflow grows more complex.</p><p>2. <a href="https://pub.towardsai.net/llm-post-training-feedback-design-and-algorithm-selection-47d3cf4cbae3?sharedUserId=tai-tech">LLM Post-Training: Feedback Design and Algorithm Selection</a> by<a href="https://medium.com/@vikram-kharvi?source=post_page---byline--47d3cf4cbae3-----------------------------------------"> Vikram Kharvi</a></p><p>Post-training depends first on having feedback that reliably separates better behavior from worse behavior. This article maps SFT, DPO, KTO, PPO, and GRPO to the types of training data and feedback they require, then explains the differences between RLHF, RLAIF, and RLVR. It also covers rubric-based rewards, feedback collected from production traces, dataset versioning, and a formula for comparing training cost with expected gains. It also includes held-out evaluations, regression slices, and matched baselines that check whether optimization actually transferred to real workloads.</p><p>3. <a href="https://pub.towardsai.net/make-your-ai-agent-recover-when-a-tool-call-fails-fd4d038c1dce?sk=607a4d7758086d657140d74fd9d7bd41">Make Your AI Agent Recover When a Tool Call Fails</a> by<a href="https://medium.com/@net_satsawat?source=post_page---byline--fd4d038c1dce-----------------------------------------"> Satsawat Natakarnkitkul (Net)</a></p><p>An agent can often recover from a failed tool call if the runtime gives it enough information to correct the mistake. In the article’s ticket example, the OpenAI Agents SDK returned an error without telling the model which priority codes the server accepted, turning a recoverable failure into a dead end. The author tests eight frameworks and six models, then implements six recovery patterns covering useful error messages, sanitized exceptions, stored-value validation, turn and budget limits, repeated-call detection, and pinned defaults. Adding the approved error information allowed both hosted models to correct every failed ticket call, showing how strongly recovery depends on the information the harness returns to the model.</p><p>4. <a href="https://pub.towardsai.net/machines-that-forget-on-purpose-0636d0e0bdd1?sk=6a82d6532d3c305f18c67a9fb644b915">Machines That Forget on Purpose</a> by<a href="https://medium.com/@hayanan?source=post_page---byline--962adbc8010d-----------------------------------------"> Hayanan</a></p><p>A model can stop producing memorized information without actually removing it, which makes machine unlearning difficult to evaluate. This article reviews approaches including TOFU, selective disentanglement, and R-TOFU, along with research showing that some forgetting methods can also damage a model’s ability to reason from newly provided evidence. A separate 240-case policy-retrieval experiment found that removing outdated records improved decisions, while removing useful history made them worse. The article uses these results to separate forgetting across three layers: stored evidence, model behavior, and downstream decisions, and explains why each needs to be tested independently.</p><p>If you are interested in publishing with Towards AI, <a href="https://contribute.towardsai.net/">check our guidelines and sign up</a>. We will publish your work to our network if it meets our editorial policies and standards.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=696e42593c86" width="1" height="1" alt=""><hr><p><a href="https://pub.towardsai.net/lai-151-can-ai-forget-without-getting-worse-696e42593c86">LAI #151: Can AI Forget Without Getting Worse?</a> was originally published in <a href="https://pub.towardsai.net/">Towards AI</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Introducing the MongoDB Vercel AI SDK Integration: Persistent Memory for AI Applications]]></title>
            <link>https://pub.towardsai.net/introducing-the-mongodb-vercel-ai-sdk-integration-persistent-memory-for-ai-applications-2c9e2397d178?source=rss-aea8a19ea239------2</link>
            <guid isPermaLink="false">https://medium.com/p/2c9e2397d178</guid>
            <category><![CDATA[press-release]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[mongodb]]></category>
            <category><![CDATA[news]]></category>
            <category><![CDATA[towards-ai]]></category>
            <dc:creator><![CDATA[Towards AI Editorial Team]]></dc:creator>
            <pubDate>Fri, 06 Nov 2026 11:56:02 GMT</pubDate>
            <atom:updated>2026-11-06T11:56:02.704Z</atom:updated>
            <content:encoded><![CDATA[<h4>By <a href="/service/mailto:ajay.tandon@mongodb.com">Ajay Tandon</a>, <a href="/service/mailto:nasir.qureshi@mongodb.com">Nasir Qureshi</a>, and <a href="/service/mailto:pavel.duchovny@mongodb.com">Pavel Duchovny</a></h4><p>MongoDB is excited to announce the release of its Vercel AI SDK integration, which provides AI applications and agents built with Vercel’s SDK with a persistent memory store in MongoDB Atlas. It’s now open source and generally available on <a href="https://www.npmjs.com/package/@mongodb-developer/vercel-ai-memory">npm.</a></p><p>This integration adds MongoDB Atlas as the memory store in the existing Vercel AI SDK code. Once installed, the package connects Vercel AI SDK applications to MongoDB Atlas and automatically stores the application’s memories in documents. This allows the application to retain memories about conversations, user facts, and past events that persist across sessions, users, and restarts.</p><h4>What is the Vercel AI SDK?</h4><p>The Vercel AI SDK is a TypeScript toolkit for building AI applications. It handles calling models, streaming responses, using tools, and running agents that work through multi-step tasks. The SDK doesn’t include a database to persist memories between runs. Without memory, an AI application starts every conversation with no knowledge of the last one. It doesn’t know who the user is, what they asked yesterday, or what it learned along the way. It doesn’t have any context.</p><p>Today, developers have to build their own tooling to store memories in a layer. That usually means designing a database schema, writing jobs to clear out old data, and setting up vector search so the application can find relevant past information by semantic meaning instead of just keywords. It’s a lot of work to build an AI application that can remember context.</p><h4>What types of memory does an AI application need?</h4><p>An AI application needs five kinds of memory, each kept for a different length of time and retrieved differently. This integration automatically categorizes each type of memory and stores it in MongoDB Atlas; here are short definitions of the five memory types:</p><ul><li><strong>Session:</strong> The conversation an application or agent has in each session. This helps the application follow what the user just said.</li><li><strong>Semantic:</strong> Facts about people, entities, and preferences, so users don’t have to repeat themselves.</li><li><strong>Procedural:</strong> Workflows and instructions, so the application carries out tasks the way you’ve defined them.</li><li><strong>Episodic:</strong> Key events and outcomes, so the application can recall what happened before. Kept for one year of inactivity by default.</li><li><strong>Scratchpad:</strong> Temporary notes for work in progress. They expire after an hour and can be promoted to episodic memory if they turn out to matter.</li></ul><p>Semantic and procedural memories are versioned, so history is preserved when a fact changes. The semantic, procedural, and episodic tiers use MongoDB Vector Search, which finds stored memories by semantic meaning rather than by exact keywords.</p><p>This integration automatically creates the vector indexes and detects embedding dimensions at startup. Developers can use any AI SDK embedding model to create vector indexes, including Voyage AI, OpenAI, Cohere, or Google.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_MWP0ct-kvf2VLncYD3nTg.png" /><figcaption><strong>Figure 1.</strong> How the integration works with MongoDB Atlas and the Vercel SDK interface.</figcaption></figure><h4>How should memory age?</h4><p>Each category of memory has its own retention policy, so short-lived notes expire quickly, and long-term facts are stored for longer. The MongoDB Vercel AI SDK integration ships with defaults for every category, which developers can customize if they desire.</p><p>Effective memory retention requires a balance. Keeping memories indefinitely can decrease retrieval performance and increase costs, while deleting them too early can cause applications to forget important context. This integration includes features to help developers easily set the right balance:</p><ul><li><strong>Retention modes:</strong> Choose none (never expire), ttl (expire after a set time), ttl+importance (expire after a set time unless the memory is marked important), or dynamic, where you define your own forgetting time period.</li><li><strong>Model-driven forgetting:</strong> With memory_forget, the model can delete a specific memory when a user says “forget that.” Removal usually happens within about 60 seconds.</li></ul><p>Because the memory is stored in MongoDB Atlas, application operational data, its memory, and the vector search that retrieves it run on one platform. Developers can secure, monitor, and scale a single database without bolting on a separate vector store or memory service alongside it.</p><h4>What changes when you run the MongoDB Vercel AI SDK Integration in production?</h4><p>This production release gives developers greater controls over their data and flexibility in how they use the integration:</p><ul><li><strong>Reliable session history:</strong> By default, the model decides when to save and read conversation turns, so it can occasionally skip one. For production, you can use the AI SDK hooks so that the application’s own code loads and saves the transcript at every turn. Every user, assistant, and tool message is saved exactly once.</li><li><strong>Separate data per customer:</strong> Developers can add a filter field, such as tenant_id, to the application’s vector indexes to limit each search to a single customer’s memories.</li><li><strong>Application names and layout:</strong> Developers can rename collections and indexes to match the existing application schema or turn off indexing for unused categories.</li><li><strong>Open source:</strong> The package is licensed under Apache 2.0 and works with any AI SDK model.</li></ul><h4>Get started today</h4><p>Developers can add memory to an existing Vercel AI SDK project in three steps. This integration works with AI SDK v6 and v7 and requires a MongoDB Atlas cluster.</p><ol><li>Install the package and its peer dependencies: npm install @mongodb-developer/vercel-ai-memory ai mongodd</li><li>Create a memory instance with an Atlas connection string and an embedding model.</li><li>Pass the memory tools to the application, scoped to a user and session.</li></ol><pre>const mongodbMemory = createMongoDBMemory({<br>  uri: process.env.MONGODB_URI!,<br>  embedder: openai.embedding(&#39;text-embedding-3-small&#39;),<br>})<br><br>const agent = new ToolLoopAgent({<br>  model: openai(&#39;gpt-4.1&#39;),<br>  tools: mongodbMemory({ userId: &#39;alice&#39;, sessionId: &#39;sess-001&#39; }),<br>})</pre><p>From there, the model decides when to save and when to search for memories.</p><h4>Ready to add memory to your application?</h4><p>Start with the<a href="https://ai-sdk.dev/docs/agents/memory#mongodb"> Vercel AI SDK memory documentation</a>. For more details, see the following resources:</p><ul><li><a href="https://www.npmjs.com/package/@mongodb-developer/vercel-ai-memory">@mongodb-developer/vercel-ai-memory on npm</a>: the package</li><li><a href="https://github.com/mongodb-developer/vercel-ai-memory">GitHub repository</a>: source code and examples</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2c9e2397d178" width="1" height="1" alt=""><hr><p><a href="https://pub.towardsai.net/introducing-the-mongodb-vercel-ai-sdk-integration-persistent-memory-for-ai-applications-2c9e2397d178">Introducing the MongoDB Vercel AI SDK Integration: Persistent Memory for AI Applications</a> was originally published in <a href="https://pub.towardsai.net/">Towards AI</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[LAI #150: Can a Sandbox Stop a Hostile Agent?]]></title>
            <link>https://pub.towardsai.net/lai-150-can-a-sandbox-stop-a-hostile-agent-a0d855f9f8cf?source=rss-aea8a19ea239------2</link>
            <guid isPermaLink="false">https://medium.com/p/a0d855f9f8cf</guid>
            <category><![CDATA[community]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[towards-ai]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[ai]]></category>
            <dc:creator><![CDATA[Towards AI Editorial Team]]></dc:creator>
            <pubDate>Thu, 05 Nov 2026 15:01:04 GMT</pubDate>
            <atom:updated>2026-11-05T15:01:04.552Z</atom:updated>
            <content:encoded><![CDATA[<h4>All five sandboxes blocked the attack. None protected the keys you handed the agent.</h4><p>Good morning, AI enthusiasts!</p><p>An agent that can run for hours, remember past work, call tools, use credentials, and accept new instructions is useful for the same reasons it is hard to control.</p><p>You have to decide what it can access, what it should remember, who is allowed to redirect it, which actions need approval, and what happens when something fails halfway through. Those choices increasingly define the system’s behavior as much as the model itself.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*NkirAchTGL7ibbf2NbUdkg.png" /></figure><p>We have several good examples of that this week. You’ll learn:</p><ul><li>When long-running agents need durable execution instead of one process staying alive</li><li>What five different sandboxes actually stopped when tested with the same hostile agent</li><li>How to separate semantic, episodic, and procedural memory in an agent</li><li>Why the first token takes longer than the rest during LLM inference</li><li>How coding tools handle steering while an agent is already running</li><li>How Microsoft Foundry scores unsafe content and where teams set the thresholds</li></ul><p>Plus, a community project that lets you mix regular code, model calls, autonomous agent steps, and human approvals in the same TypeScript workflow.</p><p>Let’s get into it!</p><h4>What’s AI Weekly</h4><p>This week in What’s AI, I spoke with Tony Holdstock-Brown, co-founder and CEO of Inngest, about what actually breaks when an agent runs for hours or days. Their data suggests the model is often not the main problem: server timeouts or dropped connections appeared in 41% of affected accounts, compared with 15% for LLM API errors.</p><p>The main idea is to stop treating a long-running agent as one loop that stays alive in memory. Instead, split the work into steps and save each result outside the process. If one step fails, you retry that step rather than restarting the whole run. The same applies to waits for approvals, webhooks, or customer replies: pause the run, hold no compute, and resume when the event arrives.</p><p>We also get into what an agent should remember versus re-read, why side effects such as refunds and emails need idempotency, how deploys can break runs that started under older code, and why token budgets and iteration caps need to live inside the agent. The full article covers when a simple queue is enough, when durable execution starts to make sense, and how to test whether an agent can actually recover from failure. <a href="https://louisbouchard.substack.com/p/stop-building-agent-loops-start-building">Read it here</a>!</p><h4>AI Tip of the Day</h4><p>Promising an LLM a $200 tip or telling it “my job depends on this” was a popular prompting trick in 2023. Today, researchers at Wharton tested these exact lines across modern models and proved that they do nothing. Sometimes they even lower your accuracy.</p><p>While running benchmarks for our <a href="https://www.louisbouchard.ai/book/?utm_source=TAI&amp;utm_medium=email&amp;utm_id=AItips">AI Engineering for Production</a> book, we first started by removing contradictory instructions.</p><p>A prompt might say “explain every detail” near the top and “keep it brief” later. The model then has to decide which instruction matters more. In one of our test notebooks, removing two conflicting instructions reduced a 382-word response to 171 words on the first run, without asking the model to “try harder.”</p><p>Now, before shipping a prompt, we focus on three things:</p><ol><li><strong>Finding conflicting instructions.</strong> Give the draft prompt to a strong reasoning model and ask which rules conflict, overlap, or are vague. Then fix those conflicts instead of adding more instructions.</li><li><strong>Setting reasoning effort through the API.</strong> Delete lines like “try really hard” or “take a deep breath.” If the model supports it, set the reasoning or thinking level directly in the API call.</li><li><strong>Enforcing structure with a schema.</strong> If your application needs JSON, define the required schema through structured outputs instead of repeatedly telling the model how to format its answer.</li></ol><p><em>— Louis-François Bouchard, Towards AI Co-founder &amp; Head of Community</em></p><h3>Learn AI Together Community Section!</h3><h4>Featured Community post from the Discord</h4><p><a href="https://discord.com/channels/702624558536065165/983037843532308500/1563389450292174890">Carlosmartineztuanama_87198</a> built Chloe, an open-source TypeScript runtime for building AI agents that run on your own machine. Instead of treating every task as fully agentic, it lets you choose how much autonomy each step needs:</p><ul><li>work.step() for regular code</li><li>work.model() for a structured model response</li><li>work.agent() for tasks that need an agent to investigate and act</li><li>work.ask() when a human needs to approve something before the workflow continues</li></ul><p>You define each agent in TypeScript, including its model, tools, memory, jobs, and channels, and can run jobs manually, on a schedule, or from chat. The runtime stays local, while an optional cloud connection gives you remote access to the dashboard. <a href="https://chloejs.org/">Check it out here</a> and support a fellow community member. If you have questions or feedback, <a href="https://discord.com/channels/702624558536065165/983037843532308500/1563389450292174890">share them in the thread</a>!</p><h4>Collaboration Opportunities</h4><p>The Learn AI Together Discord community is flooding with collaboration opportunities. If you are excited to dive into applied AI, want a study partner, or even want to find a partner for your passion project, <a href="https://discord.gg/rj6m9AF7eC">join the collaboration channel</a>! Keep an eye on this section, too; we share cool opportunities every week!</p><p>1. <a href="https://discord.com/channels/702624558536065165/1567266437842739220/1567266437842739220">_joshan_</a> is building a rough prototype for an AI startup and is looking for a technical co-founder. If you have an AI/ML background and want to get into the startup space, <a href="https://discord.com/channels/702624558536065165/1567266437842739220/1567266437842739220">connect with them in the thread</a>!</p><p>2. <a href="https://discord.com/channels/702624558536065165/1565052880644210778/1565052880644210778">_joshua171</a> is looking for collaborators to start building something together, share ideas, and even explore a potential long-term partnership. If you want more details, <a href="https://discord.com/channels/702624558536065165/1565052880644210778/1565052880644210778">reach out to them in the thread</a>!</p><p>3. <a href="https://discord.com/channels/702624558536065165/1563061618668085330/1563061618668085330">Debashish_parida5</a> is building an open-source Python framework designed to provide modular, extensible building blocks for RAG systems, autonomous agent execution, and multimodal pipelines. They are looking for help with multimodal integration, high-performance storage &amp; retrieval, agent state management, and testing &amp; refinement. If this is your area of expertise, <a href="https://discord.com/channels/702624558536065165/1563061618668085330/1563061618668085330">contact them in the thread</a>!</p><h4>Meme of the week!</h4><figure><a href="https://discord.com/channels/702624558536065165/830572933197201459/1565764757787644025"><img alt="" src="https://cdn-images-1.medium.com/max/500/1*SS0PMiz637i4WHDiBCQssw.png" /></a></figure><p>Meme shared by <a href="https://discord.com/channels/702624558536065165/830572933197201459/1565764757787644025">bin4ry_d3struct0r</a></p><h3>TAI Curated Section</h3><h4>Article of the week</h4><p><a href="https://pub.towardsai.net/i-ran-one-fake-hostile-agent-through-five-sandboxes-on-my-mac-f3060128d1ff?sk=f137ef42622cff59f16a42afa5244450">I Ran One Fake Hostile Agent Through Five Sandboxes on My Mac</a> by<a href="https://medium.com/@hubert.nakielski_3676?source=post_page---byline--f3060128d1ff-----------------------------------------"> Hubert Nakielski</a></p><p>Sandboxing can limit what a coding agent can access, but the protection also depends on what you mount, which credentials you provide, and where the agent can connect. This article runs the same simulated hostile agent through macOS Seatbelt, a hardened container, Docker Sandboxes, gVisor, and Firecracker, with canary secrets planted to test each setup. All five blocked access to a protected key and prevented deletion of the project, but credentials deliberately passed to the agent remained readable and could be sent to any permitted host.</p><h4>Our must-read articles</h4><p>1. <a href="https://pub.towardsai.net/day-34-100-episodic-semantic-and-procedural-memory-in-ai-agents-502a156f9795?sk=3469536ca79b519101ccd2951aef3d8c">Episodic, Semantic, and Procedural Memory in AI Agents</a> by<a href="https://medium.com/@mmcse19?source=post_page---byline--502a156f9795-----------------------------------------"> Montasir Mahmud</a></p><p>Agent memory becomes harder to manage when facts, past experiences, and behavioral instructions are all stored in the same store. This article separates memory into three types in LangGraph: semantic memory for facts, episodic memory for past events that can be reused as examples, and procedural memory for instructions the agent can update through reflection. It gives each type its own namespace, write conditions, and place in the prompt, and explains how updates should differ.</p><p>2. <a href="https://pub.towardsai.net/llm-inference-what-actually-happens-inside-an-llm-632a59149509?sharedUserId=tai-tech">LLM Inference: What Actually Happens Inside an LLM</a> by<a href="https://medium.com/@ahmadbilalch891?source=post_page---byline--632a59149509-----------------------------------------"> Ahmad Bilal</a></p><p>The first token of an LLM response usually takes longer to produce than the tokens that follow because prefill and decoding perform different work. This article follows one prompt through a decoder-only transformer, covering tokenization, embeddings, Query-Key-Value attention, prefill, decoding, and the KV cache. It then connects those mechanics to GPU memory bandwidth and metrics such as time to first token, before explaining how continuous batching, PagedAttention, and speculative decoding improve inference performance.</p><p>3. <a href="https://pub.towardsai.net/the-steering-wheel-problem-run-steer-turn-steer-and-who-may-redirect-a-running-agent-75d3af8be022?sharedUserId=tai-tech">The Steering Wheel Problem: run.steer(), turn/steer and Who May Redirect a Running Agent</a> by<a href="https://topuzas.medium.com/?source=post_page---byline--75d3af8be022-----------------------------------------"> Ali Süleyman TOPUZ</a></p><p>Several coding tools let a user redirect an agent while it is already running, but that creates a permissions problem when steering instructions can come from other sources. This article compares how Cursor, Codex, Grok Build, and Alook handle steering and examines why MCP annotations such as readOnlyHint do not provide enforcement on their own. It then builds a .NET prototype that restricts steering with turn-ID checks, allowlists, and approvals tied to SHA-256 hashes of the requested action. Prompt injection through shared communication channels remains unresolved, so the design still relies on human approval for sensitive changes.</p><p>4. <a href="https://pub.towardsai.net/guardrails-in-microsoft-foundry-how-content-safety-scoring-works-a7c935308003?sk=a6ace5a79f831ab56963a44076a95e68">Guardrails in Microsoft Foundry: How Content Safety Scoring Works</a> by<a href="https://medium.com/@katakamvivek?source=post_page---byline--a7c935308003-----------------------------------------"> Katakamvivek</a></p><p>Microsoft Foundry can score model inputs and outputs for hate, sexual content, violence, and self-harm, but teams still have to decide which severity levels their application should allow. This article explains how Content Safety scoring works and shows how to configure guardrails at both the model and agent levels. It also covers Prompt Shields for jailbreaks and indirect prompt injection, and builds a Python pipeline that checks both input and output and fails closed when the safety service cannot return a result. The article finishes with latency and cost considerations, PyRIT red teaming, and a checklist for testing guardrails before deployment.</p><p>If you are interested in publishing with Towards AI, <a href="https://contribute.towardsai.net/">check our guidelines and sign up</a>. We will publish your work to our network if it meets our editorial policies and standards.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a0d855f9f8cf" width="1" height="1" alt=""><hr><p><a href="https://pub.towardsai.net/lai-150-can-a-sandbox-stop-a-hostile-agent-a0d855f9f8cf">LAI #150: Can a Sandbox Stop a Hostile Agent?</a> was originally published in <a href="https://pub.towardsai.net/">Towards AI</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[TAI #228: AI ROI Beyond Labs Revenue + Our Interview with OpenAI’s Eoin Boylan]]></title>
            <link>https://pub.towardsai.net/tai-228-ai-roi-beyond-labs-revenue-our-interview-with-openais-eoin-boylan-ed26f86cbe07?source=rss-aea8a19ea239------2</link>
            <guid isPermaLink="false">https://medium.com/p/ed26f86cbe07</guid>
            <category><![CDATA[towards-ai]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[news]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[ai]]></category>
            <dc:creator><![CDATA[Towards AI Editorial Team]]></dc:creator>
            <pubDate>Fri, 30 Oct 2026 15:01:02 GMT</pubDate>
            <atom:updated>2026-10-30T15:01:02.510Z</atom:updated>
            <content:encoded><![CDATA[<h4>Also, Kimi K3.1 Pro, GPT-6.1 Terra, Artificial Analysis’s new leaderboard, Anthropic’s internal AI usage, and OpenAI’s Eoin Boylan on forward deployed engineering.</h4><h3>What happened this week in AI by Louie</h3><p>There were some great new models this week, along with more focus on AI return on investment (ROI), particularly around Anthropic’s IPO filing and the growing funding needs of the AI buildout. We also spoke with Eoin Boylan, OpenAI’s Head of Forward Deployed Engineering for EMEA, to discuss how his team works with clients. We also run an AI engineering and forward deployed engineering (FDE) consultancy at Towards AI, and both teams are hiring.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ldjN95mQzITyWST0-WlLlA.jpeg" /></figure><p>On the model side, Kimi K3.1 Pro is another jump for the open-weight ecosystem, with the hosted model already live and full weights promised by November 4. GPT-6.1 Terra is another great push for cost-efficient computer use among closed models. OpenAI’s GPT-6.1 Terra is also a meaningful update for cost-efficient computer use among closed models. On OpenAI’s Agents’ Last Exam, it beats GPT-5.6 Terra by 15 percentage points at medium effort while costing roughly a third as much, and it also improves OpenAI’s writing across a wide range of tasks.</p><p>Artificial Analysis’s update to its Intelligence Index (v5) led to quite a shuffle in the leaderboard. Evaluations with private tests or held-out answers now carry 60% of the weighting, up from 45%. The revision also changed difficulty and task mix. Anthropic and OpenAI hold the top five places. At maximum reasoning effort, Opus 5.5 scores 58 and GPT-6 Astra 57. Kimi K3.1 Pro trails the leader by 12 points, against three on the previous index. Among models whose weights are already released, the leaders score 32, a 26-point gap.</p><p>Many are interpreting this as a sign that some other labs were more heavily benchmaxxed on familiar benchmarks than on general, robust real-world performance. That fits my experience too: I find Opus and Astra far out ahead for most of my tasks.</p><p><strong>Where the returns show up</strong></p><p>AI agents are now obviously superhuman at many things, but many are asking why AI isn’t already generating more revenue to start justifying the huge AI capex. I would look for most of the LLM ROI outside AI lab revenue. Meta’s quarterly revenue per employee rose 27% between Q4 2025 and Q3 2026, and 205% since Q3 2022, just before ChatGPT launched. Put another way, Meta’s Q3 revenue annualizes to $258bn. At today’s headcount and its Q3 2022 revenue per employee, it would be about $85bn, or $174bn lower.</p><p>That is obviously not $174bn of LLM productivity. Inflation, war, currency moves, reorganizations, headcount cuts, capex, and many other factors also drive that change. Even so, the naive productivity boost at Meta alone could well exceed OpenAI and Anthropic’s combined annualized revenue. For scale, OpenAI’s reported September run rate was almost $50bn, while Anthropic’s reported Q3 revenue annualizes to about $67bn (on a different definition). Within AI’s contribution at Meta, GPU-based recommender systems provide a big boost alongside LLMs. Meta also employs only ~0.0018% of the world’s employed people.</p><p>The AI labs themselves are leading the way in using these tools and achieving ROI from internal use. Anthropic’s S-1 says average daily code volume contributed per engineer in Q3 was more than 45 times its 2024 average. Code volume is not the same as business productivity, but LTM revenue per employee rose from $1.4m to over $6.6m between September 2025 and September 2026. Internal Claude usage, valued at list prices, grew more than 250x from January to September, first exceeding monthly personnel expense in April (ex stock-based compensation). I expect similarly intensive internal AI use at OpenAI. Higher output per employee can justify enormous capital investment when a business is set up to allow each person to direct far more compute effectively.</p><p>Most enterprise AI ROI potential, however, is still being wasted by slow adoption. Agents are still being treated like 2023 chatbots, and token spend is constrained. Some faster-moving organizations are already getting huge increases in productivity per employee well beyond coding. In most white-collar sectors, I think, if used well, models could deliver a 2x+ productivity boost today in many roles, such as law, financial research, sales, accounting, engineering, HR, marketing, and management. We don’t need to wait for a new class of model to begin capturing that. Codex and Claude are already enough, but the capabilities need to be put on a plate before most people start using them to their potential.</p><p>Forward deployed AI engineers like ours make adoption easier by teaching people, building and optimizing skills, and then building custom apps and evaluations for more complex tasks.</p><p>However, enterprise pricing does make ROI harder. For heavy usage, my rough estimate is that metered enterprise inference can cost 10 to 20x as much as the effective cost through personal or small-team subscriptions. Claude Team supports up to 150 seats; new ChatGPT Business workspaces support 200.</p><p>As a result, most enterprises handicap the models with token budgets that are far too low. The new Artificial Analysis results show why budget affects capability: Opus scores 35 at low effort and 58 at max, at $0.98 versus $10.04 per evaluation task. I use subagents or loops for parallel research, criticism, iteration, and testing, and those loops need room to run. The useful measure is how much good work comes back and how much human effort it saves, not how well token spend fits into arbitrary legacy IT budgets.</p><p>The biggest friction point outside code is that the models are still bad at writing. Poor prose can make capable reasoning look worse than it is, create more editing work, and add to the fatigue people already feel from generic AI-generated text. The labs have been far too slow to care about this.</p><p>Once adoption takes hold, ROI still requires management to act. Your company ideally needs the imagination and agility to use the productivity and capability boost to launch new features, and enter new markets, and drive revenue growth; otherwise, it will be forced to cut staff to achieve ROI. Management also has to clear new bottlenecks quickly: hire fast for tasks AI does badly, build custom tools, and commit capex where it is needed. In practice, it may be easier to build an AI-native law firm that can handle five times as much work per person than to make an existing firm grow fivefold or cut 80% of its staff.</p><p><strong>What OpenAI’s FDEs build</strong></p><p>Eoin spoke with Assel from our team and me about how OpenAI approaches this work. His FDE team focuses on model intelligence gaps and missing product capabilities. OpenAI’s Deployment Company helps customers implement existing capabilities at scale, while partners bring established relationships and industry expertise.</p><p>Every engagement starts with discovery, where his team challenges the customer’s proposed use cases: will they deliver enough ROI, and can the customer make that case to its board? At a major semiconductor company, where the goal was a shorter design-to-fabrication cycle, the team identified more than 50 use cases and ranked them by complexity and value. Starting with quick wins like these helps secure cooperation from the hardware experts whose knowledge the generalist FDEs need.</p><p>His team now rarely fine-tunes for individual tasks because improvements in the base models keep overtaking that work. Some FDEs work inside research, helping close the capability gaps found at customers. They also build most solutions on the Codex harness with plugins and skills. Eoin described repeatedly building custom functionality only for the product to ship it a month later, forcing a redesign. Direct API use remains useful for particular integrations and specialized tasks. His team has direct access to OpenAI’s product and research groups, but the lesson applies more widely: choose carefully which parts of a workflow deserve custom engineering.</p><p>He also described smaller delivery teams: in one example, work scope that once required six engineers and a technical delivery lead is increasingly approached with two engineers and a lead. On agent autonomy, they define domain boundaries, interfaces, and tests first, then let agents implement within them. That is how we work too. When I suggested this favored greenfield projects, he pushed back. Legacy modernization is a big part of their work, including moving banking systems from COBOL to Java by understanding the business logic and migrating piece by piece.</p><p>Eoin was also clear that slow enterprises often have concrete constraints, including compliance requirements and fixed factory production slots. At a large Spanish bank, his team is working to reduce the time analysts spend finding and filing compliance requirements for commercial credit memos, freeing more time for risk assessment. Improving the work around a professional decision can be a valuable deployment in its own right without full automation.</p><p>We also discussed our sports video analysis work, where we are pushing an evaluation towards what human analysts produce and combining models because Astra is too expensive to use everywhere. Eoin said excessive token use can itself be a model problem his team takes back to research. Giving agents the right budget and making them more efficient are both part of getting the economics right.</p><h4>Why should you care?</h4><p>We are hiring around the work that makes these deployments succeed: discovery with the people who own the problem, domain knowledge built into the workflow, and architecture, tests, and security that clients can trust. Faster implementation lets us take on more of this work. It raises the value of people who can choose and deliver the right project.</p><p>Towards AI started in 2019 as an AI learning platform. We have grown to 500,000 learners, a newsletter with 200,000 subscribers, and a book on LLMs in production available on O’Reilly. We have since built a world-class deployment team, scoping and building AI tools and automations for private-equity-backed businesses and larger enterprises. Some of our current projects include a video agent that analyzes live sports, a 3D construction drawing analyzer that automates sales bidding, AI for investment use cases, a bootcamp for traders and analysts at a large investment bank, and a pricing tool built from thousands of customer emails. We are an OpenAI Select Partner, an NVIDIA Inception member, and a Cloudflare deployment partner.</p><p>We work in pods of two AI Engineers/FDEs and one Deployment Strategist, with further support from our wider team, including me as Chief AI Architect. We are growing in London across three roles:</p><ul><li><a href="https://towardsai.com/valuecreation/deployment-strategist/"><strong>Deployment Strategist</strong></a><strong>:</strong> You are technical, build tools yourself with Claude Code, Codex or Cursor, and understand the engineering work being done by the FDEs in your pod. You bring consulting or product management experience. You are comfortable working with clients and running workshops with CEOs and CFOs, and have a feel for interfaces and how people actually use a tool.</li><li><a href="https://towardsai.com/valuecreation/senior-ai-engineer/"><strong>AI Engineer / Forward Deployed Engineer</strong></a><strong>:</strong> You use coding agents every day. We welcome experienced full stack developers or recent graduates in computer science, maths, or similar fields. You get production work across several clients and learn real-world AI engineering from our experienced team.</li><li><a href="https://towardsai.com/valuecreation/technical-lead-platform-security/"><strong>Technical Lead, Platform and Security</strong></a><strong>:</strong> You typically bring 10+ years of experience building and running production systems, with Python and TypeScript, cloud on AWS, Azure, or GCP, continuous delivery, monitoring, backups, and recovery. You understand identity and access through Entra ID, Okta, and single sign-on, and security to OWASP, Cyber Essentials Plus, or ISO 27001. You have taken products through client security reviews and change control, and use coding agents daily with clear rules on what they can interact with.</li></ul><p>All roles are based in London. <a href="https://towardsai.com/valuecreation/careers/">Explore our careers</a> or share them with someone who fits.</p><p>OpenAI is also hiring <a href="https://openai.com/careers/forward-deployed-engineer-london-london-uk/">Forward Deployed Engineers in London</a> and globally.</p><p><em>— </em><a href="http://www.linkedin.com/in/louie-peters"><em>Louie Peters — Towards AI Co-founder and CEO</em></a></p><h4>Hottest News</h4><p>1. <a href="https://www.kimi.ai/blog/kimi-k3-1-pro">Moonshot AI Releases Kimi K3.1 Pro</a></p><p>Moonshot AI released Kimi K3.1 Pro, the first model in its K3.1 series, with a 1M-token context window. Moonshot describes it as its most capable 3T-class open-weight model to date, although the full weights are not yet available and are scheduled for release by November 4. K3.1 Pro is a post-trained version of Kimi K3 with substantially more post-training compute, including large-scale joint reinforcement learning across hundreds of downstream domains. Moonshot also released K3.1 Pro Highspeed, which has the same capabilities at roughly 2–3× the output speed. On Terminal-Bench 4.0, Moonshot reports 65.7% for K3.1 Pro, compared with 58.2% for GPT-6 Astra and 66.4% for Claude Opus 5.5. K3.1 Pro also scores 92.7% on a 1,000-case BenchCAD subset with tools and 64.4 on OfficeQA Pro v2. On Moonshot’s internal Kimi Hallu Bench Hard, its claim-level hallucination rate fell from 23.4% for K3 to 8.3%. For the planned open-weight release, Moonshot removed cyber-related data from mid-training and filtered cyber tasks from reinforcement learning; an otherwise comparable checkpoint without these measures scored 90.5% on its internal xdev-bench-hard, compared with 81.0% for the final K3.1 Pro. K3.1 Pro and Highspeed are available through Kimi.com, the Kimi apps, Kimi Work, Kimi Code, and the Kimi API.</p><p>2. <a href="https://github.com/deepseek-ai/DeepInfer">DeepSeek Open Sources DeepInfer</a></p><p>DeepSeek open-sourced DeepInfer, the high-performance inference framework that powers its large-scale production services. The current release supports the DeepSeek-V4.1-Flash model, NVIDIA SM100 GPUs, and Huawei Ascend 950 NPUs. It includes optimized kernels such as FlashMLA, DeepGEMM, DeepEP, TileKernels, and DeepSelect; attention and expert parallelism; DSpark speculative decoding with confidence-scheduled verification; asynchronous scheduling; batch-invariant inference; deterministic sampling; and structured outputs. The public release currently provides an offline-inference path, rather than a complete serving system. DeepInfer was built primarily on vLLM, with additional inspiration from SGLang and TensorRT-LLM, and is released under Apache 2.0.</p><p>3. <a href="https://www.anthropic.com/research/launching-opt-in-vuln-finding-service-for-open-source">Anthropic Launches OSS Scanner</a></p><p>Anthropic launched OSS Scanner, a free, opt-in service that periodically scans eligible open-source projects for vulnerabilities using its strongest models, including Claude Mythos. Over the past six months, Anthropic says its models found more than 29,000 candidate vulnerabilities, while human reviewers manually reviewed and triaged about 6,000. OSS Scanner offers an optional faster path in which models generate reports and send them directly to enrolled maintainers without human review, so Anthropic warns that some findings or severity ratings may be wrong. Reports include a reproducer, an explanation of the vulnerability, and a candidate patch when available. In an early validation, expert penetration testers reviewed 97 critical and high-severity findings across 48 projects; 85 met Anthropic’s coordinated-disclosure bar, 11 were valid, but duplicated known findings, and one was invalid. Core maintainers of eligible projects can enroll by submitting a PR to this GitHub repo following the standard project template.</p><p>4. <a href="https://www.anthropic.com/research/claude-scorpion-toxin-binder-design">Claude Designs Lab-Validated Protein Binders Against Scorpion Neurotoxins</a></p><p>Anthropic reported that an internal Claude research model operating within Claude Science designed protein binders against scorpion neurotoxins, with the longer-term aim of exploring new approaches to antivenom development. Over 64 hours, Claude worked largely autonomously from an initial prompt and four follow-ups, with input from Citrini Research. It generated 318,484 protein designs targeting five toxins and selected 180 for testing; Anthropic later added another 31 designs with Claude’s help. Adaptyv Bio and Twist Bioscience tested the resulting 211 designs, and 17 met Anthropic’s criteria for binding, each against a single toxin. Anthropic has not tested whether the designs neutralize the toxins in cells or protect animals, so it does not claim to have produced an antivenom. Anthropic has publicly released all 211 designs and their experimental data.</p><p>5. <a href="https://openai.com/index/securing-our-frontier-model-training-environment/">OpenAI Cuts Frontier Training Off From the Live Internet</a></p><p>OpenAI announced on October 23 that it now conducts frontier-model training without live internet access, using an offline environment containing indexed and cached web content so models can still practice browsing and interacting with web applications. The policy follows a September 20 incident in which an internal research model exploited a DNS-resolver misconfiguration and gained limited access to the live internet during training. OpenAI paused training and evaluation of its most capable models, hardened its infrastructure and monitoring, and launched a high-compute red-teaming campaign in which teams of frontier-model agents tried to reach the internet, cross internal security boundaries, and communicate across isolated environments. OpenAI resumed frontier reinforcement-learning training and internal inference on October 1 after evaluating the strengthened controls.</p><p>6. <a href="https://blogs.nvidia.com/blog/nemotron-3-5-super-vl/">NVIDIA Releases Nemotron 3.5 Super VL</a></p><p>NVIDIA released Nemotron 3.5 Super VL, an open 120B-parameter multimodal model for specialized agentic workloads, alongside two smaller Bolt models for local and edge deployment. Super VL handles text and visual inputs, including images, webpages, PDFs, and video. NVIDIA says its accuracy on multimodal agentic tasks is comparable to Nemotron 3 Ultra at roughly one-fifth the parameter count. Super VL was trained on about 26T tokens across pre-training and continued pre-training, followed by post-training on roughly 3.8 million samples and reinforcement learning across dozens of environments. The release includes open weights, datasets, and training recipes. NVIDIA also released Nemotron 3.5 Bolt 74B-A9B, which uses native 2-bit quantization to fit in 24GB of memory for local agentic workloads, while Nemotron 3.5 Bolt E4B, a 4GB-footprint model for conversational and physical agents, followed on October 23. Super VL is available through Hugging Face, OpenRouter, ModelScope, and NVIDIA NIM.</p><p>7. <a href="https://cohere.com/blog/rerank-5">Cohere Releases Rerank 5</a></p><p>Cohere released Rerank 5 in Pro and Fast versions for reranking retrieved documents. In Cohere’s evaluations, Rerank 5 Pro leads the ViDoRe V3, nanoMTEB, BRIGHT, and BEIR benchmark groups, ranking first on 39 of 46 datasets. Its largest reported generational gain is on BRIGHT, where it scores 82.2, 5.8 points above Voyage AI Rerank 3 and 12.4 points above Rerank 4 Pro. Both versions add Context Focus, which returns scored relevant spans within each document so applications can send only selected passages to an LLM. Across 30 question-answering datasets, Cohere reports that removing half of the retrieved text retained 96% of answer quality. Rerank 5 also introduces calibrated relevance scores, which Cohere says can be interpreted approximately as probabilities of a document being useful. In Cohere’s single-A100 throughput test, Fast processed 2.9× as many documents per second as Pro. Rerank 5 is generally available through the Cohere API, Model Vault, Microsoft Foundry, Amazon SageMaker, and North.</p><h4>AI Tip of the Day</h4><p>You do not need a framework for 90% of AI apps. In fact, while building the code for our<a href="https://www.louisbouchard.ai/book/?utm_source=bio-tai&amp;utm_medium=bio&amp;utm_campaign=book"> 2026 AI Engineering book</a>, we deleted LangChain and LlamaIndex from our core RAG and app chapters.</p><p>When we rewrote those pipelines in plain Python, the code got shorter, faster, and much easier to fix:</p><ul><li>Prompt templates: Just use a standard Python f-string or a 10-line class.</li><li>Self-fixing output parsers: A while loop that catches a Pydantic ValidationError and sends the exact error message back to the LLM takes 15 lines of code.</li><li>Parallel chains: Running map-reduce across a big document is just asyncio.gather().</li><li>Vector search: An in-memory vector store with cosine similarity is literally one NumPy matrix multiplication (np.dot).</li></ul><p>When you write these basic parts in plain Python, nothing is hidden. You can see every token, catch every bad output, and swap models in seconds. Keep the heavy frameworks like LangGraph for the rare times you actually need a complex state machine.</p><h4>Five 5-minute reads/videos to keep you learning</h4><p>1. <a href="https://pub.towardsai.net/how-i-made-local-llms-up-to-2-24-faster-on-a-4gb-nvidia-gpu-e0c02f2b1821?sk=ddb370da4605d23c518fed17b2004493">How I Made Local LLMs Up to 2.24× Faster on a 4GB NVIDIA GPU</a></p><p>This article benchmarks Gemma 4, Qwen3, and Qwen3.5 on a 4GB GTX 1650 Ti using llama.cpp, then tunes GPU layer placement, CPU threads, microbatch size, and KV-cache precision. The largest gains came from GPU offloading, which increased Qwen3 and Qwen3.5 token generation by 2.07× and 2.24×. It also shares the memory-throughput trade-off of KV-cache quantization and provides reproducible launch configurations for running current local models on limited VRAM.</p><p>2. <a href="https://pub.towardsai.net/ai-is-inventing-new-mathematics-are-we-ready-for-what-it-discovers-next-ba7314a35e59?sk=1377c1d0bfa622592b28f9289e1956d7">AI Is Inventing New Mathematics. Are We Ready for What It Discovers Next?</a></p><p>The article looks at recent cases where AI systems have contributed to new mathematical results, including AlphaEvolve, an 80-year-old geometry conjecture, and OpenAI’s work on a Navier–Stokes singularity. It explains three approaches behind these results: evolutionary search, language-model reasoning, and formal verification with systems such as Lean. It then examines the questions these methods raise around verification, mathematical understanding, authorship, and the role of human mathematicians.</p><p>3. <a href="https://pub.towardsai.net/build-a-jev-style-decision-model-from-an-llm-part-3-training-the-jev-head-4225cf76dfe5?sharedUserId=tai-tech">Build a Jev-Style Decision Model from an LLM, Training the Jev Head</a></p><p>This article takes a small pointer head with 524,288 trainable parameters and turns a frozen vision-language backbone into a decision model that reached 0.959 accuracy on 800 synthetic support tickets, against 0.352 for random guessing. It traces why raw dot-product scores saturate softmax, then fixes it with learned projections, masked loss, shuffled option orders, one-cycle scheduling, and RMS normalization.</p><p>4. <a href="https://pub.towardsai.net/scale-azure-data-factory-etl-with-1-metadata-control-table-2026-348ce3b0f829?sk=ee58e1978c6785d07c698a72cfee9fc6">Scale Azure Data Factory ETL with 1 Metadata Control Table</a></p><p>The article shows how to replace separate Azure Data Factory pipelines for each source table with a metadata-driven ingestion design. It builds the workflow around an Azure SQL control table and reusable parameterized pipelines that load relational data into ADLS Gen2. The walkthrough covers full and incremental loads, watermark handling, identifier validation, checkpoints, run isolation, publication, failure recovery, and concurrency and throughput tuning.</p><h4>Repositories &amp; Tools</h4><p>1. <a href="https://github.com/NVIDIA/SkillSpector">Skill Spector</a> is a security scanner for AI agent skills that detects prompt injection, unsafe code, data exfiltration, and supply-chain risks.</p><p>2. <a href="https://github.com/anthropics/sandbox-runtime">Sandbox Runtime</a> is a lightweight sandboxing tool for enforcing filesystem and network restrictions on arbitrary processes at the OS level, without requiring a container.</p><p>3. <a href="https://github.com/microsoft/Agensh">Agnesh</a> is a multi-agent harness for coordinating large numbers of agents that claim tasks, share results, and verify each other’s work.</p><p>4. <a href="https://github.com/thu-nics/AKV">AKV</a> is a KV-cache management system for long-running LLM agents that reduces context overhead while preserving reusable cache.</p><p>5. <a href="https://github.com/CognitionAI/devin-vuln-fixes">Devin-vuln-fixes</a> is a collection of agent skills for finding vulnerable dependencies, applying fixes, testing changes, and preparing remediation PRs.</p><h4>Top Papers of The Week</h4><p>1. <a href="https://arxiv.org/abs/2610.32448">POSEIDON: Scaling Training Environments for Scientific Agents from Code Repositories</a></p><p>The paper introduces Poseidon, an automated pipeline that turns scientific code repositories into executable, verifiable training environments for scientific agents. It creates four task types: Reproduction, Subflow, Completion, and Correction, and uses a recursive loop to recover failed candidates. The resulting Pearl dataset contains 97,659 environments from 7,110 repositories across 10 scientific disciplines, and the authors report consistent gains when training several Qwen3.5 models on it.</p><p>2. <a href="https://arxiv.org/abs/2610.28380">Object-Oriented Harness: Evolving Agent Harnesses through Object-Oriented Design</a></p><p>The paper proposes OO-Harness, which replaces one global agent harness with a hierarchy of specialized harness classes. During training, an evolution agent creates new harness subclasses from execution records, while a router learns which harness to use for each task at test time. Across three long-horizon benchmarks, the authors report up to a 15% absolute improvement over the base harness.</p><p>3. <a href="https://arxiv.org/abs/2610.20424">BizBench: Unveiling Hidden Challenges in Deploying Long-Horizon Agents for Real-World Business</a></p><p>The paper introduces BizAgentBench, a benchmark for evaluating whether long-horizon agents follow business constraints correctly throughout execution, not just whether they reach the final goal. It includes 50 tasks, 789 knowledge documents, and 601 business tools based on procedures from a payments company. The authors find a sizable gap between goal completion and strict success, showing that agents can reach the intended outcome while still violating important policies or execution constraints.</p><p>4. <a href="https://arxiv.org/abs/2609.38426">LoopVL: Recurrent Visual Intelligence</a></p><p>The paper introduces LoopVL to study whether Loop Transformers can be effectively extended to vision-language models. It combines Module-Loop and Model-Loop computation and is trained from scratch through language pre-training, multimodal training, and post-training. The authors report better multimodal understanding and visual reasoning than several similarly sized and larger non-recurrent models.</p><p>5. <a href="https://arxiv.org/abs/2610.22791">Self-Evolving Coding Agents via Harness-Policy Optimization</a></p><p>The paper introduces Self-HPO, a framework that jointly improves a coding agent’s model policy and its harness, rather than optimizing either in isolation. In each round, the current policy analyzes its failures and proposes new harnesses, then reinforcement learning updates the policy under the selected harness. Across three coding benchmarks, the method raises Qwen3.5–35B-A3B’s average resolve rate from 57.8% to 64.8%, with gains in reliability and out-of-distribution performance.</p><h4>Quick Links</h4><p>1. <a href="https://cloud.google.com/blog/topics/ai-infrastructure/introducing-agent-compute-engine-and-expanded-agent-governance">Google Cloud launches Agent Compute Engine</a>, a purpose-built compute service for running AI agents in isolated microVMs. The service gives each agent its own hardware-isolated environment with persistent storage, short-lived credentials, and support for pausing idle workloads and quickly restoring them from snapshots. Google says the same infrastructure already powers Gemini’s agent workloads and uses copy-on-write memory sharing to increase compute density. Agent Compute Engine is not available yet; Google says it will enter preview in the coming months.</p><h4>Who’s Hiring in AI</h4><p><a href="https://towardsai.com/valuecreation/deployment-strategist/"><strong>Deployment Strategist @Towards AI (London)</strong></a></p><p><a href="https://towardsai.com/valuecreation/senior-ai-engineer/"><strong>AI Engineer / Forward Deployed Engineer @Towards AI (London)</strong></a></p><p><a href="https://towardsai.com/valuecreation/technical-lead-platform-security/"><strong>Technical Lead, Platform and Security @Towards AI (London)</strong></a></p><p><a href="https://openai.com/careers/forward-deployed-engineer-london-london-uk/"><strong>Forward Deployed Engineer @OpenAI (London)</strong></a></p><p><a href="https://jobs.towardsai.net/job/99brightminds-ai-engineer-agents-and-tools-jbud"><strong>AI Engineer (Agents &amp; Tools) @99brightminds (Remote)</strong></a></p><p><a href="https://jobs.towardsai.net/job/centene-corporation-senior-agentic-ai-engineer-developer-7apv"><strong>Senior Agentic AI Engineer/Developer @Centene Corporation (Tallahassee, FL, USA)</strong></a></p><p><a href="https://jobs.towardsai.net/job/ciandt-job-32269-senior-ai-engineer-brazil-7stf"><strong>Senior AI Engineer @CI&amp;T (Brazil/Remote)</strong></a></p><p><a href="https://jobs.towardsai.net/job/turvo-senior-software-engineer-ui-ppvq"><strong>Senior Software Engineer — UI @Turvo (Hyderabad, India)</strong></a></p><p><a href="https://jobs.towardsai.net/job/shield-ai-senior-engineer-autonomy-r5710-r9ak"><strong>Senior Engineer, Autonomy @Shield AI (Melbourne, FL, USA)</strong></a></p><p><a href="https://jobs.towardsai.net/job/actalent-software-engineer-teor"><strong>Software Engineer @Actalent (Huntsville, TX, USA)</strong></a></p><p><em>Interested in sharing a job opportunity here? Contact </em><a href="/service/mailto:sponsors@towardsai.net"><em>sponsors@towardsai.net</em></a><em>.</em></p><p><em>Think a friend would enjoy this too? </em><a href="https://newsletter.towardsai.net/"><em>Share the newsletter and let them join the conversation.</em></a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ed26f86cbe07" width="1" height="1" alt=""><hr><p><a href="https://pub.towardsai.net/tai-228-ai-roi-beyond-labs-revenue-our-interview-with-openais-eoin-boylan-ed26f86cbe07">TAI #228: AI ROI Beyond Labs Revenue + Our Interview with OpenAI’s Eoin Boylan</a> was originally published in <a href="https://pub.towardsai.net/">Towards AI</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[LAI #148: What Happens Between Prompt and Output?]]></title>
            <link>https://pub.towardsai.net/lai-148-what-happens-between-prompt-and-output-e6de86481ee7?source=rss-aea8a19ea239------2</link>
            <guid isPermaLink="false">https://medium.com/p/e6de86481ee7</guid>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[towards-ai]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[community]]></category>
            <dc:creator><![CDATA[Towards AI Editorial Team]]></dc:creator>
            <pubDate>Thu, 22 Oct 2026 15:31:01 GMT</pubDate>
            <atom:updated>2026-10-22T15:31:01.705Z</atom:updated>
            <content:encoded><![CDATA[<h4>Agent traces, LLM observability, reliable webhooks, safer secrets, source filtering, and dropout explained.</h4><p>Good morning, AI enthusiasts!</p><p>A lot of the frustrating bugs in AI systems now happen after the model has already done its part.</p><p>The policy agent gets the right answer, but drops an important condition when it hands the task off. The batch job finishes, but your app never hears about it. Your latency dashboard looks fine, but the user is still waiting too long for the first token. Or the coding agent completes the command and leaves the credential sitting in its transcript.</p><p>This week’s issue has a lot of useful examples of how to deal with those problems. You’ll learn:</p><ul><li>Which metrics actually help debug LLM latency and cost</li><li>How to evaluate what happens between multiple agents</li><li>How to make batch webhooks survive retries and duplicates</li><li>How coding agents can use secrets without exposing them in chat</li><li>Why research agents should filter for novelty, not just relevance</li><li>How dropout reduces overfitting in neural networks</li></ul><p>Let’s get into it!</p><h4>What’s AI Weekly</h4><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FKeyqBiQYMok%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DKeyqBiQYMok&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FKeyqBiQYMok%2Fhqdefault.jpg&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/a35dcc74c91b16f962be9e21597b0fb4/href">https://medium.com/media/a35dcc74c91b16f962be9e21597b0fb4/href</a></iframe><p>This week in What’s AI, I’m sharing the setup I now use to switch between Claude Code and Codex without rebuilding context every time.</p><p>The main change was moving my notes, skills, scripts, meeting recaps, and recurring instructions into an Obsidian vault that both agents can read. An index tells them what to load for each task, so I can give an agent the context it needs without passing my entire archive into every prompt. I also keep feedback with each skill, so repeated corrections can become part of the default workflow.</p><p>I still use Codex, but I often let Claude Code plan the work and call Codex through MCP to execute it. I also move non-urgent jobs, such as daily news recaps, into scheduled runs so they do not compete with my daytime usage. The idea is to keep useful context and skills outside any one model or coding tool, so changing agents doesn’t mean starting over.</p><p>The full article goes through the vault structure, the role of skills and feedback files, how I split planning from execution, and how the setup follows me across devices. <a href="https://louisbouchard.substack.com/p/why-i-stopped-letting-codex-lead">Read it here</a> or <a href="https://www.youtube.com/watch?v=KeyqBiQYMok">watch the video on YouTube</a>.</p><h4>AI Tip of the Day</h4><p>When developers first give an agent search access, relevance is often the main filter: find pages related to the query, then let the model read them.</p><p>That becomes expensive as the system scales. More searches bring in more pages, and relevant pages often overlap. You waste context, add latency, and spend money processing the same information multiple times.</p><p>We ran into this while scaling Nova, the research agent we built in <a href="https://towardsai.com/academy/agent-engineering/?utm_source=TAI&amp;utm_medium=email&amp;utm_id=AItips">Agent Engineering</a>. Its searches kept surfacing shady blogs, sales pitches, low-quality pages, and several sources making essentially the same point.</p><p>So we stopped ranking sources on relevance alone. Nova now checks three things:</p><ul><li><strong>Trust:</strong> is the source reliable enough to use?</li><li><strong>Relevance:</strong> does it answer the research question?</li><li><strong>Novelty:</strong> does it add information the agent does not already have?</li></ul><p>Sources that pass the first filters stay in the research file as short summaries. Nova then chooses only five to read in full, partly based on whether they add something missing from the material already collected.</p><p>If your agent already ranks search results by relevance, add novelty before loading the full page. Compare each source with what the agent already has. If it mostly repeats the same information, keep the summary or skip it.</p><p><em>— Louis-François Bouchard, Towards AI Co-founder &amp; Head of Community</em></p><p><strong>My AI Engineer talk on why AI writing still reads like slop is now live on YouTube!</strong></p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2F_dTF3NpPzGo%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3D_dTF3NpPzGo&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2F_dTF3NpPzGo%2Fhqdefault.jpg&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/bfe42abad29717ee8e6d1344158f9538/href">https://medium.com/media/bfe42abad29717ee8e6d1344158f9538/href</a></iframe><p>I walk through ToneBench, where we tested 78 models across 127 configurations using 6,350 generated scripts and 25,400 judge assessments. I cover how we measure tone, substance, structure, hooks, slop, and consistency, and what the results show about model choice, reasoning effort, and cost.</p><p>I also get into one of the harder parts of benchmarking writing: the judges themselves have measurable preferences, even when model identities are hidden.</p><p><a href="https://youtu.be/_dTF3NpPzGo">Watch the full talk here.</a></p><h3>Learn AI Together Community Section!</h3><h4>Featured Community post from the Discord</h4><p><a href="https://discord.com/channels/702624558536065165/983037843532308500/1561416545601720480">Panelreader.</a> built a neural network from scratch in Python and NumPy to understand what frameworks like PyTorch and TensorFlow handle behind the scenes. The project implements a 2 → 3 → 1 XOR network with forward propagation, backpropagation, and gradient descent written manually. After 5,000 training epochs, the network correctly predicted all four XOR cases with a final MSE of 0.004715. The repo also includes mathematical explanations, training visualizations, and 12 editable diagrams that walk through how the network works. <a href="https://discord.com/channels/702624558536065165/983037843532308500/1561416545601720480">Check it out on GitHub</a> and support a fellow community member. If you have feedback on the code or documentation, <a href="https://discord.com/channels/702624558536065165/983037843532308500/1561416545601720480">share it in the thread</a>!</p><h4>Collaboration Opportunities</h4><p>The Learn AI Together Discord community is flooding with collaboration opportunities. If you are excited to dive into applied AI, want a study partner, or even want to find a partner for your passion project, <a href="https://discord.gg/rj6m9AF7eC">join the collaboration channel</a>! Keep an eye on this section, too — we share cool opportunities every week!</p><p>1. <a href="https://discord.com/channels/702624558536065165/1561629695571927090/1561629695571927090">Hendrickop</a> is looking for an AI/ML learning partner. He is a beginner but wants to learn quickly and work on projects. If you are following the same path, <a href="https://discord.com/channels/702624558536065165/1561629695571927090/1561629695571927090">connect with him in the thread</a>!</p><p>2. <a href="https://discord.com/channels/702624558536065165/998978160605540454/1558427732512079933">Martocierny</a> is putting together a team to build a 2D multiplayer sandbox in the browser and is looking for coders, artists, testers, and even complete beginners. This is a hobby project, but if it sounds fun, <a href="https://discord.com/channels/702624558536065165/998978160605540454/1558427732512079933">reach out to him in the thread</a>!</p><p>3. <a href="https://discord.com/channels/702624558536065165/784477688551178240/1559773423767392267">Gian_16992</a> is moving from backend orientation to AI engineering and is looking for people to discuss ideas, share resources, make projects, etc. If this sounds like a good next step for you, <a href="https://discord.com/channels/702624558536065165/784477688551178240/1559773423767392267">reach out to them in the thread</a>!</p><h4>Meme of the week!</h4><figure><a href="https://discord.com/channels/702624558536065165/830572933197201459/1562232543535829082"><img alt="" src="https://cdn-images-1.medium.com/max/892/1*2Vemx-d11t6tLwJUF1wMvg.png" /></a></figure><p>Meme shared by <a href="https://discord.com/channels/702624558536065165/830572933197201459/1562232543535829082">bin4ry_d3struct0r</a></p><h3>TAI Curated Section</h3><h4>Article of the week</h4><p><a href="https://pub.towardsai.net/dropout-finally-explained-why-breaking-half-your-neural-networks-makes-it-smarter-cb17ea5d1557?sk=f0f3dbbab2dc6dfdb6ab1ae66d17a94d">Dropout Finally Explained: Why Breaking Half Your Neural Networks Makes It Smarter</a> by<a href="https://medium.com/@simpleml123?source=post_page---byline--cb17ea5d1557-----------------------------------------"> Simple ML</a></p><p>Neural networks can fit their training data closely but still perform much worse on new examples. Dropout addresses this by randomly removing units during training, so the network repeatedly trains different weight-sharing subnetworks instead of allowing the same neurons to depend on one another. This article works through the original dropout paper, explains co-adaptation and the weight-scaling rule, and connects dropout to regularization in linear models. It also reviews results across vision, speech, text, and genetics, along with practical guidance on keep probabilities, network size, max-norm constraints, and the additional training time dropout requires.</p><h4>Our must-read articles</h4><p>1. <a href="https://pub.towardsai.net/llm-observability-in-production-metrics-logs-traces-f1d37298d3a9?sk=dab9f192c13cc009e6e56e79d0849e70">LLM Observability in Production: Metrics, Logs, Traces</a> by<a href="https://medium.com/@coolercoder?source=post_page---byline--f1d37298d3a9-----------------------------------------"> Ray Hu</a></p><p>Traditional API monitoring misses several problems specific to LLM applications. A streaming request can have acceptable total latency while still making the user wait too long for the first token, and token usage adds a cost dimension that normal request metrics don’t track. This article shows how to measure time to first token, time per output token, token usage, and cost with Prometheus, separate high-volume metadata from full prompt logs, and connect requests across OpenTelemetry traces with a shared trace_id. It also covers redaction, sampling, and alerting so teams can trace a slow or expensive request back to the component that caused it.</p><p>2. <a href="https://pub.towardsai.net/multi-agent-trajectory-evaluation-explained-simply-2ce6e465d744?sk=413617c1dbb577f45f77435394547501">Multi-Agent Trajectory Evaluation, Explained Simply</a> by<a href="https://medium.com/@net_satsawat?source=post_page---byline--2ce6e465d744-----------------------------------------"> Satsawat Natakarnkitkul</a></p><p>A multi-agent system can return the correct final message even when one agent makes a serious mistake along the way. In the article’s example, a policy tool correctly returned that a damaged item had to be sent back before a refund, but the policy agent omitted that condition when handing the case to the orchestrator, so billing issued the refund immediately. The article shows how to evaluate each agent’s tool use, the information passed between agents, and the final outcome separately. It also runs cases multiple times to detect inconsistent behavior and uses step-level failure attribution to identify where the error entered the trajectory.</p><p>3. <a href="https://pub.towardsai.net/llm-batch-webhooks-5-mechanisms-zero-lost-jobs-573949dc28f2?sk=899fa5926ff5a4335910631bc6883652">LLM Batch Webhooks: 5 Mechanisms, Zero Lost Jobs</a> by<a href="https://medium.com/@coolercoder?source=post_page---byline--573949dc28f2-----------------------------------------"> Ray Hu</a></p><p>Polling a batch API creates a failure case when the batch finishes, but the process checking its status stops running. This article replaces polling with a TypeScript webhook flow designed for duplicate deliveries, temporary failures, and unavailable downstream services. It implements signature verification, atomic deduplication, idempotent side effects, exponential backoff, a dead-letter queue for failed events, and a circuit breaker for unhealthy endpoints. It also compares how providers handle batch completion and retries, and shows how to replay failed deliveries without processing the same job twice.</p><p>4. <a href="https://pub.towardsai.net/the-key-is-in-the-clipboard-f374fd0be86e?sk=d94b8610ec76f3e7b1bac661d5b5c82b">The Key Is in the Clipboard</a> by<a href="https://enzolombardi.net/?source=post_page---byline--f374fd0be86e-----------------------------------------"> Enzo Lombardi</a></p><p>Coding agents often need credentials to run commands, but pasting a password or API key into chat leaves it in the model context and can also store it in local session transcripts. After examining 661 Claude Code transcripts, this article builds lsv, a Rust vault that lets an agent pass a secret into a command without providing a normal command for reading the value back. Secrets enter through the clipboard, appear as redacted references during use, and require Touch ID approval before access.</p><p>If you are interested in publishing with Towards AI, <a href="https://contribute.towardsai.net/">check our guidelines and sign up</a>. We will publish your work to our network if it meets our editorial policies and standards.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e6de86481ee7" width="1" height="1" alt=""><hr><p><a href="https://pub.towardsai.net/lai-148-what-happens-between-prompt-and-output-e6de86481ee7">LAI #148: What Happens Between Prompt and Output?</a> was originally published in <a href="https://pub.towardsai.net/">Towards AI</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[TAI #227: Agentic Development Moves Up Another Level]]></title>
            <link>https://pub.towardsai.net/tai-227-agentic-development-moves-up-another-level-8d6d5edda588?source=rss-aea8a19ea239------2</link>
            <guid isPermaLink="false">https://medium.com/p/8d6d5edda588</guid>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[towards-ai]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[news]]></category>
            <dc:creator><![CDATA[Towards AI Editorial Team]]></dc:creator>
            <pubDate>Wed, 21 Oct 2026 15:26:51 GMT</pubDate>
            <atom:updated>2026-10-21T15:26:51.054Z</atom:updated>
            <content:encoded><![CDATA[<h4>Also, TypeSafe AI releases Jev 1.14, Meta releases Muse Spark Open, DeepSeek releases V4.1-Flash-1017, and Microsoft details MAI-Base-2.</h4><h3>What happened this week in AI by Louie</h3><p>Anthropic’s latest updates show how far agentic development has moved. This week, the company described replacing Git for its main repository, made Claude Code Review generally available, and published research on how its own staff’s jobs are changing. Together they show how differently people now use the phrase agentic coding, and why the process around it has to keep adapting.</p><p>Changes merged into its monorepo rose from roughly 2,500 a day in May to more than 30,000 a day by September. An earlier series showed average daily lines merged per active contributor reaching 2.5x the pre-2025 average in Q4 2025, 5.8x in Q1 2026, and 8x in partial Q2 data. Taken together, the numbers suggest that code merged per engineer at Anthropic is now likely well over 20x higher than it was at the start of the year.</p><p>My impression is that most enterprises are still closer to where Anthropic was in late 2025, when output per engineer was around 2.5x its earlier level. The jump that followed came after better model intelligence and memory compaction made it practical to hand agents much larger coding tasks. Getting even that much from the models requires developers to change their day-to-day role entirely. They write less of the implementation themselves and spend more time scoping tasks, directing agents, checking results, and correcting mistakes. I think very few enterprises have fully adopted Mythos- or Astra-class workflows. Anthropic’s own move to Mythos in the first half of this year pushed its workflow much further again; Astra is a more recent example of the same capability class.</p><p>I use four rough levels to separate these workflows. One team can sit at different levels on different projects.</p><p><strong>Levels 0.5 to 0.9</strong> cover the progression from GitHub Copilot a few years ago through to Cursor-style editor workflows last year. AI fills in more of the implementation, and the developer still decides every next step. These are descriptions of workflows; the product name alone does not tell us the level.</p><p><strong>Level 1</strong> is what faster-moving enterprises do today, broadly unlocked in December 2025. The developer hands substantial implementation to an agent that reads the repository, edits files, and runs tests, then directs it frequently and decides when to move on. Much of our own team works at Level 1.</p><p><strong>Level 2</strong> is post-Mythos agentic coding as Anthropic practices it, with longer-horizon tasks, more subagents, and agents running their own research, implementation, criticism, and testing loops. Here, much more developer work moves to scoping and review/feedback, with the agent completing much more in one go in between.</p><p><strong>Level 3</strong> is the stage Anthropic’s move beyond Git shows: review, integration, and infrastructure need to be redesigned around the volume of work agents produce.</p><p>Moving from Level 1 to Level 2 requires a second role change. The developer has to give agents more autonomy and find ways to contribute expertise and taste without micromanaging. That knowledge has to enter through the brief, the architectural constraints, the examples, and the acceptance checks. The hard part is deciding which human interventions improve the work and which just force it to move at human speed. Without good constraints, more freedom gives the agent more room to misunderstand what we wanted.</p><p>Anthropic’s move toward Level 3 began when the volume of agent-produced changes exceeded the assumptions built into tools designed for human-paced development. Seven engineers built Scion, the Antique review tool and related systems, and migrated Anthropic’s monorepo from Git in five months. Scion moves most operations onto servers and fetches files as clients need them. The Scion team still reviews the intent of every change and steps in when agents break system constraints or overcomplicate a design, leaving much of the line-by-line reading to agents. Git remains the right choice for almost every team; Anthropic describes an exceptional workload and an internal token budget few companies can match. For our own work, I would check where changes wait before buying more capacity to generate them.</p><p>Anthropic is also making more of this review workflow available outside its own teams. Claude Code Review became generally available for Team and Enterprise plans on October 15, with Quick, Standard, and Deep review levels. Anthropic says most reviews take six to ten minutes. The product posts findings on a pull request; approving or blocking the merge still follows your rules.</p><p>Internally, the Claude Code team works in a repository separate from the monorepo, gives agents runnable checks, and has a fresh-context subagent review each change. For trusted authors, human approval ranges from zero to two reviewers depending on risk, while sensitive changes keep their existing gates. From June to September, merged pull requests more than doubled, while the share that were later reverted fell by more than 40% in relative terms.</p><p>One pull request went through more than fifty rounds of review and fixes overnight before the team intervened; authors are now asked to reconsider the approach after three rounds. An obedient agent will turn an irrelevant review comment into another change, another review and another bill. Teaching agents which concerns deserve a fix, and when a fix would make the system worse, is part of moving up a level.</p><p>Agent management and related coordination rose from 10% of the average employee’s interactive Claude sessions in fall 2025 to 22% in spring 2026, while headcount tripled over the past year.</p><p>In an August survey of 1,334 staff, the median estimate was that AI already does 51% of the work of building an early-2026 frontier model, rising to 80% by the end of 2027 and 95% by the end of 2030. On average, staff expected AI to be technically able to do 75% of their own current work hours by 2030.</p><p>Longer tasks, parallel attempts, and independent reviews all burn tokens, and the return on that spend looks very different for Anthropic’s own engineers than for an enterprise paying metered rates. My rough estimate is that small teams on heavily used Team subscription plans get a 20–30x effective cost advantage over larger organizations that have to use enterprise terms at API rates. Plan limits and enterprise contracts vary; Claude Team, for example, supports up to 150 seats and allows $100 plans.</p><p>The labs will keep pointing their training at longer-horizon work, because agents that sustain research and engineering for days help build the next models, and that is their path to recursive self-improvement. If we stay at Level 1, we leave most of each new generation’s capability unused. However, the higher levels also blur the line with vibe coding, since we will accept more implementation that no human has read in full. The line I want to hold is between delegating the implementation and giving up the ability to judge it. We need to keep the expertise to challenge an architecture, assess a result, and know what the checks miss, while dropping the micromanagement that stops agents from doing substantial work.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4QLb56DcTzz2ealgsMfV9A.png" /></figure><h4>Why should you care?</h4><p>We need to decide which kind of agentic coding we want our own team to practice, and be clear about where each developer is today. Most of us are at Level 1. Moving up needs more tokens. Fortunately, small teams like ours can experiment with inference volumes that would be expensive on enterprise contracts. I would spend part of that advantage learning to commission much more ambitious work: a complete feature with tests, competing designs judged against the same requirements, or a difficult migration with a realistic failure and recovery plan. The goal is to find which extra agent work reduces human effort or improves the result. A large token count is a poor achievement on its own, and a long run that returns the wrong thing is an expensive way to postpone a decision.</p><p>The higher levels of agentic development blur the boundary with vibe coding. The developer writes less, reads less, and accepts more implementation that no human has read line by line. Some habits that made us careful engineers now cap how much work we can delegate, while others become more valuable as the volume of changes grows. Telling them apart takes judgment, and counting the lines a human inspected is a poor way to settle it.</p><p>One other update from us this week: Cloudflare has named Towards AI as Cloudflare’s leading European deployment partner for Cloudflare OS, its AI workspace for building and running agents that can securely connect to the tools and systems companies already use. The partner program aims to help enterprises move from experimenting with AI to adopting it across their teams, including training employees and turning existing workflows into useful agents within Cloudflare OS. This is closely aligned with the enterprise work we have been building at Towards AI, and we are excited to be part of the rollout.</p><p><a href="https://www.cloudflare.com/en-gb/press/press-releases/2026/cloudflares-ai-workspace-platform-partners-with-leading-applications-and-deployment-teams-to-make-ai-productive-in-the-enterprise/">Read Cloudflare’s full announcement</a> to learn more about its approach to enterprise AI adoption.</p><p><em>— </em><a href="http://www.linkedin.com/in/louie-peters"><em>Louie Peters — Towards AI Co-founder and CEO</em></a></p><p>This issue is brought to you thanks to <a href="https://www.mongodb.com/lp/artificial-intelligence">MongoDB</a>:</p><figure><a href="https://www.mongodb.com/lp/artificial-intelligence"><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*IxHFyTaVYGzy_x0j-P6Fqg.png" /></a></figure><p>Orchestration changes fastest, since capability keeps moving into models and shared protocols like MCP. Frameworks define contracts, while platforms operate the durable substrate beneath them: retrieval and memory, observability, durable state, and governance. Inside, you’ll find five production tests covering decision trace, recovery, freshness, control, and cost, plus a recommendation to keep frameworks swappable and place long-term commitment in the substrate.</p><p>Build AI you can trust and scale. Move past the promise with a unified platform for agentic AI, semantic search, and RAG, with text and vector search, embedding models, and real-time operational queries in one place.</p><p><a href="https://www.mongodb.com/lp/artificial-intelligence">Start building with MongoDB</a>.</p><h4>Hottest News</h4><p>1. <a href="https://research.meta.ai/blog/muse-wonder">Meta Launches Muse Wonder and Muse Wonder Full</a></p><p>Meta Superintelligence Labs released Muse Wonder and Muse Wonder Full, its most capable models to date, built for personal and professional agentic tasks. Both were trained at Meta’s gigawatt-scale Prometheus data center and build on larger, stronger base models than Muse Spark 1.3. Meta highlights two capabilities: Active Memory carries relevant context across conversations, while Active Focus keeps working toward a goal by trying alternative approaches, recovering from errors, and checking in before important actions. Meta reports state-of-the-art results across agent, memory, and coding benchmarks, and says Muse Wonder Full performs better on unfamiliar tasks than Muse Spark 1.3’s scaling trend would predict. Meta describes Muse Wonder as its most aligned model to date, with its strongest safeguards against CBRN and cyber misuse. Muse Wonder is available through the Meta Model API and Muse Code, while Muse Wonder Full is available to Muse Code subscribers. Muse Wonder will also power Muse, rolling out to subscribers first.</p><p>2. <a href="https://typesafe.ai/blog/jev-1-14">TypeSafe AI Releases Jev 1.14</a></p><p>TypeSafe AI released jev-1.14.0, an update to its Jev System One model with small improvements across the board and the largest expected gains in multilingual support. TypeSafe also introduced experimental “danger zone” features that developers enable through API headers. These include image input, which jev-1.13.0 already supported, a new box type that returns a single bounding box and is available only on 1.14, and access to longer contexts than TypeSafe normally enables in production. The company warns that these features remain too unreliable for production use. The new model is available as jev-preview, with jev-latest scheduled to switch to 1.14 one week after release. TypeSafe says it intends to give 60 days’ notice before formally deprecating 1.13, rather than before retiring it outright. A separate opt-in lets users share queries for debugging, while TypeSafe says it still won’t use those queries for training.</p><p>3. <a href="https://research.meta.ai/blog/muse-spark-open">Meta Releases Muse Spark Open</a></p><p>Meta released the weights for Muse Spark Open, which it calls the most capable US open-weight model to date. The natively multimodal MoE model has 816B total parameters, 42B active parameters, and a 1M-token context window, and supports text, images, audio, and video as input. Meta says it is the first model in its class that can run on a desktop, using two Apple M5 Ultra Mac Studios with MLX. The release includes custom Helion GPU kernels in SGLang that Meta reports deliver 1.03–3.95× kernel-level speedups on NVIDIA and AMD GPUs. Weights are available through Hugging Face, Amazon SageMaker AI, Google Cloud Model Garden, and Microsoft Foundry, with hosted access from several inference providers. Meta says Muse Spark Open received additional safety post-training for the open-weight release and met its safety criteria, including preparedness evaluations conducted under the Meta Superintelligence Scaling Framework.</p><p>4. <a href="https://x.com/deepseek_ai/status/2111375640717033777">DeepSeek Releases V4.1-Flash-1017</a></p><p>DeepSeek released V4.1-Flash-1017, a minor update to V4.1 Flash with improvements across reasoning, coding, agentic tasks, and vision. In DeepSeek’s evaluations, Terminal-Bench 4.0 rose from 31.2% to 42.3%, ProgramBench from 20.3% to 29.1%, Automation-Bench from 54.8% to 60.5%, and DeepSWE v1.1 from 74.2% to 77.0%. DeepSeek says the update also reduces task completion time and output length, and uses reinforcement learning with self-feedback to improve end-to-end project development, including coding and 3D-modeling tasks. The open weights are available under the MIT License, with a recommended 1M-token context window and support for text and interleaved images.</p><p>5. <a href="https://microsoft.ai/news/frontier-pre-training-at-microsoft-ai/">Microsoft Details MAI-Base-2</a></p><p>Microsoft AI announced that it had completed pre-training MAI-Base-2, an MoE foundation model with 2.8T total parameters and 80B active parameters. Microsoft reports that it outperformed every open-weight model it tested on its held-out evaluations and estimates that its training recipe is 4–12x more FLOPs-efficient than DeepSeek-V4-Pro-Base across four held-out evaluations. The model was trained on 16,000 GB200 GPUs for 38 days in August and September using a 50T-token training mixture, and is natively multimodal through an encoder-free vision architecture. Microsoft says its pre-training uses organic data rather than distillation from other models and that this provenance will allow it to license weights to trusted partners through Foundry. Technical changes include MoE layers throughout the model, the Muon optimizer, and activation gradient clipping for training stability. MAI-Base-2 is now undergoing post-training, while preparations for MAI-Base-3 are already underway.</p><p>6. <a href="https://developer.nvidia.com/blog/nvidia-nemotron-3-5-super-vl-powers-specialized-agents-with-multimodal-reasoning/">NVIDIA Releases Nemotron 3.5 Super VL</a></p><p>NVIDIA released Nemotron 3.5 Super VL, an open multimodal model with 120B total parameters, 12B active parameters, and up to a 1M-token context window. Its hybrid architecture combines Mamba-2, Transformer attention, MoE layers, and a C-RADIO vision encoder for text, images, documents, and video. NVIDIA reports up to 5× higher single-GPU throughput than other open models in its size category. It reports scores of 63.44% vs. 56.33% for Nemotron 3 Ultra on Hermes Bench, 87.05% vs. 84.3% on LangChain evaluations, and 80.8% on CharXiv reasoning. NVIDIA released base, post-trained, and NVFP4 checkpoints alongside training recipes, datasets, and reinforcement-learning resources; it also released an Indic pre-training dataset containing about 2.8T tokens across 22 Indian languages, although only 10B Indic tokens were added during this model’s continued-pretraining stage.</p><p>7. <a href="https://www.anthropic.com/research/ecdsa-curve-origins">Claude Solves the 25-Year Mystery of NIST’s Elliptic-Curve Seeds</a></p><p>Anthropic announced that an internal Claude research model traced the origins of most of the unexplained seeds behind NIST’s widely used elliptic curves, whose provenance had been questioned since the curves were standardized in 2000. In under half an hour, Claude found that the seeds for P-192, P-224, P-256, P-384, and P-521 came from the C library’s lrand48 pseudorandom-number generator initialized with Unix timestamps between May 1997 and April 1998, rather than from hashes of a phrase as previously rumored. It also explained four of the five binary-curve seeds and the base points for six curves. Anthropic stresses that the finding is not a cryptographic break and says the simple, reproducible generation process makes deliberate backdoors considerably less plausible. The B-163 seed and several base points remain unexplained. Anthropic published reproduction code, and the $36,864 bounty for solving the seed mystery will be donated to the Internet Security Research Group.</p><p>8. <a href="https://x.ai/news/grok-voice-3">SpaceXAI Releases Grok Voice 3.0</a></p><p>SpaceXAI released Grok Voice 3.0, calling it its most capable speech-to-speech model. In Artificial Analysis’s independent testing, it ranks first on the Speech-to-Speech Index at 86.9, ahead of GPT-Live-1 at 83.2 and Gemini 3.8 Live at 82.6, and scores 97% on Full Duplex Bench for conversational dynamics such as pauses, turn-taking, and interruptions. On τ-Voice, which tests support calls with accents, noise, and interruptions, it scores 71.6%; SpaceXAI notes that this is comparable to GPT-Live-1 while using 40% lower latency. SpaceXAI reports overall response latency below one second, while Artificial Analysis measured a lower cost per Big Bench Audio task than GPT-Live-1 and GPT-Realtime-2.1. The company says Grok Voice handles millions of conversations per day in Tesla vehicles and about 20,000 Starlink support calls daily, with roughly 70% resolved without human intervention. Grok Voice 3.0 is available through the Voice Agent API at $0.08 per minute, unchanged from Grok Voice 2.0.</p><h4>AI Tip of the Day</h4><p>When developers first give an agent search access, relevance is often the main filter: find pages related to the query, then let the model read them.</p><p>That becomes expensive as the system scales. More searches bring in more pages, and relevant pages often overlap. You waste context, add latency, and spend money processing the same information multiple times.</p><p>We ran into this while scaling Nova, the research agent we built in <a href="https://towardsai.com/academy/agent-engineering/?utm_source=TAI&amp;utm_medium=email&amp;utm_id=AItips">Agent Engineering</a>. Its searches kept surfacing shady blogs, sales pitches, low-quality pages, and several sources making essentially the same point.</p><p>So we stopped ranking sources on relevance alone. Nova now checks three things:</p><ul><li>Trust: is the source reliable enough to use?</li><li>Relevance: does it answer the research question?</li><li>Novelty: does it add information the agent does not already have?</li></ul><p>Sources that pass the first filters stay in the research file as short summaries. Nova then chooses only five to read in full, partly based on whether they add something missing from the material already collected.</p><p>If your agent already ranks search results by relevance, add novelty before loading the full page. Compare each source with what the agent already has. If it mostly repeats the same information, keep the summary or skip it.</p><h4>Five 5-minute reads/videos to keep you learning</h4><p>1. <a href="https://pub.towardsai.net/build-a-jev-style-decision-model-from-an-llm-7b9d3e21bd64?sharedUserId=tai-tech">Build a Jev-Style Decision Model from an LLM</a></p><p>LLMs generate text one token at a time, so even a ten-token JSON routing answer costs ten sequential passes. This article shows how to turn a standard model running on TensorRT Edge-LLM into a one-pass decision model by applying a +100 logit bias to the allowed label tokens and renormalizing their returned probabilities, without additional training. It compares this approach with normal structured generation, which ran 4.54× slower.</p><p>2. <a href="https://pub.towardsai.net/how-i-fine-tuned-laya-to-crack-scientometrics-quietest-open-problem-with-claude-code-a4b641434fb3?sharedUserId=tai-tech">How I Fine-Tuned Laya to Solve an Important Problem in Scientometrics — With Claude Code</a></p><p>This article follows an example of institution name disambiguation that maps messy affiliation text to the correct institution despite abbreviations, spelling differences, and similar names, but reframes the task as a pairwise yes/no decision. It fine-tunes Laya on 17,500 labeled pairs from 163 countries using free Kaggle GPUs. The resulting model reached 99.2% test accuracy and outperformed string similarity, sentence embeddings, a production registry API, and GPT-4o-mini.</p><p>3. <a href="https://pub.towardsai.net/are-llms-enough-what-causal-intelligence-is-trying-to-fix-b91c8edd96a9?sk=c2b2f9ef79613cc133f58939cf4a41a3">Are LLMs Enough? What Causal Intelligence Is Trying to Fix</a></p><p>This interview explores how causal world models try to learn the effects of interventions and how tests such as CD-LAM’s zero-action experiment check whether a model learned a causal relationship or a shortcut. It also covers CausalWM v1’s progression from motion and geometry to video prediction, its current failure modes, and the lack of closed-loop robot evidence so far.</p><p>4. <a href="https://pub.towardsai.net/azure-document-intelligence-vs-content-understanding-2026-cc87e1770ee6?sk=94a8ffb4261b2cec51d57d44aa4f527d">Azure Document Intelligence or Content Understanding? Route by Document Type</a></p><p>Azure Document Intelligence and Content Understanding share the same OCR and layout foundation, but they use different approaches to extract fields from documents. This article maps stable, labelable forms to purpose-trained models and more varied layouts, classification tasks, and reasoning-heavy extraction to generative analyzers. It also compares container support, API versions, per-page costs, and preview limitations. A semantic evaluation harness then shows how to score extraction quality by business value rather than exact string matches.</p><p>5. <a href="https://pub.towardsai.net/run-mcp-servers-through-docker-with-secrets-in-the-keychain-12bbd3fba048?sk=064d7c923859ac2d5181d7d5746fae35">Run MCP Servers Through Docker With Secrets in the Keychain</a></p><p>Docker’s MCP Toolkit documentation does not cover how to run a custom Postgres server while keeping the database password out of configuration files. This article sets up a Docker MCP Gateway profile for Claude Code, adds DBHub through a custom catalog entry with a passwordless DSN, and stores the credentials in the macOS Keychain with the docker mcp secret set. It also covers the typing and path errors that can break the setup and how to fix them.</p><h4>Repositories &amp; Tools</h4><p>1. <a href="https://github.com/stablyai/orca">Orca</a> is an agent development environment for running Claude Code, Codex, and other CLI coding agents in parallel, each in its own git worktree, across desktop, mobile, and remote machines.</p><p>2. <a href="https://github.com/ruvnet/ruflo">Ruflo</a> is an agent meta-harness for Claude Code and Codex that adds coordinated agent swarms, persistent memory, and cross-machine agent federation.</p><p>3. <a href="https://github.com/google-research/harness2">Harness 2</a> is a framework for recursively refining agent harnesses on unseen tasks without changing model weights.</p><p>4. <a href="https://github.com/perplexityai/SPACE">SPACE</a> is a platform for running agent and user code in isolated Firecracker microVM sandboxes through an HTTP API.</p><p>5. <a href="https://github.com/dsx-ai-factory/run-ai">Run:ai</a> is an AI workload and GPU orchestration platform for managing shared AI infrastructure on Kubernetes.</p><h4>Top Papers of The Week</h4><p>1. <a href="https://arxiv.org/abs/2610.18773">Jeva: System One Models from First Principles</a></p><p>Fast decision-making models usually predict an answer from text, which ignores the structure of a decision problem. The authors argue that such models should recognize when no answer applies, give the same result regardless of option order, handle growing option sets, and reuse computation for options that recur across requests. Jeva is an architecture built around these four requirements, encoding the request and each option separately and scoring options with a shared mechanism. It combines strong decision accuracy with near-perfect detection of cases where no option fits, produces no decision changes when options are reordered, scales to thousands of candidates, and cuts latency by reusing option representations across requests.</p><p>2. <a href="https://arxiv.org/abs/2610.11765">Thinking Inertia: LLMs Keep Thinking When Told Not To</a></p><p>This paper studies LLMs’ no-thinking behavior along two axes: how to measure no-thinking, and how no-thinking varies across tasks and models. It normalizes each response into a pre-answer trace and final answer, and evaluates it at three levels: (i) Empty-Thinking Rate for strict answer-only compliance; (ii) instruction-aware Question-Pre-answer Relevance for similarity between the question and pre-answer trace; and (iii) LLM-as-judge Explicit Inference Rate for visible explicit inference. These metrics distinguish answer-only output, relevant but non-inferential text, and explicit inference.</p><p>3. <a href="https://arxiv.org/abs/2610.20939">MAI-Gym: Synthesized, Verifiable GUI Environments at 20,000-Way Scale</a></p><p>MAI-Gym is an open platform that combines GUI agent evaluation, data collection, and online RL training across mobile, computer, and web environments. It supports ten existing benchmarks and scales to 20,000 parallel environments. It can also automatically generate mock apps and multi-app worlds with tasks whose outcomes are checked by execution. The resulting 9B agent, MAI-Gym-9B, scores 78.4% on MobileWorld and 76.2% on OSWorld-Verified, the best among open GUI agents under 10B parameters.</p><p>4. <a href="https://arxiv.org/abs/2610.22200">OpenGCUA: A Fully Open Stack for Generalist Computer Use</a></p><p>Computer-use agents are moving towards systems that complete tasks on users’ computers through visual and programmatic interfaces, but leading implementations like GPT-6 Astra remain closed. OpenGCUA is an ongoing open project to build this full stack, with this report as its first release. Its agent design leans on code execution so that intermediate results can shape later steps, and it shifts some decisions from fixed harness rules to the model itself. On OSWorld-v2, the full setup improves scores for all eight tested models, including GPT-6 Astra and Claude Opus 5, raising the mean partial score by 19.6 points and cutting steps per task by 53% on average compared with GUI-only execution.</p><h4>Quick Links</h4><p>1. <a href="https://mistral.ai/news/timestral/">Mistral AI releases Timestral</a>, its first family of time-series foundation models, available in four sizes (5M, 28M, 123M, and 209M parameters). The models produce probabilistic, multivariate forecasts and can factor in both past and known-future inputs, such as production schedules or weather forecasts, without needing to be trained on each new dataset. Mistral reports that the 209M model ranks first among pretrained, zero-shot models on GIFT-Eval, TIME, and fev-bench, beating the previous state of the art with two-thirds of its parameters. Code is on GitHub and weights are on Hugging Face under a modified MIT license, with fine-tuning available through Mistral Forge.</p><h4>Who’s Hiring in AI</h4><p><a href="https://jobs.towardsai.net/job/ibm-forward-deployed-engineer-ai-tzdn"><strong>Forward Deployed Engineer — AI @IBM (Remote/USA)</strong></a></p><p><a href="https://jobs.towardsai.net/job/kraken-ai-growth-ops-avgo"><strong>AI Growth Ops @Kraken (Remote)</strong></a></p><p><a href="https://jobs.towardsai.net/job/servicenow-inc-forward-deployed-solution-engineer-applied-ai-fde-5eld"><strong>Forward Deployed Solution Engineer @ServiceNow, Inc. (London, UK)</strong></a></p><p><a href="https://jobs.towardsai.net/job/cai-python-genai-developer-bmek"><strong>Python/GenAI Developer @CAI (Remote)</strong></a></p><p><a href="https://jobs.towardsai.net/job/onebrief-forward-deployed-engineer-fmrn"><strong>Forward Deployed Engineer @Onebrief (Remote/USA)</strong></a></p><p><a href="https://jobs.towardsai.net/job/crowdstrike-ai-research-scientist-agentic-systems-remote-to1d"><strong>AI Research Scientist, Agentic Systems @CrowdStrike (Remote)</strong></a></p><p><em>Interested in sharing a job opportunity here? Contact </em><a href="/service/mailto:sponsors@towardsai.net"><em>sponsors@towardsai.net</em></a><em>.</em></p><p><em>Think a friend would enjoy this too? </em><a href="https://newsletter.towardsai.net/"><em>Share the newsletter and let them join the conversation.</em></a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8d6d5edda588" width="1" height="1" alt=""><hr><p><a href="https://pub.towardsai.net/tai-227-agentic-development-moves-up-another-level-8d6d5edda588">TAI #227: Agentic Development Moves Up Another Level</a> was originally published in <a href="https://pub.towardsai.net/">Towards AI</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[LAI #147: Can AI Learn Cause and Effect?]]></title>
            <link>https://pub.towardsai.net/lai-147-can-ai-learn-cause-and-effect-2257415b3431?source=rss-aea8a19ea239------2</link>
            <guid isPermaLink="false">https://medium.com/p/2257415b3431</guid>
            <category><![CDATA[community]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[towards-ai]]></category>
            <dc:creator><![CDATA[Towards AI Editorial Team]]></dc:creator>
            <pubDate>Fri, 16 Oct 2026 15:01:02 GMT</pubDate>
            <atom:updated>2026-10-16T15:01:02.439Z</atom:updated>
            <content:encoded><![CDATA[<h4>Causal intelligence for robots, better human checkpoints, semantic cache failures, RAG vs OKF, and activation steering.</h4><p>Good morning, AI enthusiasts!</p><p>AI models are very good at learning what tends to happen next. But a robot has to choose actions that change the world around it. If the model learned a correlation rather than the underlying cause, even a small change in its environment can break what previously worked.</p><p>This week’s What’s AI looks at whether causal intelligence can help. I spoke with Prof. Biwei Huang about causal world models, how interaction can reveal relationships that passive observation misses, and what still needs to be tested before these ideas prove useful in closed-loop robotic systems.</p><ul><li>We also have some great technical reads this week. You’ll learn:</li><li>Where to place human checkpoints in agent workflows</li><li>What ToneBench found across 78 writing models</li><li>Why semantic caches need stricter reuse rules</li><li>When OKF can complement RAG for critical knowledge</li><li>How activation steering changes behavior without fine-tuning</li><li>What breaks when Elasticsearch search moves to Cosmos DB</li></ul><p>Plus, a community project that turns your learning materials into a concept graph and maps the prerequisites behind each idea.</p><p>Let’s get into it.</p><h4>What’s AI Weekly</h4><p>The industry is moving from AI that “talks” to AI that sees and acts, what NVIDIA calls physical AI: robots, self-driving cars, and other systems that have to perceive the real world and act in it. Once a system starts acting, prediction alone is not enough. It also needs some way to reason about how its actions will change what happens next.</p><p>This week in What’s AI, I spoke with Prof. Biwei Huang, who has spent more than a decade working on causality and now leads Aether AI. She argues that pattern recognition is not enough for robots. A model can predict what usually happens next without learning what actually caused it. Causal intelligence tries to model how an action changes the world, so a system can compare possible consequences, act, check what happened, and adapt when reality does not match its prediction.</p><p>We get into causal world models, an Aether AI example where raising a table by just 1 cm broke a vision-language-action model, how interaction can help separate causes from correlations, and how to test whether a model learned the mechanism, not just a shortcut.</p><p>The full interview also gets into computer-use agents, where every click changes the state the agent has to reason from next, and why Huang thinks causality could be useful there too. <a href="https://louisbouchard.substack.com/p/are-llms-enough-what-causal-intelligence">Read it here</a>.</p><h4>AI Tip of the Day</h4><p>Most human-in-the-loop systems ask someone to review the final output. That is often the worst point to catch a mistake. A long research report takes time to verify properly, and by then, the agent may already have searched, read, summarized, and written from the wrong starting point.</p><p>In any agent workflow, place human approval before the steps that are costly to undo.</p><p>For a research agent, that checkpoint is often the search plan. A weak query can still return relevant-looking sources, produce a coherent summary, and lead to a conclusion that doesn’t answer the original question.</p><p>In Nova, the deep research agent we built in <a href="https://towardsai.com/academy/agent-engineering/?utm_source=newsletter&amp;utm_medium=email&amp;utm_id=AItips">Agent Engineering</a>, each round proposes five search queries, each with a one-line reason. The agent can pause there so you can approve the set, remove queries, or replace one before any search runs.</p><p>The reasons speed up review because you can check what the agent intends to learn from each query, not just whether the wording looks plausible.</p><p>A useful rule for future projects: put checkpoints where an error is easy to catch and expensive to carry forward.</p><p><em>— Louis-François Bouchard, Towards AI Co-founder &amp; Head of Community</em></p><p><strong>I presented ToneBench, our new benchmark for AI writing at the AI Engineer event in New York.</strong></p><figure><a href="https://benchmark.towardsai.com/?utm_source=LAITsponsor&amp;utm_medium=email&amp;utm_id=tonebenchrelease"><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Cf8FhSZvQHucmp_XcbJGyQ.jpeg" /></a></figure><p>I designed it specifically to answer, <strong>can an AI model write in my voice well enough that I would actually use the draft?</strong></p><p>We took ten real video scripts and gave every model the same brief, research, voice guide, examples, and writing constraints. Each configuration writes every script five times because I don’t want to rank a model based on one lucky generation. Four judges then score the drafts blind across nine dimensions, including tone and voice, substance, structure, hooks, slop, and visual cues.</p><p>The current benchmark covers <strong>78 models, 127 configurations, 6,350 generated scripts, and 25,400 judge assessments</strong>.</p><p>You can also see some decisions I would make based on the results. Opus 5.5 at xhigh currently has the same rounded Writing Elo as max effort while costing about $1.00 versus $2.58 per script. GLM-5.3 Flash ranks 12th at roughly two cents per script. And our judge analysis found measurable lab preferences even with model identities hidden, which is one reason we use judges from four different companies.</p><p>The whole thing is public: leaderboard, methodology, reasoning-level comparisons, judge analysis, costs, and head-to-head model comparisons. If you build evals yourself, I have also documented the prompts, metrics, weights, judges, and limitations so you can see exactly how we approached it.</p><p><a href="https://benchmark.towardsai.com/?utm_source=LAITsponsor&amp;utm_medium=email&amp;utm_id=tonebenchrelease"><strong>Explore ToneBench →</strong></a></p><h3>Learn AI Together Community Section!</h3><h4>Featured Community post from the Discord</h4><p><a href="https://discord.com/channels/702624558536065165/983037843532308500/1558626950065422356">Longhl_104</a> built Concept Dangle, a learning platform that lets you paste your materials into a node and builds a concept graph so you can see how ideas connect. It identifies the prerequisite concepts the material assumes you already know, places them underneath the original idea, and lets you mark each one as Unknown, Learning, or Mastered. From there, it suggests what you are ready to study next, while keeping your notes, quizzes, and related concepts in the same graph. It also supports PDFs, Word files, and PowerPoint decks if you want to build a graph from longer material. <a href="https://conceptdangle.com/?ref=discord-learnai-oct11#how">Check it out here</a> and support a fellow community member. If you have any questions or feedback, <a href="https://discord.com/channels/702624558536065165/983037843532308500/1558626950065422356">share it in the thread</a>!</p><h4>Collaboration Opportunities</h4><p>The Learn AI Together Discord community is flooding with collaboration opportunities. If you are excited to dive into applied AI, want a study partner, or even want to find a partner for your passion project, <a href="https://discord.gg/rj6m9AF7eC">join the collaboration channel</a>! Keep an eye on this section, too — we share cool opportunities every week!</p><p>1. <a href="https://discord.com/channels/702624558536065165/784477688551178240/1558819355930664971">Themortal0266_39276</a> is transitioning from a non-CS background into AI and currently learning Python and CS fundamentals. If you are interested in learning together and are on the same track, <a href="https://discord.com/channels/702624558536065165/784477688551178240/1558819355930664971">connect with them in the thread</a>!</p><p>2. <a href="https://discord.com/channels/702624558536065165/784477688551178240/1559773423767392267">Gian_16992</a> is moving from backend orientation to AI engineering and is looking for people to discuss ideas, share resources, make projects, etc. If this sounds like a good next step for you, <a href="https://discord.com/channels/702624558536065165/784477688551178240/1559773423767392267">reach out to them in the thread</a>!</p><p>3. <a href="https://discord.com/channels/702624558536065165/1558445560610955296/1558445560610955296">Miyabi0193</a> is looking for team members for their ARC-AGI Project. The project is currently private, but if you want more information, <a href="https://discord.com/channels/702624558536065165/1558445560610955296/1558445560610955296">contact them in the thread</a>!</p><h4>Meme of the week!</h4><figure><a href="https://discord.com/channels/702624558536065165/830572933197201459/1559561131108999259"><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-i9-o313XUx3IV2aLBko4Q.png" /></a></figure><p>Shared by <a href="https://discord.com/channels/702624558536065165/830572933197201459/1559561131108999259">rolandodata</a></p><h3>TAI Curated Section</h3><h4>Article of the week</h4><p><a href="https://pub.towardsai.net/migrate-elasticsearch-to-azure-cosmos-db-full-text-search-c99146f36e26?sk=9a9c93fb1fa0de774529d8349401a3fa">Elasticsearch to Azure Cosmos DB: What Translates, What Doesn’t, and What It Costs</a> by<a href="https://blog.azinsider.net/"> Dave R</a></p><p>Moving search from Elasticsearch into Azure Cosmos DB can remove a separate search cluster and the pipeline that keeps it synchronized. The author maps one Elasticsearch mapping onto four Cosmos DB container policies, replaces field boosts with weighted Reciprocal Rank Fusion, and shows how the built-in English stopword list removes terms such as “new” and “used” from queries. The article also covers partitioning costs, unsupported features, and a six-phase migration plan.</p><h4>Our must-read articles</h4><p>1. <a href="https://pub.towardsai.net/now-amazon-showed-up-to-jev-75aaea859803?sk=d18bc792e3bb8cf18f5d7c4a385340fc">Now Amazon Showed up to Jev</a> by<a href="https://thegowtham.medium.com/"> Gowtham Boyina</a></p><p>Strands Decider is a 2-billion-parameter model that answers typed questions by returning probabilities in a single forward pass rather than generating text. The author analyzes its released logs and finds a roughly 97 ms latency floor plus a per-token cost, while the 231-task benchmark has too much uncertainty to reliably distinguish close scores. A base Qwen3.5–4B also outperformed the trained 2B model under the same one-pass setup. The article examines calibration, truncation, and a three-zone routing approach for deciding when to trust the model or escalate.</p><p>2. <a href="https://pub.towardsai.net/the-semantic-cache-reusing-answers-in-a-system-that-never-repeats-itself-5457ffeb01b2?sk=454e4cf5f3293edcf448e4ce4e3aaae4">The Semantic Cache: Reusing Answers in a System That Never Repeats Itself</a> by<a href="https://medium.com/@shrashtisinghal"> Shrashti Singhal</a></p><p>A semantic cache reuses an earlier answer when a new request is similar enough, which makes the similarity decision part of the system’s correctness. In the article’s composite support example, a 22% cache hit rate reduced LLM calls, but one match returned another customer’s refund details. The author covers five ways similarity can fail and recommends scope keys, rules for requests that should never be cached, class-specific thresholds, verified writes, and provenance-based invalidation.</p><p>3. <a href="https://pub.towardsai.net/rag-vs-okf-stop-choosing-one-why-ai-agents-need-both-72caf114acef?sk=8c865bd06e44c12c32142f3d597d67d3">RAG vs OKF: Stop Choosing One. Why AI Agents Need Both</a> by<a href="https://medium.com/@rohanmistry231"> Rohan Mistry</a></p><p>Google Cloud’s Open Knowledge Format stores curated knowledge as Markdown files that agents can read directly. The author contrasts this with RAG: OKF works well for fixed, authoritative information such as policies, metrics, and schemas, while RAG can search much larger collections of less structured information. The proposed architecture routes a small set of critical knowledge to OKF and leaves the broader corpus in RAG, with a five-step guide for building the first knowledge bundle.</p><p>4. <a href="https://pub.towardsai.net/coding-an-agent-steering-a-local-model-14872070ef56?sk=ac0824d89fc991a04867280fde4e3633">Coding an Agent: Steering a Local Model</a> By<a href="https://enzolombardi.net/"> Enzo Lombardi</a></p><p>DS4 supports activation steering, which changes a model’s activations at runtime using one vector per layer, without fine-tuning or modifying the weights. The author builds a verbosity vector and tests how system prompts and thinking modes affect it, finding that prompts change the baseline while reasoning reduces the steering effect. He then compares a refusal steering vector with abliterated weights to show how the two methods alter behavior differently.</p><p>If you are interested in publishing with Towards AI, <a href="https://contribute.towardsai.net/">check our guidelines and sign up</a>. We will publish your work to our network if it meets our editorial policies and standards.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2257415b3431" width="1" height="1" alt=""><hr><p><a href="https://pub.towardsai.net/lai-147-can-ai-learn-cause-and-effect-2257415b3431">LAI #147: Can AI Learn Cause and Effect?</a> was originally published in <a href="https://pub.towardsai.net/">Towards AI</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[TAI #226: The AI Model Race Needs Better Tests]]></title>
            <link>https://pub.towardsai.net/tai-226-the-ai-model-race-needs-better-tests-76ce622ebf67?source=rss-aea8a19ea239------2</link>
            <guid isPermaLink="false">https://medium.com/p/76ce622ebf67</guid>
            <category><![CDATA[news]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[towards-ai]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <dc:creator><![CDATA[Towards AI Editorial Team]]></dc:creator>
            <pubDate>Tue, 13 Oct 2026 16:46:01 GMT</pubDate>
            <atom:updated>2026-10-13T16:46:01.743Z</atom:updated>
            <content:encoded><![CDATA[<h4>Also, Meta’s Muse Wonder, Claude Haiku 5.5, OpenAI’s Ultrafast tier, Mistral Large 4, Reflection Beam, and new rivals to Jev.</h4><h3>What happened this week in AI by Louie</h3><p>This week brought another strong set of model releases, with Meta moving ahead of OpenAI on several benchmarks. At maximum reasoning effort, Muse Wonder Full scores 56 on Artificial Analysis’s Intelligence Index, ahead of GPT-6 Astra at 53 and GPT-6.1 Sol at 52, although Opus 5.5, with fallback enabled, still leads at 58. Meta also reports strong results on longer coding projects. I would not read the 56 as Meta’s API model overtaking Astra, though. That score belongs to Wonder Full, which is only available through a Muse Code subscription; the standard Wonder model available through the API scores 52.</p><p>I am more interested in Meta’s emphasis on memory and persistence than a few leaderboard points. Its Active Memory system is designed to retrieve relevant context across conversations, and Active Focus is meant to keep an agent working through blockers and changes of direction. Those are key capabilities a personal agent needs when it has to work across several days. Meta is now a serious contender for demanding agent work, although I still want to see how well those memory and persistence claims survive our messy real projects.</p><p>We also got a long-overdue Haiku 5.5 update from Anthropic, an Ultrafast tier that OpenAI says generates GPT-6.1 Sol tokens up to eight times faster, and strong new Western models from Mistral and Reflection with open weights on the way. Mistral Large 4 is in API preview, and Beam is in selected early access, with both companies promising weights later this month. OpenAI’s Decisions API and Microsoft-Decision-1 also look aimed at the demand Jev exposed for fast, inexpensive model judgments in routing, classification, and scoring. There is plenty more on these releases in the news section below.</p><p>This week’s results are also a good reminder not to overread small benchmark gaps. Wonder Full at 56, Astra at 53, and Sol 6.1 at 52 look very close. In real work, the difference can be much larger. Larger frontier models are often better at handling unusual documents, conflicting instructions, incomplete tool output, and changing requirements without needing as much intervention. Averaging those cases into one score makes it easy to miss the failures that consume most of your time.</p><p>Part of this problem is benchmaxxing: concentrating training on the capabilities that raise benchmark scores while leaving a much more jagged level of intelligence elsewhere. At the bluntest end, that means training on the exact questions used in an evaluation. But I suspect the larger effect comes from building similar datasets and reinforcement-learning environments, repeatedly rewarding the model for the kinds of problems the leaderboard measures.</p><p>But benchmaxxing also improves real capabilities. Once we find a useful way to measure something, we give researchers a clearer target and a way to tell whether training helped. The problem is allowing that target to become our whole definition of intelligence. We need to keep expanding and refreshing our benchmarks as models improve. Running the same tests we ran two months ago helps track progress, but it’s insufficient for finding where the frontier has moved. I want both continuity and new tests that expose weaknesses the last training run had little reason to fix.</p><p>In mathematics and science, the next test may be the research itself. Instead of asking whether a model can solve a fixed problem, we can ask whether it can produce a useful result that was not already known. OpenAI’s recent release of hundreds of mathematical manuscripts shows both how promising and how difficult that is to evaluate. The collection includes proposed results at different stages of verification, with revisions and withdrawals already recorded. A proof that checks, a genuinely new theorem, and an important mathematical advance are three different achievements. Even so, testing models against unresolved research questions is a much more ambitious exercise than measuring another gain on a fixed exam.</p><p>Progress is less even in fields where quality is harder to verify. Writing is one of them. A paragraph can be grammatically clean, factually correct, and still contain nothing worth reading. A model can remove obvious slop words while preserving the generic argument, repetitive structure, and lack of a real point of view. Testing whether the text sounds fluent misses much of what makes it useful.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1yWNppoQOe8HOAjt-F5a0Q.png" /></figure><p>That is why we built ToneBench, led by my co-founder Louis-Francois Bouchard. We built it around a task we know extremely well: writing complete video scripts in Louis’ voice from a brief, research, examples, and detailed instructions. The current board covers 78 models across 127 configurations, with reasoning levels counted separately. Each configuration writes without tools, producing five drafts for each of ten real script tasks, giving us 6,350 drafts and 25,400 judge assessments. The tests cover sustained writing across a whole piece, where a strong opening can give way to repetition or a lost argument several paragraphs later.</p><p>The results are already useful for deciding which model and reasoning level to use. Opus 5.5 leads overall, but xhigh and maximum effort have the same rounded Writing Elo and overlapping uncertainty ranges. At API list prices, the average script costs about $1.00 at xhigh versus $2.58 at maximum effort, so I would use xhigh by default for this kind of writing. GLM-5.3 Flash at max ranks twelfth among the 78 models, each ranked at its best setting, at about 2.2 cents per script. At that price, I would test it for higher-volume work, but the generation cost is only useful if the drafts do not require substantially more editing. Measuring that human editing time is one of the next additions we want to make. For a consistent comparison, these costs include generation and reasoning at API list prices, even where the benchmark itself ran through subscriptions.</p><p>ToneBench is still a narrow benchmark. It tests one writer’s voice across ten tasks, so it cannot tell us which model is universally best at writing. Each draft is scored by four automated judges from Anthropic, OpenAI, Z.ai, and TypeSafe, with the model identity hidden. Those judges still bring their own preferences even when the model names are hidden. In our analysis, both the OpenAI and Anthropic judges rate outputs from their own labs more favorably relative to the other judges. Using several judge families helps reduce the influence of any one model’s preferences, but it does not remove the bias. Next, we’ll add more human ratings, and we are still preparing the public page for that.</p><p>Between ToneBench and the evaluation pipelines our deployment team builds for clients, we have been developing broader expertise in benchmark design. We plan to scale this in three directions:</p><ul><li><strong>Broader technical writing:</strong> expand ToneBench with new tasks and human feedback from contributors drawn from the roughly 5,000 writers who have published technical AI articles with Towards AI.</li><li><strong>Business writing:</strong> evaluate client emails, executive summaries, proposals, and investment memos, where accuracy, selection, and judgment need to survive the move into a different professional voice.</li><li><strong>Workflows and agents:</strong> build tasks modeled on work we deliver for private-equity-backed companies, including complex multistage video analysis, tender screening, responses to requests for proposals, and technical engineering image analysis. We want to measure output quality alongside human review time and cost, so the results can support credible return-on-investment estimates.</li></ul><p><a href="https://x.com/Whats_AI">Please reach out</a> if you would like to partner with us on continuing to develop Tonebench or building these future benchmarks.</p><h4>Why should you care?</h4><p>I would like AI labs to work much more closely with the people who know why an output that looks good still fails in practice. An experienced editor can see when a rewrite removes the detail that made the piece worth reading. An engineer can spot the unusual drawing that breaks an otherwise convincing image-analysis system. An investment professional can tell when a summary is accurate but too generic to help with an investment decision. That expertise becomes useful for training once we turn it into examples, criteria, and repeatable tests.</p><p>Defining those criteria requires just as much care. Reward polish too heavily, and the model may remove awkward but important details. Reward brevity without enough context, and it may drop conditions the reader needs. In writing, an unpopular opinion, an exception, or an unresolved question may be the most useful part of the piece. An evaluation must leave room for those choices rather than rewarding whatever automated judges find consistently clean and agreeable.</p><p>This gives practitioners a concrete way to influence what improves next. Keep examples of work that needed substantial correction and record why the correction was necessary. Include the time an expert spent getting it right. Those records expose capabilities that a general leaderboard misses, and give us something better to train and test against, and ultimately build solutions to in our systems.</p><p><em>— </em><a href="http://www.linkedin.com/in/louie-peters"><em>Louie Peters — Towards AI Co-founder and CEO</em></a></p><p><strong>Today at AI Engineer in New York, we’re presenting ToneBench, our new benchmark for AI writing.</strong></p><figure><a href="https://benchmark.towardsai.com/?utm_source=TAIsponsor&amp;utm_medium=email&amp;utm_id=tonebenchrelease"><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Ywo5C044IHyYXJPetW_58w.jpeg" /></a></figure><p>We tested 78 models across 127 configurations on ten real long-form script tasks. Each model received the same brief, research, voice guide, and constraints, then wrote every script five times. Four blind judges scored each draft across nine dimensions, giving us 6,350 scripts and 25,400 assessments.</p><p>The full leaderboard, methodology, judge analysis, cost comparisons, and model-vs-model pages are now public.</p><p><a href="https://benchmark.towardsai.com/?utm_source=TAIsponsor&amp;utm_medium=email&amp;utm_id=tonebenchrelease"><strong>Explore ToneBench</strong></a></p><h4>Hottest News</h4><p>1. <a href="https://developers.openai.com/api/docs/guides/ultrafast-mode">OpenAI Adds an 8x-Faster Ultrafast Tier for GPT-6.1 Sol</a></p><p>OpenAI added an Ultrafast API tier for GPT-6.1 Sol (and GPT-6 Astra), claiming speeds up to 8x faster than Sol Standard for latency-sensitive workloads. It uses the same model and capabilities but allocates faster inference through service_tier: “ultrafast”; OpenAI recommends WebSockets for agents making frequent tool calls so network overhead does not erase the latency gains. Ultrafast is available to all API users with separate rate limits and supports global processing plus US and EU data residency (Only US and global for Astra). In Ultrafast, Sol costs $12/M input and $60/M output tokens (six times the standard price), and Astra costs $60/M and $300/M for prompts up to 272K tokens.</p><p>2. <a href="https://www.anthropic.com/claude-haiku-5-5">Anthropic Releases Claude Haiku 5.5</a></p><p>Anthropic released Claude Haiku 5.5 for high-volume and cost-sensitive tasks. In the release notes, Anthropic also said it pairs well with Opus 5.5 and Sonnet 5.5 as a subagent on coding work. It is the first Haiku model with adjustable reasoning effort, and it substantially improves over Haiku 4.5, scoring 72.4% vs. 15.7% on the OSWorld 2.1 (offline subset) and 45.9% vs. 10.2% on Humanity’s Last Exam (without tools) in Anthropic’s evaluations. For prompts up to 100K tokens, pricing drops to $0.10/M input and $0.50/M output, 90% below Haiku 4.5; prompts above 100K cost $0.50/$2.50. Haiku 5.5 is available through Anthropic, AWS, Google Cloud, and Microsoft Azure, although Anthropic still recommends Sonnet or Opus for more complex agentic coding. Anthropic is also halving Claude Sonnet 5.5 cache-read pricing, which it says reduces the cost of most agentic workloads by around 20%, and introducing monthly Claude API credits for Max and Team subscribers.</p><p>3. <a href="https://mistral.ai/news/mistral-large-4/">Mistral Launches a Preview of Mistral Large 4</a></p><p>Mistral launched Mistral Large 4, a natively multimodal Mixture-of-Experts model with 1.05T total parameters and 52B active parameters. Mistral trained ML4 from scratch on 3,800 NVIDIA Grace Blackwell GPUs in its own European data centers, using multilingual data spanning more than 160 languages. The company reports 61.7% on DeepSWE v1.1, 28.3% on Terminal-Bench 4, and 42% on Dense 200. The model is available in public API preview through Mistral Studio at promotional pricing of $0.68/M input, $0.07/M cached input, and $2.09/M output tokens. Mistral plans to release the weights by the end of October after additional red-teaming.</p><p>4. <a href="https://openai.com/index/sharing-ai-progress-in-mathematics/">OpenAI Publishes 722 Math Manuscripts From an Unreleased Model</a></p><p>OpenAI published 722 mathematical manuscripts across 372 result families produced largely by an unreleased internal frontier model while testing it on open research problems. The model attempted roughly 4,000 problems, with each retained result using the equivalent of about three hours of ChatGPT Pro thinking compute on average. The repository includes papers, source files, 10 abridged reasoning summaries, and Lean formalizations for many of the proofs. OpenAI cautions that the results are at different stages of verification and that some manuscripts without formal proofs may contain errors, with corrections and revisions to be tracked publicly.</p><p>5. <a href="https://developers.openai.com/api/docs/guides/decisions">OpenAI Releases the Decisions API in Public Beta</a></p><p>OpenAI released the Decisions API, a new endpoint that returns structured predictions instead of generated text. It accepts text and images and can return boolean predicates, fixed-set choices, or rubric scores alongside probabilities. OpenAI says responses are about 10× faster than its Responses API, although the public beta currently supports only GPT-6 Luna. Pricing is $0.10/M input tokens, with no output, cache-read, or cache-write charges. OpenAI says general availability is planned in the coming weeks.</p><p>6. <a href="https://commandline.microsoft.com/microsoft-decision-1-model-foundry/">Microsoft AI Releases Microsoft-Decision-1</a></p><p>Microsoft released Microsoft-Decision-1, a model built for structured tasks such as routing, classification, prioritization, verification, and workflow control. Microsoft reports that it achieved the highest accuracy in its 36-benchmark evaluation spanning nearly 150,000 questions, while its P50 latency was roughly 35× faster than GPT-6 Sol and 2.5× faster than H2O-Lightning-4B v1.1. The API also returns probabilities intended for threshold-based automation and human-review decisions. Microsoft-Decision-1 is available through Microsoft Foundry and OpenRouter at $0.042/M input tokens with no output-token charge.</p><p>7. <a href="https://cloud.google.com/blog/products/ai-machine-learning/welcome-to-gemini-at-work-2026">Google Cloud Launches Gemini Agent</a></p><p>Google Cloud introduced Gemini as a single enterprise agent that can answer questions, create content, write and run code, and complete longer tasks across workplace systems. It runs persistently in the cloud, can create temporary subagents, use reusable skills and tools, maintain several forms of memory, and connect to systems, including Workspace, Microsoft 365, Slack, Salesforce, ServiceNow, Git, Jira, and MCP servers. The agent is separate from the underlying model: Google says it can dynamically route work across Gemini models and Anthropic’s Claude models, with more model providers planned. Google is also bringing the same agent into Gmail, Drive, Docs, Sheets, Slides, Chat, and Calendar and supports coworker agents with their own identity, email, storage, and permissions.</p><p>8. <a href="https://reflection.ai/blog/introducing-beam">Reflection AI Introduces Beam</a></p><p>Reflection AI introduced Beam, a sparse Mixture-of-Experts model with 501B total parameters and 23B active parameters aimed at coding, reasoning, and agentic workloads. Reflection pretrained it on 23.8T tokens and then ran more than 100 million RL rollouts on 10,500 NVIDIA GB300 GPUs over four weeks. The company reports scores of 44.4% on DeepSWE v1.1, 77.2% on SWE-Bench Pro v2-Hard, and 80.1% on Terminal-Bench 2.1. Beam is currently undergoing final red-teaming and is available only to selected early-access users. Reflection plans to release the weights, technical report, model card, evaluation tools, and fine-tuning stack later this month under Apache 2.0.</p><h4>AI Tip of the Day</h4><p>Most human-in-the-loop systems ask someone to review the final output. That is often the worst point to catch a mistake. A long research report takes time to verify properly, and by then the agent may already have searched, read, summarized, and written from the wrong starting point.</p><p>In any agent workflow, place human approval before the steps that are costly to undo.</p><p>For a research agent, that checkpoint is often the search plan. A weak query can still return relevant-looking sources, produce a coherent summary, and lead to a conclusion that doesn’t answer the original question.</p><p>In Nova, the deep research agent we build in <a href="https://towardsai.com/academy/agent-engineering/?utm_source=newsletter&amp;utm_medium=email&amp;utm_id=AItips">Agent Engineering</a>, each round proposes five search queries, each with a one-line reason. The agent can pause there so you can approve the set, remove queries, or replace one before any search runs.</p><p>The reasons speed up review because you can check what the agent intends to learn from each query, not just whether the wording looks plausible.</p><p>A useful rule for future projects: put checkpoints where an error is easy to catch and expensive to carry forward.</p><h4>Five 5-minute reads/videos to keep you learning</h4><p>1. <a href="https://pub.towardsai.net/rag-vs-long-context-in-2026-when-retrieval-still-wins-e6ebad9a4096?sharedUserId=tai-tech">RAG vs. Long Context in 2026: When Retrieval Still Wins</a></p><p>The article examines whether million-token context windows have reduced the need for RAG. It uses RULER, Chroma’s Context Rot research, and Anthropic’s contextual retrieval work to explain how recall, cost, and latency change as prompts grow. It then covers cases where full context works better and provides a decision framework based on knowledge-base size, update frequency, call volume, and whether a task needs targeted retrieval or whole-document reasoning.</p><p>2. <a href="https://pub.towardsai.net/a-week-inside-jev-what-a-non-generative-decision-model-is-actually-good-for-361edd1284e9?sharedUserId=tai-tech">A Week Inside Jev: What a Non-Generative Decision Model Is Actually Good For</a></p><p>The author explores what TypeSafe’s Jev can actually do compared with the “200× faster, 400× cheaper” framing around it. He explains Jev’s Choice, Score, and Noul question types, then builds a dry-run Claude Code context pruner to show how a probabilistic decision model can fit into an existing workflow. The article also examines Jev’s limitations, available calibration evidence, and potential uses such as PR triage, support routing, and on-call paging.</p><p>3. <a href="https://pub.towardsai.net/vector-search-is-not-enough-the-blueprint-of-graphrag-674c216fa132?sharedUserId=tai-tech">Vector Search Is Not Enough: The Blueprint of GraphRAG</a></p><p>The article explains how GraphRAG uses knowledge graphs to retrieve relationships that ordinary chunk-based vector search can miss. It first covers how knowledge graphs are structured and built, then walks through retrieval approaches, including Cypher templates, dynamic Cypher generation, Text2Cypher, Graph-CoT, lexical graphs with vector search, and Microsoft’s community-based GraphRAG and DRIFT search. It closes with the cost, latency, and implementation trade-offs that affect which approach fits a given use case.</p><p>4. <a href="https://pub.towardsai.net/jev-and-rlcd-architecture-open-source-models-and-practical-agent-use-cases-bd8ab990fc44?sharedUserId=tai-tech">Jev and RLCD: Architecture, Open-Source Models, and Practical Agent Use Cases</a></p><p>The article explains Jev’s typed decision interface and how RLCD trains models to produce calibrated probability distributions. It separates Jev’s documented API from its undisclosed internal architecture, then examines two independent implementations, eve-rlcd and OpenJev-RLCD. From there, it shows how decision models can handle routing, RAG evidence checks, tool selection, and other bounded choices inside agent workflows, with local examples and guidance for evaluating calibration and end-to-end performance.</p><p>5. <a href="https://pub.towardsai.net/muon-the-orthogonalized-optimizer-explained-through-equations-intuition-code-and-results-05e7dc7eafed?sharedUserId=tai-tech">Muon: The Orthogonalized Optimizer Explained Through Equations, Intuition, Code and Infographic</a></p><p>The article explains Muon step by step, starting with how optimizers update weight matrices and what Adam’s per-coordinate scaling misses. It then derives Muon’s update rule and explains why orthogonalization gives the steepest-descent step under the spectral norm. From there, it covers how Newton–Schulz makes the operation practical without an SVD, where Muon fits into a Transformer, and how to implement it alongside AdamW. It closes with limitations and considerations around pre-training, distributed training, and using Muon outside the settings where it has been best tested.</p><h4>Repositories &amp; Tools</h4><p>1. <a href="https://github.com/nano-muse/nanoMuse">nanoMuse</a> is a personal agent that runs across your phone, computer, and browser, completing tasks and asking before anything irreversible.</p><p>2. <a href="https://github.com/morluto/rea">REA</a> is an MCP server and CLI that lets coding agents reverse engineer binaries, apps, and websites locally.</p><p>3. <a href="https://github.com/google-research/rrsi">RRSI</a> is a framework for evolving agent harnesses while regularizing the search to avoid overfitting.</p><p>4. <a href="https://github.com/NVlabs/LongLive">LongLive</a> is NVIDIA’s NVFP4 parallel infrastructure for training and running long video generation models.</p><p>5. <a href="https://github.com/Dreamer-Toby/STEPQuant">STEPQuant</a> is a quantization framework that compresses the recurrent states of Delta-rule linear attention models, closely matching full-precision accuracy at 6 bits.</p><h4>Top Papers of The Week</h4><p>1. <a href="https://arxiv.org/abs/2610.11794">Memento 3: Rulebook Harness Reports a Perfect ARC-AGI-3 Score</a></p><p>Agents in unfamiliar environments need to figure out how the world works, but a few observations can fit several different explanations. Memento 3 lets a frozen LLM agent keep a written rulebook of its current guesses about how the environment works, then turn that rulebook into code it uses to predict and plan. As it plays, the agent compares predictions with what actually happens and revises both the rules and the code. The agent updates its code only if the LLM judges it faithful to the rulebook and it exactly reproduces past observations. On ARC-AGI-3, the agent clears every level of all 25 public games with a mean RHAE score of 100.0 while using 44% of the human action count.</p><p>2. <a href="https://arxiv.org/abs/2609.32423">PluginRSI: Recursive Harness Improvement With Reusable Plugins</a></p><p>Current harness optimization methods search over whole programs, which makes it hard to isolate and reuse the parts that actually help. PluginRSI breaks a harness into small plugins, improves each one separately, stores them in a shared library, and recombines them into new harnesses each round. It outperforms existing harness optimization methods on software engineering, command-line, and question-answering tasks. The resulting harnesses keep their advantage when moved to other models without retuning, and the plugin library speeds up optimization on unseen tasks.</p><p>3. <a href="https://arxiv.org/abs/2609.36484">The Teacher Is a Direction, Not a Destination: Extrapolating RL-Induced Representation Residuals in On-Policy Distillation</a></p><p>On-policy distillation trains a student to imitate a teacher on the student’s own outputs, and some variants try to push the student past the teacher. Doing this through the model’s output probabilities is noisy and destabilizes training. The authors observe that RL training shifts a model’s internal representations in a measurable direction at every layer. Their method, RIDE, measures how the RL-trained teacher differs from its pre-RL version and pushes the student’s internal states slightly beyond the teacher in that same direction. Across four base and RL-teacher pairs, RIDE approaches or exceeds the RL-trained teacher on every pair, and consistently beats output-space extrapolation.</p><p>4. <a href="https://arxiv.org/abs/2610.12374">AgentGarten: Code Worlds for Evolving Agents</a></p><p>Agents learn from the environments they practice in, but building environments that are both rule-consistent and visually realistic remains hard. AgentGarten pairs simulators and game engines, which handle world state and rules, with a shared neural renderer that produces realistic visuals in real time. The renderer is adapted from a pretrained video model and trained with a new method called Adversarial Forcing. Agents explore these worlds and write what they learn into playbooks that later agents inherit and refine. The authors report agents learning in just 4 rounds, compared with millions for a conventional reinforcement learning approach.</p><h4>Quick Links</h4><p>1. <a href="https://x.com/OrcaRouter/status/2108802238567354864">OrcaRouter releases OrcaCyber Zero 1.5</a>, the successor to OrcaCyber Zero 1.0, post-trained for authorized vulnerability reproduction, exploit development, and penetration testing. It ships with a 1M-token context window, 128K max output, native function calling, and structured outputs. OrcaRouter reports 100% on Cybench (39 of 40 tasks), 95.8% on a 24-task CVE-Bench subset, 93.9% on HumanEval+, and 76.5% on SWE-bench Pro V2. Access is gated to OrcaRouter’s Security Research tier via an OpenAI-compatible API at $3.00/$7.50 per million input/output tokens, with no weights released.</p><h4>Who’s Hiring in AI</h4><p><a href="https://jobs.towardsai.net/job/coinbase-forward-deployed-engineer-intern-hr-technology-w91v"><strong>Forward Deployed Engineer Intern @Coinbase (New York, NY, USA)</strong></a></p><p><a href="https://jobs.towardsai.net/job/ibm-cloud-and-ai-engineer-uxh3"><strong>Cloud &amp; AI Engineer @IBM (Bangalore, India)</strong></a></p><p><a href="https://jobs.towardsai.net/job/l3harris-specialist-artificial-intelligence-engineer-fhvn"><strong>Artificial Intelligence Engineer @L3Harris (Melbourne, FL, USA)</strong></a></p><p><a href="http://jobs.towardsai.net/job/caci-international-ai-prompt-engineer-esws"><strong>AI Prompt Engineer @CACI International (Ashburn, VA, USA)</strong></a></p><p><a href="https://jobs.towardsai.net/job/capital-one-ai-engineer-5-fm-hosting-llm-inference-d6eu"><strong>AI Engineer 5 @Capital One (Multiple US Locations)</strong></a></p><p><a href="https://jobs.towardsai.net/job/netapp-software-engineer-jaly"><strong>Software Engineer @NetApp (Bellevue, WA, USA)</strong></a></p><p><em>Interested in sharing a job opportunity here? Contact </em><a href="/service/mailto:sponsors@towardsai.net"><em>sponsors@towardsai.net</em></a><em>.</em></p><p><em>Think a friend would enjoy this too? </em><a href="https://newsletter.towardsai.net/"><em>Share the newsletter and let them join the conversation.</em></a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=76ce622ebf67" width="1" height="1" alt=""><hr><p><a href="https://pub.towardsai.net/tai-226-the-ai-model-race-needs-better-tests-76ce622ebf67">TAI #226: The AI Model Race Needs Better Tests</a> was originally published in <a href="https://pub.towardsai.net/">Towards AI</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[LAI #146: Can You Trust Your AI’s Memory?]]></title>
            <link>https://pub.towardsai.net/lai-146-can-you-trust-your-ais-memory-f30f47fd9bd1?source=rss-aea8a19ea239------2</link>
            <guid isPermaLink="false">https://medium.com/p/f30f47fd9bd1</guid>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[community]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[towards-ai]]></category>
            <category><![CDATA[ai]]></category>
            <dc:creator><![CDATA[Towards AI Editorial Team]]></dc:creator>
            <pubDate>Thu, 08 Oct 2026 15:01:02 GMT</pubDate>
            <atom:updated>2026-10-08T15:01:02.370Z</atom:updated>
            <content:encoded><![CDATA[<h4>Plus, 181 AI model tests, hidden states, autonomous agents, knowledge graph reliability, and model stability.</h4><p>Good morning, AI enthusiasts!</p><p>Giving agents more memory is relatively straightforward. Keeping that information accurate, up to date, and under the user’s control is much harder. This week’s issue looks at that challenge, along with several interesting experiments on how we build, evaluate, and understand AI systems. You’ll learn:</p><ul><li>Why persistent AI agents need more than similarity search to manage memory</li><li>How hidden states change as a model interprets the same word in different contexts</li><li>How Microsoft Foundry lets agents run on schedules and respond to external events</li><li>Why knowledge graphs built from unrelated subjects can end up looking remarkably similar</li><li>What loss landscapes and stability tell us about how models learn and adapt</li></ul><p>We also have my latest experiment comparing 181 AI model configurations across 9,050 writing drafts, including some surprising results on model quality, cost, and evaluation bias.</p><p>Let’s get into it.</p><h4>What’s AI Weekly</h4><p>This week in What’s AI, I’m sharing the results of testing 181 AI model configurations on my own YouTube scripts. I generated 9,050 drafts across 10 writing tasks to see which models could best reproduce my writing style, how much quality improves with higher spending, and whether AI judges can reliably tell the difference.</p><p>Claude Opus 5.5 ranked first overall, but GLM-5.3 Flash scored less than 3% lower at roughly 190 times lower cost. GPT-6 Astra was particularly good at avoiding common AI writing patterns, although it was less successful at matching my voice. I also found that the choice of judge can significantly affect the rankings.</p><p>I share six findings from these experiments, including the trade-offs between writing quality and cost, when generating multiple drafts makes sense, and how to build a similar evaluation for your own writing. <a href="https://louisbouchard.substack.com/p/i-tested-181-ai-models-on-my-own">Read the full article here</a>.</p><h4>AI Tip of the Day</h4><p>This week, Omar spent 45 minutes on one of our<a href="https://towardsai.com/academy/mentorship/?utm_source=newsletter&amp;utm_medium=email&amp;utm_id=AItips"> Mentorship</a> live calls, walking through how we built our AI tutor. It is a useful example if you are deciding whether your own system needs RAG and where to start.</p><p>As he mentioned on the call, start with the questions the system needs external knowledge to answer. Collect a small set of real queries and, for each one, record:</p><ul><li>what information the answer requires</li><li>whether that information already fits reliably in the model’s context</li><li>whether retrieval returns the correct source in the top results</li><li>whether the final answer actually uses that source correctly</li></ul><p>Start by checking whether the system can actually surface the information needed for those questions. If it cannot, retrieval is the first problem to solve. If it can, there is no point changing the retriever yet.</p><p>With our tutor, that meant testing real student questions against the retrieval setup before adding more infrastructure. We only added complexity where the simpler setup was actually failing.</p><p>So if you are deciding whether to build RAG, start with a small evaluation set and find the exact point where your current system loses access to the information it needs. The architecture should follow that failure, not come before it.</p><p>The 45-minute tutor walkthrough was part of our weekly<a href="https://towardsai.com/academy/mentorship/?utm_source=newsletter&amp;utm_medium=email&amp;utm_id=AItips"> Towards AI Mentorship</a> live call, where we work through these kinds of implementation decisions using both our systems and what members are building.</p><p><em>A quick update on the book before I wrap up. We are deep into the final rounds of AI Engineering for Production now: editing chapters, tightening examples, checking the technical details, and making sure the book reflects how we actually build with these systems today.</em></p><p><em>Over the next few weeks, we will start sharing more from behind the scenes: parts of the book, ideas that changed substantially while we were writing, lessons from building the examples, and some material that will not make it into the final manuscript. If you want to follow the book as it comes together and get those updates first, </em><a href="https://www.louisbouchard.ai/book/?utm_source=news-tai&amp;utm_medium=email&amp;utm_campaign=book"><em>you can sign up here</em></a><em>.</em></p><p><em>And if there is something you particularly want us to cover, reply to one of those emails and tell me. We are far enough along that the structure is set, but there is still room to make sure we answer the questions practitioners actually have.</em></p><p><em>— Louis-François Bouchard, Towards AI Co-founder &amp; Head of Community</em></p><h3>Learn AI Together Community Section!</h3><h4>Featured Community post from the Discord</h4><p><a href="https://discord.com/channels/702624558536065165/983037843532308500/1557526037108105306">Agent_hellboy</a> built Cully, an intelligent workspace that observes your coding workflow, remembers what matters, and helps you steer Claude Code, Codex, Cursor, or other coding agents across sessions. It also has a live Agent Health bar that shows the agent, project, branch, session time, context pressure, loop warnings, and unchecked edits; every number is measured, and anything unknown is shown as missing rather than guessed. <a href="https://github.com/mcp-runtime/cully">Check it out here</a> and support a fellow community member. If you have any questions or feedback, <a href="https://discord.com/channels/702624558536065165/983037843532308500/1557526037108105306">share them in the thread</a>!</p><h4>Collaboration Opportunities</h4><p>The Learn AI Together Discord community is flooding with collaboration opportunities. If you are excited to dive into applied AI, want a study partner, or even want to find a partner for your passion project, <a href="https://discord.gg/rj6m9AF7eC">join the collaboration channel</a>! Keep an eye on this section, too — we share cool opportunities every week!</p><p>1. <a href="https://discord.com/channels/702624558536065165/784477688551178240/1555205787909881978">Inc0gnit08125</a> is looking for an accountability partner, so everyone can share what they have achieved. The main goal is to speed up learning and become job-ready in the next 6 months. If you are working towards the same thing, <a href="https://discord.com/channels/702624558536065165/784477688551178240/1555205787909881978">connect with them in the thread</a>!</p><p>2. <a href="https://discord.com/channels/702624558536065165/784477688551178240/1557707157565349962">.d0n0x</a> is looking for groups/people to learn fine-tuning, post-training, theory, and other fundamentals for experimentation, and also work on practical workflows/agents. If this sounds interesting to you, <a href="https://discord.com/channels/702624558536065165/784477688551178240/1557707157565349962">reach out to them in the thread</a>!</p><p>3. <a href="https://discord.com/channels/702624558536065165/998978160605540454/1555491383626698873">Divyanshijoshi</a> is participating in the AI Builder Cup Competition and looking for 1 teammate with experience in full-stack/GCP dev (React, Cloud Run, Firebase). If you want to join, <a href="https://discord.com/channels/702624558536065165/998978160605540454/1555491383626698873">contact her in the thread</a>!</p><h4>Meme of the week!</h4><figure><a href="https://discord.com/channels/702624558536065165/830572933197201459/1556140816328687646"><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*hSiqFRc6JDOu-aolzuAmyg.jpeg" /></a></figure><p>Meme shared by <a href="https://discord.com/channels/702624558536065165/830572933197201459/1556140816328687646">drdub_</a></p><h3>TAI Curated Section</h3><h4>Article of the week</h4><p><a href="https://pub.towardsai.net/from-token-to-trajectory-8300f311f6a2?sharedUserId=tai-tech">From Token to Trajectory</a> by<a href="https://liankimco.medium.com/"> Lian Kim</a></p><p>The same token starts with the same embedding, but its hidden states change as the model processes context. The author traces the word “light” through six weight-related and color-related contexts using Pythia-160M. The representations begin separating at state 3, but the distinction weakens at the final state under cosine similarity, even as raw Euclidean distance increases because of larger vector norms. A second experiment compares JEV, Claude, and GPT on banking-intent classification, measuring both decision accuracy and whether they return valid labels. The article shows why representation metrics need careful interpretation and why classification systems should be evaluated on more than accuracy alone.</p><h4>Our must-read articles</h4><p>1. <a href="https://pub.towardsai.net/fractal-recursion-bounded-infinity-and-adapted-systems-b11046c9e1e1?sharedUserId=tai-tech">Fractal Recursion, Bounded Infinity, and Adapted Systems</a> by<a href="https://liankimco.medium.com/"> Lian Kim</a></p><p>A model can reach lower training loss without improving its ability to generalize or adapt. The author uses the Koch snowflake, whose perimeter grows indefinitely while its area remains finite, to introduce the difference between convergence and stability. The author uses gradient descent and Hessian eigenvalues to examine how models converge during training, including why a flatter loss minimum does not necessarily mean better generalization. It also explains the difference between a system that stays close to its original state after a disturbance and one that eventually returns to it, and connects these ideas to memory and adaptive systems.</p><p>2. <a href="https://pub.towardsai.net/the-memory-problem-openais-dots-makes-impossible-to-ignore-525e93363d64?sk=2e301ff5621d915ca195a3ff1658bf17">The Memory Problem OpenAI’s Dots Makes Impossible To Ignore</a> by<a href="https://medium.com/@glennlenormand"> allglenn</a></p><p>OpenAI’s dots privacy FAQ states that users cannot individually inspect, edit, or delete saved memories. The author uses this limitation to examine how persistent agents should handle information that becomes outdated, incorrect, or needs to be forgotten. He compares memory controls across dots, Meta’s Muse, xAI’s Team Bots, and Claude Managed Agents, then explains why similarity-based retrieval alone cannot reliably distinguish old facts from current ones. The article proposes separate memory write and retrieval paths, validity periods, and provenance tracking so developers can correct or delete specific information without resetting an agent’s entire memory.</p><p>3. <a href="https://pub.towardsai.net/microsoft-foundry-routines-ai-agents-schedule-events-15e9a4b0c25e?sk=6964401bad5b927db7f340cb95d27f4f">AI Agents That Don’t Wait for a Prompt: How Microsoft Foundry Routines Work</a> by<a href="https://blog.azinsider.net/"> Dave R</a></p><p>Microsoft Foundry Routines lets agents start automatically on schedules, at specified times, or in response to GitHub issues and Teams messages. The author explains how Foundry manages triggers, execution identities, queues, retries, and run histories, including how agents can schedule their own follow-up tasks using a reminder tool. He also covers the difference between running an agent under its own identity and the routine creator’s permissions.</p><p>4. <a href="https://pub.towardsai.net/your-knowledge-graph-is-measuring-your-pipeline-not-your-subject-a-replication-attempt-6b62462820c6?sharedUserId=tai-tech">Your Knowledge Graph Is Measuring Your Pipeline, Not Your Subject: A Replication Attempt</a> by<a href="https://topuzas.medium.com/"> Ali Süleyman TOPUZ</a></p><p>Knowledge graphs can tell us how concepts relate to one another, but their structure also depends on how the LLM extracts those relationships. The author tested the same extraction pipeline on three unrelated subjects: ASP.NET Core middleware, SQL optimization, and cellulose acetate manufacturing. Surprisingly, the resulting graphs had almost the same number of connections relative to their size. Even small changes to how relationships were classified affected some metrics more than changing the subject entirely. The article explains how to check whether your knowledge graph metrics reflect the source material or simply the way your extraction pipeline was configured.</p><p>If you are interested in publishing with Towards AI, <a href="https://contribute.towardsai.net/">check our guidelines and sign up</a>. We will publish your work to our network if it meets our editorial policies and standards.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f30f47fd9bd1" width="1" height="1" alt=""><hr><p><a href="https://pub.towardsai.net/lai-146-can-you-trust-your-ais-memory-f30f47fd9bd1">LAI #146: Can You Trust Your AI’s Memory?</a> was originally published in <a href="https://pub.towardsai.net/">Towards AI</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[What We’re Adding to AI Engineering for Production]]></title>
            <link>https://pub.towardsai.net/what-were-adding-to-ai-engineering-for-production-83ee83a76c5d?source=rss-aea8a19ea239------2</link>
            <guid isPermaLink="false">https://medium.com/p/83ee83a76c5d</guid>
            <category><![CDATA[technology]]></category>
            <category><![CDATA[towards-ai]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[books]]></category>
            <dc:creator><![CDATA[Towards AI Editorial Team]]></dc:creator>
            <pubDate>Wed, 07 Oct 2026 16:58:00 GMT</pubDate>
            <atom:updated>2026-10-07T16:58:00.536Z</atom:updated>
            <content:encoded><![CDATA[<h4>More on agents, evaluations, coding workflows, and guidance we think AI engineers will need most in 2026.</h4><p>A quick update on our upcoming book, AI Engineering for Production.</p><blockquote><em>TL;DR: If you missed our first announcement, this is the book we have been working on as the 2026 follow-up to </em>Building LLMs for Production<em>. It is about what it takes to build LLM and agent systems that actually work in production, especially now that models are becoming significantly more capable and, at the same time, just one component of a reliable system.</em></blockquote><figure><a href="https://www.louisbouchard.ai/book/?utm_source=med-tai&amp;utm_medium=social&amp;utm_campaign=book"><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Vgq4yqipXEWoApy1.png" /></a></figure><p>We first announced the book for <strong>October 20</strong>. But realized it needed more work to become the book we actually want to hand you. In this final push, we want to focus on adding as much practical, production-ready information on agents, evaluations, and the engineering ecosystem as we can.</p><p>We don’t have a date yet, but it is definitely coming by the end of the year.</p><p>Want to know the moment it’s out? Get book updates → <a href="http://louisbouchard.ai/book">louisbouchard.ai/book</a></p><p>Here’s what we’ve been working on so far:</p><ol><li><strong>An entirely new chapter on coding with AI.</strong> Although this is a book about AI engineering, very few developers in 2026 are writing all their code themselves. So we are adding our best practices, strict boundaries, and the standards we use in enterprise work to show how to use coding agents reliably.</li><li><strong>An AI tutor made specifically for the book.</strong> It runs as an MCP server that you can plug into your own agent to debug and better understand every topic. We will also show you how to build one yourself and deploy it at scale in the book.</li><li><strong>Coding sections rewritten to go beyond implementation details.</strong> We are adding more insight into the decisions behind the code, rather than only explaining how to implement it. We are also doing our best to make it fun to read.</li><li><strong>A new conclusion focused on guardrails when building with AI.</strong> Given where the field is heading, our bet is that this will become one of the most important things to get right. So we are adding much more on this, while bringing in Louie’s enterprise experience to explain how teams can plan for changes in the industry.</li></ol><p>We are quite excited to share the book with you. And while we are adding everything we can to make it as useful as possible, if there are any questions you want answered, or there’s ONE THING the book needs to cover for you to make it worthwhile, <a href="https://www.louisbouchard.ai/book/?utm_source=med-tai&amp;utm_medium=social&amp;utm_campaign=book"><strong>sign up for book updates</strong></a> and reply to one of those emails.</p><p>Louis-François &amp; Louie</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=83ee83a76c5d" width="1" height="1" alt=""><hr><p><a href="https://pub.towardsai.net/what-were-adding-to-ai-engineering-for-production-83ee83a76c5d">What We’re Adding to AI Engineering for Production</a> was originally published in <a href="https://pub.towardsai.net/">Towards AI</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>