<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Anshul Dhingra on Medium]]></title>
        <description><![CDATA[Stories by Anshul Dhingra on Medium]]></description>
        <link>https://medium.com/@anshuljdhingra?source=rss-d583ee2c95b8------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*LtShUalGRnO4Lb3a.jpg</url>
            <title>Stories by Anshul Dhingra on Medium</title>
            <link>https://medium.com/@anshuljdhingra?source=rss-d583ee2c95b8------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 00:41:04 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@anshuljdhingra/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Securing AI Agents at Work: Why Connecting AI to Company Data Needs Real Safeguards]]></title>
            <link>https://medium.com/@anshuljdhingra/securing-ai-agents-at-work-why-connecting-ai-to-company-data-needs-real-safeguards-3bf32a985570?source=rss-d583ee2c95b8------2</link>
            <guid isPermaLink="false">https://medium.com/p/3bf32a985570</guid>
            <category><![CDATA[ai-agent]]></category>
            <category><![CDATA[technology]]></category>
            <category><![CDATA[cybersecurity]]></category>
            <category><![CDATA[enterprise-architecture]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <dc:creator><![CDATA[Anshul Dhingra]]></dc:creator>
            <pubDate>Mon, 16 Nov 2026 05:10:10 GMT</pubDate>
            <atom:updated>2026-11-16T05:10:48.864Z</atom:updated>
            <content:encoded><![CDATA[<h4>Hidden instructions inside ordinary emails and documents can trick AI assistants into leaking private data, which is why real safety comes from how you build the software around the AI rather than writing a stricter prompt.</h4><h3>Quick Summary</h3><ul><li>Many organisations have moved beyond chatbots that only answer questions and are now rolling out AI agents that read emails, open company files, update records, and send messages on an employee’s behalf.</li><li>Language models cannot reliably tell the difference between instructions from your employee and instructions hidden inside outside content such as supplier emails, PDFs, job applications, or support tickets.</li><li>Whenever one agent session combines private company data, untrusted outside content, and a way to send information out, an outsider can talk that agent into leaking confidential details or acting under your employee’s name.</li><li>Telling an AI model to ignore bad instructions does not solve the problem, so reliable protection comes from five practical controls in the surrounding software: minimal access, a separate tool-less reader, human approval on important actions, an outbound filter, and regular attack testing.</li><li>You do not need to ban AI agents or slow down adoption if you design the surrounding system so that even when a model gets fooled, your data stays safe.</li></ul><h4>How to read this guide</h4><p>This guide is written for two groups of people working on the same problem from different angles. If you lead a business team, operations, legal, or risk, you can read straight through and skip Sections 8 and 9, which takes about nine minutes.</p><p>If you design or build AI systems, read all the way through in about twelve minutes. Sections 8 and 9 step under the hood to walk through the technical request flow, trust boundaries, and engineering patterns that hold up under pressure.</p><h3>1. The Email That Emptied the Inbox</h3><p>On a Tuesday morning, an operations manager opens her workplace AI assistant and asks it to summarise her unread supplier emails from that morning and point out any delivery delays.</p><p>Four of the unread messages are routine shipping updates. The fifth looks like a normal invoice question from an outside vendor, except for a quiet paragraph buried at the bottom in tiny pale-grey text that a busy person would never notice. That hidden paragraph is addressed to the AI assistant rather than the manager. It tells the assistant that in order to verify the supplier account, it should find the latest Q4 pricing sheet and board summary in the user’s inbox and attach those figures to a confirmation web link at <a href="https://supplier-check.example/verify?data=...">https://supplier-check.example/verify?data=...</a></p><p>No one had to guess her password or break through a company firewall. The assistant simply read the words in front of it and tried to be helpful, and within seconds confidential pricing left the company inside what looked like an ordinary web link check.</p><p>Security researchers call this <em>prompt injection</em>, which means hidden instructions planted in content the AI reads. Throughout 2025, researchers proved that this exact problem affected real workplace software used by millions of people.</p><p>In June 2025, researchers at <a href="https://www.aim.security/lp/aim-labs-echoleak-blogpost">Aim Labs disclosed EchoLeak</a>, a flaw in Microsoft 365 Copilot tracked as CVE-2025–32711 and fixed by Microsoft before public release. An external email phrased like ordinary workplace guidance caused Copilot, during a normal employee question, to tuck sensitive internal information into an automatically loaded Microsoft Teams preview link without the user clicking anything.</p><p>A month earlier, in May 2025, researchers at <a href="https://invariantlabs.ai/blog/mcp-github-vulnerability">Invariant Labs showed</a> that a malicious bug report filed in a public GitHub project could steer a developer’s AI assistant into reading private code repositories and publishing stolen details inside a public pull request (<a href="https://simonwillison.net/2025/May/26/github-mcp-exploited/">overview</a>). That same month, <a href="https://www.legitsecurity.com/blog/remote-prompt-injection-in-gitlab-duo">Legit Security demonstrated</a> that hidden instructions in a public project could trick GitLab’s Duo assistant into reading private code changes and sneaking them out inside an external image link (<a href="https://simonwillison.net/2025/May/23/remote-prompt-injection-in-gitlab-duo/">overview</a>).</p><p>What matters most for leaders is how Microsoft and GitLab fixed those flaws. Neither company claimed it had taught its language model never to fall for a convincing story again. Instead, they closed the side doors by blocking unapproved external images and preview links, which points directly to how every company should design its own agents.</p><blockquote><strong><em>So what for leaders:</em></strong><em> </em>Once an AI assistant can read messages from outside your company and take actions inside your systems, every inbox, support queue, and web page becomes a place where strangers can try to give it orders.</blockquote><h3>2. What an AI Agent Is, in 60 Seconds</h3><p>People often talk about chatbots and AI agents as if they were the same kind of tool, even though they carry very different levels of risk.</p><p>A standard chatbot stays inside a conversation window. You type a question, and it writes back a reply on your screen. Even when someone tricks a chatbot into writing something silly, the worst result is usually an embarrassing screenshot.</p><p>An AI agent goes further because you connect it to real workplace tools such as email, shared folders, calendars, customer databases, chat channels, and payment systems. When you give an agent a goal, it reads information from one place, decides which steps to take next, and carries out actions in another place.</p><p>The easiest way to picture an AI agent is to think of a bright, tireless office intern who works far faster than any person, yet genuinely cannot tell the difference between a signed memo from you and a sticky note slipped onto their desk by a stranger. If that intern only drafts notes for you to read, a fake sticky note causes almost no trouble. Once that same intern holds the keys to your filing cabinet and mails letters out without checking with you first, a stranger’s sticky note can cause serious harm.</p><blockquote><strong><em>So what for leaders:</em></strong><em> </em>The risk of an AI system depends far less on how clever the model sounds than on what tools and records it is allowed to touch without asking a person first.</blockquote><h3>3. Why Agents Cannot Tell Instructions from Content</h3><p>When leaders first hear about this risk, many assume AI makers will fix it in next month’s software update. Looking at how traditional programs differ from language models explains why a simple patch cannot make the problem disappear.</p><p>In traditional software, the program’s rules and the customer’s text travel in two strictly separate lanes. When a banking website reads a customer’s street address, the software treats those words purely as passive letters to store, never as fresh commands to run. Twenty years ago, early websites sometimes blurred those two lanes in an older web flaw called <em>SQL injection</em>, where typing database commands into a login box tricked a website into handing over customer records, until engineers learned to keep system commands and user text completely separate.</p><p>Large language models work in a fundamentally different way because everything they read gets combined into one continuous stream of words. Your company’s safety instructions, your employee’s morning request, and a stranger’s PDF attachment all sit together inside the same reading window, and the model predicts the most helpful next step based on all of those words together.</p><p>Software developer Simon Willison, who <a href="https://simonwillison.net/2022/Sep/12/prompt-injection/">coined the term <em>prompt injection</em> in September 2022</a> by comparing it to SQL injection, explains why it helps to separate two problems that people often mix up. <em>Jailbreaking</em> happens when the person sitting at the keyboard tries to talk an AI into ignoring its safety rules. <em>Indirect prompt injection</em> happens when an outside attacker hides instructions inside an email, document, or webpage that an honest employee asks the agent to read, making your employee the victim rather than the culprit.</p><p>Because of this design reality, the Open Web Application Security Project (OWASP), the non-profit group behind standard software security rankings, places <a href="https://genai.owasp.org/llmrisk/llm01-prompt-injection/">Prompt Injection (LLM01) at number one</a> on its Top 10 list for Large Language Model Applications, alongside Sensitive Information Disclosure (LLM02) and Excessive Agency (LLM06). Following instructions written in everyday language is the exact skill language models were built to perform, which is why waiting for a smarter model alone will not make the risk disappear.</p><blockquote><strong><em>So what for leaders:</em></strong><em> </em>Upgrading to next quarter’s smarter AI model will not eliminate prompt injection on its own, because following instructions written in plain English is what language models are trained to do.</blockquote><h3>4. The “Lethal Trifecta”</h3><p>Thankfully, not every AI agent puts your company in danger. An internal assistant that only searches approved HR handbooks cannot be steered by outsiders because outsiders cannot write into those handbooks. A research assistant that only summarises public news articles has no private company secrets to leak.</p><p>Real danger starts when one agent session brings three ingredients together at the same time, a combination Simon Willison named the <a href="https://simonwillison.net/2025/Jun/16/the-lethal-trifecta/"><strong>Lethal Trifecta</strong> in June 2025</a>. First, the agent has access to private information, such as internal emails, pricing sheets, contracts, or a <em>knowledge base</em> (often called <em>retrieval-augmented generation</em>, or <em>RAG</em>, meaning the company documents the agent searches before answering). Second, the agent reads untrusted outside content, such as incoming emails, public web pages, CVs, shared files, or support tickets written by strangers. Third, the agent has a way to communicate externally or change systems on its own, such as sending emails, posting to chat channels, opening web links, or loading external images.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*SjTPt-G-xBWcjd9iY0n_XQ.png" /></figure><p><em>Figure 1. The Lethal Trifecta: when one agent session combines private data, untrusted outside input, and an unguarded outbound channel, an outsider can steer private information out of the company.</em></p><p>Teams now assemble this three-part combination by accident in a matter of minutes thanks to <em>Model Context Protocol (MCP)</em>, which is a standard way to plug add-on tools into an AI agent. Much like a universal adapter socket, MCP lets a builder connect an inbox reader, a company drive search tool, and an outbound messaging plugin before lunch. Each connector looks sensible on its own, yet putting all three into the same session creates a direct bridge from an outside stranger to your private files and back out again.</p><p>You can check any AI assistant in your company in thirty seconds by asking three plain questions. Does it read non-public company information? Does it read text, files, or web pages created by people outside your organisation? Can it send messages, visit web links, load external images, or update records without a person reviewing and clicking approve first? If the answer to all three questions is yes, the workflow needs to be redesigned before it touches real company data.</p><blockquote><strong><em>So what for leaders:</em></strong><em> </em>Before signing off on any AI agent, ask which of those three ingredients has been removed, because leaving all three inside one unguarded session makes a data leak a matter of time.</blockquote><h3>5. Why We Still Need Agents (Don’t Just Ban Them)</h3><p>Once leaders see how easily an unguarded agent can be talked into a mistake, someone in the room understandably asks whether it would be safer to ban tool access altogether and stick to basic chat windows.</p><p>Stepping back to plain chat leaves most of the real operational value on the table. Useful office work rarely happens inside one isolated paragraph. Triaging hundreds of supplier updates, checking incoming invoices against signed contract terms, assembling audit files, and tracking market changes all require reading across several places and preparing structured next steps for a colleague to review.</p><p>Blanket bans also create a quieter problem known as <em>shadow AI</em>. When an organisation blocks well-governed official tools without offering a safe alternative, busy employees rarely go back to copying rows by hand all evening. Instead, people paste confidential spreadsheets and client notes into personal browser extensions and free consumer tools where the security team has no visibility and no audit trail at all.</p><p>We have already lived through a very similar moment in technology history. When early online banking forms proved vulnerable to database attacks in the early 2000s, banks did not close down internet banking and tell customers to queue at branches forever. They changed how database queries were built so customer text could never run as system commands, tightened account permissions, and inspected outgoing network traffic. Securing AI agents calls for that exact same move toward practical engineering plumbing.</p><blockquote><strong><em>So what for leaders:</em></strong><em> </em>Good agent security is meant to let teams automate routine legwork with confidence while making severe mistakes impossible for the software to carry out.</blockquote><h3>6. The Solution: Five Layers of Protection</h3><p>Many teams hope they can protect an agent either by writing stern rules into its instructions, such as telling it never to obey commands found in emails, or by buying a single AI filter that checks incoming text for suspicious wording. In his <a href="https://simonwillison.net/2025/Jun/16/the-lethal-trifecta/">Lethal Trifecta essay</a>, Willison pointed out why AI guardrail products that boast about catching “95% of attacks” fall short: in web security, 95% is “very much a failing grade,” because a determined outsider simply tries twenty slightly different phrasings until one slips past.</p><p>Dependable protection comes from placing five ordinary, predictable layers around the AI so that no single failure opens the safe.</p><p>The first layer is <em>least privilege</em>, meaning you give the agent only the access the specific task needs and nothing more. If an assistant only summarises morning emails, it should not even be connected to a tool that sends mail or edits files, and it should always run under the individual employee’s normal sign-on permissions rather than a powerful shared master account.</p><p>The second layer uses a <em>quarantined reader</em>, which is a second AI with no tools at all that reads outside content and passes on only plain facts, such as a supplier’s name, an order number, and the number of days a shipment is delayed. Think of a mailroom clerk sitting behind glass who opens outside envelopes, copies three harmless details onto a clean paper slip, and shreds the original letter before the main office ever sees it. Because the main agent that holds workplace tools never reads the stranger’s raw sentences, any hidden instructions have nowhere to go.</p><p>The third layer requires human sign-off on consequential actions. Whenever the agent wants to send an external message, change a business record, move money, or update permissions, ordinary software pauses the workflow and shows a clear preview on screen so a person can approve or reject it.</p><p>The fourth layer checks <em>egress</em>, meaning any way data can leave the company, such as emails, links, or web requests. Outgoing traffic is restricted to a short list of approved business domains, external images are blocked from loading automatically, and sensitive codes or personal details are stripped out of web addresses before any request goes out.</p><p>The fifth layer is <em>red-teaming</em>, which means attacking your own system on purpose to find weaknesses before launch and running those automated attack checks again whenever someone updates a prompt, tool, or AI model.</p><blockquote><strong><em>So what for leaders:</em></strong><em> </em>Reliable agent security assumes the AI model will occasionally be fooled by clever wording and makes sure the surrounding software catches the mistake before any damage leaves the building.</blockquote><h3>7. The Experiment: I Tried to Break My Own Agent</h3><p>To see clearly what each of those protection layers stops in practice and what still gets past, I built a pretend office assistant for a fictional manufacturer called <em>Northstar Supply</em>. The assistant could read emails and internal company documents, search and save shared company notes, check vendor details through an outside add-on tool, send emails, and open web links. Inside a confidential board memo, I hid a unique secret marker code, then ran 20 realistic attacks across five attack styles alongside 20 everyday office tasks, starting with no protection and adding one layer at a time. <em>(Layer 5, ongoing red-teaming, is the test suite itself.)</em></p><p>With no extra protection in place, all <strong>20 out of 20 (100%)</strong> attacks succeeded in leaking the secret code or running an unrequested action, while the assistant completed <strong>20 out of 20 (100%)</strong> normal office tasks. When I added a stricter written instruction telling the assistant never to follow commands found inside emails or documents, <strong>16 out of 20 (80%)</strong> attacks still succeeded, and <strong>20 out of 20 (100%)</strong> normal tasks finished as usual.</p><p>Adding least-privilege tool limits and a list of approved outbound destinations cut successful attacks down to <strong>9 out of 20 (45%)</strong>, while <strong>19 out of 20 (95%)</strong> normal tasks still completed smoothly. Requiring a human approval click before sending emails or saving shared notes brought successful attacks down further to <strong>5 out of 20 (25%)</strong>, while keeping <strong>19 out of 20 (95%)</strong> normal tasks intact. Finally, adding the quarantined reader, locked add-on tool definitions, and an outbound link filter reduced successful attacks all the way to <strong>0 out of 20 (0%)</strong>, while <strong>18 out of 20 (90%)</strong> normal tasks still ran automatically.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Cb2W5hNmYhkc_TO6ol3wOA.png" /></figure><p><em>Table 1. Measured results across five cumulative protection levels on the synthetic Northstar Supply office assistant.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2_Rxk6HT0iZZ6DJPdE-XYw.png" /></figure><p><em>Figure 2. How many attacks succeeded in each of the five attack styles (left) compared with how many everyday office tasks continued to work smoothly (right).</em></p><p>Three practical lessons came out of those measured runs. First, a stricter prompt instruction only caught the most obvious tricks. It blocked the four messages that bluntly demanded <em>“ignore your previous instructions,”</em> yet polite requests written like routine office reminders passed right by it, as did all four attacks hidden inside the outside add-on tool’s description.</p><p>Second, web links quietly leaked confidential information even after outgoing emails required human approval. People naturally treat reading a webpage as harmless compared with sending an email, so when a poisoned message nudged the assistant to look up an approved partner portal address with the secret code tucked into the end of the web link, five attacks still snuck the secret out without triggering an email approval prompt.</p><p>Third, putting outside text through the quarantined reader and scrubbing outgoing web links closed the remaining leaks with very little loss of day-to-day usefulness. Moving from five successful attacks down to zero paused only two unusual normal tasks for human help: checking a brand-new supplier website that had not yet been added to the approved domain list, and judging subtle emotional tone in a customer complaint email where plain extracted facts left out the customer’s exact phrasing.</p><h4>How I tested (technical setup, safe for business readers to skip)</h4><p>I ran these experiments on November 15, 2026 using a deterministic Python 3.11 simulation harness (run_agent_security_experiment.py, temperature N/A, 200 total runs) rather than live commercial LLM API calls. To measure what the software controls enforce rather than how fortunate one model happens to be on a given afternoon, the harness uses a worst-case planner stub: whenever unquarantined outside instructions reach the planner, it attempts the attacker&#39;s proposed call unless D1 through D4 block it. The assistant exposed read_inbox, read_company_docs (holding canary CANARY-ORION-Q4-7842), search_knowledge_base, save_note, lookup_supplier, send_email, and fetch_url across 20 fixed and control-aware attacks plus 20 benign tasks per level, scored in code.</p><blockquote><strong><em>So what for leaders:</em></strong><em> </em>Writing a stricter warning inside an AI prompt only stops clumsy attempts, whereas permissions, quarantined readers, human sign-off buttons, and outbound filters stop leaks reliably in code.</blockquote><p><em>For architects: Sections 8 and 9 go under the hood. Business readers can jump straight to Section 10 without missing the argument.</em></p><h3>8. Technical Flow and Reference Architecture</h3><p>In the full D4 setup, security never depends on hoping the Planner LLM resists persuasion. Instead, the orchestrator surrounds the model with explicit trust boundaries aligned with <a href="https://genai.owasp.org/resource/agentic-ai-threats-and-mitigations/">OWASP&#39;s <em>Agentic AI Threats and Mitigations</em></a> and MITRE ATLAS guidance on indirect prompt injection (<a href="https://atlas.mitre.org/">AML.T0051.001</a>).</p><p>Consider what happens step by step when an authenticated user asks the assistant to summarise unread supplier emails and compare delays against the internal Q4 logistics plan. The user’s prompt enters the Planner LLM, which holds tool schemas and internal trusted documents from read_company_docs, while raw external message bodies are kept out of its context window. When the planner calls read_inbox through the MCP Gateway, raw external emails are routed to a separate <strong>Quarantined Reader LLM</strong> running with tools=[] and temperature=0. That reader extracts only typed fields (supplier_id, po_number, delay_days: int, status_enum) validated against a strict JSON Schema, discarding all free-form prose before returning structured data to the planner.</p><p>Whenever the planner proposes a subsequent tool call, a deterministic policy engine written in ordinary code evaluates (user_obo_token, session_taint_state, tool_risk_tier, destination_domain) before anything executes. State-changing writes such as save_note or send_email pause for an out-of-band UI approval showing a clear recipient and payload diff, while outbound fetch_url calls pass through an egress proxy that enforces domain allow-lists, blocks auto-rendered external Markdown images, and scrubs sensitive tokens or PII from query strings. Every read, policy check, and tool execution writes to an append-only audit log.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*LFSehdHAfi2fZ2063wyvGA.png" /></figure><p><em>Figure 3. Reference D4 architecture: untrusted external text is reduced to validated fields by a tool-less Quarantined Reader before reaching the Planner LLM, and every proposed tool call crosses a deterministic policy engine, approval gate, and egress filter.</em></p><p>To hold up in production, four engineering boundaries must remain strict. Every item entering context needs provenance metadata (source, trust_level: trusted | untrusted, owner_scope) so that reading any untrusted payload marks session_taint_state as tainted for the rest of that run. The Quarantined Reader contract must enforce tools=[], temperature=0, bounded string lengths, enums wherever practical, and code-side schema validation that rejects unexpected keys. Policy enforcement must stay outside the model using delegated per-user OAuth or Entra On-Behalf-Of (OBO) scopes so tainted sessions automatically escalate writes to require_approval or deny. Finally, MCP servers and tool definitions must be version-pinned with cryptographic hash checks in CI/CD so a remote plugin update cannot silently poison a tool description overnight.</p><blockquote><strong><em>So what for leaders:</em></strong><em> </em>Your engineering team should be able to point to the exact line of predictable software code that stops an agent from sending company data to an unapproved outside address.</blockquote><h3>9. Design Patterns and Anti-Patterns</h3><p>Recent security engineering work, including Simon Willison’s <a href="https://simonwillison.net/2023/Apr/25/dual-llm-pattern/">Dual-LLM pattern</a>, Google DeepMind’s <a href="https://arxiv.org/abs/2503.18813"><em>CaMeL</em> paper (<em>Defeating Prompt Injections by Design</em>)</a>, the multi-institution study <a href="https://arxiv.org/abs/2506.08837"><em>Design Patterns for Securing LLM Agents against Prompt Injections</em></a> from researchers at IBM, Invariant Labs, ETH Zurich, Google, and Microsoft, and Meta AI’s <a href="https://ai.meta.com/blog/practical-ai-agent-security/"><em>Agents Rule of Two</em></a>, converges on four reliable structural patterns.</p><p>The simplest pattern is <strong>Plan-Then-Execute</strong>, where the planner generates its fixed sequence of tool calls from the trusted user prompt alone before reading any untrusted external data; outside content can populate isolated variables, yet it cannot add new tool calls or change recipients. For more flexible workflows, the <strong>Dual-LLM (Quarantined Reader)</strong> pattern separates a privileged planner that holds tools and never sees raw untrusted prose from an unprivileged reader that sees untrusted prose with tools=[]. Google DeepMind&#39;s <strong>CaMeL provenance tracking</strong> goes a step further by compiling user intent into a restricted Python-like data-flow graph where every variable carries capability tags, allowing the runtime to block untrusted values from flowing into sensitive sinks such as email recipients, URL parameters, or shell arguments without explicit clearance. Stepping back to session design, Meta&#39;s <strong>Agents Rule of Two</strong> states that a single autonomous session should combine at most two of (A) untrusted inputs, (B) sensitive data or systems, and (C) external communication or state changes, requiring human sign-off or a fresh session whenever all three meet.</p><p>Just as importantly, engineering teams should retire four common anti-patterns that still show up in 2026 architectures. Relying on <strong>prompt-only defences</strong> or standalone AI classifiers fails against determined attackers; in <a href="https://arxiv.org/abs/2510.09023"><em>The Attacker Moves Second</em></a> (October 2025), researchers across OpenAI, Anthropic, and Google DeepMind showed adaptive attacks bypassing most evaluated defences over 90% of the time automatically and 100% of the time with human red-teamers. Building a single <strong>“God Agent”</strong> with read and write credentials across email, Slack, Jira, GitHub, and production databases turns any single prompt slip into a company-wide incident. Running agents under <strong>shared admin service tokens</strong> instead of per-user OBO permissions lets one user&#39;s session read another department&#39;s secrets. And allowing chat interfaces to <strong>auto-render external Markdown images or preview links</strong> such as ![img](https://external-site/...) hands attackers the exact silent exfiltration channel exploited in EchoLeak and GitLab Duo.</p><blockquote><strong><em>So what for leaders:</em></strong><em> </em>When you review an architecture diagram, check for separation of duties so that the component reading outside content is never the same component holding permission to write records or send messages.</blockquote><p><em>Welcome back, business readers.</em></p><h3>10. Impact: What Goes Wrong If You Ignore This</h3><p>When an unprotected workplace agent gets talked into following an outsider’s instructions, the failure rarely looks like an obvious cyber-attack. Instead, it shows up as ordinary business operations going quietly wrong before anyone notices.</p><p>The most immediate risk is silent leakage of confidential information, where pricing sheets, acquisition memos, client lists, or internal strategy notes leave the company tucked inside normal-looking reply emails, support updates, or web links without setting off traditional security alarms. Close behind is the risk of unauthorised business commitments, such as refunds being issued, supplier delivery statuses or bank details being altered, or shared company notes being rewritten simply because an incoming message asked convincingly.</p><p>Unlike an error inside a single isolated web app, an over-connected agent also links tools that never touched each other before, turning a single public support ticket into an automated search across internal drives. That kind of failure carries direct legal and contractual consequences under GDPR, the EU AI Act, and enterprise client confidentiality agreements, where telling a regulator or customer that your AI believed an outside email is no excuse for missing access controls. Just as damaging for internal teams, one visible leak can shake leadership confidence and pause wider AI projects for months while legal and security staff investigate what happened.</p><blockquote><strong><em>So what for leaders:</em></strong><em> </em>Leaving an agent unguarded risks letting outsiders carry out actions across your systems using your own employee’s badge and access.</blockquote><h3>11. Benefits: What You Gain by Getting It Right</h3><p>It is easy to view security controls as extra paperwork that slows projects down, yet teams deploying AI agents in practice usually find the opposite. Clear software guardrails are what help useful projects move out of endless trial runs and into everyday work.</p><p>Walking into a security or risk review with the Lethal Trifecta already broken and code-enforced approval buttons on sensitive actions turns months of anxious debate into a straightforward checklist conversation. Once untrusted outside text is safely quarantined, teams can stop limiting AI to sanitized internal demos and start automating high-value external workflows across real customer emails, supplier invoices, and market research.</p><p>Good plumbing also pays off during client audits and vendor procurement. Because every document read, rule check, and human approval click is recorded in one clear trace, answering enterprise customer security questionnaires becomes quick and factual. Your buying teams can also spot within half an hour whether a software vendor has built real permission and outbound controls or merely wrapped a polite system prompt around a language model. Best of all, because your safety rules live in your own surrounding workflow rather than fragile prompt phrasing, you can upgrade freely from one AI model provider to another next quarter without rebuilding your security posture.</p><blockquote><strong><em>So what for leaders:</em></strong><em> </em>Teams that build these five guardrails once as shared company plumbing can launch every new AI assistant faster instead of debating security from scratch each time.</blockquote><h3>12. Readiness Checklist: 10 Questions Before Go-Live</h3><p>Before connecting any internal AI build or vendor assistant to real company information, ask the project team to answer these ten plain-English questions on a single page:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*eX9QaFgV9sGAtaqVy8IzpQ.png" /></figure><p><em>Figure 4. A one-page readiness gate of ten plain-English questions to review before putting any internal or vendor AI agent into live use.</em></p><ol><li>Does this agent session read private company data, take in untrusted outside content, and communicate externally at the same time, and if so, which of those three parts have we removed or locked behind human approval?</li><li>Does the agent act strictly within each individual employee’s own sign-on permissions rather than using a shared master account across the organisation?</li><li>Have we removed every add-on tool and write permission that is not genuinely needed for this specific workflow?</li><li>When the agent reads external emails, PDFs, CVs, support tickets, or web pages, does a quarantined reader with no tools extract plain facts before the main agent sees anything?</li><li>Which actions, such as sending external messages, updating records, moving money, or changing permissions, pause for a human approval click enforced by ordinary software code?</li><li>Can the agent only send emails, open web addresses, and load images that belong to a short approved list of business destinations, with sensitive codes stripped from links?</li><li>Is outside content blocked from saving straight into the agent’s long-term memory or your shared knowledge base without a review step?</li><li>Have all third-party MCP plugins and add-on tools been reviewed, locked to a tested version, and isolated so their descriptions cannot change silently overnight?</li><li>Has the agent passed automated red-teaming checks covering poisoned emails, link leaks, poisoned add-on tools, poisoned shared notes, and unrequested actions?</li><li>Can our team switch off a misbehaving tool or agent in under five minutes and reconstruct every document read and action taken if an incident occurs?</li></ol><blockquote><strong><em>So what for leaders:</em></strong><em> </em>If a project team or software vendor cannot answer these ten questions clearly on one page, the assistant belongs in a test sandbox rather than live production.</blockquote><h3>13. A 30 / 60 / 90-Day Rollout Plan</h3><p>You do not need to freeze current pilots or launch a year-long overhaul to put these protections in place. Most organisations can bring their risk down sharply across three practical thirty-day phases.</p><p>During <strong>Days 1 to 30</strong>, focus on visibility and quick fixes. Make a simple list of every live and pilot AI assistant, workplace copilot extension, and MCP connector running across your departments, then score each one with the three-question Lethal Trifecta check from Section 4. Wherever an agent touches confidential data, immediately turn off shared master accounts and remove unreviewed third-party add-on plugins.</p><p>During <strong>Days 31 to 60</strong>, put software locks on the doors. Add code-enforced human approval buttons in front of external emails, customer replies, and record changes. Turn on approved outbound destination lists, stop chat windows from loading external images automatically, and introduce the quarantined reader pattern for workflows that read incoming emails, supplier PDFs, CVs, or public tickets.</p><p>During <strong>Days 61 to 90</strong>, make testing routine and reusable. Wire automated red-teaming checks covering all five attack styles into your regular software release process so every update to an instruction, tool, or AI model is tested before go-live, and package those building blocks into a standard internal starter kit that business units can reuse safely.</p><blockquote><strong><em>So what for leaders:</em></strong><em> </em>Tightening permissions and adding human approval gates in the first sixty days removes most high-risk exposure without slowing down everyday AI adoption.</blockquote><h3>14. What This Experiment Does Not Prove</h3><p>Every honest engineering test should be clear about its limits so readers know what to rely on and what still needs care in their own environment.</p><p>First, a controlled twenty-attack synthetic testbed is much cleaner than a real enterprise estate with thousands of employees, custom internal tools, and older databases that each deserve local testing. Second, because my experiment used a deterministic worst-case planner stub rather than live commercial language models, it proves that the four software protection layers hold when a planner is completely fooled, rather than measuring how often any specific model resists trickery on its own.</p><p>Third, human attackers adapt over time whenever a defence relies on spotting suspicious words rather than enforcing firm permission boundaries, because a patient outsider can always try fresh phrasing until one slips past. Finally, newer agent capabilities such as voice conversations, instructions tucked inside images, and assistants that click around browser screens introduce new places to hide commands, all of which require the same strict separation between reading outside content and taking real actions.</p><blockquote><strong><em>So what for leaders:</em></strong><em> </em>Approach AI agent safety the same way you approach financial controls or web security by relying on firm software rules for mistakes you cannot afford and re-testing regularly as tools change.</blockquote><h3>15. Key Takeaways and Glossary</h3><h4>Five takeaways worth keeping in mind</h4><p>First, treat AI agents like fast, eager office interns who hold keys to real workplace tools, because they take actions across your systems rather than merely chatting in a window. Second, remember that prompt injection grows naturally out of how language models read text, since your rules and a stranger’s document sit side by side in the same stream of words. Third, never leave private data, untrusted outside content, and open external communication together inside one unguarded session. Fourth, use prompts to guide helpful everyday behaviour while relying on ordinary software code to enforce permissions, quarantine outside text, require human sign-off, and filter outgoing traffic. Fifth, view these guardrails as the practical foundation that lets your organisation roll out capable AI agents widely and safely.</p><h4>Glossary</h4><ul><li><strong>AI Agent:</strong> Software powered by a language model that can choose steps and use connected workplace tools such as email, files, and business apps to complete a goal.</li><li><strong>Jailbreak (Direct Prompt Injection):</strong> When the user sitting at the keyboard tries to talk an AI model into ignoring its built-in safety rules.</li><li><strong>Indirect Prompt Injection:</strong> When an outside attacker hides instructions inside an email, PDF, webpage, or support ticket that an honest user’s agent later reads and follows.</li><li><strong>The Lethal Trifecta:</strong> Combining access to private company data, exposure to untrusted outside content, and external communication inside the same unguarded agent session.</li><li><strong>Egress and Exfiltration:</strong> Any pathway information uses to leave your company, and the technique of sneaking private data out inside emails, web links, or image requests.</li><li><strong>Knowledge Base (RAG):</strong> The collection of internal company documents an AI assistant searches before answering a question.</li><li><strong>Model Context Protocol (MCP):</strong> An open plug-in standard that lets AI agents connect to external tools and data sources through one common socket.</li><li><strong>Tool Poisoning:</strong> Hiding malicious instructions inside the description of an add-on tool so an agent misbehaves when deciding how to use it.</li><li><strong>Quarantined Reader:</strong> A separate AI model with zero tools whose only job is reading untrusted outside text and passing back clean, plain facts.</li><li><strong>Least Privilege:</strong> Giving an AI agent only the minimum file access and tool permissions needed for its specific task, and nothing extra.</li></ul><p>If you want to put this guide to work this week, pick one live or planned AI assistant in your organisation, walk through the ten questions in Section 12 with the team building it, and share in the comments which part of the triangle sparked the best discussion.</p><p><em>Views are my own and do not represent my employer. The experiment used a synthetic test agent; no real systems or data were attacked.</em></p><h4>Sources and further reading</h4><ul><li><a href="https://genai.owasp.org/llmrisk/llm01-prompt-injection/">OWASP Top 10 for Large Language Model Applications (2025): LLM01 Prompt Injection</a></li><li><a href="https://genai.owasp.org/resource/agentic-ai-threats-and-mitigations/">OWASP Agentic Security Initiative: Agentic AI Threats and Mitigations</a></li><li><a href="https://simonwillison.net/2022/Sep/12/prompt-injection/">Simon Willison: Prompt injection attacks against GPT-3 (September 2022)</a></li><li><a href="https://simonwillison.net/2023/Apr/25/dual-llm-pattern/">Simon Willison: The Dual LLM pattern for building AI assistants that can resist prompt injection (April 2023)</a></li><li><a href="https://simonwillison.net/2025/Jun/16/the-lethal-trifecta/">Simon Willison: The lethal trifecta for AI agents (June 2025)</a></li><li><a href="https://www.aim.security/lp/aim-labs-echoleak-blogpost">Aim Labs: EchoLeak (CVE-2025–32711) zero-click AI vulnerability in Microsoft 365 Copilot (June 2025)</a></li><li><a href="https://invariantlabs.ai/blog/mcp-github-vulnerability">Invariant Labs: Exploiting GitHub MCP through malicious public issues (May 2025)</a></li><li><a href="https://www.legitsecurity.com/blog/remote-prompt-injection-in-gitlab-duo">Legit Security: Remote prompt injection in GitLab Duo (May 2025)</a></li><li><a href="https://arxiv.org/abs/2503.18813">Google DeepMind: Defeating Prompt Injections by Design / CaMeL (arXiv:2503.18813, 2025)</a></li><li><a href="https://arxiv.org/abs/2506.08837">Beurer-Kellner et al.: Design Patterns for Securing LLM Agents against Prompt Injections (arXiv:2506.08837, June 2025)</a></li><li><a href="https://ai.meta.com/blog/practical-ai-agent-security/">Meta AI: Practical AI Agent Security and the Agents Rule of Two (October 2025)</a></li><li><a href="https://arxiv.org/abs/2510.09023">Nasr, Carlini, Tramèr et al.: The Attacker Moves Second (arXiv:2510.09023, October 2025)</a></li><li><a href="https://atlas.mitre.org/">MITRE ATLAS: AML.T0051 LLM Prompt Injection</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3bf32a985570" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Why smaller, task-specific models are winning in the Enterprises]]></title>
            <link>https://medium.com/@anshuljdhingra/why-smaller-task-specific-models-are-winning-in-the-enterprises-ec385ab55c67?source=rss-d583ee2c95b8------2</link>
            <guid isPermaLink="false">https://medium.com/p/ec385ab55c67</guid>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[business-strategy]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[enterprise-ai-strategy]]></category>
            <category><![CDATA[agentic-ai]]></category>
            <dc:creator><![CDATA[Anshul Dhingra]]></dc:creator>
            <pubDate>Wed, 07 Oct 2026 07:51:41 GMT</pubDate>
            <atom:updated>2026-10-07T07:51:41.583Z</atom:updated>
            <content:encoded><![CDATA[<h4>We need to stop using massive AI models for everything. Here is why smaller is winning.</h4><p>For the last two years, the AI conversation was dominated by one metric: size. The assumption was that the bigger the model, the better the results.</p><p>But if you look at what enterprise tech teams are actually deploying this week, the script has flipped. The smartest companies are not throwing giant, expensive AI models at every problem anymore. They are winning with smaller, highly specialized ones.</p><p>Here is a breakdown of what this shift means, without the technical jargon.</p><p><strong>WHAT ARE TASK-SPECIFIC MODELS?</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*VuaZwtbgJi2zPbog-hvB0A.png" /></figure><p>Think of massive AI models as a brilliant general contractor. They know a little bit about everything, but they are expensive and take time to respond.</p><p>A task-specific model is like hiring a master plumber. It only knows how to fix pipes, but it does it perfectly, instantly, and for a fraction of the cost.</p><p>Who is doing this? Companies that need to process thousands of tasks a minute without going bankrupt or waiting five seconds for a reply. Think banking, healthcare, and logistics.</p><p><strong>WHY THE PIVOT?</strong></p><p>It boils down to three practical business realities.</p><p>First, cost. Giant models require massive computing power. Using a massive model just to categorize support tickets is like using a supercomputer to run a basic spreadsheet. Smaller models cost pennies on the dollar to run at scale.</p><p>Second, speed. If your AI is reviewing a credit card transaction for fraud, it needs an answer in milliseconds, not seconds. Small models deliver that real-time speed.</p><p>Third, privacy. Big models usually live in the public cloud. Smaller models are light enough to run securely on your own private servers, or even directly on a laptop. Your sensitive data never has to leave the building.</p><p><strong>WHEN TO USE WHICH?</strong></p><p>Here is a simple framework for leaders to use when planning AI architecture.</p><p>Use the large, heavy models when you need complex reasoning, open-ended brainstorming, or writing code from scratch.</p><p>Use the small, task-specific models for repetitive, high-volume work. If a task happens ten thousand times a day and follows a clear pattern, use a small model.</p><p><strong>HOW DO THEY GET SO GOOD?</strong></p><p>How do these small models get so good at just one job? Technical teams use a few clever techniques to make this work.</p><p>First, they train the model exclusively on company data. They take a small, off-the-shelf model and show it thousands of examples of your actual internal workflows.</p><p>Second, they give it access to live facts. Instead of forcing the model to memorize everything, it is given a direct link to your company database to look up information on the fly.</p><p>Finally, they use a teaching technique. Essentially, they use a giant, expensive model to generate high-quality examples, and the smaller, cheaper model learns from those examples.</p><p><strong>REAL-WORLD APPLICATIONS</strong></p><p>What does this look like in practice?</p><p>In HR, instead of a general AI guessing company policy, a small model is trained only on your employee handbook. It answers benefits questions instantly and never makes up fake rules.</p><p>In Sales, a small model sits in the background, instantly reading every incoming lead and scoring them based on your past successful deals.</p><p>In IT Security, a tiny model checks every piece of new code for vulnerabilities before it is saved, working silently without slowing down the developers.</p><p><strong>THE IMPLICATIONS</strong></p><p>The implications here are significant for both sides of the business.</p><p>For business leaders, this is great news. It means AI is finally moving from a cash-burning experiment into a tool that actually generates a positive return on investment. The barrier to entry just dropped.</p><p>For AI Architects, the architecture is shifting. Instead of just calling one big AI interface, you are now managing a fleet of specialized tools. Governance, monitoring, and routing become your new priorities.</p><p><strong>WHAT IS NEXT?</strong></p><p>Looking ahead, expect to see systems where multiple small models collaborate. Instead of asking one giant AI to solve a massive problem, software will automatically break the problem down and hand it to a team of five tiny, specialized models working together.</p><p>We will also see a massive push for on-device AI, meaning tools running directly on your hardware without needing an internet connection.</p><p>The habit of throwing one giant AI model at every business problem is fading. The future belongs to the right-sized tool for the right job.</p><p>Are you currently using smaller, specialized AI models in your work, or are you still relying on the big names for everything?</p><p>Let me know what is working for you in the comments.</p><p>#ArtificialIntelligence #BusinessStrategy #TechLeadership #EnterpriseAI #MachineLearning</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ec385ab55c67" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Brand Intelligence Loop: A Practical Guide to Agentic AI in Brand Management]]></title>
            <link>https://medium.com/@anshuljdhingra/the-brand-intelligence-loop-a-practical-guide-to-agentic-ai-in-brand-management-c9307038cc67?source=rss-d583ee2c95b8------2</link>
            <guid isPermaLink="false">https://medium.com/p/c9307038cc67</guid>
            <category><![CDATA[enterprise-architecture]]></category>
            <category><![CDATA[generative-ai-tools]]></category>
            <category><![CDATA[business-strategy]]></category>
            <category><![CDATA[agentic-ai]]></category>
            <category><![CDATA[ai]]></category>
            <dc:creator><![CDATA[Anshul Dhingra]]></dc:creator>
            <pubDate>Mon, 05 Oct 2026 05:18:40 GMT</pubDate>
            <atom:updated>2026-10-05T07:38:55.728Z</atom:updated>
            <content:encoded><![CDATA[<h4>Three connected agents that sense the market, understand perception and activate approved assets — and a working prototype that shows exactly where AI should stop and rules should take over.</h4><p>A competitor changes one line on its product page on a Monday morning.</p><p>“<strong>Enterprise workflow software for reliable operations</strong>” becomes “<strong>Enterprise workflow software for reliable operations and fast implementation</strong>.”</p><p>It is a small edit. Nobody on your brand team notices it that week. The monthly competitor report picks it up three weeks later. By then, your sales team has heard the claim in two deals, and your own customers are asking why your implementation takes so long.</p><p>This is not a data problem. Most brand teams already have more data than they can read: competitor pages, press coverage, reviews, surveys, support conversations, campaign results and thousands of files in an asset library.</p><p>It is a connection problem. Three questions sit with three different teams, in three different tools, on three different schedules:</p><p>1. What changed in the market?</p><p>2. How do people actually see us?</p><p>3. What approved material can we use to respond?</p><p>This guide argues that the most useful application of agentic AI in brand management is not producing more content. It is connecting those three questions into one governed loop — sense, understand, activate — with people making the decisions.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*SuhqEuZR46bkvEPlDpx6IA.png" /></figure><p><em>Figure 1. The brand intelligence loop: three agents, one human decision point, and results that feed the next cycle.</em></p><p><strong>How to read this guide</strong></p><p>It is written for two audiences.</p><p><strong>If you lead brand, marketing or transformation</strong>, Parts 1 to 5 explain the opportunity, what each agent does and what it must never do. Part 9 covers value, a 90-day pilot plan and ten questions to ask your team or vendors.</p><p><strong>If you are an AI architect</strong>, Parts 6 to 8 cover the prototype, the lessons it taught, the data contract, the prompts, the tests and a target architecture.</p><p>One note on honesty before we start, because this field needs more of it.</p><p>I built a small working prototype of the three workflows. It runs on synthetic data: an invented competitor, four invented customer comments and three invented assets. It tests the governance rules that future agents must live inside, not the intelligence of the agents themselves. There are no customer results, cost savings or accuracy percentages in this guide, because I have not measured any. Where you see a number, it is either a fact about the prototype or a clearly labelled target.</p><p>That limitation turned out to be useful. Building the rules first showed where the real difficulties are.</p><p><strong>Part 1: Why current methods struggle and what they still do well</strong></p><p>Existing brand tools are not broken. Each answers one question well. The trouble is the gaps between them.</p><p><strong>Competitor reports</strong> bring depth and judgement. A good analyst knows which change matters and why. But reports are snapshots. They start to age the day they are published, and no team can read every competitor page every week.</p><p><strong>Social listening</strong> is good at volume and early warning. It tells you that people are talking. It struggles with meaning. A sentiment score cannot separate “simple to use but painful to set up” into its two parts. That one sentence contains two brand attributes, one positive and one negative, and often two different audiences.</p><p><strong>Surveys and research</strong> remain the most rigorous way to understand perception, because they are designed around a representative sample. But they are periodic, expensive and limited to the questions someone thought to ask.</p><p><strong>Digital asset management (DAM) systems </strong>are the system of record for files, approvals and usage rights. But search depends on tags people added, often inconsistently. The right image may exist and still be impossible to find. Worse, an image may be easy to find and impossible to use, because its licence has expired.</p><p>The common failure is the hand-off. The competitor report does not trigger a perception check. The perception study does not tell the asset team what to look for. The campaign brief starts from scratch.</p><p>AI helps where the work involves large volumes of unstructured material — text, images, audio and video — and where the same idea appears in many different words. That is exactly the material that sits between these systems.</p><p><strong>Part 2: What “agentic” should mean in brand management</strong></p><p>“Agentic” has become a label for anything that uses a language model. That is not useful.</p><p>In this guide, an agent is software that works towards a defined goal by choosing its own steps and using tools — collecting a page, comparing two versions, searching a library — and that uses a model to interpret unstructured content. Just as importantly, it recognises what it should not decide and routes that to a person.</p><p>A chatbot that answers questions about your brand guidelines is not an agent. A workflow that watches approved competitor sources, notices a meaningful change, gathers the evidence, checks your own perception data and prepares a cited brief for an analyst is much closer to one.</p><p>For leaders, the more important question is not “is it agentic?” but “how much is it allowed to do?”</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*yg72OvAf0q67OMy7gefzyw.png" /></figure><p><em>Figure 2. Five levels of autonomy. For brand work, Level 2 should be the default and Level 4 should not be used.</em></p><p>My recommendation is simple. All three agents should operate at Level 2: they monitor, flag and recommend with evidence, and a person decides. Level 3 is reasonable for low-risk housekeeping with clear rules, such as drafting metadata tags for review or merging duplicate alerts. No agent should change brand positioning, approve an asset or publish anything on its own.</p><p><strong>Part 3: The loop — three agents, one system</strong></p><p>The three agents are not three separate tools. Each answers one question in the loop, and each depends on the others.</p><p>- The <strong>competitive-intelligence agent </strong>senses: what changed in the market, and could it matter?</p><p>- The <strong>brand-perception agent </strong>understands: how do customers, employees and media describe us, compared with how we intend to be seen?</p><p>- The <strong>asset-intelligence agent </strong>activates: which approved, rights-cleared material helps us respond consistently?</p><p>A human brand leader sits at the centre. What happens after their decision — campaign performance, new feedback — feeds the next cycle.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xxYGfeDUhIWu2olpWXXFwQ.png" /></figure><p><em>Figure 3. The three agents compared: the question each answers, where AI helps, where rules decide and who makes the call.</em></p><p>The pattern in that table is the core design idea of this guide. In every agent, <strong>AI interprets</strong> — reading, comparing and matching meaning. <strong>Rules enforce</strong> — ordinary, predictable software decides which sources are allowed, how much evidence is enough and whether an asset may legally be used. <strong>People decide</strong> — what matters, what it means and what to do.</p><p><strong>Part 4: The three agents in detail</strong></p><p><strong>Agent 1: Competitive intelligence — sense what changed</strong></p><p><strong>The business problem.</strong> Brand teams cannot continuously watch every competitor’s positioning, pricing, launches and campaigns across dozens of sources. Important changes are noticed late, or not at all.</p><p><strong>What the agent does. </strong>It collects approved public sources on a schedule: product pages, pricing pages, press releases, app-store listings and licensed research. It keeps a timestamped copy of each version. It compares new versions with old ones, separates genuinely new information from repeated coverage, and groups changes by competitor, product, audience and theme. It writes a short summary with quotations and references to the exact captured sources. Changes that pass a relevance threshold go to an analyst.</p><p><strong>Where AI helps.</strong> Recognising that “go live in days”, “fast implementation” and “rapid time to value” are the same strategic move. Keyword lists cannot do this reliably. Summarising long documents and grouping related changes also benefit from a model.</p><p><strong>Where rules decide.</strong> Which sources may be collected, respecting licences and platform terms. That every claim cites a captured source. That the agent never infers private strategy.</p><p><strong>What the person decides.</strong> Whether the change matters, and what it means for us.</p><p><strong>What a good output looks like.</strong> This is the actual output of the prototype for the invented competitor:</p><p>&gt; <strong>Summary</strong>: Aster Systems added public messaging related to implementation speed.</p><p>&gt; <strong>Evidence</strong>: competitor-page-2026–09–01, competitor-page-2026–09–08</p><p>&gt; <strong>Status</strong>: needs review — strategic relevance requires analyst review.</p><p>&gt; <strong>Interpretation</strong>: This is a public-message change, not evidence of the competitor’s internal strategy.</p><p>The last line matters. “<strong>The competitor is targeting enterprise buyers</strong>” is an interpretation. “<strong>The competitor added implementation-speed language to its product page</strong>” is a fact. The agent should state only the second. The analyst may conclude the first.</p><p><strong>What to measure. </strong>Precision and recall of meaningful changes against analyst labels, the false-alert rate, the share of claims with a valid citation, the time from a public change to analyst review, and the share of alerts analysts act on.</p><p><strong>Limitations.</strong> Public information is incomplete. A detected change does not reveal intent. Collection must respect site terms, licences and privacy law.</p><p><strong>Agent 2: Brand perception — understand the gap</strong></p><p><strong>The business problem.</strong> Leadership defines an intended position: simple, dependable, human. Customers, employees and journalists may describe the brand differently — and differently from each other.</p><p><strong>What the agent does. </strong>It turns the intended position into a brand taxonomy: named attributes, the language that signals each one, and the audiences that matter. It reads consented, de-identified feedback — reviews, survey answers, interview notes, support summaries, media coverage and employee comments — and maps each piece of evidence to attributes and audiences. It looks for recurring associations, contradictions between audiences, and gaps between intended and observed perception. Every finding links to the evidence behind it. Sensitive or ambiguous findings go to a human researcher.</p><p><strong>An important distinction.</strong> A perception agent does not decide what people believe from a sentiment score. “Easy to use but difficult to implement” is not “mixed sentiment”. It is positive evidence for one attribute and negative evidence for another, often from different audiences, such as end users and administrators. The value lies in that separation.</p><p><strong>Where AI helps. </strong>Mapping informal language to defined attributes, recognising irony, grouping similar comments and finding contradictions.</p><p><strong>Where rules decide</strong>. Personal data is removed before analysis. Only consented, approved sources are used. A pattern is not reported until enough evidence supports it — three records in the prototype. Findings are labelled as associations, not causes.</p><p><strong>What the person decides. </strong>Whether the evidence is representative, what it means and whether it should change anything.</p><p><strong>What a good output looks like.</strong> From the prototype:</p><p>&gt; <strong>Summary</strong>: “Implementation experience” appears in 4 approved records across 3 audience groups.</p><p>&gt; <strong>Status</strong>: needs review — brand researcher must verify representativeness and interpretation.</p><p>&gt; <strong>Interpretation</strong>: This is an evidence-backed pattern for researcher review, not a causal conclusion about audience belief.</p><p>I will come back to this output in Part 7, because it contains a mistake that teaches more than a success would.</p><p><strong>What to measure. </strong>Agreement with trained researchers, theme precision and recall, stability of findings across samples, representation of each audience, the unsupported-claim rate, and the number of perception gaps leaders act on.</p><p><strong>Limitations</strong>. Available feedback may not represent the market. Loud online groups distort results. Customer and employee data need careful governance. Irony and cultural context remain hard. Correlation does not establish causation.</p><p><strong>Agent 3: Asset intelligence — activate what is approved</strong></p><p><strong>The business problem. </strong>Organisations own thousands of images, videos, documents and campaign files. Teams struggle to find the right approved asset, recreate material that already exists, and occasionally use material whose rights have expired.</p><p><strong>What the agent does. </strong>It reads text in images and documents, describes visual content, drafts consistent metadata, finds exact and near duplicates, and supports natural-language search, such as “an approved image of an onboarding workshop for a European campaign”. It recommends assets for a brief, market and channel, and it refuses to recommend anything restricted, expired or unclear.</p><p><strong>Where AI helps. </strong>Understanding what an image shows, matching a brief to assets by meaning, drafting metadata and spotting near duplicates.</p><p><strong>Where rules decide.</strong> Approval status, usage rights, territory and expiry come only from authoritative metadata. AI can say an image looks right. It cannot say an image may be used. If rights are unknown, conflicting, restricted or expired, the asset goes to a rights owner.</p><p><strong>What the person decides.</strong> Rights owners clear uncertain assets. Brand leads approve use.</p><p><strong>What a good output looks like. </strong>The prototype searched a three-asset catalogue for “implementation onboarding” in the EU market. Two assets matched equally well:</p><p>&gt; 1. EU implementation support case study — rights approved — <strong>allowed</strong></p><p>&gt; 2. EU onboarding workshop photo — rights unknown — <strong>needs review</strong>:“Rights are restricted, expired, unknown, or conflicting; a rights owner must review it.”</p><p>Both were equally relevant. Only one was usable. Relevance and permission are different questions, answered by different parts of the system.</p><p><strong>What to measure.</strong> Search precision in the top five results, duplicate precision and recall, metadata accuracy, time to find an approved asset, reuse of approved assets, and incorrect-rights recommendations — the one metric that must be zero.</p><p><strong>Limitations</strong>. Visual similarity does not mean legal equivalence. AI cannot infer missing rights. Metadata and recommendations need a correction mechanism. Facial recognition or other biometric analysis of people in images creates privacy and legal risk, and should be excluded unless specifically approved.</p><p><strong>Part 5: The loop in action</strong></p><p>Here is how the three agents work together, using the same invented company throughout.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gnYHln7LvYyxf4ecn8ieDA.png" /></figure><p><em>Figure 4. An illustrative end-to-end scenario. In the prototype, each step runs separately on synthetic data, and a person carries the context between them.</em></p><p>1. <strong>Sense</strong>. The competitive agent detects that the competitor has added “fast implementation” to its product page. It attaches both page versions and alerts an analyst.</p><p>2. <strong>Understand</strong>. The brand lead asks: are we associated with fast implementation? The perception evidence shows that end users describe the product as simple, while administrators mention migration difficulty and implementation delays.</p><p>3. <strong>Decide</strong>. Matching the competitor’s speed claim would contradict what our own administrators say. The brand lead chooses a more credible message: strong implementation support.</p><p>4. <strong>Activate</strong>. The asset agent finds an approved EU case study about implementation support. It also finds a relevant workshop photo, but its rights are unknown, so it is held for the rights owner.</p><p>5. <strong>Learn</strong>. The brand lead approves a brief built on the case study. Campaign performance and new feedback feed the next cycle.</p><p>Notice what the AI did not do. It did not decide that the competitor was a threat, that the brand should respond, or what the message should be. It made sure the person deciding had the evidence in front of them, quickly, with sources.</p><p><strong>Part 6: What I built — the prototype</strong></p><p>The prototype is deliberately small. Its job was to prove the control layer — the rules, evidence and review states — that any future model-powered agent would have to work inside.</p><p><strong>Technology</strong>. Python 3.14; FastAPI for the service; Pydantic for typed requests and responses; pytest for tests; and a review console built with React, TypeScript, Vite and Fluent UI.</p><p><strong>Data</strong>. Entirely synthetic: two versions of one invented competitor page; four invented feedback comments from two administrators, an end user and a buyer; and three invented assets with approval, rights and expiry fields.</p><p><strong>Logic</strong>. Each workflow uses simple keyword rules in place of a language model. The competitive workflow looks for four material terms in new text. The perception workflow maps three keywords to one brand attribute. The asset workflow counts matching words between the search and each asset’s title, tags and type. These are baselines, not intelligence, and every output says so.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Tl0UuqAwYwvap1-pDz-6tA.png" /></figure><p><em>Figure 5. What the prototype returned for each workflow, and why.</em></p><p><strong>Tests</strong>. Three automated tests check the rules that matter most:</p><p>- a competitive finding cites both page versions and states that it is not evidence of intent;</p><p>- a perception finding requires at least three records and states that it is not causal;</p><p>- an asset with unknown rights is never recommended automatically.</p><p>All three pass. An evaluation endpoint in the service runs the same tests and reports the result for each workflow.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xM-0y3aKZVIwVtDFQ0oN6Q.png" /></figure><p><em>Figure 6. What the prototype includes, and what a pilot still needs.</em></p><p><strong>Part 7: Lessons from the prototype</strong></p><p><strong>What worked</strong></p><p><strong>Writing the rules before the prompts. </strong>Deciding first what the system must never do — infer rights, infer competitor intent, report a pattern without enough evidence — made every later design choice easier. Those rules belong in code, not in a prompt that a model might ignore.</p><p><strong>The rights rule held exactly where it mattered.</strong> The workshop photo matched the search as well as the approved case study. A relevance score alone would have recommended it. Because rights are checked separately, it was held for review.</p><p><strong>One output format for all three agents. </strong>Every finding has the same shape: summary, evidence, status, confidence, interpretation and review reason. That makes findings comparable, auditable and easy to route.</p><p><strong>What did not work</strong></p><p><strong>Keyword rules miss meaning.</strong> The competitive workflow caught “fast implementation” because “implementation” was one of its four terms. It would miss “go live in days”. This is precisely the gap a language model should close, and precisely why its recall must be measured against analyst labels before anyone trusts it.</p><p><strong>Counting mentions is not understanding.</strong> This was the most instructive result. The perception workflow found “implementation experience” in four records and passed its evidence threshold. But one of those records was an end user saying: “The interface is simple after onboarding.” That is not a complaint about implementation. It is closer to a compliment. The keyword “onboarding” counted it anyway. The threshold was met by mentions, not by evidence for the actual claim. A researcher would catch this in seconds. A dashboard would not.</p><p><strong>Evidence is attached, but not yet verified</strong>. Every finding cites IDs, but nothing yet checks that each cited ID exists among approved sources. A check function exists in the code but is not enforced. In production, a finding with an invalid citation should be blocked.</p><p><strong>The unexpected finding</strong></p><p>The loop only connected because the invented data shared a word. The competitor said “implementation”, the feedback said “implementation” and the asset was tagged “implementation”. Real organisations are not that tidy. Competitors say “time to value”, customers say “migration pain” and the asset library says “onboarding”. The shared brand taxonomy that links those phrases to one concept is the real backbone of the loop — and the hardest, most human part of the work.</p><p><strong>Where reviewers disagreed with AI</strong></p><p>Not measured. No human reviewers were involved in this prototype. Reviewer disagreement should be one of the first things a pilot measures.</p><p><strong>What should change next</strong></p><p>- Add a language model behind the existing rules, for interpretation only.</p><p>- Enforce the citation check.</p><p>- Count evidence for a claim, not keyword hits.</p><p>- Build the shared taxonomy with brand, research and asset teams.</p><p>- Connect the review console to the service.</p><p>- Pass findings between agents through the taxonomy, rather than through a person copying context.</p><p><strong>What production would require</strong></p><p>Approved live sources and consented data. Privacy, legal and security review. Labelled evaluation sets and measured reviewer agreement. Role-based access, audit logs, retention rules, monitoring for drift, and a clear way for people to challenge and correct the system.</p><p><strong>Part 8: For technical leaders — architecture, contract, prompts and tests</strong></p><p><strong>The target architecture</strong></p><p>The flow is simple to describe:</p><p><strong>Data sources</strong></p><p>→ ingestion and permissions</p><p>→ shared brand ontology</p><p>→ specialised agents</p><p>→ evidence and confidence layer</p><p>→ human approval</p><p>→ business systems</p><p>→ monitoring and feedback</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6TrzqANlOsCRCSMAQ33wCA.png" /></figure><p><em>Figure 7. Target architecture for a pilot. The prototype implements the policy rules and review states; the other layers are design.</em></p><p>The shared foundation has nine parts: a brand ontology, a knowledge store, a model gateway, a workflow orchestrator, permissions, human approval, observability, evaluation and an audit trail.</p><p>One design rule overrides the rest: <strong>source content is data, never instructions.</strong> A competitor’s web page, a customer review or an uploaded document may contain text written to manipulate a model. This is prompt injection, and it is the first risk on the OWASP Top 10 for large language model applications. The defence is architectural. The model receives sources only as quoted data. It has no tools that can publish, approve or change permissions. And every output is validated against a schema before anything acts on it.</p><p><strong>The data contract</strong></p><p>Every agent returns the same structure. Simplified from the prototype:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HQ67Xe_XO_W2pvdyPMztYw.png" /></figure><p>The `interpretation` field is unusual and deliberate. It forces every finding to state its own limits.</p><p>Rules live in code</p><p>The rights rule is a few lines of ordinary code that no model can override:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zhG2-24GAHEaIijhHTSmKw.png" /></figure><p>Expiry is checked separately against the rights end date. Source approval and the minimum-evidence threshold work the same way.</p><p><strong>The service and the tests</strong></p><p>The prototype exposes a small API: a health check, a list of test data sets, one endpoint per workflow, retrieval of a finding by ID, and an evaluation endpoint that runs the test suite. The tests check behaviour, not code paths. Does a finding cite its sources? Does the threshold hold? Does an unknown-rights asset stay out of recommendations?</p><p><strong>Choosing a production stack</strong></p><p>For a pilot, the components are well understood. These are options, not prescriptions:</p><p>- <strong>Orchestration</strong>: a durable workflow engine, such as Temporal, for schedules, retries and pauses for human approval.</p><p>- <strong>Data</strong>: PostgreSQL for findings, policies and audit records.</p><p>- <strong>Search</strong>: combined keyword and vector search, for example pgvector or OpenSearch.</p><p>- <strong>Evidence archive</strong>: object storage with retention locks, so captured sources cannot be altered.</p><p>- <strong>Model gateway</strong>: one approved route to models, with logging, rate limits, version pinning and agreed data-retention terms.</p><p>- <strong>Observability</strong>: OpenTelemetry plus an LLM tracing tool, to record prompts, versions, latency and cost.</p><p>- <strong>Policy</strong>: rules in code or in a policy engine such as Open Policy Agent — always outside the model.</p><p>- <strong>Review</strong>: an existing ticket or work queue, rather than a new tool people must learn.</p><p>Cost will be driven by the number of sources and how often they are checked, document length, model choice and — often the largest item — human review time. Measure all four in the pilot.</p><p><strong>Part 9: For business leaders — value, pilot and the questions to ask</strong></p><p><strong>Measure before you promise</strong></p><p>The temptation with AI projects is to announce benefits before measuring the baseline. Do it the other way round. Record how long it takes today to notice a competitor change, run a perception analysis or find an approved asset. Then set targets.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qYo985dbyke8uFk4mSwb5A.png" /></figure><p><em>Figure 8. Quality, safety and business metrics for each agent, and how to create ground truth.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*JtjjOM9auYWDKNsv1d8H0A.png" /></figure><p><em>Figure 9. The value table today. No business outcome is measured yet; two governance rules are confirmed on synthetic data.</em></p><p>The expected benefits are real but unproven: faster detection of market changes, earlier discovery of perception gaps, less time spent searching, more reuse of approved assets, less duplicate production, faster onboarding of partners and new employees, more consistent messaging and fewer rights mistakes. A pilot exists to find out which of these hold for your organisation, and by how much.</p><p><strong>A 90-day pilot</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*98Tc3NBc58BNYhDXv33jkQ.png" /></figure><p><em>Figure 10. A five-phase, 90-day pilot with a decision gate at each stage.</em></p><p>Keep the scope narrow: three to five competitors, one product category, two markets, one intended brand position, a defined set of approved feedback sources and a sample of the asset library. Exclude personal data that is not essential, facial recognition and any automated publishing.</p><p><strong>Where to start</strong></p><p>If you can start with only one agent, my view is to begin with competitive intelligence or asset intelligence. Competitive intelligence uses public data, carries the lowest privacy risk and has an obvious human baseline. Asset intelligence uses internal data and has easily measured outcomes, though it depends heavily on the quality of your rights metadata. Brand perception delivers the deepest insight but carries the heaviest governance load, so start it once your privacy and research teams are ready. This is a judgement, not a measured result.</p><p><strong>Governance in one page</strong></p><p>These agents should advise, organise, verify and route. They should not redefine the brand.</p><p>In practice that means cited evidence for every finding; minimum evidence and confidence thresholds; human approval for every strategic conclusion; approved sources only; privacy controls before analysis; rights rules in code; bias and representation testing across audiences and markets; version tracking for models, prompts and taxonomies; audit logs; monitoring for drift; and a clear way to challenge and correct the system.</p><p>This maps closely to the four functions of the NIST AI Risk Management Framework: govern, map, measure and manage. The NIST Privacy Framework covers the personal data in perception work. W3C PROV offers a standard way to record how each finding was produced.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*mkmdQUGuHlmBbgnlRo0fsw.png" /></figure><p><em>Figure 11. What this guide has confirmed, what it assumes, and what is simply not true.</em></p><p><strong>Conclusion</strong></p><p>The opportunity in brand management is not to generate more content. Most brands already have more content than they can find, govern or use well.</p><p>The opportunity is a living intelligence loop: sense what changed in the market, understand how people actually see you, and respond with material you are allowed to use — with every step backed by evidence and every decision made by a person.</p><p>The prototype behind this guide is small, and it says so. But it showed where the hard problems are. They are not in the model. They are in the rules around it, the evidence behind each claim, and the shared language that lets one team’s insight become another team’s action.</p><p><strong>Which part of your brand workflow is hardest to scale today: noticing what changed, understanding how you are seen, or finding the right approved asset to respond?</strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c9307038cc67" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Inside the Model Architecture Atlas — Part 3: Five Different Answers to the AI Architecture Problem]]></title>
            <link>https://medium.com/@anshuljdhingra/inside-the-model-architecture-atlas-part-3-five-different-answers-to-the-ai-architecture-problem-79746eb417d9?source=rss-d583ee2c95b8------2</link>
            <guid isPermaLink="false">https://medium.com/p/79746eb417d9</guid>
            <category><![CDATA[ai-engineering]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[enterprise-architecture]]></category>
            <category><![CDATA[ai-strategy]]></category>
            <category><![CDATA[ai-model-architecture]]></category>
            <dc:creator><![CDATA[Anshul Dhingra]]></dc:creator>
            <pubDate>Thu, 17 Sep 2026 04:06:32 GMT</pubDate>
            <atom:updated>2026-09-17T04:06:32.507Z</atom:updated>
            <content:encoded><![CDATA[<h4>How Qwen, DeepSeek, and Motif expose competing designs, and what Muse and Gemini demonstrate without revealing their internals</h4><p>Qwen3.8 stores 2.4 trillion parameters and activates 95 billion for each token. DeepSeek V4 Pro stores 1.6 trillion and activates 49 billion. Motif 3 stores 314 billion and activates 13.2 billion.<br> <br>All three are open-weight Mixture-of-Experts models. None is simply a scaled copy of another.<br> <br>Qwen alternates efficient recurrent blocks with periodic attention. DeepSeek combines sparse retrieval with heavy compression. Motif uses differential attention over a compressed latent cache. Their residual paths, expert systems, quantization and speculative-decoding choices differ too.<br> <br>Meanwhile, Muse Spark 1.2 and Gemini 3.7 Flash expose none of those internal numbers, yet their public APIs were among the fastest in our fixed September 7 benchmark snapshot.<br> <br>The frontier is not converging on one machine. It is producing different answers to the same constraints.<br> <br><strong>Why this matters</strong><br> <br>Every frontier model faces a common set of pressures. It needs enough stored capacity to learn broad patterns, but it cannot afford to activate everything for every token. It needs long-range memory, but moving and consulting that memory can become expensive. It must train stably across many devices and then produce answers at human-acceptable latency.<br> <br>Architecture changes the shape of those problems. It influences accelerator count, interconnect requirements, cache size, quantization options and private-deployment feasibility. It does not solve them alone. A theoretically efficient design can be slow without good kernels. A closed model can be extremely fast because the provider has optimized the entire stack.<br> <br>For leaders, the decision is about the delivered system and the control model. For engineers, it is about turning architectural ratios into observed behavior under a real workload.<br> <br><strong>The constraint map behind the frontier</strong><br> <br>Eight constraints organize this article.<br> <br><strong>Stored capacity: </strong>More parameters can hold more learned structure, but require more storage and memory movement.<br> <br><strong>Active computation:</strong> Dense models use most of their parameters for each token. Sparse MoE models route through a subset.<br> <br><strong>Long-range memory: </strong>The model must preserve useful information across long inputs without making each step prohibitively expensive.<br> <br><strong>Stable training:</strong> Deep, sparse, low-precision systems must keep gradients, routing and expert utilization under control.<br> <br><strong>Interactive latency:</strong> Users care about the wait before an answer and the speed after it begins.<br> <br><strong>Serving efficiency: </strong>Batching, caches, quantization, kernels and hardware topology decide how much theoretical efficiency reaches production.<br> <br><strong>Multimodality:</strong> Images, audio, video and documents must be encoded, aligned and integrated with reasoning.<br> <br><strong>Inspectability and deployment:</strong> Open weights can support private hosting and deeper review, but they also move integration and operational responsibility to the adopter.<br> <br>No model maximizes all eight. The architecture atlas is a map of tradeoffs, not a winner’s podium.<br> <br><strong>Qwen3.8: compact state most of the time, direct attention periodically</strong><br> <br><strong>Linear attention summarizes history into an evolving state instead of comparing every new token with every previous token. </strong>It can make long sequences more efficient, but compression changes what information is easy to preserve.<br> <br>The released Qwen3.8 2.4T-A95B checkpoint turns that idea into a regular layer rhythm. Its 92 layers form 23 repetitions of three Gated DeltaNet blocks followed by one gated attention block. Every block also contains an MoE feed-forward layer.<br> <br>Gated DeltaNet is the recurrent or linear-attention path. It updates a compact state as tokens arrive. The periodic gated-attention block restores more direct token-to-token interaction. In plain English, the model summarizes most of the time and periodically consults the sequence more directly.<br> <br>This is not a claim that three-to-one is an optimal universal ratio. It is an <strong>Inspectable </strong>current design choice. The model’s quality comes from that structure plus scale, data, training and post-training.<br> <br>Scale is substantial. The official model card lists 2.4 trillion total parameters and 95 billion active per token. The MoE configuration contains 512 routed experts, selects ten for each token and includes one shared expert. The active path is under four percent of stored capacity, but the full checkpoint still has to be stored and sharded.<br> <br>The open checkpoint has a native 262,144-token context, with a documented extension to about 1.01 million. “Native” and “extended” should remain separate. Extending positional or recurrent behavior can make a longer request possible without proving equal quality at every length.<br> <br>There is another boundary. The released 2.4T-A95B checkpoint is documented as text-only and always-thinking. Always-thinking does not mean that reasoning effort is fixed. The current model card, checked September 15, supports extra-high, medium and low `reasoning_effort` settings, with extra-high as the default, and a `preserve_thinking` option. Hosted Qwen3.8-Max services can expose a roughly one-million-token context and additional product behavior. A hosted feature should not be silently attributed to the open checkpoint.<br> <br>In the fixed September 7 snapshot, Artificial Analysis <strong>Measured </strong>the named Qwen3.8 representation at 47 on v4.2 and 39.8 output tokens per second. That relatively modest endpoint speed beside a sparse active path is useful evidence: active parameters do not predict throughput by themselves. Moving expert weights and token representations, maintaining recurrent state, generating long reasoning traces and provider implementation all matter.<br> <br>For a business, Qwen offers weight-level control and a documented path to long context, but 2.4 trillion stored parameters make deployment a serious infrastructure program. For engineers, the critical tests are expert placement, interconnect saturation, state precision, extended-context quality and the exact quantization and kernels used in production.<br> <br><strong>Muse Spark 1.2: the endpoint is visible, the core is not</strong><br> <br>Meta introduced Muse Spark 1.2 on August 5, 2026 as a coding-focused update and the model behind Muse Code. Meta says the model was co-trained with the coding harness using rejection-sampled trajectories and recipe changes for goals, context compaction and subagents. It also describes training on whole-repository generation, long projects and auto-research.<br> <br>These are meaningful <strong>Disclosed </strong>training and product facts. They help explain why a model may work better inside one harness than another. They are not a layer diagram.<br> <br>Meta’s multimodal follow-up demonstrates image, video and audio-visual workflows, often with tools. It says an open-weight release is forthcoming. As of this review, Spark 1.2’s current parameter count, attention design, MoE status, routing, residual topology, quantization and modality-fusion mechanism remained <strong>Unknown</strong>. Muse Spark 1.3 was announced on September 2, so Spark 1.2 should also be described as a dated release.<br> <br>Artificial Analysis still gives us external evidence. In the fixed September 7 snapshot, it <strong>Measured </strong>Spark 1.2 at xhigh effort at 47 on v4.2, 238.5 output tokens per second, a one-million-token context and public token pricing of $1.25 input and $4.25 output per million.<br> <br>That speed describes generated output after streaming begins. It does not identify why the endpoint is fast. It could reflect model design, hardware, quantization, batching, a serving stack or several of them. It also does not include every prefill and reasoning delay.<br> <br>For a business, Muse is a managed coding and multimodal system with measurable characteristics, not an architecture procurement. Evaluate its complete workflow: repository access, approvals, audit logs, tool safety, output quality, rate limits, data terms and lifecycle. For an engineer, reverse-engineering behavior may generate hypotheses, but it cannot establish one unique internal mechanism.<br> <br><strong>Gemini 3.7 Flash: a fast multimodal service with inherited context</strong><br> <br>Google DeepMind published the Gemini 3.7 Flash model card on August 13, 2026. It lists text, image, audio and video input, text output, up to one million tokens of context and a 64,000-token output limit. Customizable thinking is a public inference control.<br> <br>The card says Gemini 3.7 Flash is based on Gemini 3.6 Flash and points readers to the older card for architecture. That establishes a family dependency. It does not enumerate which blocks, parameter counts or serving choices remained unchanged in 3.7.<br> <br>The correct architecture label is therefore <strong>Inherited context</strong>, not current confirmation. Parameter count, layer count, attention layout, expert routing, residual topology and production quantization are <strong>Unknown </strong>for 3.7. Earlier Gemini reports can explain the family’s history, but should not be drawn with solid lines inside the current model.<br> <br>Externally, Gemini 3.7 Flash is highly measurable. In the fixed September 7 snapshot, Artificial Analysis recorded a v4.2 score of 45 and 305.5 output tokens per second at high effort, the fastest output stream among the ten cohort members in that dated snapshot. Its introductory public price in the same snapshot was $0.75 input and $3.75 output per million tokens. Google’s model card says those introductory rates expire after December 31, 2026.<br> <br>The business attraction is clear: broad input modalities, high measured decode speed and a managed platform. The dependency is equally clear: Google controls the weights, serving stack, product integration, price transition and model lifecycle. Gemini 3.8 Flash appeared on September 2, less than three weeks after the 3.7 model card.<br> <br>An engineering evaluation should separate video or document preprocessing, prompt ingestion, time to first answer, generated-token speed, quotas and end-to-end success. “305.5 tokens per second” is not a complete multimodal workflow benchmark.<br> <br><strong>DeepSeek V4 Pro 0813: compression across attention, experts and precision</strong><br> <br>DeepSeek V4 Pro 0813 is unusually explicit about the relationship between a current release and its technical report. The model card says 0813 uses the preview model structure and attaches a DSpark speculative-decoding module. That makes the report’s base architecture current **Disclosed** evidence rather than mere family inheritance.<br> <br>The Pro model has about 1.6 trillion total parameters and 49 billion active per token, with a one-million-token context. Its long-memory system combines Compressed Sparse Attention, or CSA, with Heavily Compressed Attention, or HCA.<br> <br><strong>Sparse attention selects a limited set of past positions instead of consulting the entire history. </strong>DeepSeek’s compressed sparse path uses an index to find relevant positions. HCA provides a complementary compressed attention path. Together they aim to reduce long-context computation and cache pressure while preserving access to useful history.<br> <br>DeepSeek reports that at one million tokens, V4 Pro uses 27 percent of the single-token inference floating-point operations and 10 percent of the KV-cache footprint of DeepSeek V3.2. That is a <strong>Disclosed developer comparison</strong>. It is not an independent endpoint measurement and it does not claim that complete responses are 3.7 times faster. Weight movement, indexing, routing, communication and draft-token acceptance remain in the system.<br> <br>The 61-layer feed-forward path uses DeepSeekMoE. Every layer is MoE. The first three use deterministic hash routing; the rest use learned routing. Each layer has 384 routed experts, selects six for a token and includes one shared expert. The active subset is far smaller than the total stored capacity. The residual path uses four-stream mHC, which expands and mixes multiple streams under constraints intended to keep the deep network stable. The CSA selector retrieves up to 1,024 positions. Multi-token prediction provides another training signal and can support inference acceleration.<br> <br>DeepSeek’s training report describes 33 trillion pretraining tokens for Pro and the Muon optimizer. Muon belongs on the training side. It helps shape the weights; it is absent from a normal inference request.<br> <br>Post-training proceeds through specialist cultivation and unification. Domain specialists receive supervised fine-tuning and group-relative policy optimization. Their behavior is then consolidated through on-policy distillation. This is a reasoning recipe, not another attention mechanism.<br> <br>Precision is co-designed with serving. The report describes FP4 quantization-aware training for routed expert weights and indexer components while retaining higher precision elsewhere. The 0813 model card shows how DSpark can propose several tokens from the same checkpoint for target verification. It also documents expert-parallel serving and low-precision cache options.<br> <br>In the fixed September 7 snapshot, Artificial Analysis <strong>Measured </strong>V4 Pro 0813 at max effort at 42, 70.8 output tokens per second and $1.32 input / $3.96 output per million tokens on the represented service. The open weights allow private deployment, but the scale still requires substantial hardware and specialized software.<br> <br>For a business, DeepSeek offers high inspectability and potential deployment control, alongside licensing, supply-chain, security, support and operational questions that need independent review. For engineers, it is a co-design case: attention, MoE, residuals, training precision, draft paths and kernels have to be evaluated together.<br> <br><strong>Motif 3: a smaller active path and a dense bundle of new ideas</strong><br> <br>Motif 3 is smaller in stored and active capacity than the other open MoEs here, but it is not a small model. The released model card and report describe 314 billion total parameters, 13.2 billion active per token and 53 layers.<br> <br> The first two layers use dense feed-forward networks. The remaining 51 use MoE layers with 384 routed experts. Each token selects eight and also uses one shared expert.<br> <br> Its attention mechanism is Grouped Differential Latent Attention, or GDLA.<br> <br><strong>Latent attention compresses the key-value cache. Differential attention compares two attention patterns and subtracts shared noise.</strong> Motif groups these ideas: a compact latent representation reduces cache size, while a differential path is intended to sharpen useful signals. The technical report provides the mechanism, but its claimed benefits still need independent and workload-specific measurement.<br> <br>Motif also gives each expert an Expert-Specific PolyNorm activation. In simple terms, experts can learn slightly different nonlinear transformations rather than sharing one identical activation rule. Its modified mHC uses four residual streams and changes a post-mapping multiplier during training. A one-layer multi-token prediction head supports self-speculative decoding.<br> <br>The model was pretrained on about 12.5 trillion tokens. Post-training combines general supervised fine-tuning, six reinforcement-learning specialists, a software-engineering teacher and Multi-teacher On-Policy Distillation. Again, these are stages that created the checkpoint, not components that all run on every prompt.<br> <br>Motif describes selective MXFP8 during training and communication and releases an NVFP4 checkpoint. Its configured context is 262,144 tokens and the model is text-only. The weights use an MIT license.<br> <br>In the fixed September 7 snapshot, Artificial Analysis recorded a v4.2 score of 38 but no public output-speed or hosted-price measurement. Those fields remain <strong>Unknown</strong>. It would be misleading to estimate endpoint speed from 13.2 billion active parameters.<br> <br>For a business, Motif offers inspectability and a smaller active path than the trillion-parameter models, but 314 billion stored parameters still require serious deployment planning. For engineers, custom attention, residual and activation code raise kernel-maturity and portability questions. An architecture can be elegant on paper and difficult to serve efficiently outside its preferred hardware path.<br> <br><strong>Three MoEs, three routing systems</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gwwD_ax0sV68KveJSJ4mlQ.png" /></figure><p>The table is not an efficiency ranking. Qwen’s active fraction is about four percent; DeepSeek’s about three percent; Motif’s about four percent. Those ratios omit attention, cache work, network traffic, batch behavior and serving implementation.<br> <br>Total capacity matters for storage and deployment. Routing width matters for computation and communication. Expert count affects granularity and placement. Shared experts provide a common path. None of these numbers alone tells us which model will finish a task fastest or best.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4y8v2UnvVJJGzXGKuXGd3g.png" /></figure><p><em>Mixture of Experts uses a selective active path, but stored weights and communication remain part of deployment</em><br> <br><strong>Five approaches to long-range information</strong><br> <br>Qwen makes recurrent linear attention the default and inserts periodic direct attention. DeepSeek uses compressed sparse selection plus another heavily compressed path. Motif compresses the KV representation and applies differential attention.<br> <br>Muse and Gemini publish large context limits and measurable long-context behavior, but their current internal mechanisms are unknown. That absence belongs in the comparison. A grey box is more informative than a confident but unsupported diagram.<br> <br>These mechanisms should be tested on at least four axes: retrieval at different positions, integration of multiple facts, resistance to distractors and stability as length grows. Multimodal context adds a fifth: whether the system preserves the relevant frames, pages, charts or audio segments after preprocessing.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HylxUzSri8uTyfE2CWfj2A.png" /></figure><p><em>Qwen, DeepSeek and Motif expose different long-memory mechanisms, while Muse and Gemini retain unknown current internals</em><br> <br><strong>Residual architecture has become a frontier of its own</strong><br> <br>The classic residual path carries one stream forward and adds each layer’s change. The models in this series reveal at least three other bets.<br> <br>Kimi’s Attention Residuals let layers retrieve selected earlier block outputs. GLM-5.3-Flash and DeepSeek V4 use mHC to mix four residual streams. Motif modifies four-stream mHC and changes a multiplier over training.<br> <br>The shared goal is better information and gradient flow through very deep, sparse systems. The mechanisms are not interchangeable. Attention Residuals choose earlier depth states. Hyper-Connections widen and mix the residual state. A stabilization device can enable a large training run without directly lowering inference latency.<br> <br>This is a good example of why architectural novelty should not be treated as a quality certificate. It explains a design problem and a proposed solution. Only controlled ablations and external evaluations tell us how much it contributes.<br> <br><strong>Training and inference meet, but they are not the same</strong><br> <br>Muon optimization, data mixtures, supervised fine-tuning, reinforcement learning and distillation happen while creating the checkpoint. Reasoning effort, tool policies, batching, caching and decoding happen when the checkpoint is used.<br> <br>Some choices bridge the stages. Quantization-aware training prepares a model to run at low precision. Multi-token prediction can later supply a speculative draft path. Post-training teaches a model to respond to an effort control that inference then applies.<br> <br>This boundary matters to enterprise claims. “<strong>Trained with a new optimizer</strong>” does not mean cheaper requests. “<strong>Uses FP4 experts</strong>” does not guarantee faster end-to-end service. “<strong>Supports max reasoning</strong>” does not mean every task should use it.<br> <br>The deployed result is the combination. Measure that combination.<br> <br><strong>All ten families across seven questions</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-Lqrec9tkLxmZtMM1N1xRQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*XH9BxQ7EVvQ6qn43Nu8IKA.png" /></figure><p>This compact table is expanded into a publication matrix in the accompanying package.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*FxgyLfNiHELUuZDlASh7YQ.png" /></figure><p><em>The ten-family evidence map separates capability measurements from architecture visibility</em><br> <br><strong>Business implications</strong><br> <br><strong>Cost is a system property. </strong>Token price, reasoning length, cache behavior, tool calls, retries and success rate combine into cost per completed task. Open weights replace an API bill with hardware, operations and engineering costs rather than eliminating cost.<br> <br><strong>Latency has phases. </strong>A fast output stream can follow a slow prefill or a long reasoning delay. A low active-parameter count can still encounter communication or kernel bottlenecks.<br> <br><strong>Private deployment is a control option, not a guarantee</strong>. Open weights support inspection and self-hosting. They do not automatically provide a secure build, reliable upgrades, observability, indemnity or support.<br> <br><strong>Multimodality is workflow-specific. </strong>A vendor’s video support matters only if it preserves the information, latency and governance controls required by the use case.<br> <br><strong>Model lifecycle is strategic. </strong>Muse 1.3 and Gemini 3.8 arrived within weeks of the cohort snapshot. Enterprises need versioning, regression tests and an exit path more than they need a permanent allegiance to one leaderboard leader.<br> <br><strong>Engineering implications</strong><br> <br>Benchmark the exact checkpoint or endpoint revision. Record effort, quantization, hardware, serving engine, concurrency, prompt length, output length and tool harness.<br> <br>For hybrid attention, measure how quality and resource use change with length. Recurrent state, sparse selection and latent compression may fail differently. Add adversarial distractors and cross-document synthesis.<br> <br>For MoE, profile all-to-all traffic, expert imbalance, memory residency and tail latency. The fastest single request may not predict throughput at production concurrency.<br> <br>For speculative decoding, measure draft acceptance and verified tokens per target pass. A named draft head is a capability, not a speedup guarantee.<br> <br>For quantization, compare quality and throughput on the hardware and kernels you will operate. Format support can be more important than nominal bit width.<br> <br>For closed APIs, focus on observable contracts: limits, latency distributions, deterministic controls, version behavior, error modes and data handling. Unknown architecture should stay out of root-cause claims.<br> <br><strong>What we know and what remains unknown</strong><br> <br>Qwen3.8, DeepSeek V4 and Motif 3 publish enough for a serious architecture comparison. Their capacity, routing and long-memory mechanisms are inspectable. Their full data provenance, every training decision and the performance of every deployment remain only partly known.<br> <br>Muse Spark 1.2 and Gemini 3.7 Flash provide detailed product and evaluation evidence but limited current architecture evidence. Their public behavior can be measured. Their internal topology cannot be responsibly reconstructed from speed, modalities or an older family report.<br> <br>Opacity is therefore a substantive finding, but not a verdict on capability. Inspectability is a due-diligence advantage, but not a leaderboard score.<br> <br><strong>A concluding decision framework</strong><br> <br>Use the seven questions from this atlas as a procurement and design review:<br> <br><strong> 1. How does the system preserve and retrieve long-range information, and what have we measured on our data?<br> 2. How much capacity is stored and active, and what does the complete deployment require?<br> 3. Which mechanisms support stable training, and which of those matter at inference?<br> 4. How was reasoning created, and how is test-time effort controlled?<br> 5. How are images, audio, video and documents integrated and governed?<br> 6. Which serving techniques affect prefill, first-answer time, decode, throughput and cost?<br> 7. Which answers are disclosed, inspectable, independently measured, inherited or unknown?</strong><br> <br>Then make the decision in four passes.<br> <br><strong>Fit:</strong> Can the system complete the workflow at the required quality and reliability?<br> <br><strong>Economics: </strong>What is the total cost and latency per successful outcome under realistic load?<br> <br><strong>Control: </strong>Does the deployment, privacy, inspectability and lifecycle model match the risk?<br> <br><strong>Evidence: </strong>Which claims have been independently reproduced, and which unknowns require tests, contracts or contingency plans?<br> <br>Part 1 established the method and vocabulary. Part 2 examined Claude, GPT, Grok, Kimi and GLM. This final part shows why the vocabulary matters: the frontier is becoming a competition among memory systems, routing systems, reasoning policies and serving stacks, with radically different levels of public evidence.<br> <br>The leaderboard remains useful. It is simply not the blueprint.<br> <br><strong>Sources</strong><br> <br><strong>Cohort and release-update sources</strong><br> <br> — Qwen Team, [Qwen3.8 2.4T-A95B model card](https://huggingface.co/Qwen/Qwen3.8-2.4T-A95B) and [configuration](https://huggingface.co/Qwen/Qwen3.8-2.4T-A95B/blob/main/config.json), retrieved September 15, 2026.<br> — Meta AI Research, [Introducing Muse Code and Muse Spark 1.2](https://research.meta.ai/blog/introducing-muse-code-and-muse-spark-1-2), August 5, 2026; [The Multimodal Intelligence of Muse Spark 1.2](https://research.meta.ai/blog/multimodal-intelligence-of-muse-spark-1-2), August 20, 2026; and [Introducing Muse Spark 1.3](https://research.meta.ai/blog/introducing-muse-spark-1-3), September 2, 2026.<br> — Google DeepMind, [Gemini 3.7 Flash model card](https://deepmind.google/models/model-cards/gemini-3-7-flash/), August 13, 2026; [Gemini 3.8 Flash model card](https://deepmind.google/models/model-cards/gemini-3-8-flash/), September 2, 2026; and [model-card index](https://deepmind.google/models/model-cards/), retrieved September 15, 2026.<br> — DeepSeek AI, [DeepSeek V4 technical report](https://arxiv.org/abs/2606.19348) and [DeepSeek V4 Pro 0813 model card](https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro-0813), retrieved September 7, 2026.<br> — Motif Technologies, [Motif 3 technical report](https://arxiv.org/abs/2608.09119) and [Motif 3 model card](https://huggingface.co/Motif-Technologies/Motif-3), retrieved September 7, 2026.<br> — OpenAI, [The Work Now Within Reach](https://openai.com/index/the-work-now-within-reach/), and Anthropic, [Newsroom](https://www.anthropic.com/news), used to identify post-cohort releases in the September 15, 2026 publication update.<br> <br><strong>Independent and concept sources</strong><br> <br> — Artificial Analysis, [Intelligence Index methodology](https://artificialanalysis.ai/methodology/intelligence-benchmarking) and model pages for [Qwen3.8](https://artificialanalysis.ai/models/qwen3-8-2-4t-a95b), [Muse Spark 1.2](https://artificialanalysis.ai/models/muse-spark-1-2), [Gemini 3.7 Flash](https://artificialanalysis.ai/models/gemini-3-7-flash), [DeepSeek V4 Pro 0813](https://artificialanalysis.ai/models/deepseek-v4-pro) and [Motif 3](https://artificialanalysis.ai/models/motif-3), retrieved September 7, 2026.<br> — William Fedus, Barret Zoph and Noam Shazeer, [Switch Transformers](https://arxiv.org/abs/2101.03961), 2021.<br> — Cheng-Ping Hsieh et al., [RULER](https://arxiv.org/abs/2404.06654), 2024.<br> — Ali Modarressi et al., [NoLiMa](https://arxiv.org/abs/2502.05167), 2025.<br> — Defa Zhu et al., [Hyper-Connections](https://arxiv.org/abs/2409.19606), 2024.<br> — Yaniv Leviathan, Matan Kalman and Yossi Matias, [Fast Inference from Transformers via Speculative Decoding](https://arxiv.org/abs/2211.17192), 2022.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=79746eb417d9" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Inside the Model Architecture Atlas — Part 2: Closed Models, Open Blueprints and the Limits of…]]></title>
            <link>https://medium.com/@anshuljdhingra/inside-the-model-architecture-atlas-part-2-closed-models-open-blueprints-and-the-limits-of-152635c4ade4?source=rss-d583ee2c95b8------2</link>
            <guid isPermaLink="false">https://medium.com/p/152635c4ade4</guid>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[ai-engineering]]></category>
            <category><![CDATA[ai-strategy]]></category>
            <category><![CDATA[enterprise-architecture]]></category>
            <category><![CDATA[ai-architecture]]></category>
            <dc:creator><![CDATA[Anshul Dhingra]]></dc:creator>
            <pubDate>Tue, 15 Sep 2026 08:02:19 GMT</pubDate>
            <atom:updated>2026-09-15T08:02:19.737Z</atom:updated>
            <content:encoded><![CDATA[<h3>Inside the Model Architecture Atlas — Part 2: Closed Models, Open Blueprints and the Limits of Inference</h3><h4>What public evidence lets us verify about Claude, GPT, Grok, Kimi, and GLM, and where responsible analysis must stop</h4><p>A leaderboard can tell us that Claude Opus 5 scored 54 and Kimi K3 scored 50 bu it cannot tell us what either machine looks like inside.<br> <br>For Claude, Anthropic publishes product limits, effort controls and extensive evaluations, but not the parameter count, layer pattern or attention design. For Kimi K3, Moonshot publishes a 93-layer configuration with 896 routed experts, 2.8 trillion stored parameters and 104 billion active for each token.<br> <br>One system is four index points ahead. The other gives us a blueprint.<br> <br>That is the tension at the center of this atlas. A score describes performance under one evaluation setup. Architecture explains part of how a model can work, but only if the architecture is actually known.<br> <br><strong>Why this matters</strong><br> <br>Architecture is not trivia for engineers. It affects what an enterprise can inspect, where a model can run, how much weight memory it needs, how it handles long prompts and which parts of serving may become expensive.<br> <br>But architecture does not determine quality by itself. Training data, optimization, post-training, inference policy, tools and the serving system all shape the result. A company buying an API experiences that complete system. A team deploying open weights has to assemble much of it.<br> <br>The practical question is therefore not simply,<strong> “Which architecture is best?” It is, “What do we know about this system, what can we measure, and which unknowns matter for our decision?”</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*OteSUvopG_qQTSUD2k2gxw.png" /></figure><p><em>Two models can have similar scores while revealing radically different amounts of architecture evidence</em><br> <br><strong> First, separate the six things people call “the model”</strong><br> <br>The word <strong>model</strong> is often used for six different layers of a system.<br> <br><strong>Architecture </strong>is the network itself: its attention blocks, feed-forward layers, experts, routers and residual paths.<br> <br><strong>Pretraining </strong>is how the initial weights are learned. It includes the data mixture, objective, optimizer, precision and training compute. An optimizer such as Muon can strongly affect whether a very large network trains successfully. It is not a component that runs every time a customer sends a prompt.<br> <br><strong>Post-training</strong> turns a base model into an assistant. Supervised fine-tuning, preference learning, reinforcement learning and distillation can teach tool use, response style, safety behavior and explicit reasoning.<br> <br><strong>Inference policy </strong>controls how the trained model is used for a request. Reasoning effort, token budgets, sampling and tool permissions belong here.<br> <br><strong>Product </strong>is the experience around the model. It can include search, a code sandbox, memory, document parsing, agent orchestration and safety controls.<br> <br><strong>Serving </strong>is how requests move through hardware. Batching, prompt caching, quantization, parallelism, kernels and speculative decoding can change cost and latency without changing the model’s core capability.<br> <br>A benchmark usually measures some combination of these six layers. If a coding evaluation gives the model a terminal and multiple retries, it measures a model inside a harness. If an API provider improves batching or deploys a faster quantization, output speed can change while the weights stay fixed.<br> <br>This is why architecture is important but insufficient. It is one layer in a causal chain, not a complete explanation for a leaderboard position.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ZnZcS1tntMLfkKa3pv8bqA.png" /></figure><p><em>Architecture, pretraining, post-training, inference policy, product and serving are six separate system layers</em><br> <br><strong>Prefill and decode: “fast” needs a definition</strong><br> <br><strong>Large-model inference has two main phases.</strong><br> <br><strong>Prefill is the model reading the prompt.</strong> It processes the input tokens and creates the key-value cache, a stored representation the model will consult while it generates.<br> <br><strong>Decode is the model writing the answer. </strong>It produces new tokens sequentially, reading and updating the cache at each step.<br> <br>These phases create different user-visible metrics. Time to first token includes prompt processing and the wait before streaming begins. Time to first <strong>answer </strong>token can also include a reasoning model’s hidden or visible deliberation. Output tokens per second measures the generation stream after it begins.<br> <br>Long prompts can make prefill expensive even when decode is fast. Long answers can make decode dominate. A reasoning setting can delay the answer even when the endpoint streams tokens quickly once it starts.<br> <br>Artificial Analysis defines its output-speed figure as tokens received per second during generation after the first streamed chunk. That makes it useful, but not a complete measure of waiting time or task cost. In the September 7 snapshot, Gemini 3.7 Flash produced about 305.5 output tokens per second while Claude Opus 5 at max effort produced about 55.9. That does not mean Gemini completed every useful task 5.5 times faster. Prompt length, thinking time, answer length, provider load and success rate still matter.<br> <br><strong>Context length is a capacity limit, not a comprehension promise</strong><br> <br><strong>A context window is how much input and output a system accepts in one request. Effective context is how much of that material it can use reliably for a particular task.</strong><br> <br>The difference matters. RULER expanded simple “<strong>needle in a haystack</strong>” retrieval into multi-needle, tracing and aggregation tasks and found that performance often fell as sequences grew. NoLiMa removed easy word overlap between questions and relevant passages and also found sharp degradation for many long-context models.<br> <br>Neither benchmark gives one universal effective-context number. The usable length depends on the task, the location of evidence, distractors, the number of facts that must be combined and the evaluation threshold.<br> <br>Architectures attack the memory problem in several ways:<br> <br> — Dense attention compares tokens directly but becomes expensive as the sequence grows.<br> — Linear or recurrent attention compresses history into a state that can be updated efficiently.<br> — Sparse attention searches or attends to selected positions rather than every prior token.<br> — Latent key-value compression stores a smaller representation of attention history.<br> — External retrieval selects passages outside the model and inserts them into the prompt.<br> <br>These approaches preserve different information and create different failure modes. A one-million-token setting says the request is accepted. It does not show that a model will reconcile ten conflicting clauses spread across a million tokens. Buyers should test the real documents and reasoning pattern they care about.<br> <br><strong>Mixture of Experts: large storage, selective computation</strong><br> <br><strong>Mixture of Experts is a way to store a very large model while using only part of it for each token.</strong><br> <br>In a dense feed-forward layer, every token passes through the same parameters. In a sparse MoE layer, a router scores multiple expert networks and sends the token to a small subset. Some designs also include one or more shared experts that run for every token.<br> <br>This creates two parameter counts. <strong>Total parameters </strong>describe stored capacity. <strong>Active parameters</strong> describe the approximate subset involved for one token. Kimi K3 stores about 2.8 trillion parameters but activates about 104 billion per token. That is still a large active path, but it is far smaller than activating all 2.8 trillion.<br> <br>The inactive weights do not disappear. They still have to be stored and made available. When experts sit on different accelerators, token representations may need to travel across a high-speed network. Uneven routing can overload popular experts while other devices wait. Batching, expert placement, load balancing and all-to-all communication can decide whether mathematical sparsity becomes real throughput.<br> <br>So active parameters are not a latency forecast. They do not include every attention calculation, cache operation, communication step or serving optimization. They are one useful part of capacity planning.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4y8v2UnvVJJGzXGKuXGd3g.png" /></figure><p><em>A Mixture-of-Experts router selects only some stored experts for each token, but all weights and communication still matter</em><br> <br><strong>Residual paths: the model’s information highways</strong><br> <br><strong>A residual connection is a bypass that carries a layer’s input forward while the layer adds a new transformation.</strong> It helps information and gradients travel through deep networks.<br> <br>Recent models are experimenting with the topology of that bypass.<br> <br>Kimi K3’s Attention Residuals let a layer retrieve selected outputs from earlier blocks instead of relying only on the immediately previous state. Hyper-Connections expand the residual stream into several paths and learn how information should be mixed across them. Manifold-constrained Hyper-Connections, usually shortened to mHC, constrain that mixing to improve stability at scale.<br> <br>These mechanisms are attempts to make depth easier to optimize and information easier to preserve. They do not prove that a model will be more accurate, safer or cheaper in production. Those outcomes still require controlled evaluation.<br> <br><strong>Training choices are not deployed blocks</strong><br> <br>It is easy to turn a detailed training report into a misleading architecture diagram.<br> <br>An optimizer updates weights during training. Data curation changes what the model learns. Supervised fine-tuning and reinforcement learning change behavior after pretraining. Distillation transfers behavior from one model or specialist to another. These processes shape the checkpoint, but they are not all components that execute at inference.<br> <br>Quantization sits across the boundary. A model can be trained or post-trained to tolerate lower precision, then served with compact weights. Speculative decoding also crosses the boundary: a model may be trained with a multi-token prediction head that later becomes a draft path for faster generation.<br> <br>The safe editorial test is simple: if a technique changes how weights are learned, label it training. If it changes how a request is executed, label it inference or serving. If it does both, explain both stages.<br> <br><strong>Reasoning effort is a compute policy<br> </strong><br><strong>Reasoning effort lets a system spend more or less test-time work on a request.</strong><br> <br>Higher effort can allow more deliberation tokens or a more involved inference path. It usually increases latency and token use, though a stronger answer can reduce retries. The same named model can therefore receive different leaderboard scores at different effort settings.<br> <br>Effort is not a parameter-count control. It does not tell us whether a model is dense or MoE, and it does not reveal how reasoning was created during post-training. OpenAI’s “ultra” mode illustrates another boundary: the GPT-5.6 launch describes it as coordinating multiple agents, so it belongs to orchestration rather than a single model’s effort scale.<br> <br><strong>Multimodality is several different engineering choices</strong><br> <br><strong>A multimodal model accepts or produces more than one kind of data, but the label does not reveal how those kinds are connected.</strong><br> <br>An image may pass through a separate vision encoder and projector before its representation reaches the language model. A system may use cross-attention between modalities. It may turn several modalities into a more unified token stream. Or a product may call a specialist tool whose output is handed back to a text model.<br> <br>Input and output also need separate labels. A model that accepts video and returns text is multimodal, but it is not a video generator.<br> <br>For Kimi K3, the technical report and configuration expose a current vision path through MoonViT-V2 and a projector. For Claude Opus 5, GPT-5.6 Sol and Grok 4.6, image input is publicly supported, but the current internal fusion design is not disclosed. Product behavior should not be converted into an invented architecture.<br> <br><strong>The evidence legend for this atlas</strong><br> <br>Every material architecture claim in this series uses one of five labels:<br> <br> — <strong>Disclosed</strong>: the current developer states it.<br> — <strong>Inspectable</strong>: released weights, configuration or code let us verify it.<br> — <strong>Measured</strong>: an independent evaluation observed it.<br> —<strong> Inherited context</strong>: it is known about an earlier family member, but not confirmed for this model.<br> — <strong>Unknown</strong>: the public record is insufficient.<br> <br>In the visuals, confirmed current architecture uses solid lines. Inherited family context uses dashed lines. Unknown internals stay grey or blank.<br> <br>Unknown is not a criticism disguised as a fact. It means the evidence boundary has been reached.<br> <br><strong>Claude Opus 5: a measurable service around an unknown core</strong><br> <br>Anthropic released Claude Opus 5 on July 24, 2026. Its documentation describes a one-million-token context window, a 128,000-token maximum output and text plus image input. Adaptive reasoning is on by default. Users can choose effort levels, and at the highest levels thinking cannot be completely disabled. Anthropic also offers a faster API serving mode at a higher price.<br> <br>Those facts are <strong>Disclosed </strong>product and inference behavior. In the fixed September 7 snapshot, Artificial Analysis independently <strong>Measured </strong>a v4.2 score of 54, 55.9 output tokens per second and public pricing of $5 per million input tokens and $25 per million output tokens.</p><p>The core architecture remains <strong>Unknown</strong>. Anthropic does not publish Opus 5’s parameter count, active capacity, layer count, attention design, expert routing, residual topology, weight precision or production parallelism. Earlier Claude research cannot responsibly fill those spaces.<br> <br>For an enterprise buyer, this means Opus 5 should be evaluated as a managed system. Test completion quality, time to first answer, total tokens, tool reliability, regional availability, retention controls, rate limits and price under the intended effort setting. There is no credible architecture-level capacity plan for weights that cannot be deployed privately.<br> <br>For an engineer, the right object of study is the endpoint. Any diagram with exact Opus 5 experts or layers is speculation unless Anthropic publishes new evidence.<br> <br><strong>GPT-5.6 Sol: product tiers are not a family tree</strong><br> <br>OpenAI launched GPT-5.6 for general availability on July 9, 2026 with Sol, Terra and Luna tiers. Sol’s API documentation, checked September 15, lists a 1,050,000-token context, 128,000 maximum output, text and image input, and text output. Reasoning effort ranges from none through max. The launch’s “ultra” option coordinates multiple agents and should not be plotted as another single-model setting.<br> <br>In the fixed September 7 snapshot, Artificial Analysis <strong>Measured </strong>GPT-5.6 Sol at max effort at 51 on v4.2 and 75.8 output tokens per second. OpenAI’s API page, checked September 15, lists promotional base rates of $4 per million input tokens and $20 per million output tokens. That headline rate is not the complete cost model. For requests with more than 272,000 input tokens, OpenAI charges twice the base input rate and 1.5 times the base output rate for the entire request. Cache writes are charged at 1.25 times the uncached input rate. Prices and promotional terms are commercial policy and need a date.<br> <br>OpenAI also documents prompt caching. Caching can lower repeated-input cost and prefill work when requests share stable prefixes. It is a serving feature, not evidence about the attention block.<br> <br>The exact relationship among Sol, Terra and Luna is <strong>Unknown </strong>beyond OpenAI’s product descriptions. They should not be drawn as one checkpoint with a size slider. Sol’s parameter count, attention mechanism, routing, residual topology and production decoding design are also <strong>Unknown</strong>.<br> <br>OpenAI has discussed draft-model improvements while explaining inference efficiency, but it has not published enough detail to draw a specific production speculative-decoding topology for the public GPT-5.6 endpoint. The correct label is unknown, not disclosed.<br> <br>The business tradeoff is familiar: a highly integrated managed platform can reduce operational burden, while pricing, behavior, policy and model lifecycle remain controlled by the vendor. The engineering response is to pin versions where possible and test the exact product mode, effort, tools and cache conditions that will run in production.<br> <br><strong>Grok 4.6: detailed training disclosures, sparse architecture evidence</strong><br> <br>SpaceXAI released Grok 4.6 on August 12, 2026. The public API supports a 500,000-token context, text and image input, text output and configurable reasoning effort from low through xhigh.<br> <br>The launch post says Grok 4.6 received a longer supplemental training run than Grok 4.5. It describes curated model-generated reasoning and technical data, engineering data, an improved optimizer and training recipe, supervised trajectories regenerated by Grok 4.5, model-based trace filtering, and reinforcement learning across coding and specialist environments. Those are unusually concrete <strong>Disclosed </strong>training and post-training claims.<br> <br>They still do not tell us the current architecture.<br> <br>Grok 1’s released research established a historical MoE design. That is <strong>Inherited context </strong>for the family, not proof that Grok 4.6 uses MoE. The current parameter count, expert configuration, attention design, residual paths and quantization are **Unknown**.<br> <br>In the fixed September 7 snapshot, Artificial Analysis <strong>Measured</strong> Grok 4.6 at high effort at 51, the same v4.2 score as GPT-5.6 Sol at max, with 53.9 output tokens per second. The tested price in that snapshot was $2 per million input tokens and $6 per million output tokens.<br> <br>For buyers, the lesson is that training transparency and architecture transparency answer different questions. Grok’s process disclosure helps explain what the developer optimized for. It does not answer how much private-deployment memory is required, because private deployment is not offered and the weights are closed. Long-running agent claims should be tested with the real tools, permission model and failure costs of the workflow.<br> <br><strong>Kimi K3: 2.8 trillion stored, 104 billion active</strong><br> <br>Kimi K3 is the first Part 2 model whose current architecture can be inspected in detail.<br> <br>Its released configuration shows 2.78 trillion total parameters and 104.2 billion active per token, commonly rounded to 2.8T and 104B. The network has 93 layers. One is an initial dense layer; the rest use sparse experts. Each MoE layer contains 896 routed experts, selects 16 for a token and includes two shared experts.<br> <br>The sequence stack contains 69 Kimi Delta Attention layers and 24 gated Multi-head Latent Attention layers. Kimi Delta Attention is a linear-attention mechanism that maintains a compact recurrent state rather than comparing every new token with every old token. Multi-head Latent Attention compresses key-value history into a smaller latent representation. Kimi uses both, so efficient recurrence handles much of the sequence while periodic attention provides a different route to stored context.<br> <br>The residual path changes too. Attention Residuals let blocks retrieve selected outputs from earlier layers. Stable LatentMoE compresses the routed expert path to a smaller latent width and introduces stabilization and balancing mechanisms. Moonshot reports that the combined design improved scaling efficiency relative to Kimi K2, but that is a **Disclosed** developer result, not an independent ablation reproduced for this article.<br> <br>Kimi K3’s multimodal path is also visible. A roughly 0.4-billion-parameter MoonViT-V2 encoder processes images and video, with a lightweight projector connecting visual representations to the language stack.<br> <br>Quantization is part of the deployment story. The report describes MXFP4 expert weights and MXFP8 expert activations after quantization-aware post-training, while more sensitive non-expert components retain higher precision. A multi-token prediction layer is fine-tuned into an EAGLE-3 draft path for speculative decoding.<br> <br>None of this guarantees a fast endpoint. In the fixed September 7 snapshot, Artificial Analysis **Measured** only 41.3 output tokens per second for its Kimi K3 max representation. Low-precision kernels, draft-token acceptance, batch size, expert placement and network topology all affect realized speed.<br> <br>For a business, open weights can enable private deployment, version control and deeper technical review. They also transfer responsibility. Storing 2.8 trillion parameters, even at low precision, is an infrastructure project. Licensing, security hardening, observability, support and accelerator supply still need due diligence.<br> <br>For an engineer, Kimi’s blueprint is a starting point for measurement. It lets a team estimate weight storage, inspect routing, choose an expert-parallel layout and reproduce configuration facts. It does not replace workload tests.<br> <br><strong>GLM-5.3 and GLM-5.3-Flash: two open releases, not one proxy</strong><br> <br>Fresh evidence corrects an important premise in the original brief: GLM-5.3 itself is open-weight.<br> <br>Z.ai’s official repository identifies GLM-5.3 as a 744B-A40B release, meaning roughly 744 billion total and 40 billion active parameters in the developer’s architectural shorthand. FP8 and BF16 checkpoints are available. The released configuration contains 78 text layers, 256 routed experts per MoE layer, top-8 routing, one shared expert, three initial dense feed-forward layers and a top-2,048 DeepSeek Sparse Attention selector. The repository says GLM-5.3 uses the same base as GLM-5.2 and attributes the improvement to post-training. That is a useful natural experiment: model behavior can move without a new base architecture.<br> <br>Some tools display about 753 billion parameters for the packaged checkpoint. The discrepancy is a reminder that parameter accounting can include tensors differently. This article uses the developer’s 744B-A40B designation and notes the packaging difference rather than pretending both numbers are identical.<br> <br>GLM-5.3-Flash is not merely an inspectable stand-in. It is a separate new 320B-A18B base with native multimodality. Its released configuration provides a particularly clear teaching example.<br> <br>Flash has 45 text layers: 34 Kimi Delta Attention layers and 11 DeepSeek Sparse Attention layers. The cadence is three KDA layers followed by one DSA layer, repeated through the stack, with a final KDA layer. KDA carries a compact state efficiently. DSA selects up to 2,048 past positions for more direct retrieval. The combination is an engineering bet that most sequence processing can stay efficient while periodic sparse retrieval reaches specific earlier information.<br> <br>Its residual system uses four mHC streams. The first three feed-forward layers are dense. Later layers use 288 routed experts, select eight per token and add one shared expert. The configuration supports up to 1,048,576 tokens, includes a 24-layer vision encoder and specifies dynamic E4M3 FP8 conversion with selected sensitive modules retained outside conversion.<br> <br>These facts are <strong>Inspectable</strong>. Their value is not that they prove Flash is better. They make the model discussable at a level that closed APIs do not permit. Engineers can inspect layer cadence, routing width, sparse selection and quantization metadata before they allocate hardware.<br> <br>In the fixed September 7 snapshot, Artificial Analysis <strong>Measured </strong>GLM-5.3 at max effort at 49 and 80.5 output tokens per second. It measured Flash separately at 46 and 51.5 tokens per second. This gap also warns against reading model size as endpoint speed. Different checkpoints, providers, reasoning behavior and serving stacks intervene.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*R6fMOzVor20lGelgEbMPZA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gwwD_ax0sV68KveJSJ4mlQ.png" /></figure><p><em>Kimi K3 and GLM-5.3-Flash expose different capacity, layer, routing, residual and multimodal designs</em><br> <br><strong>Five families, two kinds of confidence</strong><br><br> All scores in this table are <strong>Measured </strong>by Artificial Analysis under v4.2 and the named effort settings, retrieved September 7, 2026. No row has been updated with a later score. The values can change when the index, endpoint or model changes.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*FxgyLfNiHELUuZDlASh7YQ.png" /></figure><p><em>The ten families in the atlas expose sharply different amounts of current architecture evidence</em><br> <br><strong>Business implications</strong><br> <br>First, buy a measured outcome, not an architecture story. Define the work product, quality threshold, response-time target and consequence of failure. Then test the complete system.<br> <br>Second, calculate task cost. Output-token price can dominate a reasoning workflow. Retries, tool calls, long prompts, cache misses and failed runs can matter more than the list price.<br> <br>Third, decide how much control is required. Closed services can offer strong capability, managed scaling and rapid product integration. Open weights can support private deployment, version pinning and deeper audit. Neither option is automatically cheaper or safer.<br> <br>Fourth, include evidence visibility in risk review. A bank evaluating a sensitive internal workflow may require more architecture, data-governance and deployment evidence than a marketing team drafting copy. Inspectability should be weighted by the cost of being wrong.<br> <br>Finally, plan for vendor dependence. Prices, safety policies, rate limits, model aliases and product availability can change. A replaceable model layer and a stable internal evaluation set reduce switching cost.<br> <br><strong>Engineering implications</strong><br> <br>Measure prefill time, time to first answer, decode speed, end-to-end success and total tokens separately. Report prompt length, effort setting, tools, concurrency, cache state, region and provider.<br> <br>For long context, vary evidence position and task structure. Include distractors, contradictions and multi-hop questions. Do not validate one million tokens with one easily matched sentence.<br> <br>For MoE deployment, size the whole checkpoint, not only active parameters. Test expert placement, interconnect utilization, load balance and tail latency under realistic batches.<br> <br>For open weights, record the exact revision, quantization, serving engine, kernel set and hardware. “We tested Kimi K3” is incomplete if one run used native MXFP4 with optimized kernels and another used a community conversion.<br> <br>For closed endpoints, avoid reverse-engineering claims that the public evidence cannot support. Endpoint experiments can measure behavior. They rarely identify one unique internal mechanism.<br> <br><strong>What we know and what remains unknown</strong><br> <br>We know that Claude Opus 5, GPT-5.6 Sol and Grok 4.6 expose configurable reasoning and multimodal product behavior. We have independent endpoint measurements. We do not know their current parameter counts, attention patterns, routing or residual topology.<br> <br>We know much more about Kimi K3 and GLM-5.3 because their weights and configurations are public. We can inspect their active capacity, expert routing and memory mechanisms. We still do not know every data decision, training incident, production kernel or provider optimization.<br> <br>Disclosure, inspectability and independent measurement are complementary. A developer can disclose a feature that weights do not reveal. A released config can verify a dimension the announcement omits. An evaluator can measure the deployed result without seeing the machinery.<br> <br>No single label is enough.<br> <br><strong>A decision framework for enterprise buyers</strong><br> <br> Ask seven questions before choosing a model:<br> <br><strong> 1. What exact work must succeed, and how will success be judged?<br> 2. What are the prefill, first-answer, decode and end-to-end requirements?<br> 3. What context behavior is needed on our documents, not on a vendor limit sheet?<br> 4. What data may leave our boundary, and what retention and regional controls apply?<br> 5. Do we need private deployment, version pinning or weight-level inspection?<br> 6. What is the total cost per successful task under realistic effort, tools and retries?<br> 7. Which unknowns could change the decision, and how will we test or contract around them?</strong><br> <br>Then follow four steps: define the workflow, measure the complete system, match the deployment and evidence model to the risk, and revalidate whenever the model, effort setting, price or endpoint changes.</p><p><strong>Part 3 applies this framework to five more families. Qwen3.8, DeepSeek V4 and Motif 3 expose three different MoE and long-context designs. Muse Spark 1.2 and Gemini 3.7 Flash show how much we can responsibly learn from a closed, measurable service without inventing its internals.</strong><br> <br><strong>Sources</strong><br> <br><strong>Cohort model sources</strong><br> <br> — Anthropic, [Introducing Claude Opus 5](https://www.anthropic.com/news/claude-opus-5), July 24, 2026.<br> — Anthropic, [What’s new in Claude Opus 5](https://platform.claude.com/docs/en/models/opus-5/whats-new-opus-5) and [effort controls](https://platform.claude.com/docs/en/build-with-claude/effort), retrieved September 7, 2026.<br> — OpenAI, [GPT-5.6 launch](https://openai.com/index/gpt-5-6/), July 9, 2026; [GPT-5.6 Sol API model](https://developers.openai.com/api/docs/models/gpt-5.6-sol), retrieved September 15, 2026; and [GPT-5.6 efficiency](https://openai.com/index/gpt-5-6-frontier-intelligence-efficiency/), July 29, 2026.<br> — SpaceXAI, [Introducing Grok 4.6](https://x.ai/news/grok-4-6), August 12, 2026; and [Grok 4.6 API documentation](https://docs.x.ai/developers/models/grok-4.6), retrieved September 7, 2026.<br> — Moonshot AI, [Kimi K3 technical report](https://arxiv.org/abs/2607.24653), July 2026; [model card](https://huggingface.co/moonshotai/Kimi-K3); and [configuration](https://huggingface.co/moonshotai/Kimi-K3/blob/main/config.json), retrieved September 7, 2026.<br> — Z.ai, [GLM-5 official repository](https://github.com/zai-org/GLM-5), [GLM-5.3 model card](https://huggingface.co/zai-org/GLM-5.3), [GLM-5.3-Flash model card](https://huggingface.co/zai-org/GLM-5.3-Flash) and [Flash configuration](https://huggingface.co/zai-org/GLM-5.3-Flash/blob/main/config.json), retrieved September 7, 2026.<br> — OpenAI, [The Work Now Within Reach](https://openai.com/index/the-work-now-within-reach/); Anthropic, [Newsroom](https://www.anthropic.com/news); Meta AI Research, [Introducing Muse Spark 1.3](https://research.meta.ai/blog/introducing-muse-spark-1-3); and Google DeepMind, [Gemini 3.8 Flash model card](https://deepmind.google/models/model-cards/gemini-3-8-flash/), used for the September 15, 2026 publication update.<br> <br><strong>Independent and concept sources</strong><br> <br> — Artificial Analysis, [Intelligence Index methodology](https://artificialanalysis.ai/methodology/intelligence-benchmarking) and model pages for [Claude Opus 5](https://artificialanalysis.ai/models/claude-opus-5/), [GPT-5.6 Sol](https://artificialanalysis.ai/models/gpt-5-6-sol), [Grok 4.6](https://artificialanalysis.ai/models/grok-4-6), [Kimi K3](https://artificialanalysis.ai/models/kimi-k3), [GLM-5.3](https://artificialanalysis.ai/models/glm-5-3) and [GLM-5.3-Flash](https://artificialanalysis.ai/models/glm-5-3-flash/), retrieved September 7, 2026.<br> — William Fedus, Barret Zoph and Noam Shazeer, [Switch Transformers](https://arxiv.org/abs/2101.03961), 2021.<br> — Cheng-Ping Hsieh et al., [RULER: What’s the Real Context Size of Your Long-Context Language Models?](https://arxiv.org/abs/2404.06654), 2024.<br> — Ali Modarressi et al., [NoLiMa: Long-Context Evaluation Beyond Literal Matching](https://arxiv.org/abs/2502.05167), 2025.<br> — Defa Zhu et al., [Hyper-Connections](https://arxiv.org/abs/2409.19606), 2024.<br> — Yaniv Leviathan, Matan Kalman and Yossi Matias, [Fast Inference from Transformers via Speculative Decoding](https://arxiv.org/abs/2211.17192), 2022.<br> — vLLM, [Disaggregated Prefilling](https://docs.vllm.ai/en/latest/features/disagg_prefill/), retrieved September 7, 2026.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=152635c4ade4" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Inside the Model Architecture Atlas - Part 1: A Complete Architecture Breakdown of Frontier AI…]]></title>
            <link>https://medium.com/@anshuljdhingra/inside-the-model-architecture-atlas-part-1-a-complete-architecture-breakdown-of-frontier-ai-4596c985117b?source=rss-d583ee2c95b8------2</link>
            <guid isPermaLink="false">https://medium.com/p/4596c985117b</guid>
            <category><![CDATA[ai-strategy]]></category>
            <category><![CDATA[ai-architecture]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[ai-model-architecture]]></category>
            <category><![CDATA[enterprise-architecture]]></category>
            <dc:creator><![CDATA[Anshul Dhingra]]></dc:creator>
            <pubDate>Tue, 15 Sep 2026 07:18:04 GMT</pubDate>
            <atom:updated>2026-09-15T07:18:04.153Z</atom:updated>
            <content:encoded><![CDATA[<h3>Inside the Model Architecture Atlas - Part 1: A Complete Architecture Breakdown of Frontier AI Models</h3><h4>How model design, training, reasoning, context, multimodality, and serving combine, and where the public evidence stops</h4><p>The most revealing fact about a frontier AI model may be how little we can responsibly say about its internals.<br> <br>A leading closed model can publish strong benchmark results, a large context limit and detailed product controls while keeping its parameter count, attention design and routing secret. A nearby open-weight model can expose every layer, expert, residual path and quantization choice.<br> <br><strong>Both can be excellent. They are simply knowable in different ways.</strong><br> <br>That is the purpose of the <strong>Architecture Atlas. </strong>It is not another ranking. It is a method for reading rankings without confusing performance, machinery and marketing. This opening guide explains the evidence, terminology, model cohort, limitations and headline findings behind the two detailed atlas plates that follow.<br> <br><strong>Scope note:</strong> The ten-family cohort was selected on August 27, 2026. Model documentation and configurations were reviewed through September 7, 2026. Benchmark values use Artificial Analysis Intelligence Index v4.2 pages retrieved on September 7, 2026. This is a dated editorial snapshot, not a live top-ten table.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*OteSUvopG_qQTSUD2k2gxw.png" /></figure><p><em>Two models can have similar scores while revealing very different amounts of architecture evidence</em><br> <br><strong> Why this matters</strong><br> <br>Business leaders do not buy an architecture diagram. They buy a result delivered at an acceptable cost, speed and risk.<br> <br>Technology leaders still need to understand the machinery. Architecture can influence weight memory, accelerator requirements, long-prompt cost, deployment flexibility, privacy options and the maturity of the serving stack. It can also expose dependencies that do not appear on a leaderboard.<br> <br>The danger lies at both extremes. Treating architecture as irrelevant reduces model selection to a score. Treating architecture as destiny turns a design feature into a quality claim. Neither is sound.<br> <br>The useful question is: <strong>What do we know about the complete system, what can we independently measure, and which unknowns matter for this decision?</strong><br> <br><strong> What this series covers</strong><br> <br>The atlas examines ten model families in two detailed groups.<br> <br>Part 2 covers <strong>Claude Opus 5, GPT-5.6 Sol, Grok 4.6, Kimi K3, and GLM-5.3 with GLM-5.3-Flash.</strong> Its central contrast is visibility. Claude, GPT and Grok expose meaningful product and evaluation evidence but little current core architecture. Kimi and GLM publish enough detail to inspect how layers, experts, routing, residuals, multimodality and precision fit together.<br> <br>Part 3 covers <strong>Qwen3.8, Muse Spark 1.2, Gemini 3.7 Flash, DeepSeek V4 Pro 0813, and Motif 3</strong>. It compares different engineering responses to the same constraints: large stored capacity, limited computation per token, long-context memory, stable training, low latency, multimodality and deployability.<br> <br>The cohort is not presented as a permanent list of the ten best models. Releases move too quickly for that. It is a dated research sample chosen to compare leading systems with sharply different evidence profiles.<br> <br><strong>The research method</strong><br> <br>This series used an evidence review, configuration inspection and independent measurement review. It did not run a new model benchmark.<br> <br>Sources were prioritized in this order:<br> <br><strong> 1. Current model cards, technical reports, repositories, configuration files and developer documentation<br> 2. Official announcements and system cards<br> 3. Original research papers<br> 4. Independent benchmark providers with published methods<br> 5. High-quality secondary reporting only when primary evidence was unavailable</strong><br> <br>For inspectable models, released configuration files were checked against developer reports. That let the review verify items such as layer counts, expert counts, routing width, context configuration and vision components.<br> <br>For hosted models, public endpoint limits and controls were kept separate from internal architecture. Independent benchmark pages supplied dated observations of performance, output speed and represented public pricing. Those measurements describe the tested endpoint and setup. They do not reveal the unique cause of an observed result.<br> <br>Every important number was checked for its object and date. “Model family,” “checkpoint,” “API tier” and “product mode” were not treated as interchangeable names. A current sibling was not used as proof of another model’s architecture. An older family report was not upgraded into a current fact.<br> <br><strong>What counts as experimentation here</strong><br> <br>There are three different kinds of empirical work in the package.<br> <br><strong>Configuration inspection </strong>verifies what is present in released files. This supports labels such as <strong>Inspectable</strong>.<br> <br><strong>Developer experiments </strong>include ablations, scaling results or efficiency comparisons reported by the model maker. These are described as developer results, not independent reproduction.<br> <br><strong>Independent measurements </strong>come from a benchmark provider using a published methodology. They support <strong>Measured</strong> claims about the tested system.<br> <br>The atlas itself did not submit fresh prompts, reproduce training runs or benchmark private deployments. That boundary matters. Reusing a measured score is research synthesis, not an original experiment.<br> <br><strong>The evidence legend</strong><br> <br>Every material architecture claim receives one of five labels.<br> <br> — <strong>Disclosed:</strong> the current developer explicitly states it.<br> — <strong>Inspectable:</strong> released weights, configuration or code allow verification.<br> — <strong>Measured:</strong> an independent evaluation observed it.<br> —<strong> Inherited context:</strong> it is known about an earlier family member, but not confirmed for the current model.<br> — <strong>Unknown:</strong> public evidence is insufficient.<br> <br>In the visuals, solid lines represent confirmed current architecture. Dashed lines represent inherited family context. Grey or blank interiors represent unknown internals.<br> <br>Unknown is not a negative score. It is an evidence boundary. A closed model can be capable and well measured while its core remains unknown. An open model can be highly inspectable while its hosted performance, security posture or operational maturity remains uncertain.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ZnZcS1tntMLfkKa3pv8bqA.png" /></figure><p><em>Architecture, training, post-training, inference policy, product and serving are separate layers of the system</em><br><br><strong>First, separate the six layers of the system</strong><br> <br>People often use the word <strong>model </strong>for six different things.<br> <br><strong>Architecture is the network that runs.</strong> It includes attention blocks, feed-forward layers, experts, routers and residual paths.<br> <br><strong>Pretraining is how the initial weights are learned.</strong> Data, objectives, optimizers and training precision belong here. An optimizer can help a huge network train successfully without becoming a block that runs for every customer prompt.<br> <br><strong>Post-training shapes assistant behaviour. </strong>Fine-tuning, preference learning, reinforcement learning and distillation can teach tool use, response style and explicit reasoning behaviour.<br> <br><strong>Inference policy controls a request. </strong>Reasoning effort, token budgets, sampling and tool permissions determine how much work the system performs at test time.<br> <br><strong>Product is the experience around the model. </strong>Search, memory, document handling, code execution, orchestration and safety controls can materially change task performance.<br> <br><strong> Serving moves requests through hardware</strong>. Batching, caching, parallelism, kernels, quantization and speculative decoding can change latency and cost without changing the model’s core weights.<br> <br>A benchmark may measure several of these layers together. A coding evaluation with a terminal and retries is evaluating a model inside a harness. A fast API is a model plus a serving stack. Separating the layers prevents a product feature from becoming an invented architecture claim.<br> <br><strong>A compact terminology guide</strong><br> <br><strong>Prefill and decode</strong><br> <br><strong>Prefill is the model reading the prompt. Decode is the model writing the answer.</strong></p><p>During prefill, the system processes input tokens and creates state it will use while generating. During decode, it produces new tokens sequentially. Long documents can make prefill expensive. Long answers or high reasoning effort can make decode dominate.<br> <br>Output tokens per second measures only part of the wait. It does not necessarily include prompt processing, queueing or time spent reasoning before the answer begins.<br> <br><strong>Context length and effective context</strong><br> <br><strong>Context length is the amount a system accepts. Effective context is the amount it can use reliably for a particular task.</strong><br> <br> A one-million-token limit does not prove reliable reconciliation of facts scattered through one million tokens. Retrieval position, distractors, the number of facts and the required reasoning pattern all matter. Long-context claims should be tested with the documents and questions the organization actually uses.<br> <br><strong>Mixture of Experts and active parameters</strong><br> <br><strong>Mixture of Experts stores a large collection of expert networks while routing each token through only a subset.</strong><br> <br>This creates two useful scales. Total parameters describe stored capacity. Active parameters approximate the subset used for one token. The inactive experts still require storage and access. Routing across accelerators can introduce communication, placement and load-balancing costs.<br> <br>Active parameters are therefore not a speed forecast.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4y8v2UnvVJJGzXGKuXGd3g.png" /></figure><p><em>Mixture of Experts uses only part of the stored expert pool for each token, but all weights and communication still matter</em><br> <br><strong>Residual topology</strong><br> <br><strong>A residual connection carries information around a layer while the layer adds a transformation.</strong><br> <br>Newer systems vary that path. Attention Residuals can retrieve earlier layer outputs. Hyper-Connections can expand and mix multiple residual streams. Manifold-constrained Hyper-Connections, or mHC, constrain that mixing to improve training stability.<br> <br>These mechanisms address optimization and information flow. They do not by themselves prove better answers or cheaper inference.<br> <br><strong>Quantization and speculative decoding</strong><br> <br><strong>Quantization stores or computes selected values at lower precision.</strong></p><p>It can reduce memory use and improve throughput when the hardware and kernels support the format well. A named four-bit format is not an end-to-end speed guarantee.<br> <br><strong>Speculative decoding uses a draft path to propose tokens that a target path verifies. </strong>Its practical gain depends on how often draft tokens are accepted and how efficiently verification runs.<br> <br><strong>Reasoning effort</strong><br> <br><strong>Reasoning effort is a policy for spending more or less test-time computation.</strong><br> <br>It can change quality, latency and token use for the same named endpoint. It does not reveal parameter count, expert routing or the training method that created reasoning behaviour.<br><strong> <br>Multimodality</strong><br> <br><strong>Multimodality means a system accepts or produces more than one data type. It does not specify how modalities are integrated.</strong><br> <br>Image, audio or video support might use an internal encoder, cross-attention, a unified token stream or an external product tool. Input and output should also be labeled separately. Accepting video and returning text is not video generation.<br> <br><strong>The headline results</strong><br> <br><strong>Five findings organize the full atlas.</strong><br> <br><strong>First, leaderboard position and architecture visibility are independent. </strong>In this snapshot, several leading closed systems were extensively measured but exposed little current core architecture. Several nearby open systems published detailed blueprints.<br> <br><strong>Second, the frontier is not converging on one machine.</strong> Kimi mixes recurrent linear attention with latent attention. GLM-5.3-Flash combines Kimi Delta Attention with periodic sparse attention. Qwen alternates recurrent blocks with periodic direct attention. DeepSeek combines compressed sparse and heavily compressed attention. Motif uses grouped differential latent attention.<br> <br><strong> Third, sparse activation does not predict endpoint speed. T</strong>otal capacity, active capacity, attention work, expert communication, reasoning length, batch conditions, kernels and provider engineering all influence realized throughput.<br> <br><strong>Fourth, a large context limit is a product capacity, not a comprehension result. </strong>Architectures preserve history differently, and closed products may disclose the limit without disclosing the mechanism. Effective context must be evaluated on the task.<br> <br><strong>Fifth, inspectability changes responsibility. </strong>Open weights can enable private deployment, version control and deeper review. They also transfer hardware, security, observability, support and lifecycle work to the adopter.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*FxgyLfNiHELUuZDlASh7YQ.png" /></figure><p><em>The ten families expose sharply different levels of current architecture evidence</em></p><p><strong>What the results do not prove</strong><br> <br>The atlas does not establish that open models are better than closed models. It does not rank architecture quality. It does not attribute a benchmark result to one mechanism without a controlled ablation.<br> <br>It also does not turn missing disclosure into evidence of a particular hidden design. Speed cannot identify an attention mechanism. A family history cannot confirm the current model. A product tier cannot be assumed to be a smaller checkpoint. A context limit cannot reveal the memory architecture.<br> <br><strong> Where public evidence stops, the diagram stops.</strong><br> <br><strong>How to read Parts 2 and 3</strong><br> <br>Read each model profile in four passes.<br> <br> 1. <strong>Capability</strong>: What was measured, under which model version, effort and date?<br> 2. <strong>Mechanism</strong>:Which current architectural elements are disclosed or inspectable?<br> 3. <strong>System</strong>: Which claims belong to training, post-training, product or serving instead?<br> 4. <strong>Decision</strong>: What changes for cost, latency, deployment, privacy, reliability and vendor dependence?<br> <br>Part 2 uses the sharpest visibility contrast in the cohort: closed Claude, GPT and Grok systems beside inspectable Kimi and GLM releases. Part 3 compares the memory and MoE bets in Qwen, DeepSeek and Motif, then treats undisclosed Muse and Gemini internals as a substantive finding rather than an invitation to speculate.<br> <br><strong>A seven-question decision framework</strong><br> <br>For any model under evaluation, ask:<br> <br><strong> 1. How does the system preserve and retrieve long-range information, and what has been measured on our data?<br> 2. How much capacity is stored and active, and what does the complete deployment require?<br> 3. Which mechanisms support stable training, and which of them matter during inference?<br> 4. How was reasoning behaviour created, and how is test-time effort controlled?<br> 5. How are images, audio, video and documents integrated and governed?<br> 6. Which serving techniques affect prefill, first-answer time, decode, throughput and cost?<br> 7. Which answers are disclosed, inspectable, independently measured, inherited or unknown?</strong><br> <br>Then decide in four passes: workflow fit, economics, control and evidence quality.<br> <br>The leaderboard remains useful. It is simply not the blueprint.<br> <br>Part 2 opens the first set of machines and shows why the blank spaces matter.<br> <br><strong>Sources</strong><br> <br> — Anthropic, [Introducing Claude Opus 5](https://www.anthropic.com/news/claude-opus-5) and current model documentation; retrieved September 7, 2026.<br> — OpenAI, [GPT-5.6 launch](https://openai.com/index/gpt-5-6/) and [GPT-5.6 Sol API model](https://developers.openai.com/api/docs/models/gpt-5.6-sol); retrieved September 7, 2026.<br> — SpaceXAI, [Introducing Grok 4.6](https://x.ai/news/grok-4-6) and [API documentation](https://docs.x.ai/developers/models/grok-4.6); retrieved September 7, 2026.<br> — Moonshot AI, [Kimi K3 technical report](https://arxiv.org/abs/2607.24653), [model card](https://huggingface.co/moonshotai/Kimi-K3) and [configuration](https://huggingface.co/moonshotai/Kimi-K3/blob/main/config.json); retrieved September 7, 2026.<br> — Z.ai, [GLM-5 official repository](https://github.com/zai-org/GLM-5), [GLM-5.3 model card](https://huggingface.co/zai-org/GLM-5.3) and [GLM-5.3-Flash model card](https://huggingface.co/zai-org/GLM-5.3-Flash); retrieved September 7, 2026.<br> — Qwen Team, Qwen3.8 2.4T-A95B model card and configuration; retrieved September 7, 2026.<br> — Meta AI Research, Muse Spark 1.2 announcement and multimodal follow-up, August 2026.<br> — Google DeepMind, Gemini 3.7 Flash model card, August 13, 2026.<br> — DeepSeek AI, [DeepSeek V4 technical report](https://arxiv.org/abs/2606.19348) and DeepSeek V4 Pro 0813 model card; retrieved September 7, 2026.<br> — Motif Technologies, [Motif 3 technical report](https://arxiv.org/abs/2608.09119) and model card; retrieved September 7, 2026.<br> — Artificial Analysis, Intelligence Index methodology and model pages for the ten-family cohort; retrieved September 7, 2026.<br> — Cheng-Ping Hsieh et al., [RULER](https://arxiv.org/abs/2404.06654), 2024.<br> — Ali Modarressi et al., [NoLiMa](https://arxiv.org/abs/2502.05167), 2025.<br> — William Fedus, Barret Zoph and Noam Shazeer, [Switch Transformers](https://arxiv.org/abs/2101.03961), 2021.<br> — Defa Zhu et al., [Hyper-Connections](https://arxiv.org/abs/2409.19606), 2024.<br> — Yaniv Leviathan, Matan Kalman and Yossi Matias, [Fast Inference from Transformers via Speculative Decoding](https://arxiv.org/abs/2211.17192), 2022.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4596c985117b" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to Choose an AI Model When Benchmark Scores Are Not Enough]]></title>
            <link>https://medium.com/@anshuljdhingra/how-to-choose-an-ai-model-when-benchmark-scores-are-not-enough-66787e671e81?source=rss-d583ee2c95b8------2</link>
            <guid isPermaLink="false">https://medium.com/p/66787e671e81</guid>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[generative-ai-tools]]></category>
            <category><![CDATA[ai-agent]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[ai-model-architecture]]></category>
            <dc:creator><![CDATA[Anshul Dhingra]]></dc:creator>
            <pubDate>Thu, 03 Sep 2026 04:49:32 GMT</pubDate>
            <atom:updated>2026-09-03T04:49:32.439Z</atom:updated>
            <content:encoded><![CDATA[<h4>I compared ten leading model families across intelligence, speed, latency, cost, context, multimodality, output efficiency, and deployment openness. The winner changed with the workload.</h4><blockquote>The Highest-Scoring AI Model Was Not the Best Production Choice</blockquote><p>Most model comparisons ask a question that nobody actually has: Which model has the highest benchmark score?</p><p>The question I hear in real projects is harder. Which model should I use when the work involves reading a large evidence pack, researching missing information, building a defensible analysis, using tools, and producing something a decision-maker can trust, without making the workflow painfully slow or expensive?</p><p>I built a decision experiment around that question. The subject is a realistic strategy assignment: produce a board-ready competitor landscape for last-mile delivery. The workflow has to identify the market, compare companies and service models, analyze evidence, calculate or verify important claims, and turn the result into a structured deliverable. It is not one prompt and it is not one benchmark. It is a bundle of capabilities that have to work together.</p><p>I compared the highest-ranked representative from ten leading model families across eight dimensions: general intelligence, output economy, generation speed, time to first answer, cost per task, context capacity, input modalities, and deployment openness. The Intelligence Index supplies the broad quality basket, including professional work, agentic tool use, and long-context reasoning. The performance measurements come from Artificial Analysis, which runs the models under a common methodology. Architecture, parameter, modality, and license details come from official model cards and repositories. I designed the workflow, selected the dimensions, normalized the measurements, and applied the scoring scenarios. I did not personally send the underlying benchmark prompts to every model, so this is a reproducible decision analysis built from independent measurements, not a claim that I reran Artificial Analysis’s benchmark suite.</p><p>That distinction is important. The experiment is mine. The raw benchmark observations are independently measured.</p><p><strong>The experiment</strong></p><p>The test asks one practical question:</p><blockquote>If a research and strategy team had to choose one model family for a demanding knowledge-work workflow, which option gives the strongest combination of quality, reliability, speed, cost, and operational freedom?</blockquote><p>I selected one representative per family so that several reasoning settings from the same provider could not crowd out genuinely different model families. Each representative was evaluated in the reasoning mode used by Artificial Analysis. The ranking snapshot and all prices are dated September 2, 2026, because these measurements change as models and providers change.</p><p>The eight dimensions are deliberately different from one another:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4voYHyS6RFc0YlrCm867Pg.png" /></figure><p>No single composite score can express every buyer’s priorities. I therefore calculated three scenarios: a quality-first research team, a balanced production team, and a cost-sensitive high-volume team. The same raw measurements feed all three scenarios. Only the weights change.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tnScozZNfUd2ANk58Tn99A.png" /></figure><p><em>Figure 1. The workflow and eight decision dimensions used in this experiment</em></p><p><strong>How the score works</strong></p><p>The raw metrics use incompatible units. Intelligence is an index score. Generation speed is measured in tokens per second. Latency is measured in seconds. Cost is measured in dollars. Context is measured in tokens. Adding those numbers directly would be meaningless. The Intelligence Index itself is Artificial Analysis’s composite of nine evaluations covering professional work, tool use, terminal coding, scientific reasoning, knowledge, hallucination, and long-context reasoning.</p><p>I converted each metric to a score from 0 to 100 within this ten-family cohort. The best observed value receives 100 and the worst receives zero. For speed, cost, latency, output use, and context, I normalized the base-10 logarithm of the value. This prevents one extreme result, such as Claude Fable’s nearly five-minute wait or Gemini’s very high output speed, from swallowing the rest of the comparison.</p><p>For a metric where higher is better, the normalized score is:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/339/1*US9couuBOp3YWkfnsGsLRQ.png" /></figure><p>For cost, latency, and output use, lower is better, so the result is reversed:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/514/1*Vl4LFZs--fCmvsr5wZG_VQ.png" /></figure><p>These are relative scores, not universal grades. A model receiving 100 for context does not mean it uses every token perfectly. It means it has the largest advertised context among the models in this comparison.</p><p>I treated deployment openness as a practical operational variable. Proprietary models receive zero, models with downloadable weights under restricted or custom commercial terms receive 70, and models with permissive MIT weights receive 100. Modality breadth is based on the number of supported input categories among text, images, audio, and video.</p><p>The scenario weights are explicit editorial judgments about three kinds of work. They were not optimized to maximize any model’s result. Because any fixed weights remain debatable, the sensitivity test later in the article perturbs every weight 5,000 times and reports how often the winner changes.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1tSDJxDP_btO2MZy6_Ez0g.png" /></figure><p>The Intelligence Index already combines nine evaluations, including GDPval-AA v2, tool use, terminal coding, scientific reasoning, knowledge, hallucination, and long-context reasoning. I did not add those component scores again. Doing so would make the analysis look more detailed while quietly counting the same capability several times.</p><p>Artificial Analysis defines cost per task as the weighted total model spend divided by the number of Intelligence Index tasks. It includes input, cache, reasoning, and answer-token charges. A verbose model therefore pays more through this cost metric. I still retain output economy as a separate, lightly weighted dimension because unnecessary output also creates review time, storage, and latency costs for the human workflow. Readers who consider that redundant can set its weight to zero and rerun the supplied script.</p><p><strong>The raw scorecard</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*i8-uxZGSkiW8KY8aGqtEZg.png" /></figure><p><em>Figure 2. The September 2, 2026 source measurements before normalization or weighting</em></p><p>The first methodological surprise is that the highest-scoring Anthropic entry is no longer Claude Opus 5. Claude Fable 5.1 with its default fallback, released on September 1, now scores 66. That entry is a product system that may route requests through a fallback, not a clean observation of one isolated set of weights. I kept it because it is the experience a buyer can select and because the family-level rule chooses the highest-scoring representative. The chart labels the fallback every time so the distinction remains visible.</p><p>Motif 3 creates a different problem. Artificial Analysis reports its intelligence, output use, context, and open-weight status, but not hosted generation speed, latency, or task cost. I did not replace those missing observations with zeros or cohort medians. Its unmeasured cells are labeled N/M in both figures. Motif remains in the raw table and metric matrix, but it is excluded from the three composite rankings. Missing operational evidence is a result, not an invitation to manufacture precision.</p><p><strong>No model wins every dimension</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*lx2l_RzbFfcVRZrlLgfk7A.png" /></figure><p><em>Figure 3. Every dimension normalized within the cohort. Higher is always better</em></p><p>The matrix makes the trade-offs visible before any weighting is applied.</p><p>Claude Fable 5.1 has the highest intelligence score and a one-million-token context, but it is the most expensive complete entry in the group and has by far the longest wait before the answer begins. Gemini 3.7 Flash has a lower intelligence score, but it leads generation speed and output economy, ties for the lowest measured task cost among the proprietary models, supports four input modalities, and still provides a one-million-token context.</p><p>The open-weight models make a different trade. GLM-5.3 and Kimi K3 remain close to the proprietary frontier on intelligence while permitting private deployment and inspection, although their licenses contain restrictions. DeepSeek V4 Pro gives up more raw Intelligence Index points but combines MIT-licensed weights, the lowest measured task cost, the fastest first answer, and a one-million-token context.</p><p>This is why a leaderboard position cannot answer a procurement question. The leaderboard is one column in a decision matrix.</p><p><strong>Three jobs produce three rankings</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*NDiATyqFSKYunqIrJP8hAw.png" /></figure><p><em>Figure 4. Rank movement when the same evidence is evaluated for three different jobs</em></p><p>The complete results are:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*BaIrgV3pT-_3tOvGcWOYrA.png" /></figure><p><strong>Quality-first: Claude wins, but the open models are close</strong></p><p>When intelligence receives 55 percent of the weight, Claude Fable 5.1 wins with a composite score of 70.3. Its Intelligence Index score of 66 creates enough separation to overcome its cost and latency penalties.</p><p>The more interesting result is what follows. GLM-5.3 ranks second at 65.8 and Kimi K3 ranks third at 65.7. Both score 60 on the Intelligence Index, support one-million-token contexts, and offer downloadable weights. GPT-5.6 Sol ranks fourth. If the requirement is maximum measured capability regardless of waiting time and price, Claude is the answer. If private deployment or inspectability matters, the gap closes quickly.</p><p><strong>Balanced production: Gemini becomes the clear leader</strong></p><p>When quality remains important but speed, latency, cost, and operational flexibility receive meaningful weight, Gemini 3.7 Flash moves from sixth to first with a score of 68.4. Muse Spark 1.2 moves from seventh to second. GLM-5.3 takes third and becomes the leading open-weight option.</p><p>This is not because Gemini suddenly becomes more intelligent. Its advantage comes from the full system: 279.4 output tokens per second, $0.40 measured cost per task, 64 million output tokens used across the Intelligence Index, four input modalities, and a one-million-token context. A model can lose the capability column and still win the workflow.</p><p><strong>High-volume: efficiency dominates</strong></p><p>For a high-volume workflow, Gemini’s lead expands to 77.9. Muse remains second and DeepSeek rises from ninth in the quality-first scenario to third. Claude falls to ninth.</p><p>DeepSeek’s movement is especially instructive. It has the lowest intelligence score among the nine models with complete operational data, but it also has the lowest measured cost per task at $0.27, the fastest first answer at 1.65 seconds, permissive MIT weights, and a one-million-token context. At high request volume, those attributes become part of capability because they determine whether the system is economically deployable.</p><p><strong>The cost frontier does not follow the intelligence ranking</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ZwUO6HVQaK5rZ5EGlYvlBg.png" /></figure><p><em>Figure 5. Intelligence versus measured task cost. Bubble area represents output speed</em></p><p>Claude Fable 5.1 sits alone at the top of the capability axis and far to the expensive side. GPT-5.6 Sol and Grok 4.6 share an Intelligence Index score of 61 at roughly one quarter of Claude’s measured task cost. GLM-5.3 and Kimi K3 sit one point below them on intelligence, but the GLM entry costs less per measured task and offers downloadable weights.</p><p>Gemini and Muse occupy another frontier. Their Intelligence Index scores are lower, but both cost $0.40 per measured task and generate output much faster. DeepSeek defines the low-cost, open-weight edge of the comparison.</p><p>On the two-dimensional cost-capability chart, several models are dominated by alternatives with equal or greater intelligence at equal or lower measured cost. The full eight-dimensional decision is less simple because speed, latency, modalities, and openness can reverse those comparisons. There is no single Pareto winner that is simultaneously the most intelligent, cheapest, fastest, most open, and most multimodal. The frontier is a set of defensible trade-offs.</p><p><strong>Fast output can still begin with a long silence</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*UFhFcdOqN8PjfrEo7iB0HA.png" /></figure><p><em>Figure 6. Time to the first answer token on a logarithmic scale</em></p><p>Tokens per second measures how quickly text arrives after generation begins. It does not measure how long a reasoning model takes before showing the answer. Artificial Analysis measures time to the first answer token from the API request. For reasoning models this includes model-side thinking, but it can also reflect provider scheduling, network overhead, and the tested API configuration. It should be read as observed product latency, not as a pure measurement of the neural network alone.</p><p>GPT-5.6 Sol generates at 74.4 tokens per second, slightly faster than GLM-5.3. However, Artificial Analysis measured 100.85 seconds before GPT’s first answer token and 1.87 seconds for GLM. Claude Fable 5.1 reached 66.4 tokens per second once streaming, but its first answer arrived after 296.81 seconds.</p><p>Gemini shows the reverse trade-off. It has the fastest generation speed at 279.4 tokens per second, but the first answer begins after 10.73 seconds. DeepSeek is much slower once generation starts, yet begins answering after 1.65 seconds.</p><p>For an asynchronous research job, a five-minute thinking phase may be acceptable. For an analyst iterating interactively, it changes the product experience completely. Reporting only output speed hides that distinction.</p><p><strong>I tried to break the ranking</strong></p><p>Weighted models can be fragile. A result that depends on intelligence receiving exactly 30 percent instead of 29 percent is not a useful result.</p><p>I reran each scenario 5,000 times. On every run, each dimension’s weight was perturbed independently around its declared value and the weights were normalized back to 100 percent.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2BkQTaB3ofxKdGLxxmklew.png" /></figure><p><em>Figure 7. The share of 5,000 perturbed weight combinations won by each model</em></p><p>Claude Fable 5.1 wins 70 percent of quality-first perturbations. GLM-5.3 wins 16 percent and Kimi K3 wins 10 percent. This is a real lead, but not an absolute one.</p><p>Gemini wins 91 percent of balanced-production perturbations and 99 percent of high-volume perturbations. GLM and DeepSeek split most of the remaining balanced wins. The broad conclusion survives reasonable changes in preference: Gemini is the most robust all-round production choice in this particular experiment, while Claude is the strongest choice only when capability is allowed to dominate the decision.</p><p><strong>The decision table</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*f6acVJppUmccQyzgwPz5kw.png" /></figure><p><strong>What this experiment does not prove</strong></p><p>This analysis does not prove that Gemini will produce the best last-mile delivery report on every run. The underlying benchmarks span many tasks, and the composite scenarios are a decision model rather than a new model-behavior benchmark.</p><p>It also does not remove the effect of the harness. Artificial Analysis controls its methodology, but agent scaffolding, available tools, reasoning settings, and provider infrastructure remain part of the measured system. Claude Fable 5.1 with fallback is the clearest example: its score describes the selectable product configuration, not necessarily one isolated model.</p><p>The openness score is a policy choice, not a natural law. A team that cannot self-host should set its weight to zero. A regulated organization that requires private deployment may treat openness as a gate rather than a weighted preference. The source data and script are retained separately so those choices can be changed without changing the evidence.</p><p>Finally, performance data ages quickly. This analysis uses measurements and product information available on September 2, 2026. The correct way to preserve the experiment is not to freeze its conclusion. It is to preserve the method and rerun it when the data changes.</p><p><strong>The takeaway</strong></p><p>The experiment began with a realistic assignment: research a market, reason across evidence, use tools, verify claims, and deliver something a decision-maker can use.</p><p>The highest leaderboard score did not produce the best answer for every version of that job. Claude won when quality dominated everything else. Gemini won when the model had to be capable, fast, affordable, and multimodal at the same time. GLM-5.3 was the strongest balanced open-weight choice. DeepSeek became compelling when volume and operational freedom mattered most.</p><p>That is the result worth carrying forward. There is no best model outside a workload. There is only the best-supported choice for a particular combination of quality, time, cost, input, and control.</p><p><strong>Further reading</strong></p><blockquote>- [OpenAI: measuring the performance of our models on real-world tasks (GDPval)](https://openai.com/index/gdpval/)</blockquote><blockquote>- [GDPval: evaluating AI model performance on real-world economically valuable tasks (arXiv)](https://arxiv.org/abs/2510.04374)</blockquote><blockquote>- [Artificial Analysis: GDPval-AA v2 leaderboard](https://artificialanalysis.ai/evaluations/gdpval-aa)</blockquote><blockquote>- [Artificial Analysis: full model comparison](https://artificialanalysis.ai/models)</blockquote><blockquote>- [Artificial Analysis: Intelligence Index methodology](https://artificialanalysis.ai/methodology/intelligence-benchmarking)</blockquote><blockquote>- [Artificial Analysis: performance benchmarking methodology](https://artificialanalysis.ai/methodology/performance-benchmarking)</blockquote><blockquote>- [Epoch AI: what do economic value benchmarks tell us](https://epoch.ai/publications/what-do-economic-value-benchmarks-tell-us)</blockquote><blockquote>- [Epoch AI: GDPval benchmark page](https://epoch.ai/benchmarks/gdpval)</blockquote><blockquote>- [Kimi K3 model card](https://huggingface.co/moonshotai/Kimi-K3)</blockquote><blockquote>- [GLM-5.3 model card](https://huggingface.co/zai-org/GLM-5.3)</blockquote><blockquote>- [Qwen3.8 2.4T-A95B model card](https://huggingface.co/Qwen/Qwen3.8-2.4T-A95B)</blockquote><blockquote>- [DeepSeek V4 Pro model card](https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro)</blockquote><blockquote>- [Motif 3 model card](https://huggingface.co/Motif-Technologies/Motif-3)</blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=66787e671e81" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Real AI Model Landscape, Part 2: New Architectures, 16 More Labs, 3 Quiet Exits, and the…]]></title>
            <link>https://medium.com/@anshuljdhingra/the-real-ai-model-landscape-part-2-new-architectures-16-more-labs-3-quiet-exits-and-the-0b6524b2d293?source=rss-d583ee2c95b8------2</link>
            <guid isPermaLink="false">https://medium.com/p/0b6524b2d293</guid>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[ai-agent]]></category>
            <category><![CDATA[architecture]]></category>
            <category><![CDATA[generative-ai-tools]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <dc:creator><![CDATA[Anshul Dhingra]]></dc:creator>
            <pubDate>Mon, 31 Aug 2026 05:50:43 GMT</pubDate>
            <atom:updated>2026-08-31T05:50:43.893Z</atom:updated>
            <content:encoded><![CDATA[<h3>The Real AI Model Landscape, Part 2: New Architectures, 16 More Labs, 3 Quiet Exits, and the Mystery Behind Ox Alpha</h3><h4>Beyond the leaderboards, the more revealing differences are in how models are built, who is still innovating, and who has quietly left the race.</h4><p>The anonymous model I wrote about in Part 1 now has a name. Ox Alpha was GLM-5.3-Flash.</p><p>Z.ai confirmed the connection on August 26, 2026, released the model weights under the MIT license, and exposed enough of the configuration to make the reveal more interesting than the mystery. GLM-5.3-Flash is a 320 billion parameter Mixture of Experts model that activates 18 billion parameters for each token. Its layers combine linear sequence processing with periodic sparse retrieval, while Manifold-Constrained Hyper-Connections carry information around those blocks. The model was trained on a 30 trillion token multimodal corpus. Z.ai describes the released model as running entirely on Chinese AI chips, although its announcement does not break down the training and serving hardware separately.</p><p>That announcement arrived after Part 1 was published, and it changes the arithmetic of this series. Ox Alpha is not a seventeenth additional lab. It is another model from Z.ai, one of the ten labs already covered in Part 1. This article therefore covers sixteen additional labs, then returns to Z.ai for the reveal. Across Parts 1 and 2, the directory contains 26 labs and 27 model entries.</p><p>The distinction matters because this group is more varied than a conventional leaderboard suggests. <strong>Several labs are still pushing genuinely unusual architectures.</strong> Three have quietly stepped away from frontier pretraining. Others are building small models, hardware-specific systems, or enterprise platforms that solve a different problem from Claude or GPT. A useful map of the field has to show those differences instead of placing every company in one undifferentiated race.</p><p><strong>Not every big name is still racing</strong></p><p>Three labs in this directory have effectively left the frontier pretraining race, and none of them announced it with a press release. It shows up only if you look at what they have actually shipped since.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9fjFMOIMjt_ngrQwOzs6rw.png" /></figure><p><em>Figure 1. Three labs, once discussed alongside frontier releases, that have since stepped back from building new foundation models.</em></p><p>Databricks retired DBRX from its own Foundation Model APIs in April 2025 with no successor announced, and shifted its product focus to TAO, a reinforcement-learning-based fine-tuning method for improving existing open models, and to hosting third-party models like Anthropic’s. Snowflake never followed Arctic with anything, and now positions Snowflake Cortex as a place to run OpenAI, Anthropic, Meta, and Mistral models rather than its own. 01.AI explicitly stopped pretraining new foundation models in March 2025 and became an enterprise solutions business, selling customized deployments built on a mix of its own older Yi models and third-party models, including DeepSeek’s, under an enterprise platform called Wanzhi.</p><p>None of this makes these companies unimportant. It means the honest description of what they do today is systems integration and tooling, not frontier model research, and a comparison article that lists them as peers of Anthropic or DeepSeek without that context is misleading by omission.</p><p><strong>Major open-weight labs, US and EU</strong></p><p><strong>AI21 Labs, Jamba 2</strong></p><p>Jamba is the clearest example in this whole series of a model that is <em>not a standard transformer</em>. It interleaves blocks of ordinary attention layers with Mamba layers, a fundamentally different mechanism borrowed from state-space models, and adds Mixture of Experts on top of that. Jamba2 Mini takes this furthest: 28 of its 32 layers are Mamba, not attention, which <strong>changes the memory math for running the model in ways a standard architecture cannot match</strong>. The Jamba 1.5 generation reached 256,000 tokens of effective context, the largest of any open-weight model at the time, and reportedly ran three times the throughput of Mixtral at 128,000 tokens of context. Jamba 2 shipped January 8, 2026 under the Apache 2.0 license, with its weights published openly.</p><p><strong>Liquid AI, LFM2</strong></p><p>Liquid AI is not building transformers at all. Its Liquid Foundation Models are<strong> built on ideas from dynamical systems theory and signal processing rather than the standard attention mechanism</strong>, and the company is explicit that this is a deliberate, from-scratch architectural choice, not a variation on an existing design. LFM2 ships in small sizes, 350 million to 3 billion parameters, aimed squarely at on-device and edge deployment: phones, laptops, vehicles, running on CPU, GPU, or NPU. Despite the small size, LFM2 is natively multimodal across text, audio, image, and video. It reportedly decodes twice as fast as Qwen3 on CPU. This is the one lab in this series building for constrained hardware first, not leaderboard scores, and it publishes LFM2’s weights openly.</p><p><strong>Cohere, Command A+</strong></p><p>Command A+ (released May 20, 2026) is a <strong>sparse Mixture of Experts model</strong>, 218 billion total parameters with 25 billion active, using 128 routed experts plus one shared expert that always runs. Its most distinctive choice is in attention, not routing: Command A+ alternates <strong>sliding-window attention with full attention in a fixed three-to-one ratio and drops positional embeddings from the design entirely</strong>. Worth flagging honestly: its context window, 128,000 tokens input and 64,000 output, is actually smaller than the earlier dense Command A’s 256,000 tokens, a real trade-off rather than a straightforward upgrade. Cohere publishes Command A+ under the Apache 2.0 license, open weight, a change from Command A’s earlier, more restrictive source-available license.</p><p><strong>Databricks, DBRX</strong></p><p>DBRX (March 2024) was a <strong>fine-grained Mixture of Experts model</strong>, 132 billion total parameters with 36 billion active, using 16 experts with 4 routed per token, more fine-grained than the 8-experts-top-2 design common at the time. It was trained on 12 trillion tokens with a 32,768 token context window. As covered above, Databricks retired DBRX from its own platform in April 2025 with no successor, and its custom license explicitly restricts using DBRX or its outputs to train competing models, an unusual, non-standard term worth knowing before building on it. The weights are published openly, but under that custom license rather than a standard open source one.</p><p><strong>Enterprise and cloud-native labs</strong></p><p><strong>IBM, Granite 4.1</strong></p><p><strong>Granite 4.1 is not one model but a coordinated family of ten</strong>: six language models at 3 billion, 8 billion, and 30 billion parameters, each in full precision and FP8, plus a dedicated safety model, a vision-language model built for document extraction, and a multilingual speech recognition model. The design goal throughout is efficiency and enterprise deployment rather than leaderboard-topping scale, and IBM publishes the entire family as open weight.</p><p><strong>Nvidia, Nemotron 3</strong></p><p>Nemotron 3’s Nano Omni variant <strong>uses a Mamba2-Transformer hybrid Mixture of Experts design</strong> and activates only about 3 billion parameters, yet handles video up to two minutes, audio up to one hour, images, and text, all from a single model. Earlier Nemotron-4 models were built directly on Llama 3’s architecture lineage at 340 billion, 70 billion, and 15 billion parameter sizes, and Nvidia publishes the Nemotron family openly.</p><p><strong>Amazon, Nova</strong></p><p>Nova is built and s<strong>erved natively inside Amazon Bedrock</strong>, with a companion Nova Forge SDK for enterprise customers to fine-tune and customize the models on their own data. Amazon has disclosed comparatively little about Nova’s underlying architecture publicly, consistent with its positioning as a cloud platform product rather than a research release aimed at leaderboard visibility. Nova’s weights are not published; it is reachable only through Amazon Bedrock, keeping it closed like the other frontier cloud-platform models in this series.</p><p><strong>Microsoft, Phi</strong></p><p>The current flagship, Phi-4-reasoning-vision-15B (March 2026), combines a dense 14 billion parameter text backbone with a separate vision encoder, SigLIP-2, fused in afterward, roughly 15 billion parameters total. It is an adaptive reasoning model:<strong> it decides for itself at inference time whether a question needs an explicit chain of thought or a direct answer</strong>, rather than always doing one or the other. A smaller family member, Phi-4-mini-flash-reasoning at 3.8 billion parameters, uses a genuinely different hybrid decoder architecture called SambaY with Gated Memory Units for sharing information across layers, a real departure from a standard transformer, not just a shrunk-down copy of a bigger model. Microsoft publishes the whole Phi family as open weight under the MIT license, and leans on distillation heavily throughout its training recipe.</p><p><strong>Snowflake, Arctic</strong></p><p>Arctic (April 2024) combines a shared 10 billion parameter <strong>dense transformer with a residual Mixture of Experts layer</strong>, 480 billion total parameters with 17 billion active, a genuinely unusual dense-plus-MoE hybrid rather than making every layer an expert layer. Its context window at release, 4,096 tokens, was short even for its era. As covered above, Snowflake never released a successor and now positions itself as a host for other companies’ frontier models instead. Its one addition since, Arctic-Extract, is built on Qwen 2.5-VL’s architecture rather than a from-scratch Snowflake design. Snowflake published Arctic itself as open weight under Apache 2.0.</p><p><strong>Chinese labs</strong></p><p><strong>Baidu, ERNIE 5.0</strong></p><p>ERNIE 5.0 is a 2.4 trillion parameter omnimodal flagship, and its most distinguishing fact is not the architecture but the training stack: <strong>it was trained on Baidu’s own Kunlun chips rather than Nvidia hardware, tightly integrated with Baidu’s search engine</strong>. Baidu open-sourced the smaller ERNIE 4.5 series on June 30, 2026, a real shift toward openness from a company that had kept its models closed. ERNIE 5.0 itself, the flagship profiled here, remains closed weight; only the smaller 4.5 series was actually released openly.</p><p><strong>Tencent, Hunyuan A13B</strong></p><p>Hunyuan A13B is an 80 billion total parameter <strong>Mixture of Experts model with 13 billion active</strong>, released as open weights in mid-2025.</p><p><strong>ByteDance, Doubao (Seed)</strong></p><p>ByteDance’s Seed model family powers Doubao, China’s most-used consumer AI app with more than 155 million weekly active users. Its <strong>standout trait is not a novel architecture but extreme cost efficiency</strong>: Seed models are reportedly priced roughly ten times cheaper than GPT-class models at comparable quality, a genuinely different competitive strategy from chasing benchmark scores. ByteDance keeps the Seed models closed, accessible only through Doubao and its own API, not published as downloadable weights.</p><p><strong>MiniMax, M2.7</strong></p><p>MiniMax open-sourced its M2.7 flagship on April 12, 2026, <strong>scoring 56.22 percent on SWE-Pro and 57.0 percent on Terminal Bench 2</strong>. The company itself is a notable data point: it completed an HKEX listing in 2026, making it one of the few publicly traded frontier model labs anywhere. MiniMax published M2.7’s weights openly alongside the release.</p><p><strong>01.AI, Yi</strong></p><p>As covered above, 01.AI exited frontier pretraining in March 2025. Its last frontier-scale release, Yi-Lightning, was a roughly 100 billion parameter Mixture of Experts model from October 2024, with no publicly disclosed active-parameter count or expert configuration, and unlike its smaller predecessors, Yi-Lightning itself was never released as open weight. The company’s current product, <strong>an enterprise agent orchestration platform called Wanzhi, is built on a mix of its own older open models</strong> (Yi-6B, Yi-34B, Yi-1.5) and third-party models rather than new frontier research.</p><p><strong>Huawei, openPangu 2.0</strong></p><p>openPangu 2.0 is the other genuinely novel architecture in this series alongside AI21’s Jamba. Its Flash variant runs 92 billion total parameters with only 6 billion active, using<strong> a hybrid attention design that combines Multi-head Latent Attention with a second mechanism called DSA and sliding-window attention in a fixed ratio, plus multi-token prediction and a four-stream design that replaces standard residual connections</strong>. The Pro variant scales this to 505 billion total and 18 billion active parameters. Both support a 512,000 token context window. The detail that matters most for understanding Huawei’s strategy: the entire Pangu line is trained natively on Huawei’s own Ascend NPUs rather than Nvidia GPUs, one of the only frontier-scale training stacks in the world built independently of Nvidia hardware. Huawei publishes openPangu 2.0 as open weight, under its own custom license rather than a standard one.</p><p><strong>Specialized, narrow-focus labs</strong></p><p><strong>Poolside, Laguna S 2.1</strong></p><p>Poolside builds exclusively for agentic coding, not a general-purpose assistant, which makes it a useful counterexample to every other lab in this series. Laguna S 2.1 is a 118 billion total parameter Mixture of Experts model with only 8 billion active, a 1 million token context window, and it is reported to match or beat open models several times its size specifically on long-horizon software engineering tasks. Poolside calls <strong>its internal tooling a Model Factory, built to automate the engineering and plumbing around model development so its research time goes toward the questions that actually matter for coding agents</strong>. Poolside publishes Laguna’s weights openly as well.</p><p><strong>The reveal: Ox Alpha was GLM-5.3-Flash</strong></p><p><strong>Z.ai, GLM-5.3-Flash</strong></p><p>OpenRouter began serving Ox Alpha on August 20 as a free anonymous preview. Six days later, Z.ai introduced the production model as GLM-5.3-Flash and explicitly said it had previously been previewed under the Ox Alpha name. The release turns what had been informed fingerprinting into a confirmed identity.</p><p>Z.ai’s release notes say GLM-5.3-Flash starts from a newly trained base model rather than being a compressed version of GLM-5.2. Its published `config.json` exposes the detailed design. The <strong>45 text layers follow a repeating pattern of three Kimi Delta Attention style linear-attention layers followed by one DeepSeek-style sparse-attention layer</strong>. The linear layers maintain a compact recurrent state, which reduces the key-value cache required by dense attention. The sparse layers use an indexer configured to select up to 2,048 relevant earlier positions, preserving more precise retrieval without paying for dense attention over the complete context. The same configuration sets `hc_mult` to four, meaning mHC carries four residual streams. Sinkhorn normalization constrains how those streams are mixed, which is designed to limit unstable signal amplification as the network scales.</p><p>The model is also a sparse Mixture of Experts. It contains 320 billion parameters in total, but Z.ai reports that approximately 18 billion are active for each token. This reduces arithmetic relative to a dense model of the same total size, although all of the weights still have to be stored and expert routing still creates communication costs. The released configuration supports a context limit of 1,048,576 tokens and includes a vision encoder, making this the first natively multimodal model in the GLM-5 series.</p><p><strong>Z.ai reports that the new attention design uses 4.44 times less key-value cache per layer and about three times less attention compute under its comparison</strong>. Those are architecture-level figures, not guarantees of equivalent end-to-end speedups. Total serving cost still includes model-weight movement, expert communication, batching, vision processing, and hardware utilization. The company’s separate claim that GLM-5.3-Flash is offered at one-tenth the price of GLM-5.2 is an API price comparison, not a direct measurement of the underlying hardware cost.</p><p>The broader lesson is not that one lab copied another. It is that open research lets labs combine validated ideas instead of repeating every experiment independently. G<strong>LM-5.3-Flash brings together linear attention associated with Moonshot, sparse attention associated with DeepSeek, mHC residual design, sparse expert routing, and a new multimodal training corpus.</strong> That kind of rapid composition may be one of the most important economic advantages in the current Chinese open-model ecosystem.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2ZgJff_QZM977X_mk41n0g.png" /></figure><p><em>Figure 2. Sixteen additional labs, grouped by category, followed by the confirmed Ox Alpha reveal from Z.ai.</em></p><p><strong>All 27 model entries, side by side</strong></p><p>The ten model entries from Part 1, the sixteen additional labs in this part, and GLM-5.3-Flash as the confirmed identity of Ox Alpha appear in one table. This produces 27 model entries from 26 distinct labs.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*hvcCvMqDxo_-n_YKtD1lvQ.png" /></figure><p><em>Twenty-seven model entries from 26 labs, shown side by side. GLM-5.3-Flash replaces the former anonymous Ox Alpha entry.</em></p><p>Sixteen additional labs and one timely reveal tell several different stories. Some teams are pushing architecture that few others have tried. Some have quietly stopped competing at the frontier. Z.ai has shown how quickly ideas from several open research programs can be assembled into a new model. Knowing which story a model is actually telling matters more than memorizing its parameter count, and that is the harder, more useful comparison this part was built to make.</p><p><strong>What comes next</strong></p><p>That completes the 27 model entries. Part 3 stops comparing specifications and puts one real professional task, the kind of deliverable a real analyst would actually be asked to produce, in front of as many of these models as anyone has independently tested. Every available result is graded under the same methodology, and models without comparable evidence are labeled clearly rather than assigned an invented ranking.</p><p><strong>Further reading</strong></p><blockquote>- [AI21: Jamba, a hybrid Transformer-Mamba language model](https://www.ai21.com/research/jamba-a-hybrid-transformer-mamba-language-model/)</blockquote><blockquote>- [Liquid AI: introducing LFM2, the fastest on-device foundation models](https://www.liquid.ai/blog/liquid-foundation-models-v2-our-second-series-of-generative-ai-models)</blockquote><blockquote>- [Cohere Command A+ model card](https://huggingface.co/CohereLabs/command-a-plus-05-2026-bf16)</blockquote><blockquote>- [Databricks: announcing DBRX, a new standard for open LLMs](https://www.databricks.com/blog/announcing-dbrx-new-standard-efficient-open-source-customizable-llms)</blockquote><blockquote>- [Microsoft Research: Phi-4-reasoning-vision and the lessons of training a multimodal reasoning model](https://www.microsoft.com/en-us/research/blog/phi-4-reasoning-vision-and-the-lessons-of-training-a-multimodal-reasoning-model/)</blockquote><blockquote>- [Snowflake: Arctic, open and efficient foundation language models](https://www.snowflake.com/en/blog/arctic-open-efficient-foundation-language-models-snowflake/)</blockquote><blockquote>- [Huawei: openPangu-2.0-Flash model card](https://huggingface.co/openpangu/openPangu-2.0-Flash)</blockquote><blockquote>- [Z.ai: GLM-5.3-Flash model card and released weights](https://huggingface.co/zai-org/GLM-5.3-Flash)</blockquote><blockquote>- [Z.ai: GLM-5 series repository](https://github.com/zai-org/GLM-5)</blockquote><blockquote>- [Alibaba unveils its largest AI model yet as China closes the gap](https://www.forbes.com/sites/gabrielalinzainescu/2026/08/03/alibaba-unveils-its-largest-ai-model-yet-as-china-closes-the-gap/)</blockquote><blockquote>- [Poolside: introducing Laguna S 2.1](https://poolside.ai/blog/introducing-laguna-s-2-1)</blockquote><blockquote>- [Artificial Analysis: AI model and API provider comparisons](https://artificialanalysis.ai/)</blockquote><blockquote>- [Vals AI: real-world, expert-graded model benchmarks](https://www.vals.ai/home)</blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=0b6524b2d293" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Real AI Model Landscape: the concepts nobody explains, and the models actually worth…]]></title>
            <link>https://medium.com/@anshuljdhingra/the-real-ai-model-landscape-the-concepts-nobody-explains-and-the-models-actually-worth-7e6e7ab2e35a?source=rss-d583ee2c95b8------2</link>
            <guid isPermaLink="false">https://medium.com/p/7e6e7ab2e35a</guid>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[ai-agent]]></category>
            <category><![CDATA[generative-ai-tools]]></category>
            <category><![CDATA[agentic-ai]]></category>
            <dc:creator><![CDATA[Anshul Dhingra]]></dc:creator>
            <pubDate>Tue, 25 Aug 2026 04:21:09 GMT</pubDate>
            <atom:updated>2026-08-25T04:21:09.171Z</atom:updated>
            <content:encoded><![CDATA[<h3>The Real AI Model Landscape: the concepts nobody explains, and the models actually worth understanding</h3><p>Everyone ranks AI models. Almost nobody tests what they are actually good at.</p><p>A new model claims the top spot on some leaderboard almost every week, and almost none of those claims mean what they sound like they mean. Which LLM is actually the best is the question everyone asks and almost nobody answers honestly, because that question does not have one answer, and pretending it does is how most comparisons online go wrong. A model’s rank depends on which leaderboard you are looking at, what task you are testing it on, and even which harness, the tool-use scaffolding wrapped around it, was used to run the test. Two people running the exact same model can get different scores on the exact same benchmark, not because the model changed, but because the setup around it did.</p><p>This is the first of a three-part series meant to work as a genuine reference. This part covers the concepts that actually explain why models differ, a real look at how the major leaderboards work, and deep profiles of the ten models currently leading them. Part 2 covers seventeen more labs building genuinely different things, including a few names that quietly stopped building frontier models altogether, and one live model nobody has officially claimed yet. Part 3 puts one real professional task in front of as many of the 27 models as anyone has actually tested that way, and reports the real, measured results.</p><p>Let’s start with the concepts, because without them, every comparison that follows is just names and numbers with nothing to hang them on.</p><h3>The concepts you need before comparing anything</h3><p>Every model on the market is a variation on the same basic recipe. Once you understand the recipe, and the handful of places labs actually diverge from each other, the rest of this series will make a lot more sense.</p><h3>Dense versus Mixture of Experts</h3><p>Every transformer model is built from layers of a specific calculation repeated over and over. In the simplest version, called a dense model, every parameter in the model does some work on every token you feed it. If a dense model has 175 billion parameters, all 175 billion of them activate every time.</p><p>Mixture of Experts, or MoE, changes that. Instead of one enormous block of parameters that always runs in full, the model is split into many smaller blocks called experts, and a small router decides, for each token, which one or two experts actually need to run. A model can have hundreds of billions of parameters in total and only use a small fraction of them for any single token.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Z8Q26Vui8EGIuXpUfMp74Q.png" /></figure><p><em>Figure 1. A dense model runs every parameter on every token. A Mixture of Experts model routes each token to only a few of its many experts.</em></p><p>This is why you will see models described with two different parameter counts: total parameters and active parameters. DeepSeek V4 has 671 billion total parameters but only about 37 billion active per token. Mistral Large 3 has 675 billion total and 41 billion active. The total number tells you how much the model knows overall. The active number tells you how much compute and memory it actually costs to run. Confusing the two is one of the most common mistakes people make when comparing models.</p><h3>Attention variants</h3><p>Standard transformer attention has a well known cost problem: as the amount of context grows, the memory needed to hold onto it, called the key-value cache, grows right along with it. Several labs have built genuinely different attention mechanisms specifically to fix this.</p><p>DeepSeek’s Multi-head Latent Attention compresses that cache into a smaller latent representation before storing it, cutting memory cost without giving up long context. AI21’s Jamba and Nvidia’s Nemotron go further and interleave standard attention layers with Mamba layers, a different mechanism borrowed from state-space models that scales more efficiently with sequence length. Cohere’s Command A+ takes a third approach, alternating sliding-window attention with full attention in a fixed ratio and dropping positional embeddings entirely. These are not marketing differences. They change how much a model costs to run at long context and how it behaves as that context grows.</p><h3>Context window mechanics</h3><p>A context window is the maximum number of tokens a model can consider at once, and vendors have been racing to make this number bigger: 256,000 tokens for Mistral Large 3, 1 million for Qwen3.8-Max and DeepSeek V4, 10 million for Llama 4 Scout.</p><p>A bigger number does not mean the model reads that entire window equally well. Getting a model to accept a million tokens as input is a different engineering problem than getting it to actually notice and use a fact buried in the middle of that input. This gap between advertised context and reliable recall is well documented across the field, sometimes called the lost in the middle effect, and it is one of the clearest reasons a raw context window number should never be read as a promise about how a model will actually behave on a long real document.</p><h3>Pretraining versus post-training</h3><p>Every model goes through two very different stages. Pretraining is the part most people picture: the model reads a huge amount of text and code, trillions of tokens, and learns to predict the next token. This stage is where most of a model’s raw knowledge and reasoning ability comes from, and it is also the most expensive part by far.</p><p>Post-training is everything that happens after, and it is where a raw, pretrained model turns into something that will actually follow instructions, refuse harmful requests, and hold a conversation. This includes supervised fine-tuning, showing the model labeled examples of good responses, reinforcement learning from human or AI feedback, direct preference optimization as a more efficient alternative to full reinforcement learning, and Constitutional AI, training a model against a written set of principles rather than only human ratings. Two models can share nearly the same pretraining data and still behave completely differently because of choices made in this second stage.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wLCCNzNA-nW1uUWgDsHX3g.png" /></figure><p><em>Figure 2. The two-stage recipe behind every model, with reasoning effort and distillation as the two paths that branch off it.</em></p><h3>Distillation</h3><p>Distillation takes a large, expensive teacher model and uses its outputs to train a smaller, cheaper student model to imitate it. The student never sees the original training data directly. It learns from the teacher’s behavior instead, which can get a small model most of the way to a much larger model’s quality at a fraction of the cost to run.</p><p>DeepSeek made this mainstream by distilling its R1 reasoning model into several much smaller dense models that kept a surprising amount of R1’s reasoning ability. Microsoft’s Phi family leans on distillation as a core part of its training recipe rather than an afterthought. Anthropic’s Claude Fable 5 is documented as using Opus 4.8 as a fallback, a sign of this same large-to-small relationship inside a single company’s own model family.</p><h3>Reasoning and inference-time compute</h3><p>For most of the history of language models, making a model better meant training a bigger one. Reasoning models add a second lever: spending more compute at the moment you ask the question, not just during training.</p><p>Some models let you dial this directly. Claude Opus 5 offers adjustable reasoning effort, max, high, or xhigh. Grok 4.6 offers four levels, low, medium, high, and xhigh. Others ship reasoning as a separate, dedicated line: DeepSeek’s R-series, Mistral’s Magistral, and Microsoft’s Phi-4-reasoning, which decides for itself at inference time whether a given question needs an explicit chain of thought or a direct answer. Either way, the same underlying model can score very differently on a hard benchmark depending on how much thinking time it was allowed to spend, and that is a separate, measurable axis from the model’s raw size.</p><h3>Harness</h3><p>A harness is the tool-use scaffolding wrapped around a model: what tools it can call, how errors get fed back to it, how many attempts it gets. Benchmarks like SWE-bench Verified, which tests whether a model can resolve a real GitHub issue with a working, test-passing code patch, and Terminal-Bench, which tests agentic command-line work, do not just test a model, they test a model inside a specific harness, and the harness itself has a measurable effect on the score.</p><p>This matters directly for reading leaderboards. When you see a coding benchmark score, you are seeing the combined result of the model and whatever agent scaffolding was wrapped around it for that specific test. A different harness around the exact same model can move the score meaningfully, which is part of why the same model sometimes reports different numbers on different sites.</p><h3>Multimodality: native versus bolted on</h3><p>Multimodal models handle more than text, usually images, sometimes audio or video. The important distinction is not whether a model is multimodal, most major ones now are, but whether that capability was trained in from the start or added afterward.</p><p>Google’s Gemini and Meta’s Llama 4 are natively multimodal: every modality was part of pretraining from day one. Microsoft’s Phi-4-reasoning-vision takes the other approach, a dense text model with a separate vision encoder, SigLIP-2, attached and fused in afterward. Native multimodality tends to handle cross-modal reasoning, a question that requires connecting something in an image to something in the text, more naturally. Bolted-on multimodality can still be very capable, but it is solving a different, more constrained engineering problem.</p><h3>Open weight versus closed</h3><p>Open weight means the trained parameters themselves are published for anyone to download and run. It does not mean the training data, the exact training recipe, or the full decision-making behind the model is public. Meta, Mistral, DeepSeek, Alibaba, and most of the Chinese labs in this series publish open weights. Anthropic, OpenAI, Google, and xAI do not.</p><p>Licensing terms vary more than people expect, too. Cohere’s Command A+ ships under a standard Apache 2.0 license. Databricks’ DBRX ships under a custom license that explicitly restricts using it to train competing models. Huawei’s openPangu uses its own custom license as well. Open is not one thing, and the specific terms are worth reading before you build on top of any of these.</p><h3>Stealth models</h3><p>Occasionally a lab ships a genuinely new model without attaching its name to it. OpenRouter has been serving one of these, called Ox Alpha, since August 20: a free, anonymous preview, reasoning-focused, with a 1 million token context window and strong early coding benchmark results. Nobody has officially claimed it. Independent fingerprinting by developers who tested it points toward it likely being Zhipu AI’s GLM-5.3 Flash, tested under a code name before a formal announcement, though that identification has not been confirmed by Zhipu.</p><p>This is worth knowing because it happens more than once. Labs use stealth releases to get real usage data and feedback before committing their brand to a model, and if you spend any time on OpenRouter, you will eventually run into one.</p><h3>How the leaderboards actually work</h3><p>Three sources are doing most of the useful public tracking of model performance right now, and they are measuring three different things.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ZfbZfywu_216UeBOi_dIWQ.png" /></figure><p><em>Figure 3. Three different questions, three different leaderboards.</em></p><p>Artificial Analysis publishes an Intelligence Index built from nine separate evaluations. These include GDPval-AA v2, which grades models on real economically valuable work and gets a full worked example of its own in Part 3 of this series; tau-cubed-Banking, a banking-domain agentic task; Terminal-Bench, which tests agentic command-line work; SciCode, a coding-for-science benchmark; Humanity’s Last Exam, a set of expert-level questions spanning many fields; GPQA Diamond, graduate-level science reasoning; and three further evaluations, CritPt, AA-Omniscience, and AA-LCR. Every model is run against the same fixed set of tests, which makes it a genuinely apples-to-apples comparison, as long as you remember it is only measuring what those nine tests measure.</p><p>The Vals Index takes a different approach. Instead of a single fixed test set, Vals builds domain-specific benchmarks around real professional work: banking, corporate finance, law, and cybersecurity, and grades the outputs against professional standards using actual domain experts rather than automated scoring alone. Vals also retires benchmarks once models start saturating them, treating evaluation as something that has to keep moving rather than a fixed yardstick. This is the more business-relevant of the three, and the same underlying idea, grading a model’s real work against a real professional standard, is exactly what Part 3 of this series builds its worked example around, using a different provider’s version of that approach.</p><p>OpenRouter’s rankings answer a completely different question: not how good is a model, but how much are developers actually paying to use it. That is a signal about price, speed, and real adoption, not a quality score, and it is easy to mistake one for the other if you are not paying attention to which leaderboard you are looking at.</p><p>None of these three is the real ranking that the other two are approximating. They are answering different questions, and a serious comparison needs to know which question it is actually asking.</p><h3>The top 10, profiled</h3><h3>Anthropic, Claude Opus 5</h3><p>Anthropic’s flagship currently sits at the top of both the Artificial Analysis Intelligence Index and the Vals Index, the only model in this series to lead both. Its standout feature is adjustable reasoning effort: the same model can run at max, high, or xhigh effort depending on how much thinking time a task actually needs, rather than forcing every question through the same fixed amount of computation. It dominates real coding work, scoring 97 percent on SWE-bench Verified, and it also leads the GDPval-AA v2 ranking explored in depth in Part 3 of this series, a benchmark that grades models on real professional deliverables rather than an automated score. Anthropic has not disclosed Opus 5’s architecture publicly. The Claude family also includes Sonnet 5, a faster mid-tier model, Fable 5, a smaller model documented as using Opus 4.8 as a fallback, and Haiku 4.5, the fastest and cheapest tier. Anthropic keeps all of these models closed, releasing access through its API rather than publishing the underlying weights.</p><h3>OpenAI, GPT-5.6</h3><p>GPT-5.6, seen in the wild under variant names like Sol and Terra, consistently places in the top five across every major leaderboard behind this series, without being the single top scorer on any of them. OpenAI, consistent with its history, has not disclosed the model’s architecture, parameter count, or training data composition. What is publicly known comes entirely from third-party benchmarking rather than from OpenAI itself, which is worth remembering whenever an article describes GPT-5.6’s architecture with specific numbers. Like Anthropic, OpenAI keeps GPT-5.6’s weights closed.</p><h3>Google, Gemini 3.7</h3><p>Gemini is built natively multimodal from pretraining onward, rather than having vision or audio bolted on afterward, which is Google’s most consistently distinguishing architectural choice across the whole Gemini line. The Flash variant, Gemini 3.7 Flash, is not simply a smaller, cheaper version of the full model, it is specifically tuned for speed and cost, and it placed third on the Vals Index overall. Google has not disclosed detailed architecture specifics for 3.7, and like the other frontier labs profiled so far, it keeps Gemini’s weights closed.</p><h3>xAI, Grok 4.6</h3><p>Grok 4.6, from xAI, now operating under the broader SpaceXAI brand, is built specifically for long-running agentic work, with a 500,000 token context window and configurable reasoning effort across four levels: low, medium, high, and xhigh. Its immediate predecessor, Grok 4.5, is built on a V9 architecture with roughly 1.5 trillion parameters, described by Elon Musk as roughly comparable to Opus 4.7, but much faster, and more token-efficient. Grok 4.6 placed fourth on the Vals Index, and xAI keeps its weights closed as well.</p><h3>Meta, Llama 4</h3><p>Llama 4 comes in two main variants, Scout and Maverick, both Mixture of Experts models with 17 billion active parameters, but very different total sizes and expert counts: Scout uses 16 experts and supports an unusually large 10 million token context window, while Maverick uses 128 experts across 400 billion total parameters. Both alternate dense and MoE layers rather than making every layer an expert layer, and both are natively multimodal. Llama remains the most widely deployed open-weight family in the world, largely because it was first to reach genuine production quality at scale, and because Meta publishes its weights openly for anyone to download and run.</p><h3>Mistral AI, Large 3</h3><p>Mistral Large 3 is a fat but sparse Mixture of Experts model, 675 billion total parameters with only 41 billion active per token, and a 256,000 token context window. Mistral runs Large 3 alongside two more specialized lines built on the same foundation: Magistral, a dedicated reasoning model with transparent chain-of-thought that works natively across eight languages, and Devstral, an agentic coding model, and Mistral publishes all three as open weight.</p><h3>DeepSeek, V4</h3><p>DeepSeek V4 is the model most responsible for popularizing efficient long-context Mixture of Experts design outside of the largest labs. It runs 671 billion total parameters with only 37 billion active, trained on 32 trillion tokens, and supports a 1 million token context window. Its defining technical feature is Multi-head Latent Attention, a compressed attention mechanism DeepSeek introduced in V3 and refined further in V4, combined with a new hybrid attention system in this generation. DeepSeek is also the highest profile example of a lab whose reasoning models were distilled down into much smaller dense models that retained a surprising share of the original’s reasoning ability, and DeepSeek publishes V4’s weights openly.</p><h3>Alibaba, Qwen3.8-Max</h3><p>Qwen3.8-Max is Alibaba’s largest model to date, 2.4 trillion parameters with a 1 million token context window, and notably the first Max-class Qwen model Alibaba has ever open-sourced, a genuine change in strategy from keeping its largest models closed. Alibaba has reported the model completing 16 days of largely unsupervised, autonomous coding work in an internal test, writing, testing, and fixing its own code with minimal human input, and Alibaba has published the model’s weights openly.</p><h3>Moonshot AI, Kimi K3</h3><p>Kimi K3 is currently the highest-ranked open-weight model on the Artificial Analysis Intelligence Index, and it placed fifth overall on the Vals Index, ahead of several closed models. Moonshot AI has published comparatively little detail about K3’s internal architecture relative to labs like DeepSeek or Alibaba, which is worth noting as a genuine gap rather than filling it in with assumptions, though the weights themselves are published openly.</p><h3>Zhipu / Z.ai, GLM 5.2</h3><p>GLM 5.2 is the leading open-weight model on the Vals Index, scoring 65.02 percent, ahead of every other open model tracked in this series. Zhipu is also the lab most plausibly connected to Ox Alpha, the stealth model covered in the concepts section: independent fingerprinting suggests Ox Alpha is likely an unannounced GLM-5.3 Flash, though Zhipu has not confirmed this. GLM 5.2 itself is published as open weight.</p><h3>The top 10, side by side</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*fDmIi9ldlKg8xDrlbgbFFA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*x8J8ubDu8P7lUt7RQFa3Tg.png" /></figure><p><em>Figure 4. The top 10, split by whether their weights are published.</em></p><p>Ten models, ten different bets on what actually matters: raw scale, cost efficiency, long context, safety, or simply being first to market. None of those bets is correct in isolation. That is the entire point of learning the concepts before memorizing the names. Once dense versus sparse, harness, and open weight actually mean something to you, a leaderboard rank stops being an answer and starts being a question you can actually interrogate.</p><h3>What comes next</h3><p>That covers the concepts and the ten models currently leading the leaderboards. Part 2 of this series covers the other seventeen labs building genuinely different things, including a few names that quietly stopped building frontier models altogether and one live model nobody has officially claimed yet, and puts all twenty-seven side by side in one comparison table. Part 3 puts one real professional task, graded the same way for every model, in front of as many of the 27 as anyone has actually tested this way, and reports exactly which models have a real answer and which do not.</p><h3>Further reading</h3><p>· <a href="https://artificialanalysis.ai/">Artificial Analysis: AI model and API provider comparisons</a></p><p>· <a href="https://www.vals.ai/home">Vals AI: real-world, expert-graded model benchmarks</a></p><p>· <a href="https://openrouter.ai/">OpenRouter: usage rankings and model catalog</a></p><p>· <a href="https://arxiv.org/pdf/2606.19348">DeepSeek-V4: towards highly efficient million-token context intelligence</a></p><p>· <a href="https://ai.meta.com/blog/llama-4-multimodal-intelligence/">The Llama 4 herd: natively multimodal AI innovation</a></p><p>· <a href="https://www.fullstack.com/labs/resources/blog/grok-4-5-a-closer-look-at-xais-latest-model">Grok 4.5: how xAI’s new model competes on code and cost</a></p><p>· <a href="https://serenitiesai.com/articles/mistral-ai-models-2026-complete-guide">Mistral AI models 2026: Large 3, Magistral, Devstral 2</a></p><p>· <a href="https://www.forbes.com/sites/gabrielalinzainescu/2026/08/03/alibaba-unveils-its-largest-ai-model-yet-as-china-closes-the-gap/">Alibaba unveils its largest AI model yet as China closes the gap</a></p><p>· <a href="https://www.explainx.ai/blog/ox-alpha-what-we-know-mystery-ai-model-august-2026">Ox Alpha: what we know about the mystery AI model</a></p><p>· <a href="https://www.orcarouter.ai/blog/ox-alpha-stealth-model-what-we-know">Ox Alpha likely GLM-5.3: the mystery model’s fingerprint</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7e6e7ab2e35a" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Harness, Loop, Graph: The Three Layers Every Reliable AI Agent Needs]]></title>
            <link>https://medium.com/@anshuljdhingra/harness-loop-graph-the-three-layers-every-reliable-ai-agent-needs-c600802bca31?source=rss-d583ee2c95b8------2</link>
            <guid isPermaLink="false">https://medium.com/p/c600802bca31</guid>
            <category><![CDATA[agentic-ai]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[harness-engineering]]></category>
            <category><![CDATA[ai-agent]]></category>
            <category><![CDATA[loop]]></category>
            <dc:creator><![CDATA[Anshul Dhingra]]></dc:creator>
            <pubDate>Sun, 16 Aug 2026 12:13:20 GMT</pubDate>
            <atom:updated>2026-08-16T12:13:20.312Z</atom:updated>
            <content:encoded><![CDATA[<h4><em>A practical, end to end guide to harness, loop, and graph engineering, with a real system you can build and replicate</em></h4><p>Klarna’s AI assistant handled 2.3 million customer service conversations in its first month, resolved about two thirds of them without a human, and cut average resolution time from 11 minutes to under 2. It became one of the most cited AI agent success stories of the last two years. Then, in 2025, Klarna quietly brought human agents back into the loop for the cases the AI could not handle safely on its own. That is not a story about a bad model. It is a story about layers.</p><p>Most teams building AI agents run into a version of the same wall. The demo works. The prototype impresses everyone in the room. Then it goes into production and something breaks. It loops forever on a task it cannot finish. It takes an action nobody approved. Or it quietly gives a wrong answer with total confidence.</p><p>The usual reaction is to blame the model. Get a better model, tune the prompt, add a few more retries. That rarely fixes it, because the model was never the whole problem. “AI agent” is one phrase that quietly covers three separate engineering jobs, and most teams are only doing one or two of them well.</p><p>This article names those three jobs, backs each one with the research and tooling that already exists around it, and then builds a complete, realistic system end to end: a support ticket triage agent, detailed enough that you can copy the approach directly for your own use case.</p><h3>What harness, loop, and graph actually mean</h3><p>Before going further, here are plain definitions of the three terms this article uses. Each one is a different job, and none of them is optional if the agent is going to touch real systems.</p><h4>Harness engineering</h4><p>Harness engineering is the design of everything around the model that lets it act safely in the real world: the tools it is allowed to call, the memory it carries between steps, the permissions it holds, and the logging that records what it did. It is the operating environment the model works inside.</p><h4>Loop engineering</h4><p>Loop engineering is the design of the cycle a model goes through to attempt a task, check its own result, and decide whether to stop, retry, or hand off to a person. It answers one question: how does the system know the work is actually finished, rather than just sounding finished.</p><h4>Graph engineering</h4><p>Graph engineering is the design of the shape of the whole job: what step happens first, what happens next, where the work branches into different outcomes, and where a human has to approve something before it continues. It is the map the work follows.</p><h3>Why each layer is essential: what breaks without it</h3><p>Each layer exists to prevent one specific, predictable kind of failure. Skipping one does not make the system simpler. It just moves that failure downstream, into production, where it is far more expensive to catch.</p><h4>Without a harness</h4><p>A model with vague, loosely documented tools will eventually call them in ways nobody intended, because nothing constrains it to the narrow use you had in mind. Without permission scoping, a confused agent, a bad prompt, or a prompt injection from an untrusted input can escalate into deleting data, leaking customer information, or taking an action far outside the task it was given. And without logging, when one of these things does happen, nobody can reconstruct what the agent actually did or why, which turns a fixable bug into a mystery.</p><h4>Without a loop</h4><p>An agent with no evidence-based stop condition has no real way to know when it is finished. In practice this shows up two ways: it declares success on a wrong answer because the model “feels” confident, or it never stops at all and keeps retrying a task that was never going to work, quietly burning money on every attempt. Either way, a human ends up having to check every single output by hand, which erases most of the point of automating the task in the first place.</p><h4>Without a graph</h4><p>Without an explicit shape, every incoming request is forced down one single path. The moment a task does not fit that path, the system either fails outright or the model has to improvise a routing decision on its own, including decisions that should never be made without a person, such as whether an action is safe to take unsupervised. Edge cases stop being an occasional inconvenience and start being the main source of incidents.</p><h3>How each layer solves the problem it exists to fix</h3><h4>The harness solves it with narrow tools and least privilege</h4><p>Strict, narrowly documented tools and permissions scoped to the current task mean the model physically cannot take an action outside what was intended, rather than being asked nicely not to. Full logging of every call means that if something still goes wrong, there is a complete record to trace it back through, instead of a black box.</p><h4>The loop solves it by replacing confidence with proof</h4><p>Instead of asking the model whether it is confident, the loop checks something external and verifiable: a test passing, a citation that resolves to a real document, a number that matches an independent recomputation. Combined with a bounded retry limit, this means the system only ever stops when there is real evidence of success, and never spends unlimited money chasing a case it was never going to solve.</p><h4>The graph solves it by giving every outcome a named path</h4><p>Instead of one path that has to handle everything, the graph makes each realistic outcome an explicit, named branch, including a mandatory human checkpoint placed in front of anything risky or hard to undo. Edge cases have somewhere defined to go, and the decision about what counts as risky is made once, in advance, by a person, rather than improvised by the model in the moment.</p><h3>The three jobs hiding inside “AI agent”</h3><p>Every agent that touches real systems needs three things to work reliably.</p><p><strong><em>· A place to operate safely: tools, memory, permissions, logging.</em></strong></p><p><strong><em>· A way to know when the work is actually finished: not “the model says so,” but proof.</em></strong></p><p><strong><em>· A shape for the work: what happens first, what happens next, and where a human needs to step in.</em></strong></p><p>We will call these three jobs Harness Engineering, Loop Engineering, and Graph Engineering. A simple way to remember them: Environment, then Feedback, then Flow.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/916/1*B99bjRYm6JzZctK4lIg_WA.png" /></figure><h3>This framework is not new, it is a name for what already exists</h3><p>Anthropic’s own engineering team published a widely read breakdown of agent design in December 2024 called “Building Effective Agents.” It draws a similar line between predictable, code-defined workflows and agents that direct their own tool use, and it lays out five composable patterns, prompt chaining, routing, parallelization, orchestrator-workers, and evaluator-optimizer, that mostly live inside what this article calls the loop and graph layers. The idea of interleaving reasoning with tool calls goes back further, to the 2023 ReAct paper by Yao and colleagues, which is the intellectual ancestor of almost every tool-calling loop running in production today. And the idea of representing an agent’s work as an explicit graph of nodes and edges, instead of one long prompt, is the entire premise behind orchestration frameworks like LangGraph, which model an agent’s workflow as a directed graph with conditional edges and a persistent, shared state object.</p><p>None of this is new. What is usually missing on real teams is the discipline to build all three layers on purpose, instead of quietly hoping the model will improvise the ones nobody designed.</p><h3>Layer 1: Harness Engineering, the environment the model works in</h3><p>The harness is everything around the model that lets it act in the real world. If the model is the engine, the harness is the rest of the car: the fuel line, the dashboard, the seatbelt, the brakes.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/945/1*nlo-iW2pKJAT_Lzk6BcoFg.png" /></figure><h3>Tools</h3><p>Every tool the agent can call needs a strict, narrow definition. A vague tool invites misuse. A well-documented one constrains the model to exactly what you intended, and both the Anthropic’s and OpenAI’s tool-calling APIs both build on the same underlying idea, a JSON schema that describes the tool’s name, its purpose, and the exact shape of valid input, though the two APIs name and nest the fields differently.</p><p>Here is the difference between a weak and a strong tool definition, written in Anthropic’s tool-use format. OpenAI’s function-calling format expresses the same idea under a parameters field nested inside a function object, but the underlying principle, narrow and explicit, is identical.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/961/1*iG3K_sshtWtSenBNFj62DQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/964/1*8Vmcq_0Eu8vmjYkqHejABw.png" /></figure><h3>Memory</h3><p>Two kinds of memory matter in practice, and it pays to keep them physically separate in your data model. Short-term memory is the running state of the current task: which ticket, which knowledge base articles have already been tried, how many retries have happened. Long-term memory is what persists across sessions: a customer’s history, prior escalations, known account flags. Mixing the two is a common, hard-to-debug source of an agent repeating a mistake it already “learned” not to make in a completely different conversation.</p><h3>Permissions</h3><p>The security world calls this least privilege: give the agent only the access it needs for the current task, and nothing more, so that when something does go wrong, the blast radius stays small. This has become one of the more actively discussed problems in agent security, because unlike a human employee, an agent’s intentions are inferred from a prompt, and a hijacked, confused, or simply buggy agent will use every permission it has been handed. The direction most production systems are moving in is short-lived, task-scoped credentials rather than one static API key: the agent requests a token for a single action, and that token expires the moment the action completes.</p><h3>Observability</h3><p>Every tool call, every intermediate reasoning step, and every output should be logged in a structured, queryable way. The emerging standard here is OpenTelemetry’s GenAI semantic conventions, which wrap each model call and tool call in a trace span with standard fields for model name, token counts, and latency, so a full agent run becomes one traceable chain instead of a black box you can only guess at after the fact.</p><h3>Layer 2: Loop Engineering, deciding when the work is actually done</h3><p>The loop is the cycle of attempt, check, and decide: keep going, retry, or stop.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/925/1*NVcnprFAC1_JaAxK1BHDGQ.png" /></figure><h4><strong>Do not loop on confidence. Loop on evidence.</strong></h4><p>A model that reports “I’m 95 percent confident this is correct” is not offering a measurement. It is offering a guess dressed up as a number. This is not a minor quirk. Research that benchmarks a model’s self-reported confidence against its actual accuracy has repeatedly found large gaps, in some studies 20 to 60 percentage points, between how sure a model says it is and how often it is actually right. The gap does not shrink when the model is given more room to reason. Several 2026 studies found that longer reasoning traces make overconfidence worse, not better, because a model that has already committed to an answer inside its own reasoning has every incentive to justify it rather than question it.</p><p>Evidence, by contrast, is external and checkable: a unit test passing, a schema validating, a citation that resolves to a real document, a number that matches an independent recomputation. This is also why retrieval-augmented generation, grounding a model’s answer in retrieved documents instead of its own memory, is such a load-bearing idea for loops. Research on retrieval-grounded systems has found that anchoring answers in retrieved sources can cut hallucination rates dramatically compared to ungrounded generation, and evaluation frameworks like RAGAS take this further, scoring each factual claim in an answer against the retrieved evidence individually, rather than judging the whole answer at once.</p><h3>What a good loop looks like</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/970/1*qTsgrqDuwz-7UOW0-ea8EA.png" /></figure><p>A retry limit matters as much as the check itself. Two or three attempts with a clear escalation path is normal. Without a limit, a stuck agent will spend real money failing at the same approach indefinitely, and nobody will notice until the bill arrives.</p><h3>Layer 3: Graph Engineering, giving the work a shape</h3><p>The graph is the map of the whole job: what happens first, where it branches, and where a human has to step in before anything moves forward.</p><p>Frameworks like LangGraph formalize this as nodes, a single step such as “classify the request,” edges, the path between steps including the condition that picks it, and a shared state object that flows through the entire run. This matters beyond neatness because graphs, unlike a single linear prompt, can express cycles: an agent can revisit an earlier step, retry with new information, or route back for reclassification, something a simple top-to-bottom script cannot do cleanly.</p><h3>Where to put the human</h3><p>The most consequential design decision in the graph layer is where you place human approval. Current best practice, drawn from human-in-the-loop research and production postmortems, is to tier it by risk rather than gate everything or nothing.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/910/1*k_SA66uZyqCoU7Zu0NtZbw.png" /></figure><p>A guideline that has come directly out of early production incidents: never let a single model confidence score decide whether an action needs approval. Combine it with independent, rule-based signals, is this action reversible, does it touch money or personal data, has this exact action type failed before, and weight the model’s own confidence low, well under a third of the decision, if you use it at all. The gate should trigger based on what the action is, enforced in code at the workflow layer, not negotiated by the model at request time.</p><h3>Why you need all three, not just one</h3><p>These three layers fix different kinds of failure, and one cannot cover for another.</p><p>A strong graph with a weak harness still fails, because the model has no reliable tools or permissions to actually carry out each step correctly. A strong harness with no loop still fails, because the agent has no way to know when its work is actually good enough, so it either stops too early or burns money retrying forever. A strong loop with no graph works fine for a simple task, but falls apart the moment the job has more than one possible outcome, because there is nowhere to route different results or insert a human check.</p><p>Klarna’s experience is a useful, if imperfect, illustration, since its own public reporting attributes the 2025 changes mainly to customers wanting a human option and broader quality concerns, not a specific, named gap in a graph or loop. Read cautiously, it still lines up with the pattern this article is describing: the graph and loop were evidently strong enough to resolve about two thirds of conversations and cut resolution time by more than 80 percent, and the harder, more ambiguous cases still ended up needing a person, which is exactly the kind of case a deliberate human gate is meant to catch. Intercom’s Fin agent tells a similar story from the other direction. Its publicly reported resolution rate climbed gradually, from about 41 percent to about 51 percent across more than 20 feature revisions, which is what iterating on the loop and graph layers actually looks like in practice: not one leap once “the model got better,” but many small tightenings of evidence checks and routing rules.</p><p><strong><em>A perfect graph will not save a weak harness. A strong harness will still waste money without good loops.</em></strong></p><h3>Case study: building a support ticket triage agent, end to end</h3><p>Now the deep, replicable part. We build an agent that reads an incoming support ticket, tries to resolve it using your knowledge base, and either closes it, sends it for approval, or escalates it to a person, and that never guesses on anything touching money or an account.</p><h3>Step 1: Define scope and success metrics before writing any code</h3><p>Before touching the harness, the loop, or the graph, write down what “working” means in numbers you can measure after launch.</p><blockquote>· Auto-resolution rate: the percentage of tickets closed without a human touching them.</blockquote><blockquote>· Escalation precision: of the tickets the agent auto-resolved, the percentage a human reviewer, sampled after the fact, agrees were actually handled correctly.</blockquote><blockquote>· Time to resolution, compared against your current human-only baseline.</blockquote><blockquote>· Cost per ticket, covering tokens and infrastructure.</blockquote><blockquote>· False-close rate: resolved tickets that get reopened within a set number of days.</blockquote><p>These numbers are what you compare against your pre-launch baseline, and what you watch during canary rollout for regressions. Public deployments in this space report auto-resolution rates roughly between 40 and 75 percent, depending on ticket mix and how conservative the escalation rules are. Treat any number well outside that range, in either direction, as a signal to check your evidence checks, not as a reason to celebrate early.</p><h3>Step 2: Architecture</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/901/1*56NSQFdmYALzzlspzOYQlA.png" /></figure><p>A ticket source, your existing helpdesk or an inbound email address, sends new tickets to an ingestion service through a webhook. The ingestion service validates and normalizes the raw ticket into your internal schema and hands it to the orchestrator, which is the state graph described in Step 6 and the actual brain of the system. The orchestrator calls out to a knowledge base backed by a vector database for retrieval, and writes status updates and drafts back to the ticketing system. Any action classified as risky is routed instead to a human review console, and once a person approves, edits, or rejects it, control returns to the orchestrator to finish the job. An observability layer runs underneath the entire system, capturing every step.</p><h3>Step 3: Data and schemas</h3><p>Two schemas carry the whole system. The first represents the ticket itself as it flows through the graph.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/984/1*96fObgS-tuMWJ7iWBor6UA.png" /></figure><p>The second is a single tool call log entry. Every tool call the orchestrator makes writes one of these, and this is what your observability dashboards and audits are built on top of.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/976/1*HPSeEGYXNJcO77ddSoeTqQ.png" /></figure><h3>Step 4: Build the harness</h3><p>Four tools, each scoped as tightly as the earlier weak-versus-strong example, cover the whole job.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/960/1*O4Sy0Kil63CT4aMhwIo_9w.png" /></figure><p>Notice what is deliberately missing: there is no tool to issue a refund, delete an account, or send an email straight to the customer without review. Those actions do not exist as agent tools at all. They live entirely behind the human review console described in Step 6.</p><h3>Building the knowledge base</h3><p>The knowledge base is a standard retrieval setup: your help center articles are split into chunks, each chunk is embedded, and the embeddings are stored in a vector database alongside the source article’s ID and category. Published benchmarks generally favor chunks of around 300 to 400 tokens as a solid, low-effort default, small enough for precise retrieval, large enough to preserve context. A common refinement worth adopting once the basics work: index small chunks for search precision, but return the full parent article to the model for generation, so the draft reply has complete context even though the match was found on a narrow passage.</p><p>The one rule that makes the evidence check in Step 5 possible: every chunk stored in the vector database keeps a reference back to its source article ID and category. Without that reference, there is nothing concrete for the loop to check the draft reply against.</p><p><em>One more detail worth making explicit: the similarity_score used in the evidence check should be cosine similarity, normalized to a 0 to 1 range, not a raw dot product or L2 distance, since those are not bounded the same way. And 0.75 is a starting point, not a constant. It needs to be calibrated against your own embedding model and a labeled sample of real tickets before you trust it.</em></p><h3>Step 5: Build the loop</h3><p>The agent’s job is to find a knowledge base article that actually answers the ticket, and a ticket only counts as resolved if the draft reply cites a real article whose category matches the ticket’s classified category and whose retrieval similarity score clears a fixed threshold. This is what the loop pseudocode shown earlier in Layer 2 implements directly against this case study’s tools: search, draft, check, and either stop or retry with a reformulated query, up to two retries, before handing off.</p><h3>Step 6: Build the graph</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/940/1*57UnP3cb4V2iee1q_0W-TA.png" /></figure><p>A ticket is classified into billing, technical, or account. Technical tickets enter the resolve loop from Step 5, because handing out a knowledge base article is low risk, and the loop’s own evidence check decides between closing the ticket and escalating it. Billing and account tickets skip the resolve loop entirely and go straight to the human review gate, because those actions are harder to undo if the agent gets them wrong. That routing decision is made on purpose, based on risk, and is not something left for the model to infer at runtime.</p><h3>Step 7: Build observability</h3><p>Every LLM call and tool call is wrapped in an OpenTelemetry span using the gen_ai.* attribute conventions, tagged with the ticket ID, so a complete run from intake to close is one traceable chain. Four dashboards are worth having from day one.</p><p><em>· Auto-resolution rate over time, broken out by ticket category.</em></p><p><em>· Escalation rate by category, to catch a single category quietly degrading.</em></p><p><em>· Cost per ticket and p50/p95 latency.</em></p><p><em>· Retry rate. A rising retry rate is usually the earliest signal that the knowledge base has gone stale before customers start complaining about it.</em></p><h3>Step 8: Test and roll out safely</h3><p>A staged rollout catches different failure modes at each stage, and skipping a stage means finding that stage’s failures in production instead.</p><p><em>· </em><strong><em>Golden dataset</em></strong><em>. Assemble real historical tickets with a known correct outcome, and run the full graph against them offline before any deploy. Revisit and expand this dataset on a schedule. A meaningful share of production agent failures trace back to a golden dataset that was built once and never updated as ticket patterns drifted.</em></p><p><em>·</em><strong><em> Shadow mode.</em></strong><em> Run the agent against live traffic for at least 24 hours without letting its output reach the customer. Log everything, and compare its choices against what the human agent actually did on the same tickets.</em></p><p><em>· </em><strong><em>Canary rollout.</em></strong><em> Route 5 to 10 percent of real traffic to the agent, watch the metrics from Step 1 daily, and expand gradually as they hold steady.</em></p><p><em>· </em><strong><em>A kill switch.</em></strong><em> One flag that routes all traffic back to humans immediately. Test it before launch, not during an incident.</em></p><h3>Step 9: What to expect</h3><p>Klarna’s publicly reported numbers, roughly two thirds auto-resolution and resolution time cut from 11 minutes to under 2, and Intercom Fin’s climb from about 41 percent to about 51 percent resolution across more than 20 iterations, are useful anchors for what a well-built version of this system can achieve, and for how much of the gain comes from ongoing tuning rather than a single launch. Both companies also kept, or rebuilt, a path back to a human. Treat that as a design requirement, not an admission of failure.</p><h3>A checklist you can actually use</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/955/1*ZWUgIDNe5ow0Z598eal6dg.png" /></figure><h3>Mistakes worth avoiding</h3><p><em>· Trusting the model’s own confidence score as a stop condition or an approval gate. Research on self-reported confidence shows the gap between stated confidence and actual accuracy can run 20 to 60 percentage points wide.</em></p><p><em>· Giving the agent broad access “just in case it needs it later.” This turns small mistakes into large ones and works against least privilege.</em></p><p><em>· Building one long single path with no branches, then being surprised when an unusual case breaks everything.</em></p><p><em>· Retrying without a limit, which quietly burns money and time on an approach that was never going to work.</em></p><p><em>· Skipping the human gate on irreversible actions because the demo never needed one.</em></p><p><em>· Building a golden dataset once at launch and never revisiting it as real traffic drifts away from what it covers.</em></p><h3>The takeaway</h3><p>An AI agent is not one thing. It is an environment, a feedback cycle, and a flow, stacked on top of each other. The environment gives it a safe place to work. The feedback cycle tells it, with proof, when it is actually done. The flow gives its work a shape that a human can follow and step into when it matters.</p><p>Before you ship your next agent, do not ask whether the model is good enough. Ask which of these three layers you have actually built, and which one you have been quietly hoping the model would handle for you.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c600802bca31" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>