<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Advisor360° - Medium]]></title>
        <description><![CDATA[Advisor360° Engineering - Medium]]></description>
        <link>https://medium.com/advisor360-com?source=rss----e383b54c6c43---4</link>
        <image>
            <url>https://cdn-images-1.medium.com/proxy/1*TGH72Nnw24QL3iV9IOm4VA.png</url>
            <title>Advisor360° - Medium</title>
            <link>https://medium.com/advisor360-com?source=rss----e383b54c6c43---4</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 00:11:50 GMT</lastBuildDate>
        <atom:link href="https://medium.com/feed/advisor360-com" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Mind the Gap: What is a Trust Fabric]]></title>
            <link>https://medium.com/advisor360-com/mind-the-gap-what-is-a-trust-fabric-668329964dc5?source=rss----e383b54c6c43---4</link>
            <guid isPermaLink="false">https://medium.com/p/668329964dc5</guid>
            <dc:creator><![CDATA[Manu Cohen-Yashar]]></dc:creator>
            <pubDate>Tue, 20 Oct 2026 21:58:26 GMT</pubDate>
            <atom:updated>2026-10-20T21:58:25.511Z</atom:updated>
            <content:encoded><![CDATA[<h4>Engineering trust by managing the gap between expectation and reality</h4><h3>Nobody trusts a system because it’s perfect. We trust it because it’s honest about where it falls short.</h3><p>Think about the people you trust most. They aren’t the ones who never let you down. They’re the ones who tell you, early and plainly, when what you expected and what’s actually happening have drifted apart. “I’ll be late.” “This won’t be ready Friday.” “I’m not sure that number is right.” Trust lives in that honesty.</p><p>Business and technology work the same way. The gap between expectation and reality is always there. Sometimes it’s tiny, sometimes it’s wide enough to drive a quarterly forecast through. Trust doesn’t need the gap to be small. It needs the gap to be visible.</p><p>Managing that gap is an engineering problem you can solve on purpose. That’s what a trust fabric is for.</p><p>Press enter or click to view image in full size</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*bIPAZaPagGrws-j59ay30g.png" /><figcaption>manage the gap between expectation and reality.</figcaption></figure><h3>A fabric, not a single product</h3><p>A trust fabric is the <strong>confluence </strong>of <strong>infrastructure</strong>, <strong>tools</strong>, and <strong>practices </strong>that together let you measure, expose, and act on the <strong>distance between expectation and reality</strong>.</p><p>It is not a single API. It is not a single database. It is not one tool you buy, or one process you roll out with a kickoff deck and a Slack channel. It’s all of these working in concert. If someone offers to sell you “trust” in a box, check what else is in the box.</p><p>The trust equation has two sides: <strong>expectation</strong> and <strong>reality</strong>. Each needs its own discipline.</p><h3>Expectations: the side we write down</h3><p>Expectations belong to the business. They live in the product definition, the spec and its lifecycle, and the ongoing conversation with the customer. What matters is getting them into a document everyone can see. An expectation that exists only in someone’s head is a surprise waiting to happen. In practice, that’s where most of them stay until it’s too late. Marriage counselors have built a whole profession on exactly this.</p><p>Strong product management, business alignment, and healthy customer relationships are what get this side of the equation right. And “customer” covers more than it sounds like. It can be an external client, or the team two desks over. Once we agree on a service or module contract, we set expectations.</p><p>Getting there takes real effort. You have to dig into every part of the data and how it behaves to uncover the expectations nobody mentions until production breaks. Those expectations cover the whole experience of using the data: its quality, performance, and representation.</p><p>For data teams, the data product lifecycle is the core practice here. It turns “we kind of assumed the table refreshes daily” into a documented commitment someone can actually check.</p><h3>Reality: the side we measure</h3><p>If expectations are what we promise, reality is what we can prove. Proof starts with observability.</p><p>In our stack, Dynatrace covers infrastructure, Monte Carlo covers data, and Confident AI covers agentic AI workflows. That’s our choice, not a prescription. The observability market is wide, and the right tools depend on what you run.</p><p>But reality is more than a quality score. It also includes context and provenance. Where did this data come from? What does it mean to the business? What has a good data catalog learned about it? These answers are part of the description of reality, not decorations on it. We use Euno to map lineage and expose the provenance of every dataset, so a customer’s expectation about where data came from can be checked against measured lineage and the context built automatically around it.</p><h3>You can’t report what you don’t measure</h3><p>You have to measure data quality before you can report it, and you have to measure it in more than one place in its life cycle. We test data in the code, in the pipeline (using dbt tests, for example), and at the storage layer for every strategic dataset (using Monte Carlo, for example).</p><p>None of that works without upfront design. You need to define what “good” means before you can tell where to look for “bad.” Otherwise you end up monitoring everything, alerting on nothing useful, and training your team to ignore the pager. That’s not observability. That’s ambient noise with a dashboard.</p><h3>When the gap gets too wide</h3><p>Measuring the gap is half the job. The other half is doing something when it grows. That means alerting and acting when anomalies appear and the distance crosses a defined threshold, and having the communication and incident management infrastructure to respond. A gap that’s detected but never escalated is just a very well-documented problem.</p><h3>Infrastructure: where promises meet hardware</h3><p>Clients expect every request to be handled, no matter how busy the system is. They expect their data to be secure. They expect the system never to go down and to respond instantly. Reality, of course, is different. Meeting those expectations takes infrastructure with enough capacity and redundancy to absorb load spikes and failures. And all of it has to be balanced against cost, because “infinitely scalable” usually comes with an infinitely scalable invoice.</p><h3>The trust sidecar</h3><p>One practice we lean on heavily is shipping data together with its context and quality. We call it the <strong>trust sidecar</strong>.</p><p>On our GraphQL API, every record exposed through the GraphQL mesh comes packaged with a trust artifact: a relevant quality score plus metadata about that element. The consumer doesn’t have to go hunting for whether the data is reliable. The answer arrives in the same response.</p><p>This changes the conversation when things aren’t great. If the API returns weak data because that’s genuinely the best data we have, we say so. “Here’s your answer, here’s how much to trust it” is a far better experience than a confident number that quietly turns out to be wrong. Honesty, delivered at the field level.</p><h3>Trust is built, not hoped for</h3><p>Trust is the foundation that successful AI systems stand on, and it doesn’t appear by accident. It has to be engineered. It takes business and engineering rigor, a solid set of tools, and strong infrastructure, all working together to manage the distance between expectation and reality.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=668329964dc5" width="1" height="1" alt=""><hr><p><a href="https://medium.com/advisor360-com/mind-the-gap-what-is-a-trust-fabric-668329964dc5">Mind the Gap: What is a Trust Fabric</a> was originally published in <a href="https://medium.com/advisor360-com">Advisor360°</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Developing Adaptability at Advisor360°]]></title>
            <description><![CDATA[<div class="medium-feed-item"><p class="medium-feed-image"><a href="https://medium.com/advisor360-com/developing-adaptability-at-advisor360-6242853cf068?source=rss----e383b54c6c43---4"><img src="https://cdn-images-1.medium.com/max/1672/1*0hdlQTj9uoNa8O4HwzSfkQ.png" width="1672"></a></p><p class="medium-feed-snippet">A practical framework for building and assessing adaptability</p><p class="medium-feed-link"><a href="https://medium.com/advisor360-com/developing-adaptability-at-advisor360-6242853cf068?source=rss----e383b54c6c43---4">Continue reading on Advisor360° »</a></p></div>]]></description>
            <link>https://medium.com/advisor360-com/developing-adaptability-at-advisor360-6242853cf068?source=rss----e383b54c6c43---4</link>
            <guid isPermaLink="false">https://medium.com/p/6242853cf068</guid>
            <dc:creator><![CDATA[Manu Cohen-Yashar]]></dc:creator>
            <pubDate>Tue, 22 Sep 2026 12:11:52 GMT</pubDate>
            <atom:updated>2026-09-22T12:11:51.575Z</atom:updated>
        </item>
        <item>
            <title><![CDATA[Knowledge Management in Advisor360]]></title>
            <link>https://medium.com/advisor360-com/knowledge-management-in-advisor360-6fff562a4060?source=rss----e383b54c6c43---4</link>
            <guid isPermaLink="false">https://medium.com/p/6fff562a4060</guid>
            <dc:creator><![CDATA[Manu Cohen-Yashar]]></dc:creator>
            <pubDate>Mon, 17 Aug 2026 12:08:27 GMT</pubDate>
            <atom:updated>2026-08-20T18:16:21.375Z</atom:updated>
            <content:encoded><![CDATA[<h4>What we have learned about organizing knowledge, what AI actually changed, and why you should be careful of anyone trying to sell you a “company brain”</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Ep0TgetqZ37q-WR5H2LYtw.png" /></figure><p>My son left for college, and his room has been spotless ever since. My daughter still lives at home, and you cannot see her floor. Not the carpet, not the wood. At this point, the floor is a rumor. I know it is there — I built it.</p><p>Here is the thing, though. The tidy room is not the good one. It is tidy because nobody lives there. The disaster zone belongs to a teenager with friends, hobbies, strong opinions about music, and a completely normal happy life. Mess is what living things produce.</p><p>I say some version of this to every data team I work with. Your knowledge base is my daughter’s room. It is not messy because something went wrong. It is messy because it is alive. Entropy is greater than zero; the Second Law of Thermodynamics did not write an exception for your Confluence instance, and any vendor promising you a permanently organized corporate brain is quietly promising you a room where nobody lives.</p><p>This is not an abstract concern for us. At Advisor360°, knowledge is the foundation of the automation products we build — products meant to reinvent how financial advisors work. An agent that drafts a client review, assembles a meeting brief, or explains a number is only as good as what it knows and how much of that it should be trusted to act on. So we treat knowledge as a first-class engineering problem, with the budget and the evaluation rigor that implies. More on what that looks like in practice near the end.</p><p>First, let us walk through what we actually know about organizing knowledge for machines, what AI genuinely changed, and what is still wide open. Spoiler: the open part is bigger than the closed part, and that is the most interesting news in enterprise AI right now.</p><h3>The ladder from information to wisdom</h3><p>Knowledge has been the engine of human civilization since the beginning, and it helps to be precise about what we mean by the word.</p><p>The useful mental model is a ladder. Information is a pile of data points. Organize that pile inside a mental model, and it becomes knowledge. Apply reasoning to knowledge, and you get wisdom.</p><p>Information scientists call this the DIKW hierarchy, and it is usually credited to Russell Ackoff, who laid it out in a 1988 address that ran the following year as a paper titled “From Data to Wisdom.” Milan Zeleny sketched something similar in 1987. The oldest fingerprints on the idea belong to T.S. Eliot, who arranged wisdom, knowledge, and information into a descending ladder in his 1934 work “The Rock,” roughly a decade before anyone built a computer to worry about it. Poets get there first, and then engineers show up with the budget.</p><h3>Librarians solved a version of this once already</h3><p>Writing sparked the first explosion of knowledge, and printing turned that explosion into a permanent condition. Every time technology makes it easier to produce and consume information, the pile grows past the point where you can just remember where you put things. Organization stops being optional.</p><p>Librarians took that problem seriously for centuries and built genuinely sophisticated machinery for it. Taxonomies and ontologies arrange terms in a formal hierarchy that supports systematic processing. Classification schemes convert those terms into numbers so that articles and books can be clustered, shelved, and found.</p><p>Anyone old enough to remember a card catalog knows exactly what this looked like, and knows the specific satisfaction of pulling open a wooden drawer and finding the one card you needed. That drawer was a knowledge graph. It just did not have a marketing team.</p><h3>Nothing intelligent happens without knowledge in the loop</h3><p>Intelligence depends on knowledge. Every cognitive act you perform leans on something you already know, whether you pull it from memory or from a book. Nothing intelligent gets done without knowledge somewhere in the loop.</p><p>Von Neumann said it elegantly in 1945. His architecture for the modern computer requires memory that holds both data and instructions, external storage, a central processing unit, and mechanisms for input and output. Read that list again and notice that it is not really a description of a machine. It is a description of any cognitive process. The hardware just happens to be the version we can buy.</p><p>Nineteen forty-five was a good year for this line of thinking. The same year, Vannevar Bush described the Memex, a personal curated store of documents connected by associative trails. Bush got almost everything right about how a knowledge system should feel. The one thing he could not solve was who maintains it. Hold that thought, because it comes back with a vengeance in about eighty years.</p><h3>Automating the cognitive assembly line</h3><p>Humans have always tried to hand our work to technology. The Industrial Revolution automated physical processes that had been powered by human and animal muscle. Only with digital transformation did we start automating cognitive procedures.</p><p>If you want to automate a cognitive procedure, you first have to automate its components, and we have made remarkable progress on almost all of them. Look at the input side alone: punch cards, keyboards, mice, touch screens, voice. My grandmother would have needed a training course to use a computer. My daughter learned to swipe before she learned to walk.</p><p>The heart of a cognitive process is reasoning, and that is the component we have only partly automated. The old rule was brutal and simple. If you could describe a process formally, you could write a program for it. If you could not, you were out of luck.</p><p>The same rule applied to knowledge. To make information useful to a computer, you had to organize it into a structure the computer could operate on. Computers run formal processes. Information had to arrive in formal shape or not at all.</p><h3>The expensive art of telling a computer what things mean</h3><p>Data modeling is the craft of building that shape. Taxonomies and ontologies define a formal model that captures semantics alongside the raw data. They work. They are also brutally expensive to build and to keep alive.</p><p>About 10 years ago, long before this AI wave, I wanted to build a system to help people apply culinary medicine in their daily lives. I asked myself a simple question: how do I encode this domain so a computer can actually use it? Culinary medicine sits at the intersection of two enormous fields, medicine and cooking, and the intersection was more than I could model. I could have written a book. What I could not do was build a virtual expert that would stand in your kitchen and guide you, because I could not encode the knowledge. The bottleneck was never the software. It was the modeling.</p><p>Machine learning was the first real crack in that wall. It let us automate a solution to a problem we could not describe formally, which is another way of saying a problem we could not write a program for. The catch is that these were very specific kinds of problems: classification, regression, recommendation. All of them have a clean statistical profile. We could not write a program to solve the problem directly, so we wrote one that captured an approximation of that statistical profile as a model. We call that training. Then we wrote a second program that follows the model and hope the result is good enough. Because the machine captures the model, we say it “learns,” and that is how the field got its name.</p><p>Machine learning widened the space of solvable problems. It did not change the underlying deal on knowledge. Computers still needed formal organization before they could use anything.</p><h3>Most of the world’s knowledge was invisible to machines</h3><p>And most knowledge was never organized formally. A document is easy for a human to read and, until very recently, nearly opaque to a computer. Digital transformation multiplied the volume of knowledge humanity produces, and the vast majority of it lay outside what machines could access.</p><p>The numbers are worth sitting with. IDC and Gartner have put unstructured data at roughly 80 to 90 percent of all enterprise data for over a decade, and the share keeps growing. IDC also found the spending runs the other way: about 40 percent of technology spend goes to the unstructured majority while 60 percent supports the structured minority. We have been investing most of our money in the smaller half of what we know.</p><p>Knowledge engineering is the discipline that tries to close that gap and make knowledge usable by machines. Knowledge graphs are the flagship example, representing an enormous collective investment in doing representation properly.</p><p>Reliable knowledge processing is hard for reasons that go deeper than tooling. To process knowledge reliably, you have to genuinely understand it, and machines do not comprehend anything. The same sentence means different things in different contexts. Knowledge also has a life cycle. Some facts come with an expiry date, which makes retirement easy. Most facts expire because another fact shows up and makes them obsolete, which makes it almost impossible to detect their retirement automatically. Humans handle this without noticing, because we place things in context by reflex. Machines find it excruciating.</p><p>Taxonomies and ontologies give you a real superpower here: deterministic validation. You can check whether the information complies with the model, and the model can tell you what it means based on where it sits. That works. It is also expensive, complex, and covers only the slices of knowledge you paid to model in the first place.</p><p>When something is powerful but expensive to operate, expect limited adoption. That is precisely the story of RDF knowledge graphs. The technology is not dead; plenty of smart people still advocate for it, and yet the overwhelming majority of organizations do not use it. Every organization owns and manages knowledge. Almost none of them own a knowledge graph. That gap tells you something about where the technology is heading.</p><h3>Then vectors happened</h3><p>AI changed the terms of the deal.</p><p>At the core of an LLM is an enormous body of vectorized knowledge. To train a model or use one, information gets encoded into embedding vectors and served to the model as tokens. Strip away the details, and it comes down to this: information is represented as a long list of numbers describing a point in a very high-dimensional space.</p><p>The surprise was what came along for the ride. That encoding captures semantic meaning. Nobody designed it that way. It emerged. Models turned out to capture both the data and its meaning without anyone having to write an ontology first.</p><p>You can measure it. The canonical demonstration comes from Mikolov and colleagues in 2013: take the vector for “king,” subtract “man,” add “woman,” and you land near “queen.” The relationship between a monarch and a gender was never programmed. It fell out of the statistics of how words co-occur. Run the same trick on a corpus about the British monarchy and the distance from “prince” to “king” behaves a lot like the distance from “Harry” to “Charles.”</p><p>We can see the semantics. We do not fully understand them. It is a black box and a difficult one to steer. But judged from the outside, modern LLMs deliver semantic comprehension that matches or beats what an RDF knowledge graph gives you, and for the average person an LLM is dramatically easier to use. So the adoption picture flipped completely. Knowledge graphs stayed rare. Today essentially every company that owns knowledge is encoding it into an LLM in one form or another, deliberately or otherwise.</p><h3>Please keep your deterministic engines</h3><p>One important caveat before anyone gets carried away.</p><p>Some data is naturally structured or semi-structured and fits beautifully into the data models of modern databases. This is the data behind business analytics, financial operations, and very nearly every transaction in the modern economy. Those transactions depend on engines that run at massive scale and return deterministic, trustworthy, accurate results. Databases, warehouses, and analytical engines do exactly that, and they do it very well.</p><p>LLMs are extraordinarily powerful and fundamentally nondeterministic. In our world, that distinction is not a philosophical nicety. A client’s account balance, a performance figure, a fee calculation — those belong to an engine you can audit, not to a model that is feeling creative on a Tuesday. Structured data is not the problem. The problem is the unstructured mountain of documents, decks, tickets, policies, and transcripts that we need to encode before we can build intelligent systems on top of it.</p><h3>RAG, and the demo that did not survive contact with production</h3><p>RAG was the first obvious attempt at knowledge representation for AI. You have seen the demo: a folder of corporate documents, a chatbot, a few impressive questions. In 2024, that was genuinely exciting. In production, the business reality proved considerably more complicated.</p><p>There is a benchmark that captures this well, and it lands close to home for anyone in financial services. FinanceBench, published by Patronus AI in late 2023, tested models on deliberately straightforward questions about public company filings. GPT-4-Turbo paired with a retrieval system answered incorrectly or refused on 81 percent of them. Models have improved a lot since then, and that specific number is old, but the shape of the finding has proven durable: retrieval quality, not model quality, is usually what breaks first.</p><p>Almost every knowledge management system now ships a RAG chatbot. Confluence has one. SharePoint has one. Users have already learned to calibrate their expectations accordingly, which is its own kind of verdict. Many techniques improve on naive RAG with real success, and none of them constitute a complete solution. Possibly none ever will.</p><h3>The format that won by giving up</h3><p>Ward Cunningham invented the wiki in 1995, and the concept went mainstream in 2001 when Jimmy Wales and Larry Sanger launched Wikipedia.</p><p>A wiki rests on the same premise as a knowledge graph: knowledge is a web of concepts. The difference is that a wiki never asks you to formally describe what a link means. That omission is the whole reason it is easy to use, and ease of use is why it won.</p><p>Strictly speaking, concepts connected without typed relationships give you less than a knowledge graph does. Reality turned out not to care. You can build excellent intelligent systems even when semantic information is never encoded at the graph layer.</p><h3>Karpathy’s wiki, and the part everyone skips</h3><p>In early April 2026, Andrej Karpathy published a one-page GitHub gist he described as an idea file, titled simply “llm-wiki.” It picked up more than five thousand stars and five thousand forks.</p><p>The problem he names is one every heavy LLM user recognizes. Traditional RAG and chat interfaces start from scratch every session, forcing the model to parse raw documents again and again, with no structured long-term memory accumulating anywhere.</p><p>The alternative is to stop managing notes by hand. Drop raw sources into a folder, papers and articles and meeting notes, and let an agent compile, structure, and cross-reference them into a living interconnected Markdown wiki. The wiki becomes a compounding artifact. It gets richer with each source you add, rather than resetting with each question. Karpathy explicitly traces the idea back to Bush’s Memex, which brings our 1945 thread full circle.</p><p>He pointed to two fundamental elements of knowledge management, and most of the industry pays attention to only one. The first is representation, the wiki itself. The second is maintenance.</p><p>His argument on maintenance is the part I would tattoo on a whiteboard. The tedious part of running a knowledge base was never the reading or the thinking. It is the bookkeeping. Updating cross references, refreshing summaries, flagging where a new source contradicts an old claim. Humans abandon wikis because the maintenance burden grows faster than the value. LLMs, in his words, “don’t get bored.” One of them can touch fifteen files in a single pass without complaining once.</p><p>This is the thing most of us forget. A great representation that nobody maintains decays into a great representation of last year. Maintenance is a complex, tedious cognitive task that human beings genuinely hate doing, and for the first time in history we have a technology that can actually do it. That is not a minor convenience. It is the difference between a knowledge base that survives and one that quietly rots.</p><p>Which brings us back to my daughter’s room. Life produces mess. The question was never how to prevent the mess. It was who is willing to tidy up every single day forever. It turns out the answer is a machine that never gets bored.</p><h3>Nobody knows where human knowledge ends</h3><p>Solve representation and solve maintenance, and you still have not answered the most uncomfortable question in the room, which is whether the thing you built contains enough knowledge to solve real problems.</p><p>Human knowledge has no boundaries, or at least none we can measure. We constantly test how effectively people use what they know. We have never managed to measure where a person’s knowledge starts, where it ends, or how many facts and concepts it contains. We learn new things and lose or change old ones without noticing and without permission. Nobody runs a delete statement against their own memory.</p><p>Machine knowledge is the exact opposite. Its boundaries are precise and define the size and content of the database or knowledge management system. We ingest new data, update existing data, and retire old data in a controlled, auditable fashion. That control is what makes it manageable. It is also what makes it small. The internet blurred that line, and the wiki blurred it further. Knowledge represented as a web of links feels infinite the way a mind feels infinite. We do not know its edges either. A knowledge base that is not connected to the internet is constrained by design.</p><p>Human thinking is also associative, and this is the part that should worry anyone building a corporate brain. Whenever we solve a problem, we pull in knowledge from domains that have no obvious business being in the room. My son, the one with the spotless bedroom, is a musician, and it is remarkable to watch how everything routes through music for him. Every problem he touches gets some element of music theory applied to it. He tells me a striking share of physicians in the US have a musical background, because the two domains connect in ways I cannot follow. The pattern is real, and you have watched it happen on your own team.</p><p>An enterprise knowledge base gets no such luxury. It is bounded by the domains we chose to define and the facts we chose to ingest. So how would we ever declare it complete? How do we know it holds what it takes to solve a real problem rather than a representative one?</p><p>Here is the version of that problem running in production right now. Plenty of companies are building their knowledge base out of their codebase. Point an agent at the repository, let it scan everything, and it writes a set of Markdown files describing the system, cross-linked into a wiki. This is great. We do it, and I recommend it. It is the pattern from the previous section, applied to the one corpus every engineering organization already has.</p><p>But code does not tell the whole story, and it never has. The rest of the story is product documentation, the theory and academic research the design rests on, the business glossary, industry regulations and standards, conversations with customers, company history, company politics, and the legacy system nobody migrated that the agent never laid eyes on. That list is nowhere near exhaustive.</p><p>So is a knowledge base compiled from the code alone enough? Probably not. What you get is fluent, well-organized, internally consistent, and confidently missing the reason for half the decisions.</p><p>Think about a good support engineer. She closes each ticket by drawing on an uncontrolled corpus of documentation, war stories, a hallway conversation from last March, and a hunch left over from a previous job, most of which she could not list for you if you asked. Now build a support chatbot with an equivalent knowledge base. How would you know when you were done?</p><h3>Trust is a source problem</h3><p>Once you accept that knowledge will be messy and constantly rewritten, the next question is which parts of it you should believe.</p><p>In real life, we mostly resolve this by source. You would generally trust a story from an established news organization over an anonymous social post. Sometimes we lean on credentials: on a medical question, I trust my physician far more than my teenage daughter. And trust always depends on context. On the subject of celebrity drama, my daughter is dramatically more reliable than my physician.</p><p>Digitally, quality gets more complicated, but source is still the right place to start. Expiry dates are another simple and effective mechanism. Neither is close to sufficient.</p><p>The hard question is what is true when several teams document the same topic and disagree. If a knowledge base contains conflicting facts, the LLM reading it needs a basis for choosing among them. And notice that data quality problems are not evidence that something broke. Conflict and drift are the natural state. Handling them is our job.</p><p>There are partial answers. Metadata management platforms augment data with lineage and provenance so a model has something to reason with when sources conflict. Another approach builds a dependency graph between facts and terms: one fact depends on another, so when the root becomes obsolete, its children go with it, and a child is never allowed to contradict its parent.</p><p>The principle underneath all of it is simple. Digital knowledge has to travel with metadata about its quality and context, or it cannot be trusted.</p><h3>OKF, or how Google learned this lesson in six weeks</h3><p>The industry is converging on this faster than I expected.</p><p>On June 12, 2026, Google Cloud published version 0.1 of the Open Knowledge Format. OKF is a vendor-neutral, Markdown-based open specification that turns structured documentation into traversable knowledge. A bundle is a directory of Markdown files with YAML front matter, cross-linked so that an agent can navigate deliberately rather than guessing. No runtime, no SDK, no registry, and exactly one required field. If you can read a file, you can read OKF.</p><p>Then reality arrived. Once agents started writing thousands of concepts into these bundles, contributors kept circling the same worry: can a corpus that agents are actively rewriting be trusted at all?</p><p>Six weeks later, on July 25, 2026, Google shipped v0.2 with a trust layer. The spec now organizes five questions a consumer needs answered: what was this created from (provenance), how much should I trust it (trust), is it still true (freshness), is it the current version (lifecycle), and was this number produced the way we said it must be (attestation). In practice, that means frontmatter fields recording sources, who generated a concept and when, who verified it and whether that verifier was a human or a machine, a draft/stable/deprecated status, and a hard stale_after date.</p><p>Six weeks from “here is a clean format” to “we need provenance and trust or none of this works.” Knowledge management veterans spent thirty years learning that lesson. It is oddly reassuring to watch a very good team rediscover it at high speed.</p><p>My favorite piece of v0.2 is the new Attested Computation concept type, because it quietly resolves the deterministic engine tension from earlier. A concept can now carry the sanctioned way to compute a value, along with an executor that runs it and a deterministic no-LLM attester that checks the receipt. The agent may only fill in declared parameters. It may not rewrite the SQL. So the model gets to be fluent, and the finance number gets to be exact, and you can prove which is which.</p><p>For a wealth management company, that distinction is the whole ballgame. OKF is a real step forward. It simplifies and unifies knowledge base creation so the result serves humans and machines at once, and it absorbs decades of hard-won knowledge management practice. It is also not a complete solution, and to Google’s credit, the spec says so.</p><h3>The intern who read the entire internet</h3><p>There is a deeper risk sitting underneath all of this, and it is structural.</p><p>An LLM is supposed to be a reasoning engine that executes cognitively intelligent tasks. But it is built out of a colossal amount of world knowledge, and that becomes a problem the moment world knowledge contradicts your private knowledge. Having a model that knows things is often convenient. Having a model with opinions about your domain is not.</p><p>I use the intern analogy. An LLM is a brilliant intern fresh out of a prestigious school. She knows a great deal about almost every topic and nothing whatsoever about your business. With a human intern, you can say “set aside what you learned about this domain and use our internal definitions instead,” and she will simply do it. With an LLM, that instruction is hard, expensive, and sometimes impossible to enforce.</p><p>Here is a demonstration that makes the point better than any architecture diagram, and I encourage you to run it at your next team meeting.</p><p>Ask everyone to open their phone and tell their favorite chatbot to pick a random number between 0 and 10. Tell them to look at the answer and keep it to themselves. Then announce, with as much drama as you can manage, that everyone is looking at a 7.</p><p>You will be right often enough to seem psychic. The reason is that humans have a well-documented bias toward 7 when asked for a random number between 1 and 10, a finding that goes back to psychology research in the 1970s. Models trained on billions of examples of human text inherit the bias.</p><p>This is measured, not folklore. Javier Coronado-Blázquez of Telefónica Tech ran 75,600 calls across six models, seven languages, and six temperature settings. In the 1 to 10 range, 7 was selected in roughly 80 percent of cases by GPT-4o-mini, Phi-4, and Gemini 2.0. Ask for 1 to 100, and nearly everything collapses onto 37, 47, and 73. Turning up the temperature, the parameter that supposedly controls variability, does not rescue it.</p><p>Now for the second half. Have everyone go back and say: “Pick a random number between 0 and 10. I know you have a strong bias toward 7, so give me a genuinely random one.” In our own testing, this reliably produces 4, and the model will often explain, with total confidence, that it selected randomly. Yet everyone in the room is holding the same number.</p><p>That is the whole lesson in one gesture. The bias lives in the foundation weights, and at that level a prompt has very little purchase. Even asking the model to implement a proper random function does not fix it.</p><p>Fine-tuning can help. It is an expensive process that adjusts model weights, often through adapter layers rather than the full network, and it comes with real conditions. You need enough training data and a precise understanding of the behavior you want to change. In most enterprise situations, you do not know what you do not know, and finding out is its own expensive project.</p><p>And a skill, at bottom, is a well-organized way to deliver instructions and context to a model at runtime. Useful, absolutely. We use them constantly. But context is not weights. No prompt, however elegantly engineered, overrides what the foundation learned.</p><h3>How we approach this at Advisor360°</h3><p>All of the above is why we treat knowledge as infrastructure rather than documentation.</p><p>Our products automate cognitive work for financial advisors: preparing for a client meeting, assembling a review, answering a question about a household, explaining where a number came from. Every one of those tasks is a knowledge task before it is an AI task. The model is the easy part to swap. Knowledge is the factor that determines whether the output is worth acting on.</p><p>So we invest real effort and real budget in building and maintaining high-quality knowledge bases — for our agents and for our people, who turn out to want the same things from a knowledge base that agents do.</p><p><strong>We evaluate before we trust.</strong> We have built a sophisticated evaluation framework because the alternative is shipping on vibes, and in wealth management, shipping on vibes is not an option. Retrieval quality, grounding, freshness, and failure modes get measured, not assumed. When a knowledge base changes, we want to know whether the answers have improved or merely changed.</p><p><strong>We manage metadata and context deliberately.</strong> We have introduced Euno for metadata and context management, giving us live lineage, usage, ownership, and business logic stitched together into context that both our teams and our agents can consume at query time. This is the practical answer to the trust problem described earlier: when sources conflict, something has to know which one is certified, which one is stale, and who owns it.</p><p><strong>We compile wikis from our code and our documentation.</strong> We run the pattern this article has been circling — agents that read the repository and the docs and maintain a cross-linked, living knowledge base, with the bookkeeping done by a machine that does not get bored. And we hold ourselves to the caveat in the same section: code alone is not the whole story, so we keep widening the corpus.</p><p><strong>We track the research and adopt what earns its place.</strong> Formats like OKF, provenance and attestation patterns, advances in retrieval and interpretability — we read this work, test it, and adopt the parts that survive contact with our production reality. Some of it does not, and we say so.</p><p>None of this makes our knowledge base tidy. It makes it maintained, measured, and honest about its own edges. Those are different goals, and the second one is achievable.</p><h3>Nobody has solved this</h3><p>Knowledge management is not a solved problem. Anyone selling you a complete corporate brain is selling an illusion, and you should recognize it as one before the invoice arrives. That standard applies to us too, which is why we would rather tell you what we measure than tell you we have finished. We are early in a long journey.</p><p>Today an LLM is a package deal, and the package is a black box with respect to the knowledge it carries. Until we find ways to inspect and control the weights and what they encode, we stay roughly where we are.</p><p>The genuinely good news is that this is exactly what the research labs are working on. Interpretability and model editing are among the most active areas in AI right now. A breakthrough there would reshape everything in this article, and we would happily rewrite all of it.</p><h3>Where that leaves us</h3><p>Knowledge is the key to intelligent action. As we automate cognitive processes, we must automate knowledge management as well. But knowledge, like life, is a mess, and organizing a mess that keeps regenerating is genuinely hard.</p><p>We have real progress to show: graph technology, wikis, vector representations, emerging formats with trust built in. It is the beginning of the story rather than its end.</p><p>So stay curious, stay skeptical, and apply diligent, rigorous evaluation and validation to every knowledge solution you adopt. Ask the vendor which of the five OKF questions their system can actually answer. Ask what happens to a fact when the fact it depended on turns out to be wrong. Ask who maintains it on a Tuesday in eighteen months.</p><p>And if anyone tells you their knowledge base is perfectly organized, ask them who lives there.</p><p><em>The author works at Advisor360° as an AI &amp; Data Architect, where we build automation products for financial advisors and their firms. The practices described here reflect how we build and evaluate the knowledge on which those products depend.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6fff562a4060" width="1" height="1" alt=""><hr><p><a href="https://medium.com/advisor360-com/knowledge-management-in-advisor360-6fff562a4060">Knowledge Management in Advisor360</a> was originally published in <a href="https://medium.com/advisor360-com">Advisor360°</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Turning Deployment Failures into Operational Learning]]></title>
            <link>https://medium.com/advisor360-com/turning-deployment-failures-into-operational-learning-544128aa0ff6?source=rss----e383b54c6c43---4</link>
            <guid isPermaLink="false">https://medium.com/p/544128aa0ff6</guid>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[platform-engineering]]></category>
            <dc:creator><![CDATA[Tushar Murudkar]]></dc:creator>
            <pubDate>Wed, 10 Jun 2026 12:05:48 GMT</pubDate>
            <atom:updated>2026-06-10T12:05:46.305Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*WlohiVqHQZqItJtnt_KymA.png" /></figure><h3>How an AI bot helps a delivery platform learn from every failed deployment</h3><p>Every deployment leaves evidence. It shows what changed, what held, and where the system gave way. Across a large delivery platform, those signals accumulate continuously — in pipelines, logs, metrics, tickets, and team channels. Each failure contains a lesson, but extracting that lesson still takes time, context, and engineering judgment. Vizion was built to reduce that gap.</p><p>Vizion is an always-on AI Slack bot that watches CI/CD failures across QA, PreProd, and Prod. When a failure occurs, it gathers the relevant evidence, reconstructs what changed since the last successful run, drafts a root-cause hypothesis with recommended next steps, and posts the analysis back into the same channel where the failure was reported. Once an engineer approves the lesson, Vizion preserves it so future failures can be resolved with better context.</p><p>The goal is not to replace incident response. It is to reduce context-reconstruction time and make each failure easier for the platform to learn from.</p><p>What began as deployment visibility — understanding how deployments moved through the platform — is now evolving into operational learning: a system that can reason about failure patterns, retain confirmed lessons, and help teams respond with more consistency over time.</p><p>This is the story of how we taught the delivery platform to learn from failure.</p><h3>From Visibility to Action</h3><p>In my <a href="https://medium.com/advisor360-com/what-23-000-deployments-revealed-about-software-delivery-at-scale-9644b7056040">previous article</a>, I wrote that deployment data is not just operational reporting — it is architectural feedback.</p><p>Metrics alone do not improve a platform. They reveal what happened, but they do not explain why it happened or reconstruct the events that led to failure.</p><p>That work usually falls to an on-call engineer. Before they can act, they first have to reconstruct the operational context around the failure. That reconstruction is often where the delay lives.</p><p>Vizion was designed to make that context available immediately.</p><h3>One Signal, Two Purposes</h3><p>Every deployment produces a structured delivery signal: deployment data for reporting and an outcome signal that reflects success or failure.</p><p>That signal was originally built for visibility. It made deployment activity measurable across environments and gave us a more reliable view of how software moved through the platform. Vizion uses the same signal differently.</p><p>When the outcome is a failure in QA, PreProd, or Prod, Vizion treats it as the starting point for analysis. Successful runs pass through without action. Failed runs become structured opportunities to gather evidence, explain what happened, and capture what should be remembered.</p><p>The architectural point is simple: the platform does not need a separate detection mechanism to learn from failure. The same instrumentation that measures delivery can also trigger a response.</p><p>One signal serves two purposes — measurement and action.</p><h3>Vizion as an Architectural System</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6AyGdwt9fMtkqVs2HDQXzg.png" /></figure><p>Vizion is best understood not as a single AI feature, but as a set of deliberate architectural boundaries.</p><p>It operates as an event-driven pipeline where each layer owns a narrow responsibility:</p><ul><li>A <strong>listener</strong> reacts to failure events as they are emitted by the delivery platform.</li><li>An <strong>evidence layer</strong> assembles the context: logs, recent changes, the last successful run, and relevant prior analyses.</li><li>A <strong>reasoning layer</strong> drafts an explanation from that evidence and checks its output before release.</li><li>A <strong>delivery layer</strong> posts the analysis back into Slack, where the team is already working, and where needed, creates and assigns a ticket.</li><li>A <strong>memory layer</strong> preserves approved analyses as reusable priors so similar failures can retrieve the right lesson later.</li><li>A <strong>control layer</strong> keeps human review in place and determines what is allowed to become part of system memory.</li></ul><p>This separation is intentional. Models will change. Reasoning approaches will improve. The AI layer can be replaced over time without redesigning the whole system.</p><p>The durable value lives in the surrounding architecture: consistently gathering the right evidence, preserving confirmed learning, and making that learning reusable across future incidents.</p><p>The intelligence may evolve, but the architecture is what compounds.</p><h3>Context Before Reasoning</h3><p>The most important design choice in Vizion is that reasoning does not start with the model. It starts with evidence.</p><p>Before an explanation is drafted, Vizion assembles the operational context around the failure: the failed run, recent changes, logs, metrics, the last successful deployment, database migrations, and any similar prior analysis. Only then does the reasoning layer generate an explanation.</p><p>That sequence keeps the system disciplined. It reduces guesswork, makes the output easier to review, and gives the engineer a clearer starting point for action.</p><p>In practice, Vizion is not just asking AI to explain a failed deployment. It is building the conditions for a better explanation before the AI is involved.</p><h3>Learning Only Matters If It Persists</h3><p>A feedback loop has value only if the system retains what it learns.</p><p>Each Vizion analysis includes a confirmation step. When an engineer approves the outcome, they capture the lesson, the recommended fix, and the scope of that lesson — whether it applies narrowly to one repository or more broadly across the platform.</p><p>An approved lesson is the engineer-confirmed explanation and recommended fix that can be safely reused when a similar failure appears again. That approved analysis becomes reusable knowledge.</p><p>The next time a failure with the same fingerprint appears, Vizion can retrieve the prior lesson and bring it into the new analysis. A fingerprint is a stable representation of the failure pattern, not just the exact log text. That allows the system to recognize similar failures even when they do not look identical line by line.</p><p>One engineer’s judgment becomes context the next engineer does not have to rediscover. That is the difference between automation that repeats work and architecture that compounds learning.</p><h3>Feedback That Improves the Platform</h3><p>The goal is not only to explain failures. The goal is to make every interaction improve the system.</p><p>Engineers can refine the analysis, request more evidence, ask for a code-level fix, or approve the lesson so it becomes reusable. Each path adds signal back into the platform.</p><p>That distinction matters. The value of AI is not only in producing an answer; it is in helping the platform become more informed, more consistent, and easier to operate over time.</p><h3>Keeping Judgment Where It Belongs</h3><p>Vizion supports operational judgment. It does not replace it.</p><p>QA is optimized for developer feedback, while PreProd and Prod require operational ownership and ticket tracking. Vizion adapts to that difference by automating ticket creation and assignment to an on-call engineer, rather than treating every failure the same way.</p><p>Vizion gathers evidence, drafts reasoning, and preserves approved lessons. The decision still belongs to the engineer responsible for the system.</p><p>Human approval is what keeps the memory trustworthy. The platform learns only from what someone has reviewed and confirmed.</p><p>In operational architecture, that distinction matters. Assistance can scale. Accountability should remain clear.</p><h3>The Next Step: Reducing Failure Before It Happens</h3><p>Everything described so far is reactive. It reduces the cost of understanding failure after it occurs. The more interesting direction is preventative.</p><p>The next capability is a readiness assessment that evaluates a change before it reaches production, using the same signals and lessons already captured by the system.</p><p>Does the change include a database modification?<br>Is it backward-compatible with the version currently serving traffic?<br>How large is the change set since the last successful deployment?<br>Does the rollout approach match the risk profile of the change?</p><p>These are the kinds of questions a delivery platform should surface early.</p><p>The objective is to enhance human decision-making by presenting risk to teams while there is still time to respond.</p><p>That is the natural extension of operational learning: not just explaining incidents better, but helping prevent avoidable ones.</p><h3>Final Thought</h3><p>Vizion exists to amplify the engineers who own the path to production.</p><p>It gives them a system that gathers context quickly, explains failures with more consistency, and preserves confirmed lessons so the same problem does not have to be solved from scratch twice.</p><p>That is where delivery architecture creates lasting value: not only in moving software safely to production, but in building systems that learn from what happens along the way — and help teams see risk before it becomes failure.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=544128aa0ff6" width="1" height="1" alt=""><hr><p><a href="https://medium.com/advisor360-com/turning-deployment-failures-into-operational-learning-544128aa0ff6">Turning Deployment Failures into Operational Learning</a> was originally published in <a href="https://medium.com/advisor360-com">Advisor360°</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[What 23,000 Deployments Revealed About Software Delivery at Scale]]></title>
            <link>https://medium.com/advisor360-com/what-23-000-deployments-revealed-about-software-delivery-at-scale-9644b7056040?source=rss----e383b54c6c43---4</link>
            <guid isPermaLink="false">https://medium.com/p/9644b7056040</guid>
            <category><![CDATA[observability]]></category>
            <category><![CDATA[platform-engineering]]></category>
            <category><![CDATA[devops]]></category>
            <dc:creator><![CDATA[Tushar Murudkar]]></dc:creator>
            <pubDate>Wed, 03 Jun 2026 21:04:42 GMT</pubDate>
            <atom:updated>2026-06-03T21:04:41.345Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*CwkF_0Q947XioHndsuT2Pw.png" /></figure><p>Software delivery at scale is more than running pipelines successfully. It is about understanding how change flows through the entire delivery system — from code to build, from build to deployment, from lower environments to production, and from failure back into improvement.</p><p>Over the last 90 days, I looked at deployment metrics across teams, applications, platforms, and environments. The goal was not to create another reporting dashboard. The goal was to understand the delivery system itself: where change moved well, where friction repeated, and where platform design could make delivery safer and more predictable.</p><p>In that period, we captured more than <strong>23,000 deployment events across 20 teams and over 200 applications</strong>. The value was not in the number alone. The value was in what the patterns revealed.</p><h3>Deployment Metrics as an Architecture Signal</h3><p>A deployment is not just a CI/CD event. It is a signal that many parts of the delivery architecture are working together: source control, workflow design, artifact management, environment configuration, approvals, infrastructure, secrets, runtime platform, observability, and release process.</p><p>When deployments are measured consistently, they provide a practical view into the health of that architecture. They show whether change is flowing through the intended path, where teams experience friction, and where the platform needs stronger defaults or better guardrails.</p><p>That is why deployment data should not be treated only as operational reporting. It can become architectural feedback.</p><h3>Volume Shows Movement, Not Maturity</h3><p>Deployment volume showed that teams were actively moving change through the platform. That is a positive signal, but volume alone does not prove delivery maturity.</p><p>From an architecture lens, the more important question is whether change is flowing consistently, reliably, and through the intended paths. Activity matters, but flow matters more.</p><p>A mature delivery platform should make the path to production repeatable, observable, and continuously improvable. When deployment volume is viewed alongside success rate, failure reasons, hotfixes, and environment trends, it becomes a stronger signal of delivery health.</p><h3>Failure Patterns Reveal Design Opportunities</h3><p>Failure patterns were especially useful because a failed deployment is often treated as an isolated event, while repeated failures usually point to a system design opportunity. The issue may be missing validation, weak pipeline guardrails, environment drift, configuration inconsistency, unclear ownership, or poor error visibility.</p><p>The answer is not simply to ask teams to be more careful. The better question is: how do we improve the delivery architecture so this type of failure becomes less likely?</p><p>That may mean moving validation earlier, improving deployment templates, strengthening defaults, making error messages clearer, or creating better self-service guidance for teams.</p><h3>Environment Flow Shows Delivery Health</h3><p>Development, QA, pre-production, and production environments should each add confidence before change reaches users. When deployment activity is concentrated in one part of the lifecycle, it may indicate active development, but it may also point to rework, validation gaps, promotion friction, or release pressure.</p><p>For a DevOps architect, environment flow is a key signal because environments should not just be checkpoints. They should be part of a progressive confidence model where each environment reduces uncertainty instead of simply moving change to the next stage.</p><h3>Hotfix Trends Expose System Pressure</h3><p>Hotfixes are normal in real systems and they will happen. The concern is not that hotfixes exist; the concern is when they become a pattern.</p><p>When hotfix activity increases, it may point to pressure in testing, planning, compatibility, rollback strategy, production readiness, or deployment controls. Tracking hotfixes helps identify whether urgency is becoming part of the normal operating model.</p><p>A healthy delivery system does not eliminate every hotfix. It understands why they are happening, how often they are happening, and whether the trend is moving in the right direction.</p><h3>Metrics Should Improve the System, Not Judge the Teams</h3><p>Deployment metrics need context. A team with more failures may own more complex services, deploy more frequently, carry more legacy dependencies, or operate with more constraints. Metrics without context can lead to the wrong conclusions.</p><p>Used well, metrics help identify where teams need better paved roads, stronger defaults, clearer standards, earlier guardrails, or better enablement. The purpose is not to rank teams; the purpose is to improve the delivery system around them.</p><h3>From Dashboard to Feedback Loop</h3><p>The dashboard itself is not the outcome. The outcome is the feedback loop it creates.</p><p>When deployment activity is visible, repeated issues become easier to prioritize, platform improvements become easier to justify, and standards become easier to measure. A mature DevOps platform should not only automate deployments. It should learn from how teams deliver and continuously reduce friction in the path to production.</p><p>That is where deployment metrics become valuable: they connect delivery activity back to platform design.</p><h3>Where This Goes Next</h3><p>Deployment metrics are one layer of the engineering delivery picture. The next step is connecting them with other signals such as change failure rate, lead time, rollback trends, incident data, security gate performance, code quality, and production reliability.</p><p>Together, these signals can provide a more complete view of software delivery health. The goal is not to measure everything. The goal is to measure the signals that help make software delivery safer, faster, and more predictable.</p><h3>Final Thought</h3><p>The biggest takeaway is this: visibility is not just an operational benefit. It is an architectural input.</p><p>It shows how change flows, where friction repeats, where standards are working, and where the platform needs to evolve.</p><p>That is where DevOps architecture creates real value — not only by building pipelines, but by designing a delivery system that continuously improves how software moves from idea to production.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9644b7056040" width="1" height="1" alt=""><hr><p><a href="https://medium.com/advisor360-com/what-23-000-deployments-revealed-about-software-delivery-at-scale-9644b7056040">What 23,000 Deployments Revealed About Software Delivery at Scale</a> was originally published in <a href="https://medium.com/advisor360-com">Advisor360°</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Architecture Doesn’t Arrive Fully Formed]]></title>
            <link>https://medium.com/advisor360-com/architecture-doesnt-arrive-fully-formed-5a66b59ee537?source=rss----e383b54c6c43---4</link>
            <guid isPermaLink="false">https://medium.com/p/5a66b59ee537</guid>
            <category><![CDATA[software-architecture]]></category>
            <category><![CDATA[software-engineering]]></category>
            <category><![CDATA[system-design-concepts]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <dc:creator><![CDATA[Pritam Mondal]]></dc:creator>
            <pubDate>Mon, 01 Jun 2026 12:06:25 GMT</pubDate>
            <atom:updated>2026-06-01T12:06:23.770Z</atom:updated>
            <content:encoded><![CDATA[<p><em>How tracing one realistic query changed our entire graph topology — and the design principle it left behind.</em></p><h4>The most expensive structure is one that doesn’t do anything</h4><p>We discovered this while tracing an early graph design for our AI advisor platform. On paper, the design was logical. The topology was clean. In practice, something about it didn’t fit — and tracing it end-to-end showed us exactly what.</p><p>Here’s what we explored, what we found, and the design principle it produced.</p><h4>The first structural approach</h4><p>Our first approach organised the graph by domain. Each domain — household data, account information, holdings — had its own subgraph: a self-contained execution layer with its own state type and internal routing logic.</p><p>The reasoning was coherent. Financial advisory data is genuinely domain-specific. Isolating each domain would let engineers work on them independently. The structural boundaries felt like a natural fit for the problem space.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vuouQb2zHWOzat5G2SJuzw.jpeg" /></figure><h4>What tracing revealed</h4><p>When we traced a realistic query end-to-end — an advisor asking about a client’s household holdings — we watched execution move through the subgraph boundary and into the domain-specific nodes.</p><p>Most of those nodes were doing the same thing every time. Given a resolved client entity and a known intent, the data fetch was always the same. The format step was always the same. The subgraph added a structural boundary — but contributed no capability that a simple flat node couldn’t provide.</p><p>The structure was implying variability that wasn’t there.</p><p>The difference is easier to see in code than in prose:</p><pre># Early design — a subgraph per domain<br># wraps steps that are always deterministic<br>domain_graph = StateGraph(DomainState)<br>domain_graph.add_node(&quot;fetch_data&quot;, fetch_domain_data)<br>domain_graph.add_node(&quot;format_response&quot;, format_domain_response)<br>domain_graph.add_edge(&quot;fetch_data&quot;, &quot;format_response&quot;)<br><br>parent_graph.add_node(&quot;domain_workflow&quot;, domain_graph.compile())<br><br># Revised design — flat nodes on the parent graph<br># same execution path; subgraph boundary removed<br>parent_graph.add_node(&quot;fetch_domain_data&quot;, fetch_domain_data)<br>parent_graph.add_node(&quot;format_domain_response&quot;, format_domain_response)<br>parent_graph.add_edge(&quot;fetch_domain_data&quot;, &quot;format_domain_response&quot;)</pre><p>The subgraph in the first design implied that something variable could happen inside it. Nothing variable did. The boundary was a structural lie.</p><h4>The design decision</h4><p>That trace produced one principle: <strong>subgraph structure only where non-determinism genuinely requires it.</strong></p><p>The domain subgraphs were replaced by flat execution nodes on the parent graph. One subgraph was kept — entity resolution — because entity resolution is the only step in our system that produces genuinely variable outcomes. A query might resolve to one client, multiple clients, or require clarification before execution can proceed. That variability earns the subgraph boundary.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2_G4ypnQlGfzUQgUKA7k_w.png" /></figure><p>The result: a flatter, more readable graph. Less code surface. Cleaner traces in production. The same behaviour with a structure that accurately represents what the system actually does.</p><h4>Why the iteration was necessary</h4><p>We couldn’t have reached this design without exploring the first one. What the nested approach revealed only became visible when we traced a realistic query — or when a peer review asked the right question at the right moment.</p><p>Other design explorations addressed different concerns: state serialisation constraints that ruled out certain patterns, routing edge cases that surfaced in multi-turn conversations, checkpoint behaviour that differed from what the design assumed. Each exploration refined something.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*m_xd3WB69u9HZU42-bk9Uw.png" /></figure><p>Architecture that doesn’t get challenged stays wrong longer than it needs to.</p><h4>The honest framing</h4><p>Architecture isn’t delivered fully formed. A design exploration doesn’t mean “this is what we shipped” — it means “this is where our understanding was at this point.”</p><p>The value of tracking design iterations is that it makes the evolution visible. When a future team member asks “why didn’t we organise it this way?” — the answer exists. Not in someone’s memory.</p><p>The current design will have a successor. Something in it will reveal itself as worth reconsidering once enough of it is built. That’s not a problem — that’s the process working correctly. What’s next is building enough of it to find out what that something is.</p><p><em>Part 3 of a series on designing and building a production AI platform for financial advisors.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5a66b59ee537" width="1" height="1" alt=""><hr><p><a href="https://medium.com/advisor360-com/architecture-doesnt-arrive-fully-formed-5a66b59ee537">Architecture Doesn’t Arrive Fully Formed</a> was originally published in <a href="https://medium.com/advisor360-com">Advisor360°</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[From Chatbot to AI Platform: What Enterprise Scale Taught Us About Agentic Design]]></title>
            <link>https://medium.com/advisor360-com/from-chatbot-to-ai-platform-why-simple-wasnt-enough-e0e5b1d3fab9?source=rss----e383b54c6c43---4</link>
            <guid isPermaLink="false">https://medium.com/p/e0e5b1d3fab9</guid>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[agentic-ai-architecture]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[agents]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Pritam Mondal]]></dc:creator>
            <pubDate>Mon, 11 May 2026 12:32:19 GMT</pubDate>
            <atom:updated>2026-05-11T12:32:18.178Z</atom:updated>
            <content:encoded><![CDATA[<h4>Solving Context Bloat: How a Discovery at Enterprise Scale Led Us <strong>to Re-architect Our AI Platform</strong></h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5EjCRDYe12SDvvOnxWtEjg.jpeg" /></figure><h4>It started as a help chatbot</h4><p>In late 2023, we shipped a virtual assistant for financial advisors. Its job was clear and well-defined: help advisors navigate the platform — answer questions about features, guide them through workflows, surface the right information at the right time.</p><p>It was a proper production application. Not a demo, not a prototype. It did exactly what it was designed to do, and it did it well. Advisors used it. It reduced friction. That product is still live today, still doing its job.</p><h4>We saw something bigger</h4><p>As the AI landscape matured through 2024 — agents, tool use, MCP — we started watching something interesting happen. Advisors were experiencing agentic AI in other parts of their professional lives. The question we kept coming back to internally was: <em>what would it look like if our platform didn’t just answer questions, but actually worked alongside advisors in their day?</em></p><p>Not a chatbot that explains how to view a household. A platform that surfaces the household, analyses the data, and helps the advisor act on it — all through a single conversation.</p><p>That question — what if the platform could work <em>for</em> advisors, not just <em>respond</em> to them — is what set the re-architecture in motion.</p><h4>Pushing the limits — what we discovered at scale</h4><p>To explore that ambition, we pushed into agentic territory using LangGraph ReAct agents. The results were immediately compelling — multi-step reasoning, the ability to pull together information from multiple data sources in a single interaction, genuinely intelligent responses to complex queries.</p><p>So we went further. We started stress-testing this pattern against real enterprise-scale data. That’s where we discovered something specific and interesting.</p><p>Financial advisors work with large, complex households. A single household can have dozens of accounts and hundreds of positions across multiple entities. When a ReAct agent answers a question about that household, it reasons iteratively — fetching data, observing it, deciding what else it needs — before producing a response. Each reasoning step pulls more data into the context window.</p><p>At small scale, this is fine. At enterprise scale, the effect compounds. The model processes far more data than it needs to answer the question. The signal gets lost in the noise. Latency climbs. Response quality starts to degrade. And the richer the household data, the worse it gets.</p><p>What we discovered wasn’t a failure of the pattern. ReAct is genuinely powerful for exploratory, open-ended tasks — and that remains true. What we found was a precise signal about what a platform serving enterprise-scale financial data actually needs architecturally: a smarter approach to matching the execution pattern to the nature of the query.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Lamj7LxU-hbAbbgm5c0Zbw.jpeg" /></figure><h4>The architectural insight</h4><p>That signal led to a clean observation: advisor queries exist on a spectrum.</p><p>Some are well-defined — the advisor knows exactly what they want, the intent is clear, and the answer requires precise retrieval of specific data. Others are genuinely exploratory — open-ended analytical questions where iterative reasoning is exactly the right tool.</p><p>A uniform ReAct loop applies the same overhead to everything, regardless of where a query falls on that spectrum. A clear, deterministic query pays the full cost of iterative reasoning without gaining any of its benefit. At enterprise scale, that cost compounds across every query, every session, every advisor.</p><p>The architectural decision that followed was clear: build a platform designed for the full spectrum from day one.</p><ul><li><strong>Well-defined queries</strong> — classified by intent, routed deterministically to purpose-built handlers that fetch only the fields the query requires. Fast. Bounded. No reasoning loop overhead.</li><li><strong>Exploratory queries</strong> — handled by a more capable execution path where iteration is genuinely warranted. Higher cost by design, used when the query actually needs it.</li></ul><p>That’s not a patch on what existed. It’s a different category of platform — one designed to carry every capability we want to bring to advisors, now and in the future.</p><p>The split itself is the straightforward part. Designing the system that reliably knows which path a query belongs on — at runtime, without introducing its own latency overhead — is where the real architectural work lives. That’s what the rest of this series is about.<br>What we’re building</p><p>The platform we’re building isn’t a smarter version of a help chatbot. It’s a skill-based orchestration layer designed to carry every workflow an advisor touches — account data, holdings, CRM, client relationships — through a single, intelligent chat interface. Built to scale with the complexity of enterprise financial data, and designed from the ground up to absorb every capability we want to bring to it.</p><p>The rest of this series goes deep on the decisions behind that build — the graph design, the routing architecture, the security boundaries, and everything in between. We’re building something we’re genuinely proud of. And we’re just getting started.</p><p><em>This is Part 1 of a series on designing and building a production AI platform for financial advisors.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e0e5b1d3fab9" width="1" height="1" alt=""><hr><p><a href="https://medium.com/advisor360-com/from-chatbot-to-ai-platform-why-simple-wasnt-enough-e0e5b1d3fab9">From Chatbot to AI Platform: What Enterprise Scale Taught Us About Agentic Design</a> was originally published in <a href="https://medium.com/advisor360-com">Advisor360°</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Rethinking Triage: Designing Systems for Routing, Context, and Ticket Quality]]></title>
            <link>https://medium.com/advisor360-com/rethinking-triage-designing-systems-for-routing-context-and-ticket-quality-6a8a95dd79e2?source=rss----e383b54c6c43---4</link>
            <guid isPermaLink="false">https://medium.com/p/6a8a95dd79e2</guid>
            <dc:creator><![CDATA[Sofiadhirschmann]]></dc:creator>
            <pubDate>Tue, 28 Apr 2026 20:47:56 GMT</pubDate>
            <atom:updated>2026-04-28T20:47:54.731Z</atom:updated>
            <content:encoded><![CDATA[<p>As systems grow in complexity, triage becomes more than sorting and prioritizing work. It becomes a process of interpreting incomplete information and deciding where that work should go.</p><p>At Advisor360, this process was initially supported by a structured intake system for the non-production development team. Tickets included titles, descriptions, screenshots, and environment details, and were posted to a shared Slack channel for triage.</p><p>The result was a centralized and consistent workflow, but one that depended on human coordination. Routing decisions required implicit knowledge of team ownership and an understanding of past work. Developers would either assign tickets themselves or defer to managers when ownership was unclear.</p><p>As features and customer usage grew, ticket volume and complexity increased. Routing required more coordination and shared context, pulling manager time toward interpretation and assignment and away from higher-impact work.</p><p>This raised a broader question: what would it look like to design a system that reduces the need for manual interpretation and implicit knowledge in triage?</p><p>At its core, triage is a routing problem. But effective routing depends on two things: high-quality inputs and access to relevant solution context. We structured the system around these three components, each addressing a different point of friction in the triage workflow.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_zkEU3CFk07rwDD7vhoODA.png" /><figcaption><strong>Ticket Lifecycle Submission to Resolution:</strong> Tickets move from Slack intake through validation, LLM-based team routing, auto-assignment, and resolution, with non-blocking feedback for missing information and outcomes stored to support similarity matching and solution context.</figcaption></figure><p><strong>Auto-Assignment: Building a Responsibility Model</strong></p><p>In December 2025, we introduced an initial LLM-based approach to ticket routing. Before this, we explored a traditional feature engineering approach, but it struggled to capture the complexity of the problem. Ticket routing is not just classification, it requires interpreting incomplete, heterogeneous data and mapping it to implicit ownership models across teams.</p><p>LLMs proved more effective in this context. Their ability to understand semantic relationships allowed them to reason over loosely structured inputs and make more accurate routing decisions. Rather than relying on handcrafted features, we used the LLM as a unifying layer to interpret tickets and extract relevant signals.</p><p>However, performance depended on grounding the model in how teams actually think about ownership.</p><p>We began with a simple prompt using high-level descriptions of team responsibilities. Even this baseline performed well, which validated the direction. From there, we focused on capturing how managers actually think about ownership during triage.</p><p>We worked directly with team managers to make that knowledge explicit, documenting ownership in terms of services, endpoints, and responsibilities. This was incorporated into the system through prompting and endpoint mapping, allowing the model to reflect a shared and consistent view of how tickets should be routed.</p><p>As shown in the chart below, once auto-assignment was introduced, time to first assignment dropped significantly and became more consistent. Before this, assignment times were highly variable and depended on when someone noticed the ticket. Afterward, routing became near-immediate. If a ticket was not manually assigned within the first five minutes, it was automatically assigned, removing the need for constant monitoring of the triage channel.</p><p>This metric reflects a shift from reactive triage to more stable and predictable routing.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*afRTo9IJr1ZWPaL7iMXlWA.png" /><figcaption>Auto-assignment reduces time to first assignment and decreases variability, indicating faster and more consistent routing.</figcaption></figure><p>The effects go beyond speed. The chart below shows that workload distribution became more balanced. Previously, Enterprise Shared Services often served as the default for unclear or edge-case tickets and was responsible for either resolving them or rerouting them. By the end of March, assignments were more evenly distributed, reducing that dependency.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*XlavI9SREH7hfvfJchDS4g.png" /><figcaption>Enterprise Shared Services, which frequently served as the default for unclear or edge-case tickets, saw its share of volume decrease from 20% to 5% following the December 2025 auto-assignment rollout.</figcaption></figure><p><strong>Resolution Context: Turning Past Work Into Starting Points</strong></p><p>Routing addresses where work should go, but not how developers begin solving it. Once assigned, developers were often starting from scratch, even when similar issues had been solved before.</p><p>We introduced a similarity-based system to surface relevant past tickets and their resolutions.</p><p>Historical resolved tickets are embedded into a semantic index using OpenAI embeddings. New tickets are embedded on creation and compared against this index using cosine similarity. The system returns the most similar resolved tickets, including their root causes, fixes, and verification steps.</p><p>This shifts the workflow from searching manually or asking around to starting with relevant solution context immediately available.</p><p>To support this, we also improved how tickets are closed by prompting developers to document: root cause, fix applied, and verification.<br>This ensures that past work becomes reusable, not just archived.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9iGQ-ivb8TDjhJ94oht8NQ.png" /><figcaption><strong>AI Similarity &amp; Recommendations: </strong>Resolved tickets are embedded as memory, new tickets are embedded on arrival, and similarity search returns the two most relevant past tickets to speed up resolution.</figcaption></figure><p><strong>Reporting Quality: Supporting Clarity Without Rigidity</strong></p><p>The effectiveness of both routing and similarity matching depends on input quality. In practice, ticket descriptions were often incomplete, requiring follow-up and slowing down triage.</p><p>We designed a system that encourages better inputs while allowing uncertainty.</p><p>We introduced an LLM-based validator that checks whether a ticket description includes three key dimensions: what happened, where it happened, and when it happened or how it can be reproduced.</p><p>If information is missing, the system comments directly in the ticket’s Slack thread with specific feedback about what is needed. This feedback is non-blocking, allowing tickets to be created while still encouraging more complete inputs.</p><p>Together, these components reduce the amount of manual interpretation required during triage. Routing decisions no longer depend entirely on individual knowledge, past work is easier to access, and tickets arrive with more consistent context.</p><p>More broadly, this reflects a shift in how triage systems can be designed. Rather than relying on continuous human coordination, they can be structured to make implicit knowledge explicit, reuse past work, and support better inputs from the start. As systems continue to grow, these patterns become increasingly important, not just for efficiency, but for making work more scalable and predictable.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6a8a95dd79e2" width="1" height="1" alt=""><hr><p><a href="https://medium.com/advisor360-com/rethinking-triage-designing-systems-for-routing-context-and-ticket-quality-6a8a95dd79e2">Rethinking Triage: Designing Systems for Routing, Context, and Ticket Quality</a> was originally published in <a href="https://medium.com/advisor360-com">Advisor360°</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[What Would Freud Say About Agentic Trust Frameworks?]]></title>
            <description><![CDATA[<div class="medium-feed-item"><p class="medium-feed-image"><a href="https://medium.com/advisor360-com/what-would-freud-say-about-agentic-trust-frameworks-8616e2cbd09f?source=rss----e383b54c6c43---4"><img src="https://cdn-images-1.medium.com/max/1376/1*sorXyuMKiN-rxrATSmIEBw.png" width="1376"></a></p><p class="medium-feed-snippet">How psychology, sociology, and political science influence your agentic design.</p><p class="medium-feed-link"><a href="https://medium.com/advisor360-com/what-would-freud-say-about-agentic-trust-frameworks-8616e2cbd09f?source=rss----e383b54c6c43---4">Continue reading on Advisor360° »</a></p></div>]]></description>
            <link>https://medium.com/advisor360-com/what-would-freud-say-about-agentic-trust-frameworks-8616e2cbd09f?source=rss----e383b54c6c43---4</link>
            <guid isPermaLink="false">https://medium.com/p/8616e2cbd09f</guid>
            <category><![CDATA[agentic-ai]]></category>
            <category><![CDATA[trust]]></category>
            <dc:creator><![CDATA[Manu Cohen-Yashar]]></dc:creator>
            <pubDate>Sat, 04 Apr 2026 22:08:40 GMT</pubDate>
            <atom:updated>2026-04-04T22:08:39.519Z</atom:updated>
        </item>
        <item>
            <title><![CDATA[The Lie That Looks Like a Fact: Fabrication Risk in AI for Regulated Industries]]></title>
            <link>https://medium.com/advisor360-com/the-lie-that-looks-like-a-fact-fabrication-risk-in-ai-for-regulated-industries-11aea6a2c655?source=rss----e383b54c6c43---4</link>
            <guid isPermaLink="false">https://medium.com/p/11aea6a2c655</guid>
            <category><![CDATA[agentic-ai]]></category>
            <category><![CDATA[enterprise-architecture]]></category>
            <category><![CDATA[ai]]></category>
            <dc:creator><![CDATA[Venkat Peri]]></dc:creator>
            <pubDate>Fri, 03 Apr 2026 15:32:41 GMT</pubDate>
            <atom:updated>2026-04-03T15:32:40.450Z</atom:updated>
            <content:encoded><![CDATA[<h4><em>Part of a series on building trustworthy autonomous AI for wealth management.</em></h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*y6rdPyuAnnKvpPWVn2PJdQ.png" /></figure><p>There is a failure mode in AI systems that the industry consistently underestimates, because it does not look like a failure at first glance.</p><p>It does not look like a garbled response or a nonsensical output. It does not trigger an error. It passes a formatting check. It reads confidently, fluently, and specifically. It might describe a client’s concern expressed at an October meeting, a phone call about an IRA rollover in November, a documented anxiety about retirement timing that shaped everything discussed since. The advisor reads it, trusts it, and walks into the room ready to reference those moments.</p><p>None of them happened.</p><p>This is the failure mode I want to talk about: not hallucination in the abstract, but fabrication of documented specifics in contexts where trust in the output is not optional. In regulated industries — wealth management, healthcare, legal services — an AI that invents plausible facts and presents them as documented history is not producing a lower-quality output. It is producing a trust hazard. The distinction matters because it changes what you build to prevent it.</p><h3>The Pattern</h3><p>The pattern emerges from a straightforward problem. A generative model is asked to produce something — a meeting agenda, a case summary, a client brief — that is supposed to be grounded in a specific person’s documented history. The model is good at producing fluent, well-structured outputs. It has seen enough examples to know what a meeting agenda looks like, what a prior interaction summary sounds like, what the language of documented client concern feels like.</p><p>When the underlying data is rich, this works. The model draws on real CRM notes, real meeting transcripts, real documented concerns and goals, and produces something genuinely personalized and accurate.</p><p>When the data is sparse — a client with no meaningful CRM history, no documented interactions, just a portfolio snapshot — the model faces a choice it is not always equipped to make consciously: acknowledge the gap, or fill it.</p><p>It fills it. Not because it is broken. Because it was not told it was operating in a data desert.</p><p>The generated output describes specific past interactions using the same voice and confidence as an output that had real interactions to draw on. A reader cannot tell the difference from the text alone. Neither can most automated checks that evaluate formatting, coverage, or even general compliance posture. The fabricated interaction is grammatically correct, appropriately structured, and stylistically consistent with everything around it.</p><h3>Why This Is Harder Than It Looks</h3><p>The instinctive response is to call this a hallucination problem and reach for the standard toolkit: retrieval-augmented generation, better grounding, stricter prompts. Those tools help. But they do not fully address the structural cause.</p><p>The structural cause is that the generator has no reliable signal about how much real, usable data it actually has. It receives a data payload and produces an output. If the payload is sparse, it cannot distinguish between “I don’t have this information” and “this information wasn’t included in what I received.” Without that distinction, it defaults to producing a complete-sounding output — because completeness is what it has been optimized to produce.</p><p>This means that prompt engineering alone — “don’t make things up,” “only use documented facts” — helps at the margins but does not solve the problem. A model that doesn’t know it’s in a data desert will not reliably recognize that it’s fabricating. It is doing what it was asked to do: produce a comprehensive, personalized document. The problem is that comprehensiveness in the absence of data becomes fabrication by construction.</p><h3>What Catching It Requires</h3><p>When you have latency budget — in batch workflows, in overnight processing, in any context where outputs are generated before they’re needed — you can catch fabrication through post-hoc evaluation with access to ground truth.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*uKG7lg5zxk2NnbN8BrSB_g.png" /></figure><p>The approach is an evaluation pipeline that scores outputs against source data, not just against internal consistency. The key move is keeping input evaluation and output evaluation completely separate. Input evaluation asks: how much usable data does the generator actually have? Output evaluation asks: is what the generator produced grounded in that data, or did it invent?</p><p>Keeping these two pipelines independent is what makes fabrication visible as a distinct failure mode. If you collapse them into a single quality score, a well-written fabricated output can score well on accuracy (internally consistent), coverage (all sections present), and compliance (no regulatory violations) while containing invented specifics that the advisor will treat as documented facts. The fabrication hides inside an otherwise passing output.</p><p>This architectural separation — input evaluation entirely independent of output evaluation — turns out to be a general principle, not a domain-specific one. Anthropic’s engineering team arrived at the same structural insight while building a <a href="https://www.anthropic.com/engineering/harness-design-long-running-apps">harness for long-running agentic coding</a>: separating the agent doing the work from the agent judging it is a strong lever, and tuning a standalone evaluator to be skeptical is far more tractable than making a generator critical of its own work. Their evaluator needed access to a running application — ground truth the generator didn’t have at write time. The wealth management equivalent is an evaluator with access to source data that the generator processed but could not audit against itself. Same separation, different domain.</p><p>The failure signal that surfaces fabrication specifically is personalization depth: does this document draw on real, sourced details from this client’s history, or does it read like a template that could belong to anyone? And within that dimension, there is a harder check: are there specific claims presented as documented facts that cannot be traced to any source in the input data?</p><p>That check — source-traceable specifics — is what distinguishes fabrication from appropriate inference. An output that says “given the portfolio’s equity concentration, liquidity may be a concern” is an inference, and an appropriate one. An output that says “as discussed in the November call, you expressed concern about liquidity ahead of your daughter’s tuition payment” is a claim about a documented event. If that event is not in the source data, it is a fabricated fact.</p><p>The fabrication gate fires first, before any other quality routing, because a fabricated output that happens to score well on everything else is more dangerous than a lower-quality output that is honest about what it does and does not know.</p><h3>The Latency Problem</h3><p>This is where the architecture gets honest about its own limits.</p><p>The evaluation approach described above is not viable in real-time. A conversational AI answering questions about a client account, a chat-based assistant responding to an advisor’s query mid-meeting, a voice interface fielding questions about account history — none of these have the latency budget to run a separate evaluation pipeline before producing a response. The same failure mode exists. The same risk is present. The detection mechanism is not available.</p><p>This is not a solved problem, and I do not think it is useful to pretend otherwise.</p><p>What the batch experience does establish is a design principle that translates across latency budgets: the generator needs a context-awareness signal before it writes. In batch, that signal comes from a structured input evaluation — a score, a band, a data-availability summary — passed into the generation system prompt. The generator is told, explicitly, how much real data it has, and instructed to behave differently depending on the answer. When data is sparse, it produces a shorter, more conservative output that names the gaps rather than filling them. When data is rich, it draws directly from what is there.</p><p>In real-time contexts, the same principle applies at a different granularity. Before generating a response, the system should know — and communicate to the model — what is actually available in the context window for this specific query. Not just whether data exists, but whether it is fresh, sourced, and traceable. A system that has accessed a client’s real-time account data should signal that to the model differently than a system pulling from a week-old cache or producing an answer from general patterns. The model’s confidence calibration — and its tendency to fabricate — responds to these signals when they are explicit.</p><p>The implementation of this in a low-latency context is genuinely difficult. But the absence of a clean implementation is not an argument for skipping the signal. It is an argument for finding the cheapest version of it that still changes the model’s behavior in the right direction.</p><h3>The Broader Principle</h3><p>The industry conversation about AI trust in regulated industries tends to focus on authorization and compliance: is the agent allowed to do this, does the output avoid regulatory violations, is there a human in the loop at the right moments. Those are real and important questions. Prior pieces in this series have addressed them directly.</p><p>But there is a distinct layer of trust that authorization and compliance do not cover: the factual integrity of the output itself. A system can be fully authorized, completely compliant, and human-reviewed at every decision point, and still produce outputs that contain fabricated specifics that an advisor or clinician or attorney will treat as documented history.</p><p>That failure does not announce itself. It passes most automated checks. It reads well. And in a domain where a professional’s relationship with a client rests partly on the accuracy of what they believe they know about that client’s history, the failure cost is not a bad UX outcome. It is an erosion of the trust that the AI was supposed to support.</p><p>Building systems that prevent this — at whatever latency budget is available — requires treating fabrication as a distinct failure mode with its own detection strategy, not as a special case of general output quality. The first step is recognizing it as such.</p><p><em>The author leads AI and data infrastructure for wealth management at Advisor360°. The observations in this piece come from building and evaluating production AI systems that generate client-facing and advisor-facing materials at scale.</em></p><p><em>Part one of this series: [Authorization and the Decision Gateway]</em> <em>Part two of this series: [The Confidence Plane and Intent Plane]</em> <em>Part four: forthcoming</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=11aea6a2c655" width="1" height="1" alt=""><hr><p><a href="https://medium.com/advisor360-com/the-lie-that-looks-like-a-fact-fabrication-risk-in-ai-for-regulated-industries-11aea6a2c655">The Lie That Looks Like a Fact: Fabrication Risk in AI for Regulated Industries</a> was originally published in <a href="https://medium.com/advisor360-com">Advisor360°</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>