AI initially evolved as the ultimate tool to generate and develop code for software developers. Like the software development industry, the legal industry has also focused heavily on AI’s ability to produce work, specifically around drafting contracts or briefs — a crucial part of the legal workflow . However, its most consequential use may be to check whether that work (human or AI), and every step behind it, has been done correctly.

Speed alone does not establish value. As Richard Tromans points out, saving half an hour with AI achieves little if a lawyer then spends half an hour checking the result. Joshua Yuvaraj calls this the “verification-value paradox”. AI can produce legal work faster, but lawyers remain responsible for checking that it is accurate, complete and relevant. The more work AI generates, the more there is to verify. He argues that this checking can consume much of the promised efficiency gain, leaving little net value once the cost of verification is counted.

That raises a different question: Are we overlooking the part of the legal value chain where AI could have its greatest impact? Furthermore, rather than producing more legal documents, should legal AI focus on delivering higher quality results — more precision and accuracy for attorneys?

AI’s greatest value may be as an auditor

The dynamics and economics of the legal AI industry have made drafting a natural starting point. Its outputs are visible, and time savings are readily measured. The value of auditing is harder to demonstrate in a product demo: it lies in errors prevented and failures that never happen.

What if the most valuable application of AI is checking the work that humans already do?

An AI-native firm’s advantage lies in its ability to invest in long-term auditing capabilities whose return comes through better service, even when these investments and product features are not directly revenue generating.

In his analysis of 9,241 merger agreements, Dave Hoffman reported that typo patterns are more strongly associated with particular lawyer teams than with the firm itself. That strengthens the case for making quality control part of a firm’s infrastructure. An AI auditing system could apply the firm’s required checks across teams and matters, making the standard of checking less dependent on which lawyers handle the file.

A good draft does not prove a properly completed process

The distinction I am making is between evaluating an AI-generated draft and recognizing the value of AI as a general auditor in legal processes. As Richard Tromans puts it: “A law firm is not just an economic pyramid of legal labour, it is a pyramid of quality control.”

The idea of legal “evaluations” has frequently been applied in the context of evaluating outputs. However, checking an output does not establish that every required step in the matter was completed.

A good draft is not proof of a properly completed process. A demand letter may read as complete while leaving out the client’s MRI entirely. The client mentioned it at intake and the firm requested the report, but it never arrived and nobody followed up. Nothing in the letter reveals the gap; it only shows up when the draft is checked against the intake notes and the records log. An AI auditor could check the file against the evidence required for the matter.

A point many engineers miss is that much of the work in a law firm involves running processes that sit outside a Word Document, including developing and executing legal strategies and processes across high-volume personal injury, e-discovery or complex, multifaceted due diligence.

Auditing should therefore cover people, processes and outputs. Were all the required procedural steps completed? Did information get lost between teams? Is a legal conclusion adequately supported, or was material evidence overlooked?

The auditing task starts with defined checks and evidence, then identifies departures requiring attention. In law, each step matters even when the answer seems obvious and checking feels pointless. Triple-checking that a legal process has crossed the t’s and dotted the i’s is part of the work itself.

This audit task represents the work of a junior- and mid-level associate. The attention to detail that lawyers require develops from lengthy, time-tested checklists of specific items to verify. This work, unlike the more complex nuances of swaying judicial opinion based on caselaw, can easily be verified and improved with AI.

Secondly, much of this audit work is where human error occurs. Attorneys burning the midnight oil often wait to verify citations or triple-check for typos at the last minute when they are simply too tired to function effectively. An audit function well defined with AI does not make those errors.

AI can make comprehensive quality control affordable

Comprehensive auditing was historically constrained by the cost of expertise. A more experienced lawyer had to check a junior person’s work. Trust was the currency of the realm, because it was one of the highest leverage tools for a senior partner to grow their business without being constrained by the time required in quality control.

Writing down the checks did not remove the cost of performing them, but AI changes the economics of running those checks. We can separate the expertise required to design an audit from the effort required to run it across thousands of matters.

Errors prevented matter more than hours saved

The economic case depends on catching more errors than existing supervision at an acceptable cost of human review. Each finding should point to evidence, and the audit itself should be tested against known failures.

Success should be measured through errors prevented, missing evidence identified, incorrectly rejected cases recovered and process failures caught before harm occurs. Hours saved and documents generated tell only part of the story.

Applied to quality control, AI could improve the reliability of the whole legal value chain. We can now build audits designed by humans but run by machines.

The winning firm will turn legal expertise into auditing infrastructure

The AI-native law firm that wins will build the best infrastructure to lower the friction in turning the legal expertise trapped in lawyers’ heads into automated AI auditing systems.

The friction in extracting that expertise is the hard part. As Josh Kubicki puts it, “Most law firms rolled out AI without ever defining the exam it needs to pass.” We need to take the checks experienced lawyers carry out in their heads and build them into an AI auditing system that performs those checks automatically across matters. That means asking lawyers what they check and why, then translating their answers into instructions the system can follow: which documents to examine, what evidence to look for, which procedural steps to verify and when to alert a lawyer.

Human oversight remains essential, but the opportunity is to make expert supervision more consistent and scalable. Thomas Potter proposes using AI for structured verification, with lawyers concentrating their judgment on the issues those checks flag. Lawyers must review significant findings, resolve ambiguity and take responsibility for decisions.

Law firms have long rationed quality control because experienced lawyers have limited time. AI gives us a way to apply their expertise across every matter through audits they design and oversee. The value lies in making that level of care routine. A single material error caught can be worth more than hundreds of hours saved drafting.

The Best Legal AI Tool Might Be a Legal Auditor was originally published in Foremark Legal on Medium, where people are continuing the conversation by highlighting and responding to this story.

An AI-native firm is not a traditional firm that bought AI tools. It is a firm designed from the ground up around new capabilities, the way Amazon was not just a bookstore with a website.

The AI-native firm changes the form of the three functions of finding the business, minding the matter, and grinding out the work. The finder industrializes into a firm-owned acquisition engine, because as AI expands capacity the bottleneck moves from lawyer hours to quality clients. The minder’s craft, which cases to take, when to escalate, when to settle, is written down and encoded into systems, run by a small self-contained unit (“Pod”). The grinder’s production is absorbed by machines, and the lawyer moves up to developing the legal theories that open new case types.

As you cannot bill an agent’s hour, the economics must change with the business of law and the organization of the law firm. The firm shifts from selling time to underwriting outcomes. It evaluates a case up front the way an insurer evaluates a policy and ties its fee to the result instead of the clock.

Organizational

• Empowering Practice Heads with Quant Fund style Pod teams: The AI-native firm runs each practice area as a pod with dedicated engineering and marketing resources with its own P&L.
• Decentralizing Ops to the Practice Area level: The traditional firm only has a firm-level COO, with operations as overhead nobody owns. The AI-native firm additionally gives practice area heads dedicated biz ops leads inside the pod, to ensure that each practice-area gets adoption right.
• Measuring Client NPS: The traditional firm leaves the client experience unmeasured. The AI-native firm measures NPS per matter, like every other consumer industry.

Technological

• Case Data Infrastructure: The AI-native firm automatically collects data on market terms and counterparty behavior, compounding with volume, answering questions on ‘How X company reacts to Y’, or ‘Is a $500,000 settlement the average amount for cases with similar facts’.
• Legal Knowledge Infrastructure to augment the Apprenticeship model: The traditional firm transfers judgment by manual apprenticeship. The AI-native firm builds a living knowledge base based on matter correspondence model training, queryable by every lawyer in the firm.

Commercial

• Demand-Generation Engine: Client acquisition engines that are commonplace for your typical consumer or SMB tech startups become increasingly adopted by AI-native law firms, competing with legacy law firms whose client engagement strategy is manual.
• Outcome-based Business Model: The traditional firm is paid for time regardless of result. The AI-native firm acts like a case underwriting, forever changing the customer expectation, promoting contingency and flat-fee businesses that earn only when the client wins.
• Compensation structure: The traditional firm pays hours-based comp with some origination. The AI-native firm pays flat salary plus bonus plus equity, with no hourly billing. As in every other industry, good lawyers work hard because they are motivated, not because their hours are tracked. All lawyers, junior and senior, are given MSO equity, making long-term investment rational.
• Management credit: At the traditional firm, management is a tax the best lawyers avoid. At the AI-native firm, building the firm, improving processes, and expanding new practice areas is the highest-credited work in it.

The finder becomes an industrialized marketing engine

In the traditional firm, business arrives through the charisma and networks of individual rainmakers, which makes the finder the most powerful and expensive person in a firm.

In contrast, the new Finder of the AI-native firm is a consumer growth operator or a B2B sales talent. Modern startups and enterprise software companies treat client acquisition as a repeatable operating function, investing heavily in market intelligence to identify opportunities before competitors do. Consumer law and SMB companies are obvious areas of first adoption for modern client acquisition.

To give a flavor:

• Consumer law: Reimagine the client experience by replacing traditional advertising and consultations with product-led experiences that deliver immediate legal guidance, transparent case tracking, proactive communication, and self-service, setting the standard against Apple, Amazon, and Uber rather than other law firms.
• SMB law: Become the legal operating system by integrating with payroll, accounting, HR, CRM, and banking platforms to continuously monitor legal risks, automate compliance, and provide proactive guidance as businesses grow. This is similar to how traditional relationship-based SMB sales, like insurance, moved into more automated online partnership-heavy GTM motions.
• High-end transactional: Build an AI-powered opportunity engine by continuously monitoring market signals such as M&A, fundraising, litigation, regulations, executive changes, and industry news to identify emerging client needs and enable highly personalized business development.

In my conversation with Ryan Walker, ex-CTO of Casetext and CEO of General Legal, an AI-native firm focused on serving startups, he puts the constraint on the commercial rather than the technical side.

Traditional law firms generally have no GTM function, and have the expectation that business arrives inbound on reputation, with none of the machinery every tech startup built to scale a sales process.

He notes that replicating basic sales excellence techniques and the more innovative SMB sales motions that tech startups pioneer every single day, and applying them to a traditional industry like the legal industry will likely be the needle-mover to bring a tremendous advantage.

Rainmaker-only economics are deprioritized because the book of business belongs to the platform rather than to any partner. Retention inverts for the same reason: at traditional firms partners leave because their clients follow them, and the firm adds little beyond a brand and office space. At the AI-native firm, a departing lawyer walks away from a client acquisition engine, case-scoring models, and accumulated outcome data that are difficult to replicate solo.

The minder becomes the pod

The Minder is the workhorse of a traditional law firm. In litigation, this is the lawyer who keeps dozens of deadlines from slipping, decides when to push and when to settle, and deals with the client. In deal work, it is the lawyer who knows what is market in a venture financing, which terms the other side will trade and which they will fight for, and how to sequence a closing so it actually closes. In the traditional firm, all of that lives in one person’s head, which is why a matter runs only as well as the lawyer minding it, and why quality collapses the moment volume outruns the supply of careful people.

In the AI-native firm, that craft leaves the lawyer’s head and gets embedded in the machinery itself. Each practice area operates as its own decentralized Pod with a practice lead, dedicated engineers, and a dedicated business operating officer, and each pod carries its own profit and loss on top of a shared platform built on institutionalizing the Pod’s workflow. The business operations officer sits at the practice level, not only at the firm-level. Practice areas differ too much in their operations for a firm-level executive to be anything but firm-level ops management (payroll, CRM, etc).

In the AI-native firm, quality is observable per pod, per lawyer, and per case type. The client experience gets the same treatment, with NPS measured per matter.

The grinder becomes the strategic thought partner

Nobody applies to law school dreaming of document review. They apply to become the person in the room when the company’s future is being decided. The AI-native firm builds its technology stack around institutionalizing this thought leader to create the 100x Lawyer.

Theory development is this person’s highest-leverage work. The grinder develops new legal theories to pursue litigation on, and in the AI-native firm this is a research and development function with direct commercial consequence: a validated theory opens a new case type, and a case type at sufficient volume seeds a new pod. The grinder also gets involved in matters where legal creativity matters — theory selection, negotiation posture, and courtroom judgment do not resolve to pattern matching.

Research becomes quantitative. The grinder’s agents sweep case law, dockets, verdicts, and the firm’s own outcome data so that a hypothesis is tested against the whole corpus at once rather than against whatever an associate could read in a week. The lawyer directs inquiry instead of performing it.

The people model that results

Partnership comes through apprenticeship, because the old pipeline is dead. The associate model trained lawyers by burying them in production work under an apprenticeship structure, and that has not diminished significantly as laterals are prioritized in the AI-age.

The apprenticeship runs on a living knowledge base. Agents capture the firm’s emails and case correspondence, and structure them into an institutional agent memory. A young lawyer queries the knowledge base with the questions a prior generation asked a senior attorney, and gets answers drawn from how the firm’s best lawyers actually handled the situation rather than from how one mentor happened to remember it.

In my conversation with Eddie Nasser, Head of AI at Keller Postman / ex-Head of AI at Crosby, frames the progression in three stages:

“The traditional law firm apprenticeship model is single-player: one junior learns from one partner through repeated drafts and feedback.

[My] framework for AI adoption evolves that model in three stages: first, single-player querying, where a lawyer asks an AI system for help; then multiplayer querying, where an AI system can draw on the collective context and judgment of many lawyers across the firm; and finally multiplayer proactive suggestions, where agents understand what the team is doing and start surfacing the next action themselves.”

For example, noticing you are working on a particular workflow and proactively scheduling and suggesting certain actions, just like a very helpful senior partner who has desktop access to your computer.

Compensation follows a more systematic platform approach. Lawyers earn a flat salary plus bonus plus equity. Most industries, except manufacturing floors and lawyers, do not track the hours worked internally as a measure of productivity. Motivated people work hard because they are motivated and incentivized well, not because their hours are tracked.

Credit flows to the work that builds the firm in the same way it works in other industries: launching a new pod, building the systems a pod runs on, developing apprentices into partners, and lifting pod-level economics. At traditional firms, management is a tax: time spent running the firm is time not billing, so the best lawyers avoid it and firms are run by whoever could not say no. The AI-native firm credits building the firm as the highest-value work in it, and rewards the culture accordingly.

Every lawyer, senior or junior, holds equity in the management services organization. Traditional partnerships are annual profit-distribution machines with little in retained earnings, so no partner has a reason to invest in anything that pays off after retirement. Equity makes long-term investment rational, and justifies the management credit.

The Data Infrastructure difference

The shared platform rests on two data infrastructures, and they collect different data for different reasons.

The Case Data infrastructure is the firm’s Commercial Record: settlement values, case outcomes, counterparty behavior, and the negotiation patterns of counterparties like insurers.

The questions this case data infrastructure answers include:

• Based on our historical case data, if a client has X type of injuries in Georgia, what is the typical range of settlements that are offered?
• Based on our historical case data, are MFNs more common or market standard for startups investments over $50m?
• Based on our historical case data, when we deal with this insurer, what elements are they typically more sensitive to than others?

The firm collects this data because underwriting requires it; an insurer cannot price risk without actuarial history, and a contingency firm cannot decide which cases to take, what they are worth, or when to settle without an outcome history of its own.

In my conversation with Kyle Canchola, Partner at Vector Legal, an AI-native firm focused on serving investors and startups, he notes that “AI can make negotiation memory institutional: lawyers can see which positions were taken, where counterparties moved, and what tradeoffs worked across prior deals.”

The Legal Knowledge infrastructure is the firm’s Intellectual Record: the research, briefs, arguments, and case correspondence in which the firm’s lawyers actually reason.

The questions this legal knowledge infrastructure answers include:

• What fact patterns and arguments survived federal preemption in pharmaceutical warning-label cases?
• For personal injury cases, what are the different ways we have creatively tied the demands of their job to the injury at hand?
• Does Judge X favor First Amendment arguments in commercial advertising cases?

The firm collects this because judgment has always been its scarcest asset, and the one it could never store.

Eddie Nasser describes the mechanics: “The interesting question is how you actually capture legal judgment. We compare early drafts against the final drafts that went out, identify the feedback that consistently changed them, and we even have Supreme Court litigators review briefs and articulate what they would change.

Over time, that becomes a set of skills and evals that lets a junior lawyer get something closer to expert-level feedback before a draft ever reaches a partner.”

Conclusion

The traditional firm’s partner leaves and takes the clients; the AI-native firm’s lawyer leaves and takes nothing, because the acquisition engine, the scoring models, and the accumulated judgment stay.

That accumulated judgment is also what rebuilds the pipeline, now that the production work that used to train juniors is gone. The firm that builds this gets a defensible asset and a training system out of the same infrastructure; the firm that does not will find it has no seniors left to lose.

Disclaimer: Quotes / individuals mentioned do not suggest endorsement with some or all ideas in this article, and reflect the personal, not company, opinions of those quoted.

Architecture for the Outcome-Based, AI-Native Law Firm was originally published in Foremark Legal on Medium, where people are continuing the conversation by highlighting and responding to this story.

The legal system is inaccessible because legal help must be purchased before anyone knows whether a case will succeed. Rights exist on paper, but exercising them requires paying upfront for an outcome no one can guarantee.

Hourly billing forces ordinary people to absorb that uncertainty themselves. A client pays to investigate, to understand the options, to draft documents, to pursue enforcement, and plenty of valid claims are not worth pursuing once the math is done.

The next generation AI-native law firm is not a firm that uses AI. It is a firm whose economics only make sense because of AI. Contingency is the most natural expression of those economics. Contingency fees solved part of this problem by inverting who carries the risk. Lawyers took a percentage of recovery if they won and nothing if they lost, which became one of the most important access-to-justice innovations in modern legal history.

The reason contingency stayed confined to a narrow set of practice areas was generally economic. Understanding a legal problem used to require hundreds of hours on documents, damages, and procedural risk before anyone could decide if a case was viable, making contingency dangerous outside categories with very large expected recoveries.

AI breaks the constraint that kept contingency narrow. Systems can now summarize records, spot legal issues, compare claims against historical outcomes, estimate damages, and draft initial filings in minutes. The bottleneck was never appetite for risk but the cost of evaluating it, and that cost is collapsing.

The consequences of cheap evaluation extend well beyond personal injury. Wage theft alone costs American workers an estimated $50B+ a year, yet most incidents never produce a claim because damages are too small relative to legal cost. The same arithmetic kills landlord disputes, junk fees, denied benefits, and small-dollar fraud, so lowering intake and triage costs by an order of magnitude makes a huge population of real injuries viable for the first time.

Transactional work is starting to move toward contingency too. Formation, licensing, immigration, and contract negotiation have measurable outcomes that can anchor success-based fees once servicing costs fall. Eli Albrecht, an M&A lawyer, recently posted that his firm took on the largest deal in its history “mostly on contingency” after vetting the sponsor. “We are bullish on our clients,” he wrote. Operationally flexible boutiques are running these deals now, and client expectations about negotiable fees shift with every one.

Lawyers already know how to design outcome-aligned structures; they have simply built them for the wrong clients. Bankruptcy practice runs on preference stacks, success fees, litigation trusts, and structured settlements, and corporate lawyers engineer outcome-aligned deals routinely for sophisticated parties. The creativity exists; it has just been pointed at clients who could already afford lawyers.

Portfolio-based law is what this combination of capabilities makes possible. A firm can run thousands of smaller matters as a diversified book instead of placing concentrated bets on a few expensive cases, with AI compressing variance, sharpening prediction, and letting firms model legal risk the way insurers and lenders model their own. Contingency stops looking like an exception and starts looking like infrastructure.

Two objections to broader contingency come up regularly, and neither holds. The first is that it will flood courts with frivolous claims, but contingency lawyers already eat the cost of weak cases. The second, that AI cannot replace legal judgment, is true but beside the point: the argument is not that lawyers disappear, but that the cost of understanding a legal problem falls far enough to change what is worth taking on.

Outcome-based pricing is the default almost everywhere else in the economy for knowledge workers: medicine, venture capital, recruiting, asset management, and sales all run on it. Legal services have stayed anchored to hourly billing even as the marginal cost of legal work has compressed, and the profession still frames contingency as a special exception when it may be the most rational model for a system in which understanding a legal problem is cheap. The right to a contingency fee, ultimately, is the right to transfer risk from the person least able to bear it to the institution best equipped to evaluate it.

The most interesting category in legal tech is therefore not software sold to law firms but the firm itself, rebuilt around contingency economics AI has made tractable. Personal injury is the natural wedge: contingency already works there, intake volume is enormous, and the gap between marketing spend and case quality is wide enough for an AI-native operator to compound advantages quickly. US TAM is roughly $800B in resolved claim value annually, most of it priced by firms whose intake runs on rules of thumb.

From that wedge, the same operating system extends into adjacent contingency-eligible verticals and eventually into non-litigation work transactional categories where outcomes can be priced and estimated by AI systems.

The firms that win this will look less like traditional law firms with AI bolted on and more like underwriters: closed-loop systems that source claims, score them, price risk, and feed every outcome back into the model that picks the next case. That is what the next generation AI-native law firms would look like.

Forget Flat Billing — AI-Native firms will transform the Right to Contingency Fees was originally published in Messy Problems on Medium, where people are continuing the conversation by highlighting and responding to this story.

The bottleneck in Legal AI is not generating better answers. It is identifying which questions need to be asked, when they should be asked, and whether they are being asked at all.

The law is unique as a field in that there is a categorical difference between getting a question wrong and failing to ask the question that would have revealed the correct framing in the first place.

The reason lawyers are famously always saying ‘it depends’ is because giving an answer to a question is straightforward, but finding out whether the client actually fits another scenario (outside what the client knows applies to them) is what lawyers are trained to do. Clients obviously do not know the right context to give for a particular legal question, so lawyers are trained to dig up the appropriate context on their behalf. Clients may easily walk away satisfied with a surface level answer, which may not be applicable to them at all.

At the highest level, we can break legal work into 3 categories — transactional, consultative, and creative.

Transactional: This includes finding standard information, drafting paperwork to fit known scenarios, and executing predefined processes. Regulatory filings, deal documentation, and immigration forms fall squarely into this bucket. This category is the most automatable, and Legal AI has made real progress here.

Consultative: This is where a lawyer’s value comes from asking questions that materially improve a client’s legal position. It is not about executing a known path, but about discovering a better one. For example, someone may approach a lawyer asking for help with an H-1B visa. A consultative lawyer might, through questioning, realize that the person is actually better suited for a different immigration pathway they were not aware of and never asked about. The value lies entirely in the question that reframed the problem.

Creative: This involves creative interpretations of the law, novel arguments, and untested strategies. It is the least structured and most judgment-intensive form of legal work. While AI can assist here, it is also the furthest from checklist automation.

Most AI developers implicitly assume that all legal work is transactional. They are optimized for answering questions, summarizing documents, or drafting text once the problem has already been defined. It fails when the real work is figuring out what the problem actually is.

This is why lawyers are paid what they are paid. They are not primarily compensated for recall, drafting speed, or access to information. They are paid to notice what is missing, to challenge assumptions, and to ask the right question at the right moment.

Today’s Legal AI is fundamentally reactive. It waits for a prompt, a document, or a task definition. Even the most advanced systems inherit the user’s framing. If the human does not know what to ask, the system does not either.

The real inflection point comes when systems become proactive in a very specific sense: not by listing generic unknowns or abstract gaps, but by using contextual clues to determine whether alternative scenarios should be considered and which questions must be asked to confirm or rule them out.

A meaningful system is one that could interrupt a workflow to say, in effect:

“This is the wrong question for this situation. Here is the question that actually matters.” OR

“Based on our previous conversation, if you are generally less concerned with downside risk, this other thing is what you should be more concerned with.”

That is a much harder problem than it sounds, because good questions are not just about missing information. They are about assessing the probability of relevance and leverage, and asking the minimum number of relevant questions at the right time. Consider an immigration example.

A junior lawyer might ask, “It appears you are interested in applying for a H1B visa. Please show me evidence of a non-immigration intent. These are some examples”.

A more experienced lawyer might instead ask, “You have an Australian accent. Did you know it’s easier to apply for Australians to apply for an EB1 visa?”

Both are questions, but only one meaningfully shapes strategy. The difference is not access to information, but judgment about what inquiry moves the case forward.

This is also why existing legal knowledge systems fall short. Internal matter search focuses on final work product: briefs, contracts, memos. It tells you what was answered, not what was asked. It does not capture the questions that changed direction, the questions that came too late, or the questions that were never asked but should have been. As a result, firms systematically lose their most valuable form of institutional knowledge.

What progress looks like — the Question / Context Dataset

A real solution here does not look like a better answer engine. It focuses on a different dataset entirely.

Meaningful progress would require capturing how questions are actually asked inside a law firm. That means scraping internal communication at scale: emails, chat threads, document comments, and notes. This is where legal judgment lives. Not in final briefs, but in the questions lawyers ask while a matter is unfolding. Internal historical matter search is insufficient because it ignores how judgment is formed in motion.

What matters is not just the question, but when it was asked and in what scenario. The same question asked at intake, during discovery, or on the eve of a deadline carries very different weight. Each question would need to be mapped to its context, including why it was asked at that moment. Not the answer it produced, but the reasoning that made the question relevant.

This produces a fundamentally different dataset. Instead of questions paired with golden answers, it is a dataset of questions paired with intent, timing, and scenario. It captures how lawyers decide what to ask, not what they ultimately conclude.

In the immigration example used earlier, it is incredibly tedious to hard-code every possible relevant factor that could potentially be relevant (i.e. accent, etc). As such, a more practical approach would be to categorize and train on the questions that get asked more frequently in practice, and the context that triggered it.

A system trained on this kind of data would not wait to be prompted with “what are relevant questions to ask?”. It would proactively surface questions based on the probability that a given question matters in the user’s current situation, inferred from contextual clues like matter type, stage, facts, and prior decisions. Its value comes from asking the right question unprompted, early enough to change outcomes.

Until Legal AI is trained on questions rather than answers, it will remain strongest where legal judgment matters least. The real opportunity is not to automate execution, but to encode the decision-making that determines which execution path is worth pursuing in the first place.

The bottleneck in Legal AI: Asking the right questions with Question / Context Databases was originally published in Messy Problems on Medium, where people are continuing the conversation by highlighting and responding to this story.

Legal services are beginning to follow a pattern that fintech went through a decade ago with Embedded Finance. Once underlying legal services become standardized and repeatable, the primary differentiator stops being the service itself and becomes distribution.

This dynamic is well established in embedded finance. Banking, payments, and card issuance are now offered by companies that are not banks, but that already own the customer relationship. Ride-sharing platforms, payroll providers, and accounting software embed financial products at the moment they are needed, while regulated institutions remain responsible for the underlying financial activity.

This shift is already happening. Certain legal services are already product-like: incorporation, routine filings, standardized compliance reviews, and repeatable enforcement actions. Incorporation is the clearest existing example. Through Stripe Atlas, founders can form a company as part of setting up payments. The legal service is not discovered through a law firm search; it is initiated inside a payments workflow at the exact moment it becomes necessary.

Lexington Law is an example of a law firm that has built a credit reports software product as a way to generate leads for the underlying law firm.

Lexington Law firm is the counterparty to the consumer, and offers credit repair and credit monitoring solutions in a way that does not appear like a traditional law firm. This solo law firm apparently made approximately ~$300m in revenue in 2024, before undergoing bankruptcy due to unauthorized billing practices and robo-dialing.

Another example is the Rocket Lawyer and Covéa partnership.

Covéa is a large insurance group that offers consumer insurance products, and it used Rocket Lawyer as a way to embed “legal benefits” inside an insurance relationship, rather than forcing customers to separately shop for legal services. The model is described as Covéa integrating Rocket Lawyer’s digital legal documents, legal information, and related services into Covéa’s product suite so policyholders can access legal tools through a co-branded, integrated experience.

This is the same embedded distribution logic: the insurer owns the customer relationship and embeds legal help at the moment of need, while Rocket Lawyer supplies the legal platform and access to lawyers where appropriate.

Tax law offers another signal. Legal outcomes related to tax are increasingly delivered through accounting platforms and affiliated professional services structures. From the user’s perspective, legal services appear embedded within an accounting or finance workflow rather than accessed as a standalone legal engagement.

As software mediates more economic activity, these platforms are increasingly becoming the first point of contact for initiating legal services. In some cases, software acts purely as a distribution layer. In others, the software itself is the environment where the legal issue arises. In these articulated cases, users begin their legal journey inside software systems they already rely on.

Concrete examples of embedded legal initiation

• Accounting software → letters of demand:
  When an invoice becomes delinquent, a business initiates a lawyer-issued letter of demand directly from its accounting platform. The software provides timing, documentation, and access; licensed lawyers draft and issue the demand.
• HR / payroll software → labor law counsel:
  An HR or payroll platform becomes the place where an employer engages labor counsel following events such as termination, reclassification, or expansion into new jurisdictions. The platform routes context; lawyers conduct audits, provide advice, and manage remediation.
• Vendor or procurement platforms → contract enforcement:
  A vendor-management system initiates legal escalation when contractual milestones are missed or performance thresholds are breached. The platform tracks obligations and evidence; legal interpretation and enforcement remain lawyer-driven.

Distinction from Aggregators

This is different from the general trend of law services aggregators like Avvo or Rocket Lawyer, or their more modern version like Morgan & Morgan, Atticus, LawHive, Off the Record, etc. Aggregators are analogous to the NerdWallet’s of the Fintech world.

While fundamentally an interesting business model innovation, their core thesis is distinct:

• Aggregate legal services / lawyers under a consumer-facing marketplace or backend marketplace
• For a consumer-facing marketplace like Avvo, generate revenue through advertisements (similar to Amazon Ads) and a pay per lead model. Regulations typically do not allow non-lawyers to take on a fee split or % of the lawsuit or make specific recommendations of lawyers.
• For a backend marketplace like Atticus or LawHive, you create a law firm that deals with the customer intake and then makes specific lawyer recommendations and introductions. This bypasses restrictions around fee splits, and attorney recommendations.

While these aggregator models are good business models, they are fundamentally a continuation of the idea that services, and not software, are the first port of call for a customer.

Embedded legal is focused on the idea of a software flow-first approach that leads into legal services, as opposed to a legal service router. It is the difference between NerdWallet versus Shopify Capital. One aggregates customer demand, the other embeds financial services onto an initially unrelated core offering.

With AI, more custom user journeys through software can be designed to help attract customers with a problem, and triage between those who subsequently need to speak to a lawyer, and those who are satisfied with the self-help tools.

The distribution gap in law firms

Most law firms do not think in terms of scalable distribution, even as many of their underlying services become increasingly standardized. Firms remain optimized for bespoke delivery and reputation-driven sourcing. As legal services become more productized, this gap between how services are delivered and how demand is captured becomes increasingly visible.

The impact is likely to be huge, but why now?

If we think about the ratio between the number of fintechs that own the customer relationship and initial customer demand generation over their underlying banks, we can see the potential for embedded legal driven disruption to rise dramatically, especially with the ease of building tools to give initial context for the legal problem (i.e. credit monitoring tools as a segue into credit repair services), and the ability for LLMs to help with providing initial triaging and direction.

Why embedded legal could fail — and where MSO / PLLC & Alternative Business Structures helps

There are a number of business structures that either already allow ways to construct businesses mentioned above (i.e. MSO / PLLC), and/or to additionally allow non-lawyers direct equity in such structures (i.e. Arizona ABS). This is beside the fact that a group of lawyers could have always executed such a structure. There is primarily an education gap in combining demand generation, technology, capital, and business structures together in a unified direction.

• Referral and fee-sharing restrictions constrain the core examples:

In models like accounting platforms initiating demand letters, HR systems routing labor counsel, or procurement software triggering enforcement, law firms are often restricted from paying platforms for leads or sharing fees tied to individual matters. This directly limits how embedded distribution layers can monetize, even when they originate demand.

• Platforms cannot freely price legal initiation:

Unlike other software marketplaces, legal platforms generally cannot charge per letter sent, per audit triggered, or per enforcement action initiated. This makes it difficult to align incentives between software platforms and law firms using conventional SaaS or usage-based pricing.

• ABS structures can realign incentives at the entity level:

Where permitted, MSO/PLLC and Alternative Business Structures may allow software platforms and legal service providers to jointly own or participate in integrated service entities. This makes it possible to generate revenue at the business level rather than on a per-referral basis, reducing reliance on prohibited fee-splitting.

ABS enables subscription-based labor compliance offerings that bundle access to counsel into ongoing HR workflows. Lawyers retain responsibility for advice and remediation, while platforms support continuous monitoring and engagement, allowing recurring revenue without per-event referrals.

Closing

Embedded legal is not about replacing lawyers or turning legal advice into software. It is about recognizing that software platforms are increasingly where legal journeys begin, even when lawyers remain responsible for the legal work itself.

Embedded Legal: How Embedded Fintech principles will disrupt the business of Law was originally published in Messy Problems on Medium, where people are continuing the conversation by highlighting and responding to this story.

AI can now be used to map out how lawyers handle edge cases. But I am uninterested in finding out the most common denominator workflow for handling edge cases. Instead, I want to use AI like a Spartan’s armor. I find the best high-performance lawyers who want to use technology, build custom, individualized software for individual lawyers that reflects the way they specifically think and work. I don’t care (and almost prefer) if they have a more unique way of doing things, and have the performance to show for it.

Selling SaaS to law firms is painful. Law firms are basically a collection of independent contractors, each individual with their own way of doing things. Let’s lean into this! Do not sell technology to 1,000 people and increase their efficiency by 30%. Instead, sell technology to 1 person, increase their efficiency by 10,000%, and capture a clip of their efficiency gain. One exception is any software or process that directly leads to revenue gains, and not cost-savings. You can imagine the best case scenario is custom revenue-generation technology.

Simply put — the way experts look to automate themselves is different from how software companies are traditionally built, especially before AI. To be clear, a law firm is going to need both general and specialized software. Just like generalized software is going to be helpful for a lot of general work, a specialized software tool is specifically going to be helpful for extending work that not many others are doing — which basically describes a lot of a lawyers job.

How does this make sense economically?

• Cost and Flexibility of Development: AI has drastically lowered the barrier of code generation and natural language processing. Software that took millions to build before can be built in hours and for thousands of dollars.
• Lawyer Adoption: This is the hardest bit because you need to completely understand one lawyer’s idiosyncratic workflow in extreme detail, and they need to be both patient enough and willing to share. However, in this model, one does not need to convince 10,000 lawyers that automation makes sense. One just needs to convince 1 high-performing lawyer per business sector / specific valuable workflow that this is the future. This business model can take 9,999 no’s.

Is there a precedent for this?

Yes! This is how some quant funds are run. My cofounder, Tom, who briefly worked at Optiver, shares some of his experience in the industry:

Small Cross-Functional “Pod” Teams: Many quant funds use small, cross-functional teams (often called pods) that include a portfolio manager/trader plus quantitative researchers, data scientists, and software engineers​

For example, Citadel embeds engineers directly into trading teams, allowing technologists to build and iterate on solutions side by side with portfolio managers​. This real-time partnership means a researcher can devise a model, an engineer can code it, and a trader can deploy it quickly, all in sync

Quant funds heavily customize technology for each trading strategy or team. Over 50% of employees at a firm like Two Sigma are engineers, building internal systems and tools to solve “problems unique to financial services”​. Rather than one-size-fits-all software, these firms develop proprietary analytics platforms, data pipelines, and execution algorithms that fit the needs of specific traders or strategies.

Quantitative developers often tailor the software to their portfolio manager’s trading style, providing bespoke analytics, risk models, and automation

Imagine if quant funds were run as a partnership of individual traders that had a barely functioning centralized software procurement function. That’s basically the current state of law firms.

What would be reasons this could work for quant funds but not for lawyers? The main reason is that each trader can make tens to hundreds of millions of dollars, while a lawyer might only be able to bill say up to 5m. This also doesn’t apply to specific areas of non-contingency legal practices like plaintiff litigation where individual lawyers can make 9 figure incomes. Furthermore, this is the old model of a lawyer without leverage. With technology, and the ability to refer cases to other lawyers, it is possible to imagine a 100x lawyer.

A New Playbook for AI-First Law Firms

Most importantly, this approach breaks the false trade-off between quality and scale. In the current system, the best lawyers are expensive, limited in capacity, and reliant on inefficient firm structures. In this new model, they are empowered with AI and automation, allowing them to maintain elite quality while expanding their reach at an unprecedented scale.

This is where the legal industry is headed — whether existing firms recognize it or not. Just as finance was transformed by quant trading and manufacturing was reshaped by automation, law will be redefined by AI-augmented professionals who leverage technology not as a tool, but as a fundamental extension of their expertise.

The first AI-native legal platform will not be built by selling software to firms. It will be built by identifying and equipping the best individual lawyers with an infrastructure that turns them into 100x lawyers.

This is the future of law firms.

LLMs and AI systems are reshaping the way we think about product development. As LLMs introduce a level of reasoning into previously workflow-only systems, we see a rise of “Reasoning Architecture” that lies on top of traditional underlying “Technical Architecture.

What Has Changed?

The Reasoning Architecture

Unlike traditional product design, AI-driven systems must explicitly manage a new conceptual layer: the Reasoning Architecture. This layer captures expert judgment, domain-specific logic, and nuanced decision-making criteria, especially critical in subjective scenarios and edge cases where ambiguity reigns.

Reasoning Architecture is distinctly different from Technical Architecture, which remains responsible for data storage, retrieval systems (like vector databases), infrastructure, APIs, and overall system performance. While engineers maintain the Technical Architecture, domain experts — often non-technical — directly shape the Reasoning Architecture. This is because we now have an element in AI systems that directly looks at and reasons across natural language. These LLM systems are design to mimic how humans and experts in specific workflows think and automate the way the system thinks about unstructured information.

The Reasoning Planning infrastructure is essentially advanced Chain of Thought reasoning that reflects the ability for specific human experts to, in a no-code way, easily load and edit their reasoning about information in different situations.

An example of Reasoning Architecture

Reasoning architecture is essentially the mapping of how the system reasons through specific information and comes to specific types of conclusions, categorizations, or decisions based on the information at hand. It essentially acts almost like a checklist or an SOP of how to think about specific information.

This answers questions and assessments like ‘What should I spot and consider in the information given?’ or ‘I should give this category a score unless there is this other fact.’

This is different from technical questions like ‘Where is the information going to come from?’ or ‘How is this information stored?’

Reasoning Planning is different from Technical Planning as the former is focused on how to think about and categorize information versus taking actions and steps (i.e. to retrieve information, to access a website, to send an email, etc).

An example of Technical Architecture

You can see major differences in the Technical and Reasoning Architecture simply by looking at the type of nodes being represented in these graphs. Technical Architecture focuses a lot on performing actions (Doing things) versus Reasoning Architecture focuses on evaluating information (Reasoning through things).

All types of technical processes include some assumption of a Reasoning Process, but there are certain workflows, especially for highly technical white-collar knowledge-intensive fields (law, healthcare, manufacturing) where there are specific steps which are pure information reasoning, without any new external data or action being taken throughout the entire reasoning process (i.e. no Technical Workflow).

We believe reasoning architectures and reasoning maps will become more popular as people begin to focus on encoding more complex context and custom expert workflows into systems. As AI is increasingly used to map out custom workflows that are highly specialized, we believe platforms specifically designed for mapping expert reasoning processes will become increasingly useful. Most tools out there are more focused on workflows because that is the vast majority of what is a known process to developers (i.e. retrieve customer information, retrieve name and search LinkedIn for their CV, extract title & interests).

The Role of Domain Experts

To effectively manage Reasoning Architecture, new user interfaces and workflows are required. Domain experts need specialized UX tools allowing them to audit, adjust, and refine the logic and decisions made by LLMs. This approach transforms previously implicit judgments by experts (“you know it when you see it”) into clearly documented and operational guidelines, improving transparency and accountability.

There are a few ways to ensure your AI system is reasoning the way you want it to — Fine-tuning a model with reasoning capabilities or a Multi-Agent System with Chained Prompts.

In many scenarios, where you want auditability of systems, frequent changes to the reasoning flow, and most importantly, some level of deterministic reasoning that is not a black box model, without having to use millions of training examples, a Multi-Agent System is what people prefer to turn to.

Understanding Product Development

The emergence of a distinct Reasoning Architecture shifts our understanding of product development. At their core, successful products continue to revolve around clear user outcomes. Engineers still need to prioritize a deep understanding of what Users want and their pain points. Users still prioritize the ease and intuitiveness of the final product rather than understanding the complex processes running behind the scenes. Continuous iteration and user feedback loops remain essential to refining and improving products, ensuring they meet evolving user needs. Product development is obviously still centered on ideas of engineering efficient technical solutions, and delivering intuitive and seamless user experiences.

However, product development teams must explicitly manage and operationalize nuanced human judgment and expert reasoning as core elements of the product itself. How the LLM thinks about information is now as important as to how the system gets information. This requires teams to be conscious of how the LLM is navigating and reasoning through information where domain experts are no longer passive advisors but active collaborators who shape product logic directly.

Further, the explicit separation of reasoning and technical architectures elevates the importance of explainability, transparency, and accountability. Products must now clearly document and justify the logic behind their decisions, changing how teams approach both product design and iteration. Instead of purely technical feedback loops, there are parallel processes:

• Engineers optimizing performance
• Experts refining logical decision-making

Such audit-ability of reasoning steps may also go beyond essential Reasoning Development work, but also for legal explainability purposes.

Why This Distinction Matters for Product Developers & Managers:

Clearer Explainability & Accountability

• Simplifies understanding and communicating how AI-driven decisions are made, essential for compliance and user trust.

Efficient Auditing Workflows:

• Enables faster tracing and validation of the reasoning logic, speeding up troubleshooting and reducing downtime.

New Expert-Centric Workflow:

• Requires adopting specialized interfaces and iterative workflows that actively involve domain experts in continuously defining, refining, and auditing reasoning logic separately from technical iterations.

A point I find interesting is that just like how traditional product managers would have a deep understanding of the workflows of their users, to the point that they can act as consultants on best practice workflows, AI product managers are likely going to have to essentially upskill on the reasoning of their users, to the point that they can act as consultants on the specific reasoning employed by their users. From this perspective, you can see how you are likely going to have more lawyers as legal-tech AI PMs, or accountants as accounting-tech AI PMs, etc.

The Future of Product Teams

This shift necessitates evolving roles within product teams. Domain experts become integral participants, actively shaping product logic alongside engineers and product designers. Ultimately, clearly distinguishing Reasoning Architecture from Technical Architecture ensures AI-driven products remain transparent, accountable, and reliably aligned with human expertise and ethical standards.

• Why lawyers have a unique advantage when working with LLMs
• How LLM information retrieval works — and how to spot failures
• Practical examples (including a “Mock LSAT”) to debug LLM outputs

This article discusses how, despite the fact that lawyers are not engineers, a legally trained mind — when equipped with basic technical knowledge of Large Language Models (LLMs) — can uniquely contribute to their deployment and coordination. Many LLM failures stem from the way AI systems handle language. While impressive, LLMs do not think like humans.

Law is one of the few professions built on structuring reasoning and maintaining consistency through language. Lawyers, with the right technical foundation can collaborate with engineers to create expert reasoning and retrieval workflows — charting processes for how LLMs should reason about specific types of legal information.

AI offers lawyers the potential to create a programmatic version of themselves — one that retains their expertise but never tires or retires. By understanding LLM search processes like Retrieval Augmented Generation (RAG), lawyers can pinpoint why an LLM fails to retrieve relevant answers and use their semantic expertise to refine its output.

Most LLM systems use Retrieval-Augmented Generation (RAG) to pull in external information beyond their training data. This includes searching documents, websites, and other sources before generating an answer.

I think of LLMs like an intern. They can follow instructions literally, lack intuitive reasoning, and struggle to connect implicit concepts. They also have a tendency to forget information between interactions.Besides all of these, like an intern with a particular quirk in the way they think, LLMs rely on a single method — semantic similarity — to interpret questions and retrieve relevant information. Knowing this, this approach can be reverse engineered to identify failures.

For example, when searching a 300-page contract for “consequential damages,” a lawyer knows:

• Searching for “damages” via Ctrl+F won’t work since the term may appear as “indirect losses.”
• Context matters — if “consequential damages” appears in definitions, it expands the scope of relevant terms.
• Certain standard documents (e.g., side letters in partnership agreements) must be checked, even if not explicitly referenced.

Similarly, LLMs struggle when retrieving conceptually related but substantively material terms. If asked, “What liabilities does the contract impose?”, an LLM may overlook relevant sections if “consequential damages” and “indirect losses” are regarded as non-semantically similar. Lawyers, trained to be sensitive to conceptual equivalencies, can help address these gaps.

Let us go in depth into how LLMs retrieve and answer questions in a RAG system so we can start looking at more complex scenarios (and more likely scenarios you may have come across).

The basics of RAG that a lawyer would need to know: Chunking, Semantic Similarity, Chunk Filtering

Retrieval Augmented Generation (RAG) operates in three critical stages:

• Chunking: Take a document, break it into pieces called chunks, and turn each chunk into numbers
• Semantic Similarity: Take a question, turn it into numbers and see which document chunks are semantically similar to each question mathematically. In semantic similarity, words like “dog” and “animal” are mathematically close because they share meaning, whereas unrelated terms like “horseshoe” and “quantum mechanics” are distant.
• Chunk Filtering/ Top-K: Take the top few document chunks (scored by semantic similarity), give them to an LLM, and let the LLM construct the answer from that subset of document chunks, given the question.

That’s it — that’s essentially all a simple RAG system does. If I have learnt anything between building developer infrastructure and going through law school, it is that developers, like lawyers, use highly technical terms that belie relatively simple concepts to help enhance job security.

In the below image, you can see what a chunk looks like, and you can program the size of the chunks to use. The larger the chunks, the more the information. The smaller the chunks, the smaller the information. Only a certain amount of words can fit into the LLMs context window when they construct the answer. Therefore, the larger the chunks, the less chunks you can fit into the LLM.

Variable: Chunk size
Large chunks: Better for extracting generalized meaning and insight
Small chunks: Better for extracting specific facts, figures, or statistics

The only other thing to know is something called the Top-K filter. This is just a fancy way of saying ‘What is the top number of document chunks we will decide to select to give to the LLM to construct its answer’.

Top-K filtering determines how many document chunks an LLM uses to generate an answer. For example, if Top-K = 3, the system selects the three most relevant chunks. A higher Top-K captures more information but risks hallucinations, while a lower Top-K ensures focused answers but may miss key details.

One obvious limitation is imagining that you have a question which requires information that is spread across 10 document chunks (i.e. tell me the top 10 clauses that are related to this issue) and the top-k is set to 5, meaning only the top 5 document chunks are retrieved.

Variable: Top-k
Higher values: More complete responses that incorporate large amounts of information
Lower values: More focused, consistent answers

For example, consider the query: “What is the recommended dosage of acetaminophen for an adult?”

• Document 1 contains the chunk: “Take 500 mg every 4–6 hours.”
• Document 2 contains the chunk: “Do not exceed 3000 mg per day.”

If the system retrieves only one chunk, the answer may be misleading. Even worse, if neither chunk explicitly mentions “acetaminophen,” they might not be retrieved at all.

One common trick is to ask the LLM to add metadata summaries to each chunk — a summary of the document itself (i.e. imagining that the 2 documents are “Dosage instructions for Acetaminophen” and “FDA warnings about Acetaminophen” and a similar summary is appended to each chunk). We can then imagine that the combination of each sentence with its document summary gives us a level of confidence that the chunks would be more semantically similar and be appropriately picked up.

The LSATs of Debugging LLMs

Let’s apply this to real-world examples. Debugging LLM retrieval is like taking the LSAT — spotting how subtle wording affects meaning and determining where retrieval goes wrong.

Example A: Travel Itinerary Query

• Query: “What are the optimal itineraries for experiencing authentic, under-the-radar cultural venues in Paris beyond the standard landmarks?”

Underlying Sentences in the Knowledge Base:

• Sentence A: “The Musée Jacquemart-André offers a rich collection of fine arts in a historic mansion setting.”
• Sentence B: “The Eiffel Tower and Louvre Museum remain the best attractions in Paris for experiencing authentic cultural venues in Paris.”
• Sentence C: “The Canal Saint-Martin area provides a vibrant, off-the-beaten-path experience with indie boutiques and local eateries.”
• Sentence D: “For those seeking indie cultural experiences, venues like the Maison de la Photographie and the Passage des Panoramas offer unique insights into Parisian art and history.”

Intended Answer & Limitations of Semantic Similarity:
The intended relevant sentences are sentences A, C & D. However, despite the query’s focus on non-mainstream venues, the retrieval system might overweight Sentence B because of the mentions of ‘Paris’ and ‘authentic cultural venues’. In semantic similarity, the weight of ‘under-the-radar’ may not overcome the weight of the other variables mentioned, even though it is obvious to a human that that variable is the most important.

Potential Solutioning:
For queries involving travel recommendations, a knowledge graph can be established that tags locations with specific attributes — such as “under the radar,” “popular,” “nature,” or “cultural.” These attributes function as values by which conceptual similarities can be measured for the purposes of RAG retrieval.

Alongside this, an LLM dedicated to classifying the attributes can classify the question to determine the traveler’s desired experience (e.g., searching for nature-oriented venues versus cultural venues). This LLM double-checks the answer against the predefined categories, ensuring that the final output aligns with the traveler’s questions and excludes popular attractions and emphasizes under-the-radar experiences.

Similarly, another LLM can double-check the final answer and the original question to highlight any inconsistencies, and re-rank the chunks used.

Example B: Enterprise Device Management Query

• Query: “What are the best practices for remotely deploying security patches on company-owned smartphones to ensure strict regulatory compliance, minimal service disruption, and robust data integrity?”

Underlying Sentences in the Knowledge Base:

• Sentence A: “Enterprise-grade MDM solutions enable scheduled, phased rollouts of security patches, which helps minimize service disruption while maintaining oversight.”
• Sentence B: “Always make sure that your Device is receiving automatic, over-the-air updates to keep up to date with the latest in security” {This sentence was taken from a pamphlet titled ‘Consumer Device Best Practices’ where the title was many sentences away from Sentence B}
• Sentence C: “For corporate environments, it is essential to implement real-time monitoring and post-deployment audits as part of the remote patch deployment process to ensure data integrity and adherence to regulatory standards.”
• Sentence D: “Manual update procedures can sometimes be favored by organizations concerned with the stability of automated systems, but these often lead to inconsistent patch levels and increased downtime.”

Intended Answer & Limitations of Chunks:
The intended relevant sentences are sentences A, C, & D. However, although the query is clearly enterprise-focused, a RAG system might mistakenly emphasize Sentence B because it looks only at the close semantic similarity to the ‘security’ related terms. This misprioritization could cause the system to blend in consumer-level automatic update methods — even though those are considered best practice for consumers — which do not meet the strict compliance and data integrity requirements of an enterprise scenario. Consequently, the final answer might inappropriately mention automatic updates, thereby diluting the enterprise-specific best practices.

Potential Solution:
Where the underlying document structure can give additional critical context, a document structure knowledge graph can help to tie information about each text chunks with the overall document structure and context.

An example of what a document structure looks like in practice is where we combined a knowledge graph document structure of a legal contract with a multi-agent system that specifically tracks references to other clauses (or even footnotes), and is able to use the document structure to refer to other clauses, the chunks that belong to it, and bring it to the LLM accordingly.

Conclusion

Effective LLM debugging isn’t about coding — it’s about understanding language precision and retrieval logic. Lawyers, with their expertise in structuring arguments and analyzing ambiguity, can play a pivotal role in refining AI reasoning and improving retrieval accuracy.

Arguably, auditable system architectures that help to inject context and reflect specific reasoning steps are essentially in shaping technology adoption in critical industries like law.

With the right foundational knowledge, lawyers don’t need to be engineers to contribute meaningfully to AI development. Their ability to map, critique, and structure information makes them valuable partners in the future of AI-driven legal work.

Thanks to Kyle, Jose, Connor, Kenneth, Tomas, Azfer, Matthew, Amol, Timothy, Marcel for comments and feedback. Comments do not imply endorsement.

This week’s launch of Operator by OpenAI, following the release of a similar tool called Computer Use from Anthropic, marks the beginnings of a shift in how legacy ERP moats are maintained, potentially signaling the end of traditional strategies like restrictive API/data access and selective integrations as a means to create a durable competitive advantage in software.

In OpenAI’s Operator release, the demo showcased mainly consumer use cases (shopping via Instacart, reserving a restaurant via OpenTable, or booking a trip on Booking). One critical reaction has been that a lot of what is possible with the Operator tool in the browser, could be made simpler with using APIs that provide the same experience in a more reliable way.

However, the consumer-facing nature of the demo (Instacart has a highly navigable consumer-facing website, plus it makes its APIs freely available) under-emphasizes the larger enterprise opportunity around pulling data and automating data extraction and transformation from more complex systems, which the Operator tool ultimately will enable. In fact, many observers expect these consumer-facing use cases are useful to gain initial adoption and help the technology mature, for enterprise use-cases where extraction is more complex or APIs do not exist, which can be noted from the progression of early benchmark data along the spectrum of UX complexity.

Operator-like Tools for Interfacing with Legacy System of Records

Legacy ERP systems have served as a system of record and a critical backbone for businesses, integrating various processes and centralizing data. Think Salesforce for CRM, Workday for HR, NetSuite for Accounting. However they often lack modern APIs or standardized integration capabilities, making it difficult for their own customers or emerging solution providers seeking to build on top of or extend the functionalities of these systems. The net effect of these “data walls” created by ERP providers essentially entrenches their position within enterprises.

Many of these systems are still operational due to the high costs of migration and the critical functions they perform. However, Operator-like tools make it possible to unlock data trapped within these systems. Important capabilities include:

• Simulating User Interactions: Operator-like tools simulate human interactions with ERP interfaces, automating tasks such as navigating menus, exporting reports, and retrieving specific data. These systems can be programmed in natural language and operate far faster and more consistently than manual processes, significantly enhancing efficiency.
• Scaling Legacy System Interactions: Automation allows businesses to perform high-volume data extraction from legacy systems, scaling operations that were once constrained by human effort. For instance, Operator-like tools in LLMs can extract thousands of transaction records overnight, a task that would take days for human operators, increasing friction for adopting and proving third party vendors.
• Breaking Exclusivity Barriers: By enabling access to legacy ERP data without requiring proprietary API access, enterprises reliant on traditional ERP systems can increase the value and useability of their own data. This challenges the lock-in and pricing power of their vendors, and lets businesses adopt the latest in AI technology without being locked in by slow and traditional vendors.

Consider an organization using a decades-old ERP to manage payroll that does not provide APIs for easy data export. Instead of relying on costly integrations or manual data exports, Operator-like tools could simulate user actions to export payroll records. These records are then automatically formatted and used for modern LLM systems that require the data for context, bridging the gap between old and new technologies. It does not matter whether your ERP system has built APIs for one component of software to pull data for another program, because Operator-like tools can pull that data as a person would.

Another example of this use-case that is perhaps more relatable is LinkedIn data exporting. This process requires a number of steps including going into your LinkedIn account, pressing a few export buttons, waiting for an email, clicking on a link in the email inbox and downloading a ZIP file to your desktop. With Operator-like tools, all of these steps could be automated or running in the background on behalf of a networking tool a user wants to give their data to, creating new use cases across both business and personal domains.

The implications of these developments are that it may become harder for legacy companies to preserve a data moat if they are purely a system of record, and must deliver value-enhancing services on top of the data they hold.

Investment / Building Opportunities

Eroding System of Record Moats & New Openings for AI-Native Software Solutions

One of the great debates for the evolution of software in the age of AI is whether legacy SaaS players have an advantage vs newer AI-native software companies. Legacy players have two main advantages: distribution (ie: an established user base, and relationships with stakeholders at a company) and data moats, as mentioned before.

The elimination of data moats as a default advantage that incumbent ERP players possess over newer entrants has significant implications:

1. Differentiation at the point of the user: Being the system of record no longer creates an automatic advantage for software. Expect increasingly that end user preferences — based on the ability of software to automate user workflows, or offering greater flexibility of features, will increase in importance
2. Easier trial of new software providers: While still needing to overcome a distribution disadvantage (ie: no pre-existing relationship with a company), with lower switching costs it will be easier for companies to experiment with new software solutions outside of the traditional “stack” they were limited to before
3. Downward pressure on ERP margins: Price increases aren’t automatic by default, and ability to switch providers is real

This has implications for investors evaluating the startup landscape, and nascent AI-native players who are helping companies automate workflows, opening up the design space for solution providers and creating new opportunities to bypass legacy ERP providers.

‘Cloudflare/ Captcha for Operator-like tools for traditional ERPs’

We may see a very messy range of solutions pop up by enterprises to combat Operator-like tools in their ERP systems. The notion of Cloudflare for ERP SaaS may emerge. Just as bots were typically targeted against websites, Operator-like tools may be regarded as similarly undesirable botting actions against computer-based ERP systems, and we may see upcoming companies specialize in developing new techniques against Operator-like tools, to help ERP providers preserve their data moat for as long as possible.

Metadata generation and not data records as the new moats

In a traditional ERP system, data that is easily exportable can include things like customer contact details, sales pipelines, and other information that a human user would typically observe in the UI. Metadata could include things like ‘How much time a customer has spent in the Prospecting stage’. Metadata, and metadata generation, may be the more important data moat that ERP players would need to double down and focus on. Many ERP providers (and there are quite a few in niche industries who do nothing additional to the data they store) have not had good systems for metadata generation and this may change. Such ERP providers who are not providing additional metadata value addition may also represent good opportunities to disrupt.

Custom export automation and computer-driven interaction with legacy ERP systems are reshaping how businesses can access and use their data. By mimicking human workflows to overcome traditional API limitations, companies can unlock the full potential and flexibility of their system of records. As tools like Operator and Computer Use continue to improve, API moats for legacy systems of records may continue to erode.

In this article and open source repository, we wanted to demonstrate intelligent navigation of a legal document’s clauses done through a multi-agent system that leverages a lexical graph (document hierarchy) and chunk linking within a multi-graph multi-agent workflow for a regulatory document. The stack we used here was Reducto.AI, Langgraph, LlamaIndex. You can find access to the open source repository here.

Some of the very specific problems that we have come across in documents, particularly legal documents, was the need to build a document hierarchy of the different clauses within the document. This is as clauses sometimes referred to other clauses to get the full meaning and context.

In order to gain the full context, you then had to recursively navigate and retrieve any mentioned clauses (and even footnotes!), navigate the document hierarchy graph to find the mentioned clause, see if any other clauses were mentioned, and to repeat. Recursive retrieval can be done on a range of other document elements outside legal documents including page numbers, multimodal data like images, hyperlinks to other documents or external data, etc.

Legal documents have always fascinated me when it comes to graph-based traversal and retrieval, given the structured nature of the documents. There is also a very specific structured retrieval step that must also always take place in the context of a legal document in the guise of the definitions page, where every specific important term has a very specific variable or fixed definition that can change depending on the nature of the document.

In this Notebook and example, we want to show how you can easily create a document hierarchy graph of the clauses within a legal document, a graph of the definitions in the legal document, and a multi-graph multi-agent recursive retrieval process that answers questions by intelligently navigating through both the document hierarchy graph and the definition graph to gain the full context for questions posed against it.

In this example, we take a simple regulatory document issued by the Malaysian Central Bank, specifically this document governing compliance rules in a variety of organizations.

The question we want to get an answer to is the question:- How can the Board and the CCO manage control functions?

The ideal flow to answer this question would be:

• Gives me a definition of CCO from definition page
• Retrieval of information from Clause 6.3 & Clause 7.2.
• Detection of “Refer to Para 7.3 and 7.4” in Clause 7.2.
• Traverses to and retrieves Para 7.3 and 7.4
• Detection of the mention of Para 9.1 and the retrieval of Para 9.1

As a quick benchmark, when we run this through ChatGPT using GPT4o (which has embedded agentic reasoning), we get the following answer:

The management of control functions by the Board and the Chief Compliance 
Officer (CCO) involves several key responsibilities as outlined in the 
compliance document:

Responsibilities of the Board:
1. Oversight of Compliance Risk Management: The Board is responsible for 
overseeing the management of compliance risk within the financial institution. This includes approving the compliance policy, establishing the compliance function, and appointing the CCO.
2. Approval and Support for the Compliance Function: The Board must 
ensure that the compliance function and the CCO have the appropriate standing, 
authority, and independence to carry out their duties effectively. The Board 
must also ensure that the compliance function is adequately resourced.
3.Engagement with the CCO: The Board should engage with the CCO regularly to 
discuss issues faced by the compliance function and ensure the CCO has direct 
access to the Board.
4. Annual Evaluation: The Board must evaluate the effectiveness of the 
institution’s management of compliance risk at least annually, considering 
assessments from senior management, internal audit, and the CCO.

Responsibilities of the Chief Compliance Officer (CCO):
1. Coordination and Management of Compliance Risk: The CCO is responsible for 
coordinating the identification, management, and monitoring of compliance risk
across the institution. This includes ensuring compliance monitoring and 
testing are consistent across the organization.
2. Reporting and Advising: The CCO must report regularly to senior management 
on the findings and analysis of compliance risks and must ensure that reports 
are readily available to internal audit and regulatory authorities. The CCO 
also advises the Board and senior management on legal and regulatory 
requirements, keeping them informed of developments and their implications.
3. Independence: The CCO must maintain independence from business lines to 
effectively carry out the role of a control function. This includes ensuring 
that there is no conflict of interest in their responsibilities, reporting 
lines, or remuneration.
4. Resources and Training: The CCO must ensure that the compliance function 
is sufficiently resourced with officers who have the necessary qualifications 
and experience. The CCO is also responsible for ensuring that adequate training
 is provided to officers on relevant legal and regulatory requirements.

Interaction with Other Control Functions:
The compliance function should be organized in a way that allows effective 
management of compliance risk, considering the size, nature, and complexity of
the financial institution’s operations. Where responsibilities are shared 
between a dedicated compliance unit and other control functions,
clear documentation and coordination are required to ensure that the CCO can 
perform their duties effectively without impairing independence or focus.
By fulfilling these responsibilities, the Board and the CCO can ensure that 
the institution's compliance risks are managed effectively, supporting the 
overall safety and soundness of the financial institution.

This answer is interesting but incomplete because it:

• Seems to specifically pick up on the keyword ‘compliance’ and highlights all the compliance obligations that are relevant to other clauses as opposed to specifically the board and the CCO. For example, coordination across jurisdictions (Para 8.2) was raised in the context of ‘compliance’, and remuneration independence (Para 7.8) was raised in the context of ‘compliance’.
• Crucially, it seems to have missed the explicit references to Para 7.3, and 7.4 that explicitly state a key obligation between the Board and the CCO around Board approvals being required when the CCO shares control functions. It has also missed Para 9.1 about the separation between audit and compliance functions.

This is understandable given that recursive retrieval of clauses, pages and footers are not explicitly part of the typical semantic similarity retrieval processes that govern RAG.

Summary of Multi-Graph Multi-Agent Workflow Developed

Creation of the Graphs

Within this Notebook, we first extract the document structure parsed by Reducto’s document ingestion engine. The document structure breaks down every page into different elements, such as a Section Header, a List Item, or a Footer.

The elements are then combined based on the order they appear and their implied hierarchies, for example, a Section Header is a parent of a List Item. We then analyze the links in the document to identify connections between extracted elements that can be modeled in the lexical graph.

We then import the chunks and triples into the Knowledge Graph Studio using our SDK here to create the Lexical Graph.

We also built a graph of the legal definitions for the document. Uniquely in legal documents, there is a definitions section for each document that helps to define the specific way certain terms must be interpreted. This can differ by document, use-case and client. In this case, the document contained definitions on pages 4–5. These texts were extracted and passed into GPT-4o with a prompt to extract the legal terms and their definitions verbatim and return them as structured output. The output was turned into a CSV file and uploaded as a separate graph using the SDK and a predefined schema. This Definition Graph is invoked by the Definition Agent when needed to augment the context with the specific relevant definitions. In this case, the Definition Agent is invoked after the initial clauses are retrieved.

We then imported the nodes from Studio into the notebook, and used LlamaIndex to locally index the information of the nodes, preserving the embeddings generated with Studio. We use a combination of LlamaIndex’s Vector, BM25, and Keyword retrievers. The addition of the BM25 and keyword retrievers are helpful in a legal document use case where the query and retrieval process requires precision with terminology. BM25 helps to identify key terms in highly repetitive text, while the keyword retriever ensures that important terms, despite occurring infrequently, are retrieved as needed.

LangGraph was used to build a multi-agent workflow around the lexical graph with the Studio SDK and GPT-4o.Essentially, when a query is passed, the system first searches for the relevant vector chunks through the Initial Search Agent. In this case, the vector chunks are clauses or sub-clauses. The Definition Agent is subsequently called here to augment the clauses with the relevant definitions. The Router Agent then detects if there are additional linked sections or footers that it needs to refer to, and if so, to retrieve the appropriate sections and considers them. The Recursive Retrieval Agent will recursively do so if the subsequent retrieved clauses (as is the case here), refers to even more clauses.

The first clauses it retrieves are Paragraphs 6.3 and 7.2. The Definitions Graph is referenced to check if there is any additional context via the definitions section that should be included. The additional definitions for “CCO” and “Senior Management” are included.

Within Paragraph 6.3, there is the following sub-paragraph 6.3.f:

• “where the CCO also carries out responsibilities in respect of other control functions3, be satisfied that a sound overall control environment will not be compromised by the combination of responsibilities performed by the CCO.”

Based on the information of the first clause it retrieves, a Router Agent then helps to detect if there are clauses or footers mentioned in the material. In this case, a Footer (Footnote 3) was associated with the first clause. The Router Agent then triggers the Footer Parsing Agent. The Footer Parsing agent identifies the relevant footer and returns the following Footer:

• “Refer to paragraphs 7.3 and 7.4.”

Here, another traversal is required, and the Recursive Retrieval Agent is used to traverse the Lexical Graph and retrieve the chunks/clauses within Paragraphs 7.3 and 7.4.

This new information from Paragraphs 7.3 and 7.4 is combined together. Paragraph 7.4(b) contains a reference to Paragraph 9.1. here:

• “compliance function responsibilities cannot be shared with, nor can the CCO assume responsibilities for, internal audit as such practices would render the independent review process described in paragraph 9.1 ineffective.”

Paragraph 7.4 contains a link to Paragraph 9.1, which the Router Agent detects, and instructs the Recursive Retrieval Agent to retrieve it on the next pass. An Answering Agent tracks all the incoming information to summarize and ultimately formulate the final answer to return to the user.

The final answer constructed reflects our ideal flow and takes in information from the Definitions page, Paragraph 6.3 & 7.2, Footer Note 3, Paragraph 7.3, 7.4, 9.1 and is accurately summarized for the user with all the relevant information taken in with smart traversals.

To ensure we did not just get lucky, we ran the final query 3 times, which showed a successful retrieval of the relevant information.

To manage control functions effectively, the Board and the Chief Compliance 
Officer (CCO) have distinct responsibilities that they must exercise:

Responsibilities of the Board:
- Approve critical decisions regarding the CCO, including appointment, 
remuneration, and termination (Section 6.3(a)).
- Ensure the CCO has sufficient stature to engage effectively with senior 
management (Section 6.3(b)).
- Regularly engage with the CCO to discuss compliance issues and consider 
interactions without senior management present (Section 6.3(c)).
- Provide the CCO with unimpeded access to communicate with the board directly
(Section 6.3(d)).
- Support the CCO with adequate resources to perform duties effectively, 
including competent staff (Section 6.3(e)).
- Satisfy themselves that combined responsibilities, if any, do not compromise
the control environment (Section 6.3(f)).

Responsibilities of the CCO:
- Coordinate the identification and management of institution-wide compliance 
risks (Section 7.2(b)).
- Ensure consistent conduct of compliance monitoring and testing across the 
organization (Section 7.2(b)).
- Maintain independence and sufficient focus on compliance duties, even when 
tasked with additional control functions (Section 7.3).

Shared Responsibilities & Coordination:
- The board must approve any sharing of compliance function responsibilities 
between the compliance unit and other control functions (Section 7.4(a)).
- Function responsibilities, including timely communication of issues, should 
be well-defined and documented (Section 7.2(a)).
- Effective arrangements for coordination among control functions should be in
place to facilitate the CCO’s responsibilities (Section 7.2(d)).
- Compliance responsibilities must not compromise the separation of the 
internal audit function (Section 9.1).

The board should ensure comprehensive oversight, and the CCO should maintain
effective coordination and communication across the organization to manage
control functions efficiently.

In conclusion through this exercise, we have developed a system that demonstrates the following:

• A multi-graph system, with each graph representing a different process and objective within a RAG system.
• Creating RAG-capable automated lexical graphs with Reducto & LlamaIndex
• A multi-agent system that allowed for smart traversals of documents based on the way the document intends for a human to read and traverse its information, returning answers from each section and sub-section in a structured way
• A multi-graph multi-agent system that is governed with LangGraph.

Appendix

Agent code snippets

Definition Agent

• Retrieves the definitions of terms mentioned in the query.

    definitions_response = client.graphs.query_unstructured(
        graph_id=definitions_graph.graph_id,
        query=query_prompt,
    )
    
    response_text = definitions_response.answer
    term_def_pairs = response_text.split('\n')
    definitions_dict = {}
    
    for pair in term_def_pairs:
        if ':' in pair:
            term, definition = pair.split(':', 1)
            definitions_dict[term.strip()] = definition.strip()
    
    return definitions_dict

query_prompt = """Return me definitions for the terms in this query: "How can the Board and the CCO manage control functions?" Ensure the term-definition pairs are separated by newlines, properly capitalised"""

definitions_dict = definitions_search(query_prompt)


def print_prompt_definitions_dict(definitions_dict):
    prompt = "Relevant Definitions:\n"
    for term, definition in definitions_dict.items():
        prompt += f"{term}: {definition}\n"
    return prompt

print(print_prompt_definitions_dict(definitions_dict))

Router Agent

• Decides if the process should stop or continue. Also identifies relevant nodes containing footer information or links to other nodes to be retrieved by the Recursive Retrieval Agent.

def router_agent(state: AgentState) -> AgentState:
    # decide if process should should stop or continue

    starter_prompt_footer = f"""
        You are an intelligent agent overseeing a multi-agent retrieval process of graph nodes from a document. These nodes are to answer the query: 
        ```{state['query']}```
        
        Below this request is a list of nodes that were automatically retrieved. 
        
        You must determine if the list of nodes is enough to answer the query. If there isn't enough information, you must identify any relevant footer information in the nodes.
        
        A node can footer information asking to look in another section/part of the document, which will require a separate natural language search. 
        Example: If the footer says "see paragraph x", a search query e.g. "Return paragraph x to answer the query '{state['query']}'" should be made. 
    
        If there are no further nodes worth analyzing, return an empty response. ONLY RETURN QUERIES FOR FOOTERS THAT ARE RELEVANT TO ANSWERING THE QUERY
        
        Else, if any relevant nodes require a footer search, specify the node_id and the search query.
        Nodes are identified by node_id and must be quoted in backticks.     
    """
    
    starter_prompt_link = f"""
        You are an intelligent agent overseeing a multi-agent retrieval process of graph nodes from a document. These nodes are to answer the query: 
        ```{state['query']}```
        
        Below this request is a list of nodes that were automatically retrieved. 
        
        You must determine if the list of nodes is enough to answer the query. If there isn't enough information, you must identify any linked nodes that could be worth exploring.
        
        If there are no further nodes worth analyzing, return an empty response.
        
        Return a list of node_ids. ONLY RETURN NODE_IDS for NODES THAT ARE RELEVANT TO ANSWERING THE QUERY. Nodes are identified by node_id and must be quoted in backticks.
    """
    
    # collect latest nodes, and all nodes
    last_fetched_nodes_flattened: Dict[str, MultiAgentSearchLocalNode] = {}
    all_nodes_flattened: Dict[str, MultiAgentSearchLocalNode] = {}

Supervisor Agent

• Monitors the context window and prunes nodes if too much context is being used.
• Also keeps track of search failures, if no nodes or relevant information was found for footer searches or linked nodes. If there are too many repeated search failures, it will end the retrieval process prematurely.

def supervisor_agent(state:AgentState) -> AgentState:
    
    # Look for search failures. This might be an instance where multiple searches were made for certain parts of the document, but no relevant information was found.
    # This means that the search has to be ended prematurely to prevent infinite loops.
    printout = ""
    for node in state["previous_nodes"]:
        printout += node.print_node_prompt()
    for node in state["last_fetched_context_nodes"]:
        printout += node.print_node_prompt()
        
    prompt = f"""
You are a supervisor agent overseeing the multi-agent retrieval process of graph nodes from a document. The nodes are to answer the query:
```{state['query']}```


Below is a list of nodes that were automatically retrieved, followed by a list of errors. If there are many similar, repeated errors in the retrieval process , where no further linked or relevant nodes could be retrieved, return END to end the process. Else return CONTINUE. 
Return only a single word, either END or CONTINUE.
"""
    
    completion = openai_client.beta.chat.completions.parse(
        model="gpt-4o-2024-08-06",
        messages=[
            {"role": "system", "content": prompt},
            {"role": "user", "content": printout},
            {"role": "user", "content": state['search_failures']},
        ],
    )

Recursive Agent

• Retrieves information for nodes marked by the Router Agent
• Either fetches new linked nodes, or does a keyword search on the document for the footer (e.g. if the Footer asks to refer to paragraph 7.3 and 7.4 without a link, a search is done instead). An LLM call will prune redundant nodes at this step

def recursive_retrieval(state: AgentState) -> AgentState:

    current_nodes = state["last_fetched_context_nodes"]
    
    for current_node in current_nodes:
        state["previous_nodes"].append(current_node)

    new_current_nodes = []

    # look up the nodes to fetch by id    
    
    for node_id in state["node_links_to_fetch"]:
    # sometimes GPT returns node ids with or without backticks
        if node_id[0] == "`":
            node_id = node_id[1:-1]
        if node_id in local_nodes_map:
            new_current_nodes.append(local_nodes_map[node_id])
        else:
            state["search_failures"].append(f"Failed to fetch node with id: {node_id}")


    for node_id, search_query in state["node_footers_to_fetch"].items():
        # fetch nodes by keyword and bm25 search
        footer_retrieved_nodes = retrieve_with_keywords_bm25(search_query)
        # LLM prunes nodes that are not relevant
        footer_retrieved_nodes, _ = prune_nodes(search_query, footer_retrieved_nodes)

        for node in footer_retrieved_nodes:
            new_current_nodes.append(node)

        # if no nodes fetched, log failure
        if len(footer_retrieved_nodes) == 0:
            state["search_failures"].append(
                f"Failed to fetch nodes for query: {search_query}"
            )

    state["last_fetched_context_nodes"] = new_current_nodes
    state["pass_count"] += 1
    state["node_footers_to_fetch"] = {}
    state["node_links_to_fetch"] = []

    return state

Answering Agent

• Given all the information collected in the retrieval process, and the definitions from the document, it finalizes the answer to the query
• It also tries to link to sources e.g. paragraph numbers wherever possible

def answering_agent(state: AgentState) -> AgentState:
    # answer the query
    prompt = f"""
You are an answering agent. You will be given a list of document nodes that were automatically retrieved by the system. These nodes are to answer the query:
```{state['query']}```

Give references to sections/paragraphs if possible, but do not output full node ids with backticks and the hash. 
"""

Variations of the Answers retrieved through the Multi-Graph Multi-Agent system

Answer #2:

To manage control functions effectively, both the Board and the Chief 
Compliance Officer (CCO) must collaborate and establish a structured 
framework. Key measures include:

Board’s Responsibilities:
- The Board must approve the appointment, remuneration, and termination of the
CCO, ensuring they have the necessary stature to engage effectively with
senior management (S 6.3(a), (b)).
- The Board should engage regularly with the CCO to discuss compliance issues
and ensure the CCO has direct access to the Board (S 6.3(c), (d)).
- The Board must also assure that the CCO is granted adequate resources and
support, including competent officers, to fulfill their duties (S 6.3(e)).

CCO’s Role and Independence:
- The CCO should oversee the institution-wide compliance risk management and 
ensure that responsibilities are clearly documented and communicated in the 
compliance policy (S 7.2(a), (b)).
- If the CCO is involved in other control functions, they must ensure their 
independence and ability to focus on compliance responsibilities remain 
uncompromised (S 7.3).

Coordination with Other Functions:
- Proper coordination between the CCO and other control functions is essential.
This includes robust information flows and mechanisms for seeking compliance
advice (S 7.2(d)).
- The Board must approve the sharing of compliance responsibilities to ensure
there is no dilution of focus or effectiveness (S 7.4(a)).

Separation from Internal Audit:
- Compliance functions should remain distinct from the internal audit to 
maintain the efficacy of independent reviews (S 7.4(b), S 9.1).

Training and Capacity Building:
- The compliance function is responsible for training and providing guidance
to other officers on legal and regulatory requirements to manage compliance 
risk effectively (S 8.11).

By implementing these steps, the Board and the CCO can effectively manage
control functions, maintaining robust compliance and governance frameworks.

Answer #3:

The management of control functions by the Board and the Chief Compliance 
Officer (CCO) involves several key responsibilities and structures:

Role of the Board in Managing CCO Functions:
- The board must approve the appointment, remuneration, and termination of the
CCO (Section 6.3(a)).
- The board should ensure that the CCO has sufficient stature to interact 
effectively with the CEO and senior management (Section 6.3(b)).
- Regular engagement between the board and the CCO is important to discuss 
compliance issues directly (Section 6.3(c)).
- The CCO must have direct, unimpeded access to the board (Section 6.3(d)).
- There must be adequate resources and support for the CCO to perform his
duties effectively (Section 6.3(e)).

Shared Responsibilities and Independence:
- Where compliance functions are shared, the board must approve this 
arrangement, and responsibilities should be clearly defined and documented
in the compliance policy (Section 7.2).
- The CCO should not assume responsibilities for internal audit, as this can
compromise independent review processes (Sections 7.4, 9.1).
- The CCO must ensure that their independence and ability to focus on
compliance are not impaired by additional responsibilities (Section 7.3).

Responsibilities Within the Organization:
- Compliance is the responsibility of all officers within the institution. 
Business lines manage compliance risk through their managerial controls, 
while the compliance function ensures that these controls are adequate 
(Section 1.2).
- The internal audit function provides independent assurance on the quality 
and effectiveness of the institution’s controls, including those concerning
compliance (Section 1.2(c)).

Coordination Across Control Functions:
- Arrangements for coordination among control functions and the CCO must 
promote a consistent approach to managing compliance risk, with adequate 
information flows and avenues for advice (Section 7.2(d)).

By following these guidelines, the Board and the CCO can manage the
compliance control functions effectively, ensuring that compliance risks
are appropriately identified, managed, and mitigated across the organization.