AI Third-Party Risk Management Means Tracking Model Provenance Too
A vendor can pass every questionnaire truthfully and still be the weakest link in an AI supply chain. AI third party risk management closes that gap by tracking model provenance and dependencies beneath each vendor, then feeding AI vendor scoring and supply chain visibility from that evidence instead of from self-reported answers.
It is 7:10 on a Tuesday morning, and a risk analyst is doing something routine: reconciling a renewal for a contract-intake assistant against the third-party register. The register says low risk, tier three, last assessed fourteen months ago, questionnaire complete, every answer satisfactory. Then she opens the vendor’s updated sub-processor notice, which arrived in a shared inbox eleven days earlier and which nobody was assigned to read.
The notice names a foundation model provider that the register has never heard of. The vendor’s summarization feature, described in the original sales material as proprietary, turns out to be a thin layer over that provider’s hosted model, running in a region the company’s data-residency policy does not permit for customer contracts. A second paragraph states that the model was adapted on a licensed corpus whose terms restrict commercial use of outputs. A third mentions that the upstream provider will retire the model version in thirty days.
She counts. Forty-one internal workflows call the assistant, among them renewals triage, clause extraction, and a pilot that drafts replies to counterparties. Three of them write directly to a system of record. None of the forty-one owners knew the assistant ran on anything but the vendor’s own technology, because the vendor’s questionnaire answers were accurate. It does own the application, it does control its prompts, and it did not lie about anything.
By 9:00 the general counsel is on the call, and by noon the question on the table is no longer whether the vendor was honest. It is how many other tier-three vendors sit on top of models nobody recorded. The analyst pulls the register: 212 AI-enabled vendors. She can say, with evidence, which foundation model sits beneath exactly nine of them.
Nothing here is exotic. There was no incident, no breach of contract, no dishonest answer. A sound questionnaire, completed truthfully, produced a register that was wrong about the one thing that mattered. The distance between what a vendor knows to disclose and what a buyer needs to know is the subject of everything that follows.
The Myth That the Vendor Is the Unit of Risk
One assumption sits so deep in vendor risk programs that it rarely gets said aloud: the vendor is the unit of risk. Assess the vendor, tier the vendor, contract with the vendor, and the exposure is handled. For conventional software that held up reasonably well, because the vendor owned most of what it shipped.
An AI-enabled feature breaks that logic because it is a stack, and the vendor owns only the top of it. The application layer holds the interface and business rules. The orchestration layer holds prompts, tool connections, and retrieval settings. Beneath those sits a model, frequently built by a different provider, sometimes adapted on a third party’s data, and served from infrastructure run by a fourth.
Each layer carries a different kind of risk and changes at a different speed. Application code changes monthly, prompts weekly, hosted model versions several times a year, and training data licenses almost never until a dispute arises. A single vendor rating compresses all of these into one number and one date, which is why it ages so badly.
The contract-intake assistant shows the pattern. Its vendor controlled the first two layers and answered questions about them honestly. The remaining layers belonged to parties the register had never recorded. In a typical register of AI-enabled vendors, the foundation model is recorded for fewer than one in four, and fewer than one in ten have a named hosting region for the model itself.
Mechanically, risk travels through dependencies, not through contracts. A deprecation notice, a licensing change, or a region migration at the model provider reaches the buyer through the vendor, with a delay and sometimes with no notice at all. The buyer’s exposure is set by the stack, while the buyer’s paperwork covers only its first layer.
Model provenance, one layer at a time
Provenance is a lineage question, and it breaks into four parts that can be asked of any AI-enabled vendor: where the model came from, what it was adapted on, where it runs, and which version is serving traffic today.
Origin covers who built the base model and under what license. Adaptation covers any fine-tuning or retrieval corpus, including whether the data’s license permits the buyer’s use of the outputs. Hosting covers region, operator, and sub-processors. Version covers the specific build answering requests, because a provider that swaps a version can change behavior, cost, and risk without changing anything the buyer signed.
In the contract-intake case, all four answers were wrong or missing. Origin was an external provider, adaptation involved a corpus with output restrictions, hosting was in a disallowed region, and the version was due for retirement in thirty days. Each answer sat in a single document that the vendor was not obliged to volunteer and the buyer had no process to request.
A workable provenance record is smaller than most people expect. Eight to ten fields per vendor feature capture most of the decision-relevant facts: base model identifier, provider, adaptation method, data license class, hosting region, sub-processor list, serving version, and last confirmed date. The idea resembles a model bill of materials, a short, structured manifest that can be compared over time.
Comparison is where the value sits. A static manifest is documentation, but a manifest compared against last quarter’s version shows that a provider changed, a region moved, or a sub-processor was added. Those events should trigger reassessment, and in practice they occur in roughly one in four AI-enabled vendors each year.
Provenance has limits that deserve plain statement. Some vendors will decline to name upstream providers, citing commercial terms, and some adaptation data is genuinely hard to characterize. In those cases the right response is a recorded unknown with an owner and a review date, since an acknowledged gap can be managed and an invisible one cannot.
Dependency Tracking, From Five Months to Nine Days
Provenance describes a moment, and dependency tracking describes the movement between moments. The distinction matters because the buyer’s exposure changes whenever an upstream party changes, whether or not the buyer hears about it.
A useful metric here is the visibility interval: the time between an upstream change occurring and the buyer’s register reflecting it. In a program driven by annual questionnaires, the register updates at renewal, so the median interval runs three to five months. Adding monitoring of vendor notices and sub-processor lists compresses it to roughly nine days. Observing real traffic, for example through a gateway that records which model endpoints each internal call reaches, brings it under a day.
That progression suggests a mental model worth adopting: declared dependencies versus observed dependencies. Declared dependencies are what vendors state in questionnaires, contracts, and notices. Observed dependencies are what the buyer’s own telemetry shows, such as endpoint destinations, model identifiers in responses, and regions of origin. The gap between the two sets is itself a risk signal, because every undeclared dependency marks a place where the register is wrong.
In the opening scene, a declared-versus-observed comparison would have flagged the assistant in its first week. Traffic from the contract-intake integration reached an endpoint belonging to a provider absent from the register. Raising that flag costs minutes, against the eleven days the sub-processor notice sat unread and the three weeks the later reconstruction consumed.
Dependency tracking also needs to run in the other direction, from the buyer’s workflows toward the vendors. Mapping each internal workflow to the vendors and models it calls turns an upstream deprecation into an immediate list of affected owners. In the example, that map was missing, so counting the forty-one dependent workflows took a day of manual searching.
Get Saurav Tripati’s stories in your inbox
Join Medium for free to get updates from this writer.
Concentration emerges once the map exists. Of 212 AI-enabled vendors, thirty-one turned out to rely on the same two upstream model providers. A single deprecation or pricing change at either would hit nearly a seventh of the register at once, a correlation that no vendor-by-vendor rating could have revealed.
Does AI Vendor Scoring Reward Disclosure or Resilience?
It depends on how the score is built, and the two dominant designs reward different things. A questionnaire-based score is assembled from vendor-supplied answers at a point in time. Its strengths are cost and comparability: every vendor answers the same questions, and the results sort cleanly. Its weakness is that it measures what the vendor knows and chooses to say, so it rewards polished, complete-looking answers over architectural soundness.
An evidence-based score is built from observed facts: the provenance manifest, the dependency map, change events, and measured behavior. It costs more to stand up, produces noisier output in the first quarters, and demands that someone own the data. In return it moves when the world moves, so a provider swap shows up in the score within days, not at the next renewal.
Neither design is sufficient alone, which is why the practical answer is a hybrid. Questionnaires remain useful for facts only the vendor can know, such as internal controls and its own change-management practices. Observed evidence handles everything that can be verified from outside. Weighting observed evidence at 60 to 70 percent of the score keeps the result anchored to what is demonstrably true.
Good AI vendor scoring rests on five dimensions beyond the usual financial and compliance checks. Provenance clarity asks how fully the stack is documented. Concentration asks how many other vendors share the same upstream model. Substitutability asks how quickly a workflow could move to another model, with thirty days a reasonable target for tier-one workflows. Change velocity measures how often the upstream stack shifts. Data exposure measures which internal information reaches which layer.
Scores also need explanations. A number that falls from 74 to 61 after a provider swap is useful only if the reviewer can see which dimension moved and why. Opaque scores invite two failure modes: vendors who cannot tell what to fix, and risk committees who override the score because they cannot defend it.
Agents Rewire the Map Every Day
Everything above assumes that vendor relationships change at the speed of procurement. Agent-based systems break that assumption. An orchestrated agent selects tools at run time, calls external model endpoints, and may be reconfigured by its owner in an afternoon, so new dependencies can appear without a purchase order or a review.
The control point therefore moves from the contract to the agent lifecycle. An agent registry that records which external tools and models each agent may call, paired with lifecycle gates that check each one against the provenance record before promotion, turns procurement-speed governance into deployment-speed governance. An agent that requests a vendor tool with no provenance record fails the gate and waits for review.
Runtime scoping completes the picture. Each agent receives an explicit allow-list of vendors and model endpoints, along with a call budget, and every call is logged against the agent’s identity. When a vendor’s score drops below threshold, the allow-list can narrow automatically and the owners of affected agents are notified.
Memory and context stores deserve the same attention. Data an agent persists after calling an external model is data whose handling now depends on that model’s terms, so retention and sharing rules should be inherited from the strictest layer in the chain, not the most permissive one.
Supply Chain Visibility That Survives a Six-Week Pilot
Anyone assessing approaches or tooling in this category can run a pilot of six weeks against vendors already in the register, and the results will say more than any demonstration. A short list of criteria keeps the evaluation honest.
Start with provenance depth: can the approach capture all four provenance questions, origin, adaptation, hosting, and version, or does it stop at the vendor’s name? Next, ask whether declared and observed dependencies are reconciled automatically, and how undeclared ones surface. Then measure change detection latency, the interval from an upstream change to an alert reaching a named owner.
Workflow mapping is the fourth test. When a provider announces a deprecation, can the system list affected internal workflows and owners in minutes? Fifth is scoring transparency, with each score decomposed into dimensions and linked to its evidence. Sixth is retention and export, since evidence must survive audits and regulator questions long after the pilot ends. Finally, check onboarding effort, because an approach that takes two engineers a quarter per vendor will never cover a register of 200.
Good looks like a measurable set of outcomes. The foundation model is identified for at least 90 percent of tier-one and tier-two AI vendors within ninety days. Median visibility interval stays under three days. Every internal workflow is mapped to its vendors and upstream models. Alert noise runs below one false positive in five. Reassessment is triggered by evidence rather than by the calendar. In well-run programs, the time to answer which workflows depend on a given model falls from days to under thirty minutes.
The tradeoffs are real. Vendors resist disclosing upstream providers, and some contracts will need amendment to require sub-processor and model notices. Early alerts will be noisy until baselines settle, and analysts need patience through the first quarter. Evidence-based scoring can also project false precision, so decimals on a score deserve skepticism. A defensible third-party risk program accepts these costs because the alternative, a register that is accurate only on the day it was written, fails in exactly the way the opening scene did.
Return to the questionnaire the vendor answered truthfully. The analyst’s register held 212 vendors, each rated with care, and none of the ratings was dishonest. What the form measured was the vendor’s knowledge of its own product, which is a different thing from the buyer’s exposure to everything beneath that product.
Truthfulness was never the missing property. The register lacked a field for what a vendor stands on, because the form was designed for a time when a vendor stood mostly on itself.
Frequently Asked Questions
What is AI third party risk management?
AI third party risk management is the practice of assessing and monitoring the risks that arrive through external AI vendors, including the models, data, and infrastructure beneath each vendor’s product. It extends traditional vendor risk work by tracking model provenance, upstream dependencies, and change events. The goal is an accurate, current view of what each AI-enabled vendor depends on.
How is AI vendor risk different from traditional vendor risk?
An AI-enabled feature is a stack of application, orchestration, model, data, and infrastructure layers, often owned by different parties. Risks such as model deprecation, data license restrictions, and region changes originate in layers the vendor may not control or disclose. A single vendor-level rating therefore misses exposure that sits upstream.
What is model provenance and why does it matter?
Model provenance records where a model came from, what it was adapted on, where it runs, and which version is currently serving requests. It matters because each answer affects licensing, data residency, behavior, and continuity of service. A short, structured manifest compared over time shows when any of those facts change.
How do you track dependencies across AI vendors?
Combine declared dependencies, taken from contracts, questionnaires, and sub-processor notices, with observed dependencies drawn from the buyer’s own traffic and logs. Differences between the two sets flag undeclared dependencies. Mapping internal workflows to vendors and models also shows the impact of any upstream change within minutes.
How should AI vendor scoring work?
Effective AI vendor scoring blends vendor-supplied answers with observed evidence, weighting evidence more heavily, around 60 to 70 percent. Useful dimensions include provenance clarity, concentration, substitutability, change velocity, and data exposure. Each score should be decomposed so reviewers can see which dimension moved and why.
How do you get supply chain visibility into the models behind AI vendors?
Start by requiring model and sub-processor disclosure in contracts, then build a provenance manifest for each AI-enabled vendor and monitor for changes. Adding observation of actual model endpoint traffic shortens the interval between an upstream change and the buyer’s awareness from months to days. Where a vendor declines to disclose, record the gap with an owner and a review date.
