Accountable for Meaning You Do Not Own

What an accountable party needs when an agent acts on shared meaning.

Agents are arriving in business processes. Not in demonstrations of business processes, but in the processes themselves: an agent that reads incoming claims and routes them, one that assembles a purchase order and puts it forward for release, one that decides which of several supplier records a document belongs to. The work is unremarkable, which is the point of it. It is work that used to be done by a person following a procedure, and the procedure was written down somewhere.

Then something goes wrong in a way that costs money, and somebody asks who is accountable. The room produces a list rather than an answer. The vendor who supplies the model. The platform team that operates the runtime. The team that maintains the definitions the agent reads. The human who was nominally in the loop and did not intervene. Each of those holds up for about a minute.

What makes the question hard is a detail that surfaces later in the same meeting. Nobody has changed the agent’s code in a month. Nobody has changed its instructions either. Somebody changed a definition it depends on, last week, in a system that has nothing to do with agents at all.

obviously created by AI

Where the meaning comes from

An agent that is to act inside an organisation needs that organisation’s meaning rather than meaning in general. It has to know what counts as a customer here, which is not what counts as a customer at the company next door. It has to know when a contract is active, which is a matter of this organisation’s rules about signature, effective date, suspension and renewal. It has to know what delivered implies, whether delivered entails invoiceable, and which of those things stands in a relation to which.

None of that is in the model. It is also not missing from the model the way facts are missing, waiting to be supplied by better retrieval or longer memory. Organisational meaning is stipulated. What counts as an active contract here is a decision somebody made, recorded somewhere or recorded nowhere, and it could have been decided otherwise without any fact about the world being different. An agent cannot infer a commitment that was never made explicit, however good it becomes at reading. The information is not hidden from it. The information does not exist until the organisation states it.

So it has to come from somewhere, and there are two places to put it.

The first is inside the agent: in prompts, in tool descriptions, in instructions, in retrieval over whatever documents happen to exist. This is where most organisations are, and it works, in the sense that the agent behaves acceptably. It also has a property that only becomes visible later. The meaning is implicit, unversioned, untestable, and duplicated once per agent. Nobody can say what the third agent takes an active contract to be, because nobody wrote that down as a claim. They wrote it down as a paragraph.

The second is outside the agent, in a maintained model of the domain’s entities and the relations between them, which the agent reads. Calling this the better engineering is a judgement, and it is one this essay makes without proving: the meaning becomes inspectable, maintainable and governable, which are the three properties the first option lacks. That it is the direction of travel is easier to see. It also becomes a single thing that many agents depend on.

That last sentence is the hinge of everything that follows, and it is worth stating calmly. Externalising the meaning does not create the dependency. The dependency was there all along, scattered through prompts, invisible and therefore ungovernable. Externalising it is what makes it visible enough to argue about.

What changes when the reader is an agent

An ontology read by people and by deterministic software is a descriptive artifact. Somebody consults it, somebody builds against it, and a change to it takes effect when a person acts on the change. It can sit in a revised state for a year while nothing downstream moves.

An ontology read by an agent is a different kind of object. A change in meaning propagates into what the agent does, with no change to its code and no change to its configuration. The artifact has become a runtime dependency of behaviour. That is the pivot, and it is what turns the corporate ontology from a modelling topic into a governance one.

The obvious account of what changed is wrong, and worth heading off before it takes hold. It is not that the ontology stopped being static. Corporate models were never static; definitions have always been revised, and revised without much ceremony. Two other properties were doing the work, and both are gone.

The first is that consumption was deterministic. Given the data and the code, the behaviour was a function, and a function can be tested. The second is that there was a checkpoint. Consider an integration mapping, a nightly extract, a report finance runs at close. When a definition changed, somebody had to change the mapping, or re-run the extract, or adjust the report, and could see beforehand what came out of the other end, and could decide not to adopt. That is what a checkpoint looks like in practice, and it is why nobody ever needed a governance concept for it.

Note where that checkpoint sits. It is at adoption, not at every output. Nobody reviews a nightly job’s output nightly. The checkpoint was at the moment the change was taken up, and it held because the effect of taking it up could be established in advance.

Agents remove the determinism and the checkpoint together.

One case shows that the pivot is a property of consumption rather than of the artifact. A coding agent that reads the same ontology in order to build an internal system creates no runtime dependency at all, because what it produces is reviewed and shipped by somebody. The checkpoint is intact and the artifact is identical. What changed in the other case is not the model. It is who reads it, and what happens immediately afterwards.

The two answers that do not hold

Two answers are available to the room in the opening. Both are held by serious people, and both point at something real.

The first is the ownership objection, and it should be stated at its strongest, because at its strongest it sounds like a demand for rigour. I cannot be accountable for behaviour driven by definitions somebody else controls. The team that maintains the ontology can change what active means on a Tuesday, my agent will do something different on Wednesday, and I will have had no part in either. Give me ownership of the definitions my agent depends on, or stop holding me accountable for what it does. This is raised by people trying to take accountability seriously, not to escape it, and it deserves better than being waved off. What the objection is reaching for has a name. M.C. Elish described the moral crumple zone: the position in a human-machine system where a human operator absorbs the blame for behaviour they could not have controlled.¹ Anyone who has been designated the responsible party for a system they cannot steer knows the shape of that position from the inside, and asking for ownership is a reasonable thing to do from inside it.

The second is quieter and harder to argue with. Nobody can establish what caused the outcome anyway. What the agent did was determined jointly by the ontology it read, by its own configuration and instructions, and by the fact that a model of this kind does not reliably do the same thing twice. No one of the three explains the outcome by itself, and no investigation will apportion them cleanly. If causation cannot be established, the reasoning goes, then accountability cannot be established either.

Notice that none of this requires more than one agent to bite. The causal picture is already unreadable at a scale where a single team can see the whole thing.

Both answers conclude wrongly, and showing why takes a definition.

Accountability is an assignment

Accountability is an assignment. A designated party answers for an outcome, however tangled the chain behind it. It is constituted by the designation rather than derived from an investigation. Nothing about this is novel and nothing about agents requires it to be revised; it is how accountability has worked in every institution that has ever had to allocate it.

Two things follow at once. The first is that an accountable party can be accountable for things they did not do, could not have foreseen, and in some cases could not have prevented. “I did not cause it” was never by itself a defence. The second is that the murkiness answer collapses. Attribution is the empirical question of what produced the outcome; accountability is the designated question of who answers for it. Agents degrade attribution, and that much should be conceded fully. They do not thereby degrade accountability, because accountability was never derived from attribution in the first place.

The ownership objection needs more care, because the part practitioners resist is the part carrying the weight: owning an input and answering for an outcome do not compose.

Stated generally, the objection proves far too much. Every accountable role in a large organisation already operates on inputs owned by somebody else. Legal templates. HR policy. The network. ERP customising. Master data. That is the normal condition of accountable work, and not a defect in it. A CFO answers for figures computed under accounting standards she did not write and cannot amend. Nobody regards accountability for a financial close as ambiguous merely because customer master data is maintained by a central team three floors down.

The generalisation: accountability distributes along defined interfaces, not along causal chains.

The step that went missing

Which leaves a question the ownership objection cannot answer. If shared ownership were the problem, master data governance would already be intolerable. It is not intolerable. For the most part it is boring. Ask why, and the answer turns out to have nothing to do with ownership.

Get Bruno Baketarić’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

Given the data and the code, the behaviour is a function. So the consumer can take the new version of the data, run it against their own system, look at what comes out, and then decide to adopt it. Validate, then adopt. That step is what makes accountability legitimate despite foreign ownership. It is not a formality attached to the arrangement; it is the reason the arrangement is tolerable.

Compressed to a sentence: I can be accountable for a dependency I do not own, as long as I can verify its effect on my system before it takes effect.

That is what the argument turns on, and it says something the ownership frame cannot. Ownership was never the load-bearing condition. Verifiability was. Ownership looked load-bearing because in most arrangements the two travelled together, and in the arrangements where they came apart, verification quietly did the work on its own.

Now ask what an agent does to that step. Three things, and only the third is the one usually named.

The first is that the input space has come open. An extract job’s inputs are bounded by a schema; the shapes that arrive can be enumerated, because a shape outside the schema does not arrive. An agent’s inputs are bounded by whatever situations turn up, and the set of situations that turn up is not something anybody holds in advance. A regression suite tests the cases somebody thought of. The cases somebody thought of used to be most of them.

The second is that the correctness criterion has become semantic. Testing a data change means comparing outputs: this figure was one number, is now another, and the difference is explained. Testing a semantic change means judging whether the agent’s new behaviour is right under the new meaning, and that judgement is the same act of interpretation the change was supposed to settle. There is no diff to run. There is a reading to perform, case by case, on cases that cannot be enumerated.

The third is that behaviour is not repeatable. Run the same case twice and the agent may do something else. This is the property everyone names first and it is the weakest of the three, because it is the one that could in principle be engineered away. It is also the one nobody wants engineered away. An agent is put into a business process precisely because it can respond to situations that were not specified in advance, and that flexibility is the same property as the variability. Nobody is asking for an agent that does the identical thing twice. They would have written a rules engine.

Put together, the effect is not the one the room was arguing about. The fight most organisations are having is about who owns the ontology, and that fight is largely beside the point, because ownership can be settled and the problem will still be sitting there afterwards. The question nobody asked is whether a semantic change can be evaluated before it propagates. That is what the legitimacy of the arrangement has always rested on, and it is what has quietly become unanswerable.

What scale adds

Everything above holds with one agent. Several agents change the size of the problem without changing its shape, and it is worth being precise about how, because two mechanisms are at work and they call for different responses.

The first is accumulation. A small deviation early in a chain does not get corrected downstream. It is the input the next stage works from, and the stage after that works from the result, so what began as a marginal difference in what active meant arrives at the end as a case nobody would recognise. Where the chain closes on itself, and an agent’s output eventually returns as its own input, the same mechanism runs without a bound. That version is both harder to see and harder to stop.

The second is correlation. The agents are not making independent mistakes that might average each other out. They read the same definition, so a single shift lands in all of them at once and in the same direction. A shared dependency does not produce noise that cancels. It produces divergence that points one way.

Attribution, already weak with one agent, becomes hopeless here, and this is where the second objection appears: the mesh did it. A system of interacting agents, the claim runs, distributes accountability until nobody holds any.

It does no such thing. Opacity is an argument for granting the accountable party more authority, not for holding them less accountable. Whoever deploys the system and takes its upside is the assigned party, and an unreadable causal chain changes nothing about the assignment. What “the mesh did it” describes is a system nobody was assigned, which is a failure to assign rather than a discovery about agents.

What the accountable party needs

If the assignment stands and the verification step has gone, the practical question is what would make the assignment honest again. The standard answer is reversibility: the accountable party can pull the plug. That is necessary, and it is not nothing. It is also blunt. Accountability equipped with a kill switch and nothing else is real but degenerate, because it permits stop and does not permit steer. The party who can only turn the agent off is choosing between running a system whose behaviour they cannot anticipate and not running it at all.

What is required is wider, and each part of it is unremarkable on its own. Notification, so that a semantic change arrives as an announcement rather than as a surprise in production. Pinning, so that the consumer keeps operating on the version it has validated until it has validated the next one. Contestation, meaning a forum in which to escalate a change that breaks the consumer’s domain, since a change that is right centrally can be wrong locally. And withdrawal, meaning deployment that is bounded and reversible instead of total.

The test is therefore not whether somebody is accountable, but whether that party holds the rights that would let them act. Where they do not, naming a person does not produce accountability; it produces somebody to blame.

Pinning carries the argument. It converts a shared mutable dependency into a controlled one without moving ownership anywhere, which is precisely why the ownership fight is misdirected. What the objecting party wanted was never the definitions. It was the ability to decide when a change to them starts applying to their system, and pinning supplies that while leaving the ontology where it is.

Nor is it an invention. It is how the software supply chain has worked for decades: versions are published, consumers pin, consumers upgrade once they have validated. Nobody consumes a shared library by having the maintainer push new behaviour into every consumer’s production at the moment of the commit, and yet a corporate ontology is typically deployed live, to everyone, the moment somebody edits it.

The comparison is worth pressing, because three analogies circulate here and they are not equally good. Accounting standards fit partly: what counts as revenue is a semantic definition, and a CFO does answer for figures computed under rules she did not write. But that standard is exogenous, public and slow, it has an established interpretive authority in the shape of auditors and case history, and the sign-off happens on an output, once per reporting instance. Master data fits better, with the same central ownership, except that its consumption is deterministic. The supply chain fits best, for the reason that matters here: it is the only one with no per-instance sign-off. A CFO reviews a close. Nobody reviews each agent action.

None of this is without precedent in law. The EU AI Act separates the obligations of providers from those of deployers, and the deployer answers for operation without having built the model, which is assignment rather than causal finding. The instructive part is what those obligations consist of: to use the system as instructed, to monitor its operation, to inform the provider and the market surveillance authority where a risk appears, and to suspend use. The oversight the Act requires must be assigned to people holding not only competence and training but the authority to exercise it.² Even in codified law, what accompanies accountability is wider than a switch.

The tension that does not resolve

Pinning is the lever, and it carries a cost that should not be smoothed over.

What pinning protects is the individual accountable consumer. What it costs is the thing the shared model existed for. If every consumer pins a different version, then an active contract means one thing to the claims agent and another to the billing agent, and the coherence that justified externalising the meaning is exactly what erodes. The problem the ontology was built to solve returns in a slower and better-documented form. This is unresolved, and it is arguably the central design problem of corporate ontology governance, not a detail within it.

Three further questions sit underneath it, all open.

How does one regression-test a semantic change against behaviour that is not repeatable? What would a behavioural test suite for an agent under a version bump consist of, and how would anybody know it had covered enough?

Is semantic change classifiable at all? The pinning argument quietly assumes it is: that a change can be marked as breaking, additive or clarifying, and that versions are discrete objects a consumer can hold and compare. Software has that because interfaces are enumerable. Meaning may not be. If a definition can be tightened in a way that is additive for one consumer and breaking for another, the version has no single classification, and the lever above has no handle. That question sits before the remedy, not after it, and it is not answered here.

And who is the interpretive authority? Accounting has auditors, case history and a regulator. A corporate ontology has a team with a backlog. Absent an authority, contestation has no forum, and a right to escalate is a right to send an email. Nor does the record distinguish the two cases. An absent notification is detectable. An absent objection looks exactly like agreement.

None of which argues against having a corporate ontology. It remains the mechanism for coherence across agents at scale, and it is what makes it possible to exploit the flexibility these systems bring rather than merely tolerate it. The governance question was never whether to have one.

The question is not who owns the ontology. It is on what terms its consumers are permitted to depend on it. An organisation that cannot answer that has not yet assigned accountability for its agents, whatever its org chart says.

¹ M.C. Elish, “Moral Crumple Zones: Cautionary Tales in Human-Robot Interaction”, Engaging Science, Technology, and Society 5 (2019), 40–60, doi:10.17351/ests2019.260. The metaphor is taken from vehicle design: the crumple zone protects the passenger compartment by deforming on impact, and in the moral version the human operator is the part that deforms.

² Regulation (EU) 2024/1689, Article 26. The obligations named here are those of paragraphs 1, 2 and 5. Article 26 applies to high-risk systems as the Act defines them, which is narrower than the range of agent deployments this essay describes; the structural point does not depend on the classification.