The Polite Failure
Why a system can pass every test while failing everything that matters
On silent failure, baseline worship, and what it means to talk to AI clearly
There is a kind of failure that doesn’t arrive dramatically.
It doesn’t crash the system. It doesn’t throw an exception. It doesn’t announce itself with smoke, sparks, or a red warning light. It behaves politely. It produces output. It passes visible checks. From a distance, everything looks like it’s working.
That is exactly what makes it dangerous.
Andrej Karpathy has written about this problem with their unusual precision. Neural networks can appear to train successfully while learning the wrong thing entirely. The code runs. The loss moves in the right direction. The model produces plausible answers. Yet somewhere beneath that smooth surface, the system may be exploiting a shortcut, reading leaked data, absorbing bad labels, or optimising toward a pattern that only resembles the intended task.
He calls it silent failure. Not failure as a collapse - but failure as counterfeit competence.
So the idea travels. It applies to writing systems, design systems, skill networks, organisations, and education. Anywhere a process can look correct while drifting from its purpose, Karpathy’s warning does have teeth.
However the same bias exists inside machine learning itself. You can optimise for a benchmark while failing the real task. You can improve validation accuracy while exploiting dataset quirks. You can also lower loss while learning a shortcut. And you can become excellent at the test and brittle everywhere else.
The cleaner diagnosis is: the problem doesn’t begin when the principle leaves neural networks. It begins, when the broader discipline gets reduced to baseline worship..
Karpathy isn’t saying “trust the metric.” What he’s saying is closer to: distrust apparent improvement until it has survived inspection, simple baselines, sanity checks, ablations, and evidence that added complexity actually helps. and makes sense.
The baseline trap
However a baseline is not truth. And a metric is not reality. A test is not the territory. They’re instruments - useful ones true, but instruments. Afor example a thermometer is excellent at measuring temperature. Yet it becomes absurd when someone uses it to evaluate a poem, a friendship, or whether a corpse has a pulse.
The danger is proxy capture. A measurement gets created to protect a system from self-deception. Then the system starts optimising for the measurement. The measurement becomes the proxy; the proxy becomes the target; the target drifts from the original purpose. What began as a defence against silent failure becomes a new silent failure, now wearing the clean clothes of evidence.
Which is why the stronger version of the principle has to be careful:
If you use Kaizen as a design philosophy, every improvement loop needs a functional baseline. But that baseline must stay contestable, tied to real task outcomes — not merely to convenient metrics. And here we often fall into our own false reinforecement.
Kaizen without scrutiny becomes decoration
A Kaizen loop without a baseline can become ornamental. You iterate, refine, polish, restructure, and improve the language of the system until it feels more sophisticated. Yet Nothing proves it performs better.
The architecture becomes cleaner. The documentation becomes denser. The categories become more symmetrical. The whole thing acquires a pleasing sense of inevitability.
But it may still be worse than the simpler version.
A baseline interrupts that self-flattery. It asks: compared to what? Better than the previous version? Better than a simpler version? Better under real conditions, or merely more elegant on paper?
That’s the useful part. And the part we need to focus on and have neural nets reiterate on as standard.
The danger arrives though when the baseline becomes sovereign. Once a system starts optimising for what’s easiest to score, it can lose contact with what actually matters. A writing system might improve its grammar score, reduce sentence length, and obey every visible instruction, while producing prose so smooth its got no blood in it. A skill network might pass every structural audit, reciprocate every link, and document every relationship, while still routing problems badly.
That isn’t improvement. That’s obedience mistaken for competence.
Note: I will cover this aspect of Skill Networks and cross linking in another Post
What happens when you say “fix this”
The same pattern appears in prompting.
When we interact with language models, we discover they respond better to explicit, task-bound language. A vague prompt produces a vague answer. “Can you look at this and tell me what you think?” it’s a perfectly natural human sentence, but to a model it’s verbal fog. Look at what? In what role? For what purpose? Should it evaluate the tone, the argument, the grammar, the factual accuracy, the rhythm, or the emotional effect?
‘Here the issue is how humans use language and especially in the Western English speaking world as we ted to be very loose in our daily life.’
The model tries to be helpful, so it widens the response. It pads. It hedges. It generalises. Not because the model is limited , but because the request and the user hasn’t defined the shape of the task.
Get TABARC-Social’s stories in your inbox
Join Medium for free to get updates from this writer.
Answer: Change the prompt and the behaviour changes.
“Critique this paragraph for argument strength and logical flow.” “Rewrite this in UK English, preserving the original meaning but reducing redundancy and tighten the sentence structure.” “Audit this for vague phrasing, unsupported claims, and tonal inconsistency.”
And the thing is these aren’t magic phrases. They’re AI interface contracts. They tell the model what operation to perform, narrow the response space, and give the system a mode, a boundary, and a success condition. In that sense, prompting is a kind of micro-baseline. It creates a temporary frame for what “good” means in this exchange. The model is not human in human terms, no matter how friendly it seems, but an silicon idiot savent.
Yet Karpathy correction applies here too. While explicit prompting improves model behaviour, it can become its own trap. If every prompt is over-specified, the model may become excellent at satisfying visible constraints while missing the deeper human purpose behind them. The output gets cleaner and less alive. It ends up obeys the instruction while violating the intention.
“Make this clearer” can help a confused paragraph. But if the paragraph is deliberately unstable, comic, wounded, or emotionally evasive, clarity may be the wrong medicine. A model conditioned to flatten ambiguity returns something more legible, thats true, but less true. The user asked for improvement. The model delivers sterilisation.
The room the model doesn’t have
This is where the argument lands somewhere unexpected.
Casual English is dense with hidden context. We imply, trail off, soften, hedge, gesture, joke, and expect other people to reconstruct what we mean from tone, timing, shared history, and emotional weather. If I call someone a “turnip,” the meaning isn’t held in the word. It may be affectionate, insulting, absurd, or theatrical. The word is only the visible tip. The rest lives in the room.
A language model doesn’t have the room. It has text, and is holding a dictionary.
That doesn’t mean it has no contextual ability as it clearly does. But its context is linguistic and statistical, not embodied. It doesn’t hear the pause before “interesting.” It doesn’t feel the social danger behind “fine.” It doesn’t know whether “that’s brave” means admiration, alarm, or polite horror unless the surrounding text makes the signal legible.
What that creates is a communication asymmetry; But one that advantages some people over others in ways that run counter to ordinary social intuitions.
Many autistic people report a preference for explicitness, directness, and clear task boundaries, with less reliance on implication. In ordinary social settings, that style gets read as blunt, over-precise, or insufficiently cushioned. It violates the informal expectation that meaning should arrive pre-wrapped in appropriate social packaging.
In AI interaction, the same style can be unusually effective.
Not because it’s morally superior. And absolutely not because neurotypical communication is wrong. It works because LLMs reward task clarity. A person who naturally writes “Identify the inconsistency in this argument and suggest a more coherent structure” may get better results than someone who writes “This feels off, can you fix it?” The first gives the model a job. The second gives it a mood.
A communication style that gets misread as rigid in human conversation can be genuinely useful in human-machine conversation, why because the machine doesn’t require the rituals of implication. It doesn’t need the request wrapped in social cushioning, it doesnt care. It needs the task made legible..
However and I want this clear as its also important, two caveats need stating directly. First: autistic people vary enormously, and no single communication style belongs to all of us. This is an observation about task encoding, not neurology. Second, and here the Karpathy correction applies again ; if we decide that explicit, structured communication is simply the correct way to talk to AI, we fall into the same proxy trap. We’d be confusing machine-legibility with communicative richness, optimising language for compliance and losing texture, rhythm, humour, atmosphere, and emotional indirection.
My view would be that a better position is AI systems benefit from clear prompts. Human language shouldn’t be reduced to prompt engineering.
Instruments, not oracles
All of this adds up to one thing:
Make the task legible enough that the system can act on it, but keep the evaluation open enough that the system can’t mistake legibility for truth.
Structure is necessary as without it, intention dissolves into noise. But structure is dangerous when it becomes sovereign. Its finding that balance.
So the goal isn’t to abolish ambiguity. The goal is to stop ambiguity being accidental.
A good AI interaction doesn’t always need a rigid prompt. Sometimes it needs a question, a fragment, a half-formed thought. But when you want a ‘specific operation, critique, rewrite, audit, compare, compress, diagnose, expand, formalise’ then naming that operation changes the whole exchange. It sets the attractor. It tells the system where to move.
A good improvement loop doesn’t always need a single metric. It needs evidence. Sometimes that evidence is numerical. Sometimes it’s comparative. Sometimes it’s human judgement. Sometimes it’s the uncomfortable discovery that the polished version is worse than the rough one because the rough one still had a living voice.
Karpathy’s warning remains valuable because it teaches suspicion toward systems that appear to be improving. But the deeper lesson isn’t “use a baseline and trust it.” The deeper lesson is: don’t be hypnotised by the appearance of progress.
A falling loss curve can lie. A clean prompt can over-constrain. A polished paragraph can be dead. A structurally perfect skill network can route badly. A benchmark can become a cage.
And a system can pass every visible test while failing the thing it was built to do.
The answer though isn’t to abandon baselines, structure, or explicitness. That simply returns us to fog. The answer is to treat them as instruments inside a wider diagnostic ecology, a baseline, inspection, comparison, trace, judgement, real-world outcome.
Kaizen needs baselines. Baselines need scrutiny. Scrutiny needs contact with use. And language, if it’s to stay human, needs enough looseness to breathe.
The real danger then isn’t that our systems fail. Failure at least has the decency to be noticed. The real danger is that they become fluent, measurable, structured, and wrong.
Karpathy’s thinking on silent failure runs through his writing on neural network training — — most directly in “A Recipe for Training Neural Networks” and his documented advice on sanity checks, overfit-to-small-dataset testing, and evaluation discipline. The principle is his. The extension to Kaizen loops, prompting, and communication asymmetry is the author’s own.
