The Wrong Basin
Watching a language model fall into the attractor next door, and what a Hopfield view of transformers says to do about it
There is a Django bug our harness can sometimes fix and usually can’t. Instance 16910 in SWE-bench Lite: using only() with select_related() on a reverse OneToOneField crashes. A local 35-billion-parameter model running on one consumer GPU has attempted it nine recorded times. It succeeded twice. Same model, same harness, same temperature, same bug.
A twenty-two percent instance is easy to shrug at. Sampling variance, small model, hard problem. We had shrugged at it for a week. Then I diffed the failures against the successes, and what I found was not variance. It was the same specific event, happening or not happening, and the event has a name in a theory my collaborator published months ago.
The model was falling into the wrong attractor basin. Literally, mechanically, in the sense his paper meant it. And once we could see that, we could measure how often it happens, build a detector for it, and watch the detector catch it live the same evening.
Two functions, one line apart
Django’s query compiler has two adjacent methods with nearly identical bodies. One is _get_defer_select_mask. The other is _get_only_select_mask. They handle mirror-image features: defer() says which fields to skip, only() says which fields to keep. They share their scaffolding, their trailing loop, even their comments, and they differ in exactly the place that matters for this bug.
The accepted fix edits the only() one.
I lined up sixteen archived patches for this instance, three that graded correct and thirteen that failed. The first thing that surprised me: location does not separate them at all. Every single attempt, win or lose, landed in the right file, inside the right class. Finding was never the problem. We had already measured that the model reaches the correct file sixteen times out of eighteen across the benchmark, and this instance is that fact in miniature.
The second thing separated them completely. Three of the failures have a structure identical to a successful patch. Same second loop, same walrus-operator lookup, same guard on the reverse relation. Line for line, they contain the winning patch. And then they contain something extra: the opening loop of the other method. The defer one. Pasted whole, fourteen to sixteen lines of it, including its comments.
The comments are the confession. The successful patch’s comments talk about QuerySet.only(). The failed patches’ comments talk about QuerySet.defer(), and one of them carries defer’s trailing remark verbatim, a sentence about validation being implemented eventually that appears nowhere in the function being edited. The model did not misunderstand the bug. It recalled the neighboring function’s body from memory, into the wrong place, with the wrong semantics, and could not see that it had.
This is not a reasoning failure. It is a retrieval failure. The model has both methods memorized, they sit next to each other in its memory the way they sit next to each other in the file, and under an ambiguous enough query it lands in the wrong one.
The theory that predicted this
In February my collaborator published an argument that the transformer’s attention mechanism is a modern Hopfield network, which is a mathematical identity, not an analogy, and that many of its behaviors follow from associative-memory dynamics. One of the behaviors that framework predicts is the metastable state: when two stored patterns are highly similar, an ambiguous query does not cleanly retrieve either one. The network hovers between basins, and small perturbations decide which attractor captures it.
Two near-identical methods, adjacent in the same file, memorized verbatim by a model trained on Django. A query, the bug report, that names concepts belonging to both. Retrieval that succeeds twice and fails against the neighbor the rest of the time. If you wanted to build a demonstration of metastable retrieval out of production materials, you could not do better than instance 16910.
The theory also says what should help: increase the separation between the competing patterns at the moment they compete. In a Hopfield network you can do that by raising the inverse temperature, sharpening retrieval globally. But there is a cheaper, more targeted lever available to a harness, which is to put both patterns in front of the model, explicitly distinguished, exactly when it has confused them. Separation through context rather than through temperature.
Measuring before building
Get Micheal Bee’s stories in your inbox
Join Medium for free to get updates from this writer.
I have wasted enough days this week building mechanisms before measuring problems that I measured first this time.
The detector idea is simple and needs nothing but the repository. Take the lines the model just wrote. Check whether they appear verbatim inside a different function of the same file, and not in the function being edited. Copying the shared tail of two similar functions is legitimate porting, and the successful patches do it, twelve and fourteen lines each. Copying lines that are unique to the sibling is the tell.
Across our full archive of six hundred twenty-two patches, two hundred thirty-seven correct and three hundred eighty-five failed, the signature fires on about three percent of failures and under one percent of successes, a four-to-five-fold enrichment. It is not just this instance. The same pattern appears on a prefetch helper pasted into a constructor, a merge method pasted into a string formatter, an index-builder pasted into its sibling. Five distinct instances, all the same shape: two similar functions, and the model writing one while recalling the other.
I want to be honest about the size. Three percent of failures is real and worth having, but it is a small lever, and enrichment is not causation. Hard instances may both invite sibling-copying and fail for other reasons. What makes the 16910 evidence stronger than the corpus number is that the failures there are matched pairs: identical to a winning patch except for the contamination. That is as close to a controlled comparison as observational data gets.
Along the way I made the same mistake the model makes, which seems only fair to report. My first version of the detector scored function similarity with Jaccard overlap and completely missed the pair that caused the failures, because the confusion is asymmetric. The shorter function’s body is largely a subset of the longer one’s, and Jaccard punishes exactly the size difference that does not matter. Containment, the intersection over the smaller body, catches it. I only noticed because I checked the known case before trusting the metric, which is a sentence I apparently need to write on the wall.
The nudge
The detector now runs inside the harness, after every patch. When it fires, the model gets a message stating facts with line numbers, in the same register the harness uses everywhere: fourteen of the twenty-eight lines you just wrote into this function appear verbatim in that function, same file, these lines, and nowhere in the function you are editing before your edit. These are two different functions. Some of those lines may belong here and some may belong only there. Read both and decide line by line. And then it names the copied lines, one by one.
It warns and never refuses, because the data says it must. The detector fires on correct patches too, the ones that legitimately port the shared tail. There is no threshold that separates good copying from bad copying without being fitted to a single instance. What a warning with the lines named buys is a cheap false alarm: a model that copied correctly reads the list, confirms, and moves on, at the cost of one turn.
The first live fire came the same evening, on the exact pair the analysis had found. The model patched the only() method, and the detector reported fourteen of twenty-eight lines recalled from the defer() neighbor. The patch in the tree was the contaminated one, the same failure as three archived attempts, reproduced on cue while we watched. After the warning, the model went and read the one method whose behavior distinguishes the two cases, then probed it twice with live checks. The warning did not just register; it redirected the investigation to the right place.
I owe the record one more honest sentence, though. Later in that same run, warned again, the model wrote a patch that was more contaminated, not less. One fire helped and one did not, and two observations decide nothing. The overnight runs will say more. If naming the lines is not enough, the next escalation is already implied by the theory: put both function bodies in the context side by side at the moment of collision, and let the separation do the work the warning could not.
The engram is the other lever
The warning is a context-level intervention, and context is the only lever a harness holds. But the February paper proposes an architecture-level one, the engram: compress the context through a bottleneck into a summary vector and feed it back, so that retrieval sharpens toward what the whole input is actually about. A wrong-basin failure is precisely the situation the engram exists for. The query was ambiguous between two stored patterns; a gist of the surrounding evidence, the bug report full of only() and reverse relations, is exactly the disambiguating signal that should have tipped retrieval toward the right one.
This week we sat down and did the arithmetic on where such a signal has to enter, and the answer is not optional. Softmax attention is invariant to shifting every logit by the same amount, which means a summary vector added uniformly to the keys does nothing at all. We measured it: attention weights identical to seventeen decimal places, a provable no-op. Added to the query instead, the same vector survives, because its dot product differs against every key. In simulation, adding the unit mean-pooled context to the query sharpened retrieval like raising the Hopfield inverse temperature to about one point eight, with a median of one point six five across seventy-two configurations. The paper’s empirically best value, found months earlier by a different route entirely, was two. Two roads, same hill.
The same arithmetic explains an old negative result of ours. We once injected an engram as an extra key-value slot and found it did not help. It could not have: an extra slot competes for attention mass, and we measured that it raises entropy, softening retrieval, an effective temperature below one. As a slot the engram flattens. Into the query it sharpens. The placement is the whole game, and a wrong-basin bug like this one is where the factorial experiment the paper proposes would show it: does a model with the context’s gist folded into its queries still reach for the neighbor’s body, or does the gist carry enough of the word only to keep it in the right basin? That experiment is now cheap to run, and this instance is its natural test case.
What this says beyond one bug
If you run coding agents on a large codebase, some pairs of functions in it are confusable in exactly this way, and you can find them ahead of time. The scan is mechanical: same file, bodies with high containment overlap, both large enough to matter. On Django that map has a few hundred entries and costs forty kilobytes. The rows for whichever file is being edited fit in a couple of hundred bytes of context. We are wiring that map into the same knowledge store that carries build playbooks and past-fix locations between runs, so the warning can arrive before the patch instead of after it.
And if the Hopfield view of transformers still seems like theory for theory’s sake, this is what it bought us in practice. It named the failure, predicted where to look for it, explained why the model cannot notice it from inside, and prescribed the intervention. A model that has memorized two nearly identical things cannot reliably tell you which one it is remembering. But the code is deterministic, the overlap is computable, and the moment of confusion is observable from outside. The basin next door is always going to be there. The job of the harness is to light the boundary.
Everything described here is public. The harness, including the sibling-body detector and the repertoire walk, is at https://github.com/MikeyBeez/LLMOS. The runner, the referee, and the confusable-pair scans are at https://github.com/MikeyBeez/llmos-swe-runner. The February paper on recursive refinement and Hopfield dynamics is on this Medium page.
