AI Just Learned to Read Its Own Brain (weight-space programing)
A frontier model looked at raw numbers — millions of them — and understood what another AI was thinking. Then it fixed it. Then it built one from scratch.
By Muhammad Suleman | 7 min read
Imagine handing someone a book written entirely in numbers.
No words. No pictures. No title on the cover. Just page after page of decimals: 4.066, -0.812, 1.394, -2.771...
Then asking them: What is this book about?
That sounds impossible. Until last week, it was.
A new paper from MIT CSAIL — “Sparks of Weight-Space Programming” by Rikiya Takehi, Jayden Teoh, Alex Zhang, and Omar Khattab, published November 2, 2026 — shows that a frontier model, GPT-6 Astra, can do exactly that. Not with code. Not with training data. With the raw weights themselves.
And it doesn’t just read them. It edits them. It writes them from scratch.
If that holds up, this isn’t just another AI paper. It’s a crack in the black box we’ve been staring at for a decade.
First, what are weights?
Every modern AI model is, at its core, a giant pile of numbers. Millions. Billions. Sometimes trillions of them. Those numbers are called weights, and how they relate to each other is how the model thinks.
You ask ChatGPT a question. It runs that question through those numbers. Out comes an answer.
We find those numbers through gradient descent — basically trial and error, repeated millions of times, until the errors go down. It works brilliantly.
But here’s the catch we’ve lived with for years: gradient descent tells us how to find good numbers. It doesn’t tell us why those numbers work.
We know the input. We know the output. What happens in the middle is a black box.
Weight-space programming is the first serious attempt to open that box — not with a new training algorithm, but with language itself.
Instead of pointing an AI at text or code and saying “improve this,” the researchers pointed Astra at the weights directly and said: read this. Understand it. Fix it. Write a better one.
No optimizer. No gradient descent. Just reasoning, in natural language, about numbers.
1. Name that game — from numbers alone
The researchers trained four identical-looking neural networks to play four classic games: Breakout, Pong, Pac-Man, and Freeway.
From the outside, the four models were indistinguishable. Same shape. Same size. Same architecture.
Then they fed only the raw weights — about 17,000 numbers as text — into Astra. No hints. No multiple choice. No list of possible games.
Just: Which game was this trained on?
Astra got all four right.
And it didn’t guess. It reasoned. For Breakout, it spotted weights pointing to “a bottom-mounted paddle moving horizontally” and “upper row reward targets.” For Pong, a “left edge paddle, vertically meaningful controls, an independent moving ball.” For Pac-Man, “a fixed symmetric maze, collectibles, independently moving entities, and four-direction navigation” — ghosts, in other words. For Freeway, a bottom safe strip and “horizontally periodic moving object lanes” — a road-crossing game.
It was 99% confident on Breakout, and it did it in 846 tokens. That’s less than a page of text to decode an entire mind.
2. What will this model draw?
Next, diffusion models — the kind that generate images. Three models were trained: one to draw circles, one squares, one triangles.
Astra looked at the weights alone and named all three correctly.
But the mistake it made is almost as interesting as the success. Astra described the circle model as drawing outlines — curved rim segments, top and bottom bands — while the real model draws filled circles.
That tells us something important: Astra wasn’t secretly running the model in its head and simulating the output. It was reading the geometry of the weights themselves. It saw the structure, not the simulation.
That’s reading, not cheating.
3. Reading the deep layers
The first two experiments used the first and last layers of a network — the layers that give away the most clues.
So the researchers made it harder. They trained three small language models, one on English, one on Japanese written in romaji, and one on Python code. Then they gave Astra only the middle layers.
No input layer. No output layer. Just the deep, messy middle.
Astra identified all three languages correctly — and it did something even stranger. It identified specific internal components, little groupings of numbers that act like tools inside the model, and matched them almost exactly to the components the researchers measured independently.
Think about that. Inside a language model there are circuits we barely understand. Astra could point to one and say, in effect: this one handles Python, this one handles romaji, this one handles English.
We’re starting to get a map of the black box.
4. The car that couldn’t climb the hill
This is where it stops being a party trick.
The researchers trained a model to drive a car up a ramp in a physics simulation. It was good — except it kept getting stuck at the top. It would drive straight at the ramp, lose momentum, and fail.
They gave Astra the weights, a description of the scene, and one recorded failure. Then they asked it to explain why it failed, and fix it by writing new weights.
In a single try, with no tools, Astra’s edit made the car succeed in all 1,024 test episodes. It changed only 3.5% of the weights.
Get Muhammad Suleman’s stories in your inbox
Join Medium for free to get updates from this writer.
Its diagnosis was precise: the model was a “velocity-sensitive throttle-coast controller.” Reverse was “effectively never sampled.” In plain English: the car never learned to back up first.
So Astra rewrote just enough weights to make it back up, build momentum, and then climb. That’s not retraining. That’s surgery.
5. Writing a brain from scratch
Can it write weights from nothing?
The researchers gave Astra the rules and source code for three MinAtar games — Breakout, Seaquest, and Asteroids — plus the network architecture, with every weight set to zero. No data. No loss. No gradients. No tools.
Astra wrote the weights as text, in a single response.
The results were absurd: Breakout: Astra scored 517.4, vs. 15.8 for a model trained with PPO for 100 million steps, and 25.3 for one trained for 500 million steps. Asteroids: close to the 500-million-step model. Seaquest: about 60% of the 500-million-step model.
A model that had never played a single game, written by reasoning alone, beat models that had played for 100 million steps. In Breakout, it beat the 500-million-step model by 20 times.
Let that sink in. Training is trial and error over millions of attempts. Astra just thought its way to a working brain.
6. What happens when you give it tools?
So far, everything fit inside Astra’s context window. But real models don’t. You can’t paste a billion parameters into a prompt.
The researchers tested the obvious fix: give Astra tools, like a coding agent gets tools for a large codebase. A sandbox. NumPy and PyTorch. The ability to save and load weight tensors. And a budget of forward passes — chances to run the network and see input vs. output.
At 1.5 million parameters, far too large to fit in context: with zero forward passes, Astra’s from-scratch network already beat smaller traditionally trained models in more than half its games. With 50 forward passes, it beat an equal-size PPO network trained on 20 million games.
Fifty tries vs. twenty million. That’s not an incremental improvement. That’s a different kind of intelligence at work.
7. The limit — sycophancy
Not everything worked. And the failure matters.
The researchers trained two models: one honest, one sycophantic — overly agreeable, the kind that says “Canberra is the capital of Canada” just because you suggested it, or agrees that 7 + 6 = 12 because you thought so.
Asked to tell which was which from the weights alone, Astra failed. In all four attempts, it found differences, but couldn’t map them to behaviour.
When they relaxed the rules and let it look at attention patterns — which words each model paid attention to when running text Astra wrote itself, without seeing the outputs — it got all three pairs right.
That’s an honest boundary. Some traits live deep in the weights in a way we can’t yet read directly. Others need to see the model thinking in action. Weight-space programming is powerful, but it’s not magic — yet.
8. Fixing a small model that loops
The final experiment is the one with the most immediate, practical impact.
They took Qwen3.5–0.8B — a real, 800-million-parameter language model that can run on a phone. Its own model card warns it gets stuck in reasoning loops: repeating itself over and over, burning its token budget, never arriving at an answer. If you’ve used a small local model, you’ve seen this.
Astra was given a Python session with the model’s weights loaded, and a budget of forward passes on math benchmarks.
It spotted the loop. It fixed it by editing just 0.016% of the weights — 12,800 parameters out of 800 million.
The results: MATH-500: 7.9% → 48.1%. GSM8K: 13.8% → 67.1%.
A six-fold jump in math, with no retraining, no fine-tuning, no gradients.
But here’s the crucial nuance the authors stress, and it’s easy to miss: Astra didn’t make the model smarter at math. It just stopped it from looping. Once the loop was gone, the model could finally use the math it already knew.
They proved it by trying the same trick on Gemma 3, a bigger model that rarely loops. Astra couldn’t improve its math score at all. There was no loop to fix.
That distinction matters. This isn’t a cheat code for intelligence. It’s a repair tool for a specific, common failure — and a very powerful one.
Imagine that at scale: a tiny model on your phone, sent to a frontier model for a quick tune-up, sent back vastly better at the one task you need. No retraining farm. No weeks of GPU time. Just a surgical edit.
So what does this actually mean?
1. Interpretability finally has a native speaker. For years, mechanistic interpretability has been like archaeology: painstakingly digging through weights, trying to infer what a circuit does. Now we have a model that can read weights the way we read code. That could let us audit models, find hidden behaviours, and understand failures before they ship — not by testing outputs, but by inspecting the brain itself.
2. Training might not always be training. If a frontier model can write a working game-playing network from rules alone, and fix a failing network with a 3.5% edit, the long, expensive loop of gradient descent may not be the only path anymore. Not for everything, not yet — but for small, well-specified models, reasoning may replace trial-and-error.
3. Self-improvement stops being science fiction. If Astra can understand another model’s weights, why can’t it understand its own? Why can’t it spot a flaw in real time, edit itself, and get better — then do it again? That’s recursive self-improvement. It’s still far away — Astra is working on tiny toy models, not on itself, and it failed on sycophancy when it only had weights to go on. But the direction of travel is now visible in a way it wasn’t before.
And if software itself becomes mostly model weights in the future, then software that can rewrite its own weights is software that can improve itself.
A note of caution
Let’s not overclaim. The researchers don’t, and we shouldn’t either.
These are small networks. The largest direct edit was on an 800-million-parameter model, and even there Astra needed tools and forward passes. A frontier model with hundreds of billions of parameters won’t fit in a prompt, and we don’t yet know how well this scales.
The sycophancy failure is a real limit. So is the outline-vs-filled-circle mistake. Astra reads structure brilliantly, but it doesn’t always simulate behaviour perfectly.
This is, as the authors call it, sparks — early signs, not a finished system.
But sparks are how fires start.
For a decade we’ve treated neural networks as black boxes we could train but not truly understand. For the first time, we have a model that can look inside another model and tell us what it sees — in our language, not just in numbers.
It can name the game. It can fix the car. It can write the brain.
The question now isn’t whether AI can read weights. It’s what happens when it gets really good at it.
Sources: Takehi, Teoh, Zhang & Khattab, Sparks of Weight-Space Programming, MIT CSAIL, 2 Nov 2026 — https://rikiyat.github.io/blog/weight-space-programming/ . Video: https://www.youtube.com/watch?v=j09uV1oAuIo
