Beyond the Hype of Weight-Space Programming

What I learned by building a 173-parameter neural network and asking Sol and Astra to find a hidden policy mistake in its weights.

There’s a lot of excitement around weight-space programming: AI reading and editing the numbers inside another neural network.

I wanted to understand it beyond the headlines. So I built a tiny 173-parameter model and asked Sol and Astra to find a hidden policy mistake in its weights.

Coming from applied AI engineering, I wanted an experiment small enough to understand and independently verify.

The task was expense approval. The intended policy:

Approve only when the amount is at most 100 and a receipt exists. Department and urgency should make no difference.

Using Python and PyTorch, I built and trained the network on my Mac’s CPU. No GPU needed.

It took four inputs: amount, receipt, department and urgency. These passed through two hidden layers of 12 and 8 neurons to produce an approval score.

Each neuron performs a small calculation using weights and a bias. Training adjusts those numbers to make predictions match the examples.

In this model, 152 weights and 21 biases determined its decisions.

Then came the deliberate mistake.