3.6 Bits Per Parameter: Putting a Number on What a Model Actually Memorizes
An ICML 2026 oral gives the memorization literature something it has been missing: a definition of how much training data a model has memorized that you can actually compute.
The opening move is to split “memory” into two things that usually get conflated. One is the information a model has stored about a particular training example. The authors call this unintended memorization. The other is what the model has picked up about the process that generated the data, which is just generalization.
Why the usual test doesn’t hold up
A lot of earlier work took the view that if you can get a model to emit a sentence from its training set, it memorized that sentence. That isn’t rigorous. Push hard enough on the prompt and you can get a model to produce almost any string, so successful extraction proves much less than it appears to.
The failure runs in the other direction too. A model that has learned addition will hand you the correct answer to an equation it has never seen. Nothing was recited there. Reproducing the right output and having stored the example are different claims.
Memorization as compression
The reframing is information-theoretic. If having the model in hand lets you compress a piece of data into fewer bits, then the model must be storing information about that data.
In practice you approximate the compressed length with the model’s log-likelihood, the probability score it assigns to a stretch of data. When −log p(x) is small, close to zero, the model holds information about x. What that measurement does not tell you yet is which kind of information. It could be rote storage of the example, or it could be a rule the model learned that happens to cover it.
Subtracting generalization back out
To isolate memorization you need a baseline that knows all the regularities but has memorized nothing. The ideal baseline is the true data distribution.
Get ComfyAI’s stories in your inbox
Join Medium for free to get updates from this writer.
With that in place, memorization(x) is the length needed to describe x under the true distribution, minus the length needed to describe x under the true distribution plus the target model. Whatever the target model shaves off the description is the part it memorized.
The obvious problem is that nobody has the true distribution. For experiments on real text, the authors substitute a much larger model trained on a far bigger corpus, take its −log p(x) as the generalization baseline, and treat the remainder as unintended memorization.
Random tokens make the measurement clean
To measure capacity without that approximation getting in the way, they train directly on uniformly random tokens.
Random data has no structure to learn, so generalization is zero by construction and no reference model is needed at all. The true distribution here is just uniform randomness, and its description length comes straight from a formula: a fixed number of bits per token. Every bit the model saves against that number can only be capacity it burned on rote storage.
The number that comes out is about 3.6 bits per parameter for GPT-style models. Roughly 3.5 under bf16 and 3.8 under fp32, and it holds steady across models from a few hundred thousand parameters up to tens of millions.
What this explains
Once you can measure capacity, some familiar training curves stop looking mysterious.
When the dataset is small relative to what the model can hold, the model stores examples one at a time, and unintended memorization climbs as you add data. Then memory fills up. Past that point additional data simply cannot be stored, so the model shifts toward learning general rules instead, and unintended memorization starts to fall. That turning point is where grokking begins.
Double descent lines up with the same threshold. Test loss falling, then rising, then falling again turns out to happen exactly at the moment the dataset outgrows the model’s capacity.
What the paper really buys you is a budget. Capacity stops being a vague intuition about model size and becomes a quantity in bits that you can compare against the size of your dataset, which is what makes the phase transitions in between legible.
| Find papers faster on arXivSub with AI summary (CVPR、ICCV、ECCV、ICML、ICLR、NeurIPS、AAAI、MICCAI、CoRL、RSS、ACL、EMNLP、ICRA、IROS、IJCAI)
