Why Cadence artificial brains must sleep and dream

A patch net asleep. The world’s port is closed, each patch keeps its context and its record cells, and the dreams rising from the store dissolve into the lattice of the slow weights.

Cadence is an open-source neural network architecture developed by Pragma Research (preprint). Its brains are built from bounded patches with local state, ports, a record store and one local repair rule. This article is about the part of the design that sounds least like engineering: the brain has to sleep, and while it sleeps it has to dream, and both of these are theorems about tables rather than metaphors.

This is a guide to one mechanism, told through things everyone has done: hearing a thing once and knowing it, sleeping on a problem, and waking up with the answer.

TL;DR: A Cadence brain learns in two phases because it has two memories with opposite virtues. By day, what it observes is written once into a record store that holds it exactly; its slow weights do not move. By night, the slow weights learn from the store’s own completions of the day’s cues, with the world closed. Those completions are its dreams. On a sentence grammar, the store alone got 81.5% of never-taught combinations right; after one night of 8,000 dreams and no input, the slow weights alone got 100%, while a control that stayed awake with the teacher reached 90.5%. Wagner and colleagues measured the human version in 2004: 13 of 22 people found a puzzle’s hidden rule after a night’s sleep, against 5 of 22 awake[^wagner]. Sleep adds nothing. It moves what was held into the part of the brain that generalises.

Sleeping on it

You read a code base for three days, function by function, and each function makes sense on its own. On the fourth morning, before coffee, the vulnerability is there, whole: input from this endpoint reaches that query through a helper you read on day one and had filed under harmless. Nothing was read overnight. The pieces were all in your head at bedtime. In the morning they were in a different arrangement, and the arrangement was the finding.

Learning a language goes the same way. You hear “das Buch, das ich gelesen habe” once and can repeat it. You cannot build “der Film, den ich gesehen habe” from it, and a week later you can, and you cannot say on which day you learned the rule, because you did not learn it awake.

In 2004 Ullrich Wagner, Steffen Gais, Hilde Haider, Rolf Verleger and Jan Born put this under measurement in Lübeck[^wagner]. Subjects practised a number-reduction task that had a hidden shortcut nobody told them about. Eight hours later they were tested again. Of the group that had slept in between, 13 of 22 had found the shortcut. Of the two groups that had stayed awake, one through the night and one through the day, 5 of 22 in each. Practice was equal and the information was equal. The only thing the sleepers had done that the others had not was sleep. The paper is called Sleep inspires insight, which is about as far as Nature lets a title go.

Neuroscience has an account of why, assembled over fifty years. David Marr proposed in 1971 that the hippocampus is a fast store for single episodes and the cortex a slow learner of regularities[^marr]. Matthew Wilson and Bruce McNaughton showed in 1994 that hippocampal cells which had fired together while a rat explored a track fired together again while it slept afterwards, in the same order, faster[^wilson]. James McClelland, McNaughton and Randall O’Reilly explained in 1995 why a brain needs two systems at all: a slow learner that took every fact directly would overwrite the old ones, so facts are held fast and taught to the slow system by replay[^cls]. Susanne Diekelmann and Born reviewed the evidence in 2010[^born].

The computational side is older. Francis Crick and Graeme Mitchison proposed in 1983 that dream sleep is a phase in which the brain runs without input and cleans up its own patterns[^crick]. Geoffrey Hinton, Peter Dayan, Brendan Frey and Radford Neal turned the idea into an algorithm in 1995, the wake-sleep algorithm, in which a network learns in one phase from the world and in the other from samples it generates itself[^hinton].

Cadence brains do what these accounts describe, and they do it with two operations that every patch has anyway. The rest of this article says exactly what those operations are, why the night cannot be skipped, and what a dream is when written down.

Cadence in one page

A Cadence brain is a set of patches. A patch has a bounded local state, ports through which it hears and speaks, a record store, and a learning rule that acts only on what the patch itself can see. Everything a patch does is settling: it finds the state that agrees with what its ports hold and with what its weights and records say. Understanding, in this architecture, is a settled state. It is what the patch is in, and nothing it emits.

The patch in this article is the record patch. Picture a row of channels, each remembering its own past at a rate it has learned, some forgetting within a word and some holding for a sentence. A word arrives, each channel mixes a little of it into what it holds, and the row of channels is the context:

l_t = sigmoid(g + G u_t)          how much of the past each channel keeps
z_t = tanh(B u_t + b)             what the present input contributes
h_t = l_t * h_{t-1} + (1 - l_t) * z_t

What to carry forward from this is that the context is a weighted memory of everything heard, and that the weights B and G decide what is remembered and for how long.

The output of the patch is the sum of two things:

y_t = C h_t + c + m_t

The first part, C h_t + c, is the slow readout. The matrices B, G, C and their biases are the slow weights, and they are the part of the patch that generalises. The second part, m_t, is the read of the record store, and the store is where the day's events go.

The store works like a library with a peculiar cataloguing system. The reading at a moment (the input and the context side by side) is projected by a fixed random matrix onto a great many cells, and only the handful of cells that respond most are kept. That handful, at unit length, is the key k. The store is a table M with one row per cell, and the read is the average of what the active cells hold:

m = k M

When the outcome y is witnessed, the store writes the part of it that the slow readout got wrong, by the delta rule:

M <- M + eta * k^T (r - k M),    r = y - (C h + c)

Two theorems about this rule carry the rest of the article, and both are proved in the Cadence Lean library[^cadence]. First, one write at rate one makes the read at that key exactly r: a single observation is held exactly, after one exposure. Second, a write at key k changes the read at any other key q by eta times the overlap q . k times the correction. Readings that share no cells leave each other alone. Readings that share many cells learn from each other whether they meant to or not. Keep that second theorem in view; it is the whole mechanism of dreaming.

The slow weights learn by a different rule, and it has a shape worth remembering. The patch is settled twice: once free, with only the input, and once with its output nudged toward the observed outcome. The difference between the two settled states, divided by twice the nudge, moves the weights. For this patch that difference equals the gradient of the loss (the two agree to eight decimals), and it is the same idea as equilibrium propagation[^scellier]. No error signal travels backwards through anything. Each weight moves on the difference between two states it can see: one clamped to the world, one free.

How a Cadence brain acquires memory

By day, a Cadence patch writes. Every observed outcome goes into the record store in one write, and the slow weights stay where they are. The consequences are ones you can feel from the inside.

A fact told once is usable at once. In the language brain we built, a reader can type “I just read The Glass Harvest by Mira Chen, it is about a gardener who grows memories in glass jars”, and three turns later the brain says the sentence back, word for word, because it went into the store in one write. Twenty-four such sentences, none of which the slow weights had ever seen, were each recited exactly after a single write. With the store switched off, none of them were.

The store also generalises, by overlap, and further than a table of exact records has any right to. We gave a record patch with random, untrained slow weights one pass of writes over 10,672 pairs of a message and its sentence. The messages come from a grammar of 1,276 of them, with 6,173 sentences; a message says something like “tell the kind of this book: dystopian, classic, novel”, and its sentence is “it is a classic dystopian novel by”. A quarter of the combinations of subgenre, era and form were held out entirely. With no gradient step anywhere, the store then produced a sentence inside the grammar for 81.5% of the combinations it had never met, and preferred the grammatical member in every minimal pair we tried: a against an, is against are, deal against deals, both against all. The writes took about a minute.

Where a dream comes from. A reading is coded by its most active cells. A reading the store never met shares cells with taught readings that agree with it in part, and its read is the average of what those cells hold. That average is a sentence the grammar accepts, most of the time.

How does a table of exact single observations answer correctly for a case it never saw? By the second theorem. The never-taught reading “dystopian, early, novel” shares its act cells and its subgenre cells with both taught neighbours, shares “novel” with one and “early” with the other, and its read assembles “an early dystopian novel” from the two of them. The grammar’s regularities live in which readings share which cells. Nobody wrote a rule. The average of the neighbours is the rule.

The day has two limits, and they are the reason a night is necessary.

The first: the store generalises by overlap and no further. The 81.5% is the neighbours’ agreement. Where the neighbours disagree, or where a combination has no near neighbour, the store is silent or wrong, and more writing of the same pairs does little about it. Two passes gave 85%. Re-reading the corpus twice more with the teacher present gave 90.5%.

The second: the store is finite and shared. When the 24 conversational sentences went into a store that also held a library of 2,765 sentences, the library’s recall fell from 100% to 97.3%. Every write disturbs its neighbours by their overlap, exactly as the theorem says, and a store that is written all day fills up and starts interfering with itself.

The slow weights have the opposite virtues. They are smooth: a regularity learned from some combinations extends to all of them. They are large: in the trained language brain, all 2,765 read sentences were recited by the slow weights alone. And they need many presentations of a regularity before they hold it. By day that is exactly what they cannot have. The corpus is heard once and gone. The reader’s sentence was said once. If the slow weights tried to learn each observation as it arrived, they would need many small steps per item, and each step would disturb what they hold, which is the overwriting that McClelland and colleagues described[^cls].

So at bedtime the brain is in a specific condition. Everything it learned today is in the store, exactly and locally. The slow weights know none of it. And the only copy of the day is the store.

Why dreaming is necessary, and what a dream is

The night’s problem is to move the day’s knowledge from the store into the slow weights with the world closed. The slow weights need presentations. The presentations have to come from somewhere. The only place the day survives is the store. So the presentations are the store’s own completions of the day’s cues, and that is what a dream is.

Get Bernhard Mueller’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

Written down: for each cue x_i of the day, the patch settles from rest with its store and reads out

d_i = argmax_group ( C h(x_i) + c + m(x_i) )

the category the patch would choose awake, for every slot of the output. The pair of cue and dream, (x_i, d_i), is then taught to the slow weights by the ordinary two-phase contrast, with writing switched off. Over a night, thousands of such pairs are taught for several passes, and nothing outside the patch is consulted. What to carry forward: a dream is the patch's own answer to a question it was asked during the day, taught to itself as if the world had said it.

The two phases. By day the cue is heard, the outcome is witnessed, and the store takes it in one write while the slow weights stay frozen. By night the day’s cues are replayed, the store completes each one, and the slow weights learn the completions as if they had been observed. At dawn the store is rewritten against the moved weights.

Two things happen in this procedure that make it more than a copy.

First, the dream of a cue the store has never met is the neighbours’ agreement, the 81.5%. Those dreams are taught too. The slow weights, being smooth, learn the agreement as a rule and extend it to the combinations where the store was silent. The night is where overlap turns into generalisation.

Second, the dream is the answer of the whole patch, slow readout and store together. On the first night the slow readout knows nothing and the dream is the store’s. On later nights the slow readout knows the regularities, the store supplies what it has heard since, and the dream is their sum. The two memories are consulted together and the result is written into the one that lasts.

There is a detail the implementation has to respect, and it is worth stating because it is where the obvious version fails. A record holds the residual against the slow readout as it stood when the record was written. If the weights move and the dreams are recomputed from the moved weights plus the stale residuals, the dream drifts and the weights follow their own drift. In a toy with 24 outcomes, that version was right about 6 of them after sixty such steps, which is a brain talking itself into nonsense with great conviction.

So the night fixes its dreams at bedtime, learns them, and at dawn writes each dream back into the store against the moved weights, so that the store holds only what the weights did not take. In the same toy, this version was right about 24 of 24, with the weights alone and with the store.

What a night of sleep did

We ran Wagner’s design on the language brain. The day was one pass of record writes over the 10,672 taught pairs, with the slow weights untouched. Five arms started from the same bedtime state.

ArmWhat happens after bedtimeNever-taught kinds validOther held-out messages validno sleepnothing81.5% (store alone; weights alone 0%)88.5%more practice, awaketwo more passes over the corpus, teacher present90.5%92.0%sleep, replay8,000 dreams from the day’s cues, 1,200 slow updates, corpus closed100% (weights alone and with the store)89.0%sleep, recombination8,000 dreams from cues that recombine the day’s features, many never taught83.0%87.5%teacherthe corpus itself, 1,200 slow updates100%98.0%

Of the 8,000 dreams, 90.95% were sentences inside the grammar. The slow weights learned them in 1,200 updates. Afterwards, with the store switched off, the weights alone produced a valid sentence for every one of the 200 never-taught combinations tested. Nothing entered during the night. The control that stayed awake, kept the teacher and re-read the corpus twice more got to 90.5%. The night cost 81 minutes on a laptop, most of it dreaming, which is also roughly the proportion in people.

Sleep measured twice. Left, Wagner and colleagues: the share of people who found a puzzle’s hidden rule after eight hours, awake or asleep. Right, the Cadence record patch: the share of never-taught sentence kinds said correctly at bedtime, after two more awake passes with the teacher, and after one night of dreams with the corpus closed.

The second column is the instructive one. On the other held-out set, sentences whose messages the store had mostly met but whose completions it got wrong at bedtime, the night changed nothing: 88.5% became 89.0%, where the teacher with the corpus reached 98%. The dreams were the store’s completions, and the store’s completions of those messages were wrong, so the slow weights learned the errors as faithfully as the regularities.

The recombination arm says something more. Its cues mixed the day’s features at random, so many of them were combinations never taught, and the dreams for those were the store’s guesses at 81.5% accuracy. Taught as facts, the guesses held the weights at 83%. Dreaming what you met lets the weights extend the pattern; dreaming what you never met teaches your guesses as facts, and the two are different nights.

That is a precise statement of what sleep does and does not do. Sleep redistributes what is held. It does not correct it. Go to bed with a wrong belief about a code base and you wake up with the same wrong belief, more firmly held and better generalised. Correction needs the world again: a re-reading of the disputed passage, a critic, another day. In the human literature the same limit appears as sleep consolidating false memories along with true ones. The experiment that follows this one adds a dawn re-reading of only the cues whose dreams were unsure, a few percent of the corpus, to see whether that closes the gap.

The epiphany as a change of valley

The results above are about language, where the dreams are sentences. The code audit is about reasoning, and the same mechanism describes it, at the scale of the whole state rather than one cue.

Awake, a patch never settles freely. Something is always clamped: the file you are reading, the sentence you are hearing, the question you were asked. The rest of the state settles around the clamp and finds the nearest consistent configuration, the nearest valley in the energy. Read another file and another part is clamped, and the state settles into another valley. Each valley agrees with what was pinned at the time. None of them agrees with everything at once, because everything was never unpinned at once.

Sleeping on it. Awake, a focus clamps part of the state and each settle finds the nearest valley. Asleep, nothing is clamped and the whole store drives the state, which can leave its valley for one that agrees with everything held. The curve is illustrative.

Asleep, nothing is clamped and the whole store is the drive. The state can move to the configuration that agrees with everything held, which the focused settles could not reach because each of them held the rest fixed. A vulnerability that spans three functions is such a configuration: each function is consistent alone, and the tainted path exists only when all three are settled together. The morning epiphany is the report of that settle. No information arrived; a lower valley was found.

This is the two-phase learning rule again, at another scale. Per batch, the rule compares a clamped settle with a free one. Per day, waking is the clamped phase and sleep is the free one, and consolidation is the difference between what the day held under its clamps and what the night finds without them. The brain that Crick, Mitchison and Hinton described is the same rule run over hours instead of milliseconds, which is a satisfying thing to find in one’s own equations, and also slightly alarming.

The reasoning experiment that tests this directly is designed and is work in progress: pieces that are each consistent alone and inconsistent together, shown one at a time under a clamp, then a free settle over the store at night, scoring whether the finding appears only after the night and in no control that stays awake with equal compute.

Why this matters for language and for deep reasoning

Two consequences follow for anyone building learning systems, and they are the reasons sleep is central to Cadence rather than a feature.

For language, the data economy is different in kind. A next-word learner acquires grammar by gradient over a corpus: many passes, every weight moving at every token. The record patch acquired a grammar’s regularities from one pass of writes and one night of dreams over its own store. On the same held-out grammar, a two-layer record patch, a GRU and a two-layer transformer trained by cross-entropy for fifty epochs all reach 100%; the patch got there with one pass and a night. On a twelfth of the data at equal updates, the patch’s held-out loss was the lowest of the three at every width: 0.32 nats per word against 0.34 for the GRU and 0.36 for the transformer at width 64. With half or all of the pairs the three land within 0.03 nats of one another. The difference between them is how little each needs.

Children hear a construction a handful of times and use it in fresh combinations after a few nights. The mechanism here does the same thing for the same reason, and it is the first one I know of that does.

For reasoning, the point is the persistent state. A transformer holds nothing between tokens. To write the next word of an audit it must have the whole audit in its window and rebuild the structure from the string at every step, which is why its cost grows with the window and why it cannot sleep on anything: there is nothing there to sleep on. A patch holds a settled state and a store between events, and the store is what the night works on. The finding that spans three functions read on three days is reachable because the three are held together in one store and settled together once the clamps come off. Understanding a code base, on this account, is a configuration that agrees with everything held, reached offline, and read out in the morning.

What has and has not been shown

The grammar is finite, with 1,276 messages and 6,173 sentences; “generalises” means to unseen combinations inside it, as judged by an acceptor. The dreams were sentences read out word by word from the store. The settled-sentence speaker, in which every word of a sentence is committed together before the first is spoken, is the design and is work in progress. Sleep replays the store’s errors, and the correction experiment is work in progress. No comparison with a transformer at scale has been made, and the settled-state account of reasoning is a design with a planned test.

What is measured is this: one pass of writes gives a grammar for 81.5% of its never-taught combinations, one night of the store’s own dreams lifts the slow weights from 0% to 100% of them with the world closed, and staying awake with the teacher gets 90.5%. Wagner’s subjects would recognise the shape of that, at least the ones who had slept.

[^wagner]: U. Wagner, S. Gais, H. Haider, R. Verleger and J. Born, Sleep inspires insight, Nature 427, 352–355 (2004). doi:10.1038/nature02223. 13 of 22 subjects gained insight into the hidden rule after sleep; 5 of 22 in each of the two wake groups.

[^marr]: D. Marr, Simple memory: a theory for archicortex, Philosophical Transactions of the Royal Society B 262, 23–81 (1971). doi:10.1098/rstb.1971.0078.

[^wilson]: M. A. Wilson and B. L. McNaughton, Reactivation of hippocampal ensemble memories during sleep, Science 265, 676–679 (1994). doi:10.1126/science.8036517.

[^cls]: J. L. McClelland, B. L. McNaughton and R. C. O’Reilly, Why there are complementary learning systems in the hippocampus and neocortex, Psychological Review 102, 419–457 (1995). doi:10.1037/0033–295X.102.3.419.

[^born]: S. Diekelmann and J. Born, The memory function of sleep, Nature Reviews Neuroscience 11, 114–126 (2010). doi:10.1038/nrn2762.

[^crick]: F. Crick and G. Mitchison, The function of dream sleep, Nature 304, 111–114 (1983). doi:10.1038/304111a0.

[^hinton]: G. E. Hinton, P. Dayan, B. J. Frey and R. M. Neal, The wake-sleep algorithm for unsupervised neural networks, Science 268, 1158–1161 (1995). doi:10.1126/science.7761831.

[^scellier]: B. Scellier and Y. Bengio, Equilibrium propagation: bridging the gap between energy-based models and backpropagation, Frontiers in Computational Neuroscience 11, 24 (2017). doi:10.3389/fncom.2017.00024.

[^cadence]: The record theorems (one-shot memory, the interference law, exact storage and capacity, pattern separation) are stated and proved in the Cadence paper and its Lean library: floatingpragma.io/cadence/paper.pdf and github.com/muellerberndt/cadence. The language experiments cited here are receipts R102 to R108 of that paper’s evidence map.

[^docs]: The record patch guide, with the sections “Acquisition in two phases” and “A grammar from records alone”: docs/record-patch.md.