Why I Bet on Hierarchical Temporal Memory
How a model built to imitate the neocortex — not for machine learning — became my favorite machine learning method.
Most machine learning I’ve encountered is built the same way: pick a task, define a loss, optimize until the benchmark climbs. Hierarchical Temporal Memory (HTM) came at the whole thing backwards — and that inversion is what pulled me in. It wasn’t designed for any machine learning task, but to replicate a specific piece of machinery in the neocortex. Only later was it discovered to be good for anomaly detection.
This is the story of how that difference turned into a bet.
It started as a piece of the cortex, not a solution to a problem
HTM sequence learning is derived from a single layer of the cortical column — the basic compositional unit of the neocortex. The theory comes from Jeff Hawkins, whose 2004 book On Intelligence laid out the ambition, and from Numenta, the company he co-founded to turn that ambition into working algorithms. The goal was simply to capture how that cortical circuit represents an input and predicts what comes next. The model’s knack for anomaly detection was a pleasant surprise, not a design goal.
That’s a fundamentally different starting point. Almost every mainstream architecture begins with the task and works backward to a mechanism. HTM began with the mechanism — the biology — and the tasks came after. It’s the difference between designing a tool and reverse-engineering an organ.
Loyalty to the cortex gave it a native data structure
Because HTM is a cortical system above all, it doesn’t lean on the dense vectors most machine learning runs on. It represents everything with its own native data structure: the Sparse Distributed Representation (SDR).
An SDR is a large array of cells — a couple thousand, say — of which only around 2% are active at any moment. That looks wasteful, but the sparsity is the whole point. With so few cells on, two unrelated inputs almost never activate the same ones, so many patterns can coexist in the same space without colliding, and the number of distinct representations available is astronomically large.
The active bits aren’t arbitrary, either. Each one stands for something about the input, so similar inputs share active cells and dissimilar ones don’t — overlap literally measures semantic similarity. That buys two properties that matter enormously for anything operating in a messy world: SDRs are robust to noise and to faults, because you can lose or flip a good fraction of the active cells and still recognize the pattern from what remains, and they degrade gracefully instead of breaking.
Most important for sequence learning, the column structure lets one sparse pattern carry two things at once — the spatial content of an input (what it is) and its temporal context (where it sits in a sequence). The same feature in two different contexts lights up different cells, which is what makes prediction possible. And with so few cells active, all of it runs on very little energy. These aren’t stylistic flourishes; they’re the non-negotiable requirements of a representation an animal has to survive on — and exactly what a dense, data-center-scale vector is not.
It learns like living tissue — Hebbian, not backprop
On top of that sparse structure, HTM learns the way tissue does: Hebbian. Connections form and strengthen between cells that fire together, and weaken when they don’t. There’s no global backward pass, no gradient threaded through every weight in the network.
That’s a different paradigm from the backpropagation lineage that has dominated neural networks — not just a new loss or activation function. Today’s leading sequence models are genuinely effective: LSTMs learn long-range dependencies, and the transformers behind LLMs capture sequential structure of staggering complexity. But none of them are constrained to the brain. They can grow as large as the hardware allows, revisit their entire training history in massive batches, and burn as much compute as a data center can supply — their design answers to the loss function, not to what a cortical circuit could plausibly implement. HTM accepts the opposite discipline: whatever it does has to be something the tissue could actually do.
Complexity it grows, not complexity it’s handed
That discipline changes what nodes and links even are. In a backprop network they define one enormous function whose millions of weights are the tunable dials — and its complexity is fixed the moment you pick the architecture. A deep network is a high-capacity function from its very first training step, whether or not the data warrants it. Regularizers like dropout trim capacity at the edges, but the model may still start out far more expressive than the problem needs — which is exactly why it can memorize noise and has to be actively defended against overfitting.
Get Sam Heiserman’s stories in your inbox
Join Medium for free to get updates from this writer.
HTM inverts this. It starts essentially empty and grows connections as it needs them — strengthening links from a latent pool of potential ones to capture specific recurring transitions in the sequence it sees. Complexity accretes to match the structure in the data instead of being provisioned up front. A system that builds only what it observes isn’t predisposed to overfit the way a fixed high-capacity function is — there’s no reserve of unused expressiveness lying around waiting to memorize noise.
This is the deeper reason the biology matters. The brain runs on a hard budget — limited space, limited energy, no way to summon more compute. Under that constraint you can’t win by scaling up; you need a data structure like the SDR and a local learning rule that grows only what it must. Not superhuman at one narrow task, but good enough across many that the animal survives.
How far it sits from everything else
The payoff of all this is a balance that’s genuinely hard to find: HTM learns complex sequences from very little data while resisting overfitting. Most methods manage one or the other. Set it beside the other common ways to model sequences and the divergence is stark — and it traces back to that one difference in how each system decides its own complexity.
Then they proved it wasn’t just a nice story
Biological elegance is easy to fall in love with, and easy to be fooled by. I wanted rigor too — and this is where HTM closed the deal. Numenta built the Numenta Anomaly Benchmark (NAB): scores of real-world streaming datasets, a scoring method designed for the realities of real-time detection (rewarding early, catching anomalies as they emerge), and a head-to-head comparison against the leading unsupervised time-series anomaly detectors of the day. HTM held its own.
What sold me wasn’t just the ranking. The NAB corpus spans genuinely different real-world domains — server and network telemetry, machine and ambient sensor readings, traffic, online-ad and social-media signals — and HTM performed across all of them with a single configuration, no per-dataset hyperparameter optimization. That matters more than it might sound. A method that has to be re-tuned for every new domain is partly overfit to the benchmark itself, but a method that works out of the box across domains shows exactly the general, robust competence the biological story predicts it should have. The grow-only-what-you-observe design wasn’t just an argument on paper — it showed up in the results.
So the foundation was both things at once, biologically faithful to the cortical circuit and engineering-validated on a sturdy benchmark.
What you give up
None of this comes free, and it would be dishonest to pretend otherwise. The same commitments that make HTM robust also close doors that mainstream deep learning leaves wide open.
The biggest is that HTM assumes the world arrives as a stream — sequential, temporal data — and builds that assumption into its bones. That’s a natural fit for time series, sensor feeds, and streaming detection, but it isn’t the universal hammer a transformer has become; you don’t reach for HTM to classify a static image.
The other is that you can’t buy your way to more capability. In deep learning, when you want more power you make the model bigger — more layers, more parameters, more features, more compute — and, remarkably often, it works. HTM has no equivalent lever. Its capacity is bounded by the SDR and the size of the region, so you can’t conjure an arbitrarily large model with an arbitrary number of features just by scaling the network up. The discipline that keeps it from overfitting is the same discipline that caps its raw scale.
And it’s simply less mature. It has a fraction of the tooling, tutorials, and person-years that have poured into the backprop ecosystem, and it’s been tried on far fewer problems — so some of what it can and can’t do is still an open question rather than a settled fact.
Why the bet
Knowing all of that — the imposed temporal structure, the ceiling on scale, the thin tooling — I made the bet anyway. A model that began as a fragment of cortex. That brought its own sparse, energy-cheap data structure. That learns by forming and pruning connections instead of chasing a gradient. That scales the way a space- and energy-bound brain has to scale — generalist, robust, continual. And that still stood up to serious benchmark testing.
Each of those is a departure from the mainstream recipe. Together they’re a coherent bet: that the road to robust, continually-learning, general intelligence might look less like a bigger transformer and more like the thing sitting between our ears. That’s the bet I made — and why HTM, of everything I’d seen, is the one I chose to chase.
