Exploring HTM -Hierarchical Temporal Memory (HTM) machine learning framework and Java
One area of artificial intelligence that has fascinated me over the years is Jeff Hawkins’s work at Numenta — from NuPIC and Hierarchical Temporal Memory to his more recent work with Monty and the Thousand Brains Project.
Many software engineers are fascinated by the possibilities of artificial intelligence. Over the years, that interest has taken many forms: genetic programming, artificial life, statistical machine learning and, more recently, generative AI built with large language models.
Modern LLMs are based on transformer neural networks. Although neural networks were loosely inspired by the brain, they do not attempt to reproduce the brain in a biologically faithful way. Hawkins’s research takes a different approach. It asks whether understanding the structure and operation of the neocortex can lead to fundamentally different forms of machine intelligence.
Hierarchical Temporal Memory
HTM stands for Hierarchical Temporal Memory. It is a theory and computational framework inspired by the structure of the neocortex.
HTM systems learn continuously from streams of data. Rather than being trained once on a fixed dataset, an HTM model can learn changing temporal patterns as they arrive. Its computational model includes neuron-like cells, sparse distributed representations and connections that form as the system learns sequences. HTM has been applied particularly to anomaly detection. After learning the normal patterns in a data stream, the system can recognize when a new input does not match what it expected.
Monty and the Thousand Brains Project
Monty and the Thousand Brains Project extend these ideas even further. The Thousand Brains Theory proposes that intelligence does not depend on a single central model of the world. Instead, many cortical columns learn complementary models using reference frames. These columns can independently recognize parts of an object or situation and then reach a broader conclusion through a process resembling voting.
Monty is an experimental framework for exploring and testing these ideas computationally, particularly through sensorimotor learning. The goal is not simply to process a large static dataset, but to learn through movement, observation and continued interaction with an environment.
Today’s LLMs are extraordinarily sophisticated, but they learn and operate very differently from biological brains. I find Monty interesting because it explores whether a system can become more adaptive by learning through interaction in a way that may be closer to how brains build models of the world.
Why I Used Java
As part of exploring these concepts, I used an AI language model to help refactor a Java implementation of HTM. Why Java? I wanted to examine HTM through an object-oriented language that I already understand well. Java makes it easier for me to inspect the relationships among cells, columns, connections, sequences and predictions as concrete objects.
The project builds with Maven and includes unit tests. It also contains a Java Swing visualizer that demonstrates several of the core concepts.
Get Berlin Brown’s stories in your inbox
Join Medium for free to get updates from this writer.
I built and tested the project with:
- OpenJDK 25.0.2 LTS
- Apache Maven 3.9.12
Compile and run the visualizer with:
mvn compile exec:java@visualizerUnderstanding the Visualizer
The visualizer simulates a room thermostat producing 24 hourly temperature readings each day:
12 AM 64°F → 1 AM 63°F → ... → Noon 76°F → ... → 11 PM 65°F → next daySuppose the paused screen displays:
Thermostat input: Day 2 · 6 PM · 73°F
Previous guess: 6 PM · 73°F (plus internal column IDs)
Next sensor guess: 7 PM · 71°F (plus internal column IDs)
Anomaly: 0.000This tells us several things. Before receiving the current input, the model predicted a temperature of 73°F at 6 PM. The thermostat then reported exactly that value. Having correctly recognized the current point in the sequence, the model now predicts that the next reading will be 71°F at 7 PM.
An anomaly score of 0.000 means that the current input was completely expected based on the sequence the model had learned.
If the thermostat suddenly reported a highly unusual temperature, the anomaly score would increase. The model would not need a human-written rule defining the temperature as unusual. It would recognize the reading as anomalous because it did not match the learned temporal pattern.
That is what makes HTM particularly interesting to me: prediction and anomaly detection emerge from learning sequences rather than from manually programming every possible condition.
Recommended Reading
For anyone interested in the ideas behind this work, I recommend Jeff Hawkins’s books:
- On Intelligence
- A Thousand Brains: A New Theory of Intelligence, with a foreword by Richard Dawkins
The following resources provide more information about HTM and the Thousand Brains Project:
- Continuous Online Sequence Learning with an Unsupervised Neural Network Model
- Thousand Brains Project
HTM and Monty have not replaced transformers or become the dominant direction in artificial intelligence. They represent a different research path — one based on the possibility that understanding how the brain learns may reveal capabilities that current AI systems still lack.
For me, implementing and visualizing these ideas in Java makes the research more concrete. Instead of only reading descriptions of cells, columns, sequences and predictions, I can inspect the objects, run the tests and watch the model learn.
Htm.java code is available at:
