Spiking Neural Networks Explained
How brain-inspired computing could redefine the future of AI
This article is the fifth installment in my Computational Neuroscience series. In the previous post, we explored the Leaky Integrate-and-Fire (LIF) neuron, one of the foundational models of neural computation. We now take the next step, from individual neurons to networks of spikes, and explore how the brain’s event-driven way of computing is inspiring a new generation of AI.
TL;DR
- The Crisis: Modern AI is phenomenally capable and phenomenally wasteful. Training frontier models may consume tens of gigawatt-hours of electricity. Your brain runs the same class of intelligence on 20 watts.
- The Shift: Spiking Neural Networks (SNNs) are not “better ANNs.” They are a fundamentally different computational paradigm, temporal, event-driven, and sparse where information lives in the timing of discrete pulses, not the magnitude of continuous numbers.
- The Destination: Neuromorphic chips, event cameras, and cortical computing architectures are converging toward a new kind of intelligence: one that runs on milliwatts, adapts after deployment, and computes only what matters, only when it matters.
The Hidden Energy Crisis Eating AI Alive
Estimates suggest frontier model training may consume tens of gigawatt-hours of electricity. A single inference request on a large language model burns roughly 10× the energy of a Google search. The data centers powering modern AI are on track to consume more electricity by 2030 than some mid-sized countries do today.
Now sit with this for a moment: your brain, the same three-pound organ that learns languages, writes poetry, and navigates complex social dynamics operates on roughly 20 watts of power. This is not a marginal difference. It is a chasm so wide that it demands a fundamental rethinking of how we build intelligent machines. And increasingly, researchers believe the answer was never buried in some future breakthrough. It was in our heads the whole time.
The intuitive thought coming from the brain that it is not slow because it’s efficient. It is both fast and efficient. The trick is that it almost never computes everything at once, only the parts that matter, at the moment they matter.
The reason biology achieves this is deceptively elegant. The brain is event-driven and sparse. Neurons only fire when they have something important to say. Between messages, they rest. In a conventional deep learning model, every neuron fires on every forward pass, a massive, synchronous, continuous computation. It’s like leaving every light in a skyscraper on at all times, whether anyone is in the building or not.
This is the foundational insight behind what is often referred to as the third generation of neural networks: SNNs. The first generation were perceptrons, static linear models. The second, the deep learning revolution, rich but power-hungry. The third promises something different: intelligence that computes only when necessary, encodes information in time itself, and, eventually, runs on hardware that barely draws power when silent.
ANNs vs. SNNs: The Core Contrast
Before going deeper, it helps to have the full picture side by side.
How SNNs Actually Work, From the Ground Up
The Contrast: Continuous vs. Discrete. Before we can understand SNNs, we need to feel the difference between what they do and what conventional artificial neural networks do. In a standard ANN, a neuron computes a real-valued output: f(x) = σ(Wx + b). That sigma is some smooth activation function ReLU, sigmoid, tanh etc. The output is a continuous floating-point number, flowing forward through the network in a single synchronized sweep. Clean. Mathematically tractable. Computationally expensive.
A spiking neuron communicates in binary temporal events, spikes. A 1 at a specific moment in time means the neuron fired. A 0 means it didn’t. That’s it. The information isn’t in the magnitude of a number; it’s in the timing and frequency of these discrete pulses.
The Biological Engine: The LIF Neuron
The workhorse model of computational neuroscience and the building block of most practical SNN architectures, is the Leaky Integrate-and-Fire (LIF) neuron. The name is the manual. It does three things:
Integrate: The neuron accumulates incoming signals as a membrane potential, V(t). Each upstream spike contributes a small charge, like water dripping into a bucket. Mathematically:
Here, τ is the membrane time constant, R is a resistance term, and I(t) is the incoming synaptic current at time t. The potential builds as long as signals keep arriving.
Leak: The bucket has a hole. When inputs stop or slow down, the membrane potential decays back toward the resting potential. This is the “leaky” part, mathematically captured by the −V(t) term. Without this, neurons would accumulate spurious historical charge forever, making the network unmanageable.
Fire: When V(t) hits a threshold ϑ, the neuron fires a spike, a single binary event and resets to a baseline Vreset:
That reset is crucial. After firing, the neuron enters a brief refractory period, it’s temporarily unable to fire again, regardless of input. This mirrors biology and, conveniently, creates a natural temporal rhythm.
What makes this profound is the role that time plays. In an ANN, time is a nuisance, data is flattened, sequences are chunked, everything is processed in a single synchronous batch. In an SNN, time is a first-class data dimension. The precise moment a spike occurs carries meaning. Two neurons that both fire once per second can communicate completely different information depending on when they fire relative to each other, a concept called spike-timing-dependent plasticity.
The Multi-Dimensional Language of Spikes
Here’s the misconception that derails most first-encounters with SNNs: they are not just ANNs with binary activations. The computational richness of spiking networks comes from the multiple distinct ways information can be encoded in spike patterns.
Rate Coding is the most intuitive. A neuron firing 5 spikes per second encodes a weak stimulus; 50 spikes per second encodes a strong one. It’s the closest analog to traditional ANN activations, frequency as magnitude. But it’s also the least interesting, because it throws away the richest dimension available: time.
Temporal Coding is where SNNs achieve true expressive power. A spike at t = 10 ms carries an entirely different computational meaning than a spike at t = 30 ms not just in isolation, but relative to every other spike in the network. The brain doesn’t wait to count how often a neuron fires; it extracts meaning from when it fires.
Population Coding distributes representations across groups of neurons. No single cell encodes “face”, a sparse ensemble of distinct neurons jointly encodes the concept, each contributing a fragment of the overall representation. This mirrors biological cortical dynamics and provides robustness: damage one neuron, the code survives.
Rank Order Coding is perhaps the most elegant. The first neuron to fire carries the primary information payload. Subsequent neurons add refinement. This scheme enables ultra-fast, single-pass inference; the brain can begin recognizing an object from just the earliest arriving spikes, before the full pattern has even propagated.
Together, these four schemes give SNNs a vocabulary that no continuous-valued ANN can natively express. Time isn’t bolted on as an afterthought via sequence tokenization, it is the fundamental medium of computation.
Event Cameras: The Perfect Sensor Pair
There’s a fundamental mismatch that catches newcomers off guard: pairing a traditional 30 FPS frame-based camera with an SNN largely defeats the purpose. A standard camera produces dense, synchronous image tensors, the exact format SNNs are not designed to process. You’ve imported the inefficiency you were trying to escape. The natural sensor partner for a spiking network is the Event Camera, also called a Dynamic Vision Sensor (DVS).
Event cameras operate nothing like conventional imaging hardware. Each pixel monitors its local patch of the scene independently and asynchronously. When a pixel detects a change in brightness of anything moving, appearing, or disappearing, it immediately fires an event, a discrete spike encoding its location, timestamp, and the sign of the brightness change. When nothing moves, no data is generated at all.
The analogy is precise: a static wall produces zero data. A moving object produces a torrent of microsecond-resolution discrete spikes tracking every edge, every contour, every trajectory in real time. Latency measured in microseconds. No motion blur. No wasted bandwidth on unchanged background pixels.
Get Nikita S Raj Kapini’s stories in your inbox
Join Medium for free to get updates from this writer.
The research ecosystem driving this field has coalesced around three benchmark datasets: N-MNIST (the neuromorphic equivalent of handwritten digit recognition), DVS128 Gesture (hand gesture classification from event streams), and CIFAR10-DVS (spiking versions of the standard 10-class image dataset). These are the benchmarks where SNN architectures prove or fail their mettle on genuine neuromorphic inputs.
Learning Without Backpropagation: STDP
The field’s most fundamental unsolved problem is that you cannot train spiking networks with standard backpropagation. The reason is almost poetically simple: backprop requires computing the derivative of your activation function. In a standard ANN, that’s a smooth sigmoid or ReLU. A spike, however, is a Heaviside step function, its derivative is zero everywhere except at the threshold, where it’s technically undefined. You can’t backprop through discontinuities.
The elegant hack keeping the field alive is surrogate gradients: during the forward pass, neurons spike normally (discrete, non-differentiable). During backpropagation, a smooth proxy function substitutes for the derivative, a sigmoid, a piecewise-linear ramp, a Gaussian bump. You’re lying to the optimizer about what the activation function looks like. Remarkably, it works.
But there’s a deeper, more biologically honest alternative: Spike-Timing-Dependent Plasticity (STDP).
The foundational rule can be stated simply: neurons that fire together, wire together; neurons that fire out of sync, lose their link.
The mechanics are precise. If pre-synaptic neuron A fires just before post-synaptic neuron B, there is a causal relationship, A likely contributed to B’s firing. The synapse between them strengthens (Long-Term Potentiation, LTP). If neuron A fires after neuron B, no causal link exists. The connection weakens (Long-Term Depression, LTD).
STDP is biologically plausible in ways backprop almost certainly isn’t. The brain has no mechanism to send error signals backward through layers. Learning happens locally, at the synapse, based only on the relative timing of pre- and post-synaptic spikes. This makes STDP the backbone of unsupervised SNN learning, and the focus of libraries like BindsNET that prioritize biological realism over benchmark performance.
Why You Can’t Just Run This on a GPU
Here’s a frustrating reality that catches many practitioners off guard: simulating an SNN on conventional hardware captures very little of its promised efficiency. In many cases, it can even be slower and more power-hungry than running a comparable ANN.
The culprit is the Von Neumann bottleneck. In conventional computing architectures such as CPUs, GPUs, and TPUs, memory and computation are physically separated. Every operation requires data to move between storage and processing units over a shared bus. This movement consumes energy, generates heat, and often dominates the overall cost of computation.
For dense ANN workloads, this overhead is tolerable because most neurons participate in every forward pass. The data being moved is actively contributing to computation. SNNs, however, are inherently sparse and event-driven. At any given moment, a large fraction of neurons may be silent. While this reduces meaningful computation, conventional hardware still performs dense matrix operations and incurs the same memory-transfer costs. As a result, much of the potential efficiency of SNNs is lost in simulation.
The benefits of spiking networks emerge when the hardware itself follows the same principles as the brain: local memory, event-driven computation, and asynchronous communication.
This is the promise of neuromorphic hardware. Instead of separating memory and processing, neuromorphic chips co-locate them. Synaptic weights are stored alongside the circuits that use them, minimizing data movement. Computation is triggered only by incoming events rather than by a global clock, allowing inactive regions of the chip to consume almost no dynamic power.
The leading neuromorphic chips today are Intel’s Loihi 2, IBM’s TrueNorth, and Manchester University’s SpiNNaker platform. Loihi 2 can simulate up to one million neurons while consuming milliwatts. TrueNorth demonstrated a 4,096-core chip running inference tasks at 70 milliwatts, orders of magnitude below any GPU equivalent. SpiNNaker is particularly interesting for neuroscience: it’s a massively parallel platform designed to simulate brain-scale models in real time.
The lesson is simple: SNNs are not inherently energy-efficient on conventional hardware. Their efficiency comes from the combination of spiking computation and hardware designed to exploit its sparsity.
The Pragmatic Bridge: ANN-to-SNN Conversion
Researchers rarely train massive SNNs entirely from scratch. The dominant industrial pipeline is more pragmatic:
- Train a standard, highly accurate ANN using mature deep learning toolkits on conventional hardware.
- Convert the continuous activations into equivalent spiking frequencies or threshold-crossing dynamics.
- Deploy the converted network onto neuromorphic silicon to capture energy efficiency without losing baseline accuracy.
This conversion path isn’t perfect, some accuracy degradation occurs, and temporal dynamics require careful calibration but it bypasses the most painful parts of direct SNN training while still reaching the hardware that makes the efficiency case. For teams targeting edge deployment today, it’s the most viable on-ramp.
The Elephant in the Room: SNNs vs. Transformers
The field can’t avoid the question: why not just build spiking transformers?The tension is architectural. Standard attention mechanisms are intrinsically dense and globally connected, every token attends to every other token, producing a full quadratic interaction matrix. Spikes are sparse and localized, activity propagates through specific pathways, not global broadcast. These are fundamentally opposing computational philosophies.
And yet the research is happening. Spikformer introduced spiking self-attention mechanisms that replace the softmax attention map with spike-based approximations, achieving competitive vision classification while dramatically reducing energy per forward pass. SpikeGPT extended the experiment to language, demonstrating that autoregressive generation is at least possible in a spiking framework, even if far from the capability frontier. Both projects are early-stage, with significant gaps to close.
The provocative thesis is gaining traction: the future may not be Transformers vs. SNNs, but rather Transformers structurally reimplemented using spiking principles, retaining the attention-based computational graph while replacing the dense, synchronous arithmetic with sparse, event-driven spike dynamics. Whether that retains what makes transformers powerful, or merely produces a power-efficient shadow of them, remains an open question.
Beyond SNNs: Toward Cortical Computing
Here’s the honest frontier: individual spiking neurons, however faithfully modeled, are not the brain’s primary unit of computation. The brain’s power emerges from its organization.
The cortex is structured in six-layer columns, repeating micro-circuits stacked across the entire surface of the brain, each one implementing a similar local computation. These columns don’t just process; they predict. The dominant contemporary theory of predictive coding holds that the cortex is fundamentally a hierarchy of prediction machines. Each layer sends predictions downward about what it expects to receive, and upward propagates only the error, the surprising residual between prediction and reality. Computation is proportional to surprise, not stimulus intensity.
Combine this with sensorimotor loops, where motor commands alter sensory predictions in real time, grounding cognition in action and voting mechanisms across distinct cortical regions, where high-level decisions emerge from distributed consensus rather than centralized readout and you get a picture of intelligence that looks nothing like a feedforward network of any kind.
The most ambitious researchers in neuromorphic AI aren’t asking “how do we make spiking neurons more efficient?” They’re asking: what does it look like to build artificial systems organized like the cortex is organized? That question is substantially harder, substantially less funded, and arguably the one that matters most in the long run.
The Developer Ecosystem: Your SNN Toolkit
If you’re coming from a deep learning background, the SNN tooling landscape can feel sparse by comparison. But in the past three years, the ecosystem has matured considerably. Here’s the practical map:
- snnTorch (PyTorch): The most approachable library for DL practitioners. Built natively on PyTorch, it supports training via backpropagation through time (BPTT) and converting trained CNNs into spiking variants. If you want to go from zero to a working spiking model fast, start here.
- Norse (PyTorch): A functional PyTorch library that treats SNNs as differentiable programs. Strong support for surrogate gradient methods. More flexibility than snnTorch for custom neuron models and novel training algorithms.
- BindsNET (PyTorch): Designed with biological plausibility in mind. Strong support for unsupervised learning rules like spike-timing-dependent plasticity (STDP). Best for researchers interested in biologically realistic learning, not just energy-efficient inference.
- Brian2 (Python): The gold standard for high-fidelity computational neuroscience simulation. Not designed for ML workflows, but unparalleled for modeling precise biological dynamics. If your work touches connectomics or neural circuit modeling, Brian2 is the tool.
- Lava (Intel Loihi; Hardware Deployment): Intel’s open-source framework for deploying on Loihi neuromorphic hardware. The abstraction layer above the chip. If you’re targeting real neuromorphic inference, not simulation. Lava is the bridge between your model and the silicon.
- SpikingJelly (PyTorch): Optimized for spiking computer vision pipelines, particularly with event cameras (neuromorphic vision sensors). Strong benchmark suite and support for the N-MNIST, CIFAR10-DVS neuromorphic datasets.
The Long Game
SNNs remain a young and rapidly evolving field. The training methods are more complex, the tooling ecosystem is smaller, and neuromorphic hardware has yet to achieve the ubiquity of GPUs. What makes the field exciting is not the promise of replacing existing models overnight, but the possibility of expanding what intelligent systems can be. By embracing sparse, event-driven computation and hardware designed around those principles, SNNs open a path toward AI systems that are more efficient, adaptive, and naturally suited to continuous interaction with the real world.
The trajectory over the last five years has been steep. Loihi 2 is 10× more efficient than Loihi 1. snnTorch closed the PyTorch integration gap. Hybrid convolutional-spiking architectures have quietly started delivering competitive numbers on tasks that previously seemed out of reach. And neuromorphic principles have already found profound success in practical applications, from autonomous drone navigation under power constraints and real-time EEG seizure detection, to early implementations of auditory prosthetics like cochlear implants that leverage event-driven signal processing.
Whether the future belongs to SNNs, transformers, neuromorphic hardware, or something yet to be discovered is ultimately beside the point. The goal is not to replicate the brain perfectly or chase the latest wave of hype. The goal is to build intelligent systems that solve real problems in the most efficient, scalable, and sustainable way possible.
As AI continues to evolve, the question is not whether we can build larger models. It is whether we can build smarter ones. The future will belong not to the systems that consume the most computation, but to those that achieve the most with the least.
