Memory Needs for AGI

The Hidden Truth About Bandwidth Requirements

Memory needs for AGI are far beyond what current AI systems require, especially when it comes to bandwidth. To match human-level intelligence, an AGI system must handle tens of terabytes of memory with bandwidths reaching 1000 GB/s or more. This is because the brain’s linguistic cortex, which supports complex cognition and language, operates with immense data flow and storage that current models simply don’t replicate.

I remember the moment I first grasped the scale of this challenge. I was working on a project involving large language models, and while training a 70 billion parameter model, I noticed the memory demands were already pushing the limits of our hardware. Yet, when I read about the memory bandwidth needed to emulate human brain functions, it was clear that even our biggest models were just scratching the surface. The sheer volume and speed of memory access required for true AGI seemed almost unimaginable.

This realisation sparked a journey to understand why memory bandwidth is so critical for AGI and how current technology stacks up. It’s not just about having a lot of memory but being able to move data quickly enough to support real-time learning and inference. Without this, an AGI system would be bottlenecked, unable to process the complex, multimodal information…