The Book That Almost Killed Neural Networks
How We Got Here — Episode 2 of 26. We are walking the AI timeline one contribution at a time.
Every time you call Claude, every time a coding agent decides whether to read the file or run the test — you are using a multi-layer neural network. Tens of layers deep. Hundreds of billions of weighted connections. That architecture is the foundation of the entire modern AI stack.
In 1969, two researchers at MIT proved — with mathematical precision — that this exact architecture was impossible.
Not impractical. Not difficult. Impossible. Computationally impossible to train. A dead end. The field believed them. Funding collapsed. Frank Rosenblatt, the psychologist whose perceptron we walked through last Tuesday, was publicly humiliated in the book’s introduction. Two years later, he was dead in a boating accident on his 43rd birthday.
That is the standard story of the first AI Winter. Minsky killed neural networks. I want to walk through it carefully today, because the standard story is too clean. The actual story is more interesting. And the actual mathematical claim is more nuanced. And the man who supposedly killed neural networks had, twenty years earlier, built one of the first neural networks ever attempted.
That is where Episode 2 starts.
Quick recap
If you didn’t catch Episode 1 — the one-paragraph version is this. In 1958, a psychologist at Cornell named Frank Rosenblatt built the perceptron. It was the first machine that learned a yes-or-no decision from labeled examples by adjusting numerical weights. He proved, mathematically, that if a linear solution to the problem existed, his training procedure would always find it. He then built the Mark I Perceptron in hardware — a wall of photocells and motors and adjustable knobs that the Navy paid for. The press, the Navy press office in particular, oversold it spectacularly. The New York Times in July 1958 ran a piece predicting the machine would eventually walk, talk, see, write, reproduce itself, and become conscious.
It was an extraordinary technical achievement followed by an extraordinary publicity overreach.
Two years before Rosenblatt’s Mark I, a younger researcher named Marvin Minsky — Rosenblatt’s classmate from Bronx Science — had been working on a related machine of his own. That part of the story almost never gets told, and it changes the whole frame.
Minsky was a believer first
The standard cartoon of Marvin Minsky has him as the skeptic. The symbolic-AI guy. The MIT establishment figure who hated neural networks on principle and finally took them down with a book in 1969.
That cartoon is wrong on its own terms.
In 1951, while Minsky was a graduate student at Princeton, he built a machine called the SNARC — the Stochastic Neural Analog Reinforcement Calculator. It used a tangle of vacuum tubes, capacitors, motors, and clutches to simulate a small network of artificial neurons. He designed it to model a rat learning its way through a maze. The weights were stored as the positions of physical clutch plates. When the system “learned,” motors physically rotated the clutch plates by small amounts — the same idea Rosenblatt would later use with potentiometers.
Read that again. The man who would, eighteen years later, be remembered as the executioner of neural networks built one of the first neural network learners in history. Before Rosenblatt’s perceptron. Before any of it.
Minsky was a believer. He started where Rosenblatt started — convinced that intelligence would emerge from networks of simple units adjusting themselves through exposure, not from hand-written logical rules.
What changed between 1951 and 1969 was not Minsky’s opinion of whether neural networks were possible. He had already proved them possible, in hardware, with his own hands. What changed was his estimate of the path forward.
By the mid-1960s, Minsky had come to believe that the way to actual humanlike intelligence was not bottom-up — millions of small units adjusting weights — but top-down. Build representations. Write the rules. Encode what we know about how thought works, and let the machine reason over those representations. That was the symbolic AI program. That was what he and his colleagues at MIT were pursuing.
And there was only so much money to fund AI research in the United States in 1968.
The funding fight nobody likes to talk about
This part of the history gets sanitized in the academic retellings. I want to put it back in.
In the mid-to-late 1960s, the dominant US funder of AI research was the Defense Advanced Research Projects Agency (DARPA, then ARPA) and the Office of Naval Research (ONR). The total pool was finite. Rosenblatt’s connectionist program — neural networks, learning from examples, the perceptron lineage — had become the flashy bet. He had built actual hardware. He had compelling press coverage. He was the kind of researcher who showed up to demonstrations driving a sports car and playing piano for reporters. The funders liked him.
Minsky’s symbolic AI program at MIT had to compete for the same dollars. And by 1968, they were losing the public-facing war.
This is the part you have to hold in your head while you read the book they were about to write. Perceptrons, when it landed in 1969, was not a disinterested technical critique. It was a salvo in a fight over which research program would inherit the field. Minsky was not pretending otherwise. In private correspondence and later interviews, he acknowledged that part of the motivation for the book was to redirect the field’s resources toward the work he believed was more promising.
That does not make the math in the book wrong. The math is actually impeccable. We are going to walk through it. But the framing of the math — the choice of which limitation to surface, which to ignore, and how aggressively to extrapolate from a specific limit to a general death sentence — that framing was strategic. It was a fight for the future of a field.
I keep saying this because the standard rendering of this story misses it. Perceptrons was not science indifferent to its consequences. It was science with a stake in the outcome.
What the book actually proved
The book itself is short — about 250 pages, beautifully typeset, originally published by MIT Press in 1969. The technical centerpiece is a series of theorems about what a single-layer perceptron cannot compute.
The most famous of these is the XOR theorem.
XOR — exclusive or — is the simplest logical function that requires more than a straight line to separate. Two inputs. The output is 1 if the inputs disagree (0,1 or 1,0) and 0 if they agree (0,0 or 1,1). Plot the four points on a 2D graph. The two “trues” sit at opposite corners. The two “falses” sit at the other opposite corners. Try to draw a single straight line that puts the trues on one side and the falses on the other. You cannot. Not at any angle. Not at any position.
A perceptron, in the architecture Rosenblatt built in 1958, can only learn problems where the two classes are separable by a single hyperplane. XOR is not such a problem. Therefore, a single-layer perceptron cannot learn XOR.
Minsky and Papert proved this with formal mathematics in Perceptrons. The proof is correct. It has never been overturned. It is, technically, a real limitation of the 1958 architecture.
They went further. They proved a series of theorems about which problems a perceptron could and could not represent, depending on properties of the input. The book is full of careful, detailed work. As a piece of mathematics, it is excellent.
But the limit they proved was about a single layer. One row of perceptrons feeding one output. Everyone in the field — including Rosenblatt — already knew that adding a hidden layer between the input and the output would, in principle, let you represent XOR and a great many other things. The question was not whether multi-layer networks could represent harder problems. The question was whether you could train them.
That is where the book made its lethal claim.
The “computationally impossible to train” claim
In the book’s discussion section, Minsky and Papert addressed the multi-layer question directly. Their assessment, paraphrased: a multi-layer network might in principle solve problems a single-layer network cannot, but “we consider it to be an important research problem to elucidate (or reject) our intuitive judgment that the extension is sterile.”
Get Cordero Core’s stories in your inbox
Join Medium for free to get updates from this writer.
That sentence sounds careful. In context, it was devastating. Translated into the field’s actual response to the book, it became: Minsky has shown multi-layer networks are a research dead end. No promising training procedure was known in 1969. Rosenblatt’s convergence theorem worked only for a single layer. There was no analog for stacks. Without a way to train them, multi-layer networks were beautiful toys with no engineering future.
And in 1969, that was technically true. The training algorithm that would unlock multi-layer networks — backpropagation, Episode 3 of this series — had not yet been published in a form anyone could use. Paul Werbos’s doctoral dissertation, which contained the first usable version, would not appear until 1974. The paper that actually convinced the field, the Rumelhart-Hinton-Williams paper, would not appear until 1986. Seventeen years after Perceptrons.
So in 1969, if you took Minsky and Papert at their word, the picture was this. Single-layer perceptrons can’t solve interesting problems. Multi-layer networks could in principle, but no one knows how to train them, and the leading expert on the subject thinks the extension is sterile.
The field believed them. Funding for neural network research collapsed. Connectionism, as it was called, lost its place in the AI research portfolio for roughly a decade. The first AI Winter began.
What the book said about Rosenblatt personally
It was not just the math that landed hard. The book’s introduction included a passage about Rosenblatt’s writing — his ambitious 1962 monograph Principles of Neurodynamics — that called his work, in essence, scientifically unserious. The exact phrase the field remembered was that much of his writing was “without scientific value.”
I want to slow down here, because this is the human moment in the technical story and it gets glossed over.
Inside the field, in 1969, Rosenblatt was a serious researcher with a serious result. He had built the first machine that demonstrably learned from examples. He had proved a convergence theorem. He had built the hardware in his own lab. He was the founder of an entire research program.
Having his work labeled “without scientific value” in a book from MIT Press, co-authored by the most influential AI researcher of the era, was not a technical critique. It was a public unseating. The field read it that way. Rosenblatt’s funding dried up. His program at Cornell lost momentum. The press coverage dried up too — the press that had once chased him to write breathless headlines about machines that would walk and talk now had nothing to say.
Two years later, on July 11, 1971 — his 43rd birthday — Frank Rosenblatt died in a boating accident in Chesapeake Bay.
You will read different versions of how to interpret this in the history. Some of the more aggressive renderings frame it as essentially a consequence of the book. That framing is too much, I think. There is no evidence the death was anything other than what it appears to have been — an accident, on the water, on a summer Sunday. But the human cost of Perceptrons on Rosenblatt as a working scientist was real, and his death two years later meant he never got to see the vindication that was coming.
I want this episode to sit with that for a beat before we keep moving. The history of this field is full of papers. It is also full of people whose careers got broken by other people’s papers. We owe Rosenblatt the acknowledgment.
What Minsky was wrong about
Here is the mathematical truth, with the benefit of fifty-six years of hindsight.
Minsky and Papert’s theorems about the single-layer perceptron were correct. The single-layer perceptron really cannot learn XOR. That part of the math is permanent.
Their assessment that the multi-layer extension was “sterile” — that one was wrong. And it was wrong in the most consequential way it could have been wrong. The extension to multi-layer networks turned out to be the entire engineering future of AI.
What broke the wall was a small algorithmic move with a long history. The intuition is simple: if you can compute how much each weight in the final layer contributed to the error, you can also compute, with a chain-rule trick, how much each weight in the previous layer contributed. And the previous layer before that. All the way back to the input. Once you have those numbers, you can update every weight in the network in the direction that would have produced a better answer. You train the whole stack, not just the last layer.
That move is backpropagation. It is the algorithm that makes deep neural networks possible. Without it, every neural network in the world would still be a single-layer perceptron and we would still be stuck where Minsky said we were stuck.
Paul Werbos worked out a usable version in his Harvard doctoral dissertation in 1974, although it sat largely unread. David Rumelhart, Geoffrey Hinton, and Ronald Williams published the paper that convinced the field in 1986. By the late 1980s, multi-layer networks were learning things that single-layer perceptrons could not. The “sterile” extension turned out to be the entire field.
Minsky lived to see all of this. He died in 2016, at 88. By that point, deep learning had already become the dominant research program in AI, and his colleagues at MIT were running labs that depended on the multi-layer architecture he had once written was sterile. He did, in a 1988 expanded edition of Perceptrons, remove the most personal language about Rosenblatt and add a dedication to him. For some readers in the field, that was a meaningful gesture. For others, it was too late.
In 2004, the Institute of Electrical and Electronics Engineers — IEEE — established the Frank Rosenblatt Award for outstanding contributions to computational intelligence. It is, in a sense, the field’s official posthumous correction. Rosenblatt was the founder, the original thinker, the one who saw the path. The dispute was real. The verdict, after all the math came in, was not the one Minsky’s book had written.
What this episode is actually about
The standard rendering of this story makes it about a villain. Minsky killed neural networks. I do not think that rendering survives a careful read of the actual history.
What I think the story is about is this. Two researchers who started in roughly the same place — both believers, both with their hands on early neural network hardware — diverged on the question of which path to bet on. One of them, by accident of position and reputation, had a much bigger megaphone than the other. He used it. He proved a real mathematical limit, and then he extrapolated from that limit to a general claim about the field that turned out to be wrong in the most consequential possible way. The field believed the wrong extrapolation, and we lost a decade.
The lesson I keep coming back to is not “be nicer in your research.” It is something harder to act on. Mathematical proofs are not field-killing or field-saving on their own. The framing around them is. Perceptrons the book contains the same theorems whether you read it in 1969 expecting it to settle the field or in 2026 expecting it to be a careful study of one architecture’s limits. The math has not changed. What changed was what the field decided the math meant.
When you read someone declaring a path “computationally impossible” — especially in a field moving as fast as this one — the right question is not “is the math correct?” The math is often correct. The right question is “what is the gap between the specific limit the math proves and the general claim being built on top of it?” That gap is where decades get lost.
What this still teaches you about your stack
Bring this back to your laptop.
When you use any modern AI tool — Claude for coding, GPT for analysis, an agent that routes between tools — you are running a multi-layer neural network. The thing Minsky and Papert called sterile. The architecture that, for almost a decade, the field believed could not be trained.
The reason it can be trained, and the reason the whole modern stack works, is the backpropagation algorithm and the activation functions that made it numerically stable. We are going to walk through both, slowly, in Episode 3.
But the meta-lesson from this episode — the one I want you to carry between now and next Tuesday — is the engineering instinct about what kind of claim to trust. When a respected expert tells you a path is impossible, you can take that seriously without taking it as final. The path Minsky and Papert called impossible turned out to be the entire engineering future of their own field. That is the most expensive “impossible” call I am aware of in the modern history of computing.
You probably encounter smaller versions of this every quarter. This architecture won’t scale. This approach won’t generalize. This tool won’t work for the use case. Sometimes the person saying it has the math. The question is whether the math actually generalizes to the claim, or whether you are watching a specific limit get extrapolated into a sweeping verdict.
Rosenblatt did not get to see the vindication. He died two years after the book, in the same year his program lost its funding. But the irreducible unit he built — the perceptron — kept running, stacked deeper than anyone in 1969 thought possible, and is now running inside every model you touch.
The book that almost killed neural networks did not, in the end, kill them. It cost the field a decade and broke a man’s research career, but it could not kill them, because the mathematics of multi-layer networks were not actually impossible. They were just unsolved in 1969.
That is the human and technical truth I wanted to land this episode on.
Next Tuesday: backpropagation. The algorithm that broke the wall. We walk through how Paul Werbos worked it out in 1974, why nobody noticed for a decade, what Rumelhart, Hinton, and Williams did in 1986 that finally got the field’s attention, and the calculus trick that makes every deep neural network in the world actually trainable.
If Episode 2 was about the decade we lost, Episode 3 is about the algorithm that ended the winter.
If this series resonates with someone you work with, send it their way. The goal is that no one in research software engineering should have to keep faking their way through the foundations of a discipline they are now using every day. The point of walking the history Tuesday by Tuesday is that, by the time we get to the modern stack, the modern stack will not feel like magic. It will feel like the engineering it actually is.
Cordero is a senior research software engineer at the Scientific Software Engineering Center at the University of Washington’s eScience Institute. If this landed, forward it to the researcher in your department who keeps almost-getting around to learning where their tools came from. Tuesday at a time, that is exactly what we are doing.
