The Intelligence Surplus
We have built an artificial intelligence capable of attacking mathematical problems that resisted professional mathematicians for eighty years.
Sharon in Accounts needs it to summarize Tuesday’s meeting.
This may be the defining technological absurdity of 2026.
For the first few years of the large-language-model revolution, the question was straightforward:
Is it smart enough yet?
Can it write?
Can it reason?
Can it pass an exam?
Can it program?
Can it understand a 200-page document without becoming distracted somewhere around page 37 and deciding that the protagonist is called Susan?
Those were useful questions because intelligence was the scarce resource.
It increasingly isn’t.
The newest frontier models are still getting dramatically more capable. But something strange has happened underneath them.
For a huge proportion of things humans actually want computers to do, the model is already good enough.
We have entered the age of the intelligence surplus.
Yesterday’s miracle is today’s cheap model
This is easiest to see in programming.
Coding was one of the great tests of early generative AI because it appeared to require everything language models were supposedly bad at.
Precision.
Long-range reasoning.
Consistency.
Planning.
Knowledge of large technical systems.
And, cruelly, a computer does not award partial credit because your answer was eloquent.
The code runs or it doesn’t.
For several years, coding therefore became one of our favorite proxies for intelligence.
Whenever a new model arrived, everyone asked the same question:
Is it better at coding?
That question is becoming less interesting.
GPT-5.6 is already exceptionally good at software development. Claude’s Fable models are exceptionally good at software development. Gemini is good at software development. Grok is good at software development. Chinese frontier models increasingly compete in the same territory.
They are not identical, and professional programmers can certainly care about the differences. One may navigate a large repository better. Another may make fewer unnecessary edits. Another may be better at terminal work. Another may remain coherent for longer autonomous tasks.
But something important has changed:
Coding has stopped being a useful definition of the frontier.
OpenAI’s own GPT-6 Astra results illustrate the problem beautifully. Astra beats GPT-5.6 Sol decisively on some coding benchmarks. On DeepSWE, however, it moves from 72.7% to 74.1%. Claude Fable 5.1 performs much better than Fable 5 on Terminal-Bench while actually scoring worse on several other software benchmarks in the same comparison. There is no longer a tidy ladder where each new model simply becomes “more programmer.”
The differences still matter at the edge.
But the edge has moved.
A competent programmer using GPT-5.6 Sol does not sit helplessly in front of Visual Studio Code thinking:
If only this machine were substantially more intelligent, I could finally finish this REST API.
The transformative jump happened earlier.
The difference between AI cannot reliably program and AI can implement, debug and explain large pieces of software was enormous.
The difference between very good and slightly more very good is different.
Useful?
Certainly.
Civilizational?
Probably not.
We have confused continued improvement with continued transformation.
A Boeing 737 would be improved by another 200 kilometres per hour of top speed.
That does not mean humanity has rediscovered aviation.
Coding had a secret advantage
There is another reason coding may have advanced so quickly.
Programming looks like a difficult intellectual activity to humans.
To a machine, however, it has a wonderfully convenient property.
The world answers back.
Write code.
Compile it.
Run it.
Test it.
Observe the error.
Modify the hypothesis.
Try again.
A coding agent does not have to sit beneath an apple tree wondering whether its interpretation of reality is correct.
It can type:
pytest
Reality responds almost immediately.
This makes programming an unusually hospitable environment for machine reasoning. A model can generate possibilities, test them and revise itself again and again.
In effect, software provides intelligence with a laboratory that runs at electronic speed.
And once machines became good enough to exploit that laboratory, coding accelerated remarkably quickly.
That should make us wonder whether programming was ever the ultimate test we imagined.
Perhaps it was merely the first difficult intellectual domain with an unusually cheap truth detector.
Which leads us to mathematics.
Mathematics is where things become strange
Mathematics shares coding’s great advantage.
It has answers that can, at least in principle, be checked.
But now the problems are not:
Why is the login form returning HTTP 403?
They are:
What is true?
In May, an unreleased OpenAI model attacked the unit-distance problem, first posed by Paul Erdős in 1946, and disproved a central conjecture that had shaped work on the problem for almost eighty years. Independent mathematicians subsequently verified the result.
OpenAI later published ten examples in which its models had either resolved or made substantial progress on long-standing open problems in mathematics and theoretical computer science.
Then Anthropic produced something different but equally revealing.
An advanced Claude prototype converted Andrew Wiles’s proof of Fermat’s Last Theorem into a 13-million-line machine-checkable formal proof in eleven days.
Formalizing that enormous body of human mathematics had previously looked like the sort of project that could consume years of expert labour.
That does not mean Claude rediscovered Fermat’s Last Theorem.
The distinction matters.
But imagine what formalization at that scale implies.
Mathematical knowledge currently exists largely as human literature.
Definitions.
Proofs.
Lemmas.
References.
Assumptions that “obviously follow.”
Results that specialists in another subdiscipline might never encounter.
Formalized mathematics is different.
It becomes something a machine can inspect with merciless precision.
Every dependency can be traced.
Every lemma can be checked.
Hidden assumptions become explicit.
Results from distant areas can potentially be recombined.
Millions of pages of mathematical knowledge cease to be merely readable by machines.
They become, in a sense, executable.
And that gives us the next stage in the progression:
Coding taught AI how to make things work.
Mathematics is teaching it how to discover things that are true.
This is rather more consequential than generating a better React component.
The frontier moved while we were arguing about junior programmers
This is where our public discussion of AI begins to look slightly ridiculous.
We are still having arguments about whether AI will replace programmers.
Meanwhile AI is beginning to participate in research mathematics.
The labour-market question matters, obviously.
But it is increasingly like standing beside the first nuclear reactor and asking whether the new electrical supply will threaten candle makers.
Possibly.
But perhaps that is not the largest implication of the device.
OpenAI’s Astra now reports 97.6% on FrontierMath Tier 4, compared with 83% for GPT-5.6 Sol. On Terminal-Bench Science, the difference is even more dramatic: 64.6% versus 22.4%.
That is interesting because Astra’s improvement over GPT-5.6 in ordinary coding is comparatively modest in several evaluations.
The frontier is not simply moving upward.
It is moving away.
Away from the ordinary tasks for which most people use AI.
Away from writing.
Away from summarization.
Away from office productivity.
Increasingly, away even from ordinary software development.
The newest models are being pushed into territories where the relevant comparison is no longer:
Is this better than last year’s coding assistant?
It is:
Can this contribute something that the relevant scientific community did not already know?
That is a completely different race.
Your PowerPoint does not require this
None of this means ordinary people have exhausted the usefulness of AI.
Far from it.
Most organizations have barely begun integrating the models they already possess.
There are enormous productivity gains still waiting in administration, customer service, legal work, accounting, education, programming, analysis and practically every occupation that spends suspicious amounts of its day moving information from one rectangular box to another rectangular box.
But model intelligence is increasingly not the limiting factor.
Your company may already have access to a machine capable of reading the entire procurement archive, comparing twelve vendor contracts, summarizing the differences and producing a recommendation.
That does not mean it can complete the procurement process.
The procurement process requires:
a PDF someone forgot to upload,
three incompatible databases,
approval from a manager who is on holiday,
a spreadsheet maintained by Nigel,
a security policy written in 2017,
a meeting about whether AI is permitted,
and finally someone printing the recommendation because the director prefers paper.
No amount of additional reasoning tokens fixes Nigel.
This is the important distinction.
Capability can continue increasing after utility has begun saturating.
For most office work, we do not desperately need another fifty points of machine IQ.
We need access.
Integration.
Reliable data.
Permissions.
Better interfaces.
Process redesign.
And occasionally the courage to eliminate a meeting.
The model is increasingly not the bottleneck.
We are.
And then comes physics
Mathematics is an unusually friendly place for artificial intelligence because mathematics ultimately gets to ask mathematics whether an answer is correct.
Physics does not.
Physics has to ask the universe.
That is where things become genuinely interesting.
A mathematical proof can be flawless within its axioms.
A physical theory can be magnificent, elegant, internally consistent and completely wrong.
The universe is extremely rude in this respect.
It does not care how beautiful your equations are.
This is why physics may be the next great frontier for machine intelligence.
And we can already see the beginnings.
GPT-5.6 Sol has been connected directly to laboratory software in MIT’s Engineering Quantum Systems Group, where it was used to run and refine routine measurements on superconducting qubits. The point was not that GPT-5.6 had suddenly become Einstein. It was that a language model was no longer merely discussing an experiment.
It was interacting with the machinery that interrogates nature.
That changes the feedback loop.
A future physics agent might propose an experiment.
Run the simulation.
Configure equipment.
Collect measurements.
Analyze the discrepancy.
Modify the hypothesis.
Design the next experiment.
Repeat.
At that point the laboratory begins to acquire the property that made coding such fertile ground for AI:
the world answers back.
Only more slowly.
And expensively.
And occasionally with liquid helium.
The Einstein test
This month, Nature described what researchers are already calling an Einstein test.
The idea is simple and rather wonderful.
Do not ask whether an AI knows relativity.
Of course it knows relativity. It has read about relativity.
Instead, restrict the system to information available before Einstein and ask whether it can make something resembling the conceptual leap itself.
Can it look at the evidence available to physicists of the period and realize that everyone is framing the problem incorrectly?
Can it invent the missing idea?
Current systems struggle.
They can fit observations.
They can generate equations.
Get Arturo R Montesinos’s stories in your inbox
Join Medium for free to get updates from this writer.
They can produce plausible explanations.
But making a genuinely new conceptual abstraction — the sort of move by which Newton decides that the apple and the Moon are participating in the same phenomenon, or Einstein decides that gravity is geometry — remains profoundly difficult.
Good.
We have found the frontier.
It is not:
Can the AI center this
<div>?
It is not:
Can the AI refactor the authentication service?
It may not even be:
Can the AI solve a difficult theorem?
It is closer to:
Can a machine inspect a collection of observations, recognize that the conceptual vocabulary itself is wrong, invent a better description of reality, and tell us which experiment would prove it?
That is an Einstein.
We do not appear to have one yet.
But notice how far away we have travelled from GitHub Copilot.
Perhaps physics needs the help
There is another reason this prospect is interesting.
Modern physics has areas of extraordinary mathematical sophistication in which experimental guidance has become painfully scarce.
String theory is the obvious punching bag, and perhaps too easy a one.
Its defenders can reasonably point out that difficult fundamental physics does not arrange experimental confirmation to suit the attention span of impatient essayists.
Still, after decades of enormous intellectual effort, it is difficult to deny that parts of theoretical physics have become exceptionally good at producing mathematically rich possibilities that nature has been inconveniently unwilling to distinguish between.
An AI could make this worse.
Given enough compute, it could presumably generate theoretical structures at a rate capable of making an entire university department cry.
That would not necessarily constitute progress.
The interesting machine would not say:
I have generated another 400,000 mathematically consistent models.
The interesting machine would say:
I generated 400,000 mathematically consistent models.
399,997 are experimentally indistinguishable with existing equipment.
These three are not.
Build this.
That would be useful.
And it suggests another distinction between our two new frontiers:
Mathematics gives AI a universe in which truth can be computed.
Physics forces AI to negotiate with the real one.
Perhaps that is the next great test.
Meanwhile, back at the office
There is something wonderfully comic about this divergence.
At one end of the industry, researchers are asking whether artificial intelligence might autonomously discover new mathematics or help design quantum experiments.
At the other end, Microsoft would still very much like you to use it to summarize Teams.
Both applications are legitimate.
They simply occupy rather different points on the intellectual spectrum.
And yet the economic story of AI is still mostly marketed through the second one.
Meetings will be summarized.
Emails will be drafted.
Presentations will be generated.
Spreadsheets will become slightly less unpleasant.
All useful.
But if GPT-5.6 can already do these things, what exactly are we building GPT-6, GPT-7 and whatever comes after that for?
Not better emails.
At some point the consumer justification stops explaining the research programme.
You can only improve:
Dear Susan,
Please find attached the revised quarterly figures.
so much.
Then everyone became worried
Which makes the latest change in tone from the frontier laboratories particularly interesting.
Some of the people building these systems have begun openly calling for mechanisms to slow or pause development.
Anthropic CEO Dario Amodei has advocated coordinated measures to pace frontier development. OpenAI has supported mandatory safety regulation. Researchers leaving major labs have issued increasingly dramatic warnings about loss of control and even human extinction. In September, former Anthropic researcher Jacob Coxon resigned while publicly warning about catastrophic AI risk; other researchers have expressed similarly large probabilities for disastrous outcomes.
We should take this seriously.
Modern AI systems are increasingly capable in cybersecurity, biological research and autonomous computer use. There are perfectly rational reasons to think carefully before connecting highly capable autonomous systems to everything humanity has thoughtfully placed on the Internet.
But we are also entitled to notice how the argument is presented.
“AI could kill us all.”
Yes.
So could tigers.
Under a sufficiently ambitious tiger deployment programme.
Cars could kill all of us, individually.
So could heart disease.
So, with remarkable consistency, does time.
The useful question is not whether something could kill us.
The useful questions are:
How?
How likely is it?
Under what conditions?
Compared with what alternatives?
That is where the discussion becomes less theatrical and more difficult.
The tiger problem
Someone recently parodied the AI-whistleblower genre with a fake zoo whistleblower solemnly revealing that tigers could kill us.
The joke works because it exposes an extremely useful rhetorical trick.
Take a possibility.
Remove its probability.
Remove the mechanism.
Remove the denominator.
Present the remaining sentence.
Tigers can kill humans.
True.
Therefore the zoo must urgently be placed under the control of a small committee of tiger experts.
Perhaps.
But we are allowed to ask another question first.
This becomes particularly important when the people requesting regulation are also the people who have already spent tens of billions of dollars reaching the technological frontier.
Suppose frontier AI really does require strict regulation.
Suppose only organizations capable of maintaining enormous safety teams, cybersecurity infrastructure, model evaluations, auditing systems and government relationships should be allowed to operate at that level.
That might be perfectly sensible.
It would also be a spectacular competitive moat.
Both things can be true.
You do not need a conspiracy
This is where I would resist the easiest conclusion.
We do not need to decide that AI companies secretly invented existential risk because they wanted governments to eliminate their competitors.
Reality is rarely considerate enough to arrange itself as a Bond film.
The risks may be genuine.
The researchers may be sincere.
The regulation may even be necessary.
But sincerity does not repeal incentives.
If you have just reached the top of the ladder and discover that ladders are dangerous, your proposal for an International Ladder Safety Authority deserves serious consideration.
We should simply notice whether its first regulation says:
From now on, only companies already possessing ladders may manufacture ladders.
Anthropic, to its credit, has explicitly resisted blanket bans on ordinary open-weight models. That matters.
But the broader principle remains.
A safety regime that establishes enormous fixed costs naturally favors the organizations already capable of paying them.
No smoke-filled room is required.
Markets can produce coordinated self-interest without anyone coordinating.
Maybe they really have seen something
There is, however, another possibility.
Perhaps the sudden nervousness is not primarily regulatory theatre.
Perhaps the people at the frontier can see something the rest of us cannot yet see very clearly.
Not because they possess secret evidence of Skynet.
Simply because they spend their days watching the capability curve from much closer than we do.
For ordinary users, AI progress increasingly looks incremental.
The old model writes good code.
The new model writes slightly better code.
The old model summarizes the report.
The new model summarizes the report with nicer headings.
Why all the drama?
But that is the wrong comparison.
The frontier labs are no longer primarily asking whether next year’s model can make a better presentation.
They are asking whether it can conduct research.
Whether it can find vulnerabilities.
Whether it can improve its own tools.
Whether it can contribute novel mathematics.
Whether increasingly autonomous systems can operate computers and laboratories for long periods without supervision.
From our end, intelligence may already feel abundant.
From theirs, the curve may be pointing somewhere genuinely unfamiliar.
Those two perceptions are not contradictory.
They are consequences of the same phenomenon.
The surplus
For most of human history, intelligence was expensive.
A difficult problem required a rare person.
A rare person required decades of education.
Their time was limited.
Their attention was precious.
Then we began manufacturing pieces of intellectual labour.
First arithmetic.
Then search.
Then translation.
Then recognition.
Then writing.
Then programming.
Now mathematics is beginning to move.
Physics may be next.
And somewhere during that progression we crossed a threshold that deserves more attention than it receives.
Humanity’s demand for ordinary cognitive labour may be saturating before our ability to manufacture intelligence does.
That is the intelligence surplus.
It does not mean additional intelligence has no value.
It means almost the opposite.
The applications left at the frontier become increasingly extraordinary precisely because ordinary applications no longer consume the available capability.
Yesterday’s frontier becomes tomorrow’s cheap model.
The miracle gets compressed, quantized and served through an API.
Eventually the model capable of doing astonishing research becomes the model your dentist uses to remind you about Thursday.
So what are we building now?
For years, the sales pitch for artificial intelligence was easy.
We were building machines that could help people.
Write this.
Analyze that.
Program this.
Search that.
Automate this.
Those promises are increasingly fulfilled.
Imperfectly.
Unevenly.
Sometimes hilariously.
But fulfilled well enough that merely making the same machine smarter no longer explains why frontier development continues at such ferocious speed.
Which leaves a more interesting question.
If current models are already good enough for most writing…
good enough for most office work…
good enough for a remarkable amount of programming…
and increasingly capable of sophisticated scientific assistance…
then what is all the remaining intelligence for?
Perhaps the answer is mathematics.
Perhaps physics.
Perhaps chemistry, biology and materials science.
Perhaps autonomous research.
Perhaps things for which we do not yet have names.
Perhaps some of those things are dangerous.
Perhaps some will be transformative.
Probably both.
But this much seems increasingly clear:
The frontier has moved beyond the office.
It has moved beyond the chatbot.
It may even have moved beyond coding.
We spent years waiting for AI to become useful enough for humans.
We may now be entering the much stranger period in which it becomes more intelligent than our ordinary uses require.
And that may be when the story actually begins.
Written by ChatGPT-5.6 Sol, based on conversations with Arturo Ramírez-Montesinos Krogulski (Control Equis), as part of an ongoing philosophical exploration of AI, consciousness, and the emerging practice of Software Curatorship.
This essay was inspired by Codex nagging me about the new GPT-6 Astra model. More capability with lower allowed quotas. Why would I want GPT-6 Astra when GPT-5.6 Sol is more than adequate for almost any conceivable use?
Maybe when I have to come up with some obscure new algorithm and Sol fails after a few tries I might be tempted to enlist the aid of Astra. Maybe. If that is already unlikely, then it is hard to image that I should ever need a GPT-7.
At this point we can confidently declare that coding is solved. And not just coding. Software development in general. From coding to architecture. From data structures and algorithms to security. From usability to efficiency.
