Ricardian Economics in the Data Center
The world’s most expensive computers spend most of the day waiting. A 200-year-old economics paper explains why, and what to do about it.
In 1817 David Ricardo published an argument that still catches people out. England and Portugal both made cloth and wine. Portugal made both more cheaply. The obvious conclusion was that Portugal should make everything.
Ricardo showed the opposite. Portugal’s edge in wine was larger than its edge in cloth, so every hour at the loom was an hour not spent at the vineyard. Both countries ended up richer by specializing and trading. What mattered was not who was better, but what each gave up. That is comparative advantage.
Two centuries later, comparative advantage is rearranging how AI models are served, and turning the data centers that serve them into something new: the inference factory.
This is an argument about inference, the serving of models. Training is a different case: one enormous synchronized job. Inference is where the work divides, and it is the larger prize. McKinsey’s model has it at 93 GW by 2030, against 62 GW for training.
The GPU is Portugal
Answering a prompt is two jobs, not one. First the model reads everything you have given it and builds a working memory of that input. Engineers call this prefill: one heavy burst of arithmetic. Then it writes the reply one token at a time, pulling that entire working memory back through the chip for every token. This is decode, a memory problem rather than a math problem.
So prefill is compute-bound and decode is memory-bound, and running both on the same device means each degrades the other. Utilization during generation routinely sits in the teens.
The GPU is Portugal. It is better than the alternatives at nearly everything. The question is never what a chip is good at. It is what it should specialize in, and what it should hand to someone else.
You do not put the master toolmaker on the stamping press. He would run it perfectly well, better than most. But while he is stamping, nobody is making tools, and the press does not care who is feeding it. Decode is the stamping press: one narrow operation repeated millions of times, rewarding raw throughput and indifferent to versatility. There is silicon built to do nothing else, and it does it faster per watt than a GPU can.
The gains from acting on this are not marginal, and they were measured on exactly this split. A 2024 systems paper found that refusing to let prefill and decode share a device served 7.4x more requests inside the same latency budget, on identical GPUs. Microsoft’s Splitwise went further and tried different silicon for each phase, and found that phase-split clusters could deliver up to 2.35x the throughput at the same cost and power, with the mixed-silicon designs among the most cost-efficient. Nvidia’s figures for pairing Rubin racks with its new decode silicon reach 30x the work per watt, though that stacks a hardware generation and a change in numerical precision on top of the split.
The analogy breaks in two places.
The first is reallocation. Ricardo’s countries could shift land from vines back to sheep, slowly, but they could. A data center reallocates in lumps, at the moments it adds or recycles capacity. Operators do choose their mix, but rarely on the merits: allocation, export licenses and delivery slots decide most of it, and a great deal of what is installed today was bought to train models rather than to serve them. Nor is the lock-in only physical. Silicon has five to seven years of useful life in it, but it needs at least four of those earning to return what was underwritten. Then a second clock runs against the first. The building has fifteen years or more, and was specified around whatever the silicon looked like on the day it was drawn. That one is the harder problem, and I will come back to it.
The second break is the one I spend most of my working week inside, and I think it is the more serious. Ricardo assumed producers trade on productivity. This market trades on creditworthiness. Buying accelerators and building the halls to hold them increasingly depends on vendor credit support: offtake commitments, residual value comfort, prepayments, equity, or simply a name a lender trusts behind the asset. Nvidia and AMD have balance sheets that can carry that weight, and they have used them to unlock capacity that would not otherwise have been built. AMD’s July agreement with Core Scientific, where it stands behind a neocloud tenant’s fifteen-year lease, is that pattern in a single deal. A specialist accelerator company, however good its silicon, cannot do the same. That asymmetry is nobody’s design, but it is real. What gets financed is shaped as much by who can stand behind an asset as by what the asset does best, and a chip can be the right answer for decode and still struggle to get funded. Comparative advantage has nothing to say about that, and it is a better reason to doubt the pace of this transition than any technical objection I have heard.
The market has noticed — specialization in action
Nvidia moved first and moved hard. On Christmas Eve 2025 it paid a reported $20 billion for Groq’s intellectual property and engineering team, its largest transaction ever. You do not spend that defending a position. You spend it because you have concluded that no single chip is the right answer to both halves of the job, and you intend to own both halves. The platform it now ships is five rack types, with the decode rack beside the GPU rack. Even the single market has districts.
AMD reached the same conclusion and took a different route to it. In July 2026 it paired its Helios racks with Cerebras wafer-scale engines: Helios ingests the prompt, Cerebras generates the tokens. Their own modeling puts the combination at up to 5x the tokens per second per watt of Cerebras systems running alone. Nvidia bought its decode engine, AMD partnered for one, and both were reading the workload correctly well before the market was.
Both are integrated products, though, and that is a real advantage: engineered together, with the co-design already done. The general version is harder. Dividing a live workload across whatever specialized silicon happens to be in the building, without handing the efficiency straight back to the network, is a different claim, with a larger prize because the specialists are more specialized, and worse problems in scheduling, compatibility and data movement. That is a systems problem rather than a compiler problem, and it is where Gimlet Labs is working, dispatching fragments of a live model to whichever silicon in the building suits them, across Nvidia, AMD, Intel, Arm, Cerebras and d-Matrix. Generating production kernels across that many architectures is hard work in its own right. The dispatching is harder still, and the part I find most interesting, because solving it is what makes the dispatcher nearly as strategically important as the chips it coordinates.
Gimlet’s published figures are 3 to 10x faster on frontier workloads, or 5 to 10x at a fixed power envelope. If Nvidia has built a single market and AMD has signed a bilateral treaty, Gimlet is proposing a trading bloc. These are less competing bets than different answers to how open the system should be, which is why Gimlet works with both.
The Suez Canal problem — the limits of comparative advantage
Here is the condition today’s infrastructure does not yet meet.
Ricardo’s argument only works if shipping is cheap. If moving the cloth costs more than specializing saves, the gains vanish. He treated transport as a footnote because in his model it was one.
History was less obliging. The great expansion of world trade waited fifty years, for the steamship, the telegraph and a ditch. Suez opened in 1869 and cut roughly four thousand miles off the voyage from London to Bombay. The economics had been sound the whole time. They were unprofitable until somebody dug the canal.
We are at that point now. Training had already put the fabric under strain, with thousands of accelerators synchronizing across a cluster many times a second. Specialized inference adds a different kind of load on top: moving a model’s working memory between machines mid-request, at short notice, in volume. Agentic workloads compound both. Networks were built to move files between servers. Add capacity, add a box. Add a box, add a hop. Add a hop, add delay.
Get Zachary Cefaratti’s stories in your inbox
Join Medium for free to get updates from this writer.
Gimlet built for this rather than discovering it late. Translating working memory between vendor formats costs tens of microseconds, which is trivial. Moving it between machines costs milliseconds, which is affordable only if the machines are close. Hence different chips in one building on one fast network, not a fleet in West Texas trading with a fleet in Abu Dhabi, or in East Texas for that matter. Get the geography wrong and the freight eats the gains. Software can make the handoff as cheap as the network beneath it permits, and no cheaper.
Digging the canal — capitalizing on specialization
Specialization makes the networking layer, a less glamorous corner of the data center, suddenly the most interesting one.
It is also the strangest line in the budget. After the accelerators, the fabric is the largest hardware spend in a modern AI build, and it is the only one of the big four that does not earn. Compute, memory and storage all sit inside what the customer is billed for. Nobody buys a hop. The network is the trade route between the money makers, and a trade route is pure cost until something moves along it.
It consumes power as well as capital, and in a large cluster the switches and optics take a real share of the megawatts on site. Every one of those is a megawatt not making tokens. Underbuild it, though, and you are worse off, because idle accelerators waste more than an oversized fabric ever could. You cannot economize your way out of that and you cannot spend your way out of it either. The only exit is a fabric that does more per box, per watt and per hop, and that raises the yield on everything it connects. Suez was the same shape of problem: the canal made nothing, and turned specialization on either side of it into a business.
In March 2026 a company called Eridu came out of stealth with more than $200 million to attack exactly this. Its founder, Drew Perkins, helped write one of the protocols the early internet ran on, then built and sold two optical networking companies, the second of which Nokia bought for $2.3 billion. His co-founder came from networking silicon at Broadcom and Marvell. John Doerr, Hudson River Trading, Capricorn and Matter Venture Partners are among the backers. TSMC is manufacturing the chips.
Their bet is that this cannot be fixed by tuning what exists. Eridu is designing switch silicon from a clean sheet, pulling functions onto the chip that today require whole racks of separate equipment and tiers of optical links between them. Fewer boxes, fewer hops, less delay.
Nvidia’s networking arm, Broadcom, Arista and Cisco are all pushing the same way, which is usually a sign the problem is real. The difference is the shape of the curve. Switch silicon has doubled roughly every two to three years for a generation, which is Moore’s law, and respectable for a mature architecture. Accelerators left that curve a decade ago, through new architectures rather than smaller transistors, and compute has since improved by multiples per generation rather than a doubling. Eridu is making the same break in networking. Its target is 10x switch capacity in two years, against an industry cadence of 2x every two to three years, and it gets there by rethinking the switch rather than shrinking it. That is the accelerator story replayed in the fabric, and it is why we backed the company at Matter. As models grow and their work is split into more pieces, demand for bandwidth inside the building rises faster than demand for compute. A doubling every few years does not close that gap. A step change of that size does. In shipping terms, it is the difference between a faster ship around the Cape of Good Hope and a canal.
Rebuilding the ports — what specialization does to the building
The canal was not the only thing that had to change. Containerization arrived in 1956 and needed far more than ships. It needed ports rebuilt: deeper berths, gantry cranes, marshalling yards, rail links. The ports that adapted, Rotterdam among them, became giants. The ones that could not, like the London docks, were derelict inside twenty years. The cargo changed composition and the buildings could not follow.
Agentic workloads make all of this more urgent, not less. An agent does not send one prompt and wait. It reasons in steps, calls tools, retrieves, re-reads its own output and loops, which means a single task now spans several kinds of computation. Demand rises and its composition fragments at the same time. A workload that was already two jobs is becoming five or six, and each of them has a different appetite. That is the case for specialization stated twice over, and it is why the buildings that serve inference have to change rather than merely expand.
Almost all AI capacity today is built on an assumption of uniformity. Identical racks, identical density, identical cooling, replicated across a hall. That is the correct design for a monolithic training fleet and the wrong one for a mixed inference estate. Heterogeneous inference wants zones rather than repetition: liquid-cooled racks at densities that differ by multiples, wafer-scale systems with floor loading a conventional hall was never engineered to carry, SRAM-heavy decode racks whose thermal profile looks nothing like a GPU rack’s, all close enough to trade with each other and all refreshing on different clocks.
The obvious objection is that the vendors will fold all of this inside the rack and the building will look uniform again. Nvidia’s NVLink Fusion is built to admit outside silicon into its scale-up domain, and d-Matrix signed up in recent weeks. I think that draws the boundary in the wrong place. Nvidia’s own reference factory ships as five rack types, and its Groq decode rack sits beside the Rubin rack, not inside it. Its next scale-up domain is designed to span eight racks. Whoever packages the mix, a decode rack and a prefill rack still have different densities, different thermal profiles and different refresh cadences, and the fabric between them is still where the yield is won or lost. Packaging changes who draws the zones. It does not remove them.
Which brings me back to the second clock, and the question I find hardest to answer when I put these buildings in front of lenders: how do you underwrite a fifteen-year building when the composition of what goes inside it turns over at least twice, in a direction nobody can specify at financial close?
What a building designed for that looks like is a larger question than heterogeneity alone, since a training factory can be monolithic and still be nothing like a cloud hall, and I take it up in a companion piece. The short version is that an AI factory, of either kind, is not a data center with GPUs in it, and the developers who have understood this, DataOne among them, started from first principles rather than adapting what came before. DataOne specified for 60 to 1,000 kW per rack and for heterogeneous infrastructure from the outset, at a time when very few were planning for either.
My concern is for everyone adapting legacy designs instead. A big hall specified for one chip at one density, financed over fifteen years and filled in year one, is an unhedged bet that neither will change. Stranded capital here will not look like empty buildings. It will look like full ones that cannot take the next generation, and then need capital and downtime to reconfigure, or else lock the tenant into hardware that is wrong for the workload. A good deal of capacity is being built with that risk carried rather than priced.
What I think this means
Two things, pointing opposite ways.
You cannot retrofit your way into this. A hall already racked, cooled and cabled for one chip at one density will not become heterogeneous because someone read Ricardo. Plenty of operators do hold mixed fleets already, mostly by accident of procurement, but scattered across sites where they were never going to trade with each other. I am in the middle of exactly this question with more than one of them right now, and the honest starting point is always the same: the chips are fine, they are just in the wrong buildings.
What is actually available is the next increment. Most operators will specify it by repetition: another Cerebras build beside the Cerebras build, more AMD racks beside the AMD racks, because procurement, support and operational familiarity all pull that way. The alternative is to treat an existing monolithic estate as one half of a pair and build the next hall as the complement rather than the copy, whether that is another vendor’s silicon or the same vendor’s decode racks, with a fabric and an orchestration layer between them. The same capital completes something instead of duplicating it, and makes each megawatt count. On greenfield it is easier still, because nothing has been fixed yet.
The threat is that the price falls whether or not you participate. Output that cost around $30 per million words in early 2023 now costs well under a dollar. Whoever captures the efficiency sets the new price, and everyone else sells raw compute hours into a market that has repriced beneath them.
I have spent the last two years helping former bitcoin mining sites find AI tenants, so I watched the opposite outcome from close range. Mining is one computation repeated endlessly. No second phase with a different appetite, nothing to trade. So the industry did not specialize, it consolidated: CPUs to GPUs to FPGAs to ASICs, until nearly everyone ran the same chip doing the same thing and competed on power price and access to the newest generation. Margins went where you would expect, which is why those sites are now looking for AI tenants. Total consolidation is what happens when a workload has one profile. Training, for now, has one, which is why training fleets are monolithic and should be. Inference has several. If it consolidates anyway, it will mean I read the workload wrong. If training starts to divide too, and reinforcement learning already looks a lot like inference, the argument widens. I would not underwrite that yet.
The strongest objection I get to all of this is that it proves too much. If output per watt rises several times over, does the industry not simply flood itself with tokens nobody has priced for, and destroy the economics it set out to improve? Somewhere out there, the argument goes, are the edges of Jevons paradox.
I do not think we find them, for two reasons. The first is pace. This shift arrives at the speed of buildings and credit, not software. Capacity has to be specified, financed, permitted, energized and filled, and the bankability constraint described earlier throttles both how quickly it comes and what goes into it. Supply that arrives over five years gets absorbed; supply that arrives in five months may not. The second is that demand has swallowed every efficiency gain so far. Prices have collapsed since 2023 and aggregate spending has risen, because each drop makes a new class of application economic. Agents are the clearest case: a single agentic task consumes an order of magnitude more tokens than a chat exchange, and agents only become affordable as the price per token falls. Cheaper tokens do not shrink this market. They move the frontier of what is worth building.
The inference factory of the next decade will not be a hall of identical GPUs. It will be GPUs, CPUs and purpose-built accelerators, each running the work where it holds the advantage, connected by a network fast enough to make the trade worth making.
Ricardo would have recognized the argument. He would also have asked who is digging the canal.
Disclosure: Eridu, DataOne and Core Scientific are clients of Dalma Capital. Matter Venture Partners, where I am an Operating Partner, is an investor in Eridu. Dalma Capital engages in commercial discussions and activities with other companies mentioned here.
