The Wrong Optimization

There is a recurring argument in enterprise AI right now: most business and government workloads do not require enormous foundation models. They require smaller, cheaper, more efficient domain-specific models, ideally running privately on infrastructure the organization controls.

There is some truth in that argument, especially around the private infrastructure.

Private and sovereign deployment is a legitimate need—and in some environments, essential. For governments, regulated industries, defense, finance, healthcare, and organizations handling proprietary data, control over infrastructure, data, and inference can be strategically important. And for a narrowly bounded task, a small specialized model may indeed be cheaper and faster than a much larger general-purpose model.

But this leads to the wrong long-term conclusion.

The problem is not the engineering logic. The problem is the time horizon: it optimizes around a resource constraint that is disappearing extraordinarily quickly.

In computing, what counts as “too large,” “too slow,” or “too expensive” is rarely fixed. It is a temporary relationship between an application and the hardware available to execute it. We have watched that relationship collapse repeatedly with memory, storage, CPU cycles, network bandwidth, and cloud compute. Entire generations of engineering practice were built around conserving resources that later became cheap enough to treat almost casually.

AI inference appears to be following the same trajectory.

A recent analysis from Epoch AI estimates that since 2023, the cost of achieving a fixed level of AI performance across several mathematics, science, and reasoning benchmarks has fallen by roughly 47 percent per quarter—about thirteenfold per year on average. The authors appropriately caution against extrapolating that rate indefinitely, but the magnitude of the trend is difficult to ignore. (Epoch AI)

Hardware is moving in the same direction. NVIDIA reports substantial reductions in inference cost across successive GPU generations, with improvements coming from both hardware and the surrounding inference software stack. (NVIDIA)

And increasingly, big does not even mean computationally big. Pruning, quantization, sparsity, and sparse-activation architectures are separating nominal model size from the resources required to run it. SparseGPT, for example, showed that very large language models can tolerate substantial weight pruning with surprisingly little degradation on evaluated tasks, while mixture-of-experts architectures activate only a fraction of total model capacity for a given token. (SparseGPT)

What counts as an “unacceptably large model” is therefore moving from both directions: hardware is getting better while models themselves are getting cheaper to execute.

The architectural implication is clear. If a large general-purpose model costs twenty times as much as a specialized model, efficiency may dominate the decision. If that ratio falls to two times, the calculation changes. If it approaches parity, the question changes completely.

The comparison is no longer merely cheap versus expensive. It becomes cheap, narrow capability versus slightly more expensive general capability with far greater flexibility, reuse, and optionality.

And the latter may be economically superior even when it remains computationally less efficient.

Computational efficiency and economic efficiency are not the same thing.

A specialized model has costs beyond inference: training data, development, evaluation, deployment, retraining, monitoring, governance, integration, and maintenance. A foundation model already deployed within a common architecture may use more compute for a particular task while still producing a lower total cost of ownership.

We have seen this pattern before—over and over.

Before the smartphone, we built specialized devices for specialized functions: calculators, GPS receivers, MP3 players, digital cameras, PDAs, handheld organizers, and mobile telephones. The smartphone did not replace them because it was the theoretically optimal implementation of every function. It replaced them because the marginal cost of carrying excess general-purpose capability became negligible.

A smartphone is absurdly overpowered if all you want to do is calculate a tip. Yet almost no one argues that we should carry a dedicated calculator because it is computationally more efficient.

The excess capability became economically irrelevant.

That is the lesson here. The argument is not that a massive foundation model is necessary for every bounded task. It is that as the marginal cost of excess general-purpose capability approaches insignificance, there is progressively less reason to remove that capability in the first place.

What looks wasteful when compute is scarce becomes valuable optionality when compute is abundant.

The historical pattern is remarkably consistent: general-purpose capability begins scarce and expensive, specialization emerges to work around that scarcity, the general platform becomes cheap and abundant, and specialization moves upward into software and applications.

We are watching that happen again with machine intelligence.

This does not mean model efficiency becomes irrelevant. Some workloads will always demand extreme efficiency because of latency, power, throughput, hardware constraints, or transaction volume. But that is different from saying the future belongs to increasingly specialized domain models.

The strategic question is whether model efficiency will remain important enough to justify putting the specialization—and therefore much of the intellectual property—inside the model itself.

I would strongly bet against it.

Specialization is moving somewhere else.

Foundation Models Were Always Supposed to Be Just That— Foundations

The term foundation model is often used casually (and incorrectly) as a synonym for “very large model,” especially in popular discussion of LLMs.

That misses the point.

When Stanford researchers introduced the term in 2021, they described models trained broadly, generally at scale, that could be adapted to many downstream tasks. The important word was foundation: the model was not the finished product. It was the common base upon which other systems would be built.

Traditional machine learning encouraged us to think in terms of task-specific models. A financial problem suggested a financial model. A legal problem suggested a legal model. A medical problem suggested a medical model.

In that architecture, specialization lived primarily inside the model.

Foundation models change where specialization can live.

The emerging architecture is closer to: Foundation model + domain context + tools + policy + orchestration = specialized system

The specialization no longer has to be encoded entirely in model weights. It can be introduced at runtime through the information, tools, constraints, and environment supplied to a broadly capable model.

Fine-tuning remains one way to specialize a foundation model, but it is no longer synonymous with specialization. Retrieval, structured context, tool access, domain ontologies, memory, system instructions, workflow rules, and validation layers can all specialize behavior while leaving the underlying model untouched.

Instead of building a dedicated financial model—or fine-tuning a foundation model to behave like one—we can increasingly place a general model inside a specialized financial system.

A recent paper, Context Is King: How In-Context Specification Shapes the Geometry of Concepts, provides an intriguing piece of evidence for this direction. David and Fomin showed that capable language models can construct and use conceptual structures specified in context even when those structures conflict with strong relationships learned during pretraining. Larger models in the families they studied did this more cleanly than smaller ones.

That does not prove that domain-specific training is unnecessary or that context can replace fine-tuning in every application. But it demonstrates something more fundamental: the behavior and internal representations used by a sufficiently capable model can be substantially determined by its operating context rather than solely by what was fixed into its parameters during training.

That means specialization can increasingly become a runtime property of the system rather than a training-time property of the model.

And runtime specialization is easier to change.

• If regulations change, change the policy layer.
• If financial data changes, change the retrieved context.
• If a different database becomes authoritative, change the tool connection.
• If business rules change, change the orchestration.
• If a better foundation model becomes available, you can often replace the model without rebuilding the entire system above it.

Compare that with specialization embedded deeply in model weights, where change may require new training data, retraining, reevaluation, redeployment, version management, and another governance cycle.

Fine-tuning will not disappear. There will remain workloads where adjusting model weights is useful or necessary. But it is becoming one specialization mechanism among many rather than the defining one.

What has changed most rapidly is the sophistication of the layers above the model. Retrieval has improved. Tool use has improved. Context windows have expanded. Agentic orchestration has become practical. Structured outputs, policy enforcement, memory, verification, and routing are becoming ordinary architectural components.

More and more of what makes an AI system specialized now exists outside the model. This is why foundation model should be taken literally. A foundation is not the building. It is the part you build everything else on.

The model provides capability. The system provides specialization.

The Model Is Becoming Infrastructure

If specialization increasingly lives in the system, then the strategic value of the model itself changes.

The model still matters technically. A weak model constrains the system around it, and models differ materially in reasoning, latency, context handling, tool use, modality support, reliability, and cost.

But technical importance is not the same thing as durable differentiation. Databases, CPUs, cloud platforms, and operating systems are sophisticated and indispensable. Yet most application companies do not create strategic advantage by building their own database engine, processor, or operating system.

They build on top of them.

Foundation models are moving in the same direction.

As capable models become cheaper, easier to deploy, more interchangeable, and increasingly abstracted behind common interfaces, the competitive question shifts from “Which model do we own?” to “What have we built around it?”

That suggests a simple design principle:

Build your moat in the layers that survive when you replace the model.

That is where proprietary value increasingly lives: data, retrieval, tools, permissions, workflows, policy, verification, evaluation, orchestration, human escalation, and the accumulated knowledge of how an organization actually operates.

The underlying model may change.

The system should survive.

Model-routing architectures already point in this direction. Microsoft Foundry, for example, can abstract model selection into a routing layer, choosing among models based on factors such as capability, latency, and cost rather than embedding a single model permanently into application logic. (Microsoft)

That is more important than simply saying “use the largest model available.” The application talks to an intelligence layer. The intelligence layer determines what should execute the task. A simple extraction may go to a small model. Routine reasoning may go to a mid-sized model. Complex analysis may go to the strongest available foundation model. A tightly bounded workload may still use a specialized model.

The application does not need to care, and that is what commoditization looks like. The underlying technology does not become unimportant.

It becomes infrastructure.

Nor does this mean small or specialized models disappear. There will always be workloads where latency, power, throughput, privacy, hardware constraints, or sheer transaction volume make a narrow model the correct engineering choice. If a five-million-parameter classifier perfectly handles a billion repetitive events per day, replacing it with a massive reasoning model would be wasteful.

The more interesting question is what happens to the middle.

On one side are tiny, highly efficient models solving tightly bounded machine tasks. On the other are increasingly capable foundation models operating inside specialized systems. The category that may become hardest to justify is the expensive, purpose-built domain model whose primary advantage is that it knows one field better than the general model. That advantage becomes less compelling as general models improve, inference costs fall, and domain specialization moves into context, tools, and orchestration.

The market begins to bifurcate:

• bounded machine task → small specialized model
• open-ended cognitive task → general foundation model + specialized system

The middle gets squeezed from both directions, and this is why the larger change is not really about model size.

It is about the level of abstraction at which AI engineering is being done.

Traditional machine learning asked, “What model should we train?” The first generation of foundation-model engineering asked, “Which model should we fine-tune?”

The emerging question is broader: “What combination of models, context, data, tools, policies, verification, workflows, and humans should solve this problem?”

That is a different discipline.

It is less about model engineering and more about intelligence systems engineering because systems must outlive models. Build the system around the assumptions and peculiarities of one model, and the model becomes part of your technical debt. Build specialization above an interchangeable foundation layer, and the system can evolve as the models underneath it improve.

The goal is not merely to use a foundation model.

The goal is to architect the model as a replaceable component.

The best model available today will not be the best one available next year. Model families will change. Architectures will change. Hardware will change. Pricing will change. Some models will disappear entirely. A well-designed AI system should absorb those changes rather than be invalidated by them.

Models will continue to matter. Some will be larger, some smaller, some specialized, and some general. Engineers will continue optimizing all of them.

But increasingly, that is infrastructure work.

The strategic value is moving upward.

The durable question is no longer, “What model should we build?”

It is:

What system can we build when intelligence itself becomes a commodity?

The AI Model Is Not the Product was originally published in IQumulus on Medium, where people are continuing the conversation by highlighting and responding to this story.

Sovereign Over What?

Sovereign AI is becoming one of those terms that everyone seems to understand until they are asked to define it.

The concept did not begin as shorthand for “run an LLM on your own server.” Its roots are broader: technological sovereignty, data sovereignty, national infrastructure, jurisdiction, strategic autonomy, and the ability to retain meaningful control over increasingly critical computational systems.

In 2023, Luca Belli described AI sovereignty as a country’s capacity to understand, develop, and mobilize AI while retaining control, agency, and self-determination over those systems. More recent work from Stanford’s Institute for Human-Centered AI extends the concept beyond governments, defining it in terms of a state or enterprise acting deliberately according to its strategic interests across the development, deployment, and governance of AI. Stanford also makes an important qualification: the term remains poorly defined, with interpretations ranging from near-complete technological self-sufficiency to strategic interdependence.

That ambiguity is important because an increasingly common interpretation reduces sovereignty to deployment topology:

On-premises is sovereign. Cloud is not.

That is not a useful engineering definition, nor is it correct.

The European Commission’s Cloud Sovereignty Framework makes the same broader point from another direction. It evaluates strategic control, legal and jurisdictional exposure, data and AI, operations, supply chain, technological autonomy, security, and compliance—not simply where a server is located.

The emerging pattern is clear: sovereignty concerns control and dependency across a system, not merely ownership or location of components.

That distinction is especially significant in AI because the model is only one component. A real AI system may include models, prompts, retrieved data, policies, identity systems, databases, tools, execution environments, logs, orchestration, external APIs, human approvals, hardware, networking, and update mechanisms. Hosting the model yourself tells me something important about one component. It tells me very little about the rest.

A locally hosted model with unrestricted filesystem access, uncontrolled tool execution, poor credential isolation, externally transmitted logs, and no meaningful audit trail is not made sovereign by the location of its weights.

Conversely, an external dependency does not automatically prove the absence of control. The relevant questions are what that dependency is permitted to do, what information it receives, how replaceable it is, what jurisdiction governs it, and what happens when it fails or changes.

This is why I use a deliberately operational definition:

AI sovereignty is the effective ability of a specified owner to define and enforce the conditions under which an AI system uses data, exercises authority, depends on external systems, and continues operating.

Sovereignty engineering is the engineering of the controls required to establish and preserve that condition—not simply deciding where the model lives, where data is transmitted, or where inference occurs.

Each relevant dimension must be explicit. Specified owner matters because sovereignty is meaningless without identifying whose control is at issue. Define and enforce matters because a policy that exists only as a prompt is not a control boundary. Conditions matter because sovereignty is rarely absolute. And continues operating matters because control over a system that disappears when a vendor changes a product, revokes an API, modifies a model, or suffers an outage is limited control.

So the first serious question should not be:

Is the model local?

It should be:

Sovereign over what?

Data? Execution? Infrastructure? Models? Tools? Credentials? Updates? Operational continuity? Jurisdiction? The supply chain?

The answer may be all of them or only some. But until those boundaries are stated, sovereign AI is a label, not an architecture and not a solution.

This is also not a wholly new engineering problem. AI sovereignty inherits decades of arguments about internet sovereignty, cloud sovereignty, data residency, jurisdiction, vendor dependence, and technological autonomy.

What has changed is the object being controlled, not the underlying concern. An AI system does more than store and transmit information. It consumes context, generates probabilistic outputs, retrieves additional information, selects tools, and may be permitted to take actions.

The general engineering principle is old.

The specific contract needs new terms.

A Firewall Is Not an Architecture

The most common mistake in current discussions of sovereign AI is to treat physical or network locality as if it were the entire architecture.

Running a model on-premises may satisfy an important requirement. Air-gapping a system may satisfy a stronger one. Open weights may improve inspectability and reduce dependence on a vendor. Private infrastructure may improve data control.

But none of those properties, by themselves, establishes sovereignty.

A system can be entirely local and still be badly governed. It can expose credentials, write to systems it should only read, accept retrieved content as trusted instructions, execute tools without meaningful authorization, leak sensitive information through logs, or depend on opaque updates and unverified software.

The model may never leave the building while control has already been lost elsewhere in the system.

A firewall answers one question:

What network traffic may cross this boundary?

It does not answer what the model may read, what the application may execute, which tools can be invoked, who authorizes consequential actions, which instructions are trusted, who can update the system, or what dependencies remain outside the organization’s control.

This is not a new security principle. Zero-trust architecture exists precisely because network location is not a sufficient basis for trust. NIST SP 800–207 rejects implicit trust based merely on location or ownership. AI does not invalidate that principle; it makes it more important.

The same distinction applies to open-weight models. They may enable local inference, independent deployment, inspection, modification, and migration away from a vendor. But access to weights does not automatically provide training provenance, software supply-chain integrity, operational security, governance, or control over the surrounding system.

Likewise, private inference does not necessarily mean private operation. A system may run inference locally while transmitting telemetry, logs, embeddings, metadata, diagnostics, or application data elsewhere. The relevant question is not merely where inference occurs, but where information flows and who controls those flows.

This matters even more as AI systems become agentic. A model-driven system may interpret context, choose tools, generate parameters, select actions, and propose multistep behavior dynamically.

That changes the security problem.

The model should not define its own authority, and that has nothing to do with the model’s location; it has to do with engineering in accessing the model—regardless of its location.

A model may recommend opening a file, executing a command, or calling an external service. It should not decide whether those actions are permitted. Those decisions belong to the system surrounding the model.

This distinction between capability and authority is fundamental. A highly capable model may have almost no authority, while a weak model with excessive authority can create a dangerous system. Model capability should therefore never automatically increase execution authority.

The same principle applies to model interchangeability. Different workloads may legitimately use different models, but changing an inference provider should not silently change what data may leave the system, where it may be retained, which tools may be invoked, which actions require approval, or which jurisdictions have been accepted.

The model may change. The authority boundary should not.

Several other terms are also routinely conflated. Provenance identifies where an input, model, configuration, or output came from. Traceability lets us reconstruct the path through which a result was produced. Auditability lets an independent reviewer inspect evidence of that process. Explainability concerns whether behavior can be meaningfully explained, while interpretability concerns whether some internal mechanism or representation can be understood.

These are related capabilities, but they are not synonyms. Owning a model does not automatically make it explainable, logging every inference does not make it interpretable, and complete traceability does not prove that the system was correctly authorized. Operating behind a firewall establishes none of them by itself.

The engineering mistake is to take one useful control and promote it into a complete theory of sovereignty. Local execution, air gaps, open-weight models, and private infrastructure can all be valuable controls, and in some environments they may be essential. But none is sufficient on its own.

A firewall can be part of the architecture.

A firewall is not the architecture.

Sovereignty Is an Engineering Contract

We have seen this pattern before.

Every major expansion of networked computing produces a version of the same fear: once sensitive information leaves a physically controlled environment, control has been lost. We heard it with online banking, electronic medical records, cloud computing, remote administration, SaaS, and large-scale data platforms.

The concern was never irrational. External systems introduce real dependencies, attack surfaces, and legal and operational questions. What changed was not that those risks disappeared, but that we learned to engineer acceptable conditions under which those systems could be used.

Modern organizations routinely transmit financial data, medical information, authentication credentials, intellectual property, and operational data across infrastructure they do not physically own. We do not treat transmission itself as proof that control has been surrendered. We ask who receives the data, how it is protected, what may be retained, who is authorized to act, what jurisdiction applies, what can be audited, and what happens during failure.

That is the engineering contract.

AI does not repeal that contract. It extends it.

The general principle is old; the specific contract needs new terms.

One concerns context. AI systems consume unusually rich and unstructured information: proprietary strategy, customer data, source code, correspondence, retrieved documents, operational state, and combinations of information that may become sensitive only when assembled together. The relevant question is not whether a network connection exists, but what information is permitted to cross it, under what conditions, and for what purpose.

Another concerns authority. AI systems increasingly select tools, generate parameters, choose workflows, retrieve information, and propose or initiate actions dynamically. Capability must not imply permission.

A third concerns trust boundaries. Retrieved documents, external data, tool responses, and model outputs are inputs to the system. None should be able to redefine the policy under which the system operates. An instruction embedded in a document remains data unless a trusted mechanism grants it authority.

A fourth concerns dependency. An external inference provider may be entirely acceptable within a sovereign architecture, but the dependency must be explicit. The organization should know what data is transmitted, what guarantees exist, what happens if terms or behavior change, and what capabilities remain if the provider disappears.

Finally, the contract must survive change. Models are replaced, APIs evolve, providers change defaults, tools are added, and orchestration becomes more capable. A boundary that is correct today can become incorrect after a routine update. Replacing a model must not silently broaden data access, adding a tool must not implicitly grant new authority, and local inference must not fall back invisibly to an external provider.

For a given workload, the engineering contract should therefore define four core boundaries:

Data boundaries specify what information may be used, transmitted, processed, and retained.

Authority boundaries specify what the system may read, write, execute, modify, or delegate, and which actions require approval.

Dependency boundaries specify which external systems are accepted, what guarantees they must satisfy, and what happens when they fail.

Operational boundaries specify who may update, reconfigure, disable, audit, recover, and replace components.

Those boundaries must be enforced outside the model’s discretion.

A prompt telling a model not to reveal confidential information is not equivalent to preventing the transfer. A system instruction telling an agent not to execute dangerous commands is not equivalent to restricting the execution interface. Asking a model to remain within a directory is not equivalent to filesystem containment.

Prompts influence behavior. Controls enforce authority.

A sovereign architecture therefore places policy in mechanisms the model cannot simply reinterpret: identity and access controls, network policy, tool permissions, filesystem restrictions, approval gates, data-routing rules, execution sandboxes, and audit mechanisms appropriate to the system.

This does not mean every workload requires maximum isolation. A public-information summarizer, a hospital decision-support system, an industrial controller, and a defense application should not have identical contracts. Their requirements differ because their data, authority, consequences, dependencies, and threat models differ.

The useful question is not whether a system is “local enough” or “private enough.” It is whether the architecture satisfies its declared contract.

And that contract must be testable.

Sovereignty that exists only in documentation is aspiration. Sovereignty that survives adversarial and failure testing is an engineering property.

Sovereignty is not the absence of dependencies. It is control over which dependencies are accepted, what authority they receive, and what happens when they fail or change.

Let the Requirements Choose the Deployment

Once sovereignty is treated as an engineering contract, the local-versus-cloud argument becomes much simpler.

The deployment should follow from the requirements. It should not define them.

Too many discussions begin with a preferred architecture and work backward: the model must be local, the system must be air-gapped, the organization must own the hardware, or external inference must be prohibited.

Sometimes those conclusions are correct. But they are conclusions, not premises. A better sequence is:

Define the workload. Identify the protected assets. Establish the non-negotiable boundaries. Eliminate architectures that violate them. Compare the remaining options. Verify the result.

If information cannot legally or operationally leave a controlled environment, external processing may be excluded immediately. If operation without outside connectivity is mandatory, local execution may be required. If physical control demands extremely low latency, local inference may again be the correct choice.

Another workload may legitimately use an external provider under explicit technical, contractual, and jurisdictional controls. A third may use a partitioned architecture in which sensitive data and deterministic operations remain local while approved tasks use external inference.

All three can be rational designs.

None is sovereign merely because of its topology.

A deployment model is an output of the sovereignty analysis.

Underlying technology will continue to change. Models will improve. Hardware will change. Local inference will become practical for workloads that currently require external compute. External providers will introduce new controls and deployment options. Dependencies that are unacceptable today may become acceptable, while others may become unacceptable later.

A sovereignty architecture should survive those changes.

If an organization can replace one model with another without changing its data policy, execution authority, approval requirements, logging rules, or accepted dependency boundaries, then the model has been correctly treated as a component of the system rather than the definition of the system.

This also separates locality from independence. A system can run entirely on local hardware while depending on external package repositories, model updates, proprietary drivers, licensing infrastructure, or remote administration. Conversely, a system can use an external inference service while retaining control over most of its data, workflows, tools, policies, and operational decisions.

The question is not whether a dependency exists. It is whether the dependency has been identified, bounded, accepted, and tested.

This is the distinction between optimization and authority.

Optimization chooses the best option within the allowed set. Authority defines the allowed set.

Authority comes first.

Control Is the Point

The argument for sovereign AI becomes weaker when sovereignty is reduced to a deployment slogan.

Local models, private infrastructure, air gaps, open-weight models, and cloud services can all be appropriate under the right conditions. None is sufficient by itself to establish sovereignty.

Sovereignty is far better understood as an engineering property than as a hosting preference. The question is whether the owner can define and enforce the conditions under which the system uses data, exercises authority, accepts dependencies, and continues operating.

Complete technological independence is rarely realistic. Modern systems depend on hardware vendors, operating systems, software libraries, networks, suppliers, standards, and jurisdictions. Sovereignty is therefore not the elimination of every dependency. It is the deliberate management of dependency and authority.

Sovereign AI should not become a rejection of cloud computing, foundation models, or external services. Those are tools. The engineering question is whether they satisfy the contract for the workload.

Sovereign AI is not established by where the model is located or whether the system crosses a network boundary. It is established by sovereignty engineering. It is established by control.

State what you control. State what you depend on. Enforce the boundary. Verify that it holds.

That is soveriegn.

Sovereign AI Is An Engineering Contract was originally published in IQumulus on Medium, where people are continuing the conversation by highlighting and responding to this story.