Why I’d Pick Boring Infrastructure for AI Agents
The model is the new part of an AI agent. The AI infrastructure around it, the parts that store progress, retry failed calls, wait for approvals and hold secrets, should be the oldest, dullest technology you can find.
By boring I mean software that has failed in public for years, so its failure modes are written down: message queues, databases with a known backup story, workflow engines, and mutual TLS, where two services each prove who they are before they talk. Your on-call engineer has seen these break and knows what to do when they page.
Get Tony’s stories in your inbox
Join Medium for free to get updates from this writer.
An agent already brings one unpredictable piece, because the model can answer the same question two different ways. I don’t want a second unpredictable piece underneath it. When a run fails, I want to know the problem is in the prompt or the tool, and that the layer holding the run together did what its docs say.
That’s one reason I like Dapr under agents. It’s a graduated project of the CNCF, the foundation that also hosts Kubernetes, which is about as boring a credential as cloud software gets.
Durability alone doesn’t finish the job. A production agent still needs “a verifiable identity, authorization on every privileged call, step-level observability, and cost controls,” as this Diagrid post lists them. I’d pick boring for those too: https://www.diagrid.io/blog/durable-execution-runtime-primitive-agents
