Why Declarative API Orchestration Keeps Failing And How We’re Fixing It

Companies like Apollo, StepZen (now IBM), Tailcall, and others have repeatedly attempted the same declarative approach to API orchestration. The problem? It just doesn’t work for complex use cases.

I’ve been there myself. Back in 2018 I built a GraphQL API Gateway using this exact approach. Since then, the idea has been copied repeatedly, but the issues haven’t gone away. It works for hello world demos. But enterprise-scale needs? Not so much.

Why Declarative API Orchestration Breaks Down

To explain, let’s look at this example using Apollo Connectors:

type Booking {
  id: ID!
  checkInDate: String!
  checkOutDate: String!
  status: BookingStatus!
  hostReview: Review
    @connect(
      source: "listings"
      http: { GET: "/review?bookingId={???}" }
      selection: """
      id
      text
      rating
      """
    )
}

Beyond the fact that Apollo Connectors are locked behind a GraphOS Enterprise paywall, it has some deep technical flaws.

Directives Are a Bad Fit for Infra as Code

GraphQL Directives weren’t designed to serve as infrastructure. They lack type safety and extensibility. You can’t scope a directive to fields of a specific type. GraphQL SDL doesn’t support maps, so basic mapping logic requires workarounds, such as a selection field that only functions with specialized tooling.

It’s Verbose — and Hides the Data Model

Using @connect requires specifying mappings, parameters, HTTP methods, and more — all directly in the SDL. As your schema grows, this becomes hard to follow. A clean SDL should describe your model, not contain implementation details.

It Leaks Internal Details into the Contract

You end up designing your GraphQL contract around how easy it is to map existing services, not how your API should look. That tight coupling makes the SDL brittle and harder to evolve.

It Triggers the N+1 Problem

Add a hostReview field, and now fetching 50 bookings means 50 review API calls. Declarative connectors can’t express batching relationships, so the Router can’t optimize with a Dataloader pattern. You’re stuck with poor performance.

It’s Hard to Test or Debug

In theory, if all services are perfectly composed and follow spec, things might work. In practice, the only way to test is to boot the Router, open a playground, and manually check every query.

A good system should let you:

• Write automated tests against mocks
• Debug mapping logic with a standard toolchain
• Verify correctness without spinning up infrastructure

Why We Waited — and What Changed

We avoided declarative orchestration for years. It just didn’t scale, and it wasn’t testable. But recent advances in generative AI changed the equation.

By combining LLMs with a fresh approach to orchestration, we’ve developed something new: gRPC Plugins.

Introducing gRPC Plugins: LLM-Centric API Orchestration

The core idea is simple: break orchestration into a clear contract and a plugin implementation. Engineers define the SDL. LLMs generate the plugin. You review and deploy.

Get Stefan 🚀’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

Here’s how it works:

1. You write GraphQL SDL (or prompt an LLM to do it).
2. A compiler converts it into a gRPC Protobuf plugin.
3. The Router loads the plugin and communicates over gRPC.
4. The plugin code is written in plain Go — debuggable, testable, and mockable.
5. Cursor Rules help the LLM know exactly what to do, step by step.

The workflow is fast, safe, and developer-friendly.

Why We Chose Hashicorp’s go-plugin

We considered WebAssembly and embedded scripting engines, but neither matched our goals for testability and speed.

The go-plugin library hit the mark:

• It’s used in production by Terraform, Vault, and others.
• It runs over Unix Domain Sockets for performance.
• It supports any language, not just Go — so Python, Node.js, and others are on the roadmap.

It enables us to build lightweight, scalable plugins without introducing deployment complexity.

Cursor Rules: Making LLMs Reliable Developers

Cursor lets us define rule-based prompts that guide LLMs through plugin development — compiling code, writing tests, and building binaries — all without touching the wrong files. The whole thing is bootstrapped with one CLI command:

wgc router plugin init hello-world

From there, Cursor and the LLM take over.

Cosmo gRPC Plugins vs Apollo Connectors

Cosmo gRPC Plugins vs Apollo Connectors

Final Thoughts

Declarative orchestration always sounds elegant — until you need to support real-world use cases. Then you find yourself building a VM, a DSL, and a half-baked language no one wants to learn.

Apollo has done exactly that. They’re teaching classes and running webinars just to make Connectors usable.

We think there’s a better way.

gRPC Plugins let LLMs handle the heavy lifting. No new languages. No DSLs. Just simple, readable code that’s easy to test and debug.

It’s time to design tools with LLMs in mind.

To get started, check out the Router Plugins documentation.

Read the full technical breakdown on WunderGraph’s blog