Systems · Decisioning · Applied AI

When Rules Aren’t Enough and LLMs Aren’t Trustworthy: A Hybrid Pattern for Real-Time, Multi-Outcome Eligibility

Most enterprises run their eligibility logic on infrastructure that hasn’t fundamentally changed since the data-warehouse era. Here’s what we learned building a system that combines symbolic rule evaluation with LLM reasoning — and why neither approach works alone.

Venugobal Theagarajan & Muthu Rangarajan ~15 min read•June 2026

1. The problem nobody admits to

Walk into any large enterprise — a telecom, a bank, an insurer, a retailer — and ask a simple question: “At this exact moment, does customer X qualify for promotion Y?”

The honest answer, in most organizations, is: we’ll know tomorrow morning.

That answer sounds absurd until you understand the machinery underneath. Eligibility decisions are typically governed by condition-based rules — “customer must hold an active subscription tier AND be in good credit standing AND have held the account > 90 days AND not currently be enrolled in a conflicting program.” Authoring those rules requires coordination between a business analyst (who knows the intent), a data engineer (who knows the schema), and an application developer (who deploys the change). Each iteration is a multi-day cycle. Each rule, once deployed, runs against a snapshot — a list of entity IDs extracted overnight that satisfied the criteria at midnight.

This architecture has three structural deficiencies that compound on each other:

1. The eligibility explosion. A modern promotional catalog isn’t ten offers — it’s hundreds, each with overlapping criteria across product tier, entitlement type, tenure, channel, geography, and enrollment status. A single interaction may require evaluating dozens of outcomes to find the best match.
2. The staleness gap. The customer who changed their subscription during a chat session at 3:47 PM does not exist, as far as the rules engine knows, until the overnight batch re-runs. They cannot lose a benefit they no longer qualify for, and they cannot gain one they just earned, until tomorrow.
3. The “why not?” void. When a customer is denied an offer, conventional systems can say only that they were denied. They cannot say what specifically failed, what would need to change, or what the customer has already done toward qualifying.

Each of these is independently expensive. Together, they produce the modern enterprise’s worst customer-experience failure mode: confident wrongness at machine speed. The system tells a customer they qualify; by the time they reach checkout, they don’t. The system tells them they don’t qualify; in reality they would, if anyone could explain how.

The conventional rules engine is a beautifully engineered answer to a question nobody asks anymore: “Who qualified for this offer last night?” The question commerce actually asks is: “Does this person qualify right now, and if not, what would change that?”

2. Why neither pure approach works

There are two natural reactions to this problem, and both fail in instructive ways.

The “make it deterministic and fast” school

The first reaction is to keep the rules-engine paradigm but accelerate it: move evaluation in-memory, push it closer to the request path, add caching, parallelize. This is the path most decisioning platforms have taken over the last decade. It produces lower latency. It does not produce explainability, and it does not solve the authoring bottleneck. A faster waterfall is still a waterfall.

It also tends to produce a different pathology: the binary engine. Most rules engines return one verdict per evaluation call. To check ten promotions, the consuming application makes ten calls. Each call re-fetches the customer’s data. The data layer’s round-trip count grows linearly with the catalog. By the time you have fifty active promotions, the call graph for a single eligibility check looks like a denial-of-service attack against your own customer-data API.

The “just ask an LLM” school

The second reaction, more recent, is to throw an LLM at it. Feed the model the customer’s profile and the offer catalog; ask it to decide. This works astonishingly well on a demo and astonishingly poorly in production.

The failure modes are predictable and severe:

• Non-determinism. The same customer can get different answers from the same model on consecutive calls. For a regulated decision — pricing, credit, eligibility for a financial benefit — this is not a quirk; it is disqualifying.
• No audit trail. “The model said so” is not a defensible position when a customer disputes a denial, or when a regulator asks how a decision was reached.
• Catastrophic confidence. LLMs are excellent at sounding correct while being wrong. A rule that says “tenure_days >= 90" either fires or doesn't. An LLM asked "has this customer been with us long enough?" will hallucinate a yes when the actual number is 87.

The lesson here is not that LLMs are bad at decisioning. The lesson is that they are bad at the parts of decisioning that need to be deterministic and auditable, and excellent at the parts that need judgment, language, and adaptation. The trick is partitioning the problem so each technology handles what it’s good at.

3. The hybrid pattern

What we landed on, after several iterations, is an architecture with a clear contractual seam between two very different kinds of computation. We call it the Arbiter pattern, and it looks like this:

The discipline is in the separation:

Notice what the LLM is not doing: it never decides eligibility. It never overrides a rule. If the deterministic layer says “no,” the LLM’s job is to explain the no — not to litigate it.

4. The five mechanisms that make it work

That separation is the easy half. The harder half is the set of design decisions that operationalize it. Five mechanisms, in particular, do most of the work.

4.1 Outcome groups, not outcome rules

Conventional engines model one rule per outcome. If you have a tiered promotion — premium tier, mid-tier, basic tier, declined — you author four rules and the application makes four calls.

The Arbiter pattern inverts this. An outcome group is a named collection of related outcomes evaluated together in a single pass. The application makes one call against the group; the engine returns every outcome the customer satisfies, plus structured information on the ones they don’t. The grouping level is configurable — you might group by product line (all enterprise license tiers), by program tier (all premium-eligible outcomes), or by channel (everything available in self-service).

This is not just an API ergonomics win. It changes the round-trip economics. A consuming application that previously made N calls to evaluate N offers now makes one. The customer data is fetched once, mapped once, evaluated against N condition expressions in-memory, returned once.

4.2 One condition tree, two execution forms

The same logical condition — built once, in a no-code visual builder — needs to run in two very different contexts:

• Against a single customer record at evaluation time (low-latency, in-memory).
• Against a population of millions during segment design (high-throughput, columnar, analytical).

Writing those as two separate expressions is a recipe for drift. The mechanism we use serializes the condition tree once and derives both execution forms from the same representation. Modify the rule, both forms regenerate together. The author never sees either of them; they see drag-and-drop AND/OR groups.

Conceptually it looks like this (illustrative pseudo-code, not the actual derivation logic):

# A serialized condition tree (the canonical form)
condition_tree = {
  "op": "AND",
  "children": [
    {"field": "subscription_tier", "op": "in", "value": ["premium", "enterprise"]},
    {"field": "tenure_days", "op": "gte", "value": 90},
    {"field": "program_eligible", "op": "eq", "value": True}
  ]
}

# Derived form 1: single-row evaluation (DataFrame query)
# Derived form 2: population analytics (SQL predicate)
# Both regenerated from the same tree on every edit.

The actual derivation logic, the operators supported, and the mapping between filter grammar and analytical grammar are implementation details. The principle — single source of truth, dual execution targets — is the part worth understanding architecturally.

4.3 Real-time over batch, always

The system stores condition expressions, not customer lists. There is no nightly batch that materializes “who qualifies.” When an evaluation request arrives for customer X, the engine fetches customer X’s current attributes from registered data sources, runs the expressions, returns the result. The expressions are persisted; the qualification is computed on demand.

This sounds expensive. In practice it is not, because the cost of fetching one customer’s data and running a few dozen in-memory expressions is dominated by the network round-trip to the data source, which you were going to pay anyway. What you avoid is the overnight batch that recomputes eligibility for tens of millions of customers who didn’t change state — and the staleness that batch produces for the ones who did.

4.4 Projected-state evaluation

Here is a scenario most rules engines cannot represent at all. A customer is mid-session, about to downgrade their subscription tier. The change would disqualify them from a loyalty benefit they’re currently enrolled in.

Conventional system: the customer confirms the change. The benefit remains active in the database until the overnight batch runs. The customer receives a notification the next day that they’ve been de-enrolled, by which point the change is irreversible and the customer is angry.

The Arbiter pattern evaluates the outcome group twice: once against current data, once against projected data reflecting the pending action. If an outcome currently satisfied would not be satisfied post-action, the system can interrupt the action with a real-time disqualification notification: “This change will remove you from the loyalty benefit you’re currently enrolled in. Continue?”

This is the kind of capability that sounds like a UX nicety but is actually a structural property of evaluating expressions in real time against arbitrary state, including hypothetical state.

4.5 LLM gap-analysis as a contract, not a freeform call

When the deterministic layer returns unmatched outcomes, the LLM’s job is to explain the gap. The risk — the reason most LLM-decisioning experiments fail — is treating this as a freeform “explain why” prompt and trusting whatever comes back.

The discipline is to constrain the LLM to operate on a structured contract: the failing conditions, the current field values, and the values that would satisfy the conditions are all supplied as structured input. The LLM’s job is narrative generation over a pre-computed gap, not gap discovery. It cannot invent a reason; it can only describe one.

Get Venu Theagarajan’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

The specifics of that contract — the prompt shape, the structured-output schema, the validation passes — are the part we keep abstract here. The architectural principle is what matters: the LLM consumes deterministic output and produces language; it does not produce decisions.

The seam: if you can trace every customer-facing claim back to a specific condition expression and a specific data value, you have an auditable system regardless of how natural the language sounds. If you can’t, you have a beautiful demo.

4.6 Payload Templates and Action Templates: separating what to deliver from what to do with the mechanisms

When an outcome is matched, two distinct things need to happen: the system must know what to return to the caller, and it must know what to do as a consequence. Most rules engines collapse these into a single monolithic action block. The Arbiter pattern treats them as separate, independently versioned constructs.

A Payload Template defines the structured data attached to a matched outcome — the terms, values, eligibility window, and any fields the consuming application needs to render or act on. Payload Templates are authored once per outcome and rendered at evaluation time against the matched entity’s live attributes. A premium outcome in a financial-services context might carry an interest-rate adjustment and a credit-limit increment; the same pattern in a SaaS context might carry an unlocked feature set and a trial-extension duration. The template is the contract; the engine fills it.

An Action Template defines the side-effects triggered when an outcome fires: an API call to a downstream system, a message to a queue, a database write, an email or notification dispatch, an audit-log entry — or a rule chain. The action type is selected from a registered catalog; the parameters are resolved from the matched entity’s data at runtime, not hard-coded at authoring time.

Rule chaining deserves particular attention. Unlike a static pipeline that always invokes a fixed next step, the Arbiter’s rule-chain action type is outcome-driven: a downstream rule group is only triggered when a specific outcome fires, and the mapping between the upstream outcome’s payload and the downstream group’s input fields is explicitly authored at design time. This makes the chain inspectable — every hop is a named, versioned link — while remaining flexible enough that the same upstream outcome can fan out to different downstream groups depending on context. A matched eligibility outcome might trigger a fulfillment rule group, which in turn triggers a notification rule group, each with its own condition evaluation and its own Payload Template, all from a single originating evaluation call.

The separation matters for governance as much as for engineering. A compliance team can review and approve Payload Templates independently of the condition logic. An operations team can update an Action Template — changing which queue receives a fulfillment message, for instance — without touching the rule that triggers it. Each concern has its own change lifecycle and its own audit trail.

5. The segmentation problem: rules that haven’t shipped yet

One discovery surprised us. The hardest part of running a rules engine in production is not running it — it’s authoring rules with confidence that they’ll work.

A business analyst designing a new promotion needs to answer questions like:

• If I add this condition, how many customers drop out of the eligible population?
• Across my three tiers (premium / standard / basic), how does the population distribute?
• Are my tiers commercially viable, or is the premium tier going to qualify forty people?

Conventionally, these questions require a data-extraction request to the technology team. The analyst submits criteria; the team runs a query; the results come back days later. Each iteration consumes a cycle. Most rules get shipped under-tested because the iteration cost dominates the design discipline.

The fix is a live segmentation capability — and it is more than a query tool. Three properties, taken together, distinguish it from what a BI dashboard or a SQL notebook provides.

Multi-source data connectivity. The analyst does not load a pre-extracted flat file and hope it reflects current schema. The segmentation canvas connects directly to the same registered data sources — REST APIs, relational databases, data warehouse tables, pipeline-derived views — that the production engine uses at evaluation time. Field mappings authored for production rules are available immediately in the segmentation context. Adding a new data source to the rule engine makes it available in the segmentation canvas on the next session, with no additional ETL step.

Instantaneous candidate view. Each condition the analyst drags onto the canvas triggers a re-evaluation of the full population in-memory, against the live data, returning an updated qualifying count within seconds. The analyst does not submit criteria and wait; they interact with a live system. Adding a tenure threshold, tightening a tier filter, introducing an exclusion condition — each edit produces an immediate answer. The iteration cost that previously consumed days is compressed to seconds per hypothesis.

Distribution visualization. When a rule group contains multiple named outcomes, the segmentation canvas renders the population distribution across all outcomes simultaneously — how many entities fall into each tier, how many are excluded by each condition, where the overlaps lie. An analyst designing a three-tier offer can see instantly that the premium tier qualifies 4% of the population and the standard tier qualifies 61%, and can adjust thresholds before the rule ships rather than after it has produced commercially unviable results in production.

All three capabilities operate on the same condition tree, the same field mappings, and the same data sources as the production evaluator. There is no separate “analytics mode” with a different execution model. The interesting subtleties — how column types are profiled, how derived fields are expressed, how unmatched fields are flagged rather than silently treated as null, how mapping persists across sessions — are implementation details this article does not elaborate further.

The architectural point is this: the same condition tree that runs single-record evaluation in production also drives the population analytics during design. One canonical form, two execution targets, dual-purpose use throughout the rule’s lifecycle. The author and the production system share a single source of truth.

6. What the system actually looks like in production

Some concrete numbers, drawn from the reference implementation we built to validate the pattern:

The detail that matters most is not in the table: any rule edited through the visual builder takes effect on the next evaluation request, with no deployment. That property — rule logic as data, not as code — is what makes the iteration cost low enough that analysts will actually iterate.

7. The shape of the decisioning call

To make the pattern concrete, here is what a single evaluation call returns. The shape is the point; specific field names and the structured output contract used by the LLM layer are illustrative.

{
  "entity_id": "E-882441",
  "outcome_group": "renewal_eligibility_q2_2026",
  "evaluated_at": "2026-06-14T19:22:08Z",

  "qualified": [
    {
      "outcome": "premium_renewal_with_upgrade",
      "matched_conditions": [/* … */],
      "payload_template": "premium-renewal-v3",
      "payload": { "discount_pct": 20, "extension_months": 12, "unlocked_features": ["analytics", "api_access"] },
      "action_template": "renewal-fulfillment-v2",
      "actions_dispatched": [
        { "type": "api", "target": "fulfillment-service", "status": "queued" },
        { "type": "rule_chain", "target_group": "notification_dispatch", "status": "triggered" }
      ]
    }
  ],

  "unmatched": [
    {
      "outcome": "premium_renewal_standard",
      "gap": {
        "failing_conditions": [
          { "field": "subscription_tier", "current": "standard", "required": "premium" }
        ]
      },
      "guidance": "Upgrading to a Premium subscription would qualify this account for the full renewal benefit."
    }
  ],

  "projected": {
    // populated only when the request includes a pending action
    "would_lose": [],
    "would_gain": []
  }
}

The guidance string is the only field whose value is LLM-generated. Everything else — the qualified list, the failing conditions, the payload resolved from the Payload Template, the actions dispatched from the Action Template, the projected deltas — is deterministic output. An auditor can trace the guidance back to the failing_conditions that produced it, and can trace every dispatched action back to the named Action Template that authorized it. The customer-facing language is generative; the substance behind it is symbolic.

8. What we still get wrong

No architecture pattern survives contact with production unscathed. The Arbiter pattern, as we’ve built it, has open problems we don’t pretend to have solved:

• Cross-call consistency for narrative guidance. The LLM-generated guidance is regenerated per call, which means a customer asking the same question twice might get two slightly different phrasings. For some use cases this is fine; for regulated channels (collections, credit) it isn't, and we cache aggressively, but caching introduces its own staleness problems.
• Ranking under judgment. When multiple outcomes match and a “best offer” needs to be selected, the LLM’s ranking is a judgment call. We’ve found that judgment to be reasonable in practice but not reliably explainable. We default to deterministic ranking (e.g., by benefit value or priority weight) and only fall through to LLM ranking when the deterministic ordering produces ties.
• The data freshness contract. “Real-time” is a marketing word; in practice it means “as fresh as the upstream data source allows.” If a customer’s subscription change hasn’t propagated to the entity-data API yet, the engine evaluates against the pre-change state. We’ve moved a lot of complexity into pipeline-chained data acquisition with computed-field derivation, but the staleness of somebody else’s system is not a problem any rules engine can fix by itself.
• Schema evolution. Field schemas evolve. Rules reference field names. The mapping between source data and named rule fields needs to evolve too, and we’ve gone through two iterations of how to handle field renames without breaking rules that reference the old names.

9. What we’d tell a team starting from scratch

If you are designing decisioning infrastructure today — for eligibility, fraud, pricing, routing, or any other domain where conditions are complex and decisions need to be both correct and explainable — here is the short version of what we’d recommend:

1. Decide what’s deterministic and stay disciplined about it. Anything an auditor might ask about needs a symbolic answer. Don’t let an LLM into that part of the pipeline.
2. Put the LLM at the boundary, not in the core. Generative AI is excellent at the seam between your deterministic system and the human consuming its output. That’s where it belongs.
3. Store expressions, not lists. The moment you materialize “who qualifies,” you’ve committed to a staleness model. Store the conditions and re-execute on demand.
4. Single source of truth for the condition tree. The same rule needs to run single-record at evaluation time and population-scale at design time. Build one representation, derive both execution forms.
5. Group outcomes, batch the API. N outcomes should not require N round-trips. One call, one data fetch, all outcomes in the group evaluated together.
6. Treat “why not?” as a first-class output. If your system can only say “denied,” your customer experience is broken regardless of how fast or how accurate the denial is.
7. Make rule authoring no-code, and make it interactive. If a business analyst needs to file a ticket to ship a rule, the rule won’t ship. Period.

10. The point of all of this

Rules engines have been around for forty years. LLMs have been around for four. The instinct is to treat them as competing solutions to the same problem — one old, one new, may the best paradigm win.

That framing is wrong. They solve different parts of the same problem. The deterministic engine is the only thing that can give you correctness, reproducibility, and audit. The LLM is the only thing that can give you language, judgment, and the assisted-experience layer that turns a denial into a conversation.

The interesting design work is in the contract between them — the seam where deterministic output becomes LLM input, the structured constraints that keep generative reasoning from drifting into hallucination, the architectural discipline that makes the system explainable end-to-end even though one of its layers is a stochastic black box.

We don’t think the Arbiter pattern is the only way to solve this. We do think the partitioning — symbolic for decisions, generative for language and judgment — is durable, and that any system trying to do real-time eligibility at enterprise scale will end up looking something like it, whether or not anyone calls it by this name.

The rules engine isn’t dead. It just needs better company.

Acknowledgements & References

1. [1]Neuro-Symbolic Artificial Intelligence: The State of the Art
   Pascal Hitzler & Md. Kamruzzaman Sarker (eds.). IOS Press, 2022. Theoretical backdrop for the deterministic/generative partitioning described in Section 3. doi:10.3233/FAIA342
2. [2]Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
   Lewis et al. NeurIPS, 2020. Landmark pattern for grounding generative output in retrieved structured facts — precursor to constraining LLM reasoning over deterministic gap data. arXiv:2005.11401
3. [3]Faithful Reasoning Using Large Language Models
   Creswell et al. arXiv, 2022. Chaining LLM reasoning through verifiable symbolic intermediates — directly related to the structured-contract approach in Section 4.5. arXiv:2208.14271
4. [4]Rules Engines: The Good, the Bad, and the Hazardous
   Martin Fowler. martinfowler.com, 2009. The canonical practitioner critique of rules-engine architectures — articulates the authoring bottleneck and transparency problems that motivated the Arbiter pattern. martinfowler.com/bliki/RulesEngine.html
5. [5]Structured Outputs
   OpenAI Platform Documentation, 2024. Constrained decoding to enforce schema compliance in LLM responses — the mechanism underlying the structured-contract call in Section 4.5. platform.openai.com/docs/guides/structured-outputs
6. [6]OpenAPI Specification 3.1
   Linux Foundation / OpenAPI Initiative. Machine-readable contract format used to describe the Arbiter evaluation endpoint and its structured request/response schemas. spec.openapis.org/oas/v3.1.0

About the authors.
Venugobal Theagarajan: Principal ArchitectDriving innovative data architecture across the full design-to-delivery lifecycle for customer-facing and back-office systems.
Muthu Rangarajan: Principal Engineer, Site ReliabilityFocused on platform reliability and production engineering for large-scale distributed systems.

The reference implementation discussed in this article was built collaboratively. Numbers in the production-shape section are drawn from the working system; design decisions reflect lessons that only became visible after the system was in users’ hands.