Migrating to a DSL for Load Access & Visibility

A freight load board search can return thousands of loads. But not every carrier should see every load — or every attribute on a load.

By Nick Raff, Nipun Dave, and David Rajec

A broker might require a carrier to have six months of operating authority before exposing a rate. Other policies might hide the exact address, contact information, or booking details.

We originally implemented many of these policies using LaunchDarkly. That worked when the question looked like:

Should this user see X?

It became problematic when the question became:

For several thousand loads, which loads and which attributes should this carrier see?

Our LaunchDarkly integration did not provide an efficient batch primitive for this workload. We were evaluating rules repeatedly across thousands of records in Node.js, making visibility evaluation a significant part of search latency.

More fundamentally, we were using a feature-flag system as a record-level authorization and data-masking engine. So we built one.

A DSL for load access

We created a domain-specific language for expressing load visibility:

rule "New Carrier" {
    target load
    when carrier.authorityAgeMonths < 3
    then deny
        code AUTHORITY_TOO_NEW
}

rule "Hide Rate" {
    target load_attributes
    when carrier.authorityAgeMonths < 12
    then warn rate, ratePerMile
        code RATE_RESTRICTED
        because "Rate requires additional authority history"
}

The language distinguishes rules affecting the entire load from rules affecting individual attributes. It supports show, warn, and deny, boolean expressions, comparisons, membership operators, defaults, and some/every quantifiers.

Rules operate against a deliberately constrained context:

• load → including details like the broker and equipment type required
• carrier → including details regarding the carrier’s authority age and insurance maximums
• user → general user details

There is no arbitrary JavaScript and no generated SQL.

Why Langium instead of Peggy?

We considered both Peggy and Langium.

Peggy is a lightweight parser generator and would have been sufficient if all we needed was: text → parser → AST. But parsing was only part of the problem.

We wanted people outside the implementation team to safely modify policies. That required type checking, semantic validation, useful diagnostics, autocomplete, formatting, and an IDE-like browser editor.

Langium gave us that larger language-engineering stack:

Grammar
  |
  |-- Parser
  |-- Typed AST
  |-- Validation
  |-- Completion
  |-- Formatting
  |-- Language Server

Our implementation includes custom validators, completion, and formatting providers, and the same language package powers our Monaco-based administrative editor.

That means we catch errors before rules reach production. Load rules cannot specify attributes. Attribute rules must identify the attributes they affect. Field references are checked against the context schema. Operators are type-checked, and nullable fields require appropriate guards.

Langium gave us the foundation for a small, typed programming language.

Creating and editing visibility rules

A DSL is only useful if changing a policy is safer than changing application code.

We built an internal rule-management interface around Monaco. An administrator selects a broker and can inspect its currently active policy or create a new draft. The UI provides separate editing surfaces for load-level and attribute-level rules.

The workflow looks like this:

Validate while editing

The browser runs the same Langium compiler used by the backend. After edits are completed, both the load and attribute rule sets are compiled and validated. If either is invalid, the draft cannot be saved or activated. This catches errors at authoring time rather than when a carrier performs a search.

Save changes as a draft

Edits do not immediately affect production traffic. They are first persisted as a draft.

The server validates the DSL again before storing it, so client-side validation is a usability feature rather than a security boundary.

The UI maintains separate active and draft states and can show a diff between them before activation. If a draft already exists, the active version becomes read-only so changes continue against the pending draft rather than accidentally forking another edit.

Activate atomically into the runtime path

When a draft is activated, the server promotes the new rule set, handles replacement of the previous active rules, compiles the new DSL, and invalidates the active compiled-rules cache. Subsequent searches therefore pick up the new policy without an application deployment.

Active rules can similarly be expired when a policy should no longer apply.

Get Nick Raff’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

The resulting operational model is:

Policy changes behave more like configuration deployment than code deployment — but retain many of the safety properties of code.

They have syntax, types, validation, drafts, diffs, activation, and explicit rollback/expiration semantics.

Making rule evaluation fast

Replacing LaunchDarkly only helped if our replacement was substantially cheaper on the search path. We therefore optimized both fetch/compile time and evaluation time.

1. Cache compiled rules

Parsing and validating the DSL is considerably more expensive than evaluating an already-built AST, so we avoid doing it during every search.

The language package maintains an LRU cache of up to 1,000 compiled rule sets and deduplicates concurrent requests attempting to compile the same rule text.

The server maintains another cache keyed by active broker rule-set ID, while the active broker-to-policy mapping is cached separately. Rule activation or expiration invalidates the relevant caches.

The common request path therefore becomes:

cached compiled AST → evaluation

instead of:

fetch → parse → validate → build AST → evaluation

2. Load shared context once

Carrier and user information is shared across every candidate load in a search. We load that visibility context once and reuse it across candidate evaluations. Only the load-specific facts change.

3. Don’t evaluate loads with no rules

The visibility stage creates evaluation inputs only for loads whose broker has an active rule set. Loads from brokers without access policies bypass the evaluator entirely and remain visible by default.

4. Eliminate entire brokers before fetching loads

Some policies depend only on broker, carrier, and user context, and not on an individual load. We inspect the compiled AST to identify those policies. When a load-visibility rule contains no load-specific fields, we can evaluate it once per broker before querying the load database.

Denied brokers are inserted directly into the search’s excludedBrokers filter:

Their loads never enter the candidate set.

Policies referencing individual load fields are deferred until candidate records exist.

This gives us a hybrid execution model without having to compile an arbitrary DSL into SQL.

5. Evaluate compiled ASTs locally

For remaining candidates, rule execution is entirely in-process.

The evaluator walks the compiled expression tree against structured facts, while the server processes candidate evaluations with bounded concurrency of 10.

The result is not just a boolean. It contains visibility decisions for the load and its controlled attributes, together with the rule and reason responsible for the decision.

Denied records are removed during load consolidation, while attribute visibility influences which values can be selected when multiple sources represent the same load.

The resulting system

The final search path looks roughly like this:

In production, the ruleset latency can be described in two steps:

1. Fetch visibility rules: P50 → 10.7ms and P90 → 50ms
2. Evaluate visibility rules: P50 → 0.9ms and P90 → 38ms

The interesting result is that actual policy evaluation is usually sub-millisecond at P50.

Most requests don’t pay parsing costs. Context is reused. Loads without policies are skipped. Entire brokers can sometimes be eliminated before candidate retrieval. The remaining rules execute against an in-memory typed AST.

The result is not simply a faster replacement for LaunchDarkly.

It is a policy system designed around the computational shape of the problem we actually have.

The larger lesson

Feature flags were the right abstraction when the question was:

Is this feature enabled for this user?

Once the question became:

Which parts of thousands of domain objects can this customer see?

we needed something different.

Our policies now have a complete lifecycle:

Author → Validate → Draft → Review → Activate

where validation and evaluation is driven by the DSL:

Grammar → AST → Compile → Cache → Evaluate → Explain

They are versionable, testable, cacheable, understandable by humans, and integrated directly into the load-search pipeline.

Sometimes a configuration problem becomes complex enough that the right abstraction isn’t another collection of if statements, JSON rules, or feature flags. It’s a language.