Propagating curve and price changes through a pricing graph
In FICC trading systems, one curve tick affects many things. A 1bp move on the SOFR swap curve can make prices wrong for hundreds of swaps, caps, and cross-currency instruments. If you build or maintain a front-office pricing platform, you know the hard part is not the math. The hard part is the graph. How do you push changes through a dependency network? You must not produce old prices, use too much CPU, or show middle states to traders.
This post talks about dirty marking, fan-out, and ordering in a pricing DAG. It also talks about failure modes during volatile market times.
The Graph is a DAG, Not a Queue
Pricing systems are directed acyclic graphs (DAG). Leaf nodes are market data inputs. Examples are spot FX, deposit rates, swap curves, and volatility surfaces. Internal nodes are bootstrappers, interpolators, and curve builders. Root nodes are instrument pricers and risk calculators. When a leaf node updates, every downstream node that depends on it must recompute.
The simple way is to use pub/sub for each instrument. But this breaks quickly. If SOFR and EURIBOR both update, a cross-currency swap that depends on both might calculate price twice. Or worse, it calculates once with a mixed state (new SOFR, old EURIBOR). Traders do not care about your event throughput. They care that the PV on their screen is consistent. This means the propagation layer must understand dependencies, not just message routes.
Dirty Marking is Better Than Eager Evaluation
In a pricing graph, calculating everything on every tick wastes CPU. Most instruments do not trade every second. Many risk jobs run periodically. The standard engineering pattern is dirty marking. When an input changes, you mark dependent nodes as stale. You do not calculate them immediately. Recalculation happens on demand. For example, when a trader queries a ticket, when a risk job runs, or when a threshold is crossed.
This is a cache invalidation problem. You track a version or timestamp for each node. When a leaf updates, you walk the adjacency list and set the dirty flag downstream. The tradeoff is latency versus compute. Eager evaluation gives fresh prices instantly. But it burns CPU on instruments nobody looks at. Dirty marking saves CPU cycles. But it needs a fast and deterministic walk when a price is requested.
In practice, hybrid approaches are best. Critical liquid instruments get eager updates. Examples are on-the-run treasuries and major swap tenors. Illiquid or complex structures stay dirty until queried. The key is to make the dirty walk cheap. Keep adjacency lists in memory. Use bitsets for dependency tracking. Avoid lock contention during the mark phase. Lock-free structures or read-copy-update (RCU) patterns work well here. This is very true in C++ or Java engines. In these engines, GC pauses or mutex contention can make P99 latency very high.
When Fan-Out Meets Ordering
Fan-out makes pricing graphs messy. One curve node might feed dozens of interpolators. These interpolators feed hundreds of pricers. If you process updates asynchronously without ordering guarantees, you will see “tearing”. Tearing means a downstream node reads a new curve value but an old volatility surface. It produces a PV that never existed in the real market.
The fix is topological ordering combined with batched propagation. Do not push events node-by-node. Instead, collect input changes over a short time window. 1 to 50ms is typical for front-office systems, depending on the asset class. Then, compute a topological sort of affected nodes. Evaluate them in dependency order. This is how modern pricing engines avoid middle states. You do not wait for the market. You align your internal clock.
Get Arnold Liu’s stories in your inbox
Join Medium for free to get updates from this writer.
Frontend frameworks like React or libraries like RxJS solve similar problems. They use `combineLatest` or concurrent rendering. But backend pricing graphs need stricter guarantees. In Java or C++ systems, this often uses a custom DAG scheduler. It has lock-free queues for each topological level. In Kafka architectures, you can use partition keys aligned to curve families. A stream processor keeps state stores for each node. It only emits data when all upstream dependencies for a batch window are ready.
Engineering the Propagation Layer
Let us talk about the system design. A typical modern front-office stack routes market data through a feed handler. It normalizes the data and publishes to a message bus. From there, the pricing graph consumes the data, marks nodes dirty, and sometimes computes. Results are pushed to React screens via WebSockets or SSE. They are also saved for risk.
A few practical rules shape the design:
- Idempotency: Market data replays happen. Feed handlers disconnect and reconnect. Your graph must handle duplicate ticks. It must not double-mark or corrupt state. You must use sequence numbers per source. You must use idempotent upserts into the node state store.
- Backpressure: During volatile times, tick rates go up fast. If your graph cannot keep up, you do not want Out Of Memory (OOM) errors. You also do not want to drop curve updates silently. Apply backpressure at the ingestion layer, not the pricer. Use bounded queues for buffers. Apply last-write-wins rules based on sequence numbers for market data. Degrade gracefully. For example, skip non-critical tenors before dropping core curves.
- Partitioning: Do not put all curves on one Kafka partition. Partition by asset class or curve family. This way, independent updates can process in parallel. Cross-asset dependencies need a join layer. For example, FX forwards need two rate curves. The join layer must respect ordering across partitions. Stream-table joins or a dedicated DAG coordinator can handle this well.
Failure Modes in Production
Theory is clean. Production is not clean. Here are the failure modes I see break pricing graphs, and how to defend against them:
Stale Leaves: A feed handler silently stops updating a specific tenor. The graph marks dirty once, then never again. Downstream prices become wrong. Defense: Use heartbeat monitoring for each market data key. Also use a “max age” circuit breaker. It forces a full refresh or flags the instrument as unpriceable on the UI.
Ordering Violations: A network partition or consumer lag causes curve updates to arrive out of order. They arrive out of sequence relative to volatility updates. The graph computes a mixed state. Defense: Use logical timestamps or monotonic batch IDs per window. Add a validation step. It rejects computations if upstream versions do not match the expected snapshot. Never compute with mixed-generation inputs.
Fan-Out Storms: A macro news event invalidates half the graph at once. The dirty mark phase is fast. But the next eager recomputation for risk uses too much CPU. Defense: Use tiered recomputation. Calculate liquid instruments first. Calculate risk metrics second. Calculate historical scenarios last. Use work-stealing thread pools. Limit concurrent evaluations per node type. Pre-warm interpolation caches when possible.
Testing Gaps: Unit tests cover individual pricers. But they miss graph-level race conditions. Defense: Use deterministic replay. Record market data ticks for a session with sequence numbers. Feed them into a test harness that runs the full DAG. Check that PV outputs match a golden snapshot. Event-sourcing test runners or Kafka replay tools make this possible. If your graph is not deterministic under replay, it is not ready for production.
Closing Thoughts
Pricing graphs combine distributed systems theory and trading desk reality. Dirty marking saves CPU. Topological batching prevents tearing. Idempotent propagation keeps the system stable during feed drops. The math is well-known. The engineering difficulty is in the edge cases.
If you build or maintain a pricing graph, invest early in deterministic replay. Invest in version-aware invalidation and graph-level metrics. Do not add observability only after a stale-price incident. Remember that the market will test your ordering guarantees. It will test them whether you are ready or not. It is better to find the bugs in staging than on a live trading screen.
If you solved similar DAG propagation problems, I want to hear about it. If you have a different idea on batching windows versus eager evaluation, please share. This space keeps changing. The best patterns usually come from sharing ideas across different teams and tech stacks.
