<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Chong Yao on Medium]]></title>
        <description><![CDATA[Stories by Chong Yao on Medium]]></description>
        <link>https://medium.com/@chongyao.robin?source=rss-a3d8293bb683------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*R3ZAbND0OOtilcCzVC2R1Q.png</url>
            <title>Stories by Chong Yao on Medium</title>
            <link>https://medium.com/@chongyao.robin?source=rss-a3d8293bb683------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sun, 22 Nov 2026 06:57:01 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@chongyao.robin/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[From Prompt to Playthrough — II: Making the AI Layer Observable]]></title>
            <link>https://medium.com/@chongyao.robin/from-prompt-to-playthrough-ii-making-the-ai-layer-observable-e5c70db04a62?source=rss-a3d8293bb683------2</link>
            <guid isPermaLink="false">https://medium.com/p/e5c70db04a62</guid>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[software-architecture]]></category>
            <dc:creator><![CDATA[Chong Yao]]></dc:creator>
            <pubDate>Fri, 22 May 2026 04:34:45 GMT</pubDate>
            <atom:updated>2026-05-27T02:34:47.591Z</atom:updated>
            <content:encoded><![CDATA[<h3>TL;DR</h3><p>In <a href="https://medium.com/@chongyao.robin/from-prompt-to-playthrough-building-an-ai-interactive-fiction-engine-47cef1a7e59b">Part 1</a> I walked through the core engine of <a href="https://github.com/ZaphyrRobin/Mythwright">Mythwright</a>: a six-step pipeline turning player choices into LLM-generated chapters, a three-tier memory architecture, drift-based narrative gravity, and non-destructive branch rollback. The Future Work section flagged observability gaps as a TODO; what I didn’t realise until I tried to <em>use</em> the engine was how much else was missing — a structured prompt+schema pipeline, a real eval harness, and durable workflows for when LLM calls fail halfway.</p><p>This post is what filled those gaps: <strong>the production-side layer</strong>. Not the engine that generates a chapter, but the scaffolding that tells you <em>whether the chapter is any good</em>, <em>what it cost</em>, <em>where the latency went</em>, and <em>what to do when something halfway through breaks</em>.</p><h3>The System</h3><p>Five tools, in the order I added them:</p><ol><li>BAML — schema + prompt + streaming parser as a single source of truth</li></ol><p>2. Langfuse— the product view: cost, prompt versions, eval scores</p><p>3. Jaeger + OpenTelemetry — the engineering view: span tree, latency, errors</p><p>4. Restate — durable workflows for when LLM calls take 10+ seconds</p><p>5. A custom Python eval harness — catching prompt regressions before they ship</p><p>Each one solved a specific pain point I hit in production. None of them came for free.</p><p>Here’s where each piece lives. Part 1’s diagram had Story Engine → LLM Router → Ollama/Claude with a flat Storage layer. Part 2 inserts BAML in the LLM seam, fans OTel spans out to two backends, adds an optional Restate path, and parks the eval harness off to the side as an offline tool:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*a3WKuZvWnL6n4ER0DS4kNA.png" /><figcaption>The main request pipeline — from clients down through the dual execution paths to the LLM providers</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*C6SJJFf1cBOlMlzIZtaj9g.png" /><figcaption>Every chapter generation emits OpenTelemetry spans carrying latency, tokens, and cost — these fan out to two observability backends</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HI4v-hXgr6JsGcqgaHiMuQ.png" /><figcaption>Finally, the offline evaluation path runs independently of the request flow — it skips the API and service layers entirely, calling BAML directly</figcaption></figure><h3>The pain points that motivated all of this</h3><p>Three things broke in Part 1’s engine that I couldn’t see from outside:</p><p><strong>Prompts silently regressed</strong>. I’d tighten a constraint in the prompt, ship it, and not notice for days that a different fixture had started failing. The only signal was player drop-off, which is <em>way</em> too downstream.</p><p><strong>Costs were invisible</strong>. Each chapter is a non-trivial Claude call (1500–2500 chars of narrative + 4 options + a memory summary). I had no idea what a single playthrough cost me, or which prompt change pushed token count up 30%.</p><p><strong>Retries leaked</strong>. When a Claude call timed out, my naive retry sometimes inserted duplicate <em>Chapter</em> rows, because the retry happened <em>before</em> the persist transaction committed. The pipeline wasn’t idempotent and there was nothing tracking step-level state.</p><p>I needed observability + structure. Here’s what each tool gave me.</p><h4>1. BAML: schema + prompt + parser as one file</h4><p>Part 1’s <em>ChapterValidator</em> used tiered error handling — hard errors (invalid JSON, missing required fields) retried the full LLM call up to 3 times, soft errors (HP out of range, location not in known list) got patched in place. That tier-1 retry path was where a lot of pain hid.</p><p>The original flow was: LLM returns a JSON string → Python parses it → if structure is wrong (missing field, wrong type, fewer than 4 options) → retry the whole LLM call. With smaller local models the retry rate was substantial — Mistral 7B routinely garbled the JSON shape. Claude was rarer but not zero, and every retry was a wasted 10-second round-trip.</p><p>BAML is a schema DSL written by <a href="https://www.boundaryml.com/">BoundaryML</a>. You write your output schema and prompt in the same file:</p><pre># baml<br>enum Direction {<br>    MainArc @alias(&quot;main_arc&quot;) @description(&quot;Advances the main story arc&quot;)<br>    SideQuest @alias(&quot;side_quest&quot;) @description(&quot;Pursues an optional thread&quot;)<br>    CharacterDevelopment @alias(&quot;character_development&quot;)<br>    Exploration @alias(&quot;exploration&quot;)<br>}<br><br>class OptionItem {<br>    id int @description(&quot;0-3, matches array index&quot;)<br>    label string<br>    hint string<br>    direction Direction<br><br>    @@stream.done<br>}<br><br>class ChapterOutput {<br>    narrative string @description(&quot;1500-2500 char second-person narrative&quot;)<br>    stats_update StatsUpdate<br>    options OptionItem[] @description(&quot;Exactly 4. At least one main_arc.&quot;)<br>    memory_summary string @stream.done<br>    importance_score float @stream.done<br>}<br><br>function GenerateChapter(...) -&gt; ChapterOutput {<br>    client OllamaMistral  // overridden per-request via ClientRegistry<br>    prompt #&quot;<br>        {{ _.role(&quot;system&quot;) }}<br>        You are the story engine for an interactive fiction game...<br>    &quot;#<br>}<br></pre><p>The <em>@@stream.done</em> markers are doing real work. They tell BAML’s parser “only emit this field once it’s fully formed”. So <em>narrative</em> streams token-by-token to the player (great UX), but <em>options</em> shows up atomically as a 4-element array — no half-rendered buttons mid-stream.</p><p>What I got for free:</p><p>- <strong>Structural retries vanished.</strong> BAML’s Rust-side parser is forgiving in ways my Python JSON parser wasn’t. Malformed quotes, extra commas, trailing whitespace — fixed transparently at parse time.</p><p>- <strong>Streaming UX without manual diffing</strong>. I subscribe to the partial stream, push tokens to SSE, and let BAML’s <em>@stream.done</em> markers decide when a structured field is safe to emit.</p><p>- <strong>One source of truth for prompts</strong>. The prompt lives next to the schema. When I change the schema (add a stat, tweak an option field), the prompt is <em>right there</em> — no more “edit the dataclass and forget to update the prompt”.</p><h4>2. Langfuse: the product view</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_mbsWNXgu3XwaQY8L66nLQ.png" /><figcaption>Langfuse home — traces, model cost, scores at a glance</figcaption></figure><p>I self-host Langfuse v3 in docker-compose. The trade I considered was <em>Braintrust</em> (SaaS, slick) vs <em>Langfuse</em> (self-host, OSS). I went OSS because:</p><p>1. <strong>Data stays local</strong>. Every player session, every prompt version, every chapter narrative — all of it sits in my own Postgres + ClickHouse + MinIO stack. No vendor account, no usage metering.</p><p>2. <strong>OTel-native</strong>. Langfuse v3 ingests <em>OpenTelemetry</em> traces directly. I don’t need a Langfuse-specific SDK or a separate instrumentation path — the same OTel spans I send to <em>Jaeger</em> fan out to Langfuse with one config line.</p><p>3. <strong>Free</strong>. This matters when you’re iterating on prompts and racking up trace volume.</p><p>What Langfuse gives me that Jaeger doesn’t:</p><p>- <strong>Real token costs</strong>. When BAML emits the <em>gen_ai.usage.input_tokens</em> and <em>gen_ai.usage.output_tokens</em> span attrs, Langfuse multiplies them by Anthropic’s pricing and shows me USD per trace, per session, per project. I can see <strong>this playthrough cost $0.23</strong> without writing a single line of cost-tracking code.</p><p>- <strong>Prompt version tracking</strong>. Every BAML prompt I push gets a version stamp. When a chapter looks weird, I can scroll back and ask <em>what prompt did this trace use?</em></p><p>- <strong>Score attachment</strong>. This is the big one for eval (see section 4). I can upload eval scores as <em>score</em> records tied to specific traces, and Langfuse plots them over time — trivial regression detection.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2t9kEQpBJFBay9mI1ICG_Q.png" /><figcaption>Langfuse tracing — chapter.generate span tree</figcaption></figure><p><em>Langfuse tracing view — the full chapter.generate span tree (load → context → gravity → llm_call → verifier → validate → persist) with per-stage latency.</em></p><h4>3. Jaeger: the engineering view</h4><p><em>Jaeger UI— the same trace as a engineering-style waterfall. Useful for chasing latency hot-spots and inspecting span attributes.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*MCcypXnE1uJze-JoSX-w7g.png" /><figcaption>Jaeger trace — chapter.generate waterfall</figcaption></figure><p>OpenTelemetry’s nice property is that <strong>one instrumented codebase fans out to multiple backends</strong>. I export to Jaeger and Langfuse from the same SDK, with the same spans, just two different OTLP endpoints in config:</p><pre>```<br># server/app/telemetry.py<br>exporter_jaeger = OTLPSpanExporter(endpoint=&quot;http://localhost:4319&quot;) # gRPC<br>exporter_langfuse = OTLPSpanExporter(endpoint=&quot;http://localhost:3002/api/...&quot;) # HTTP+Basic<br>```</pre><p>Jaeger is where I look when something feels <em>slow</em>. Langfuse tells me a trace took 15 seconds total. Jaeger tells me 12 of those seconds were in <em>chapter.llm_call</em> (expected — Claude takes ~10s for 2KB of narrative), 2 seconds were in <em>chapter.persist</em> (uh, why?), and the rest spread across short setup spans.</p><p>That <em>why is persist slow</em> question is exactly the kind of thing you can’t answer with product analytics. Jaeger’s waterfall layout makes the answer visual: oh, <em>persist</em> is fanning out to memory promotion and arc-progress updates that I should have batched.</p><h4>4. The eval harness: catching prompt regressions before they ship</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Inex3Hy4VsfpoiiQxNYJPw.png" /><figcaption>Eval scores in Langfuse</figcaption></figure><p><em>Eval upload — every </em><strong><em>python -m evals.run</em></strong><em> lands as a Langfuse session with one trace per fixture and one score per scorer.</em></p><p>This is the single most useful tool I added, and the one I almost didn’t build because I assumed eval frameworks were heavyweight.</p><p>The harness is ~650 lines of Python in <em>server/evals/</em> (driver + scorers):</p><p>- <strong>6 fixtures</strong> representing distinct test cases: a freeform low-drift chapter, a high-gravity main-arc push, a chapter where Permanent Memory should trigger via the keyword oath, a stress case where <em>known_locations</em> is artificially limited (does the LLM honor the list or make one up?), one where a story rule should fire, one expecting diverse option directions.</p><p>- <em>7 scorers</em>, each a pure function (<em>input, output, expected) -&gt; Score(0.0..1.0, metadata)</em>:</p><p>- <em>structure</em> — exactly 4 options, ≥1 <em>main_arc</em>, non-empty narrative</p><p>- <em>narrative_length</em> — 1500–2500 chars</p><p>- <em>location_known</em> — chosen location is in the world’s known list</p><p>- <em>direction_diversity</em> — at least N distinct option directions</p><p>- <em>pm_keyword</em> — <em>memory_summary</em> matches the PM-promotion regex when expected</p><p>- <em>main_arc_count</em> — ≥N <em>main_arc</em> options for high-gravity fixtures</p><p>- <em>llm_judge</em> — Claude Haiku scores narrative on world consistency / pacing / character voice, 1–5 → 0–1</p><p>Six of seven are deterministic. Only <em>llm_judge</em> is stochastic, so I run it 3+ times before drawing conclusions.</p><p>The whole thing exits 0/1/2 like a normal CLI tool, so CI can gate on it:</p><pre><br>```<br>python -m evals.run --model claude-haiku-4-5-20251001<br># 0 = pass, 1 = quality regression (avg &lt; 0.5), 2 = config error<br>```</pre><p>After every prompt change, every BAML version bump, every model switch — I run this and check the avg score. The current baseline (against Claude Haiku) is 0.97 across 6 fixtures. If a prompt change pushes that below 0.9, the change doesn’t ship.</p><p>What I deliberately didn’t build:</p><p>- <strong>A DSL</strong>. OpenAI Evals and Promptfoo both have YAML-driven fixture formats. My fixtures are Python dataclasses. They’re slightly more verbose, but they’re real code I can refactor and that my IDE understands.</p><p>- <strong>A full framework like Braintrust or DeepEval</strong>. My eval is a few hundred lines and I understand every one of them. The cost of growing beyond that is a real consideration I’ll revisit at 50 fixtures, not 6.</p><p>The eval uploads to Langfuse as a session, which means I get the score history <em>and</em> the actual narrative output for every fixture across every run. Browsing <em>what did this fixture produce 30 runs ago vs today</em> is suddenly a 2-click operation.</p><h4>5. Restate: when a 10-second LLM call dies halfway</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*b5BtsubThMk9w_fiaTvvPA.png" /><figcaption><em>Restate UI — 5 durable steps of one chapter workflow, each replayable from journal on worker restart.</em></figcaption></figure><p>Restate is the optional one. Default off (<em>USE_RESTATE=false</em>). I built the path because I wanted to know if my chapter pipeline could survive a worker crash mid-generation, and the answer with the default in-process design was <em>no</em>.</p><p>The problem: a chapter generation is 5 steps spread over 10–15 seconds. If the process crashes in the middle (OOM, SIGKILL during deploy, hung HTTP socket), the player sees a half-streamed narrative and a 500 error, the LLM tokens are wasted, and there’s nothing to retry against because no state was committed.</p><p>Restate is a Rust-based durable execution engine. You declare a workflow handler and run each step inside it:</p><pre><br>chapter_service = restate.Workflow(&quot;ChapterWorkflow&quot;)<br><br>_RETRY_LLM        = restate.RunOptions(max_attempts=3, initial_retry_interval=...)<br>_RETRY_PURE_CPU   = restate.RunOptions(max_attempts=2)<br>_RETRY_NO_RETRIES = restate.RunOptions(max_attempts=1)  # DB writes — never duplicate<br><br>@chapter_service.main()<br>async def run(ctx: restate.WorkflowContext, payload: dict) -&gt; dict:<br>    context          = await ctx.run_typed(&quot;load_context&quot;,     _step_load)<br>    chapter_data     = await ctx.run_typed(&quot;generate_chapter&quot;, _step_generate, _RETRY_LLM)<br>    validate_result  = await ctx.run_typed(&quot;validate&quot;,         _step_validate, _RETRY_PURE_CPU)<br>    verifier_result  = await ctx.run_typed(&quot;verify&quot;,           _step_verify,   _RETRY_PURE_CPU)<br>    return             await ctx.run_typed(&quot;persist&quot;,          _step_persist,  _RETRY_NO_RETRIES)<br></pre><p>Each <em>ctx.run_typed</em> writes its result to a RocksDB journal before continuing. If the worker crashes between steps, the next worker restart picks up from the last committed journal entry. No duplicate Chapter rows, no wasted LLM spend, no half-state.</p><p>Two design choices I want to highlight:</p><p>- <strong>persist has max_attempts=1</strong>. Every other step retries (LLM rate limit → exponential backoff, validator failure → twice, etc.). Persist does not, because DB writes are not idempotent — a retry would insert a duplicate Chapter row. If persist fails the whole workflow fails, and that’s the right call.</p><p>- <strong>The narrative still streams via Redis sideband.</strong> Restate’s journal is JSON only — you can’t stream tokens through it. So while the workflow runs out-of-process, the <em>generate_chapter</em> step writes the running narrative to <em>redis://chapter:partial:{workflow_id}</em> every partial, and the API-side bridge polls it every 300ms. Restate-mode tokens arrive a bit chunkier than direct mode, but the SSE wire format is identical to the player.</p><h4>What I’d do differently</h4><p>- <strong>Stand up Langfuse on day 1, not later.</strong> I shipped Part 1’s engine with zero observability and paid for it with slow iteration. The cost of <em>set up an OTel exporter + Langfuse container </em>up front is way less than the cost of trying to retrofit observability into a multi-stage async pipeline later.</p><p>- <strong>Write the eval harness before writing the third prompt iteration.</strong> I changed the chapter prompt maybe 12 times across Part 1’s dev cycle. Without an eval, every change was a vibe check. With an eval, every change is a number.</p><p>- <strong>Don’t enable Restate until you actually need it</strong>. Direct mode is faster (continuous streaming vs 300ms Redis polling), simpler (one process, not three), and 80% of dev work is fine without journal-level durability. Restate becomes worth it when you have <em>paying</em> users whose LLM spend you can’t afford to lose.</p><h3>What’s next</h3><p>The next set of TODOs to be considered,</p><p>- <strong>Multi-tenancy</strong>. Right now every playthrough shares one Anthropic key (or one Ollama instance). For real users, I need per-tenant rate limiting, per-tenant cost attribution, and per-tenant memory isolation.</p><p>- <strong>Prompt A/B testing</strong>. Langfuse supports it but I haven’t wired it. The vision: ship a new prompt to 5% of playthroughs, watch eval scores + player retention diverge, promote or roll back.</p><p>- <strong>A real CI gate</strong>. The eval runs in CI today as <em>continue-on-error</em>. Tightening that to a hard gate is the easy part; the hard part is making sure flaky <em>llm_judge</em> runs don’t false-fail.</p><p>The repo is at <a href="https://github.com/ZaphyrRobin/Mythwright">Mythwright</a>, all of the above is there, including the <em>docs/</em> folder where I keep the deep-dive notes. If you’re building an LLM app and any of this resonates, I’d love to compare notes.</p><h3>Appendix</h3><p>Part 1: <a href="https://medium.com/@chongyao.robin/from-prompt-to-playthrough-building-an-ai-interactive-fiction-engine-47cef1a7e59b?postPublishedType=repub">From Prompt to Playthrough — I: Building an AI Interactive Fiction Engine</a></p><p>Github: <a href="https://github.com/ZaphyrRobin/Mythwright/?source=post_page-----47cef1a7e59b---------------------------------------">https://github.com/ZaphyrRobin/Mythwright/</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e5c70db04a62" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[From Prompt to Playthrough — I: Building an AI Interactive Fiction Engine]]></title>
            <link>https://medium.com/@chongyao.robin/from-prompt-to-playthrough-building-an-ai-interactive-fiction-engine-47cef1a7e59b?source=rss-a3d8293bb683------2</link>
            <guid isPermaLink="false">https://medium.com/p/47cef1a7e59b</guid>
            <category><![CDATA[ai-agent]]></category>
            <category><![CDATA[ai-fiction]]></category>
            <category><![CDATA[memory-management]]></category>
            <dc:creator><![CDATA[Chong Yao]]></dc:creator>
            <pubDate>Wed, 15 Apr 2026 03:36:07 GMT</pubDate>
            <atom:updated>2026-05-22T16:05:19.951Z</atom:updated>
            <content:encoded><![CDATA[<h3>Introduction</h3><p>I’ve been a web-novel reader for years. <strong>Qidian </strong>and its English sister <strong>Webnovel</strong> publish on the order of millions of words a day across thousands of active serials, and <strong>Royal Road</strong> has built a thriving second home for English-original serial fiction — LitRPG, progression fantasy, the works. Between the two, that’s where most of my reading hours go.</p><p>If you’ve ever binged a long-running serial — 800 chapters in, your favorite character finally about to make their move — you know the feeling: <em>this side-arc is dragging, can we just push the main story forward?</em> For as long as fiction has existed that frustration had no answer. The author writes at their pace; you read at theirs.</p><p>LLMs change that. The reader can now choose which branch to push forward, dive into a side arc when it’s interesting, or skip it when it isn’t. Think <strong>Dragon Age</strong>, <strong>Skyrim</strong>, or <strong>The Witcher</strong> — open-world RPG agency — but text-first, story-first, with the main plot always a click away instead of buried under combat.</p><p>This isn’t theoretical. The category has a real history and real money behind it:</p><ul><li><strong>May 2019</strong> — <em>AI Dungeon</em> launches as a hackathon project on GPT-2. After the December 2019 relaunch on the full GPT-2 model, it hit 100K players in a week and 1.5M by mid-2020.</li><li><strong>2021</strong> — <em>NovelAI</em> launches, focused on writers and long-context memory.</li><li><strong>Sep 2022</strong> — <em>Character.AI</em> launches; peaks at <strong>28M monthly active users</strong> in mid-2024.</li><li><strong>Aug 2024</strong> — Google effectively re-acquires Character.AI’s team and tech in a deal valued around <strong>$2.7B</strong>.</li><li><strong>2024–2026</strong> — Explosion of platforms: <a href="https://aidungeon.com/">AI Dungeon</a>, <a href="https://novelai.net/">NovelAI</a>, <a href="https://character.ai/">Character.AI</a>, <a href="https://dreamgen.com/">DreamGen</a>, <a href="https://feelin.ai/">Feelin</a>, Worldsmith, KoboldAI…</li></ul><p>The market is following the players. Interactive storytelling apps are projected to grow from <strong>$2.5B (2025) to $4.7B by 2033</strong>. The broader AI-companion category is on a path from <strong>$38B to $436B by 2034 (~31% CAGR)</strong>. Over <strong>65% of gamers under 35</strong> now prefer story-rich titles.</p><p>The industry hasn’t settled on a name yet — <em>AI interactive fiction</em>, <em>living stories</em>, <em>sandbox fiction</em>, <em>adaptive storytelling</em> are all in circulation. I’ll just call it <strong>AI interactive fiction</strong>: a category that didn’t exist before LLMs and is now ~6 years old as a real consumer product.</p><p>Every time I played one of these, the same engineering questions kept nagging at me:</p><ul><li>How does the system <strong>keep the main plot moving</strong> instead of letting the user (or the LLM itself) drift and never pull back?</li><li>How do you handle <strong>rollback</strong> — if the user regrets the last choice, can they unwind it without losing everything downstream?</li><li>How do you manage <strong>short-term, medium-term, and long-term memory</strong> so the story remembers what mattered and forgets what didn’t?</li><li>How do you let the user <strong>pick the cheapest LLM</strong> that’s still good enough — or bring their own key?</li></ul><p>So I built a demo app called Mythwright to find out. This article is a deep dive into how it works.</p><h3>System arch</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*i2jFnfxGIDJtZdFhc2osmQ.png" /><figcaption>High level architecture</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*fLEfxhFHDfPIoqREbivnKw.png" /><figcaption>Storage layer</figcaption></figure><pre><br>Player picks option (1 of 4)<br>         │<br>         ▼<br>┌────────────────────────────────────────────────────────┐<br>│ 1. Context Builder   → world + stats + memory window   │<br>│ 2. Drift / Gravity   → 1–2:0.0  3–4:0.3  5+:0.7        │<br>│ 3. LLM Router        → Ollama · Mistral · Claude       │<br>│ 4. Validator         → 3 retries, silent patches       │<br>│ 5. State Updater     → Postgres + Redis writes         │<br>│                        STM→LTM @ 0.7                   │<br>│                        LTM→PM on keyword match         │<br>│ 6. SSE Stream        → narrative_token → complete      │<br>└────────────────────────────────────────────────────────┘<br>         │<br>         ▼<br>   Chapter rendered</pre><p>Mythwright splits cleanly into two halves: a <strong>creator side</strong> for world-building and a <strong>player side</strong> for playthroughs. Creators define the world’s rules, map, characters, and story arcs once. Players then drop into that world and drive the story forward — every chapter they read is generated on demand by a 6-step pipeline.</p><h3>Workflow</h3><h4>Step 1 — Assemble Context</h4><p>Load the world’s system_prompt, resolve the player&#39;s current location from the Location table, pull current_stats, and extract the chosen option text + direction tag from the previous chapter.</p><h4>Step 2 — Build Memory Window</h4><p>The memory manager fetches three tiers and concatenates them into a &lt;memory_window&gt; block:</p><ul><li><strong>PM (Permanent)</strong> — always included. Cached in Redis with 10-min TTL.</li><li><strong>LTM (Long-Term)</strong> — entries with importance_score &gt;= 0.7. Cached with threshold-versioned keys, 5-min TTL.</li><li><strong>STM (Short-Term)</strong> — last N entries from Redis ZSET, where N = player’s memory_length setting (default 20).</li></ul><h4>Step 3 — Evaluate Story Rules &amp; Drift</h4><p>The engine checks all StoryRule entries against current state. If a rule’s trigger_condition matches, its effect gets injected into the prompt.</p><p>Separately, it computes <strong>narrative gravity</strong> from the player’s drift count (chapters since last main_arc choice). At drift 3–4 the gravity is 0.3 (gentle hints); at 5+ it’s 0.7 (the world itself pulls the player back to the main plot).</p><h4>Step 4 — Call LLM (JSON mode)</h4><p>The LLM router resolves the player’s chosen model to a provider (Ollama local or Claude API). The full prompt is sent in <strong>JSON mode</strong> (non-streaming) so the response is a structured object.</p><p>Provider routing:</p><ul><li>Ollama: mistral</li><li>Claude: claude-sonnet, claude-haiku, claude-opus — API key passed per-request, never stored</li></ul><h4>Step 5 — Validate &amp; Patch</h4><p>The ChapterValidator applies tiered error handling (max 3 retries):</p><ul><li><strong>Hard errors (retry):</strong> invalid JSON, missing required fields, dead characters appearing alive</li><li><strong>Soft patches (silent fix):</strong> pad/truncate to exactly 4 options, clamp HP to [0,100], snap unreachable location to nearest valid, ensure at least one option tagged main_arc</li></ul><h4>Step 6 — Update State &amp; Stream</h4><p>In a single transaction: create Chapter row with full snapshot, update playthrough stats/location, write STM to Redis, check memory promotions, advance arc progress.</p><p>The narrative is then streamed to the client as SSE narrative_token events (3 chars at 20ms intervals for a typing effect), followed by narrative_end and a final chapter_complete payload with stats, options, and memory state.</p><h3>Data model</h3><pre>## Creator Side<br><br>- **World** — title, description, `world_rules` (JSON), `map_data` (JSON), `system_prompt` (base prompt for all chapters)<br>- **Location** — semantic ID, name, description, atmosphere, connections (adjacency graph), danger_level, items<br>- **CharacterTemplate** — name, role (NPC/companion/antagonist), personality, stats, schedule, knowledge, secrets<br>- **StoryArc** — ordered waypoints (plot beats), priority, `is_main` flag for main quest vs side quest<br>- **StoryRule** — trigger_condition (JSON state match) + effect (prompt injection). Fires when chapter/location/stats match.<br>- **Gift** — player character traits rolled at creation. Weighted by rarity: common 60%, rare 30%, legendary 10%.<br><br>## Player Side<br><br>- **Playthrough** — player name, talents, gifts, current_stats (JSON), current_location, chosen LLM model, configurable `memory_length`<br>- **Branch** — parent_branch, fork_point_chapter, is_active. Enables non-destructive rollback.<br>- **Chapter** — narrative_text, stats_snapshot, 4 options (each tagged with direction), chosen_option, `snapshot` (full state for rollback)<br>- **ArcProgress** — tracks current_waypoint index and `drift_count` (chapters since last main_arc choice)<br>- **Memory** — type (STM/LTM/PM), content, importance_score, `base_importance_score` (immutable anchor for decay), chapter_created</pre><h3>Prompt Assembly</h3><pre>&lt;world&gt;        rules, tone, constraints<br>&lt;location&gt;     name, atmosphere, danger_level<br>&lt;stats&gt;        hp, power_level, inventory, companions<br>&lt;memory&gt;<br>  permanent:   covenants, origin, oaths<br>  long_term:   [score ≥ 0.7] major events<br>  short_term:  last N chapter summaries<br>&lt;rules&gt;        matched story constraints<br>&lt;arc&gt;          next waypoint + drift<br>&lt;gravity&gt;      0.0 – 1.0<br>&lt;choice&gt;       player&#39;s selected option<br><br>&lt;output&gt; JSON:<br>  narrative          1500–2500 chars, 2nd person<br>  stats_update       hp, location, inventory<br>  options[4]         {id, label, hint, direction}<br>                     directions: main_arc | exploration | side_quest | character_dev<br>  memory_summary     one line → STM<br>  importance_score   0.0 – 1.0</pre><h3>Three-Tier Memory System</h3><p>The most interesting design decision. Memory mimics human recall: short-term fades fast, important events persist, and core identity never disappears. It’s essentially a sliding context window with importance-based promotion — cache eviction policies, but for narrative.</p><pre>              ┌──────────────────────────┐<br>              │   LLM Response           │<br>              │   memory_summary + score │<br>              └────────────┬─────────────┘<br>                           ▼<br> ┌─ STM (Redis ZSET) ────────────────────────────────────┐<br> │ [Ch.1|0.4] [Ch.2|0.8] [Ch.3|0.3] ... [Ch.N+1] ◄ NEW   │<br> │                                                       │<br> │ count &gt; memory_length → evaluate oldest:              │<br> └────┬──────────────────────┬───────────────────────────┘<br>      │                      │<br>  score &lt; 0.7            score ≥ 0.7<br>      │                      │<br>      ▼                      ▼<br>   EVICTED            PROMOTE to LTM<br>                            │<br>                            ▼<br> ┌─ LTM (PostgreSQL + Redis cache) ──────────────────────┐<br> │ Decay: current = base * 0.95 ^ (age_chapters / 10)    │<br> │                                                       │<br> │ [0.81] &quot;Dragon battle&quot;     age:10  → stays            │<br> │ [0.68] &quot;Elven alliance&quot;    age:20  → fading (&gt; 0.3)   │<br> │ [0.25] &quot;Ancient ruins&quot;     age:50  → EVICTED (&lt; 0.3)  │<br> └───────────────────────────────────────────────────────┘<br><br> Keyword match on memory_summary:<br>   &quot;covenant | death | origin | god | oath | permanent&quot;<br>           │<br>           ▼<br> ┌─ PM (Permanent) ──────────────────────────────────────┐<br> │ score: max(importance, 0.95)  │  no decay │ no evict  │<br> │                                                       │<br> │ &quot;Origin: born in shadow realm&quot;                        │<br> │ &quot;Covenant: protect Migran until world ends&quot;           │<br> │ → Always in prompt regardless of memory_length        │<br> └───────────────────────────────────────────────────────┘</pre><h3>Tech Difficulties</h3><h4>1. Subplot &amp; Drift Management</h4><p>Open-world interactive fiction has a fundamental tension: players can ignore the main plot indefinitely. The system solves this with <strong>narrative gravity</strong> — an escalating pull that increases the longer the player drifts from the main arc.</p><h4>How it works</h4><p>The ArcProgress model tracks a drift_count — chapters since the player last chose an option tagged main_arc. The system converts drift into a gravity float:</p><ul><li><strong>Drift 0–2:</strong> gravity 0.0 — free exploration, no interference</li><li><strong>Drift 3–4:</strong> gravity 0.3 — the LLM weaves subtle hints into the environment</li><li><strong>Drift 5+:</strong> gravity 0.7 — the world reacts directly: NPCs bring news, earthquakes happen, the main plot comes to the player</li></ul><p>Each option in the LLM output is tagged with a direction (main_arc, side_quest, exploration, character_development). The validator enforces that at least one option is always main_arc. When the player picks it, drift resets and the arc&#39;s waypoint index advances.</p><p>This feels natural to the player — they never see a forced option, but the world itself keeps pulling them back. Like how open-world games escalate main quest urgency the longer you ignore it.</p><h4>2. Rollback via Branch Forking</h4><p>Players can rewind to any previous chapter. The system uses non-destructive branching — no data is ever deleted.</p><pre> Player at Ch.5, rolls back to Ch.3:<br><br> Branch A (inactive, preserved)      Branch B (active, NEW)<br> ├─ Ch1  investigate                 │ parent: A, fork: Ch3<br> ├─ Ch2  fight                       │<br> ├─ Ch3  negotiate  ◄─ fork          └─ Ch3+ regenerated<br> ├─ Ch4  flee                           (player picks differently)<br> └─ Ch5  &quot;Dead end&quot;<br><br><br> Ch3 snapshot restores state:<br>   stats, location, STM, LTM/PM active flags<br><br><br> After more rollbacks:<br><br>   A ─ Ch1..Ch5            (inactive)<br>        └─► B ─ Ch3..Ch4    (inactive)<br>             └─► C ─ Ch3..  (active)<br><br> Every branch preserved — switch anytime.</pre><h4>How rollback works</h4><p>When a player decides chapter 7 was a mistake and wants to fork from chapter 4, the engine does five things:</p><ol><li><strong>Deactivate</strong> every existing branch on this playthrough (one atomic UPDATE).</li><li><strong>Create a new branch</strong> with parent_branch_id pointing back to the old one and fork_point_chapter = 4.</li><li><strong>Copy chapters 1–4</strong> from the old branch into the new branch.</li><li><strong>Restore state</strong> from chapter 4’s snapshot JSON — stats, location, active memory IDs, the lot.</li><li><strong>Rebuild Redis STM</strong> from the copied chapters so the next generation has the right context window.</li></ol><h4>Why fork instead of delete?</h4><p>Three reasons, in order of importance:</p><ul><li><strong>Players change their minds.</strong> “Wait, chapter 4a was actually better” should always be possible. Branches are first-class — you can switch back at any time.</li><li><strong>History is data.</strong> Preserving every branch turns rollback into an analytics signal: <em>which chapters cause the most rollbacks?</em> That’s gold for tuning the story engine.</li><li><strong>No lossy operations.</strong> Deleting forward state means reversing LTM promotions, un-decaying importance scores, and resurrecting evicted memories. Forking sidesteps every one of those.</li></ul><p><strong>💡 The trick:</strong> every chapter stores a full snapshot JSON — {stats, location, stm_entries, ltm_entry_ids, pm_entry_ids}. Rollback is just <em>&quot;restore from snapshot.&quot;</em> No state reconstruction, no replay, no reversal logic. The complexity of multi-tier memory disappears the moment you treat each chapter as a checkpoint.</p><h4>3. Hot-Swapping LLM Models Mid-Playthrough</h4><p>A player can switch from Mistral to Claude between any two chapters via PATCH /api/playthroughs/{id}/settings. The new model has zero conversation history with this story — so how does it pick up seamlessly?</p><h4>Why this works: the prompt IS the context</h4><p>Traditional chatbots keep state in the LLM’s conversation thread — swap the model, lose the thread. Mythwright doesn’t work that way. <strong>Every chapter generation is a single, self-contained LLM call with no conversation history.</strong> The entire context is rebuilt from scratch each time, from durable state:</p><ul><li>World prompt + location → PostgreSQL</li><li>Stats, inventory, companions → Playthrough.current_stats</li><li>Memory window (PM + LTM + STM) → Redis + PostgreSQL</li><li>Story rules, arc direction, narrative gravity → derived from game state</li><li>Previous chapter’s chosen option → stored on the Chapter row</li></ul><p>The new model receives the exact same prompt the old one would have. Continuity lives in the memory system, not the model.</p><h4>What the model switch actually changes</h4><p>The player is trading one set of characteristics for another — never the story itself:</p><ul><li><strong>Writing style</strong> — Claude leans literary; Mistral is more direct. Tone shifts; plot doesn’t.</li><li><strong>Instruction-following</strong> — some models nail the JSON schema better. The validator catches the rest.</li><li><strong>Cost vs. quality</strong> — local Ollama for casual chapters, Claude for climactic ones.</li></ul><p>This is the architectural payoff of the memory system. Because context lives externally — in Redis and PostgreSQL, not the LLM’s context window — <strong>the model becomes a stateless, replaceable execution engine</strong>. Swap it anytime. The story’s memory belongs to the system, not the model.</p><h4>4. How streaming works here (SSE)</h4><p>The LLM generates a chapter token-by-token, and we want players to see those tokens as they arrive — not wait 5 seconds for the full response. We use <strong>Server-Sent Events (SSE)</strong>: a one-way HTTP stream from server to browser, dead simple compared to WebSockets.</p><p>No library needed — FastAPI’s StreamingResponse on the server, browser&#39;s native EventSource on the client.</p><pre>Browser                    FastAPI                    LLM<br> ───────                    ───────                    ───<br>    │                          │                        │<br>    │  POST /chapters/generate │                        │<br>    ├─────────────────────────►│                        │<br>    │                          │  stream tokens         │<br>    │                          ├───────────────────────►│<br>    │                          │                        │<br>    │  event: narrative_token  │◄── &quot;The&quot;               │<br>    │  data: {&quot;text&quot;:&quot;The&quot;}    │                        │<br>    │◄─────────────────────────┤                        │<br>    │                          │◄── &quot; dragon&quot;           │<br>    │  event: narrative_token  │                        │<br>    │  data: {&quot;text&quot;:&quot; dragon&quot;}│                        │<br>    │◄─────────────────────────┤                        │<br>    │                          │     ... (many tokens)  │<br>    │                          │                        │<br>    │                          │  validate + persist    │<br>    │                          │                        │<br>    │  event: chapter_complete │                        │<br>    │  data: {id, options,...} │                        │<br>    │◄─────────────────────────┤                        │<br>    │                          │                        │<br>    │  [connection closes]     │                        │</pre><p>Two event types do all the work:</p><ul><li><strong>narrative_token</strong> — fired per token during generation. The frontend appends to the visible text, giving the typewriter effect.</li><li><strong>chapter_complete</strong> — fired once at the end, after the validator passes and the chapter is written to Postgres. Carries the final chapter ID, the 4 options, updated stats, and memory IDs.</li></ul><p>If the validator rejects the LLM output (max 3 retries), the stream just keeps going — the player sees a fresh set of tokens replace the old ones. If it fails all 3, we emit a chapter_error event and the client surfaces a retry button.</p><p><strong>Why not WebSockets?</strong> Bidirectional comms would be overkill — the player only sends one thing (their choice), which is a normal POST. Everything else is server-push. SSE auto-reconnects, works through proxies, and needs zero extra infrastructure</p><h3>Demo Preview</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*KcnYoKhN0iI-maO8beIxLg.png" /></figure><h4>Creator</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ViRmT2hyeRB4nUh-lTxLUw.png" /></figure><h4>Player</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*pUrmiFWT0zpdZTeQ0Hh06g.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*svxEpXdJpFNHn3nfwAsoRw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*XmTptCGQnW6ajKBdbzQLpQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0eSMTdhHVXCA6cv0tEB9dw.png" /></figure><h3>TODO</h3><ul><li><strong>Auth &amp; accounts</strong> — JWT auth, per-user ownership, OAuth for creators. API keys move from per-request to a per-user encrypted vault.</li><li><strong>Billing &amp; quotas</strong> — real per-provider token counts, a balance ledger, and quota enforcement on the next-chapter endpoint.</li><li><strong>World marketplace</strong> — public discovery, ratings, and fork-a-world remixing. Needs an ACL layer on the World model.</li><li><strong>Rate limiting &amp; abuse</strong> — per-user and per-IP limits on chapter generation (the expensive endpoint), CAPTCHA on signup, moderation on creator prompts.</li><li><strong>Observability</strong> — correlation IDs across the SSE stream, Prometheus metrics per LLM provider, and analytics on which chapters trigger rollbacks.</li></ul><h3>Conclusion</h3><p>Now go enjoy your own interactive fiction.</p><p>I strongly believe this is a blue-ocean market. For as long as fiction has existed, web novels and tabletop D&amp;D and text-adventure games lived on completely separate islands — different audiences, different formats, different economics. LLMs are quietly collapsing the distance between them. The novel becomes playable. The D&amp;D session becomes readable. The text adventure becomes infinite.</p><p>What’s emerging isn’t just a new product category — it’s a new medium, one that simply couldn’t exist before generative AI made on-demand prose cheap and good enough. Qidian and Webnovel showed there’s a global appetite for serial fiction at industrial scale. AI Dungeon and Character.AI showed players will spend hours with stories they help shape. The gap between those two worlds is exactly where the next billion-dollar storytelling platform gets built.</p><p>Mythwright is my small bet on what the engine for that future looks like — worlds as data, chapters as checkpoints, memory as a tiered system, and the LLM as a swappable component instead of a black box. The code is open and the architecture is documented. If any of this resonates, fork it, remix it, build something better.</p><p>The story’s just getting started.</p><p><strong>Github</strong>: <a href="https://github.com/ZaphyrRobin/Mythwright/">https://github.com/ZaphyrRobin/Mythwright/</a></p><h3>Follow up</h3><p>Part II: <a href="https://medium.com/@chongyao.robin/from-prompt-to-playthrough-ii-making-the-ai-layer-observable-e5c70db04a62">From Prompt to Playthrough — II: Making the AI Layer Observable</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=47cef1a7e59b" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Demystifying a Personal AI Trading Agent — Architecture for Real-Time Decision Loops]]></title>
            <link>https://medium.com/@chongyao.robin/demystifying-personal-ai-trading-agent-crypto-edge-in-prediction-markets-3714cceaa4bf?source=rss-a3d8293bb683------2</link>
            <guid isPermaLink="false">https://medium.com/p/3714cceaa4bf</guid>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[ai-trading-bot]]></category>
            <category><![CDATA[prediction-markets]]></category>
            <category><![CDATA[polymarket]]></category>
            <category><![CDATA[crypto]]></category>
            <dc:creator><![CDATA[Chong Yao]]></dc:creator>
            <pubDate>Thu, 12 Mar 2026 22:13:45 GMT</pubDate>
            <atom:updated>2026-05-13T01:05:55.548Z</atom:updated>
            <content:encoded><![CDATA[<h3>TL;DR</h3><p>Crypto prediction markets are a different animal from traditional exchanges. You’re not trading the asset — you’re trading a binary question: will BTC close above $68,000 tonight? That distinction creates a specific type of mispricing that doesn’t exist in spot or futures markets. This post covers how I built a system to find and trade those mispricings automatically, what edge sources are durable, and where AI fits (and doesn’t fit) in the architecture.</p><h3>What Is a Prediction Market?</h3><p>A prediction market is an exchange where you trade contracts on the outcome of real-world events. Each contract settles at $1 if the event happens, $0 if it doesn’t. The market price at any moment reflects the crowd’s collective estimate of the probability — a contract trading at 67¢ implies a 67% chance of YES.</p><p>This is structurally identical to a binary option. In traditional finance, a binary (or “digital”) option pays a fixed amount if the underlying closes above a strike at expiry — and zero otherwise. The pricing mechanics are the same: both are bets on a binary outcome at a future point in time, both have a defined max profit and max loss, and both can be modeled using the same probability frameworks.</p><pre>Binary Option (finance):<br>  Pays $1 if S_T &gt; K at expiry T<br>  Price = e^(-rT) × P(S_T &gt; K)     ← risk-neutral probability<br><br>Prediction Market Contract (prediction_market):<br>  Pays $1 if event resolves YES<br>  Price = P(event = YES)             ← market-implied probability</pre><p>The key difference is what determines the underlying probability. For a vanilla binary option on a stock, the probability is derived from the Black-Scholes framework using implied vol. For a prediction market crypto contract, the market sets the price through order flow — and that crowd-sourced price is often wrong in systematic, exploitable ways.</p><p>The other meaningful difference is who participates. Equity binary options are primarily traded by institutional desks with sophisticated pricing models. Prediction markets on prediction market attract a much broader mix — retail traders, hobbyists, and directional speculators — which is precisely why the mispricings persist.</p><p>Why crypto specifically? Among the asset classes available on prediction market — weather, economics, sports, politics, crypto — crypto markets are the least efficiently priced relative to the underlying asset. The weather and economic markets attract professional forecasters who actively arbitrage model mispricings. Crypto threshold markets, by contrast, are dominated by retail participants who trade on sentiment and momentum rather than calibrated probability estimates. The gap between what a lognormal vol model says and what the market prices is structurally wider in crypto than anywhere else on the platform.</p><p>There are three distinct sources of that disagreement:</p><p><strong>1. Longshot Bias</strong></p><p>The most robust and well-documented edge in prediction markets. Contracts priced at 5–15¢ (the “tails”) are systematically overpriced relative to their true probability of settling YES. This is the crypto market equivalent of the favorite longshot bias in horse racing: retail participants overestimate the probability of unlikely outcomes.</p><p>The practical implication: selling tail contracts — those priced well below 50¢ — captures a structural premium that doesn’t require any view on where BTC is going. You’re not predicting direction; you’re collecting the bias.</p><p><strong>2. Volatility Mismatch</strong></p><p>The market prices each contract using some implied view of how much the asset will move before expiry. When that implied view diverges from what a calibrated volatility model says — either too high or too low — there’s an edge.</p><p>In practice this means: when market-implied vol is elevated relative to recent realized vol (the same VRP signal from options markets, now applied to binary contracts), tail contracts become even more overpriced than their structural bias alone would suggest. When a fear spike hits BTC and OTM contracts reprice upward, the sell signals get better, not worse.</p><p><strong>3. Mean-Reversion on Extreme Moves</strong></p><p>After large, rapid price moves, crypto assets have a documented tendency to mean-revert over short horizons. When BTC drops 8% in 4 hours, the probability that it closes below a strike that was deep OTM an hour ago has jumped — but often overshoots relative to the actual settlement probability. The market prices in fear faster than the underlying distribution warrants.</p><h3>High-Level Architecture</h3><p>The system separates into four layers: data ingestion, probability modeling, risk gating, and execution + feedback.</p><pre><br>┌──────────────────────────────────────────────────────────────┐<br>│                        DATA LAYER                            │<br>│                                                              │<br>│   Binance/CoinGecko        prediction market Market API      │<br>│   (spot price, vol,        (live contract prices,            │<br>│    24h change, volume)      strikes, order book)             │<br>└──────────┬────────────────────────┬─────────────────────────┘<br>           │                        │<br>           ▼                        ▼<br>┌──────────────────────────────────────────────────────────────┐<br>│                   PROBABILITY MODEL                          │<br>│                                                              │<br>│   Per-asset volatility estimation                            │<br>│   → lognormal distribution over settlement horizon           │<br>│   → P(price &gt; strike) for each contract                      │<br>│                                                              │<br>│   IV Spike Detector                                          │<br>│   → monitors OTM contract prices as fear proxy               │<br>│   → detects vol regime shifts in real time                   │<br>│                                                              │<br>│   Calibration Engine                                         │<br>│   → adjusts thresholds based on historical accuracy         │<br>└──────────┬───────────────────────────────────────────────────┘<br>           │<br>           ▼<br>┌──────────────────────────────────────────────────────────────┐<br>│                      RISK GATES                              │<br>│                                                              │<br>│   [1] Edge threshold gate  → min edge required to trade      │<br>│   [2] Longshot bias gate   → price-tier-aware thresholds     │<br>│   [3] Timing gate          → lead time to settlement         │<br>│   [4] Risk Manager         → position limits, kill switch    │<br>│                                                              │<br>│   All 4 must pass — any gate can veto                        │<br>└──────────┬───────────────────────────────────────────────────┘<br>           │<br>           ▼<br>┌──────────────────────────────────────────────────────────────┐<br>│                     EXECUTION                                │<br>│                                                              │<br>│   AutoExecutor → prediction market API → limit order         │<br>│   Cross-strategy ranking (crypto vs weather vs options)      │<br>│   Volatility circuit breaker (pauses on black-swan moves)    │<br>│                     │                                        │<br>│                     ▼                                        │<br>│            CryptoSignalLog (SQLite)                          │<br>└──────────┬───────────────────────────────────────────────────┘<br>           │<br>           ▼<br>┌──────────────────────────────────────────────────────────────┐<br>│                   FEEDBACK LOOP (daily)                      │<br>│                                                              │<br>│   CalibrationEngine  → mispricing surface adjustments        │<br>│   MonteCarloSizer    → Kelly-fraction position sizing        │<br>│   EdgeDriftMonitor   → contract size multiplier              │<br>└──────────────────────────────────────────────────────────────┘</pre><h3>The Probability Model</h3><p>For each active prediction market event (daily or weekly), the scanner fetches all live strikes and computes a model probability for each one. The model answers: given current price, current volatility, and time to settlement — what is the true probability that the asset closes above this strike?</p><p>The volatility estimate is the key input. Getting it wrong directly destroys edge. The system uses three inputs in combination:</p><p>Current market data — the live prediction market orderbook itself contains an implied vol signal. Liquid strikes near 50¢ (at-the-money) are the most efficiently priced and serve as an anchor for calibrating the vol estimate.</p><p>Recent realized vol — how much has the asset actually moved over the past 1 and 7 days? This is the ground truth that market-implied vol should track. When the two diverge significantly, there’s a signal.</p><p>Vol calibration mode — the system can blend implied and realized estimates depending on market conditions. During normal regimes, market-implied vol is a better forward estimate. After large moves, realized vol is more informative.</p><p>The resulting probability estimate is compared to the prediction market market price. The difference — after fees — is the edge in basis points.</p><h3>Edge ≠ Signal</h3><p>Computing edge is easy. Knowing when to act on it is the harder problem.</p><p>A 200bps edge on a 5¢ contract is not the same as 200bps on a 45¢ contract. The tail contract has asymmetric risk: if it settles YES, the loss is (100 — price)¢ per contract — up to 95¢ on a 5¢ sell. Position sizing must account for this.</p><p>The system handles this through price-tier-aware thresholds. Contracts in different price ranges require different minimum edge levels to trade, reflecting the actual risk-adjusted EV rather than a flat bps cutoff. Deep tail sells (≤10¢) and mid-range sells (11–25¢) have separate position limits enforced in the auto-executor.</p><h3>The IV Spike Detector</h3><p>The IV Spike Detector runs continuously alongside the scanner. It monitors the average bid prices of OTM prediction market contracts as a real-time implied volatility proxy — the same signal that DVOL captures on Deribit, but extracted directly from prediction market prices.</p><p>When OTM contracts reprice significantly upward — people panic-buying protection against extreme moves — the detector fires. This triggers two effects:</p><p>1. Signal quality improves: the VRP is at its fattest during fear spikes, so sell signals generated during spikes have historically higher EV.</p><p>2. Thresholds relax: the minimum edge required drops because the base rate of tail contracts being mispriced goes up during panic.</p><p>This is the same logic as the options strategy: fear creates the best selling opportunities. The IV spike detector makes the system regime-aware rather than applying static thresholds regardless of market conditions.</p><h3>Risk Gates</h3><p>Four gates run in sequence. All must pass.</p><pre>Signal<br>  │<br>  ▼<br>[Gate 1] Edge Threshold<br>  │  Raw edge_bps &gt; calibrated minimum for this asset + price tier?<br>  │  → SKIP if insufficient edge after fees<br>  │<br>  ▼ PASS<br>[Gate 2] Longshot Bias Adjustment<br>  │  Price-tier-aware threshold — cheap contracts need more edge<br>  │  Timing multiplier — edge threshold rises closer to settlement<br>  │  (less time = less opportunity to be right = need better odds)<br>  │<br>  ▼ PASS<br>[Gate 3] Calibration Check<br>  │  Historical mispricing surface for this (price, hours_ahead) bucket<br>  │  Adjusts threshold based on observed accuracy in similar conditions<br>  │<br>  ▼ PASS<br>[Gate 4] Risk Manager<br>     Max concurrent positions per asset<br>     Max spend per cycle<br>     Tail-sell exposure limit (Gross exposure limit per position type.)<br>     Kill switch: win/loss streak circuit breaker<br>     → BLOCK if any hard limit hit</pre><p>The timing gate deserves explanation. A contract priced at 30¢ with 20 hours to settlement has very different risk from the same contract with 2 hours left. Near expiry, the probability is more “locked in” — the asset either is or isn’t going to cross the strike. Conversely, with plenty of time, mean-reversion can work for or against you. The threshold adjusts dynamically based on lead time.</p><h3>Execution and Position Sizing</h3><p>Once a signal clears all gates, the AutoExecutor places a limit order on prediction market. Position sizing uses a Kelly-fraction approach through the MonteCarloSizer:</p><p>For each (action, edge_bucket, lead_time_bucket) combination, the MC sizer has a learned Kelly fraction based on historical outcomes in similar conditions. The fraction is capped conservatively (never more than 2% of portfolio per trade) and scaled further by the <em>EdgeDriftMonitor</em> if recent edge quality has been declining.</p><p>For each group, 10,000 Monte Carlo paths are simulated by resampling historical returns. The raw Kelly fraction</p><pre>f = E[R] / Var[R]</pre><p>is discounted by the coefficient of variation of those paths:</p><pre>f* = (E[R] / Var[R]) × (1 - CV_edge), capped at 2%</pre><p>High CV means the edge is path-dependent and volatile — the discount shrinks position size accordingly. Naïve Kelly ignores this and can recommend dangerously large fractions. The table rebuilds daily as new settled trades accumulate.</p><p>The cross-strategy executor ranks crypto signals against weather and options signals by edge_bps and executes the best opportunities first within a shared spend budget. This means crypto signals compete for capital — they only execute if they’re the best use of the budget that cycle.</p><p>One additional protection specific to crypto: a volatility circuit breaker. If any tracked asset has moved more than 8% in 24 hours, crypto auto-execution pauses for that cycle. This prevents the system from trading into a black-swan event where the lognormal vol model underestimates tail risk.</p><h3>The Feedback Loop</h3><p>The calibration and feedback layer runs daily after settlement. Three components:</p><p><em>CalibrationEngine</em> maintains a mispricing surface: a grid indexed by price and time-to-settlement buckets.. For each cell, it tracks the difference between implied probability and realized frequency — how often did contracts in that bucket actually settle YES, versus what they were priced at? When a systematic bias is detected, it adjusts the edge threshold for that bucket at signal-generation time.</p><p>This is the system correcting for its own blind spots. If BTC weekly contracts at 15¢ have been settling YES more often than the model expects, the calibration engine tightens the threshold for that bucket — requiring more edge before the system sells there again.</p><p><em>MonteCarloSizer</em> groups historical trades by (action, edge_bucket, lead_time_bucket) and computes a Kelly fraction for each group.</p><p>The Kelly fraction is discounted by the coefficient of variation of observed PnL, which penalizes high-variance outcomes even if expected value is positive. This is a risk-aware Kelly implementation, not naïve Kelly.</p><p><em>EdgeDriftMonitor</em> watches whether the edge_bps on recent signals is declining relative to a rolling baseline. If it is, it applies a contracts multiplier below 1.0 — silently reducing position sizes before the decay becomes visible in PnL. It’s an early-warning system for market efficiency improvements at specific strike ranges.</p><h3>The Data Model</h3><pre>CryptoSignalLog<br>├── asset                  (BTC, ETH)<br>├── ticker                 (KXBTCD-xxxx)<br>├── strike_price           (threshold in USD)<br>├── action                 (sell_yes | buy_yes)<br>├── model_prob             ← our probability estimate<br>├── prediction_market_price_cents     ← market&#39;s implied probability (×100)<br>├── edge_bps               ← key selection criterion<br>├── hours_ahead            ← used by MC sizer + calibration engine<br>├── confidence<br>├── suggested_contracts<br>├── executed / settled<br>└── pnl_cents / settlement_price_usd</pre><p>Every field that matters for calibration is stored at signal creation time. Market conditions at entry drive performance analysis — not conditions at close.</p><h3>Early Results</h3><p>After several months of paper trading on prediction market’s demo environment, the directional signal is encouraging. A few patterns worth noting:</p><p>Sell_yes (tail sells) generate significantly more PnL per trade than buy_yes. This is consistent with the longshot bias thesis — the premium per contract is larger on the sell side, and win rates have held above the break-even threshold across both BTC and ETH.</p><p>Higher average edge at entry correlates with better realized outcomes, which validates that the edge computation is capturing something real rather than noise.</p><p>The caveat applies here: demo results don’t fully reflect live execution. Slippage, partial fills, and market impact in a real account will compress the numbers. The edge sources — longshot bias and vol mismatch — are academically documented and structurally sound, but the magnitude of the realized advantage in a live account remains to be confirmed.</p><h3>Where AI Fits</h3><p>AI is not in the hot path. Signal generation and execution are fully deterministic — no LLM is making trading decisions in real time.</p><p>AI is used in one place: the Fear &amp; Greed integration. The auto-executor reads the Bitcoin Fear &amp; Greed Index and applies regime-aware adjustments: e.g</p><pre>Extreme Fear (index &lt; 20):<br>  expand the spend budget (+50%),<br>  lower the crypto edge threshold (-100bps).<br>  Fear spikes create the best contrarian opportunities.<br><br>Extreme Greed (index &gt; 80):<br>  tighten the budget (-30%),<br>  raise thresholds (+200bps).<br>  Complacency compresses the mispricing the system exploits.</pre><p>This isn’t AI in the LLM sense — it’s a simple regime signal. But it reflects the same design philosophy as the options agent: quantitative signals tell you what the market is doing; sentiment and regime signals tell you whether to lean in or pull back.</p><p>The more interesting AI application would be natural-language synthesis of the calibration reports — turning the raw mispricing surface updates into a coherent daily narrative about which markets are becoming more or less efficient. That’s on the roadmap.</p><h3>What We Tried That Didn’t Work</h3><p>Not every idea made it into the system. A few approaches looked promising on paper and failed in practice — worth documenting because the failure modes are instructive.</p><p>Cross-platform arbitrage. The same event often trades on multiple prediction markets simultaneously with different prices — a clean arbitrage in theory. In practice, Polymarket is banned for US users, which rules out the most liquid alternative. Other platforms either lack the contract depth or the API access needed to execute both legs reliably. Cross-platform arb stays on the shelf until the regulatory picture changes.</p><p>Options cross-market arb. A related idea: use Alpaca’s options API to hedge a prediction market crypto position, or identify mispricings between the two. The problem is prediction market’s options market is too thin. There aren’t enough contracts, at enough strikes, with enough volume to support a strategy that depends on consistent two-sided liquidity. The signals would be sparse and the fills unreliable. Not worth building against a market that small.</p><p>Order flow and CVD-based signals. Cumulative Volume Delta (CVD) — tracking whether buyers or sellers are initiating more volume — is a legitimate edge signal in deep CLOB markets like CME futures or Binance perpetuals. The prediction market doesn’t publish the order-level data needed to compute it. The public API returns best bid/ask and volume, not individual trade direction. Without L2 book data, CVD-style signals can’t be computed reliably.</p><p>Full integer programming for internal arb. The Polymarket arbitrage literature describes using integer programming to find optimal portfolios across a large set of logically related contracts. At Polymarket’s scale — hundreds of correlated contracts on the same event — the combinatorial approach is necessary. The prediction market crypto markets typically have 20–30 strikes per asset per expiry. A simplified iterative optimizer covers the same ground without the solver dependency, and is easier to debug when something goes wrong.</p><p>Momentum-based directional signals. The intuitive approach: if BTC is up 5% today, buy YES on near-term strikes. The problem is that prediction market markets reprice quickly on large moves — by the time the directional signal is clear, the contracts have already adjusted. Momentum signals that work in spot markets arrive too late to be actionable in prediction markets, where price discovery is fast and contracts are short-dated. The vol mismatch approach — looking at <em>*how*</em> the market repriced relative to a calibrated model — captures the same information more reliably.</p><h3>Stack</h3><pre>Backend: FastAPI, SQLAlchemy, SQLite<br>Frontend: React, TypeScript, Vite, TailwindCSS<br>Market: prediction_market (crypto threshold markets)<br>Price data: Binance WebSocket + CoinGecko<br>AI: Anthropic Claude (regime synthesis, roadmap)<br>Scheduler: Custom in-process task runner</pre><h3>Summary</h3><p>Prediction markets create pricing inefficiencies that don’t exist in traditional asset markets — because participants are trading binary outcomes, not the underlying asset, and behavioral biases show up differently. The longshot bias is the most structural of these: reliably documented, mechanically exploitable, and persistent enough to build a system around.</p><p>The architecture separates three concerns cleanly: modeling the true probability, gating on risk-adjusted edge, and learning from settled outcomes. Each layer can improve independently, and none of them require AI to function — AI is reserved for tasks that genuinely need reasoning rather than computation.</p><p>A word on the future of this edge. As more AI agents enter prediction markets, mechanical mispricing will compress. Longshot bias will narrow as automated systems learn to sell tails. Vol mismatch signals will get arbed faster. This is already happening in equity options — decades of systematic vol selling have compressed the VRP. The same will happen here.</p><p>But human nature doesn’t change. Fear, overconfidence, recency bias, and the tendency to overprice dramatic outcomes are not bugs that get patched — they’re features of how people process uncertainty. The specific numerical edge will shift, but the structural fact that markets are priced by humans (and by AI agents trained on human behavior) means there will always be positive-expectation opportunities for a well-calibrated model. The edge moves; it doesn’t disappear. The job is to keep finding it.</p><h3>Resources</h3><p><a href="https://x.com/RohOnChain/status/2017314080395296995">https://x.com/RohOnChain/status/2017314080395296995</a></p><p><a href="https://x.com/Mikocrypto11/status/2030499843819368666">https://x.com/Mikocrypto11/status/2030499843819368666</a></p><p><a href="https://x.com/w1nklerr/status/2018453276279070952">https://x.com/w1nklerr/status/2018453276279070952</a></p><p><a href="https://en.wikipedia.org/wiki/Frank%E2%80%93Wolfe_algorithm">https://en.wikipedia.org/wiki/Frank%E2%80%93Wolfe_algorithm</a></p><p><a href="https://www.cultivatelabs.com/crowdsourced-forecasting-guide/how-does-logarithmic-market-scoring-rule-lmsr-work">https://www.cultivatelabs.com/crowdsourced-forecasting-guide/how-does-logarithmic-market-scoring-rule-lmsr-work</a></p><h3>Disclaimer</h3><p><em>All code is personal use. This is not financial advice.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3714cceaa4bf" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Demystifying Personal AI Trading Agent — Starting with Options]]></title>
            <link>https://medium.com/@chongyao.robin/demystifying-personal-ai-trading-agent-starting-with-options-04a18307dc6f?source=rss-a3d8293bb683------2</link>
            <guid isPermaLink="false">https://medium.com/p/04a18307dc6f</guid>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[ai-agent]]></category>
            <category><![CDATA[options-trading]]></category>
            <dc:creator><![CDATA[Chong Yao]]></dc:creator>
            <pubDate>Wed, 11 Mar 2026 22:53:08 GMT</pubDate>
            <atom:updated>2026-03-14T01:36:12.023Z</atom:updated>
            <content:encoded><![CDATA[<p><em>A deep-dive into the system architecture — from signal generation to self-calibrating feedback loops.</em></p><h3>TL;DR</h3><p>AI is quietly reshaping what individual engineers can build. A few years ago, building a real-money trading system with multi-source data ingestion, quantitative signal generation, and an AI reasoning layer would have required a hedge fund budget and a team of quants. Today a single engineer can assemble all of that on few days — with Claude, open brokerage APIs, and free macro data. Personal financial AI agents are becoming a legitimate category: not robo-advisors built by fintech companies, but purpose-built systems that individual engineers design, control, and iterate on themselves.</p><p>This post is an account of building one such agent — a personal low cost automated options trading system. It covers the architecture, where AI genuinely earns its keep, and how the system learns from its own trade history. This is one form this kind of personal agent can take.</p><h3>The Core Idea</h3><p>There’s a well-documented phenomenon in options markets: implied volatility (IV) consistently overstates how much an asset will actually move. The gap between what the market <em>prices in</em> and what <em>actually happens</em> is called the Volatility Risk Premium (VRP).</p><p><strong>Implied Volatility (IV)</strong> is the market’s forward-looking vol estimate, backed out from option prices via Black-Scholes:</p><pre>C_market = BS(S, K, T, r, σ_implied)<br>→ solve for σ_implied = IV</pre><p><strong>Realized Volatility (RV)</strong> is what actually happened over the past N days:</p><pre>RV = √(252/N × Σ(ln(Sᵢ/Sᵢ₋₁))²)</pre><p><strong>Volatility Risk Premium (VRP)</strong> is simply the gap:</p><pre>VRP = IV - RV</pre><p><strong><em>Empirically, VRP &gt; 0 most of the time — the market systematically overpays for protection.</em></strong></p><p>The strategy exploits this: when markets panic and IV spikes, sell a <em>bull put spread </em>— collect rich premium upfront, cap your maximum loss, and let time decay work in your favor.</p><p>The edge isn’t prediction. It’s identifying <em>when</em> the VRP is fat enough to justify the risk, and having systems disciplined enough to pass on everything else.</p><pre>Max Profit  = Net Credit Received<br>            = P_short_bid - P_long_ask<br><br>Max Loss    = (K_short - K_long) × 100 - Net Credit<br><br>Breakeven   = K_short - Net Credit<br><br>P&amp;L at expiry:<br>  if S_T &gt; K_short:           profit = Net Credit      (full premium)<br>  if K_long &lt; S_T &lt; K_short:  profit = S_T - K_short + Net Credit<br>  if S_T &lt; K_long:            profit = -Max Loss       (capped)</pre><p><strong>How VRP drives the edge:</strong></p><p>The theoretical fair value of a put uses RV as σ. The market prices it using IV. Since IV &gt; RV:</p><pre>Market Put Price  &gt; BS Fair Value (using RV)<br><br>Edge = Market Bid - BS_fair(σ = RV)<br>     = premium overpricing in dollar terms<br><br>Edge is normalized to the risk taken, expressed in basis points</pre><p>So you’re not predicting direction. You’re collecting the spread between what the market <em>charges</em> for vol and what vol <em>actually costs</em> — that gap is structurally positive in calm-to-fearful regimes, which is why the strategy has positive EV over time.</p><p>The IV/RV ratio tells you how fat that gap is right now:</p><pre>IV/RV ratio &lt; 1.0   →  no premium, skip<br>IV/RV ratio moderate →  elevated VRP, signal candidate<br>IV/RV ratio high     →  fear spike, maximum VRP — best entry window</pre><h3>High-Level Architecture</h3><p>The system separates into four concerns: data ingestion, signal generation, risk gating, and execution + feedback. AI is used selectively — for tasks that genuinely require reasoning, not just computation.</p><pre><br>┌──────────────────────────────────────────────────────────────┐<br>│                        DATA LAYER                            │<br>│                                                              │<br>│  Alpaca Options API   FRED Connector   News Aggregator       │<br>│       (prices,            (macro         (headlines,         │<br>│     chains, fills)      indicators)      sentiment)          │<br>└──────────┬────────────────┬──────────────────┬───────────────┘<br>           │                │                  │<br>           ▼                ▼                  ▼<br>┌──────────────────────────────────────────────────────────────┐<br>│                     AI AGENT LAYER                           │<br>│                                                              │<br>│   AgentOrchestrator  ──────────────────────────────────┐     │<br>│   (Claude Sonnet)                                      │     │<br>│                                                        │     │<br>│   Phase 1: GATHER  → fetch macro + market data         │     │<br>│   Phase 2: ANALYZE → run scoring tools                 │     │<br>│   Phase 3: DECIDE  → generate trade thesis             │     │<br>│   Phase 4: REPORT  → daily narrative synthesis         │     │<br>│                                                        │     │<br>│   DailySynthesizer  (Claude Sonnet)                    │     │<br>│   → macro + vol regime narrative report               ◄┘     │<br>└──────────────────────┬───────────────────────────────────────┘<br>                       │<br>                       ▼<br>┌──────────────────────────────────────────────────────────────┐<br>│                   SIGNAL GENERATION                          │<br>│                                                              │<br>│  Options Scanner (every 30 min)                              │<br>│  │                                                           │<br>│  ├─ Black-Scholes pricing → theoretical fair value           │<br>│  ├─ IV/RV ratio computation (30-day realized vol)            │<br>│  └─ Edge scoring → signal ranking                            │<br>│                          │                                   │<br>│                          ▼                                   │<br>│                   SellPutSignal{}                            │<br>│              short leg + long leg + net_credit               │<br>│              + edge_bps + iv_rv_ratio + annual_roi           │<br>└──────────────────────┬───────────────────────────────────────┘<br>                       │<br>                       ▼<br>┌──────────────────────────────────────────────────────────────┐<br>│                      RISK GATES                              │<br>│                                                              │<br>│   [1] Vol Regime Gate     → macro vol score check            │<br>│   [2] Per-Underlying Gate → calibrated IV/RV threshold       │<br>│   [3] Risk Manager        → position limits + kill switch    │<br>│                                                              │<br>│   All 3 must pass — any gate can veto                        │<br>└──────────────────────┬───────────────────────────────────────┘<br>                       │<br>                       ▼<br>┌──────────────────────────────────────────────────────────────┐<br>│                     EXECUTION                                │<br>│                                                              │<br>│   AutoExecutor → Alpaca API → multi-leg order fill           │<br>│                                     │                        │<br>│                                     ▼                        │<br>│                          OptionSignalLog (SQLite)            │<br>└──────────────────────┬───────────────────────────────────────┘<br>                       │<br>                       ▼<br>┌──────────────────────────────────────────────────────────────┐<br>│                   FEEDBACK LOOP (daily)                      │<br>│                                                              │<br>│   OptionsCalibrator   → per-underlying IV/RV threshold       │<br>│   EdgeDriftMonitor    → position size multiplier             │<br>│   ExecutionAnalyzer   → fill quality + maker/taker split     │<br>└──────────────────────────────────────────────────────────────┘</pre><h3>Where AI Is Used (and Where It Isn’t)</h3><p>This is worth being explicit about. AI is not used in the hot path — it doesn’t make the trade decision. That’s intentional. Real-time options order placement requires deterministic, auditable logic, not a language model making probabilistic calls.</p><p>AI is used in two places:</p><h4>1. Multi-Phase Market Analysis Agent</h4><p>The `AgentOrchestrator` runs a 4-phase agentic loop using Claude Sonnet with tool use. It gathers data across sources, scores it through quantitative tools, then synthesizes a trade thesis:</p><pre>Phase 1: GATHER<br>Claude calls tools:<br>→ fetch_macro_data() # FRED indicators<br>→ fetch_market_data() # options market context<br>→ fetch_news_headlines() # news aggregator<br><br>Phase 2: ANALYZE<br>Claude calls scoring tools:<br>→ score_macro_health() # 100-pt composite score<br>→ get_sector_weights() # macro → sector rotation<br>→ get_vol_regime() # VIX + realized vol context<br><br>Phase 3: DECIDE<br>Claude synthesizes a trade thesis<br>(no order tools available in this phase - read-only)<br><br>Phase 4: REPORT<br>Claude writes a structured daily brief</pre><p>The tool-calling loop runs until stop_reason != “tool_use” or the max tool cap is hit — standard agentic pattern with a deterministic exit condition.</p><pre># Simplified agentic loop<br>while True:<br>    response = anthropic_client.messages.create(<br>        model=&quot;claude-sonnet-4-5-20250929&quot;,<br>        tools=phase_tools,<br>        messages=messages,<br>    )<br>    if response.stop_reason != &quot;tool_use&quot;:<br>        break<br>    # Execute tool calls, append results, continue<br>    for block in response.content:<br>        if block.type == &quot;tool_use&quot;:<br>            result = execute_tool(block.name, block.input)<br>            messages.append(tool_result_message(block.id, result))</pre><h4>2. Daily Narrative Synthesis</h4><p>A separate <em>DailySynthesizer</em> (also Claude Sonnet) takes the scored data and generates a human-readable daily analysis report — covering macro regime, equity sector rotation, and implied volatility context. This runs at 11:00 AM UTC after all settlement processing completes.</p><p>This is where AI earns its keep: converting a dense JSON blob of 30+ macro indicators into a coherent investment narrative takes genuine reasoning, not just template substitution.</p><h3>Choosing the Right Underlying Assets</h3><p>Not every option-able stock is worth running this strategy on. Universe selection is one of the highest-leverage decisions in the system — a bad underlying will generate signals that look statistically sound but fail structurally.</p><p>The VRP strategy works best on underlying that satisfy four conditions:</p><p>1. <em>Liquid options market</em>. You need tight bid-ask spreads on both legs of the spread. Illiquid options turn theoretical edge into realized slippage. Practically, this means ETFs and large-cap single stocks with significant open interest at your target strikes.</p><p>2. <em>IV mean-reversion tendency</em>. The whole thesis depends on elevated IV returning to its baseline. Underlyings where IV is structurally elevated for fundamental reasons (imminent earnings, regulatory events, binary outcomes) don’t mean-revert — they resolve. Single-stock exposure around earnings is particularly dangerous. ETFs (SPY, QQQ, GLD, IWM, TLT) are naturally diversified, which smooths the vol profile.</p><p>3. <em>Predictable realized volatility</em>. You’re pricing puts using 30-day realized vol as your σ estimate. If realized vol is erratic — big single-day moves with no pattern — the BS estimate is unreliable. Broad ETFs win here too; their vol is driven by aggregate factors, not idiosyncratic news.</p><p>4. <em>Macro alignment</em>. The macro health scorer determines the overall vol regime. Underlyings that behave distinctly from their sector or the broader market create misalignment between the macro gate and the actual risk. For example, a healthcare ETF (XLV) responds differently to rate environments than SPY. You need to understand how each underlying’s IV responds to macro regimes — and whether your vol gate is actually calibrated for it.</p><p>The practical starting point: a core set of the most liquid ETFs (broad market, sectors, commodities, rates) plus a small number of mega-cap stocks with deep options chains. Avoid: near-earnings single stocks, low-float names, leveraged ETFs (their vol structure is mechanically distorted), and anything with thin open interest at OTM strikes.</p><p>The per-underlying calibration loop refines this over time — underlying that consistently underperform get higher IV/RV thresholds, which suppresses signals from them naturally. But good universe design is faster and more reliable than calibration alone.</p><h3>Signal Generation: The Math</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zjD64CnFoBUEdcrbFkIIsg.png" /></figure><p>For each of underlying, the scanner computes:</p><p>Step 1 — Theoretical fair value via Black-Scholes put pricing:</p><pre>BS_put(S, K, T, r, σ) where:<br>S = underlying spot price<br>K = strike price<br>T = time to expiry (years)<br>σ = 30-day realized volatility</pre><p>Step 2 — IV/RV ratio: market implied vol divided by 30-day realized vol. A ratio above 1.0 means the market is overpricing vol relative to recent history — the basic VRP signal.</p><p>Step 3 — Edge in basis points: the dollar difference between market bid and BS fair value, normalized to the risk taken (max loss of the spread). A signal requires all three thresholds to clear simultaneously. Most candidates fail at step 2 or 3.</p><h3>Risk Gates: Where Most Signals Die</h3><p>Three gates run in sequence. All must pass.</p><pre>Signal<br>  │<br>  ▼<br>[Gate 1] Vol Regime Gate<br>  │  macro vol_score in &quot;fear zone&quot; (elevated but not crisis)?<br>  │  iv_rv_ratio above minimum threshold?<br>  │  → BLOCK if complacency (vol too low) or crisis (vol structurally rising)<br>  │<br>  ▼ PASS<br>[Gate 2] Per-Underlying Threshold<br>  │  calibrated min_iv_rv_ratio for THIS underlying<br>  │  (adjusted daily by OptionsCalibrator feedback loop)<br>  │<br>  ▼ PASS<br>[Gate 3] Risk Manager<br>     max concurrent positions (configurable)<br>     max collateral per trade (configurable)<br>     kill switch: drawdown circuit breaker<br>     → BLOCK if any hard limit hit</pre><p>The vol regime gate deserves explanation. Selling vol in a structurally rising VIX environment (2022 is the canonical example) destroys capital even when IV/RV looks attractive. The gate uses the macro health scorer’s `volatility_score` — derived from FRED VIX data — to distinguish “fear spike in a mean-reverting regime” from “sustained structural vol expansion.” Only the former is tradable.</p><h3>Execution</h3><p>Once a signal clears all gates, the `AutoExecutor` places a net-credit multi-leg order via Alpaca:</p><pre>Sell PUT  (short leg) — OTM, collects premium<br>Buy  PUT  (long leg)  — lower strike, same expiry, caps max loss<br>Net credit received   = short_bid - long_ask<br>Max loss              = (spread_width - net_credit) × 100 × qty</pre><p>Exit logic runs every 15 minutes:</p><pre>Exits trigger at predefined profit and loss thresholds,<br>plus a time-based exit at expiry</pre><p>Every fill is written to `OptionSignalLog` with the full entry context: IV/RV at entry, edge_bps, signal type, and eventually P&amp;L.</p><h3>The Feedback Loop: How the System Learns</h3><p>This runs daily at 8:00 AM UTC, after settlement.</p><p><em>OptionsCalibrator</em> reads all settled <em>OptionSignalLog</em> rows, groups by underlying, and adjusts entry thresholds:</p><pre>for each underlying with sufficient settled trades:<br>    if win_rate below lower bound or avg_pnl &lt;= 0:<br>        min_iv_rv_ratio += step  # raise bar, suppress signals<br>    elif win_rate above upper bound and avg_pnl &gt; 0:<br>        min_iv_rv_ratio -= step  # lower bar, allow more signals<br>    else:<br>        hold                     # insufficient signal, wait<br>    # clamped to [floor, ceiling] — never leaves sane range</pre><p>Per-underlying IV/RV overrides are applied at scan time without mutating the shared config.</p><p><em>EdgeDriftMonitor</em> detects if the edge (in bps) on recent signals is decaying relative to the few-days baseline. Position sizes scale down progressively as edge quality degrades below baseline. This is an early-warning system for a crowding or efficiency shift in the underlying.</p><p>The key design principle: <em>all adjustments are bounded, logged, and reversible</em>. No process hard-codes new parameters autonomously. Humans see every change in the calibration log.</p><h3>The Data Model</h3><pre>OptionSignalLog<br>├── underlying          (SPY, QQQ, GLD, ...)<br>├── contract_symbol     (OCC format)<br>├── strike_price / expiration_date / dte<br>├── premium_bid / net_credit / max_loss_dollars<br>├── iv_rv_ratio         ← key calibration input<br>├── edge_bps            ← key selection criterion<br>├── annual_roi_pct<br>├── signal_type         (categorized by market condition at entry)<br>├── signal_style        (spread or uncovered, based on regime)<br>├── executed / settled<br>├── pnl_dollars / pnl_pct / close_reason<br>└── iv_spike_active / iv_spike_severity</pre><p>Every field that matters for calibration is stored at signal creation time, not reconstructed later. This is important: market conditions at entry are what drive performance analysis, not conditions at close.</p><h3>Stack</h3><pre>Backend:    FastAPI, SQLAlchemy, SQLite<br>Frontend:   React 19, TypeScript, Vite, TailwindCSS<br>Brokerage:  Alpaca API<br>Macro data: FRED connector (Fed Reserve economic data)<br>AI:         Anthropic Claude<br>Scheduler:  Custom cron-style task scheduler (no Celery dep)</pre><p>The whole system runs on a single machine — no cloud infra, no message queues and most APIs are free. SQLite handles the data layer cleanly at this scale. The scheduler is a lightweight in-process task runner registered at FastAPI lifespan startup.</p><h3>Summary</h3><p>The design intentionally separates <em>what the system knows</em> (quantitative signals) from <em>what the AI reasons about</em> (macro narrative and fundamental context). Neither alone is sufficient. Together, they cover different failure modes: the quant layer catches structural mis-pricing, the AI layer catches regime shifts and fundamental landmines that pure technicals miss.</p><p><em>All code is personal use. This is not financial advice.</em></p><h3>Follow up Reading</h3><p><a href="https://medium.com/@chongyao.robin/demystifying-personal-ai-trading-agent-crypto-edge-in-prediction-markets-3714cceaa4bf">https://medium.com/@chongyao.robin/demystifying-personal-ai-trading-agent-crypto-edge-in-prediction-markets-3714cceaa4bf</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=04a18307dc6f" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Unlocking Liquidity at Scale: The Power of Global Bids in NFT Marketplace]]></title>
            <link>https://medium.com/@chongyao.robin/unlocking-liquidity-at-scale-the-power-of-global-bids-in-nft-marketsbackground-03f53459013f?source=rss-a3d8293bb683------2</link>
            <guid isPermaLink="false">https://medium.com/p/03f53459013f</guid>
            <category><![CDATA[nft]]></category>
            <category><![CDATA[blockchain]]></category>
            <category><![CDATA[system-design-concepts]]></category>
            <dc:creator><![CDATA[Chong Yao]]></dc:creator>
            <pubDate>Fri, 30 May 2025 23:17:48 GMT</pubDate>
            <atom:updated>2025-06-11T20:57:39.435Z</atom:updated>
            <content:encoded><![CDATA[<h3>Background</h3><p>This article outlines the product workflow for <strong>Global Bid</strong>, a bid type distinct from standard 1-to-1 bidding.</p><p>The standard 1-to-1 bidding model requires a bidder to place separate bids for each edition of an NFT, resulting in multiple transactions and higher gas fees.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tMiMXTiXyJ3_74Ct66Ju0Q.png" /><figcaption>Standard 1:1 bidding</figcaption></figure><p>A <strong>Global Bid</strong> allows a bidder to place an offer visible to all owners of NFT editions within the same collection. In our product, NFTs that share the same media (image/text/video) and metadata (excluding ID) are grouped into a series and labeled as Edition 1 through N. Each edition may be owned by different individuals.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*j5Hu62AnIhbjNl13P4U-kA.png" /><figcaption>Global bidding</figcaption></figure><p>When a <strong>Global Bid</strong> is placed, it applies to all editions in the series. Any edition owner can accept the bid, and the first to do so fulfills the order, transferring ownership and completing the transaction.</p><h3>System Architecture</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*t2QUwmzWshpO23ejdVeEWA.png" /></figure><p>For more details of the system level, please refer to</p><p><a href="https://medium.com/@chongyao.robin/inside-the-backend-of-an-nft-marketplace-entire-system-4f10efc6897d">Inside the Backend of an NFT Marketplace — Entire System</a></p><h3>Use Cases</h3><p>A critical aspect of this feature is determining the <strong>live bid</strong> for each edition, especially when multiple bids and owners are involved. Let’s explore several scenarios involving two bidders and two edition owners for the same NFT series.</p><h4>1. Two Global Bids on the Same NFT Series</h4><ul><li><strong>Bidder A</strong> places a global bid of <strong>$200</strong></li><li><strong>Bidder B</strong> places a global bid of <strong>$100</strong></li></ul><p>Only the highest bid ($200) is considered the <strong>live bid</strong>, visible to both <strong>Owner A</strong> and <strong>Owner B</strong>.<br>If <strong>Owner A</strong> accepts the $200 bid, the NFT transfers to <strong>Bidder A</strong>, and the previous $100 bid becomes the new live bid. Now:</p><ul><li><strong>Owner B</strong> sees the $100 bid for Edition 2</li><li><strong>Bidder A</strong>, now the owner of Edition 1, also sees the $100 bid if it applies to Edition 1 again</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*dM9_TH74rpFrTN7xyiFLIg.png" /></figure><h4>2. Global Bid Cancellation</h4><p>If <strong>Bidder A</strong> cancels their $200 bid before it is accepted, the $100 bid from <strong>Bidder B</strong> becomes the new live bid for both editions.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*opqwG6vtFRAI6E7oltI9kw.png" /></figure><h4>3. Global Bid vs Standard Bid</h4><ul><li><strong>Bidder A</strong> places a <strong>Global Bid</strong> of <strong>$200</strong></li><li><strong>Bidder B</strong> places a <strong>Standard Bid</strong> of <strong>$100</strong> on <strong>Edition 1</strong></li></ul><p>The live bid for all editions is the higher global bid ($200). Both <strong>Owner A</strong> and <strong>Owner B</strong> see the $200 offer.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*loqTpXRYrlcJoSMYh8q3AA.png" /></figure><h4>4. Accepting Global vs Standard Bids</h4><ul><li><strong>Bidder A</strong> places a <strong>Standard Bid</strong> of <strong>$200</strong> for <strong>Edition 1</strong></li><li><strong>Bidder B</strong> places a <strong>Global Bid</strong> of <strong>$100</strong> for all editions</li></ul><p>Outcome:</p><ul><li>Edition 1’s live bid is <strong>$200</strong> (standard)</li><li>Edition 2’s live bid is <strong>$100</strong> (global)</li></ul><p>If <strong>Owner A</strong> accepts the $200 standard bid:</p><ul><li>The global bid ($100) becomes the live bid for <strong>Edition 2</strong></li></ul><p>If <strong>Owner B</strong> accepts the $100 global bid:</p><ul><li>The global bid is consumed</li><li><strong>Edition 1</strong> now has no live bid</li><li><strong>Edition 2</strong> is transferred to <strong>Bidder B</strong></li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*MOVQ5F5zCf9aNFMOAOxNDA.png" /></figure><h3>Complexity and Edge Cases</h3><p>Many combinations exist involving bid placement, cancellation, acceptance, outbidding, and ignoring — across both standard and global bids, multiple bidders, and multiple owners. These edge cases require comprehensive design of both <strong>on-chain</strong> and <strong>off-chain</strong> logic, including:</p><ul><li>Combination mapping and flow diagrams</li><li>Robust QA and scenario testing</li><li>Use of <strong>row-level locks</strong> in the database</li><li><strong>Distributed cache locks with TTL</strong> to avoid double spends, race conditions, and deadlocks</li></ul><p>For Event indexer, please refer to<br><a href="https://medium.com/@chongyao.robin/inside-the-backend-of-an-nft-marketplace-event-indexer-51445bc7ff66">https://medium.com/@chongyao.robin/inside-the-backend-of-an-nft-marketplace-event-indexer-51445bc7ff66</a></p><h3>Smart Contract</h3><p>On-chain data should remain minimal and efficient. A global bid on-chain record should include:</p><ul><li>bidder_address</li><li>NFT_address</li><li>bid_amount</li><li>bid_id</li></ul><p>The smart contract should support critical methods such as:</p><ul><li>place_bid</li><li>accept_bid</li><li>cancel_bid</li><li>get_bid</li><li>compute_commissions</li></ul><h3>Conclusion</h3><p>The <strong>Global Bid</strong> feature has already driven <strong>$100K in volume in its first quarter</strong>, proving its effectiveness in enhancing liquidity on both the bidder and seller sides. It also brings significant <strong>gas savings</strong> by reducing redundant transactions and enabling broader visibility across NFT editions.</p><h3>Acknowledgements</h3><p>PM: <a href="https://www.linkedin.com/in/landon-langford-bb74ba3a/">Landon Langford</a><br>Engineering: <a href="https://www.linkedin.com/in/chongyao/">Chong Yao</a>, Yash Nelapati, <a href="https://www.linkedin.com/in/nick-geoca-bb416933/">Nick Geoca</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=03f53459013f" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Transform a Static FAQ Page into a RAG-Based AI Chatbot]]></title>
            <link>https://medium.com/@chongyao.robin/transform-a-static-faq-page-into-a-rag-based-ai-chatbot-458febe8ebcb?source=rss-a3d8293bb683------2</link>
            <guid isPermaLink="false">https://medium.com/p/458febe8ebcb</guid>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[chatbot-development]]></category>
            <category><![CDATA[retrieval-augmented-gen]]></category>
            <category><![CDATA[llm-applications]]></category>
            <category><![CDATA[backend-development]]></category>
            <dc:creator><![CDATA[Chong Yao]]></dc:creator>
            <pubDate>Wed, 21 May 2025 15:21:28 GMT</pubDate>
            <atom:updated>2026-01-04T15:22:26.931Z</atom:updated>
            <content:encoded><![CDATA[<h3>Background</h3><p>At my previous company (<a href="https://makersplace.com/">MakersPlace</a>), we had a rich <a href="https://makersplace.com/faq/">FAQ page</a> full of useful questions and answers. However, many users preferred asking support questions in community channels like Discord or on social media platforms such as Twitter (X), instead of searching the FAQ page directly. This often overwhelmed the support team and led to inefficiencies.</p><p>An AI-powered chatbot that could answer FAQs would greatly mitigate this issue by providing quick, automated responses and easing the support load. Unfortunately, we lacked the bandwidth to build this solution at the time. Now, as a side project, I implemented a working prototype leveraging open-source tools and local models.</p><h4>FAQ page example</h4><p><a href="https://makersplace.com/faq/">Frequently Asked Questions</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*FAgL1xq8xQAkQ6pnWVJ0hg.png" /></figure><h3>System Architecture</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*NpHMC_YIcUOY0bpG7vZe_w.png" /></figure><h3><strong>Tech Stack</strong></h3><ul><li><strong>Embedding:</strong> PGVector with HuggingFace Sentence-Transformers</li><li><strong>Vector Retrieval:</strong> PGVector</li><li><strong>Large Language Model (LLM):</strong> Local Mistral 7B</li><li><strong>Web Crawler:</strong> BeautifulSoup</li><li><strong>Frontend</strong>: React.js + Vite</li><li><strong>Cache</strong>: Redis</li><li><strong>Backend</strong>: FastAPI</li></ul><h3>Data Ingestion</h3><ol><li><strong>Web Crawling:</strong><br> Crawl the FAQ or documentation page to extract questions and answers.</li><li><strong>Document Storage:</strong><br> Save each Q&amp;A pair as a Document object, then generate vector embeddings using HuggingFace’s sentence-transformers models.<br>Note: for most of the FAQ pages, the questions and answers are one to one mapped. Thus, we could optimize our documents implementations before embedding.</li></ol><pre>documents = []<br>for question, content in entries:<br>    if content:<br>        documents.append(<br>            Document(<br>                page_content=content,<br>                metadata={<br>                    &quot;type&quot;: &quot;content&quot;,<br>                    &quot;original_question&quot;: question or &quot;&quot;<br>                }<br>            )<br>        )<br><br>    if question:<br>        documents.append(<br>            Document(<br>                page_content=question,<br>                metadata={<br>                    &quot;type&quot;: &quot;question&quot;,<br>                    &quot;paired_content&quot;: content or &quot;&quot;<br>                }<br>            )<br>        )</pre><p><strong>3. Vector Indexing:</strong><br> Store these embeddings in a PostgreSQL database enhanced with the pgvector extension for efficient similarity search.</p><h3>Query Handling</h3><ol><li><strong>Retrieve Relevant Documents</strong><br> When a user asks a question, it will first try to get the answer from the Redis cache. <br>The cache key will be generated by applying the embedding model to the question.<br>If cache missed, start to use PGVector to retrieve the most relevant documents based on vector similarity.</li><li><strong>Prompt Construction</strong><br> Combine the retrieved context documents with the user’s question to create a structured prompt.</li><li><strong>LLM Inference</strong><br> Pass the prompt to a local LLM (Mistral 7B Instruct model with llama.cpp backend) for answer generation.</li><li><strong>Response</strong><br> Return the generated best-answer output to the user.</li><li><strong>Cache<br></strong>Update the cache value using the same embedding model to generate the cache key based on the question.</li></ol><h3>Important Considerations</h3><h3>PGVector Setup</h3><ul><li>You need to install and enable the <strong>pgvector</strong> extension in PostgreSQL before use.</li><li>Store metadata columns as <strong>JSONB</strong> rather than JSON to improve query performance and enable advanced filtering.</li></ul><h3>Prompt Design</h3><ul><li>Use a <strong>Chat Model</strong> prompt format (messages with roles like system, user, assistant), not a simple completion prompt.</li><li>Local models like <strong>Mistral 7B instruct</strong> expect <strong>structured instructions</strong> in the prompt to produce meaningful answers.</li></ul><pre># Chat model<br>[<br>  {&quot;role&quot;: &quot;system&quot;, &quot;content&quot;: &quot;You are a helpful assistant.&quot;},<br>  {&quot;role&quot;: &quot;user&quot;, &quot;content&quot;: &quot;What is blockchain?&quot;}<br>]<br><br># prompt<br>prompt = f&quot;&quot;&quot;You are a helpful assistant. Use the context below to answer the question.<br><br>Context:<br>{context}<br><br>Question:<br>{question}<br><br>Answer:&quot;&quot;&quot;</pre><h3>Max Tokens in LLM</h3><ul><li>max_tokens defines the max tokens generated in output, <em>not</em> including input tokens.</li><li>Roughly, 1 token ≈ 4 English characters or ~0.75 words.</li><li>Example (Mistral-7B):<br>Context window = 8192 tokens.<br>If prompt length = 2000 tokens, max output tokens ≈ 8192–2000 = 6192 tokens.</li><li>Exceeding the context window causes truncation or errors.</li></ul><h3>Future improvements</h3><p>This simple prototype is intended solely for learning and exploring AI chatbot technologies. To evolve it into a fully developed, production-grade product, we should consider implementing the following improvements.</p><ul><li><strong>Better Prompt Engineering:</strong><br> Enhance prompt templates for improved relevance and more natural responses.</li><li><strong>Multi-Model Support:</strong><br> Support different LLMs and quantization formats for efficiency and quality tradeoffs.</li><li><strong>Testing &amp; Evaluation:</strong><br> Add automated tests, benchmarks, and user feedback loops to improve accuracy and robustness.</li><li><strong>Docker &amp; Kubernetes Deployment:</strong><br> Containerize the app for local and cloud deployment, including scalable Kubernetes setups.</li><li><strong>Security &amp; Access Controls:</strong><br> Add authentication and rate limiting for production-grade deployment.</li></ul><h3>Result Examples</h3><h4>Web UI</h4><p>Visit local frontend React UI: <a href="http://localhost:3000/">http://localhost:3000</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*nqEN5c4T-gZT8NmzXqTqnA.png" /></figure><p><strong>API doc</strong></p><p>Visit local backend API: <a href="http://localhost:8000/docs">http://localhost:8000/docs</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4EpG2W5wLEc9vdDimdhVPA.png" /></figure><h4>CLIs</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*s-ps3nND00NaRNs8-9VJsA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_h2ug6WQgWTIJH0XXdfmDg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xgb1x6aCNzEvyW8-VQNcfg.png" /></figure><h3>Conclusion</h3><p>This project demonstrates how you can build a fast, local, and open-source AI chatbot to answer FAQs using Retrieval-Augmented Generation (RAG). By combining efficient vector search with a capable local LLM, you can provide relevant answers quickly and reduce the burden on human support.</p><p>This approach is highly customizable and extensible, making it a strong foundation for building AI-powered support bots that respect data privacy by keeping everything local.</p><h4>Github Repository</h4><p><a href="https://github.com/ZaphyrRobin/ai_faq_chatbot">GitHub - ZaphyrRobin/ai_faq_chatbot: Example of building AI RAG-based Chatbot on Makersplace FAQ page</a></p><p><em>For more posts like this, subscribe to my newsletter: </em><a href="https://zephyrrobin.substack.com/">zephyrrobin.substack.com</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=458febe8ebcb" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Inside the Backend of an NFT Marketplace — Entire System]]></title>
            <link>https://medium.com/@chongyao.robin/inside-the-backend-of-an-nft-marketplace-entire-system-4f10efc6897d?source=rss-a3d8293bb683------2</link>
            <guid isPermaLink="false">https://medium.com/p/4f10efc6897d</guid>
            <category><![CDATA[backend]]></category>
            <category><![CDATA[software-engineering]]></category>
            <category><![CDATA[blockchain]]></category>
            <category><![CDATA[nft-marketplace]]></category>
            <dc:creator><![CDATA[Chong Yao]]></dc:creator>
            <pubDate>Mon, 12 May 2025 00:54:37 GMT</pubDate>
            <atom:updated>2026-01-04T15:22:44.361Z</atom:updated>
            <content:encoded><![CDATA[<h3>Inside the Backend of an NFT Marketplace — System Overview</h3><p><a href="https://www.linkedin.com/in/chongyao/">Chong Yao</a> | MakersPlace</p><h3>Background</h3><p>In February 2025, MakersPlace — a well-known NFT art marketplace — was sunset. It rose to prominence in 2021 through close collaborations with prominent digital artists like Beeple — best known for the iconic NFT collections <a href="https://makersplace.com/product/everydays-the-first-5000-days-1-of-1-45997/"><em>EVERYDAYS: THE FIRST 5000 DAYS</em></a>, and many others. As one of the core backend engineers, I was deeply saddened by its closure, but I remain incredibly grateful for the experience I gained building real-world Web3 infrastructure at scale.</p><p>Now, while preparing for my next journey, I’ve taken some time to reflect on the backend architecture of MakersPlace. This chapter provides a comprehensive breakdown of the entire system.</p><h3>System Overview</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*d2U5fpAGWM3Dm2ByfkMOrQ.jpeg" /><figcaption>System Architecture</figcaption></figure><h3>Server Architecture</h3><p>We use <strong>Django</strong> primarily for backend logic and internal tools, as all user-facing frontend code has been migrated to <strong>React.js</strong>, and business logic is exposed via a <strong>GraphQL</strong> API layer.</p><p>Additionally, we leverage Django’s <strong>admin panel</strong> to support internal workflows, enabling non-engineering teams to configure product flows without code deployments.</p><h3>Machine Learning Services</h3><p>Our machine learning pipeline is detailed by Sudhir <a href="https://medium.com/makersplace/discovering-art-with-ai-ecac9ae787c9">here</a>. The ML model is hosted in a dedicated <strong>Finder Service</strong> that connects to Django via <strong>gRPC</strong>.</p><p>It performs <strong>K-Nearest Neighbor (KNN)</strong> search queries against <strong>OpenSearch</strong>, returning semantically similar artworks based on vector embeddings.</p><h3>Data Storage and Caching</h3><p><strong>Redis Cluster</strong> is utilized for:</p><ul><li>Caching API and backend query results.</li><li>Acting as a <strong>Celery</strong> broker for task queues.</li><li>Implementing distributed locks with TTL to prevent race conditions in critical sections.</li></ul><p><strong>AWS RDS (MySQL)</strong> is our primary transactional database, configured with read replicas to handle increased query load.</p><p><strong>AWS S3</strong> is used for blob storage of raw and processed media files, including images, videos, and CSVs.</p><p><strong>AWS OpenSearch</strong> powers our indexing, autocomplete, and full-text search capabilities, enabling fast and accurate content discovery.</p><p>For on-chain persistence, NFT media is pinned to <strong>Pinata (IPFS)</strong> and occasionally mirrored on <strong>Arweave</strong> for permanence.</p><h3>Asynchronous Processing</h3><p><strong>AWS Lambda</strong> handles lightweight, event-driven video transcoding tasks.</p><p><strong>Celery + Redis</strong> power all asynchronous job queues, supporting retry logic with exponential backoff.</p><p><strong>Apache Airflow</strong> schedules periodic jobs and manages long-running data workflows, tightly integrated with Celery workers.</p><p>Whenever a sale was completed, it is automatically posted by our <a href="https://x.com/MPSalesBot">X (formerly Twitter) bot</a>.</p><h3>Data Streaming</h3><p>AWS Kinesis is used solely for streaming and aggregating page view counts.</p><h3>Web3 Integration</h3><p><strong>Nightswatch</strong> is our custom transaction engine that submits on-chain transactions via <strong>Alchemy</strong> APIs. Full details:</p><p><a href="https://medium.com/@chongyao.robin/inside-the-backend-of-an-nft-marketplace-transaction-post-engine-3d4fbd09f416">Inside the Backend of an NFT Marketplace — Transaction Post Engine</a></p><p><strong>EventsWatcher</strong> listens to real-time blockchain events via <strong>Alchemy WebSockets</strong>, filters relevant smart contract events, and triggers downstream <strong>Celery tasks</strong> for indexing and database updates. Full details:</p><p><a href="https://medium.com/@chongyao.robin/inside-the-backend-of-an-nft-marketplace-event-indexer-51445bc7ff66">Inside the Backend of an NFT Marketplace — Event Indexer</a></p><h3>Monitoring &amp; Logging</h3><p><strong>Papertrail</strong> aggregates in-application logs for visibility and auditing.</p><p><strong>Scout</strong> is used to profile and measure performance across our API endpoints.</p><p><strong>AWS CloudWatch</strong> monitors infrastructure-level metrics (e.g., CPU, memory), triggering alerts via <strong>Slack</strong> and <strong>PagerDuty</strong> for on-call notifications.</p><p>Custom alerts and critical logs are also piped into Slack to ensure high visibility during abnormal behavior or failures.</p><h3>Security</h3><p>All credentials, secrets, and third-party API keys are stored securely in <strong>AWS Systems Manager Parameter Store</strong>, minimizing risk of exposure.</p><p>API-level authentication, authorization, and rate limiting are enforced at the <strong>Django</strong> and <strong>GraphQL</strong> layers.</p><h3>Content Delivery Network (CDN)</h3><p><strong>Cloudflare</strong> serves as our global CDN, optimizing the delivery of static assets such as JavaScript bundles, media, and fonts for fast and secure user access.</p><h3>Optimization and Scaling</h3><p>To handle growing user and data volume, we have identified the following scaling and optimization opportunities:</p><ol><li><strong>Horizontally scale</strong> Redis, RDS (MySQL), and Django EC2 instances using auto-scaling groups.</li><li>Introduce a <strong>high-throughput message broker</strong> (e.g., <strong>Apache Kafka</strong>) to replace Redis as a task queue backend, enabling better stream handling and reliability.</li><li>Add a <strong>time-series database</strong> (e.g., <strong>InfluxDB</strong>, <strong>ClickHouse</strong>) to store and analyze user engagement data more efficiently than using relational databases.</li><li>Optimize real-time stream processing with AWS Kinesis and Apache Flink.</li></ol><h3>Conclusion</h3><p>MakersPlace is built on a modular, scalable architecture that blends traditional web tech with Web3 and machine learning. By leveraging Django, React, AWS, and blockchain services, we deliver a seamless, performant NFT marketplace. As the platform grows, we’re set up to scale further with improvements in async processing, observability, and onchain integration — ensuring we stay reliable, fast, and future-ready.</p><h3>Acknowledgements</h3><p>Thank you to all the engineers whose contributions made this system possible!</p><p><a href="https://www.linkedin.com/in/brian-shen/">Brian Shen</a>, <a href="https://www.linkedin.com/in/chongyao/">Chong Yao</a>, <a href="https://www.linkedin.com/in/minhduc3791/">Duc Nguyen</a>, <a href="https://www.linkedin.com/in/jasondotwang/">Jason Wang</a>, <a href="https://www.linkedin.com/in/softwareengineer123/">Kyle Graves</a>, <a href="https://www.linkedin.com/in/martincastroalvarez/">Martin Alvarez</a>, <a href="https://www.linkedin.com/in/mcskinner/">Michael Skinner</a>, <a href="https://www.linkedin.com/in/mihail-i-0407b01/">Mihail Ionescu</a>, <a href="https://www.linkedin.com/in/nick-geoca-bb416933/">Nick Geoca</a>, <a href="https://www.linkedin.com/in/nikitamatusevich/">Nikita Matusevich</a>, Nouman Saeed, <a href="https://www.linkedin.com/in/sanjay-raghavan/">Sanjay Raghavan</a>, <a href="https://www.linkedin.com/in/konerusudhir/">Sudhir Koneru</a>, Yash Nelapati</p><p><em>For more posts like this, subscribe to my newsletter: </em><a href="https://zephyrrobin.substack.com/">zephyrrobin.substack.com</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4f10efc6897d" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Inside the Backend of an NFT Marketplace — Event Indexer]]></title>
            <link>https://medium.com/@chongyao.robin/inside-the-backend-of-an-nft-marketplace-event-indexer-51445bc7ff66?source=rss-a3d8293bb683------2</link>
            <guid isPermaLink="false">https://medium.com/p/51445bc7ff66</guid>
            <category><![CDATA[software-engineering]]></category>
            <category><![CDATA[nft-marketplace]]></category>
            <category><![CDATA[backend]]></category>
            <category><![CDATA[blockchain]]></category>
            <dc:creator><![CDATA[Chong Yao]]></dc:creator>
            <pubDate>Sun, 11 May 2025 01:04:18 GMT</pubDate>
            <atom:updated>2026-01-04T15:22:54.784Z</atom:updated>
            <content:encoded><![CDATA[<h3>Inside the Backend of an NFT Marketplace — Event Indexer</h3><p><a href="https://www.linkedin.com/in/chongyao/">Chong Yao</a> | MakersPlace</p><h3>Background</h3><p>An event indexer is a system that reads blockchain transaction events from blocks, processes them, and loads the resulting data into a database. It serves as a crucial bridge between on-chain activity and off-chain systems, ensuring accurate and timely data synchronization.</p><p>Internally referred to as the “<em>Event Watcher</em>,” our event indexer has recently undergone a major architectural refactor to improve reliability, scalability, and maintainability.</p><h3>System Overview: Version 1</h3><p>The first version of our event indexer was designed to ingest on-chain Ethereum events via Alchemy’s WebSocket API, process them, and persist the results into our backend database. It powers features such as bid/sale updates, transfers, and user notifications, etc…</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/751/1*spAbT6g0k19LlviKi_fz7g.jpeg" /><figcaption>System Architecture of EventWatcher v1</figcaption></figure><p>Example of an event.</p><pre>{<br>  &quot;event&quot;: &quot;BidAccepted&quot;, # Event name<br>  &quot;contractAddress&quot;: &quot;0xAbC1234567890dEf1234567890aBC1234567890a&quot;,<br>  &quot;blockNumber&quot;: 1109432,<br>  &quot;transactionHash&quot;: &quot;0x9f7bc6d0800f9e8e98349e63ad44397a1c55cc44dfdfe2cbf4ed2d7fa0aa0c14&quot;,<br>  &quot;logIndex&quot;: 0,<br>  &quot;seller&quot;: &quot;0x1111222233334444555566667777888899990000&quot;,<br>  &quot;buyer&quot;: &quot;0xaaaa2222bbbb3333cccc4444dddd5555eeee6666&quot;,<br>  &quot;tokenId&quot;: 291,<br>  &quot;amount&quot;: 10000,<br>  &quot;timestamp&quot;: 1719493664<br>}</pre><p>Example of event_name → processing_function mapping.</p><pre>EVENT_NAME_TO_TASK = {<br>    bid_accepted: process_bid_accepted_task,<br>    ...<br>}</pre><p><strong>Key Components:</strong></p><ul><li><strong>Async Event Loop:</strong><br> Listens to the Ethereum blockchain through Alchemy’s WebSocket API to detect the latest block in real-time.</li><li><strong>Warm-Up Logic:</strong><br>On startup or restart, the EventWatcher replays unprocessed blocks by looping from the last synced block in our database to the latest block on-chain. This ensures that we don’t miss any events after server restarts.</li><li><strong>Async works:</strong><br>We scaled to 10+ Celery async workers for task processing.</li></ul><p><strong>Block Processing Flow:</strong></p><ol><li>Fetch logs from the block.</li><li>Filter out irrelevant logs and retain only those emitted by our smart contracts.</li><li>Use a mapping (event_name → processing_function) to dispatch events to Celery tasks.</li><li>Persist the processed event results into the database.</li><li>Update the EthereumTransaction table&#39;s status to MINED.</li></ol><h3>Limitations of Version 1</h3><p>While this initial version functioned well under normal conditions, it exhibited several critical flaws that affected reliability and developer experience:</p><ul><li><strong>Race Conditions &amp; Deadlocks:</strong><br> Concurrent Celery workers could attempt to process the same event multiple times, leading to duplicate inserts or inconsistent state.</li><li><strong>Legacy Bugs in Business Logic:</strong><br> Some event flows had hidden bugs in their Celery task logic due to outdated assumptions or unclear contracts.</li><li><strong>Lack of Monitoring &amp; Logging:</strong><br> Debugging production issues often required manual tracing through logs and reconstructing event data, which was time-consuming and error-prone.</li><li><strong>No Alerting System:</strong><br> When event processing silently failed, on-call engineers weren’t notified, causing delayed issue resolution and customer complaints.</li><li><strong>Incorrect MINED Status Update:</strong><br> The system waited until after successful event processing to mark a transaction as MINED, even though it was already mined at the point of log retrieval.</li><li><strong>Out-of-Order Event Processing:</strong><br> Events from the same contract were not grouped or sequenced, leading to processing out of order — an issue that broke assumptions in downstream logic which is important in some of the use cases.</li></ul><h3>Why It Mattered</h3><p>As one of the core on-call engineers, I discovered that roughly <strong>60% of Asana tickets</strong> originated from the EventWatcher system. These included user issues like:</p><ul><li><em>“Why didn’t I receive my bid confirmation email?”</em></li><li><em>“Why does Etherscan show the NFT was transferred but nothing updated on the site?”</em></li></ul><p>Clearly, a major overhaul was necessary.</p><h3>Daily Reporting for Visibility</h3><p>To raise awareness and help leadership prioritize the refactor, I introduced a daily report system. An Airflow job collected metrics across the full bid lifecycle — from frontend user actions to blockchain confirmation and final database updates.</p><p>This was posted to our internal Slack/Discord channels and included:</p><ul><li>Average success rate per flow</li><li>End-to-end latency (including blockchain mining time)</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5_MZCrvNBEhZuBn1-151dA.png" /><figcaption>Daily report of critical flows</figcaption></figure><p>The data made it clear: success rates weren’t consistent, and delays in processing were non-negligible. It’s clear that the average timing for some of the flows is quite long — though this includes on-chain mining, which ideally should complete within 60 to 90 seconds.</p><h3>System Overview: Version 2</h3><p>With leadership buy-in, we kicked off a major refactor. Working closely with senior engineers, our CTO, and EM, we redesigned the architecture for robustness, idempotence, and observability.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6exGtxwwx1gSi4BV-7Wl9w.jpeg" /><figcaption>System Architecture of EventsWatcher V2</figcaption></figure><p><strong>Key Improvements in v2:</strong></p><ol><li><strong>Task Queue Tables:</strong></li></ol><ul><li>Introduced a new database (TASK_QUEUE_DATABASE) with two tables: ActiveJob and ArchiveJob.</li><li>Each job was identified by a hash of the event_name and event_data to ensure uniqueness.</li></ul><pre>class JobStatusEnum():<br>    NEW = 1<br>    PENDING = 2<br>    COMPLETED = 3<br>    FAILED = 4<br><br>class ActiveJob():<br>    job_id = models.CharField(unique=True)<br>    event_name = models.CharField()<br>    event_data_hash = models.CharField(unique=True)<br>    event_data = JSONField()<br>    status = models.PositiveSmallIntegerField(choices=JobStatusEnum.choices())<br><br>    class Meta:<br>        indexes = [models.Index(fields=[&quot;event_name&quot;, &quot;event_data_hash&quot;])]<br><br>    def save(self, *args, **kwargs):<br>        if not self.job_id:<br>            self.job_id = # do hash on f&quot;{self.event_name}:{str(self.event_data)}&quot;<br>            self.event_data_hash = # do hash on self.event_data<br>        super().save(*args, **kwargs)<br><br>class ArchiveJob(models.Model):<br>    # Same fields to ActiveJob</pre><p>2. <strong>Event Sequencing:</strong></p><ul><li>Events from the same smart contract are now grouped and processed in sequence using a Celery chain, ensuring order-dependent logic remains consistent. For example, when minting multiple identical media NFTs, we aim to preserve the sequential edition order and ensure it aligns with the metadata stored on IPFS.</li></ul><pre>from celery import chain<br>def group_events_in_order(events_to_contract: List[Dict]) -&gt; None:<br>    tasks = []<br>    for event_data in events_to_contract:<br>        tasks.append(process_celery_task.si(event_data))<br>    chain(tasks).delay()</pre><p><strong>3. Atomic Processing Check:</strong></p><ul><li>Before executing a Celery task, the system checks if a job with the same hash is already in progress or completed.</li><li>If it’s in progress, the event is dropped to avoid duplication.</li><li>This guarantees <strong>idempotent</strong> and <strong>race-condition-free</strong> execution.</li></ul><p><strong>4. Indexing &amp; Cleanup:</strong></p><ul><li>Built composite indexes on (event_name, event_data_hash) for fast lookups.</li><li>A scheduled Airflow job migrates the completed jobs from ActiveJob to ArchiveJob every hour to control table growth. <br>Since the ArchiveJob serves solely as a cold storage mechanism, we’ve opted for batch updates and deletes, which are sufficient for our needs. An alternative approach—using stream processing via database CDC (Change Data Capture) triggered by ActiveJob status change commits—introduces significant overhead and is not well-suited for our system’s use case.</li></ul><p><strong>5. Monitoring &amp; Alerting:</strong></p><ul><li>Added an Airflow DAG that monitors ActiveJob for an unusual number of stuck or pending jobs.</li><li>Alerts are pushed to Slack and PagerDuty to ensure timely on-call responses.</li></ul><h3>Follow-up Topics</h3><h4><strong>Why Build Your Own Event Indexer</strong></h4><p><strong>1. Full Data Ownership &amp; Control</strong></p><ul><li>You have complete control over what gets indexed and how it’s stored.</li><li>Easily customize which smart contracts, events, or tokens to track.</li><li>Avoid vendor lock-in — you’re not subject to third-party API changes, limitations, or data retention policies.</li></ul><p><strong>2. Reduced Latency &amp; Better Performance</strong></p><ul><li>A self-hosted indexer can deliver real-time or near-real-time event tracking tailored to your product needs.</li><li>Third-party APIs often introduce delays due to rate limiting, aggregation, or caching layers.</li></ul><p><strong>3. Custom Logic &amp; Filtering</strong></p><ul><li>Apply business-specific rules, e.g., only index ERC-721 Transfer events with certain metadata criteria.</li><li>Enrich indexed data with off-chain sources such as royalties, collection flags, or user profiles.</li><li>Easily expand to new chains — I successfully prototyped this for L2 networks like Base and Arbitrum.</li></ul><p><strong>4. Cost Efficiency at Scale</strong></p><ul><li>While initial development is more resource-intensive, self-hosting can reduce long-term costs, especially with high query volumes (e.g., millions of events).</li><li>No per-request or per-token fees, unlike some commercial API providers.</li></ul><p><strong>However,</strong> given the maturity of today’s event indexing providers, if I were building the system from scratch now, I might opt for an external solution instead of maintaining our own at the early stage.</p><h4>How to Scale it</h4><ol><li><strong>Deploy multiple instances of the EventWatcher</strong><br> Each instance can be assigned to handle a subset of:</li></ol><ul><li>Specific smart contracts</li><li>Specific blockchains (for multi-chain support)</li></ul><p><strong>2. Scale Celery workers and improve task infrastructure</strong></p><ul><li>Increase the number of Celery workers to handle higher throughput</li><li>Consider upgrading to high-performance message brokers like <strong>Kafka</strong> or <strong>AWS SQS</strong> for better scalability and reliability</li></ul><p><strong>3. Shard or partition database writes</strong></p><ul><li>Partition by <strong>contract address</strong>, <strong>chain</strong>, or <strong>timestamp</strong> to distribute load and avoid write bottlenecks</li></ul><h3>Conclusion</h3><p>With all the improvements and architectural changes, the redesigned event indexer is now:</p><ul><li>Indexing latency dropped from minutes to seconds.</li><li>Improved task success rate from 40% to 95%.</li><li>Reduced on-call noise by 60% through better retries and observability.</li><li>Equipped with monitoring, alerts, and detailed logs for faster debugging</li><li>Processes events efficiently while preserving correct sequencing and data integrity.</li><li>Our system is designed to scale easily across multiple blockchains.</li></ul><p>This overhaul significantly improved both the developer and user experience, laying a solid foundation for future scaling and multi-chain support.</p><h3>Acknowledgements</h3><p>Engineering: <a href="https://www.linkedin.com/in/chongyao/">Chong Yao</a>, Yash Nelapati, <a href="https://www.linkedin.com/in/jasondotwang/">Jason Wang</a>, <a href="https://www.linkedin.com/in/mcskinner/">Michael Skinner</a>, <a href="https://www.linkedin.com/in/mihail-i-0407b01/">Mihail Ionescu</a></p><p><em>For more posts like this, subscribe to my newsletter: </em><a href="https://zephyrrobin.substack.com/">zephyrrobin.substack.com</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=51445bc7ff66" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Inside the Backend of an NFT Marketplace — Transaction Post Engine]]></title>
            <link>https://medium.com/@chongyao.robin/inside-the-backend-of-an-nft-marketplace-transaction-post-engine-3d4fbd09f416?source=rss-a3d8293bb683------2</link>
            <guid isPermaLink="false">https://medium.com/p/3d4fbd09f416</guid>
            <category><![CDATA[blockchain]]></category>
            <category><![CDATA[backend]]></category>
            <category><![CDATA[nft-marketplace]]></category>
            <category><![CDATA[software-engineering]]></category>
            <dc:creator><![CDATA[Chong Yao]]></dc:creator>
            <pubDate>Sat, 10 May 2025 00:48:07 GMT</pubDate>
            <atom:updated>2026-01-04T15:24:04.721Z</atom:updated>
            <content:encoded><![CDATA[<h3>Inside the Backend of an NFT Marketplace — Transaction Post Engine</h3><p><a href="https://www.linkedin.com/in/chongyao/">Chong Yao</a> | MakersPlace</p><h3>Background</h3><p>This chapter focuses on one of the most critical services in our system: the transaction posting engine. Yash, our CTO and the first engineer at Pinterest, was a huge fan of <em>Game of Thrones</em> — so naturally, he named this service <strong>NightsWatch</strong>.</p><p>The transaction posting engine plays a critical role in Web3 architecture, serving as the essential bridge between off-chain systems and on-chain execution.</p><h3>System Overview: NightsWatch Transaction Engine</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*WxSjNmn47HR3dR5G69ZZGA.jpeg" /><figcaption>System Architecture of NightsWatch</figcaption></figure><p>In high level,</p><ul><li><strong>gRPC Entry Point</strong>: Async workers communicate with the NightsWatch server via gRPC calls, such as queue_ethereum_transaction.</li><li><strong>Thread Pool</strong>: NightsWatch handles incoming requests using a thread pool of up to 10 workers.</li><li><strong>Async Event Loop</strong>: Separately, a dedicated async loop runs every 3 minutes to call post_pending_transactions. This fetches up to 10 Ethereum transactions in a PENDING state from the database.</li></ul><p>Both queue_ethereum_transaction and the event loop share a validation layer that checks:</p><ul><li>If the user has provided enough gas.</li><li>If the wallet address pending transaction queue is full.</li></ul><p>Major wallet addresses (those owned by the company) are separated by features — such as minting vs. bidding/sales/auctions — and each has its own queue (max size 10) to prevent nonce conflicts.</p><p>If validation passes:</p><ol><li>The processing function need to acquire the cache lock (Redis) of blocking_timeout with the cache key of the transaction id to prevent race condition and deadlock when generate the onchain nonce</li><li>Fetch the correct nonce for the wallet.</li><li>Atomically update the transaction’s status to TRYING to prevent race conditions.</li><li>Send the transaction via Alchemy’s API.</li><li>If successful, the response includes a transaction hash, and the status is updated to POSTED.</li></ol><p>Once the transaction is confirmed on-chain and a corresponding event is emitted, an <strong>event indexer</strong> (covered in the next chapter) processes it and updates the transaction status to MINED.</p><p>Other gRPC methods — such as updating, speeding up, or signing a transaction — follow similar pattern and state logic.</p><p>Thanks to detailed state transitions, per-wallet queues, and a controlled event loop, NightsWatch achieves consistency, reliability, and resilience in transaction handling.</p><h3>Why Use Queues for Transactions?</h3><p>Blockchains, especially Ethereum, enforce <strong>strict nonce ordering</strong> for transactions from a single address. This means:</p><ol><li>Transactions must be processed in <strong>first-in, first-out (FIFO)</strong> order.</li><li>Submitting multiple transactions out of order or too quickly can lead to: — Nonce mismatches. <br> — Dropped, stuck, or replaced transactions. <br> — Wasted gas fees.</li></ol><p>By implementing a per-wallet queue:</p><ol><li>Only one active transaction is allowed at a time per address.</li><li>We prevent race conditions and avoid overloading the blockchain node.</li><li>It provides back pressure to control throughput and retries safely.</li></ol><h3>Benefits of a Dedicated Transaction Posting Engine</h3><h4>Fine-Grained Nonce Control</h4><p>Ethereum requires each wallet’s transactions to have incrementing nonces. Managing this at the wallet-queue level guarantees:</p><ul><li>Nonce correctness.</li><li>Proper sequencing.</li><li>No accidental collisions.</li></ul><h4>Scalable Across Wallets</h4><p>Each wallet has an isolated queue. That means:</p><ul><li>We can safely process different accounts <strong>in parallel</strong>.</li><li>Horizontal scaling is easy — just spin up more workers to handle additional queues.</li></ul><h4>Extensible Architecture</h4><p>A dedicated engine allows you to:</p><ul><li>Add new blockchain support. <br>(Actually, I successfully prototyped this on L2 Base and Arbitrum)</li><li>Plug in modules for fraud detection, analytics, or VIP handling.<br>(For VIP users, we waive the gas fee when posting transactions)</li><li>Integrate fee boosters or transaction speed-up logic.</li></ul><h4>Transaction Batching and Rate Limiting</h4><p>We can,</p><ul><li>Per-wallet rate limits (e.g., max N txs per minute).</li><li>Conditional batching (e.g., post in bulk if queue depth is high).</li><li>Time-based prioritization (e.g., promote older or high-priority txs).</li></ul><h4>Built-In Retry and Resilience</h4><p>Failures (due to low gas, RPC errors, reorgs) don’t have to be catastrophic. We can:</p><ul><li>Retry failed transactions.</li><li>Escalate gas prices.</li><li>Alert or notify users.</li><li>Re-enqueue intelligently.</li></ul><h3>How to scale</h3><ul><li>Run <strong>multiple posting engine instances</strong>, each responsible for a subset of addresses or Redis streams.</li><li>Each worker fetches and posts txs for its assigned wallets.</li><li>Increase the number of workers and increase the wallet address queue size</li></ul><h3>Conclusion</h3><p>Building a dedicated transaction posting engine like <em>NightsWatch</em> gave us fine-grained control over Ethereum transaction flow, ensured nonce consistency, and improved system reliability under Web3-scale demands.</p><blockquote><strong>And Now His Watch Is Ended</strong></blockquote><h3>Acknowledgements</h3><p>Leaders: Yash Nelapati, <a href="https://www.linkedin.com/in/sanjay-raghavan/">Sanjay Raghavan</a>, <a href="https://www.linkedin.com/in/mcskinner/">Michael Skinner</a></p><p>Engineering: <a href="https://www.linkedin.com/in/chongyao/">Chong Yao</a>, <a href="https://www.linkedin.com/in/jasondotwang/">Jason Wang</a>, <a href="https://www.linkedin.com/in/mihail-i-0407b01/">Mihail Ionescu</a>, <a href="https://www.linkedin.com/in/martincastroalvarez/">Martin Alvarez</a>, <a href="https://www.linkedin.com/in/nick-geoca-bb416933/">Nick Geoca</a></p><p><em>For more posts like this, subscribe to my newsletter: </em><a href="https://zephyrrobin.substack.com/">zephyrrobin.substack.com</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3d4fbd09f416" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>