<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Ankit Gupta on Medium]]></title>
        <description><![CDATA[Stories by Ankit Gupta on Medium]]></description>
        <link>https://medium.com/@techankit1990?source=rss-7feaf4a9dc79------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/proxy/1*TGH72Nnw24QL3iV9IOm4VA.png</url>
            <title>Stories by Ankit Gupta on Medium</title>
            <link>https://medium.com/@techankit1990?source=rss-7feaf4a9dc79------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 15:24:40 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@techankit1990/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[AI-Assisted Payments Triage — Without Losing Trust in the Answer]]></title>
            <link>https://medium.com/@techankit1990/ai-assisted-payments-triage-without-losing-trust-in-the-answer-68613403cbef?source=rss-7feaf4a9dc79------2</link>
            <guid isPermaLink="false">https://medium.com/p/68613403cbef</guid>
            <category><![CDATA[langchain]]></category>
            <category><![CDATA[langgraph]]></category>
            <category><![CDATA[agentic-ai]]></category>
            <category><![CDATA[langfuse]]></category>
            <dc:creator><![CDATA[Ankit Gupta]]></dc:creator>
            <pubDate>Wed, 29 Jul 2026 14:11:00 GMT</pubDate>
            <atom:updated>2026-08-08T09:00:01.046Z</atom:updated>
            <content:encoded><![CDATA[<p>Every payments operations team knows this moment: a payment fails somewhere between five different systems, and the first hour of the incident is spent doing detective work — pulling logs from a PayID service, a gateway, a ledger, cross-referencing correlation IDs by hand, before anyone can even say <em>what</em> went wrong, let alone <em>why</em>.</p><p>I built a proof-of-concept, AI-assisted payments triage system to close that gap for Australian payment rails. The interesting part isn’t “we added an LLM to a support tool.” It’s the design decisions that make an AI system something a payments ops team can actually trust with production incidents. Three of those decisions are worth sharing.</p><h3>01 Cost control</h3><h4>Let the cheapest, most certain answer win first</h4><p>Not every payment failure is a mystery. Most are a known error code — insufficient funds, an invalid BSB, a PayID that resolves to the wrong account. For those, we don’t need an LLM at all: a deterministic engine scans the incident timeline, matches known error signatures against a maintained catalogue, and returns a root cause with full confidence, instantly and for free.</p><p>The AI reasoning layer only switches on when the deterministic pass comes up empty — a timeout with no error code, a cascading failure across systems, an outcome that’s genuinely ambiguous. Even then, the system escalates in stages: first a focused investigation agent that reasons system-by-system and stops as soon as it’s confident, and only for the hardest cases, a second “reflection” loop where one AI agent proposes a root cause and a second, adversarial AI agent is required to challenge it — checking that every claim is grounded in an actual log line, not invented — before it’s allowed to reach a human.</p><h3>02 Trust boundary</h3><h4>Architecture as a trust boundary, not just a tech choice</h4><p>The system is built in four strict layers — the API surface, the orchestration logic, the core business rules, and the integrations (LLM providers, databases, log sources) — with dependencies only allowed to point inward. That’s a familiar software pattern, but here it does something extra: it keeps AI-specific policy in one place instead of scattered across every feature.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/601/1*Udd4U8kzEwD1sXHRjonl6A.png" /></figure><p>Concretely, that means every single call to a language model — regardless of which service triggers it — passes through the same three-layer wrapper: resilience (retries and circuit-breaking so a flaky provider doesn’t take down the product), PII scrubbing (emails, card numbers, BSBs, phone numbers, tax file numbers are stripped before anything leaves the systems and before it’s logged), and tracing (so every prompt, response, and decision is recorded for audit). A new feature can’t accidentally bypass any of it, because there’s only one road out to a model provider.</p><h3>03 Feedback loop</h3><h4>Ship the feedback loop with the feature, not after it</h4><p>The easiest mistake with an AI feature is to launch it and have no way of knowing, three weeks later, whether it’s still any good. We treated evaluation as a first-class part of the product, not a follow-up project, built around three simple ideas working together.</p><blockquote>A living exam, sourced from real usage. Every time an analyst rates a chat answer, that interaction becomes a candidate for a “golden dataset” — a growing exam of real, human-approved cases. Before any prompt or model change ships, it gets replayed against that exam.</blockquote><blockquote>A live dashboard of trust signals, not vanity metrics: how often the system escalates to a human, how confident it was when it did, and the actual star ratings analysts give it — all visible in near real time.</blockquote><blockquote>A fast way to ask “what if.” A lightweight tool to test a prompt tweak against a slice of real cases in seconds, without spinning up a notebook.</blockquote><h3>Addendum: Design Considerations &amp; Architectural Decisions</h3><h4>ADR-001: Structured Input Instead of Free-Text Chat</h4><p><strong>Decision</strong></p><p>The system accepts only payment identifiers that match predefined regex patterns instead of unrestricted natural language.</p><p><strong>Rationale</strong></p><p>Although conversational interfaces are attractive, free-text input introduces additional concerns that were outside the scope of this POC:</p><ul><li>Prompt injection protection</li><li>Input sanitization</li><li>LLM-based intent detection</li><li>Authentication and authorization considerations</li><li>Expanded security review</li></ul><p>Restricting inputs to known payment ID formats allows deterministic validation before any AI component is invoked.</p><p><strong>Trade-off</strong></p><p>Lower conversational flexibility in exchange for:</p><ul><li>Smaller attack surface</li><li>Predictable execution paths</li><li>Lower latency</li><li>Easier testing</li></ul><p>This keeps the AI focused on incident analysis rather than interpreting user intent.</p><h3>ADR-002: LangGraph + FastAPI Without MCP</h3><p>The orchestration layer is implemented using LangGraph and FastAPI only.</p><p><strong>Rationale</strong></p><p>The POC requires:</p><ul><li>workflow orchestration</li><li>state management</li><li>API exposure</li></ul><p>These capabilities are sufficiently handled by LangGraph and FastAPI.</p><p>Model Context Protocol (MCP) was intentionally excluded because:</p><ul><li>There are no external AI tools requiring standardized discovery</li><li>Tool interoperability is not the problem being validated</li><li>Introducing MCP would increase architectural complexity without improving the core hypothesis</li></ul><p>Future production implementations may benefit from MCP as additional enterprise tools are integrated.</p><h3>ADR-003: Human-Based Evaluation</h3><p><strong>Decision</strong></p><p>Evaluation is currently performed using human reviewers instead of LLM-as-a-Judge.</p><p><strong>Rationale</strong></p><p>The primary success criteria are operational:</p><ul><li>Was the failure identified correctly?</li><li>Was the root cause accurate?</li><li>Would an operations engineer trust the explanation?</li></ul><p>These questions are best answered by experienced payment operations engineers.</p><p>Using human evaluation also avoids introducing another LLM whose judgments would themselves require validation.</p><p>Future versions may incorporate automated evaluation frameworks after sufficient human-labelled datasets have been collected.</p><h3>The Takeaway</h3><p>The flashy part of AI-assisted payments triage is the model. The part that actually earns a payments team’s trust is everything around it: knowing when to stay out of the way, making the “safe path” the only path, and treating “is this still accurate?” as a question the system answers continuously, not one you ask once at launch.</p><p><em>Built as a proof of concept exploring AI-assisted incident triage for Australian payment rails, with LangGraph-based reasoning agents, deterministic-first triage, and Langfuse-backed observability.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=68613403cbef" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>