<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Ankit Bharti on Medium]]></title>
        <description><![CDATA[Stories by Ankit Bharti on Medium]]></description>
        <link>https://medium.com/@ankitbharti?source=rss-1f2baa41252e------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*uLKmLGEEOYHll2FqQVho2Q@2x.jpeg</url>
            <title>Stories by Ankit Bharti on Medium</title>
            <link>https://medium.com/@ankitbharti?source=rss-1f2baa41252e------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Mon, 16 Nov 2026 15:22:01 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@ankitbharti/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[I Built a Multi-Agent AI System to Audit iOS Apps for Accessibility – Here’s What I Learned]]></title>
            <link>https://medium.com/@ankitbharti/i-built-a-multi-agent-ai-system-to-audit-ios-apps-for-accessibility-heres-what-i-learned-177c94542142?source=rss-1f2baa41252e------2</link>
            <guid isPermaLink="false">https://medium.com/p/177c94542142</guid>
            <category><![CDATA[agentic-ai]]></category>
            <category><![CDATA[ios]]></category>
            <category><![CDATA[accessibility]]></category>
            <dc:creator><![CDATA[Ankit Bharti]]></dc:creator>
            <pubDate>Sun, 02 Aug 2026 09:45:14 GMT</pubDate>
            <atom:updated>2026-08-02T09:56:30.043Z</atom:updated>
            <content:encoded><![CDATA[<p><strong>A weekend experiment in combining LangGraph, FastAPI, and SwiftUI to catch VoiceOver gaps before they ship</strong></p><p>Accessibility testing on iOS has a familiar shape: it happens late, it happens manually, and it happens only if there’s time left at the end of the sprint. VoiceOver walkthroughs are tedious, WCAG criteria are easy to half-remember, and by the time someone notices a missing label or a broken reading order, the screen has already shipped.</p><p>I wanted to know if an agentic pipeline could change that – not by replacing human judgment, but by doing the tedious first pass automatically and handing a human a clear, actionable report instead of a blank screen and a checklist.</p><p>That question turned into a weekend build: <a href="https://github.com/Ankit-GenAI-Projects/ios_agentic_a11y_auditing"><strong>ios_agentic_a11y_auditing</strong></a></p><h3><strong>The shape of the problem</strong></h3><p>Accessibility auditing breaks down into a few distinct steps, and each one is a reasonable unit of work for an agent:</p><ol><li>Get to the right screen and extract what’s actually on it, structurally.</li><li>Normalize that into something you can check rules against.</li><li>Check it against specific WCAG criteria.</li><li>Reconcile overlapping findings into one coherent report.</li><li>Propose a fix – but don’t apply it without sign-off.</li></ol><p>That decomposition mapped cleanly onto a graph of specialized agents rather than one large prompt trying to do everything at once. So I built it as one.</p><h3>The pipeline</h3><p>The system is a five-stage LangGraph pipeline:</p><p><strong>Navigator</strong>. Reaches the target screen and pulls the raw accessibility tree. It supports two modes: live capture from a booted iOS Simulator via <a href="https://github.com/mobile-next/mobile-mcp"><strong>mobile-mcp</strong></a> (a tool-calling agent driving the simulator directly), or a deterministic fallback from a pre-captured JSON fixture. The fallback mattered more than I expected – it’s what let me build and test the rest of the graph without a Simulator in the loop every time.</p><p><strong>Normalizer</strong>. Raw accessibility trees are messy and inconsistent. This stage maps them into a structured schema – label, traits, hint, value, role, reading order – that the rule-checking agents can reason about consistently.</p><p><strong>Label Presence and Reading Order agents. </strong>These run in parallel and check the normalized tree against specific WCAG success criteria: 1.1.1, 1.3.1, 2.4.3, 2.4.6, and 4.1.2. Splitting these into separate agents rather than one “check everything” agent made the checks easier to reason about individually, and easier to extend later.</p><p><strong>Adjudicator</strong>. Findings from parallel checks can overlap or conflict. This stage consolidates everything into a single prioritized report.</p><p><strong>Fix Agent</strong>. Proposes a concrete SwiftUI fix per finding – then stops. LangGraph’s <strong>interrupt()</strong> halts execution until a human explicitly approves or rejects each fix. Nothing gets applied automatically. Given that this is generated code touching a real codebase, that felt like the only responsible default.</p><blockquote>Every LLM-capable node in the graph has two implementations, selected automatically by whether OPENAI_API_KEY is set: a real agentic path, and a deterministic rule-based fallback. The test suite runs against the fallback by default, which made the pipeline something I could actually validate rather than just eyeball.</blockquote><h3>Two front doors, one backend</h3><p>Both a Gradio demo UI and a native SwiftUI client talk to the same FastAPI backend – neither talks to LangGraph directly. That separation was deliberate: the Gradio UI was for fast iteration on the pipeline itself, while the native app was for testing whether the API actually felt usable from a real iOS client – polling run status, showing findings, expanding fix details.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/949/1*rWMbTITRRKxxXQcPEJaZSA@2x.jpeg" /></figure><p><strong>The Gradio UI – findings and proposed fixes for a completed, approved run.</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/603/1*6MnCxV7-HEF6Stpm-HT4Aw@2x.jpeg" /></figure><p><strong>The native SwiftUI AuditorApp – kicking off a new audit run.</strong></p><p>The API surface is small and boring on purpose:</p><blockquote>POST /audits. # start a run</blockquote><blockquote>GET. /audits/{id}/status. # poll status / proposed fixes</blockquote><blockquote>POST /audits/{id}/approve. # resume the interrupted graph</blockquote><blockquote>GET. /audits/{id}/report. # fetch the final structured report</blockquote><h4>Grading my own homework</h4><p>The hardest part of a project like this isn’t getting an agent to produce output – it’s knowing whether the output is *correct*. So instead of testing against a real production app, I built a small SwiftUI design system specifically as the audit target: fifteen components, seven of them seeded with deliberate VoiceOver gaps, ten correct control cases, including a few “broken vs. fixed” pairs of the same underlying pattern. That gave me a known-answer key (documented in `GROUND_TRUTH.md`) to measure the auditor’s precision against, rather than trusting it on faith.</p><p>That decision – building a deliberately broken test fixture before trusting the auditor’s output – was probably the single most useful thing I did all weekend.</p><h4>What’s out of scope, on purpose</h4><p>Current scope is VoiceOver only: label presence and reading order. Contrast, touch-target size, and dynamic type evaluation are explicitly deferred. The pipeline was structured – parallel category-agent fan-out into a single adjudicator – specifically so those can be added later as additional parallel nodes without restructuring anything that already works.</p><p>There’s also no persistent run store yet (runs live in an in-memory dict that resets on restart), status is poll-only rather than streaming, and live `mobile-mcp` Simulator capture is implemented but not yet verified end-to-end, since I didn’t have a Simulator available in the environment I built this in.</p><h4>Why this was worth a weekend</h4><p>I didn’t set out to build a production accessibility tool. I set out to see whether agentic orchestration – specialized agents, parallel evaluation, a human-in-the-loop interrupt – actually holds up on a problem I care about, rather than a toy example. It does, at least at this scale. The bigger lesson was less about LangGraph and more about the discipline of building a ground-truth test target before trusting any of it.</p><p>If you’re curious about the code, architecture diagrams, or the ground-truth mapping to WCAG criteria, it’s all here:</p><p><a href="https://github.com/Ankit-GenAI-Projects/ios_agentic_a11y_auditing">github.com/Ankit-GenAI-Projects/ios_agentic_a11y_auditing</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=177c94542142" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>