<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Sufyan Malik on Medium]]></title>
        <description><![CDATA[Stories by Sufyan Malik on Medium]]></description>
        <link>https://medium.com/@sufyanbinimran?source=rss-540e35bc4b8d------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*ORXMXaY0_FQHhONMH_6yqg.png</url>
            <title>Stories by Sufyan Malik on Medium</title>
            <link>https://medium.com/@sufyanbinimran?source=rss-540e35bc4b8d------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 18:05:11 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@sufyanbinimran/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[I Built an AI Pipeline That Profiles 48,000 Competitors for $100. Here’s How.]]></title>
            <link>https://medium.com/@sufyanbinimran/i-built-an-ai-pipeline-that-profiles-48-000-competitors-for-100-heres-how-456776e658eb?source=rss-540e35bc4b8d------2</link>
            <guid isPermaLink="false">https://medium.com/p/456776e658eb</guid>
            <dc:creator><![CDATA[Sufyan Malik]]></dc:creator>
            <pubDate>Tue, 30 Jun 2026 04:41:35 GMT</pubDate>
            <atom:updated>2026-06-30T04:41:35.302Z</atom:updated>
            <content:encoded><![CDATA[<h3>A field report on building a self-healing, evidence-driven SERP enrichment pipeline — where getting the model choice right mattered more than getting the architecture right.</h3><p>There’s a particular kind of problem that looks easy from a distance and gets harder the closer you get to it.</p><p>“Profile our competitors automatically” is one of those problems. It sounds like a weekend project. Scrape some search results, throw them at an LLM, write the answers to a database. Done.</p><p>A weekend later, you have a system that confidently tells you a personal-injury firm in Miami specializes in maritime law because the LLM read the wrong website. You have a $400 cloud bill because something re-ran. You have records silently missing because a prompt tweak last Tuesday quietly broke the JSON parser, and nobody noticed for nine days.</p><p>This is the story of how I built that system properly — for a law firm trying to scale Google Ads spend toward millions of dollars without burning half of it. And how the hardest parts turned out to be the ones I expected to be easiest.</p><h3>The setup: a market where every click is expensive</h3><p>Personal-injury law is one of the most expensive keyword categories on Google. A single click on <em>“car accident lawyer near me”</em> can cost <strong>tens to hundreds of dollars</strong>. The firms competing in this space don’t grow by being clever — they grow by spending. A lot. Eventually, millions.</p><p>But spend without intelligence is just expensive guessing. And in this market, guesses are punishing:</p><ul><li>Bid on a competitor’s branded term you can’t outrank → you pay every click, win none.</li><li>Target a city where a bigger firm dominates the local pack → your ads run, your phone doesn’t.</li><li>Miss a cheap, high-intent long-tail keyword for six months → that’s six months of cases going to whoever found it first.</li></ul><p>What the firm wanted, in one sentence: <strong>a continuously updated map of who’s advertising for what, in which cities, and how big a threat each one is</strong> — so they could pour money into the right ad groups with confidence.</p><p>The classical version of this is an analyst with a spreadsheet, opening competitor websites in fifty tabs and writing down what they do, where they operate, and how serious they look. That works for ten competitors. It does not work for ten thousand.</p><p>So: automate it. Use an LLM. Easy.</p><p>Except it isn’t.</p><h3>The five ways naive automation goes wrong</h3><p>Before I wrote a single line of pipeline code, I made a list of how this would fail if I built it the obvious way. The list mattered more than the architecture diagram, because every design decision downstream traced back to one of these:</p><p><strong>1. The AI will confidently name the wrong firm.</strong> This is the failure mode that ruins everything. If the model misidentifies a firm even once, every other field in that record — practice areas, geography, threat score — is now attached to the wrong company. One wrong identity poisons fifteen correct facts. Across thousands of records, the dataset becomes untrustworthy in a way that’s almost impossible to audit after the fact.</p><p><strong>2. Quality will break silently.</strong> You change a word in the prompt to make it “a little more precise.” Two records start failing validation. You don’t see it because they fail quietly. A week later you’ve lost 14% of new firms and your dashboard still looks fine.</p><p><strong>3. The cheap-looking shortcut is the expensive one.</strong> The instinct is to fetch raw HTML cheaply and then have the LLM interpret it. But that’s two paid steps where one would do — and the cheap scrape tier often returns snippets, or JavaScript-rendered shells, or noisy markup that costs more tokens to clean than to just have the model read the page itself.</p><p><strong>4. You will pay twice for the same work.</strong> A retry fires, a sensor double-triggers, somebody re-runs yesterday’s job. Without deduplication, your bill scales with your bug count.</p><p><strong>5. You will run on a timer when there’s nothing to do.</strong> “Daily at 3am” sounds responsible. It also means you fire a full enrichment pass on days when zero new firms appeared in the SERPs — paying for work that produces no new information.</p><p>Each of these became a design constraint. The system I shipped was, more than anything else, a system that refuses to make any of those five mistakes.</p><h3>The architecture, in four moves</h3><p>The pipeline has four stages. Each one was kept dumb on purpose — the intelligence lives in <em>how they connect</em>, not in any single one of them.</p><h3>Stage 1 — Scrape</h3><p>A controlled scraper calls a SERP API for every target term, in every target city, on the chosen device class. It pulls back the parsed Google results page — paid ads, organic, local pack, people-also-ask, knowledge panel, related searches — and writes the raw structured response to the warehouse.</p><p>The scrape schedule lives in a spreadsheet, not in code. This sounds trivial. It is not. It means non-engineers can decide what to track without filing a pull request, and the engineering team isn’t a bottleneck on what the system pays attention to.</p><h3>Stage 2 — Flatten</h3><p>Raw SERP responses are deeply nested and irregular. Some results have ad extensions, some don’t. Some have sitelinks, some have callouts, some have nothing. Local-pack results are shaped completely differently from paid ads.</p><p>A SQL transformation layer explodes all of that into <strong>one row per individual result item</strong> — every paid ad, every organic blue link, every local listing — tagged with result_type, the city and state and device, and the advertiser domain extracted from the URL.</p><p>This is the part of the system I was most careful with. Everything downstream reads from this flattened table. It’s the <strong>shared contract</strong> between the rest of the pipeline, and getting it right meant the LLM enrichment stage and the keyword mining stage could be built independently without coordinating on schemas. One table, one truth, tested.</p><h3>Stage 3 — Enrich</h3><p>For every <em>new</em> advertiser domain — meaning every firm we’ve never profiled before — the LLM does one job: read the firm’s own website and return a strict, schema-validated profile.</p><p>What that profile contains:</p><ul><li><strong>Identity</strong> — name, whether it’s actually a law firm, entity type, rough size.</li><li><strong>Practice areas</strong> — which kinds of cases they handle (auto, premises, workers’ comp, mass tort, etc.) with the supporting text.</li><li><strong>Geography</strong> — which states they’re licensed in, which markets they advertise into.</li><li><strong>Threat score</strong> — a 0–10 number rolled up from practice-area overlap with the firm, geographic overlap, and search visibility. Bucketed into High / Medium / Low.</li></ul><p>This stage is where most of the actual money goes and most of the actual risk lives. I’ll come back to it.</p><h3>Stage 4 — Mine</h3><p>Separately, from the <em>paid</em> results only, a different process extracts two kinds of candidate keywords:</p><ul><li><strong>Advertiser names</strong> — who’s paying to appear, per market.</li><li><strong>Related-search terms</strong> — Google’s own “people also searched for” lists, surfaced alongside the ads.</li></ul><p>These get aggregated per (term, city, state) — how often they appear, in how many markets, at what average ad position — then checked against the firm&#39;s master keyword list and flagged if they look branded (matching a known competitor name from Stage 3).</p><p>Nothing is thrown away. Every candidate is annotated (<em>new vs. already targeting</em>, <em>branded vs. generic</em>, <em>needs review vs. ready</em>) and handed to the ad-automation team. The pipeline doesn’t decide what becomes an ad group. People do. The pipeline just makes sure the people see everything worth seeing.</p><h3>The decision that mattered more than the architecture</h3><p>You can spend weeks architecting a pipeline and still ship a broken system if you pick the wrong model. I learned this the way I learn most things: by almost picking the wrong one.</p><p>The intuitive choice for “read a website and extract structured information” is the biggest, smartest model available. More context, more reasoning, more careful answers. Obviously better.</p><p>So I built an evaluation set — a couple hundred firm websites, hand-labeled with the correct answers — and ran both candidate models on the same inputs. Same prompt. Same temperature. Same tool access.</p><p>Here’s what came back.</p><p>The larger model got the <strong>firm’s identity wrong on 40% of records.</strong> Forty percent. Sometimes it confused a firm with a similarly named one in another state. Sometimes it read directory pages or sponsored listings on the same domain and pulled the wrong entity. Sometimes it just hallucinated a plausible-sounding firm name that didn’t exist.</p><p>The smaller, faster model — Gemini 2.5 Flash on Vertex AI — got identity wrong on <strong>0%</strong> of the same set. Its errors, when it had them, were minor and recoverable: occasional missing practice area, occasional under-confident threat score. Stuff you could re-run.</p><p>Read those numbers again. The “smarter” model was wrong about <em>who the firm even was</em> almost half the time. And remember: wrong identity poisons every other field. A 40% identity error rate doesn’t mean “60% useful data.” It means <strong>the entire dataset is unusable</strong>, because you have no way of knowing which 60%.</p><p>I picked Flash. It was cheaper <em>and</em> more accurate <em>and</em> faster. The lesson generalized: <strong>measure the failure mode that actually breaks your product, not the benchmark that looks good in marketing slides.</strong></p><h3>Engineering the cost down to $0.002 per firm</h3><p>Once Flash was settled, the next question was: how cheap can I make this without losing quality?</p><p>I benchmarked three approaches on the same eval set.</p><p><strong>Approach A: Live web search add-on.</strong> Give the model a search tool, let it find supporting pages, summarize across them. Felt premium. Cost more. Produced essentially identical accuracy on this task because the firm’s own website already contained everything we needed. Switched off.</p><p><strong>Approach B: External crawling service, then LLM.</strong> Use a dedicated crawler to fetch the page, pass the cleaned text to the model. One service we tested cost roughly <strong>2.5x more</strong> than reading directly. Another returned only snippets — not full page content — which forced the model to guess at things it couldn’t see.</p><p><strong>Approach C: URL-context tool, single LLM call.</strong> Give the model the firm’s URL and let it read the page itself, in one call, returning the structured profile. Cheapest. Most accurate. Done.</p><p>Cost settled at roughly <strong>$0.002 per firm enriched.</strong> That’s $100 per ~48,000 firms. Modeled forward to ten million firms, the pipeline is still in the low five figures.</p><p>Two cost surprises worth flagging:</p><ol><li>Some firm websites are absurdly content-heavy — multi-megabyte JS-rendered SPAs with every blog post inlined. These cost several times the average per call. I instrumented per-call costs in observability so we can flag them and decide whether to truncate or skip.</li><li><strong>De-duplication does most of the work.</strong> Only <em>new</em> domains get enriched. If a firm appears in 200 SERP scrapes, the LLM runs once. The contract table makes this trivial — SELECT DISTINCT domain WHERE domain NOT IN profiled_domains.</li></ol><h3>Self-healing: the part that lets me sleep</h3><p>Here’s the unsexy truth about LLM pipelines: <strong>individual calls will fail.</strong> Timeouts. Rate limits. A weird URL. A page that returns 503. A response that almost-but-not-quite matches the schema.</p><p>The naive response is to retry-three-times-and-fail. The better response is to assume failure is the normal case and build the system to recover from it without losing data or demanding attention.</p><p>What I built instead:</p><ul><li>Every record that fails enrichment is <strong>held in a “needs retry” table</strong> with the failure reason captured.</li><li>On the next pipeline run, anything in retry gets another attempt — up to <strong>three retries</strong> total.</li><li>The moment a retry succeeds, the record is promoted to the trusted set automatically. No human in the loop.</li><li>Only after three failed attempts does a record surface for human review, and at that point the reason is already structured (“URL unreachable”, “schema validation failed on field X”, “model returned null name”).</li></ul><p>Combined with <strong>exactly-once batch processing</strong> (each batch of new domains is processed precisely once, even if upstream sensors fire multiple times), this means: no duplicate spend, no lost data, no silent drops, and almost no babysitting.</p><p>The pipeline is event-driven, not timer-driven. Stage 3 doesn’t fire because it’s 3am — it fires because a fresh scrape and a fresh flatten have both completed and there are new domains to look at. On days with no new data, it does nothing, which is exactly correct.</p><h3>Observability: making the invisible visible</h3><p>The thing about LLMs in production is that they fail in ways that don’t look like failures. The response is well-formed JSON. The schema validates. The cost looks normal. And the answer is wrong.</p><p>You don’t catch this with logs. You catch it with measurement.</p><p>Every LLM call in the pipeline is traced — cost, tokens in, tokens out, latency, the prompt version that produced it, the parsed output, and an <strong>automated quality score</strong> that flags low-confidence or anomalous answers. Prompts are versioned outside the code, so improvements ship without deploys and can be A/B tested against each other.</p><p>Two things this unlocked:</p><ol><li><strong>Forward cost modeling.</strong> Because we measure cost per call across thousands of real firms, we can project spend at any scale — from a hundred firms to ten million — before scaling. No surprises in the cloud bill.</li><li><strong>Drift detection.</strong> When prompt v7 quietly drops practice-area extraction quality vs. v6, the quality score moves. We see it. We roll back. The dataset stays trustworthy.</li></ol><p>A human-reviewed quality pass is being added on top of the automated scores — not because the automated scores are wrong, but because validation is one of those places where belt-and-suspenders is the right answer.</p><h3>The tech stack, briefly</h3><p>For people who want the actual names:</p><ul><li><strong>Dagster</strong> — orchestration. Asset-based, event-sensor driven, exactly-once execution semantics.</li><li><strong>Decodo</strong> — SERP scraping. Parsed Google results across geos and devices, with the rendering and pagination handled.</li><li><strong>Google BigQuery</strong> — warehouse. Stores raw scrapes, the flattened contract table, validated profiles, retry queue.</li><li><strong>Google Vertex AI · Gemini 2.5 Flash</strong> — LLM enrichment. With the URL-context tool for single-call website reads.</li><li><strong>Langfuse</strong> — observability and prompt management. Per-call traces, automated quality scoring, versioned prompts.</li><li><strong>dbt-style SQL transformations</strong> — the flatten layer. With data tests on every assumption.</li><li><strong>Pydantic</strong> — schema validation on every LLM output. Malformed responses never reach the warehouse.</li><li><strong>PostgreSQL</strong> — the firm’s master keyword store, read-only.</li><li><strong>Python + SQL</strong> — everything else.</li></ul><p>The stack matters less than the principles. A different combination of tools could solve the same problem. What can’t be substituted is the discipline of evaluating before assuming, designing for failure as the default case, and measuring everything you can’t see.</p><h3>What I’d tell anyone building one of these</h3><p>If you’re building an LLM pipeline in production, six things from this project that I’d hand to you on a postcard:</p><p><strong>1. Pick the model on your eval set, not on the leaderboard.</strong> The leaderboard tells you which model is generally better. Your eval set tells you which model is better <em>at the thing your business actually loses money on</em>. Those are not the same question.</p><p><strong>2. Identity is the load-bearing field.</strong> If your output has any kind of “what entity is this about” question, that field decides whether everything else is data or garbage. Validate it hardest.</p><p><strong>3. One contract table, many consumers.</strong> The flatten layer was the boring part of this project and the part that paid back the most. Spend the time on it. Test it. Treat it like an API.</p><p><strong>4. Single calls beat split steps.</strong> Every additional moving part is another place where state gets lost, costs accumulate, or quality degrades. If the model can do the job in one call, give it the job in one call.</p><p><strong>5. Design for failure as the default case.</strong> Retry queues, structured failure reasons, exactly-once semantics, automatic promotion on success. None of this is glamorous. All of it is what makes the difference between a pipeline you maintain and a pipeline that maintains itself.</p><p><strong>6. Measure what you can’t see.</strong> Cost, quality, drift, latency, prompt version — if it isn’t traced, it’s a future incident you haven’t found yet.</p><h3>The outcome</h3><p>The firm now has something they couldn’t get any other way: a continuously refreshed, structured, scored view of who they’re up against, in every market they advertise in, updated as the SERPs themselves change.</p><p>They can scale paid-search spend toward millions of dollars with budget aimed at validated terms, in validated markets, against competitors whose threat level they actually know. The wrong-firm error rate that would have wrecked the dataset is zero. The unit cost is predictable enough to model spend at ten-million-firm scale. And when something fails — because something always fails — the system handles it before anyone notices.</p><p>That’s the whole story. The architecture isn’t exotic. The model isn’t bleeding edge. The stack isn’t proprietary. What made it work was treating every decision as falsifiable, every failure mode as the default case, and every claim about the system as something to be measured rather than assumed.</p><p>Most production AI systems don’t need to be smarter than what they’re built on. They just need to be honest about what they are.</p><p><em>Thanks for reading. If you’re working on something similar — SERP intelligence, competitor enrichment, LLM pipelines that have to be cheap and trustworthy at scale — I’d be glad to compare notes.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=456776e658eb" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How I Built a Payments Troubleshooting Chatbot That Cut Diagnosis Time From an Hour to Four Seconds]]></title>
            <link>https://medium.com/@sufyanbinimran/how-i-built-a-payments-troubleshooting-chatbot-that-cut-diagnosis-time-from-an-hour-to-four-seconds-3157dc44e78f?source=rss-540e35bc4b8d------2</link>
            <guid isPermaLink="false">https://medium.com/p/3157dc44e78f</guid>
            <dc:creator><![CDATA[Sufyan Malik]]></dc:creator>
            <pubDate>Sun, 28 Jun 2026 20:14:43 GMT</pubDate>
            <atom:updated>2026-06-28T20:14:43.497Z</atom:updated>
            <content:encoded><![CDATA[<p><em>A self-hosted, read-only AI agent that explains — in plain English — why a customer’s payment is stuck or failed. Here’s how it was designed, what almost went wrong, and the engineering decisions I’d make again.</em></p><h3>The 60-minute problem nobody wanted to own</h3><p>The platform I was building for moves money all day. Deposits in, withdrawals out, across <strong>more than 100 payment providers</strong> — global card processors, e-wallets, and a long tail of regional bank gateways. On paper it works. In practice, a meaningful slice of payments get <strong>stuck</strong> (parked in an early state like init or pre-sold for far too long) or <strong>fail</strong> outright.</p><p>When that happens, the customer sees their money frozen. They open a support ticket. Support escalates to engineering. And here’s where the pain compounded:</p><p>To explain <strong>one</strong> stuck or failed payment, a senior engineer had to hand-grep logs across several disconnected systems — the transaction facts in the warehouse, a comment/event timeline, and a separate lifecycle table — and stitch the story together by eye.</p><p>That was <strong>20 to 60 minutes of senior-engineer time per ticket</strong>. Repetitive. Linear with volume. And exactly the wrong people doing the work.</p><p>The brief was simple to state:</p><blockquote><em>Build an internal chatbot. Drop in a transaction. The bot reads the logs across systems. It tells us why the payment is stuck or failed.</em></blockquote><p>The target was to get diagnosis under five minutes. We landed at <strong>roughly four seconds</strong> for a grounded answer.</p><p>Here’s how.</p><h3>The five constraints I refused to bend on</h3><p>Before writing a line of code, I locked down a short engineering constitution. Every later decision had to pass through it:</p><ol><li><strong>Self-hosted.</strong> No external SaaS in production. No customer data leaving the building.</li><li><strong>Read-only.</strong> The agent <em>diagnoses</em>. It never writes, retries, refunds, or escalates.</li><li><strong>Grounded and PII-safe.</strong> Every answer is backed by real tool output or it says “insufficient context.” It never invents data. It redacts personal information.</li><li><strong>Authenticated with full audit.</strong> SSO, JWT, role-based access, and a trail of every query.</li><li><strong>Observability and evaluation first.</strong> Nothing merges without traces, logs, and a regression eval set.</li></ol><p>Treating these as non-negotiable up front saved me from a dozen tempting shortcuts later.</p><h3>The 80/20 that made a one-week pilot possible</h3><p>The warehouse had <strong>146 payment-related tables across 20+ databases</strong>. I did not try to boil that ocean.</p><p>When I actually traced what answered the question “why is this payment stuck or failed?”, I found I only needed <strong>four tables</strong>:</p><p>Table Role balance_history The transaction header — current state, type, provider, amount, currency, timestamps, and the per-customer scope key balance_history_comment The chronological event timeline per transaction — the gold mine payment_system A tiny lookup table mapping provider IDs to human names fact_balance_transaction_lifecycle Pre-computed lifecycle timestamps and a clean failed_reason string</p><p>The join spine across all of them is a single transaction ID. That’s it.</p><p>The most important insight: <strong>the comment timeline already contained the structured story</strong> per transaction. Which meant the bot never had to touch the 150-million-row raw log table. That single observation is what made the pilot achievable in a week instead of a quarter.</p><p>I explicitly scoped out the other ~140 tables, the internal-transfer table (zero stuck/failed matches in practice), and a heavy PII-laden federated payments table (no extra signal). Less surface, less risk, less time.</p><h3>The architecture</h3><p>Everything runs inside the host’s internal Docker network. <strong>Exactly one port is publicly reachable</strong> — the chat UI. Every other service is internal only.</p><pre>Browser<br>  → TLS edge (reverse proxy)<br>  → SSO / OIDC auth proxy<br>  → UI service          (Next.js — the only public port)<br>      → API gateway      (FastAPI + JWT + the agent)<br>          → ClickHouse    (read-only — payment facts)<br>          → PostgreSQL    (read/write — chat history + agent checkpoints)<br>      → LLM call          (provider abstraction — swappable)</pre><p>The user’s account ID rides on the auth token. <strong>Every single read against the warehouse is scoped by it.</strong> There is no “look at everyone” mode. This is the backbone of the isolation guarantee I’ll come back to in a minute.</p><p>Conversations and agent checkpoints are keyed by {account_id}:{thread_id} in PostgreSQL — so each user&#39;s history is physically isolated, not just filtered.</p><h3>The agent pipeline</h3><p>I built the agent as a LangGraph with eight nodes, run in order on every turn:</p><ol><li><strong>Classifier</strong> — routes the request type.</li><li><strong>Payment-relevance check</strong> — is this even about payments? Off-topic gets a polite decline.</li><li><strong>Guardrail</strong> — a fast safety classifier that blocks abuse, prompt injection, write/refund requests, PII fishing, and cross-customer probing. Own-account aggregates are allowed.</li><li><strong>Follow-up detection</strong> — works out whether this turn continues an earlier one.</li><li><strong>Sufficient-context check</strong> — do I have enough (who, when, which transaction) to answer?</li><li><strong>Agent loop</strong> — the tool-calling core. It first classifies <em>scope</em>: <strong>OPEN</strong> (ask one WHO/WHEN question, no tool calls yet), <strong>SPECIFIC</strong> (I have a transaction ID), or <strong>BULK</strong> (a list, top-N, or “most” request). Prior turns get replayed in so context carries.</li><li><strong>Synthesizer</strong> — writes the grounded, customer-voice answer.</li><li><strong>Validator</strong> — a final grounding check, PII redaction, and ban-phrase scrub before anything leaves the system.</li></ol><p>The agent only has access to a small read-only adapter — five methods, all scoped by account ID. It can fetch a transaction’s full context, get an account summary, count by state, search recent stuck/failed payments, and look up provider names. That’s the entire blast radius.</p><h3>Two ways to ask, one philosophy</h3><p>I wanted the bot to work for two very different customer behaviors:</p><p><strong>The specific case:</strong> <em>“Why is my transaction 6303335 stuck?”</em> — the customer already has an ID. I go straight to the facts.</p><p><strong>The open case:</strong> <em>“What’s stuck on my account?”</em> — the bot asks <strong>one</strong> clarifying WHO/WHEN question, then answers.</p><p>I deliberately kept it to a single question. A chatbot that interrogates you with a form is a chatbot nobody uses. One question, then results.</p><h3>The bug that justified the whole eval suite</h3><p>This is the part I’m most willing to tell on myself about.</p><p>Early on, the function that fetched a transaction’s “full context” looked fine. It scoped the <strong>header</strong> by account ID — so far so good. But the <strong>timeline</strong> and the <strong>lifecycle row</strong> were keyed by transaction ID alone.</p><p>That meant a customer could read another customer’s event history just by guessing a transaction ID.</p><p>I did not catch this in code review. I did not catch it in unit tests. The behaviour-eval suite caught it on its very first run, the first time I asked it to verify isolation.</p><p>The fix was small: when the header isn’t the caller’s, the timeline and lifecycle return empty. The lesson was bigger:</p><blockquote><strong><em>Behaviour evals are not a nice-to-have. The single most important bug in this project was found by an eval, not by a human reading code.</em></strong></blockquote><p>I also tightened the response so a missing transaction and someone-else’s-transaction return the <strong>identical</strong> “not on your account” message — so a customer can’t even probe whether another customer’s transaction exists.</p><p>A regression test sits in the suite now. It can never come back.</p><h3>The other subtle bug: counting wrong</h3><p>Customers asked “how many of my payments are stuck vs failed?” all the time. The first version answered by scanning recent rows with a cap and bucketing them in code.</p><p>It was quietly under-reporting whenever a customer had more transactions than the cap.</p><p>I replaced it with a real count() GROUP BY state aggregate. Small change, outsized trust impact.</p><blockquote><em>Wrong numbers erode confidence faster than missing features. If you ship counts, make them real counts.</em></blockquote><h3>The currency trap</h3><p>When a customer has transactions in multiple currencies and asks for “the biggest one,” the obvious thing is to sort by amount. The obvious thing is wrong.</p><p>100 of one currency is not 100 of another, and I had no FX conversion tool in the read-only surface. So I built a guard: when the result spans multiple currencies, the bot <strong>does not rank by amount.</strong> It names the currencies and refuses to pretend they’re comparable.</p><p>A small refusal is better than a confident lie.</p><h3>Grounding, in one sentence</h3><p>Every answer the bot ever produces is either backed by tool output it can point to, or it says “insufficient context.” A final validator scrubs PII and ungrounded phrasing before the answer leaves the system.</p><p>There is no “use your best guess” mode. There is no creative-license fallback. If the tools come back empty, the bot says so. That single discipline is what makes the system trustworthy enough to put in front of a customer.</p><h3>On the LLM (the honest version)</h3><p>The model sits behind a provider abstraction — swappable with one environment variable.</p><p>The <strong>target</strong> is fully self-hosted: vLLM serving Llama 3.1 8B Instruct on an internal GPU box with prefix caching on. That satisfies the “nothing leaves the building” constraint completely.</p><p>During the pilot — before the GPU box was provisioned — the synthesizer called a hosted frontier model behind the same abstraction. <strong>Only the synthetic dataset was ever sent to it. Never real customer data.</strong> And because of the abstraction, flipping to the self-hosted model is a one-file change.</p><p>This is a pattern I’d recommend to anyone building production LLM systems: treat the model as a swappable component from day one. Your runtime stack will outlive your model choice.</p><p>The fast safety classifier always runs on the smaller, cheaper model. There’s no reason to pay frontier prices to detect abuse.</p><h3>The deployment</h3><p>The whole stack ships as a Docker Compose project onto a single internal Linux host:</p><ul><li><strong>UI</strong> — Next.js chat front end (the only public port)</li><li><strong>Gateway</strong> — FastAPI + JWT + the LangGraph agent (internal only)</li><li><strong>ClickHouse</strong> — payment facts (internal only)</li><li><strong>PostgreSQL</strong> — chat history and agent checkpoints (internal only)</li></ul><p>Data lives in named volumes that survive restarts. Deploy is a single scripted command — sync, build, up.</p><p>A few operational lessons got baked straight into the compose file so they can’t be undone:</p><ul><li>The database healthcheck hits an explicit IPv4 loopback because IPv6 is disabled in the container. Using localhost made the container appear unhealthy forever even though the DB was up.</li><li>The warehouse user is provisioned inside the image, not via env vars at runtime, because the env-var path triggered a startup race on a temp container that collided on ports.</li></ul><p>These are the kinds of details you only learn by watching a deploy hang at 2am.</p><h3>What I’d tell anyone building something like this</h3><p>A few principles, ranked roughly by how much pain they saved me:</p><ol><li><strong>Write a constitution first.</strong> Five non-negotiables. Refer back to them when you’re tempted by a shortcut. Most architectural arguments dissolve when one of the rules clearly settles them.</li><li><strong>Find the minimum data.</strong> Four tables out of 146 was the difference between a one-week pilot and a one-quarter project. The question “what’s the smallest dataset that answers this?” is undervalued.</li><li><strong>Wire behavior evals before features.</strong> They will find the bug you missed. They found mine on day one.</li><li><strong>Scope every read by the user’s identity.</strong> Not at the UI. Not at the gateway. At the data layer. Defense in depth means assuming the layer above you is wrong.</li><li><strong>Make refusals cheap and confident lies impossible.</strong> “Insufficient context” beats a hallucinated answer every time, especially in finance.</li><li><strong>Treat the model as swappable.</strong> Provider abstraction with a one-env-var switch. Your stack should not break when you change models.</li><li><strong>Counts must be real counts.</strong> If you’re showing numbers, they have to be exact. A capped scan is not a count.</li></ol><h3>The headline result</h3><p>Payment diagnosis went from <strong>20–60 minutes</strong> of senior-engineer log-grepping to a grounded, plain-language answer in <strong>about four seconds</strong>, with true per-customer data isolation, full audit, and zero ability to mutate data.</p><p>The customer gets their answer the moment they ask. The senior engineers get their time back. The system tells the truth or it tells you it can’t.</p><p>That’s the whole product.</p><p><em>If you’re building an AI agent that touches real customer data, the boring parts — grounding, isolation, evals, read-only by default — are the parts that decide whether you ship a product or an incident. Build those first.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3157dc44e78f" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How Computer Vision Fixed Checkout Queues for Big Retail Stores]]></title>
            <link>https://medium.com/@sufyanbinimran/how-computer-vision-fixed-checkout-queues-for-big-retail-stores-5e9c34316849?source=rss-540e35bc4b8d------2</link>
            <guid isPermaLink="false">https://medium.com/p/5e9c34316849</guid>
            <dc:creator><![CDATA[Sufyan Malik]]></dc:creator>
            <pubDate>Sun, 28 Jun 2026 18:45:07 GMT</pubDate>
            <atom:updated>2026-06-28T18:47:04.466Z</atom:updated>
            <content:encoded><![CDATA[<h3>How We Used Existing Store Cameras to Solve Retail’s Checkout Walkout Problem.</h3><p>It’s a Friday evening at a big grocery store. The kind of evening the finance team loves on paper. The aisles are full, the baskets are filling, and the day should print money.</p><p>Then I watched it happen, over and over, on the camera feeds: a customer walks in through the entrance, takes three steps, looks toward the checkout, sees the queues curling back into the aisles — and turns around. Walks straight back out. The basket was never even picked up.</p><p>That moment, multiplied by thousands, is the story of this project.</p><h3>The number that started everything</h3><p>We didn’t want a hunch. We wanted a number. So we sat with the footage and the entry/exit counts from a few stores — Carrefour, Imtiaz, Al-Fatah — and measured the walkouts on busy days.</p><p>The number came back ugly. On peak days — Fridays, festivals, anything with heavy footfall — <strong>45 to 50% of the people who entered left without buying anything.</strong> Not because prices were wrong. Not because shelves were empty. Because the checkout <em>looked</em> unbearable.</p><p>That’s the cruel part. The loss was biggest on exactly the days that should have been the most profitable. It wasn’t a pricing problem or a product problem. It was a <em>perception-of-wait</em> problem — and perception is fixable.</p><h3>The constraint that shaped the whole design</h3><p>When we took this to stakeholders, the brief was blunt: solve it cheaply, reuse what we already have, and make the return obvious. No big hardware bills. No rip-and-replace.</p><p>At first that sounds like a handcuff. It turned out to be the best thing that could’ve happened, because it forced the key realization:</p><p><strong>The stores were already sitting on the answer.</strong></p><p>Every one of them had high-quality CCTV — already mounted, already wired, already running across a NAT-configured network. We didn’t need to sell cameras. We needed to make the cameras <em>think</em>. Software on top of feeds they’d already paid for. Near-zero capital cost, fast payback, and a pitch that practically closes itself.</p><blockquote>Suggested image: a simple “before / after” — a crowded entrance view vs. a calm one, or a shot of an existing CCTV camera.</blockquote><h3>The big idea: read the journey, not the person</h3><p>Here’s the mental model we built the system around — the journey a real customer takes through a store, broken into moments a camera can actually catch.</p><p>A customer walks through the gate. The entrance camera detects the person, crops the face, reads a rough demographic (age band, gender), and mints a lightweight, short-lived ID for them. Think of it as a little ticket: <em>person #4471, roughly 30s, male, entered 18:42.</em></p><p>That ticket is the start of everything. The question we have to answer next is deceptively simple: <strong>did this person actually make it to a till, or did they bail?</strong></p><p>How we answered that question is where the real engineering lives.</p><h3>The fork in the road</h3><p>There were two honest ways to confirm someone reached a checkout, and choosing between them decided the cost and the accuracy of the entire product.</p><p><strong>Option A — follow them everywhere.</strong> Track the person from the entrance, through every aisle, all the way to the counter. It’s the obvious approach, and it’s a trap. It’s expensive (you’re running tracking across dozens of cameras at once) and it’s fragile (people vanish behind shelving, two shoppers cross paths and their IDs swap, someone stops to read a label for two minutes and the track dies). You end up paying more to get <em>worse</em> data.</p><p><strong>Option B — fingerprint at the door, re-check at the till.</strong> Don’t follow them at all in between. At the entrance, take a face <em>embedding</em> (a compact mathematical fingerprint) plus the demographics, and stash it. Then only re-check at the checkout zone. If the same face shows up at a till, that’s a match — a confirmed shopper. If it never shows up, they left.</p><p>We chose Option B, and it’s the decision I’m proudest of. It’s far cheaper, it has dramatically fewer errors, and — this is the point — it answers the <em>only</em> question that actually matters. We never needed to know which yogurt someone looked at. We needed to know whether the door-to-till promise was being kept.</p><h3>Drawing the map: zones and tripwires</h3><p>A camera sees pixels. To turn pixels into meaning, we gave each camera a map.</p><p>During setup, we draw polygons directly on each camera’s view: an <strong>entrance zone</strong>, one <strong>checkout polygon per lane</strong>, and a <strong>tripwire line</strong> across the doorway for counting people in and out.</p><p>After that, it’s geometry. Every detected person has a “foot-point” — roughly where they’re standing on the floor — and we test that point against the polygons. So at any instant the system knows exactly which zone a person is in: at the door, in lane 6, somewhere in between.</p><blockquote>Suggested image: a camera frame with colored polygon overlays on the lanes and a line across the entrance.</blockquote><h3>Who actually counts as a customer</h3><p>Not everyone in the building is a shopper. Staff walk around. Browsers wander and leave. Delivery people pass through.</p><p>So we defined a <strong>valid customer</strong> precisely: someone whose ID was created at the entrance <strong>and</strong> later matched inside a checkout zone. Entry-only IDs (people who left) don’t count. Till-only blips don’t count. You need both ends of the journey.</p><p>That single rule quietly cleans up the data. And for staff specifically, we keep them off the books with a known-faces allowlist, so an employee crossing a checkout zone forty times a day doesn’t pollute the numbers.</p><h3>Reading the wait — the honest way</h3><p>Counting heads in a queue is easy and misleading. A line of four people paying cash can take longer than a line of seven tapping cards. So we measured <em>time</em>, not just bodies.</p><p>For each lane polygon, we time how long every person dwells from the moment they join the queue to the moment they reach the till. Average that across the people currently in the lane and you get a live, realistic wait estimate.</p><p>Then we weight it with the factors that genuinely move service speed: cash payers are slower than card payers, and older shoppers tend to be a touch slower than younger ones. The result is a wait number that reflects what’s actually about to happen, not a naive headcount.</p><h3>The whole point: a nudge, at the right moment</h3><p>Everything above exists to power one tiny, valuable action.</p><p>Each lane has a threshold — a combination of queue length and predicted wait. The instant a lane crosses it (say, a predicted wait over four minutes, or more than eight people deep), the system pings the floor manager: <em>“Lane 6 is overloaded — open another till.”</em></p><p>A manager opens a lane. The pressure releases. And the next customer who walks through the entrance sees a calm, moving checkout instead of a wall of people — so they stay.</p><p>We didn’t solve the queue at the queue. We solved it at the <em>door</em>, by keeping the visible rush from ever forming.</p><h3>Under the hood: how the video actually flows</h3><p>Here’s the plumbing, because the architecture is half the story.</p><p>The cameras already speak <strong>RTSP</strong> over the store’s network, so a lightweight frame-reader service on-site (OpenCV / GStreamer) just pulls those streams.</p><p>The brains run <strong>on-premises, on small GPU edge boxes sitting right next to the cameras.</strong> This was deliberate, for two reasons. First, latency — an alert that arrives ninety seconds late is useless. Second, privacy and bandwidth — raw video never leaves the building. Only tiny structured events travel upstream.</p><p>Every meaningful moment becomes a small JSON event — <em>“entered,” “matched at lane 6,” “lane 6 wait = 5 min”</em> — published onto <strong>Apache Kafka.</strong> Kafka is the spine of the system. It decouples the cameras from everything downstream, so onboarding a new store is a configuration job, not a re-plumbing job.</p><p>The face fingerprints used for re-identification live in <strong>Redis with a TTL</strong> — they automatically expire after the session ends. That’s good for storage, and it’s good for privacy: we’re not building a permanent biometric archive, we’re matching a face to itself for a few minutes and then forgetting it.</p><p>Finally, the numbers land in two places. Events stream into <strong>ClickHouse</strong> for the real-time dashboards — the sub-second views the operations staff actually watch. And they roll up into <strong>BigQuery</strong> for the historical questions, like how this month’s Fridays compare to last month’s.</p><blockquote>Suggested image: a clean architecture diagram — Cameras → Edge inference → Kafka → (Redis) → ClickHouse / BigQuery → Dashboards.</blockquote><h3>The models — and the part everyone gets wrong</h3><p>We didn’t train from scratch. That would’ve been slow, expensive, and unnecessary. We stood on the shoulders of strong pretrained models:</p><ul><li><strong>YOLO</strong> for detecting people in the frame.</li><li><strong>RetinaFace / SCRFD</strong> for finding faces on the person crop.</li><li><strong>ByteTrack / DeepSORT</strong> for keeping a consistent ID within a single camera.</li><li><strong>ArcFace (InsightFace)</strong> for the face embedding — the fingerprint we match at the till.</li><li><strong>InsightFace’s genderage model</strong> for the rough demographics.</li></ul><p>But pretrained weights only get you to the starting line. Store CCTV is its own strange world — overhead angles, glare, dense crowds, motion blur — so we annotated a sample of each store’s own footage and fine-tuned the person detector for those exact conditions.</p><p>And here’s the part teams routinely get wrong: <strong>demographics recalibration.</strong> Off-the-shelf age and gender models are trained mostly on Western datasets, and they skew badly on a Pakistani crowd. If we’d shipped them as-is, the demographic numbers would’ve been confidently wrong. So we recalibrated them on local faces. It wasn’t optional — accuracy depended on it. If you take one modeling lesson from this post, take that one.</p><h3>From frames to dashboards: the data pipeline</h3><p>Stepping back, the end-to-end flow looks like a clean data-engineering pipeline:</p><ul><li><strong>Ingestion</strong> — RTSP streams → frame reader → on-prem inference → Kafka producer.</li><li><strong>Orchestration</strong> — <strong>Dagster</strong> runs the batch side: nightly aggregations, the demographics-drift checks, and retraining jobs.</li><li><strong>Transformation</strong> — <strong>dbt</strong> turns raw events into clean, trustworthy tables: sessionizing each visit, computing per-lane wait curves, and building the entrance-to-checkout conversion funnel.</li><li><strong>Warehousing</strong> — <strong>ClickHouse</strong> for the hot, real-time layer; <strong>BigQuery</strong> for the historical depth.</li><li><strong>Dashboarding</strong> — <strong>Grafana</strong> for the live operations wall (lane status, active alerts), and <strong>Superset</strong> for the stakeholder view (conversion lift, peak-hour patterns, recovered customers).</li></ul><p>It’s a genuinely <strong>hybrid</strong> setup: inference stays on-prem for latency and privacy, while warehousing, dashboards, and training live centrally.</p><h3>Keeping it alive: MLOps and drift</h3><p>A computer-vision system isn’t “done” when it ships — it quietly rots if you ignore it. A camera gets nudged, lighting changes, the crowd shifts, and one day your stats are subtly lying to you.</p><p>So we kept the operations lean but real. Models are versioned in a registry (<strong>MLflow</strong>), shipped through the existing CI/CD path (<strong>Bitbucket → Jenkins</strong>), and run as services on <strong>Nomad.</strong> We watch for <strong>drift</strong> — if the demographic mix or detection confidence starts sliding, that’s our cue to retrain. And <strong>Grafana</strong> keeps an eye on model health and pipeline lag, so we catch a sick camera or a degrading model <em>before</em> it corrupts the numbers downstream.</p><h3>A word on privacy</h3><p>Because this involves cameras and faces, it deserves to be said plainly. Raw video never leaves the store. The face embeddings are ephemeral — Redis expires them after the session, so there’s no growing biometric database. The outputs are aggregate — counts, waits, conversion — not a dossier on any individual. And staff are excluded by design. The system is built to optimize a checkout floor, not to surveil the people on it.</p><h3>What I’d tell you if you’re building this</h3><p>A few hard-won lessons, stripped down:</p><ol><li><strong>Reuse the infrastructure before you buy any.</strong> The cheapest, fastest win was discovering the cameras were already there. The constraint made the product.</li><li><strong>Answer the one question that matters, not every question you could.</strong> Full tracking was tempting and wrong. Fingerprint-and-recheck was cheaper <em>and</em> more accurate because it stayed focused.</li><li><strong>Measure time, not bodies.</strong> A queue’s pain is its wait, and wait depends on who’s paying and how.</li><li><strong>Recalibrate models on your real population.</strong> Pretrained demographics will betray you on a crowd they’ve never seen.</li><li><strong>Act at the source.</strong> We never fixed the queue at the queue. We kept the rush from ever showing at the door.</li></ol><h3>The stack, at a glance</h3><p><strong>Computer Vision:</strong> OpenCV, YOLO, RetinaFace/SCRFD, ByteTrack/DeepSORT, ArcFace (InsightFace), InsightFace genderage <strong>Video &amp; Edge:</strong> RTSP, GStreamer/FFmpeg, on-prem GPU edge servers <strong>Streaming &amp; State:</strong> Apache Kafka, Redis (TTL) <strong>Warehousing:</strong> ClickHouse (real-time), BigQuery (historical) <strong>Orchestration &amp; Transform:</strong> Dagster, dbt <strong>Dashboards:</strong> Grafana (ops), Superset (business) <strong>MLOps &amp; DevOps:</strong> MLflow, Bitbucket, Jenkins, Nomad <strong>Language:</strong> Python</p><p><em>If you’re working on retail analytics, computer vision, or streaming data pipelines — or you just want to argue about whether full tracking was really the wrong call — I’d love to hear from you.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5e9c34316849" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[️‍♂️ Crime Scene Investigator AI— Using Vision-Language Model]]></title>
            <link>https://medium.com/@sufyanbinimran/vlm-fine-tuning-and-downstream-application-development-bf1dcf7c3a0c?source=rss-540e35bc4b8d------2</link>
            <guid isPermaLink="false">https://medium.com/p/bf1dcf7c3a0c</guid>
            <dc:creator><![CDATA[Sufyan Malik]]></dc:creator>
            <pubDate>Thu, 01 May 2025 18:16:25 GMT</pubDate>
            <atom:updated>2025-05-04T17:38:04.566Z</atom:updated>
            <content:encoded><![CDATA[<p><em>An end-to-end system combining fine-tuned BLIP and Gemini Pro LLM to analyze crime scenes</em></p><h3>📌 Introduction</h3><p>Imagine uploading a crime scene photo — and instantly receiving a <strong>detailed investigation report</strong>:<br> — Description of the scene<br> — Possible evidence<br> — Suspects and reasoning</p><p>This is exactly what <strong>CrimeLens AI</strong> delivers.<br> We built an app that <strong>combines a fine-tuned image captioning model (BLIP)</strong> with a <strong>powerful LLM (Gemini Pro)</strong> to simulate a <em>professional crime scene investigator</em>.</p><p>👉 <strong>Try the live app here:</strong> <a href="https://vlm-fine-tuning-and-downstream-application-development-mg9s2ff.streamlit.app/">CrimeLens AI Streamlit App</a></p><h3>Step 1️⃣ Fine-tuning BLIP for Image Captioning</h3><p>We started by fine-tuning <strong>BLIP (Bootstrapping Language-Image Pretraining)</strong> to improve its ability to describe <strong>crime scene images</strong>.</p><p><strong>Dataset</strong>:<br> We curated a dataset of crime-related images (e.g., evidence markers, forensic scenes) and fine-tuned BLIP so it could generate <strong>precise, context-aware captions</strong> for such images.</p><p><strong>Fine-tuning Codebase</strong>:<br> → <a href="https://github.com/sufyanbinimran/VLM-fine-tuning-and-downstream-application-development.git">GitHub Repo (Fine-tuning code)</a></p><p><strong>Model weights</strong>:<br> → <a href="https://huggingface.co/sufyanbinimran/blip-finetuned/commit/85ef4f8a8816b6bd4a773af421853ca5b1bcacaf">Hugging Face BLIP fine-tuned model</a></p><p><strong>Example Output</strong></p><p>Uploaded Crime Scene ImageBLIP Description<em>”A handgun lying on a blood-stained wooden floor next to yellow evidence marker number 3.”</em></p><h3>Step 2️⃣ Passing Scene Description to Gemini Pro LLM</h3><p>Once BLIP generates a scene description, we pass it to <strong>Gemini 1.5 Pro</strong>, a powerful large language model.</p><p>We crafted a <strong>crime investigation prompt</strong> that guides Gemini to think like a forensic expert:</p><pre>You are a professional crime scene investigator. Carefully analyze the scene description to determine if any crime took place, identify possible suspects, evidence, and explain your reasoning with clear details.</pre><pre>Scene Description: A handgun lying on a blood-stained wooden floor next to yellow evidence marker number 3.</pre><pre>Your detailed investigation report:</pre><h3>Step 3️⃣ LLM Generates Investigation Report</h3><p>The LLM produces a structured and well-reasoned report like:</p><pre>Based on the scene description, it appears to be a possible homicide. The presence of a handgun next to a blood-stained floor suggests a violent incident. Evidence marker number 3 indicates official investigation underway.</pre><pre>Possible Evidence:<br>– Handgun (potential murder weapon)<br>– Bloodstains (for DNA analysis)<br>– Fingerprints on gun</pre><pre>Next Steps:<br>– Collect fingerprints from weapon<br>– Match DNA from bloodstains<br>– Identify recent owners of handgun</pre><h3>🚀 App Interface — Streamlit Deployment</h3><p>We built an elegant <strong>Streamlit app</strong> with a <strong>crime investigation theme</strong> to let users upload images and receive reports.</p><blockquote><em>🔗 </em><strong><em>Live App</em></strong><em>: </em><a href="https://vlm-fine-tuning-and-downstream-application-development-mg9s2ff.streamlit.app/"><em>Try CrimeLens AI</em></a></blockquote><p><strong>App Features</strong>:<br> ✅ Upload crime scene image<br> ✅ Optional: Ask a custom investigation question<br> ✅ AI generates both <strong>Image Description</strong> &amp; <strong>Investigation Report</strong></p><p><strong>Interface Preview</strong>:</p><p>Upload ImageAI Generates Report</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Uj7zvi0X8zcbmrcQF1htaA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/988/1*yHOlz4g35x2yixipw3qDIQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/975/1*DEOKwxLJMkBBMfTHqiAfNA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1013/1*bktO5kxrvEEFWFMkXn4BCA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/921/1*nMl7hgzHX2ZEbvSgiH3-EA.png" /></figure><h3>🛠️ Tech Stack</h3><p><strong>BLIP (Hugging Face)</strong>Image Captioning<br><strong>Gemini 1.5 Pro (Google AI)</strong>LLM reasoning<br><strong>PyTorch</strong>BLIP Model loading<br><strong>Streamlit</strong>App interface<br><strong>Hugging Face Hub</strong>Model hosting<br><strong>Google Generative AI SDK</strong>Gemini API integration</p><h3>🗂️ Full Resources</h3><p>– <strong>Hugging Face Model Repo</strong> → <a href="https://huggingface.co/sufyanbinimran/blip-finetuned/commit/85ef4f8a8816b6bd4a773af421853ca5b1bcacaf">BLIP Fine-tuned Model</a><br> — <strong>GitHub Code Repo</strong> → <a href="https://github.com/sufyanbinimran/VLM-fine-tuning-and-downstream-application-development.git">CrimeLens AI GitHub Repo</a><br> — <strong>Live Streamlit App</strong> → <a href="https://vlm-fine-tuning-and-downstream-application-development-mg9s2ff.streamlit.app/">CrimeLens AI App</a></p><h3>✨ Conclusion</h3><p><strong>CrimeLens AI</strong> shows the power of <strong>Vision-Language Models (VLMs)</strong> + <strong>LLMs</strong> in solving complex, real-world problems.<br> Our system demonstrates how <strong>fine-tuned image understanding</strong> and <strong>language reasoning</strong> can augment <strong>digital forensic investigations</strong>.</p><blockquote><em>⚡ </em><strong><em>Next step:</em></strong><em> We plan to integrate </em><strong><em>object detection</em></strong><em> and </em><strong><em>metadata extraction</em></strong><em> for even deeper crime scene insights.</em></blockquote><h3>🙌 About the Author</h3><p><em>Muhammad Sufyan Malik</em> is a Computer Science student and AI researcher passionate about building smart applications with <strong>LLMs</strong>, <strong>Vision Models</strong>, and <strong>Generative AI</strong>.</p><h3>🔗 Follow the Project</h3><p>🌐 <a href="https://huggingface.co/sufyanbinimran/blip-finetuned/commit/85ef4f8a8816b6bd4a773af421853ca5b1bcacaf">Hugging Face Model</a><br> 💻 <a href="https://github.com/sufyanbinimran/VLM-fine-tuning-and-downstream-application-development.git">GitHub Repository</a><br> 🚀 <a href="https://vlm-fine-tuning-and-downstream-application-development-mg9s2ff.streamlit.app/">Streamlit App</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=bf1dcf7c3a0c" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[GazePlay — Giving Voice to the Unspoken]]></title>
            <link>https://medium.com/@sufyanbinimran/gazeplay-giving-voice-to-the-unspoken-1e922ad72651?source=rss-540e35bc4b8d------2</link>
            <guid isPermaLink="false">https://medium.com/p/1e922ad72651</guid>
            <dc:creator><![CDATA[Sufyan Malik]]></dc:creator>
            <pubDate>Tue, 15 Apr 2025 05:50:47 GMT</pubDate>
            <atom:updated>2025-04-15T05:55:56.895Z</atom:updated>
            <content:encoded><![CDATA[<h3>GazePlay — Speak Without Saying a Word 👀💬</h3><p><em>An AI-Powered Assistive Communication App for Children with </em><strong><em>Cerebral Palsy</em></strong></p><h4>🎥 What Is Cerebral Palsy (CP) and What Are We Trying to Do?</h4><p>To understand our mission better, it’s important to first see what life is like for a child with CP. In the video below, you’ll see how a CP child uses a Tobii Dynavox device to communicate using eye movements.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FisrqUgi57Ww%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DisrqUgi57Ww&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FisrqUgi57Ww%2Fhqdefault.jpg&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/0a25bf89e4f173ac3d639dcfc5894497/href">https://medium.com/media/0a25bf89e4f173ac3d639dcfc5894497/href</a></iframe><h4>📱 App Overview</h4><p>Here’s a quick glimpse of how GazePlay works in real-time:</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2F8kyHkgv_4to%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3D8kyHkgv_4to&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2F8kyHkgv_4to%2Fhqdefault.jpg&amp;type=text%2Fhtml&amp;schema=youtube" width="640" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/ffc9587aa25e0dc8284e6facaa4a8bcc/href">https://medium.com/media/ffc9587aa25e0dc8284e6facaa4a8bcc/href</a></iframe><h3>📌 Motivation</h3><p>Growing up in a joint family system, I witnessed something that shaped my perspective on technology forever. My cousin, Hassan, is a child diagnosed with <strong>Cerebral Palsy (CP)</strong>. Hassan, like many others with CP, faces severe challenges: he cannot walk, has difficulty speaking, and his brain development restricts his ability to interact the way most children do.</p><p>His mother, a resilient woman, runs a local NGO dedicated to supporting special children. She always wished for Hassan to interact with the digital world — something most of us take for granted. This sparked an idea: <strong>Why not dedicate our Final Year Project (FYP) to solving this challenge using technology?</strong></p><p>While researching, we came across the <strong>Tobii Dynavox</strong>, a specialized device for eye and head-controlled communication. While incredibly powerful, its high cost makes it inaccessible for most families — especially in underdeveloped or middle-income communities.</p><p>So we asked ourselves:<br> <strong>“Can we build a free, AI-powered alternative using widely available tools like mobile phones?”</strong><br> The answer became <strong>GazePlay</strong> — a free, Android-based app that runs on eye-blinking and head movements, offering a powerful voice to children who otherwise remain unheard.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*TTkxJXUAcWX0UgwGyNH9ww.jpeg" /><figcaption>This is my Cousin Hassan born with CP condition</figcaption></figure><h3><strong>🔧 How the App Works</strong></h3><p>Children with CP often face <strong>difficult and challenging lives</strong> due to their physical and cognitive limitations. Our mission was to ensure quality interaction, fun, and learning within the time they have. We designed GazePlay around three key pillars:</p><h3>🎮 Section 1: Activity Console</h3><p>This module is designed to give CP children autonomy and expression through three rich interfaces:</p><h4>1. Icon-Based Communication (AAC System)</h4><p>Children can select icons (like sign language, daily items) to build sentences. It’s a visual language where they can express wants, needs, and thoughts.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/313/1*HqlRgEJhhvQTEVkPLg6MEA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/306/1*ejA77auT9FV1jYwp9JHIhA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/307/1*xfShW0YEuDVzWJ2erCNVKw.png" /></figure><h4>2. Places Module</h4><p>Children can pre-define frequently visited locations (e.g., home, therapy center) to easily communicate where they want to go — <strong>giving them a sense of independence and the ability to express their choices to guardians, teachers, or caregivers.</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/307/1*21NCacs2ylW1cCmp6bj9AA.png" /></figure><h4>3. Topics Module</h4><p>This includes subcategories like:</p><ul><li><strong>About Me</strong>: Introducing themselves to others</li><li><strong>Needs</strong>: Like needing food, water, or restroom</li><li><strong>Emotions</strong>: Expressing how they feel</li><li><strong>Greetings &amp; Social Phrases</strong>: To interact like any child would</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/312/1*OBW0dYsWrg1wHGhLRAn6Ew.png" /></figure><h3>📚 Section 2: Learning Hub</h3><p>Helping CP children grow in a child-friendly environment is essential. Here’s how our learning section works:</p><h4>1. Alphabet Tracing</h4><p>Children can gaze on animated dots to trace alphabets and form simple words like “A for Apple”.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/311/1*141gGlEpH8JkeYnl-ev-4w.png" /></figure><h4>2. Icon Learning</h4><p>To use icon-based communication effectively, kids need to learn the meaning of icons. This module teaches them through gamified interaction.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/304/1*Ic_VyBNJrtEenj56r9MA2A.png" /></figure><h4>3. AI-Powered Story Generation</h4><p>CP children often go through unpredictable emotional shifts. We trained a <strong>CNN-based emotion detection model</strong> that reads their facial cues in real-time. When a child is upset, the app auto-generates comforting and personalized stories to calm them down.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/312/1*QiNgTrlqJFWtQs1ooJCSJA.png" /></figure><h3>☎️ Section 3: Quick Contacts</h3><p>A simple yet critical feature: children can quickly access and connect with their guardians or parents.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/304/1*z--HoyOjEsxorsYXJNi39w.png" /></figure><h3>👨‍👩‍👧 Guardian &amp; Admin Modules</h3><h3>Guardian Panel</h3><p>Guardians can view their child’s learning and interaction progress through <strong>charts and analytics</strong>, helping them better understand growth areas.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/307/1*sgzSndceqJryHx_LAwIJ9A.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/308/1*sNb5vTokvKx-clk7h8LpMw.png" /></figure><h3>Admin Panel</h3><p>Admins have full <strong>CRUD control</strong> (Create, Read, Update, Delete) over users, including guardians and children, to maintain secure and efficient usage.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/521/1*C9XCvA3FWnMLw3GC5ipucw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9QM9xs4ZPHt-jNzk6plmEw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/439/1*sK3DRPEiJBKR2LwvqtfELw.png" /></figure><h3>🧠 Eye Tracking &amp; Head Movement Integration</h3><p>The entire app is controlled using <strong>head movements and eye blinking</strong>, using the front-facing camera:</p><ul><li><strong>Head Movement</strong>: Works like a cursor</li><li><strong>Eye Blink</strong>: Used to click or select</li><li><strong>Gaze-Based Mode</strong>: Long gaze acts like a tap</li></ul><p>You can switch between modes based on comfort. Simply place the device on a <strong>tripod attached to a wheelchair or table</strong>, and you’re ready to go.<br> 📸 <em>[Insert eye tracking + head control image]</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ct14zrgVlgTuNvvvBDKgYg.png" /><figcaption>Blink Mode</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HrbE3wTncGIExLCYNsBzlg.png" /><figcaption>Gaze Mode</figcaption></figure><h3>🛠️ Tools &amp; Technologies</h3><p>We combined modern AI tools with cross-platform mobile development to bring GazePlay to life.</p><p><strong>Frontend</strong></p><ul><li><strong>React Native</strong> (for Android app)</li><li><strong>HTML, CSS, JS</strong> (Admin Panel)</li></ul><p><strong>Backend &amp; AI</strong></p><ul><li><strong>Python &amp; OpenCV</strong> for real-time camera input</li><li><strong>Flask + Flask-SocketIO</strong> for communication</li><li><strong>TensorFlow CNN model</strong> for mood detection</li><li><strong>MongoDB</strong> for storing user data</li><li><strong>GitHub</strong> for version control</li></ul><blockquote><em>📦 Key Libraries Used:</em></blockquote><p>Flask==2.3.2 <br>Flask-SocketIO==5.3.4 <br>mediapipe==0.10.0 <br>numpy==1.24.3 <br>opencv-python==4.8.0.76 <br>pyautogui==0.9.53 <br>tensorflow==2.12.0 <br>eventlet==0.33.3 <br>python-engineio==4.5.1 <br>python-socketio==5.8.0</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*V-SJen3X55j0hgFVq1VV9w.png" /></figure><h3>🖼️ App Poster</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*RUTBWpe4CTnlaj2B64FSaQ.jpeg" /></figure><h3>📌 Note:</h3><p>GazePlay isn’t limited to children with Cerebral Palsy. Any individual with similar motor impairments — especially those who rely primarily on eye movements for communication — can benefit from this app. Our goal is to make interaction accessible for <em>everyone</em> who needs a voice through their gaze.</p><h3>💡 Final Thoughts</h3><p><strong>GazePlay isn’t just an app. It’s a voice. It’s independence. It’s inclusion.</strong><br> We hope to keep iterating and building on this platform, making it smarter, more accessible, and more impactful — all for those like Hassan who deserve to be seen, heard, and understood.</p><p>If you’re a developer, researcher, or NGO — let’s collaborate to expand this mission.<br> Feel free to reach out or contribute on GitHub:<br> 👉 <a href="https://github.com/sufyanbinimran/GazePlay.git">https://github.com/sufyanbinimran/GazePlay.git</a></p><h3>👥 Team Behind GazePlay</h3><p>Muhammad Sufyan Malik · Mohid Usman · Sidra Malik</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/797/1*iJvNANt5qRu7WlniUOruHw.png" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1e922ad72651" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[ RAG for Diagnostic Reasoning for Clinical Notes (DiReCT)]]></title>
            <link>https://medium.com/@sufyanbinimran/rag-for-diagnostic-reasoning-for-clinical-notes-direct-a9cf31cb263a?source=rss-540e35bc4b8d------2</link>
            <guid isPermaLink="false">https://medium.com/p/a9cf31cb263a</guid>
            <dc:creator><![CDATA[Sufyan Malik]]></dc:creator>
            <pubDate>Tue, 25 Mar 2025 22:30:07 GMT</pubDate>
            <atom:updated>2025-03-30T02:56:53.863Z</atom:updated>
            <content:encoded><![CDATA[<p><em>Leveraging Retrieval-Augmented Generation for Clinical Decision Support</em></p><h3>📌 Project Overview &amp; Motivation</h3><p>In modern healthcare, clinicians often face the challenge of extracting meaningful insights from massive clinical datasets. With patient records, lab results, and physician notes scattered across systems, efficient diagnostic reasoning becomes difficult.</p><p>Our project, <strong>DiReCT (Diagnostic Reasoning for Clinical Notes)</strong>, addresses this challenge by leveraging the <strong>Retrieval-Augmented Generation (RAG)</strong> framework. By integrating powerful retrieval models with generative LLMs, our system efficiently answers clinical queries and generates informative summaries grounded in real patient records from the <strong>MIMIC-IV-Ext dataset</strong>.</p><h3>🔎 Methodology</h3><h3>1️⃣ Dataset Exploration &amp; Preprocessing</h3><p>We started by exploring the <strong>MIMIC-IV-Ext Direct dataset</strong>, identifying useful subsets like physician notes and discharge summaries. Preprocessing steps included:</p><ul><li>Handling missing values</li><li>Text cleaning</li><li>Tokenization</li><li>Creating document embeddings for retrieval</li></ul><h3>2️⃣ Retrieval System Design</h3><p>We experimented with two retrieval approaches: ✅ <strong>BM25 (Sparse Retrieval)</strong> for lexical matches<br> ✅ <strong>Dense Retrieval (Sentence Transformers)</strong> for semantic understanding</p><p>An efficient indexing strategy was designed to balance speed and accuracy, ensuring precise document retrieval for clinical queries.</p><h3>3️⃣ Generative Model Integration</h3><p>We selected a robust open-source LLM(Using Falcon-7B-Instruct) capable of handling medical contexts. Key tasks:</p><ul><li><strong>Prompt Engineering:</strong> Designing context-rich prompts combining user queries with retrieved notes</li><li><strong>Pipeline Design:</strong> Seamlessly feeding retrieved data to the generative model to produce coherent responses</li></ul><h3>4️⃣ Frontend Development with Streamlit</h3><p>To ensure usability for clinicians and researchers, we developed an interactive <strong>Streamlit app</strong> that:</p><ul><li>Accepts user queries</li><li>Displays retrieved documents</li><li>Shows the AI-generated diagnostic summary</li></ul><p>✅ <strong>Check the Live App:</strong><br> 👉 <a href="https://rag-for-diagnostic-reasoning-for-clinical-notes-bspwfhgsmt8mkj.streamlit.app/">https://rag-for-diagnostic-reasoning-for-clinical-notes-bspwfhgsmt8mkj.streamlit.app/</a></p><h3>5️⃣ Evaluation &amp; Analysis</h3><p>We evaluated:</p><ul><li><strong>Retrieval performance:</strong> Precision &amp; Recall</li><li><strong>Generation quality:</strong> Accuracy, coherence, and relevance</li><li><strong>Error Analysis:</strong> Observed that improving context quality directly enhanced answer correctness</li></ul><h3>🌐 Ethical and Privacy Considerations</h3><p>Handling clinical datasets requires:</p><ul><li>Strict adherence to privacy guidelines</li><li>Ensuring no sensitive patient data is exposed</li><li>Maintaining model explainability to avoid clinical misguidance</li></ul><p>📈 Images &amp; Visuals</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3kjuEK6-Xl4-2cwYtIA1hw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*SrU52FCpmAjl5oK7ntZ32Q.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*FScH9yRvx7rCAo8KeNu53A.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*bBY80AmN4t8fkrH1aypA7g.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*AG5mYyAZ4O3WDbJ_fk8VJA.png" /></figure><h3>📚 Key Findings &amp; Lessons Learned</h3><ul><li><strong>Context quality is critical:</strong> Strong retrieval boosts generation quality</li><li><strong>Dense retrieval enhances semantic understanding</strong></li><li>Streamlit adds significant usability for non-technical users</li><li>Real-world deployment demands focus on <strong>explainability</strong> and <strong>trustworthiness</strong> in clinical AI</li></ul><h3>📂 GitHub Repository</h3><p>Explore the full code, dataset pipeline, and model implementation:<br> 👉 <a href="https://github.com/sufyanbinimran/RAG-for-Diagnostic-Reasoning-for-Clinical-Notes.git">GitHub — RAG for DiReCT</a></p><h3>🚀 Live Streamlit Demo</h3><p>Interact with our RAG system live:<br> 👉 <a href="https://rag-for-diagnostic-reasoning-for-clinical-notes-bspwfhgsmt8mkj.streamlit.app/">https://rag-for-diagnostic-reasoning-for-clinical-notes-bspwfhgsmt8mkj.streamlit.app/</a></p><h3>✨ Conclusion</h3><p>Our project demonstrates the potential of <strong>Retrieval-Augmented Generation</strong> in clinical decision support. By grounding generative models in real patient data, we can enhance diagnostic accuracy, save time, and improve patient care.</p><h3>✅ Connect With Us</h3><p>For collaborations or feedback, feel free to connect!<br> 🔗 GitHub: <a href="https://github.com/sufyanbinimran">Sufyan Malik</a><br> 🔗 LinkedIn: <a href="https://www.linkedin.com/in/sufyan-malik-887672204/">Sufyan Malik — LinkedIn</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a9cf31cb263a" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Fine-Tuning GPT for Math Riddles & Meme Repair: A Fun AI Experiment]]></title>
            <link>https://medium.com/@sufyanbinimran/fine-tuning-gpt-for-math-riddles-meme-repair-a-fun-ai-experiment-af412794013a?source=rss-540e35bc4b8d------2</link>
            <guid isPermaLink="false">https://medium.com/p/af412794013a</guid>
            <dc:creator><![CDATA[Sufyan Malik]]></dc:creator>
            <pubDate>Sat, 15 Mar 2025 11:21:45 GMT</pubDate>
            <atom:updated>2025-03-15T11:21:45.998Z</atom:updated>
            <content:encoded><![CDATA[<h3>Introduction</h3><p>Large language models like <strong>GPT-2</strong> and <strong>TinyLlama</strong> can do more than just chat — they can be fine-tuned for specific tasks like <strong>math riddle generation</strong> and <strong>fixing viral math memes</strong>! In this project, I fine-tuned models to:</p><p>1️⃣ <strong>Generate clever math riddles</strong> 🤯<br> 2️⃣ <strong>Correct incorrect viral math memes</strong> 😂</p><p>Let’s dive into the process!</p><h3>📌 Task 1: Math Riddle Factory</h3><h3>🛠️ The Goal:</h3><p>Train a model to generate math riddles like:</p><p>💡 <em>“What number becomes zero when you subtract 15 from half of it?”</em></p><h3>📊 Steps Taken:</h3><p>1️⃣ <strong>Dataset Creation:</strong> Manually curated <strong>30 riddles</strong> with solutions.<br> 2️⃣ <strong>Fine-Tuning:</strong> Trained GPT-2/TinyLlama for <strong>2–3 epochs</strong> on this dataset.<br> 3️⃣ <strong>Testing:</strong> Asked the model to generate <strong>5 riddles</strong> and checked their validity.</p><h3>🧠 Best Riddles Generated:</h3><p>✅ <strong>Riddle 1:</strong> <em>I am a two-digit number. My tens digit is twice my ones digit, and their sum is 12. What am I?</em><br> <strong>Solution:</strong> 48</p><p>✅ <strong>Riddle 2:</strong> <em>I am an odd number. Take away one letter, and I become even. What number am I?</em><br> <strong>Solution:</strong> Seven (remove “s” → “even”)</p><p>✅ <strong>Riddle 3:</strong> <em>You add me to myself and get 36. What number am I?</em><br> <strong>Solution:</strong> 18</p><h3>📌 Task 2: Math Meme Repair</h3><h3>🤣 The Problem:</h3><p>The internet is full of <strong>incorrect viral math memes</strong>. My goal? Train a model to <strong>fix them with sass &amp; accuracy</strong>.</p><h3>📊 Steps Taken:</h3><p>1️⃣ <strong>Dataset Creation:</strong> Collected <strong>20 incorrect math memes</strong> (e.g., PEMDAS mistakes).<br> 2️⃣ <strong>Fine-Tuning:</strong> Trained a model to map <strong>wrong math statements → correct explanations</strong>.<br> 3️⃣ <strong>Testing:</strong> Fed in <strong>3 new incorrect memes</strong> and checked its explanations.</p><h3>⚡ Before/After Example:</h3><p>❌ <strong>Wrong Meme:</strong> <em>8 ÷ 2(2+2) = 1?</em><br> ✅ <strong>Corrected:</strong> <em>8 ÷ 2(2+2) = 16 (Follow PEMDAS: 8 ÷ 2 × 4 = 16)</em></p><h3>😂 Error Rating System:</h3><p>My model rates bad math with:<br> 🔥 <strong>“90% sass, 10% patience”</strong><br> 🚨 <strong>“Mathematical crime detected!”</strong></p><h3>🚀 Final Thoughts &amp; Learnings</h3><p>✅ <strong>Fine-tuning even small models can make them specialized &amp; fun!</strong><br> ✅ <strong>GPT can learn logical patterns well — but still needs human checks.</strong><br> ✅ <strong>Memes may lie, but math never does.</strong></p><h3>📂 GitHub &amp; Live Results</h3><p>📌 <strong>GitHub Code:</strong> <a href="https://github.com/sufyanbinimran/Fine-tuning-Deepseek-math-for-generation.git">github.com/sufyanbinimran/Math-Riddle-Meme-Repair</a><br> 📌 <strong>Sample Model Outputs (Images Below!)</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/542/1*w8hSfNXrZfUlsrg5jPRC-A.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*UWYKltDNUXot3TRPMp21Bw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*osDTbdt16LukxlsUdD4dAw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*85LdMS_kps1h8JO1qc9h1A.png" /></figure><p>Let me know your thoughts! What’s the funniest bad math meme you’ve seen? 😆</p><p>#AI #MachineLearning #DeepLearning #Transformers #Math #FineTuning #GPT</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=af412794013a" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Fine-Tuning a Model for Math Riddles, Meme Repair, and Emoji Math Solver]]></title>
            <link>https://medium.com/@sufyanbinimran/fine-tuning-a-model-for-math-riddles-meme-repair-and-emoji-math-solver-09b4bff16619?source=rss-540e35bc4b8d------2</link>
            <guid isPermaLink="false">https://medium.com/p/09b4bff16619</guid>
            <dc:creator><![CDATA[Sufyan Malik]]></dc:creator>
            <pubDate>Thu, 13 Mar 2025 16:18:33 GMT</pubDate>
            <atom:updated>2025-03-13T16:18:33.926Z</atom:updated>
            <content:encoded><![CDATA[<h3>Introduction</h3><p>Fine-tuning a language model allows us to specialize it for specific tasks. In this project, we tackled three creative applications of fine-tuning: generating math riddles, fixing incorrect math memes, and solving emoji-based math problems. We used GPT-2/TinyLlama for training, deployed our models using Streamlit, and documented the process in this blog.</p><h3>1. Math Riddle Factory</h3><h3>Goal</h3><p>Train a model to generate math riddles and their correct solutions.</p><h3>Steps</h3><ol><li><strong>Dataset Creation:</strong> We manually curated 30 math riddles, ensuring a balance of difficulty levels.</li><li><strong>Fine-Tuning:</strong> Using GPT-2, we trained the model for 3 epochs on our dataset.</li><li><strong>Testing:</strong> We generated 5 riddles and checked their correctness.</li></ol><h3>Sample Riddles</h3><ul><li><strong>Q:</strong> What number becomes zero when you subtract 15 from half of it? <strong>A:</strong> 30.</li><li><strong>Q:</strong> A farmer has 10 cows. If all but 3 run away, how many are left? <strong>A:</strong> 3.</li><li><strong>Q:</strong> I am a number. Double me and add 5 to get 15. What am I? <strong>A:</strong> 5.</li></ul><h3>2. Math Meme Repair</h3><h3>Goal</h3><p>Correct viral math memes with incorrect calculations.</p><h3>Steps</h3><ol><li><strong>Dataset Creation:</strong> Collected 20 incorrect math memes, mainly focusing on errors in PEMDAS and fractions.</li><li><strong>Fine-Tuning:</strong> Trained a model to correct these mistakes and provide explanations.</li><li><strong>Testing:</strong> Tested with 3 new incorrect memes to evaluate explanation clarity.</li></ol><h3>Example</h3><ul><li><strong>Wrong:</strong> 8 ÷ 2(2+2) = <strong>1</strong></li><li><strong>Correct:</strong> 8 ÷ 2(2+2) = <strong>16</strong> (PEMDAS applied correctly: 8 ÷ 2 × 4 = 16)</li><li><strong>Model’s Error Rating:</strong> 90% sass, 10% patience 😆</li></ul><h3>3. Emoji Math Solver</h3><h3>Goal</h3><p>Fine-tune a model to solve math problems written entirely in emoji.</p><h3>Steps</h3><ol><li><strong>Dataset Creation:</strong> Generated 30 problems like 🍎 + 🍎 + 🍎 = 12.</li><li><strong>Fine-Tuning:</strong> Trained GPT-2 on these emoji-based equations.</li><li><strong>Testing:</strong> Solved 3 new emoji problems.</li></ol><h3>Example</h3><ul><li><strong>Input:</strong> 🚗 + 🚗 + 🚗 + 🚗 = 20</li><li><strong>Solution:</strong> 🚗 = 5</li></ul><h3>Deployment</h3><ul><li><strong>Streamlit App:</strong> We deployed the model on Streamlit, making it accessible for interactive testing.</li><li><strong>GitHub Repository:</strong> <a href="https://github.com/sufyanbinimran/Generative-AI.git">Link to our code</a></li><li><strong>Medium Blog:</strong> You are reading it!</li><li><strong>LinkedIn Post:</strong> (Add link)</li></ul><h3>Conclusion</h3><p>Fine-tuning models on custom datasets unlocks creative applications. From math riddles to emoji solvers, we explored how AI can bring fun and learning together.</p><p>Would love to hear your thoughts! Drop a comment below. 🚀</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=09b4bff16619" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Convert Pseudocode to C++ (and Vice Versa) Using Transformers]]></title>
            <link>https://medium.com/@sufyanbinimran/building-a-streamlit-app-for-pseudocode-to-c-conversion-with-transformers-9c2f4a4633ff?source=rss-540e35bc4b8d------2</link>
            <guid isPermaLink="false">https://medium.com/p/9c2f4a4633ff</guid>
            <dc:creator><![CDATA[Sufyan Malik]]></dc:creator>
            <pubDate>Thu, 27 Feb 2025 16:14:30 GMT</pubDate>
            <atom:updated>2025-03-15T10:44:25.793Z</atom:updated>
            <content:encoded><![CDATA[<h3>Introduction</h3><p>Writing code efficiently across different programming paradigms can be challenging. Many developers and students struggle when converting between pseudocode and C++. To solve this problem, we built a <strong>Transformer-based model</strong> that automatically converts:</p><ul><li><strong>Pseudocode to C++</strong></li><li><strong>C++ to Pseudocode</strong></li></ul><p>Using deep learning techniques, our model understands the structure of both languages and translates code seamlessly.</p><h3>Motivation</h3><p>Students and developers often need to translate algorithms from pseudocode to C++ or vice versa. Manual conversion can be time-consuming and error-prone. Our model aims to make this process <strong>faster and more accurate</strong>, helping:</p><ul><li><strong>Students</strong> learning programming</li><li><strong>Educators</strong> who create programming assignments</li><li><strong>Developers</strong> working across multiple languages</li></ul><h3>How It Works</h3><p>We used <strong>Transformer-based Sequence-to-Sequence (Seq2Seq) models</strong> to perform this translation. The process includes:</p><ol><li><strong>Data Preprocessing</strong>: Tokenizing and vectorizing C++ and pseudocode samples.</li><li><strong>Model Training</strong>: Training two separate models:</li></ol><ul><li>Model 1: <strong>C++ to Pseudocode</strong></li><li>Model 2: <strong>Pseudocode to C++</strong></li></ul><p><strong>3.Inference &amp; Conversion</strong>: Converting new inputs based on learned patterns.</p><h3>Architecture</h3><p>Our model architecture consists of:</p><ul><li><strong>Multi-Head Attention Mechanism</strong></li><li><strong>Positional Encoding</strong></li><li><strong>Feedforward Neural Networks</strong></li><li><strong>Layer Normalization</strong></li></ul><p>Below is a high-level view of our Transformer architecture:</p><h3>Features</h3><p>✅ <strong>Bidirectional Conversion</strong> — Convert in both directions (C++ ↔ Pseudocode)<br>✅ <strong>AI-Powered Accuracy</strong> — Trained on a large dataset for precise translations<br>✅ <strong>Fast &amp; Efficient</strong> — Generates results in seconds<br>✅ <strong>Web-Based UI</strong> — Built using <strong>Streamlit</strong> for ease of use</p><h3>Live Demo</h3><p>You can try out our model <strong>live on Streamlit</strong>:<br><a href="https://your-streamlit-app-link.com/">🔗</a><a href="https://sufyanspseudo-code-to-c-code.streamlit.app/">https://sufyanspseudo-code-to-c-code.streamlit.app</a></p><h3>Code &amp; Repository</h3><p>You can check out our <strong>GitHub repository</strong> for complete code, model weights, and dataset:</p><p>🔗 <a href="https://github.com/sufyanbinimran/Pseudo-code-to-C-code">GitHub Repository</a></p><h3>Example Usage</h3><h3>Pseudocode to C++ Conversion</h3><p><strong>Input:</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/699/1*O0-erQ5eTZVT9MgJxYyvxw.png" /></figure><p><strong>Output (C++ Code):</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/717/1*2NePvwglFgns3xH_eNwNOA.png" /></figure><h3>C++ to Pseudocode Conversion</h3><p><strong>Input:</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/723/1*jbS9mpKvuwfPI7vmtwmnyg.png" /></figure><p><strong>Output (Pseudocode):</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/714/1*Qiaer79fFfa0tIvKyRhoiA.png" /></figure><h3>Challenges Faced</h3><p>🔹 <strong>Handling Edge Cases</strong> — Some syntax variations required additional preprocessing.<br>🔹 <strong>Training on Limited Data</strong> — We manually curated a dataset to improve accuracy.<br>🔹 <strong>Maintaining Code Structure</strong> — Keeping indentation and formatting consistent was tricky.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9c2f4a4633ff" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[ Introducing Shayar AI: AI-Generated Urdu Poetry with LSTM & Text-to-Speech ]]></title>
            <link>https://medium.com/@sufyanbinimran/introducing-shayar-ai-ai-generated-urdu-poetry-with-lstm-text-to-speech-efe28e9afd85?source=rss-540e35bc4b8d------2</link>
            <guid isPermaLink="false">https://medium.com/p/efe28e9afd85</guid>
            <dc:creator><![CDATA[Sufyan Malik]]></dc:creator>
            <pubDate>Sat, 15 Feb 2025 09:28:33 GMT</pubDate>
            <atom:updated>2025-02-16T13:51:20.670Z</atom:updated>
            <content:encoded><![CDATA[<h3>🎭 The Fusion of AI &amp; Poetry</h3><p>Poetry is an art form that transcends time, emotion, and culture. But what if artificial intelligence could craft poetry just like legendary poets? Enter <strong>Shayar AI</strong>, a deep learning-based model that generates Urdu <em>ghazals</em> from a single input word.</p><p>Shayar AI leverages <strong>Long Short-Term Memory (LSTM)</strong> networks trained on a vast dataset scraped from <strong>Rekhta.org</strong>, one of the largest repositories of Urdu poetry. This allows the AI to learn poetic structures, wordplay, and nuances, ensuring that each generated <em>ghazal</em> feels natural and emotionally resonant.</p><h3>🎯 How Shayar AI Works</h3><p>Shayar AI is powered by a combination of <strong>deep learning and NLP</strong> to generate high-quality poetry. Here’s how it works:</p><ol><li><strong>Dataset Preparation</strong> 📝</li></ol><ul><li>A custom web scraper extracts <em>shers</em> and <em>ghazals</em> from Rekhta.org.</li><li>The dataset is cleaned and preprocessed, removing unwanted characters and normalizing text.</li></ul><p><strong>2.LSTM Model Training</strong> 🤖</p><ul><li>An LSTM-based recurrent neural network (RNN) is trained on the dataset.</li><li>The model learns sentence structures, rhyme schemes, and poetic flow.</li><li>The AI generates poetry when given a single Urdu word as input.</li></ul><p><strong>3.Text-to-Speech (TTS) Integration</strong> 🎙️</p><ul><li>Play.ht is integrated to convert AI-generated poetry into <strong>natural-sounding speech</strong>.</li><li>Users can both <strong>read and listen</strong> to the poetry for an immersive experience.</li></ul><h3>🌟 Features of Shayar AI</h3><p>✅ <strong>Personalized Poetry Generation:</strong> Users enter a single word, and the AI crafts a unique <em>ghazal</em>.<br>✅ <strong>Trained on Authentic Data:</strong> Uses a vast corpus from Rekhta.org to maintain authenticity.<br>✅ <strong>Natural-Sounding Audio:</strong> Play.ht integration brings poetry to life through voice synthesis.<br>✅ <strong>AI-Driven Creativity:</strong> The model mimics human-like poetic structure and emotion.<br>✅ <strong>Scalability:</strong> The system can be expanded to support multiple languages and styles.</p><h3>🎨 Images &amp; Visuals</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*OaHoTvtOY8nvfPLmirIZRQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/975/1*-JO45PNgi0Sa87owXUIvDQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/975/1*VzCv3zYxNZ0tarb30ygKGw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/885/1*Y57MGC9Rw_XRjoxQqt-PPA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/929/1*OstmJ22wF4-OjMKAR0_vBQ.png" /></figure><h3>🔥 Future Enhancements</h3><p>🔹 <strong>Improved Rhyme and Theme Consistency:</strong> Enhancing the model to generate more contextually rich poetry.<br>🔹 <strong>Interactive Poetry Experience:</strong> Enabling users to provide feedback for better AI learning.<br>🔹 <strong>Multilingual Support:</strong> Expanding beyond Urdu to generate poetry in other languages like Persian and Hindi.<br>🔹 <strong>Voice Modulation:</strong> Enhancing TTS for better emotional expression in poetry recitation.</p><h3>📢 Try It Out!</h3><p>Shayar AI is a step toward <strong>combining AI with artistic creativity</strong>. Whether you’re a poetry lover or a tech enthusiast, this project showcases how deep learning can contribute to literature and art.</p><p>🔗 <strong>Check out the project here:</strong> <a href="https://lnkd.in/gSVU_K34">Shayar AI on GitHub</a><br>You can try this app: <a href="https://shayar-ai-kxackc3nih6deiy2qgx5qq.streamlit.app/">https://shayar-ai-kxackc3nih6deiy2qgx5qq.streamlit.app/</a> 🚀</p><p>Let’s redefine poetry with AI! ✨📖</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=efe28e9afd85" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>