<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Herman Wandabwa on Medium]]></title>
        <description><![CDATA[Stories by Herman Wandabwa on Medium]]></description>
        <link>https://medium.com/@hermanwandabwa?source=rss-58e995b6d0e3------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/2*NtahlS4LHDjwlEGqWqFLXw.jpeg</url>
            <title>Stories by Herman Wandabwa on Medium</title>
            <link>https://medium.com/@hermanwandabwa?source=rss-58e995b6d0e3------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 00:35:37 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@hermanwandabwa/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[How to Diagnose RAG Failures with a Groundedness Judge]]></title>
            <link>https://medium.com/data-science-collective/how-to-diagnose-rag-failures-with-a-groundedness-judge-be3924f924ba?source=rss-58e995b6d0e3------2</link>
            <guid isPermaLink="false">https://medium.com/p/be3924f924ba</guid>
            <category><![CDATA[llm-as-a-judge]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[retrieval-augmented-gen]]></category>
            <category><![CDATA[llm-evaluation]]></category>
            <category><![CDATA[large-language-models]]></category>
            <dc:creator><![CDATA[Herman Wandabwa]]></dc:creator>
            <pubDate>Tue, 08 Sep 2026 11:48:51 GMT</pubDate>
            <atom:updated>2026-09-08T11:48:51.545Z</atom:updated>
            <content:encoded><![CDATA[<h4>Learn how to evaluate RAG systems, detect hallucinations, and trace failures across retrieval, context, and generation using a groundedness judge.</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*uyD2leAoAl7ZmzRxwqlu8w.png" /><figcaption>Image generated by the author</figcaption></figure><p>It’s been a while since I wrote my last article on building a RAG chatbot. It must have been this one <a href="https://medium.com/data-science-collective/from-data-to-dialogue-development-of-a-retrieval-augmented-generation-rag-chatbot-for-fitness-b9fbaf818ace">here</a> that even had a scraper for getting webpages and building an indexer as well as an embedder for the RAG application. I even went further with <a href="https://medium.com/data-science-collective/building-a-rag-system-with-mmr-for-safaricoms-smart-assistant-1e9ba91b9bfe">this one</a> and built a re-ranker to get cleaner and more relevant documents at retrieval time. Please go through the two, especially if you do not have enough experience in this area or are just refreshing your skills.</p><p>Building a RAG application is never enough, especially if it&#39;s deployed in production.<em> </em>What if it sometimes starts to give wrong answers? How do you even figure this out? That question sounds simple until it isn&#39;t. A RAG application has at least seven places where things can quietly break, and I’ll help you understand them in detail in the next section. It has several moving parts, including chunking, embeddings, retrieval, reranking, context assembly, generation, and the prompt that ties them together. When the final answer is wrong, that alone tells you very little about which part actually failed. I’ve watched engineers spend a week fine-tuning a generation prompt to fix what turned out to be a chunking bug and nothing prompt-related.</p><p>There’s a fairly mechanical way to localise where a RAG system failed. Once you can do that, the next challenge is measuring groundedness at scale, because reading every transcript by hand stops being practical once you move beyond the PoC stage.</p><p>That is where an LLM-as-judge comes in. But it introduces its own failure modes, and they can be just as easy to miss as the RAG issues you are trying to diagnose.</p><p>This article covers both sides of the problem: how to trace RAG failures back to their source and how to build a groundedness judge you can actually trust.</p><h4><strong>The Anatomy of a RAG Failure</strong></h4><p>Every RAG pipeline is a chain, and the first job is to work out which link failed before you start changing prompts. The map below is the framework I use to localise the problem. It is not exhaustive, and you can extend it to fit your own system and use case.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gub28aoTofw1A0EcewBxrw.png" /></figure><p>The ordering in the above table is deliberate. Basically, <strong>fix retrieval before you touch generation. </strong>No amount of prompt engineering repairs a context window that never had the right fact in it. I’ve seen teams do this backwards, where they rewrite the system prompt five times when the real problem was that the reranker was demoting the one passage that mattered. Cheap, high-leverage fixes (better chunking, hybrid search) come before expensive ones (fine-tuning a reranker, retraining anything).</p><p>Here’s how I actually use the table. Three real-shaped symptoms:</p><p>1. <em>The bot gives confidently wrong answers when the documents don’t cover the topic at all—that&#39;s</em> not a retrieval problem, as there’s nothing to retrieve. It’s a <strong>generation/abstention failure, </strong>as the model isn’t recognising <em>“I don’t have this”</em> as a valid answer.</p><p>2. <em>The bot gives correct-sounding but outdated information — </em>That’s almost never chunking. It’s a <strong>stale index, </strong>as the docs changed and nobody re-embedded.</p><p>3. <em>Simple questions work fine, but anything requiring two facts from two documents falls apart—</em>that&#39;s a <strong>retrieval failure specific to multi-hop queries </strong>where a single dense-retrieval pass isn’t built to fetch two things that don’t individually resemble the query.</p><p>The right move around this should be to pull the actual retrieved context for a failing example and look at it before touching anything. If the fact isn’t in there, you have a retrieval problem. If it is, and the model still gets it wrong, you have a generation problem. That’s a simple check that can save you days of work.</p><h4><strong>Why You Can’t Just Eyeball This at Scale</strong></h4><p>Reading transcripts works for a few examples. But this becomes quite difficult when the numbers run into thousands. This is where an <strong>LLM-as-judge </strong>comes in, and specifically, a judge for <em>groundedness </em>that is sometimes<em> </em>called<em> faithfulness. </em>It basically tries to answer whether<em> </em>every claim in the answer traces back to something in the retrieved context.</p><p>Here’s the subtle problem people get wrong immediately: <strong>groundedness is not the same thing as correctness. </strong>An answer can be true and still ungrounded (the model happened to know the right fact from its own training data, not from your documents), and an answer can be grounded and still wrong (the retrieved passage itself was outdated or incorrect). If you’re evaluating a RAG system, groundedness is the metric you actually want, because it tells you whether <em>your retrieval pipeline </em>did its job, independent of whether the underlying documents were right.</p><h4><strong>Building the Judge</strong></h4><p>A single “rate this 1–5 for groundedness” prompt sounds reasonable and is, in my experience, close to useless. Normally such ratings will give you a number with no way to know <em>why</em>, and it’s exactly the kind of holistic judgement LLMs are inconsistent at. What works much better is decomposing the answer into atomic claims and checking each one individually.</p><p>Here’s the judge prompt I’d start from:</p><pre><br>SYSTEM:<br>You are a groundedness evaluator. You will be given a QUESTION, a CONTEXT<br>(retrieved passages), and an ANSWER generated by an AI assistant. Your job<br>is NOT to judge whether the answer is true in general, or well-written, or<br>relevant — only whether it is fully supported by the CONTEXT.<br><br>Do the following:<br><br>1. Decompose the ANSWER into a list of atomic claims (each claim = one<br>   discrete factual statement, not a compound sentence).<br>2. For each claim, decide:<br>   - SUPPORTED: directly stated or reasonably entailed by CONTEXT<br>   - CONTRADICTED: conflicts with CONTEXT<br>   - UNSUPPORTED: not addressed by CONTEXT at all (NOT the same as &quot;false&quot;)<br>3. Quote the exact span of CONTEXT used as evidence for each claim<br>   (or &quot;none&quot; if UNSUPPORTED/CONTRADICTED).<br>4. Give a one-sentence rationale per claim.<br>5. groundedness_score = (# SUPPORTED) / (# total claims).<br><br>Be strict: a claim that adds specificity not in CONTEXT (context says &quot;a<br>discount applies,&quot; answer says &quot;a 20% discount applies&quot;) is UNSUPPORTED.<br><br>Return valid JSON only:<br>{<br>  &quot;claims&quot;: [{&quot;claim&quot;: &quot;...&quot;, &quot;verdict&quot;: &quot;...&quot;, &quot;evidence_span&quot;: &quot;...&quot;, &quot;rationale&quot;: &quot;...&quot;}],<br>  &quot;groundedness_score&quot;: 0.0,<br>  &quot;num_claims&quot;: 0<br>}<br><br>QUESTION: {question}<br>CONTEXT: {context}<br>ANSWER: {answer}</pre><p>A few design choices worth calling out explicitly, because they weren’t obvious to me the first time around either:</p><ul><li><strong>Claim-level, not holistic</strong> — A single overall score is hard to debug. Per-claim verdicts show you exactly which statement failed, which is far more useful when you need to fix the underlying issue.</li><li><strong>The evidence span is mandatory</strong> — requiring the judge to cite the supporting text forces it to ground its verdict in evidence rather than plausibility. It also lets a human reviewer verify the judgement quickly without rereading the entire context.</li><li><strong>UNSUPPORTED and CONTRADICTED are separate categories for a reason</strong> — a claim that conflicts with the evidence is a different failure from one that may be true but is not supported by the retrieved context. These problems require different fixes, so collapsing them into a single “bad” category throws away useful diagnostic information.</li><li><strong>The “be strict” instruction is deliberate</strong> — LLM judges can be overly lenient with borderline cases, especially when an answer sounds plausible. Explicitly asking for strict evaluation helps reduce that tendency.</li></ul><h4><strong>Calibrating the Judge</strong></h4><p>This is one part that is always forgotten, yet it&#39;s the one that actually determines whether any of the above is worth anything really. <strong>You have to validate the judge against humans before you trust it.</strong></p><p>It&#39;s actually a simple process in practice. Take 100–200 examples, have a human rate groundedness independently, and compute agreement (<a href="https://pmc.ncbi.nlm.nih.gov/articles/PMC3900052/">Cohen’s kappa</a>), not raw accuracy, because raw accuracy is misleadingly high whenever most answers are grounded (a judge that always says “grounded” looks great on accuracy and is worthless). While you’re at it, check for the usual LLM-judge biases:</p><ul><li><strong>Position bias</strong>—does the verdict change if you reorder the context passages?</li><li><strong>Verbosity bias</strong>—does the judge favour longer answers, independent of whether they’re actually more grounded?</li><li><strong>Self-preference bias</strong>—if your judge model is the same family as your generator model, it may rate that family’s outputs more charitably.</li><li><strong>Leniency drift</strong>—does agreement quietly degrade over weeks as the judge model or prompt version shifts?</li></ul><p>If kappa comes back weak in a specific category, that’s a flag to fix your rubric for that category and not a flag that your RAG system is broken there. This distinction trips people up constantly, and it’s exactly what the next section is about.</p><h4><strong>Reading/Interpreting Results</strong></h4><p>Say your dashboard shows this, broken out by query type. It&#39;s always advisable to look at the number like an<strong> operator and NOT a scoreboard.</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-oXl3u8zudVX5AuTJiJlTw.png" /></figure><p>The overall average here looks like a mild dip, which might not be very accurate. Here’s how I’d actually read this, row by row.</p><ol><li><strong>Simple factual lookup </strong>is stable and high on both groundedness and agreement. This is the easy case for both your system and your judge. It’s not interesting on its own, but if it ever drops, that tells you something fundamental broke (indexing, base retrieval, or the judge prompt itself), not something specific to reasoning or ambiguity.</li><li><strong>Multi-hop / comparison </strong>dropped hard, and κ=0.71 is only moderate, so some of that drop might be judge noise on longer, harder-to-decompose answers. Before assuming the RAG system has regressed, pull five failing transcripts and check whether both passages needed to answer the question are actually present in the context. If only one made it in, then this is a <em>retrieval-recall</em> problem. A single-shot dense retrieval pass often fails to fetch two facts that don’t individually resemble the query. The fix is mostly around query decomposition or multi-query retrieval, not prompt tweaking. If both passages are there and the model still botches the comparison, that’s a synthesis failure, and “<a href="https://arxiv.org/abs/2307.03172">lost in the middle effect</a>” (the model anchoring on the first passage and under-weighting the second) is the usual suspect.</li><li><strong>Numeric / financial figures</strong> has κ=0.52. That’s weak agreement, and it’s the loudest signal on the whole table, as it means the judge itself can’t be trusted here yet. Numeric equivalence, though, is genuinely hard for an LLM judge to check (“20%” vs. “roughly a fifth” vs. “$20 off a $100 item” are all the same claim but just syntactically different). My instinct here is not to touch the RAG system at all until the rubric is fixed, as chasing a number you don’t trust is likely to be a waste of engineering time.</li><li><strong>Policy/procedural</strong> is flat and healthy, like the first row, but for a different underlying reason: it’s usually drawn from one well-structured document rather than needing synthesis across several. If this one ever dips, my first suspect wouldn’t be reasoning. It would be a numbered list getting split across chunk boundaries, so the model only sees steps 1–3 and invents 4–5.</li><li><strong>Out-of-scope </strong>is the one I’d fix first, even though n=20 is the smallest segment here. κ=0.90, which means the judge is highly reliable in this category. This is because “did it fabricate an answer with no supporting context” is much easier to verify than nuanced numeric or multi-hop claims. That reliability means I trust this drop as real. And it’s the most dangerous failure mode on the table. Basically, the system went from correctly saying “I don’t know” to confidently fabricating, which is worse than a wrong number because it’s delivered with identical confidence to a right answer. Likely, the cause here could either be retrieval is returning “closest-but-irrelevant” passages that the model treats as license to answer, or a recent prompt/model change quietly dropped the abstention instruction. Check the retrieval similarity scores for this segment first, and if they’re near the noise floor and the model still answers, you’re missing a relevance threshold gate before generation.</li></ol><p>That’s the whole exercise, really, and the lesson is simple:</p><ul><li><strong>don&#39;t trust the aggregate,</strong></li><li><strong>check judge reliability before you trust the score,</strong></li><li><strong>segment by category,</strong></li><li><strong>hypothesise which stage broke from actual evidence, then</strong></li><li><strong>prioritise by severity times confidence </strong>and<strong> </strong>NOT by which bar moved the most.</li></ul><p>A few things worth calling out, especially for new developers in evaluations:</p><ol><li>Groundedness judging adds a real LLM call per evaluated response. At any meaningful volume, sample rather than judge everything, and <a href="https://www.prolific.com/resources/what-is-stratified-sampling">stratify your sample</a> by query category so rare-but-important segments (like out-of-scope) don’t get drowned out by high-volume easy queries.</li><li>Re-validate your judge’s kappa whenever you change the judge model <em>or </em>the judge prompt. They both count as a new judge.</li><li>A judge tuned on one domain’s numeric or jargon quirks won’t automatically transfer to another. Recalibrate per domain, not once globally.</li></ol><p>FYI, all examples and figures in this piece are illustrative, not pulled from a live production system, and the framework and prompt structure, though, are exactly what I’d make use of as a starting point.</p><h4><strong>Conclusion</strong></h4><p>We covered a lot in this article: a stage-by-stage map for localising RAG failures, why <em>groundedness</em> and <em>correctness</em> are different things worth measuring separately, how to build a claim-level judge instead of a single blunt score, and probably the most skipped step, which is how to calibrate that judge against humans before trusting a single number it produces.</p><p>The only limitation I can think of here is the assumption that I made that everything above assumes a single, clean block of retrieved context. That assumption falls apart the moment your system starts calling tools, chaining multiple retrieval steps, or reasoning across its own intermediate outputs, which is exactly what an agentic system does. I’m in the process of writing Part 2 of this article, especially on what “groundedness” even means when there’s no single context block left to check against, and how the judge design above needs to change.</p><p>If you found this useful, please clap, leave a comment, or share it and let me know if you’ve hit any groundedness failures I haven’t covered here. Don’t forget to <a href="https://medium.com/@hermanwandabwa">follow me</a> for Part 2, and I’m always happy to connect on <a href="https://www.linkedin.com/in/wandabwaherman/">LinkedIn</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=be3924f924ba" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science-collective/how-to-diagnose-rag-failures-with-a-groundedness-judge-be3924f924ba">How to Diagnose RAG Failures with a Groundedness Judge</a> was originally published in <a href="https://medium.com/data-science-collective">Data Science Collective</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Letting an LLM Query Your Database Tables Safely]]></title>
            <link>https://medium.com/data-science-collective/letting-an-llm-query-your-database-tables-safely-7e16224138c8?source=rss-58e995b6d0e3------2</link>
            <guid isPermaLink="false">https://medium.com/p/7e16224138c8</guid>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[ai-agent]]></category>
            <category><![CDATA[ai-tool-calling]]></category>
            <category><![CDATA[databases-and-llms]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <dc:creator><![CDATA[Herman Wandabwa]]></dc:creator>
            <pubDate>Tue, 30 Jun 2026 12:57:44 GMT</pubDate>
            <atom:updated>2026-06-30T19:59:56.425Z</atom:updated>
            <content:encoded><![CDATA[<h4>How tool calling connects language models to live data while keeping permissions, validation, and execution inside your code</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qvBXeh39wVr-NWhOre-G6A.png" /><figcaption>Image generated by the author</figcaption></figure><p><strong>Stuck behind a firewall. Read the article for free </strong><a href="https://hermanwandabwa.medium.com/7e16224138c8?source=friends_link&amp;sk=395f139b045d9dfd2c0a355c75e0787a"><strong>HERE</strong></a></p><p>I’m certain that most of you have heard about agentic AI systems in the past year or so. Don’t worry if the term is new to you. I’ll simply describe them as AI systems that can autonomously plan, make decisions, and take multi-step actions toward a goal rather than just responding to single prompts the way conventional Large Language Models (LLMs) like ChatGPT do. <strong><em>Tool calling</em></strong>, sometimes also called <strong><em>function calling</em></strong>, is a fundamental part of the agentic AI ecosystem. It is the mechanism that allows a Large Language Model (LLM) to request the execution of an external function as part of its response as opposed to the normal question-and-answer approach that is common in LLMs. Therefore, instead of only returning text, the model can decide that it needs a tool, choose which tool to call, and provide the arguments that should be passed into that tool. The most important detail is that the <strong>LLM only produces a structured request, while the actual execution happens in the code</strong>.</p><p>That small distinction matters a lot. If a user asks about the weather, the LLM might decide to call a weather function with the city name as an argument. The code then calls the weather API, gets the result, and sends it back to the model so that it can produce a final answer. In that setup, the LLM is no longer just a passive text generator but instead becomes part of a workflow that can interact with external systems.</p><p>I’ve always wondered how/whether tool calling can happen at a database table level, and it&#39;s the reason I chose to experiment a little bit and write this article. FYI, this is something that I haven&#39;t really tried before, so it&#39;s also a learning process for me. Imagine an LLM asking your code to look up a customer, count orders, check payment status, or answer a question from live business data and then build a dashboard from the output.</p><p>In this article, we will build the idea slowly starting with one narrow database tool, then give the model a choice between two tools, and finally look at the tempting shortcut where the model writes SQL itself.</p><h4>The tool calling loop</h4><p>Before we touch the database, it is worth making the tool-calling loop clear. Normally, a user in the tool call iteration sends a message, the model reads the message, and it decides whether one of the available tools can help; then it returns a structured tool call instead of a normal text answer.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ShhUqVXNZBJ5y9EzC2sGNw.png" /><figcaption>Tool Calling Loop</figcaption></figure><p>Generally, the tool calling is as follows:</p><p>A user asks a question, then: <br>→ The model chooses a tool<br>→ Model returns a tool call<br>→ Your code receives that tool call<br>→ Your code checks whether the call is allowed<br>→ Your code runs the selected function<br>→ Your code sends the result back to the model<br>→ The model uses that result to generate the final answer</p><p>The model’s first response might look like this:</p><pre>{<br>  &quot;name&quot;: &quot;get_customer_orders&quot;,<br>  &quot;arguments&quot;: {<br>    &quot;customer_email&quot;: &quot;jane@example.com&quot;<br>  }<br>}</pre><p>One thing to note in relation to the structured request mentioned earlier is that the model doesn’t touch the database. It just asks the code to do something, which then decides whether to run the function, how to run it, and what result to send back. That gap between the model asking and your code acting is where all the safety controls need to live.</p><h4>Setting up a small database</h4><p>To keep the example simple, we will use SQLite. SQLite makes sense in this example, as we can quickly prototype through it without the need for a server, API key, and extra infrastructure to worry about. We will create a tiny shop database with two tables, one for customers and one for orders, then use that database to show how tool calling works in practice. By the way, all the code is <a href="https://github.com/wandabwa2004/LLMs/tree/dev/Agents/tool_calling">here</a>. Just follow the instructions in the <a href="https://github.com/wandabwa2004/LLMs/blob/dev/Agents/tool_calling/README.md">Readme</a> file to set up and run every example in here.</p><pre>import sqlite3<br><br>conn = sqlite3.connect(&quot;shop.db&quot;)<br><br>conn.executescript(&quot;&quot;&quot;<br>CREATE TABLE customers (<br>    id     INTEGER PRIMARY KEY,<br>    name   TEXT,<br>    email  TEXT UNIQUE<br>);<br>CREATE TABLE orders (<br>    id           INTEGER PRIMARY KEY,<br>    customer_id  INTEGER REFERENCES customers(id),<br>    product      TEXT,<br>    quantity     INTEGER,<br>    status       TEXT,<br>    order_date   TEXT<br>);<br>INSERT INTO customers (name, email) VALUES<br>    (&#39;Jane Doe&#39;,     &#39;jane@example.com&#39;),<br>    (&#39;Nikos Pappas&#39;, &#39;nikos@example.com&#39;);<br>INSERT INTO orders (customer_id, product, quantity, status, order_date) VALUES<br>    (1, &#39;Mechanical Keyboard&#39;, 1, &#39;shipped&#39;,    &#39;2026-06-10&#39;),<br>    (1, &#39;USB-C Cable&#39;,         3, &#39;delivered&#39;,  &#39;2026-05-22&#39;),<br>    (2, &#39;Monitor Stand&#39;,       1, &#39;processing&#39;, &#39;2026-06-18&#39;);<br>&quot;&quot;&quot;)<br>conn.commit()<br>conn.close()</pre><p>I believe two customers and three orders as examples should be fine for now. It is small enough to understand without getting lost, but still real enough to show the pattern properly. You can add more complexity levels to it. Let&#39;s go through a few tool call examples:</p><h4>1. A single safe tool</h4><p>Let’s say we are building a support assistant and a user asks, <em>“What has </em><a href="/service/mailto:jane@example.com"><em>jane@example.com</em></a><em> ordered?”</em> We do not want the model to guess, because there is a high chance that it will happily invent a plausible answer if we let it, a concept called <a href="https://www.ibm.com/think/topics/ai-hallucinations">hallucination</a>. We want it to fetch the real answer from the database.</p><p>The safest way to start is NOT to give the model SQL as in the code below. That is the important design choice. We are not saying, <em>“Write whatever SQL you want.”</em> We are simply saying, <em>“Here is one tool. It takes one email address. That is all.”</em></p><pre>tools = [<br>    {<br>        &quot;type&quot;: &quot;function&quot;,<br>        &quot;function&quot;: {<br>            &quot;name&quot;: &quot;get_customer_orders&quot;,<br>            &quot;description&quot;: &quot;Look up all orders placed by a customer, given their email address&quot;,<br>            &quot;parameters&quot;: {<br>                &quot;type&quot;: &quot;object&quot;,<br>                &quot;properties&quot;: {<br>                    &quot;customer_email&quot;: {<br>                        &quot;type&quot;: &quot;string&quot;,<br>                        &quot;description&quot;: &quot;The customer&#39;s email address, for example jane@example.com&quot;<br>                    }<br>                },<br>                &quot;required&quot;: [&quot;customer_email&quot;]<br>            }<br>        }<br>    }<br>]</pre><p>Notice the shape of this tool where the model can only provide a customer email. It cannot choose a table, change the query, ask for every customer, or delete anything. It has one blank to fill in, and that is what makes this version “boring” but in a good way.</p><p>Now we send the user message together with the tool definition.</p><pre>messages = [<br>    {&quot;role&quot;: &quot;user&quot;, &quot;content&quot;: &quot;What has jane@example.com ordered?&quot;}<br>]<br>response = client.chat.completions.create(<br>    model=&quot;gpt-4o-mini&quot;,<br>    tools=tools,<br>    messages=messages<br>)<br>print(response.choices[0].message)</pre><p>Instead of answering directly, the model returns a tool call.</p><pre>ChatCompletionMessage(<br>    content=None,<br>    role=&#39;assistant&#39;,<br>    tool_calls=[<br>        ChatCompletionMessageToolCall(<br>            id=&#39;call_xyz789&#39;,<br>            type=&#39;function&#39;,<br>            function=Function(<br>                name=&#39;get_customer_orders&#39;,<br>                arguments=&#39;{&quot;customer_email&quot;: &quot;jane@example.com&quot;}&#39;<br>            )<br>        )<br>    ]<br>)</pre><p>The key detail is content=None where the model has not given us a final answer yet. It has only asked us to run a function, and now the code has to decide what to do with that request.</p><p>Here is the actual function that queries the database:</p><pre>def get_customer_orders(customer_email: str):<br>    conn = sqlite3.connect(&quot;shop.db&quot;)<br>    conn.row_factory = sqlite3.Row<br>    rows = conn.execute(<br>        &quot;&quot;&quot;<br>        SELECT product, quantity, status, order_date<br>        FROM orders o<br>        JOIN customers c ON c.id = o.customer_id<br>        WHERE c.email = ?<br>        ORDER BY order_date DESC<br>        &quot;&quot;&quot;,<br>        (customer_email,)<br>    ).fetchall()<br>    conn.close()<br>    return {<br>        &quot;email&quot;: customer_email,<br>        &quot;orders&quot;: [dict(row) for row in rows]<br>    }</pre><p>The most important part of this function is the ?. The email is not pasted into the SQL string. It is passed as a bound parameter, which means it is treated as data, not as SQL. So even if the user tries something nasty like &#39; OR 1=1 --, it does not become part of the query logic. It is just treated as a strange email address. That small detail is doing a lot of safety work.</p><p>Now we extract the tool call, run the function, and send the result back to the model.</p><pre>import json<br>tool_call = response.choices[0].message.tool_calls[0]<br>arguments = json.loads(tool_call.function.arguments)<br>result = get_customer_orders(**arguments)<br>messages.append(response.choices[0].message)<br>messages.append({<br>    &quot;role&quot;: &quot;tool&quot;,<br>    &quot;tool_call_id&quot;: tool_call.id,<br>    &quot;content&quot;: json.dumps(result)<br>})<br>final_response = client.chat.completions.create(<br>    model=&quot;gpt-4o-mini&quot;,<br>    tools=tools,<br>    messages=messages<br>)<br>print(final_response.choices[0].message.content)</pre><p>Now the model can turn the raw rows into a normal answer.</p><pre>Jane has placed two orders: one Mechanical Keyboard, which has shipped, and three USB-C Cables, which have been delivered.</pre><p>That is the basic pattern. The model picked the tool, our code ran the query, and the model turned the result into a readable answer. For many real systems, this narrow-tool pattern is the version you actually want. It is not flashy and simply gets to the point.</p><h4>2. Letting the model choose between tools</h4><p>One tool is useful, but real assistants usually need more than one capability. Maybe the user wants to look up a customer’s orders or they just want to know how many orders are still processing. Instead of writing one giant function that tries to do everything, we can give the model a small set of clear tools, just like the below, and let it decide which one fits the user’s question.</p><pre>tools = [<br>    {<br>        &quot;type&quot;: &quot;function&quot;,<br>        &quot;function&quot;: {<br>            &quot;name&quot;: &quot;get_customer_orders&quot;,<br>            &quot;description&quot;: &quot;Look up all orders placed by a customer, given their email address&quot;,<br>            &quot;parameters&quot;: {<br>                &quot;type&quot;: &quot;object&quot;,<br>                &quot;properties&quot;: {<br>                    &quot;customer_email&quot;: {<br>                        &quot;type&quot;: &quot;string&quot;,<br>                        &quot;description&quot;: &quot;The customer&#39;s email address&quot;<br>                    }<br>                },<br>                &quot;required&quot;: [&quot;customer_email&quot;]<br>            }<br>        }<br>    },<br>    {<br>        &quot;type&quot;: &quot;function&quot;,<br>        &quot;function&quot;: {<br>            &quot;name&quot;: &quot;count_orders_by_status&quot;,<br>            &quot;description&quot;: &quot;Count how many orders currently have a given status&quot;,<br>            &quot;parameters&quot;: {<br>                &quot;type&quot;: &quot;object&quot;,<br>                &quot;properties&quot;: {<br>                    &quot;status&quot;: {<br>                        &quot;type&quot;: &quot;string&quot;,<br>                        &quot;enum&quot;: [&quot;processing&quot;, &quot;shipped&quot;, &quot;delivered&quot;],<br>                        &quot;description&quot;: &quot;The order status to count&quot;<br>                    }<br>                },<br>                &quot;required&quot;: [&quot;status&quot;]<br>            }<br>        }<br>    }<br>]</pre><p>One safety detail to note in the above code is that the status parameter has an enum. What this means is that the model cannot just send any random status value. It has to choose one of the values that we allow: processing, shipped, or delivered. Again, this is a small lock, and small locks matter especially in systems that are likely to be accessed by many users.</p><p>The second function can then be defined as follows, and it uses a bound parameter.</p><pre>def count_orders_by_status(status: str):<br>    conn = sqlite3.connect(&quot;shop.db&quot;)<br>    count = conn.execute(<br>        &quot;SELECT COUNT(*) FROM orders WHERE status = ?&quot;,<br>        (status,)<br>    ).fetchone()[0]<br>    conn.close()<br>    return {<br>        &quot;status&quot;: status,<br>        &quot;count&quot;: count<br>    }</pre><p>Here, the model supplies the status, but our code controls the SQL. What this means is that if the user asks a question like, <em>“How many orders are still processing?”</em>, the model should be able to choose the count_orders_by_status tool and pass &quot;processing&quot; as the status.</p><pre>messages = [<br>    {&quot;role&quot;: &quot;user&quot;, &quot;content&quot;: &quot;How many orders are still processing?&quot;}<br>]<br>response = client.chat.completions.create(<br>    model=&quot;gpt-4o-mini&quot;,<br>    tools=tools,<br>    messages=messages<br>)<br>tool_call = response.choices[0].message.tool_calls[0]<br>print(tool_call.function.name)<br>print(tool_call.function.arguments)</pre><p>The response should look like this:</p><pre>count_orders_by_status<br>{&quot;status&quot;: &quot;processing&quot;}</pre><p>This is the part that starts to feel agentic. The model reads the user’s request, looks at the tools available, decides which one fits, and fills in the arguments. We did not write a rule that says, <em>“If the user says processing, call this function.”</em> The model made that decision from the tool descriptions and schema.</p><p>That decision-making is the useful part of tool calling. The model is no longer just producing text but choosing between actions. The safety boundary stays in the code. The model can ask for an action, but the application decides if it is allowed.</p><h4>3. The tempting shortcut</h4><p>At some point, the narrow-tool approach might start to feel annoying. You might want to answer questions like, <em>“Which product sold the most units?”, “Which customers ordered in June?”</em>, or <em>“How many orders were delivered last month?” </em>Writing a separate function for every possible question gets tiring, so there is an obvious shortcut of giving the model one tool that accepts SQL.</p><pre>tools = [<br>    {<br>        &quot;type&quot;: &quot;function&quot;,<br>        &quot;function&quot;: {<br>            &quot;name&quot;: &quot;run_query&quot;,<br>            &quot;description&quot;: &quot;Run a read-only SQL query against the shop database and return the rows&quot;,<br>            &quot;parameters&quot;: {<br>                &quot;type&quot;: &quot;object&quot;,<br>                &quot;properties&quot;: {<br>                    &quot;sql&quot;: {<br>                        &quot;type&quot;: &quot;string&quot;,<br>                        &quot;description&quot;: &quot;A single SELECT statement&quot;<br>                    }<br>                },<br>                &quot;required&quot;: [&quot;sql&quot;]<br>            }<br>        }<br>    }<br>]</pre><p>With the above code, the model can write a query for a question we did not plan for. The result here is that if the user asks a question like, <em>“Which product sold the most units?”</em>, then the model is likely to produce a query like this:</p><pre>SELECT product, SUM(quantity) AS total<br>FROM orders<br>GROUP BY product<br>ORDER BY total DESC<br>LIMIT 1</pre><p>That is powerful, but it creates a much bigger safety problem. If the model can send a useful SELECT query, it can also try to send DROP TABLE customers or DELETE FROM orders. The model still does not execute the SQL, but if your code blindly obeys it, then that distinction does not help you much.</p><p>If you are going to let the model write SQL, then you need guardrails in the code. At minimum, open the database in read-only mode like below:</p><pre>def run_query(sql: str):<br>    conn = sqlite3.connect(&quot;file:shop.db?mode=ro&quot;, uri=True)<br>    conn.row_factory = sqlite3.Row<br>    try:<br>        rows = conn.execute(sql).fetchall()<br>    finally:<br>        conn.close()<br>    return {<br>        &quot;rows&quot;: [dict(row) for row in rows[:50]]<br>    }</pre><p>The mode=ro part matters because it makes the database connection itself read-only. If the model tries to run a write operation, the database should reject it. That is much better than writing a comment that says “only run safe queries.” Just remember comments on their own do not protect your database. The database connection can.</p><p>But read-only mode is still not enough. A safer version would also check that the query is a plain SELECT, limit the number of rows returned, set a timeout, block access to sensitive tables, log every query, review unusual queries, and use a restricted database user in production. Even then, read-only does not mean <strong>harmless</strong>. A SELECT query can still leak data, as it can read customer emails, payment records, internal notes, or anything else the database user can access. Read-only protects data from being changed. It does not protect data from being revealed.</p><h4>The model also needs to know the schema</h4><p>There is another practical issue with letting the model write SQL. It needs to know what tables and columns exist. With the narrow tools, the schema is hidden inside the function. The model does not need to know that there is an orders table or a customer_id column. It only needs to know that it should provide an email address or a status value.</p><p>With raw SQL, the model needs more context. You either have to describe the schema in the prompt or give the model a separate tool that describes the schema. Without that, the model is guessing, and models are very good at guessing confidently, something you might not want near your database.</p><p>For this tiny example, the schema is easy to describe because we only have two tables. In a real system, the schema might include dozens of tables, confusing column names, old fields that should no longer be used, and sensitive tables that should never be queried by an assistant. So schema exposure is not just a convenience issue. It is also a safety and design issue.</p><h4>Reading is one thing. Writing is another.</h4><p>Everything so far has been read-only on purpose. The moment you let the model write to the database, you are in a different category of risk. A read tool might answer, <em>“Order 18 is still processing.”</em> A write tool might actually mark order 18 as shipped. Those are not the same kind of action.</p><p>For write operations, I would not let the model act directly. I would make it propose the action first, then require confirmation. The assistant should say something like, <em>“I am about to mark order 18 as shipped. Should I continue?”</em> Only after the user confirms should the code execute the update. The model can help prepare the action, but it should not be allowed to quietly commit it.</p><p>This matters because the model can be wrong in very normal ways. It can misunderstand the user, pick the wrong record, confuse two similar customers, or generate an update that looks reasonable but changes the wrong thing. When the action is only a read, the damage is usually limited to a bad answer or a data exposure problem. When the action is a write, the model can alter the state of your system.</p><h4>Conclusion</h4><p>That’s the end of this article, and I hope that you’ve benefitted. The core lesson is simple. The model asks, but your code decides. That was true in the simple tool-calling example, and it becomes much more important when the tool connects to your database. Every safety feature lives in the gap between the model asking and your code acting. The bound SQL parameter, the enum, the read-only connection, the row limit, the schema restrictions, and the confirmation step are not model features. They are engineering decisions.</p><p>So my rule would be simple. Do not give an LLM unrestricted SQL access to a real database. That is not a feature but an incident waiting to happen. As a rule of thumb, start with narrow tools. Give the model small, specific actions with clear parameters and bind every value safely.</p><p>Only reach for raw SQL when you have a strong reason, and only after you have put the locks back in place. A model can look at a user’s question, decide which tool to use, query real data, and turn the result into a useful answer. That is a big shift, but the responsibility is also real. The model can ask for anything. It&#39;s your decision to make on what it&#39;s allowed to have.</p><p>If you found this useful, please clap, leave a comment, or share it with someone working on agentic AI, RAG, or LLM systems that make multi-step decisions. As always, you can find my other articles on my <a href="https://medium.com/@hermanwandabwa">profile</a>, and I am always happy to connect on <a href="https://www.linkedin.com/in/wandabwaherman/">LinkedIn</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7e16224138c8" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science-collective/letting-an-llm-query-your-database-tables-safely-7e16224138c8">Letting an LLM Query Your Database Tables Safely</a> was originally published in <a href="https://medium.com/data-science-collective">Data Science Collective</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Turning Patient Number Predictions into Staffing Numbers Without Fancy AI]]></title>
            <link>https://medium.com/data-science-collective/turning-patient-number-predictions-into-staffing-numbers-without-fancy-ai-05b8f5da3d0e?source=rss-58e995b6d0e3------2</link>
            <guid isPermaLink="false">https://medium.com/p/05b8f5da3d0e</guid>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[forecasting]]></category>
            <category><![CDATA[data-analytics]]></category>
            <category><![CDATA[optimisation]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Herman Wandabwa]]></dc:creator>
            <pubDate>Mon, 22 Jun 2026 00:06:00 GMT</pubDate>
            <atom:updated>2026-06-24T21:30:32.341Z</atom:updated>
            <content:encoded><![CDATA[<h4>How I turned a patient-volume forecast into a low-cost roster with wait-time targets and uncertainty built in</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*pN9dar0BhJfPvyj-aHMibQ.png" /><figcaption>Image generated by the author</figcaption></figure><p><em>Stuck behind a firewall? Read the article for FREE </em><a href="https://hermanwandabwa.medium.com/05b8f5da3d0e?source=friends_link&amp;sk=ffa93c9f303c2f8ecb1e8a8610526ee5"><strong>HERE</strong></a><em>.</em></p><p><em>A follow-up to </em><a href="https://medium.com/data-science-collective/predicting-daily-patient-volume-for-a-melbourne-urgent-care-clinic-d4ca30007991"><strong><em>Predicting Daily Patient Volume for a Melbourne Urgent Care Clinic</em></strong></a><em>. All data is simulated for educational purposes. As always, the full code is open-sourced and is </em><a href="https://github.com/wandabwa2004/urgent_care_forecast/tree/main/ml-pipeline/src/optimization"><em>here</em></a><em> and in this </em><a href="https://github.com/wandabwa2004/urgent_care_forecast/blob/main/notebooks/06_staffing_optimization.ipynb"><em>notebook</em></a><em>. You can just clone the full </em><a href="https://github.com/wandabwa2004/urgent_care_forecast/tree/main"><em>repo</em></a><em> for access to the full implementation.</em></p><p>In the last <a href="https://medium.com/data-science-collective/predicting-daily-patient-volume-for-a-melbourne-urgent-care-clinic-d4ca30007991">article</a>, I built a model to predict how many patients would walk into a fictional Melbourne urgent care clinic each day. XGBoost performed best in the predictions, with an average error of about 17 patients.</p><p>The performance was impressive, but again, a forecast doesn’t roster a clinic, and this is the second bit that I thought of and wanted to achieve.</p><p>For rostering managers, the first question when they see such numbers won’t be about the R². Their concern will just be on the number of <strong>doctors and support staff to schedule the following day, including the cost of doing so.</strong></p><p>That’s the gap this article closes, and I think I promised to address this in the <a href="https://medium.com/data-science-collective/predicting-daily-patient-volume-for-a-melbourne-urgent-care-clinic-d4ca30007991">first part</a> of this series. The first article predicted patient numbers on any given day. This one should be able to tell the actual number of doctors, nurses, and admin staff to roster; the shifts they sit on; and the cost. I love optimisation and thought of adding this optimisation layer on top of the forecast. FYI, I actually started writing on optimisation very early on. Some of the articles on capacity optimisation can be found <a href="https://medium.com/data-science/capacity-optimization-in-freight-trains-part-1-4918f35a6433">here</a> and <a href="https://medium.com/data-science/capacity-optimization-in-freight-trains-with-real-data-d21296e2b56">here</a>.</p><p>Let me walk you through what I did in this piece. Meanwhile, I’ll be happy to be challenged, as I’m aware there are so many ways around this optimisation problem.</p><h4><strong>The core idea: Don’t roster to the forecast</strong></h4><p>The easiest approach in rostering is to schedule staff numbers according to the forecast. For example, if the model says the forecasted patient numbers are 130, you also just choose an arbitrary number of 13 staff members or any number and hope for the best. This approach in most instances could also be influenced by past rosters.</p><p>The trouble is that a point forecast is only a <strong>central estimate</strong>, meaning real demand will often land above it. And when it does, the cost is not theoretical. Patients will pile up in the waiting room while the in-charge person rings around for a last-minute locum.</p><p>So the real question is not, &quot;<strong>What is the forecast?</strong>” It is <strong><em>“How much extra should the rostering manager plan for, and what does that extra buy them?”</em></strong></p><p>I answered this question in two stages, and the whole <a href="https://github.com/wandabwa2004/urgent_care_forecast/blob/main/notebooks/06_staffing_optimization.ipynb">notebook</a> with usable code is built around them:</p><pre>| <strong>Stage              | The question it answers                                            |The tool                         |</strong><br>|--------------------|--------------------------------------------------------------------|---------------------------------|<br>| 1 — Requirements   | How many of each role, per hour, to keep waiting times acceptable? | Demand scenarios + <strong>Erlang-C</strong>     |<br>| 2 — Rostering      | Which shifts, at the lowest cost, cover those requirements?        | <strong>OR-Tools</strong>                        |</pre><p>These two are well-known ideas from operations research that sit on top of a machine-learning forecast. I’ll walk you through them.</p><h4><strong>Turning one forecast into a range of demand</strong></h4><p>A single forecast is useful, but staffing decisions depend on understanding how <strong>busy it could possibly get</strong>, not just what is expected on an average day.</p><p>To do that, I used the model’s past mistakes. During time-series cross-validation, each fold makes predictions on days it has never trained on. I then record how far off those predictions are, calculated as actual − predicted.</p><p>That gave me a pool of <strong>300 out-of-fold errors</strong>, built earlier in <a href="https://github.com/wandabwa2004/urgent_care_forecast/blob/main/notebooks/04_modeling.ipynb">04_modeling.ipynb</a> and saved to disk. You can re-run this to get the exact numbers for the next part.</p><p>The <a href="https://github.com/wandabwa2004/urgent_care_forecast/blob/main/notebooks/06_staffing_optimization.ipynb">staffing notebook</a> loads that pool, resamples from it, and adds those errors back onto the point forecast. Therefore, one forecast ends up becoming a thousand “what-if” days:</p><pre>rng = np.random.default_rng(a.random_seed)<br>residuals = np.load(&quot;ml-pipeline/models/cv_residuals.npy&quot;)   # 300 past errors<br>scenarios = point_forecast + rng.choice(residuals, size=1000, replace=True)<br>planned_demand = np.quantile(scenarios, 0.90)                # plan to the P90 day</pre><p>Two things about those errors are worth knowing, and the notebook prints both:</p><ul><li>Their <strong>average </strong>is<strong> +7.2</strong>, which means the model tends to under-predict a little. Therefore, taking the forecast at face value already starts the roster too low.</li><li>Their <strong>right tail is “fat”</strong>. Those are the surge days, such as thunderstorm asthma days or the day after a public holiday. A neat bell-curve assumption would smooth them away, and resampling the model’s real errors keeps them in the picture.</li></ul><p>The plan is to the <strong>90th percentile (P90)</strong> of those thousand “what-if” days. What this means is that for a forecast of 130 patients, the P90 day works out to about <strong>164 patients</strong>, which is roughly a <strong>26% buffer</strong>.</p><p>In plain terms, I am staffing so that demand only overruns the plan about one day in ten.</p><p>That “one day in ten” is not a guess. It is a setting I choose upfront, which is the coverage value in the configuration. So instead of discovering the risk level later through long queues, I set it deliberately before the roster is built.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/773/1*RbGuJb8oX-7qkNAMLnWavg.png" /><figcaption>Demand distribution with the P90 plan line. <em>Demand for a forecast of 130 patients. The dashed line is the raw forecast; the red line is the P90 day (164) I actually plan for. The long tail on the right is exactly what the buffer is there to catch.</em></figcaption></figure><h4><strong>Spreading the day into hours</strong></h4><p>Two things to understand here. 164 patients is still not a staffing number, and patients do not arrive evenly across the day<strong>. </strong>A clinic can be calm at 8am and slammed at 6pm. What matters here is whether enough people are rostered during the busy hours.</p><p>The <strong>queuing model</strong> in the next step needs an hourly arrival rate, but my simulation only produces daily totals. So, I worked around this by spreading each simulated day across opening hours using a typical urgent-care shape: a morning build-up, a midday plateau, and an after-work bump.</p><p>This shape is an ASSUMPTION, and I must admit that it’s one of the biggest sources of risk in this pipeline. The moment a real clinic has actual hour-by-hour arrival data, then this curve MUST be replaced with theirs.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/764/1*INLXGEiOdT8VU6l23G_Dug.png" /><figcaption>Intraday arrival profile. This is the assumed hour-by-hour arrival for a 164-patient day. The tallest bar is the busiest hour, where the staffing has to survive, not the daily average.</figcaption></figure><h4>How many doctors are actually needed?</h4><p>Now we know how many patients arrive in the busiest hour. The next question is simple: how many doctors does it take to keep the wait acceptable?</p><p>This is where <a href="https://www.callcentretools.com/tools/erlang-calculator/"><strong>Erlang-C</strong></a> comes in. Do not worry if the name sounds intimidating. It is an old queueing formula from the days of telephone switchboards, but the idea is straightforward.</p><p>Give it three things:</p><ul><li>how many patients arrive per hour</li><li>how fast one doctor sees patients, which is 4 patients per hour if each consult takes 15 minutes</li><li>how many doctors are on duty</li></ul><p>From that, it estimates the chance that a patient has to wait and the probability that the wait exceeds a chosen threshold.</p><p>Instead of using Erlang-C to calculate waiting times for a fixed number of doctors, I use it to find the number I actually care about, the <strong>fewest doctors needed so that no more than 20% of patients wait longer than 30 minutes</strong>. This could also be adjusted to your liking.</p><p>Here is the heart of it, simplified:</p><pre>def prob_wait_exceeds(c, lam, mu, t_hours):<br>    &quot;&quot;&quot;Chance a patient waits longer than t, with c doctors on duty.&quot;&quot;&quot;<br>    if c &lt;= lam / mu:           # not enough doctors: the queue never settles<br>        return 1.0<br>    pw = erlang_c(c, lam / mu)  # chance of waiting at all<br>    return pw * math.exp(-(c * mu - lam) * t_hours)</pre><p>The line of interest in the above code is this if c &lt;= lam / mu.</p><p>Here, lam / mu is the number of doctors needed just to keep up with the arriving work. If the doctors on duty can only match that load, or fall below it, the queue does not stabilise and will keep growing, and waits spiral.</p><p>That is the difficult part of queueing. The difference between a calm clinic and an overwhelmed one can be a single clinician on the wrong side of that line.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/768/1*fqy81PnEhlDpzAd5U52PFQ.png" /><figcaption>Erlang-C wait-probability curve at the peak hour. Chance of waiting more than 30 minutes as doctors are added during the busiest hour. The drop is steep, not gradual. The dotted line marks the fewest doctors needed to bring that risk below the 20% target.</figcaption></figure><h4><strong>Allowing for an uncertain day</strong></h4><p>Because demand itself is uncertain, the <strong>number of doctors</strong> needed in the busy hour is uncertain too. It changes across the thousand “what-if” days.</p><p>So there is no need to staff for the average number of doctors required. Instead, the staffing should be to the <strong>90th percentile</strong> of doctor demand, using the same one-day-in-ten safety margin that I applied when forecasting patient demand.</p><p>For example, that day lands at <strong>6 doctors at the peak hour</strong>. At that staffing level, the expected wait is about <strong>3 minutes</strong>, and the chance of waiting more than 30 minutes stays inside the target.</p><p>The output below is a plot showing how many doctors, nurses, and admin staff are needed in <strong>each hour</strong> of the day.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/773/1*Yb19MMHWZel1BhXV4aPgBQ.png" /><figcaption><em>How many doctors the busy hour needs across the thousand what-if days. I plan to the red line (the P90), so a worse-than-typical day is still covered.</em></figcaption></figure><h4>Building the actual roster with Google’s OR-Tools</h4><p>A good question to ask here is that if Erlang-C already tells me how many doctors are needed each hour, then why do I need a second tool?</p><p>The answer here is that knowing you need 6 doctors at 6pm is not a roster. A typical emergency care clinic will have personnel working in <strong>shifts</strong>, not single hours. The real problem is finding the cheapest mix of shifts that covers the hourly requirements while respecting the staffing rules.</p><p>This now presents a <a href="https://www.sciencedirect.com/topics/computer-science/combinatorial-problem">combinatorial problem</a>, and it is exactly the kind of thing <a href="https://developers.google.com/optimization"><strong>Google OR-Tools</strong></a> is useful for.</p><p>I give the optimiser the hourly requirements from part 1 above and three shifts to choose from: Early (08–14), Mid (11–17), and Late (14–20). It then picks the cheapest mix that covers the day.</p><pre>rec = optimizer.recommend(point_forecast=130)<br>print(rec.bodies_to_roster) # {&#39;doctor&#39;: 11, &#39;nurse&#39;: 8, &#39;admin&#39;: 4}<br>print(rec.headcount_by_shift) # Early: 6 doctors, Late: 5 doctors, ...<br>print(rec.daily_cost) # 13980.0</pre><p>So for a 130-patient forecast, the recommended roster is <strong>11 doctors, 8 nurses, and 4 admin staff</strong> across the day, at a cost of about <strong>$14,000 </strong>(of course, the base costings can be reconfigured to match the ground truth). The busiest hour still stays inside the waiting-time target.</p><p>Two small design choices matter here. First, coverage is a <strong>soft</strong> target with a heavy penalty. What this means is that the solver does not simply give up and say “impossible” when perfect coverage cannot be found. It still returns the best usable roster it can and flags any hour it could not fully cover. Second, I intentionally kept the solver settings fixed so that the same input gives the same roster.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/768/1*05Px1lcR4aZc-glBqa5lNQ.png" /><figcaption><em>The solid line is how many doctors each hour needs; the dashed line is how many the chosen shifts actually deliver. The solver’s job is to keep the dashed line on or above the solid one, as cheaply as possible.</em></figcaption></figure><h4><strong>Does it actually work?</strong></h4><p>The real test is whether the forecasting and rostering pipeline would have worked on days the model never saw. A model can look impressive on historical data, but what matters is whether its staffing recommendations hold up when faced with genuinely unseen demand.</p><p>So I ran the full pipeline on the real test-set forecasts from <a href="https://medium.com/data-science-collective/predicting-daily-patient-volume-for-a-melbourne-urgent-care-clinic-d4ca30007991">Part 1</a>, using the full 90-day held-out period. The first sanity check was simple: <strong>would the P90 plan have covered the actual demand?</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gjUujjW5ItdWQZ5eEAUHpg.png" /></figure><p>Across all 90 days, the P90 plan covered actual demand on <strong>85 days</strong>, or about <strong>94%</strong> of the time, against a target of 90%.</p><p>That’s slightly cautious, and I’m comfortable with that. It comes mostly from the fact that rosters move in whole people, not fractions of a doctor. Every hour’s requirement rounds up to the next clinician, and shifts can only ever over-cover, so a little extra slack is baked in.</p><p>The misses are the interesting part. Look at <strong>2025–10–06:</strong> <strong>257 patients against a forecast of 159</strong>. That is not mainly an optimisation failure. The root cause is a forecasting miss on a genuine freak day. No sensible buffer would have caught it without making normal days absurdly overstaffed That is exactly why the system has a separate standby flag for known shock conditions.</p><p>The average day cost is about <strong>$13,600</strong>, or roughly <strong>$1.22 million</strong> over the 90-day test period. That is the kind of number a manager can actually budget around and trade off.</p><h4>Which knob actually moves the cost?</h4><p>A roster is only as good as its assumptions, so I tested the two that matter most. One behaved exactly as expected, and the other was more interesting.</p><p><strong>Doctor consult time</strong> is the big lever here. This makes sense as the longer each visit takes, the fewer patients each doctor can handle per hour, so the roster will need more doctors.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*CRm97-chHnnI63OVitGnOg.png" /></figure><p>If three minutes are shaved off the average consult, then the model drops two doctors from the day. The <strong>wait-time target</strong> was another one that I looked at. I loosened it from “only 5% of patients may wait more than 30 minutes” to “40% may wait more than 30 minutes,” and the doctor count barely moved. I also capped utilisation at <strong>85%</strong>, meaning staff cannot be run flat out. That cap already forces enough doctors into the roster to keep queues under control. So relaxing the wait target buys almost nothing unless you also relax how hard you are willing to run people. Only when I pushed utilisation to 95% did the wait target start to matter, and even then it removed just one doctor at the loosest setting.</p><p>That’s the kind of insight you only get by building the full pipeline. The lever a manager thinks they’re pulling is acceptable waiting time, but it turns out to matter far less than one they rarely talk about, which is how hard they’re willing to run their staff.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*k1u2W_VxRKmT58UH7gcBPg.png" /><figcaption><em>Cost rises almost in a straight line with consult time as the dominant lever. Right: doctors against the wait target at two “busyness” caps. At 85% (red) the line is flat; only at 95% (orange) does easing the target shed even one doctor</em></figcaption></figure><h4>Every assumption is on the table</h4><p>One of the easiest ways for a forecasting or staffing model to lose credibility is for important assumptions to disappear into the background. Targets, service times, coverage levels, and cost estimates all influence the final recommendation, so they need to be visible rather than buried in code or spreadsheets. In this pipeline, nothing is allowed to hide behind a magic number. Every recommendation carries its key assumptions with it, making it clear what drove the result and allowing a manager to challenge the inputs before trusting the output.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/996/1*ZZJtPiO-inLQMDQa_-VfHA.png" /></figure><p>These all live in one file, <a href="https://github.com/wandabwa2004/urgent_care_forecast/blob/main/ml-pipeline/src/optimization/staffing_config.yaml">staffing_config.yaml</a>, so they can be changed without touching the code. Every result echoes them back, so the assumptions stay visible instead of quietly shaping the roster in the background.</p><h4>Some caveats</h4><p>I would be doing you a disservice if I presented this as plug-and-play. I admit that there are a number of limitations in my approach that you need to be aware of:</p><ul><li><strong>The hourly arrival shape is assumed, not measured.</strong> This is the biggest risk in the pipeline. Before trusting this in a real clinic, please replace the assumed curve with actual hour-by-hour arrivals.</li><li><strong>Erlang-C assumes patients wait.</strong> It does not model people who give up and leave. <a href="https://www.callcentrehelper.com/a-beginners-guide-to-the-erlang-a-formula-140998.htm">Erlang-A</a> would be more realistic because it handles abandonment, but that should be treated as a service-risk signal, not a shortcut to leaner staffing.</li><li><strong>The errors are treated as one pool.</strong> Every day inherits the same broad error spread. That means that the uncertainty layer does not fully distinguish an ordinary busy Monday from a genuinely shock day. That is why the standby recommendation keys off known shock drivers rather than relying only on the tail of the simulation.</li><li><strong>Costs and service times are placeholders.</strong> They are plausible Melbourne figures, but a real clinic should tune them against actual pay rates, award rules, appointment lengths, and observed consult times.</li><li><strong>Roles do not cover for each other.</strong> Each role is sized separately. I do not model a nurse practitioner taking load off doctors, or admin bottlenecks slowing down clinical throughput. A real clinic would maybe want some of those interactions included.</li></ul><h3>Conclusion</h3><p>That’s it for now. In <a href="https://medium.com/data-science-collective/predicting-daily-patient-volume-for-a-melbourne-urgent-care-clinic-d4ca30007991">Part 1</a>, we turned a simulated clinic into a daily patient-volume forecast. In this part, we turned that forecast into something a rostering manager can actually use. This was the lowest-cost roster, sized to cover demand on about nine days in ten, with a waiting-time target, a cost estimate, and every major assumption exposed.</p><p>The shape of the solution is general. <strong>Erlang-C</strong> answers how many people are needed. <strong>OR-Tools</strong> answers which shifts to use and at what cost, and the model’s own past errors decide how much buffer to carry.</p><p>Remember you can always swap the clinic for a call centre, a museum, or a supermarket checkout, and the skeleton barely changes.</p><p>There is plenty I would still extend. A good path would be using real hourly arrival data to get rid of the biggest assumption, a queuing model that handles walkouts, and smarter uncertainty that can recognise genuinely “dangerous” days before they hit. This is something that I’ll leave for you to work out.</p><p>As usual, the optimisation notebook can be found <a href="https://github.com/wandabwa2004/urgent_care_forecast/blob/main/notebooks/06_staffing_optimization.ipynb">here</a> with supporting files <a href="https://github.com/wandabwa2004/urgent_care_forecast/tree/main/ml-pipeline/src/optimization">here</a>. If this was useful, then please leave a comment or share it with someone working on forecasting, rostering, or operational planning. You can find my other articles on my <a href="https://medium.com/@hermanwandabwa">profile</a>, and I am always happy to connect on <a href="https://www.linkedin.com/in/wandabwaherman/">LinkedIn</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=05b8f5da3d0e" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science-collective/turning-patient-number-predictions-into-staffing-numbers-without-fancy-ai-05b8f5da3d0e">Turning Patient Number Predictions into Staffing Numbers Without Fancy AI</a> was originally published in <a href="https://medium.com/data-science-collective">Data Science Collective</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Building a Multi-Agent Deep Research Agent with LangGraph]]></title>
            <link>https://medium.com/data-science-collective/building-a-multi-agent-deep-research-agent-with-langgraph-203547b5fb12?source=rss-58e995b6d0e3------2</link>
            <guid isPermaLink="false">https://medium.com/p/203547b5fb12</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[langgraph]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[large-language-models]]></category>
            <category><![CDATA[agentic-ai]]></category>
            <dc:creator><![CDATA[Herman Wandabwa]]></dc:creator>
            <pubDate>Wed, 27 May 2026 11:56:27 GMT</pubDate>
            <atom:updated>2026-05-28T15:55:38.657Z</atom:updated>
            <content:encoded><![CDATA[<h4>A practical walkthrough of parallel specialist agents, citation auditing, and cited markdown report generation</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*JSAtFWSKooSbFU4XYeVR4g.png" /><figcaption>Image generated by the author</figcaption></figure><p><em>Stuck behind a firewall? Read the article for FREE </em><a href="https://open.substack.com/pub/hermanwandabwa/p/building-a-multi-agent-deep-research?r=hfb9c&amp;utm_campaign=post&amp;utm_medium=web&amp;showWelcomeOnShare=true"><strong>HERE</strong></a><em>.</em></p><p>Many so-called deep research agents are not doing deep research. They are search loops with nicer packaging where the usual pattern is simple. Put an LLM in a loop, attach a search tool like <a href="https://www.tavily.com/">Tavily</a>, let it gather links, then ask it to write a report. That can be useful, but it is not the same as structured research. Proper research needs planning, parallel investigation, claim checking, citation discipline, and enough restraint to avoid writing before the evidence has been tested.</p><p>That is the gap I wanted to explore.</p><p>Imagine you are a strategy analyst at a telco like <a href="https://www.safaricom.co.ke/">Safaricom</a>, and leadership asks for a brief by Friday on how major African telcos are using agentic AI in customer support. A basic chatbot might give you a confident answer, but there is a real chance it will hallucinate, miss important context, or produce something too generic to be useful. A human analyst would likely do a better job, but they may spend days reading reports, blog posts, press releases, vendor pages, and case studies and still miss useful details or weak citations.</p><p>What you really want in that situation is something closer to a small research team. The work needs to be split across scouting, planning, parallel research, claim checking, citation verification, and final synthesis.</p><p>That is the idea behind the design of this deep research agent. It is not just a search loop. It is a LangGraph-based research workflow designed to plan, investigate, challenge, verify, and write with structure. In this article, I will walk through what I built, why I built it this way, and how the system handles a real research problem from the initial question to the final report.</p><p>As always, the code is open source <a href="https://github.com/wandabwa2004/deep_research_agent/tree/dev">HERE</a>, so you can clone it, change the research question, and run it yourself. The one important caveat is that the output should not be treated as ground truth, as this system is designed to improve the structure and discipline of research and not replace proper verification. You should still check important claims, citations, and conclusions before using the output in any serious setting.</p><h4>TLDR</h4><p>A user question enters a LangGraph state machine and is routed as a meta-question, clarification, shallow lookup, or deep investigation. The deep path scouts the topic, plans the research, sends six specialised researchers to work in parallel, checks for gaps, audits citations, and then drafts and refines the final report. The key design choice is simple. The writer cannot run until the citations have been audited. If the run is interrupted with Ctrl+C, the system still produces the best partial report it can from the work already completed.</p><h4>1. The graph</h4><p>Here is the topology of the graph. It looks intimidating at first, but the idea is simple. Each node has just one job.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*E_tWmQj6_LIx7iwZwWq_Aw.png" /><figcaption>Graph Topology</figcaption></figure><p>I will walk through each node in order because some of the design choices only make sense once you see the full arc.</p><ul><li>classifier: This is the first node to see each question. It decides whether the question should go to meta_response, need_clarification, shallow_research, or deep_research. Since this is only a routing decision, and can easily run on the cheaper model tier.</li><li>shallow_research: Runs a small ReAct loop for simple factual lookups. It performs up to three Tavily searches and returns a cited answer. If the answer is not strong enough, the flow escalates to scout and joins the deep research path.</li><li>scout: This runs first for deep questions and does not answer the question. It maps the territory by identifying the main subtopics, key players, debates, and likely evidence sources. The architect uses this output to plan the research.</li><li>architect: Reads the scout output and turns it into a report outline and a set of ResearchTask objects. Each task includes a specialist role, a research angle, and a target section. The architect also decides which specialists to deploy, usually three to six depending on the question.</li><li>Send: Handles the parallel dispatch in LangGraph. The architect returns one Send per ResearchTask, and each branch runs independently. Running six specialists in parallel means the round takes roughly as long as the slowest branch.</li><li>specialistuses the same node code across all specialist roles, but receives a different prompt from its ResearchTask. Each specialist can search, extract, reason, and stop when it has enough evidence. Each branch returns a SpecialistNote with evidence, quotes, and source URLs. The evidence_gatherer looks for primary sources and hard numbers while the skeptic looks for weak claims and counter evidence while the mechanism_explorer explains how things work. The None safe list reducer keeps parallel writes from overwriting each other.</li><li>synthesis: Combines all SpecialistNote outputs into a structured brief mapped to the architect outline. This is not the final report, but a grouped draft structure built from raw research notes.</li><li>gap_analyzer: This checks whether the brief is complete enough to write. If it is, the flow moves to citation_audit and if not, then it creates follow-up ResearchTask objects and sends specialists into another parallel round. The loop has a hard cap called MAX_DEEP_ITERATIONS, enforced in Python routing code rather than left to the model.</li><li>citation_auditreceives all sources gathered by the specialists, removes duplicates, classifies source quality, and maps claims in the brief to source IDs in the registry. Here, the registry acts as a gate whereby anything outside it cannot be cited.</li><li>writerreceives the brief, outline, and citation registry. It drafts the final report using only approved source IDs and if a claim has no source in the registry, then the writer must leave it out. This node runs on the heaviest model tier because final report quality depends heavily on it.</li><li>refineronly sees the draft. It does not see the evidence, registry, or brief. Its job is to improve tone, flow, and formatting without adding new facts or citations. That separation is deliberate because a refiner that cannot see the sources cannot invent better-looking citations.</li></ul><p>Once the refiner returns, the run ends. The CLI writes the final report to --out or stdout and exits.</p><p><strong>2. The 12 agents</strong></p><p>There are 12 distinct role types in the graph, although some have already appeared in the walkthrough above. One of those role types is specialist, and it has six prompt variants that share the same node code. These are evidence_gatherer, mechanism_explorer, comparator, skeptic, horizon_scanner, and domain_specialist.</p><p>The architect chooses which subset to dispatch for each question. For example, a regulatory question might use the <em>evidence gatherer, skeptic,</em> and <em>domain specialist, </em>while a benchmark question might use the <em>evidence gatherer, comparator,</em> and <em>horizon scanner</em>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/930/1*_rrOYSklkeqW8-7rzVkmDA.png" /></figure><p>For those who may not have heard of this term, a <strong>ReAct loop</strong> is just an LLM in a small loop where, at each step, the model can either think out loud, call a tool (here it’s Tavily search or extract), or decide it’s done. Think of each specialist as a junior analyst with a stack of browser tabs and a deadline.</p><p>It&#39;s worth knowing that the six <em>specialists </em>share the same node code and the same Tavily tools. It&#39;s only their prompts that make them different. Its also possible for the <em>skeptic </em>and the <em>evidence gatherer</em> to read the same article and produce different notes, because one is told to look for weak claims and the other is told to look for hard numbers.</p><p><strong>3. The Safety Properties</strong></p><p>This is what distinguishes this deep-research build from, say, a single-agent search loop. I enforced four properties by the graph topology and the code and not by trust in the model, as follows:</p><ol><li><strong>The writer can only run after </strong><strong>citation_audit</strong> - The only edge into the writer comes from the auditor. This is checked by <a href="https://github.com/wandabwa2004/deep_research_agent/blob/dev/tests/test_graph_build.py">this</a> unit test, tests/test_graph_build.py::test_writer_only_runs_after_citation_audit.</li><li><strong>The writer can only cite audited sources</strong>—The writer receives the citation registry from citation_audit and can only use source IDs from that registry. This prevents unaudited sources from appearing in the final report.</li><li><strong>The refiner cannot add new facts</strong>—The refiner only receives the draft. It does not see the evidence, registry, or brief. That limits it to improving tone, flow, and formatting.</li><li><strong>The iteration cap is enforced in code</strong>—The model does not decide when the loop stops. Once the run hits the iteration cap, the router sends the flow to citation_audit, even if gap_analyzer still says more research is needed.</li></ol><p>The relevant routing snippet is as follows:</p><pre>def _route_after_gap(state: ResearchState) -&gt; list[Send] | str:<br>    iterations = state.get(&quot;iterations&quot;, 0)<br>    gap = state.get(&quot;gap_analysis&quot;)<br><br>    #Cap is enforced in code, not in the model&#39;s judgement.<br>    if iterations &gt;= MAX_DEEP_ITERATIONS:<br>        return &quot;citation_audit&quot;<br><br>    if gap and gap.status == &quot;insufficient&quot; and gap.follow_ups:<br>        return [Send(&quot;specialist&quot;, task) for task in gap.follow_ups]<br><br>    return &quot;citation_audit&quot;</pre><p>That single if iterations ≥MAX_DEEP_ITERATIONS line is the structural guarantee that the agent terminates. Without it, a gap_analyzerthat keeps returning “insufficient” would loop forever as each cycle fires off six fresh Tavily searches, six specialist LLM calls, and a synthesis pass. Such an endless loop can easily cost hundreds of dollars before one notices.</p><p><strong>4. Ctrl+C “Magic”</strong></p><p>Deep research runs can take a few minutes. That gets annoying when you are testing prompts, so I added two parameters to the CLI to help: — max-timebudget and a Ctrl-C handler.</p><p>Both routes call finalize_from_partial(state)in src/deep_research/finalize.py. This function looks at how far the graph got and produces the best useful report it can from the available state.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7I7ebW2DKpqfzRjHs2cGwA.png" /></figure><p>The function uses the richest output available in the current state. A completed report is returned as it is. A draft goes through the refiner. Specialist notes are turned into a structured brief, audited, written into a report, and refined. A shallow answer gets audited and finalised. If there is not enough useful state to work with, the CLI returns a short interrupted placeholder.</p><p>The main caveat is that the time check happens between nodes, so a long specialist call can still run past the limit. In normal use, though, Ctrl+C gives you a cited partial report instead of losing the whole run.</p><p><strong>5. Running it</strong></p><p>Running the agent is simple once you’ve set it up following the instructions <a href="https://github.com/wandabwa2004/deep_research_agent/blob/dev/README.md">here</a>.</p><pre># CLI<br>deep-research &quot;What is the state of fusion energy commercialization in 2026?&quot;<br>deep-research &quot;Compare LangGraph vs CrewAI vs AutoGen&quot; --out report.md<br><br># Force a path, bypassing the classifier<br>deep-research &quot;What year was LangGraph released?&quot; --path shallow<br>deep-research &quot;Compare A vs B vs C&quot;               --path deep<br><br># Early exit, get a report from partial state instead of waiting it out<br>deep-research &quot;...&quot; --max-time 180   # soft 3-minute budget<br>deep-research &quot;...&quot; --max-iter 1     # one cycle, no follow-ups<br><br># Or just press Ctrl-C, the CLI catches it and finalizes what&#39;s in state.</pre><p>For the Safaricom-analyst scenario at the top of the article, the command would be something like this:</p><pre>deep-research &quot;How are major African telcos deploying agentic AI in customer support in 2025-2026?&quot; --out brief.md</pre><p>The output is a cited markdown brief with claims that can be traced back to source URLs.</p><p>The default setup uses heavier and more expensive models for the writer and specialist roles. When you are still testing, you can reduce cost by switching those roles to cheaper models through environment variables.</p><pre>WRITER_MODEL=claude-sonnet-4-6 #can work with any model <br>SPECIALIST_MODEL=claude-haiku-4-5-20251001 #can work with any model</pre><p>The provider is automatically detected from the model name. For example, models starting with claude will use the Anthropic provider, while OpenAI model names will use OpenAI. This means you can mix OpenAI and Anthropic models in the same run.</p><h4><strong>Known limitations and what comes next</strong></h4><p>A few things to be aware of with this solution:</p><ul><li>A single deep research run can send 15 or more search queries and use several expensive model calls. The default setup is powerful, but it is not cheap.</li><li>The citation audit is only a first pass. It judges source quality using URL signals, so high quality means the source looks authoritative, but this does not mean the source has been fully verified.</li><li>Long runs are not resumable by default. If you need resumability, then it&#39;s better to add something like an SqliteSaver checkpointer inside graph.py. I may cover this in a later article.</li><li>The iteration cap is currently global. If one section has gaps, all follow up specialists can still run in parallel. That works, but it is wasteful. A smarter version would only rerun the specialists responsible for weak sections.</li></ul><p>That’s it for now. The starting point for this build was based on the analogy that a single LLM in a search loop is not deep research.</p><p>From there, I built a graph that routes the question, scouts the landscape, plans the work, dispatches specialists, synthesises notes, audits citations, writes the report, and refines the final output. The safety rules are built into the graph rather than left to model behaviour.</p><p>The result is a research agent that can take a question, dispatch up to six specialists, audit citations before writing, and return a cited markdown brief in a few minutes. This could be used by anyone in search of a detailed report in any area, including very niche ones. As I mentioned at the start, outputs from this system and related agents are not ground truth and should not be treated as ground truth. However, it should give you a stronger scaffold than a single agent with a search tool attached.</p><p>More importantly, it is something you can adapt. You can change the question, rewrite the specialist prompts, swap model providers, and tune the depth of research depending on the problem.</p><p>In Part 2, I want to look at three improvements.</p><ul><li>Targeted follow-up routing, so only specialists with incomplete sections run again.</li><li>Checkpointer backed resumability for longer research runs.</li><li>A proper evaluation pass using a small set of test questions.</li></ul><p>As always, all the code is open source <a href="https://github.com/wandabwa2004/deep_research_agent">HERE</a>. Feel free to clone it and adapt it for your own use case.</p><p>If you found this useful, please clap, leave a comment, or share it with someone working on agentic AI, RAG, or LLM systems that make multi-step decisions. You can find my other articles on my <a href="https://hermanwandabwa.medium.com/">profile</a>, and I am always happy to connect on <a href="https://www.linkedin.com/in/wandabwaherman/">LinkedIn</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=203547b5fb12" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science-collective/building-a-multi-agent-deep-research-agent-with-langgraph-203547b5fb12">Building a Multi-Agent Deep Research Agent with LangGraph</a> was originally published in <a href="https://medium.com/data-science-collective">Data Science Collective</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Predicting Daily Patient Volume for a Melbourne Urgent Care Clinic]]></title>
            <link>https://medium.com/data-science-collective/predicting-daily-patient-volume-for-a-melbourne-urgent-care-clinic-d4ca30007991?source=rss-58e995b6d0e3------2</link>
            <guid isPermaLink="false">https://medium.com/p/d4ca30007991</guid>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[time-series-forecasting]]></category>
            <category><![CDATA[healthcare]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Herman Wandabwa]]></dc:creator>
            <pubDate>Sun, 10 May 2026 21:35:11 GMT</pubDate>
            <atom:updated>2026-05-22T02:14:25.346Z</atom:updated>
            <content:encoded><![CDATA[<h4><strong>How I built an XGBoost forecasting pipeline to turn patient demand into clinician rostering decisions</strong></h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jHP1e7eUombb6q3O--jmBA.png" /><figcaption><em>Image generated by the author</em></figcaption></figure><p><strong>Author’s note:</strong> <em>This article uses independently created code and synthetic data for illustration. The methods described are standard forecasting and machine learning techniques commonly used across many application areas.</em></p><p><em>Stuck behind a firewall. Read the article for free </em><a href="https://hermanwandabwa.substack.com/p/predicting-daily-patient-volume-for-a-melbourne-urgent-care-clinic-d4ca30007991?triedRedirect=true"><strong>HERE</strong></a></p><p>Running a walk-in urgent care clinic comes with a recurring operational headache of figuring out how many clinicians to roster on any given day. When they are too few, then chances are that the waiting room backs up and patients drift to the next nearest Emergency Department (ED). On the contrary, then you’re paying for doctor hours that you don’t need and vice versa. Most clinics base their patient numbers on a mix of last year&#39;s numbers, gut feel, and a bit of guesswork, especially around flu season. That’s the problem I wanted to tackle in this article, where I turn urgent care demand into something that can actually be forecasted, day-by-day, with a confidence interval that a rostering manager can plan against.</p><p>Forecasting urgent care demand is not the same as forecasting foot traffic at a shopping center. For example, if a mall misses the mark by 50 customers on a Tuesday, there is a high chance that nothing much will happen. Maybe the food court will just be overstaffed or the budget takes a small hit. However, in urgent care, the cost of being wrong is heavier. Understaffing means longer waits for people who may be genuinely unwell. On the contrary, overstaffing creates the opposite problem where it burns money that thin-margin clinics might not have. So this isn’t just a neat forecasting exercise but an operational problem with real consequences.</p><p>The clinic I’m building this around is a walk-in urgent care in Melbourne, and the city, as always, is a good fit for the exercise, and hey, I live here too. Melbourne’s health demand drivers are unusually rich, from Victorian public holidays (where EDs typically receive higher numbers as GPs will not be at work) to the Southern Hemisphere winter flu season. The city also experiences the October–November hay fever window and odd freak events like the <a href="https://knowledge.aidr.org.au/resources/storm-thunderstorm-asthma-victoria/">2016 thunderstorm-asthma</a> that overwhelmed emergency services across the city.</p><p>I’ve split the article into two parts. Part 1 (this one) <em>covers synthetic data generation, feature engineering, model training,</em> and <em>evaluation</em>. Part 2 will cover deployment with a FastAPI backend serving predictions over REST, a React + Tailwind dashboard for the rostering team, and a Supabase layer for logging predictions against real outcomes. As always, all code is open-sourced <a href="https://github.com/wandabwa2004/urgent_care_forecast/tree/main/notebooks">HERE</a>. Clone it and adapt for your own clinic or venue.</p><h4><strong>1. The Problem and Why Prediction Matters</strong></h4><p>Walk-in urgent care clinics sit awkwardly between general practice and emergency departments. They handle the complaints that can’t wait for a three-day GP appointment but don’t quite need an ambulance, i.e., sprains, cuts, chest infections, children’s fevers, minor fractures, post-injury follow-ups, etc. This makes their demand erratic. For example, on a typical Tuesday they might see 90 patients, whereas numbers can hit 200 on Easter Monday, half of them being people whose regular GP was closed for the holiday.</p><p>A properly thought-through predictive model quantifies this upfront. It should be able to output a number, a confidence interval, and a staffing tier that the rostering manager can actually rely and act on. With such a model in place, the question shifts from “<em>How many patients will walk through the door?”</em> to “<em>Do we need the Low, Medium, or High clinician roster today?”</em> Let&#39;s walk through what I built and how I did it.</p><h4><strong>2. Dataset Design &amp; Simulation</strong></h4><p>I didn’t have access to real clinic records (and wouldn’t publish them if I did). So I generated a synthetic dataset that reflects realistic urgent care patterns in Melbourne. This is useful not just because of privacy but because it lets you control the signal-to-noise ratio and demonstrate the methodology cleanly before the approach gets applied to messy real-world data.</p><p>The dataset spans January 2023 to December 2025, giving 1,096 daily records.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*MozAmkfQmRp3pYNssMCqHw.png" /><figcaption>Daily Patient Volume over Time</figcaption></figure><p>The light line shows raw daily patient counts, while the red line shows the 30-day moving average. The smoothed line makes the synthetic demand pattern easier to read: flu-driven winter peaks, a softer spring rise from hay fever, quieter summer months, and occasional spikes from thunderstorm asthma events or heatwaves.</p><p>Each record captures the following:</p><ul><li><strong>Temporal features</strong>: day of week, month, quarter, week of year</li><li><strong>Weather</strong>: temperature (Melbourne-specific ranges), precipitation (mm), relative humidity, weather type (Sunny / Partly Cloudy / Cloudy / Rainy)</li><li><strong>Calendar events</strong>: Victorian public holidays, Victorian school holidays, and crucially, the <em>day after</em> a public holiday</li><li>The feature set also captures key local health and weather drivers: winter flu season with a July peak, hay fever season from October to November, rare thunderstorm-asthma events, and temperature-extreme flags for days above 35°C or below 5°C.</li></ul><p>It&#39;s worth pointing out that several of these drivers have the <em>opposite effect </em>of what they’d be in other contexts. For example, rain might <em>reduce </em>visits to entertainment joints but <em>slightly increases</em> urgent care demand (slips, falls, and people who can’t put off the chest infection any longer). I spent some good time on these sign choices because getting them wrong would produce a plausible-looking dataset that models can be trained on, but the learned relationships would be backwards. This is exactly how a seemingly validated model ends up giving bad rostering advice in production.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3D5pV3nAUGiQftC6GtGXLw.png" /><figcaption>Weather and Illness Driver Impact on Patient Volume</figcaption></figure><p><em>Left: mean daily patients by weather type. Rainy days run slightly above sunny days. Middle: patients vs. temperature. Right: pollen index vs. patients, flat until index hits ~7, then climbs sharply (the hayfever-to-asthma pathway).</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*43BTzblksSneo769lSoE5A.png" /><figcaption>Temporal Patterns in Patient Volume</figcaption></figure><p>Above are four views of the same temporal story.</p><p>The plots suggest a strong calendar effect. Mondays are the busiest day, sitting about 15% above midweek levels. July and August are the peak months, which lines up with winter pressure, and Q3 has the highest quarterly demand. In the heatmap, Monday in July is the busiest day-month combination.</p><p>The patient count itself is generated using <a href="https://www.sciencedirect.com/topics/computer-science/multiplicative-factor">multiplicative factors</a> applied to a base of 100 patients/day:</p><pre># Day of week effect (Monday is peak due to weekend GP backlog)<br>day_multiplier = {0: 1.15, 1: 1.00, 2: 0.95, 3: 0.95, 4: 1.00, 5: 0.85, 6: 0.75}<br>df[&#39;patients&#39;] *= df[&#39;day_of_week&#39;].map(day_multiplier)<br># Seasonal effect (Southern Hemisphere - winter = flu season)<br>season_multiplier = {<br>1: 0.85, 2: 0.90, 3: 0.95, 4: 1.00, 5: 1.10, 6: 1.30,<br>7: 1.45, 8: 1.35, 9: 1.15, 10: 1.10, 11: 1.05, 12: 0.90<br>}<br>df[&#39;patients&#39;] *= df[&#39;month&#39;].map(season_multiplier)<br># Calendar effects - NOTE: public holidays INCREASE demand here (GPs closed)<br>df.loc[df[&#39;is_public_holiday&#39;] == 1, &#39;patients&#39;] *= 1.40<br>df.loc[df[&#39;is_day_after_public_holiday&#39;] == 1, &#39;patients&#39;] *= 1.22<br>df.loc[df[&#39;is_school_holiday&#39;] == 1, &#39;patients&#39;] *= 1.15<br># Epidemiological drivers<br>df.loc[df[&#39;is_flu_peak&#39;] == 1, &#39;patients&#39;] *= 1.50<br>df.loc[df[&#39;is_hayfever_season&#39;] == 1, &#39;patients&#39;] *= 1.05<br>df.loc[df[&#39;is_thunderstorm_asthma&#39;] == 1, &#39;patients&#39;] *= 2.30<br># Weather impact (different sign from the museum case)<br>df.loc[df[&#39;weather_type&#39;] == &#39;Rainy&#39;, &#39;weather_factor&#39;] = 1.08<br>df.loc[df[&#39;temp_extreme_hot&#39;] == 1, &#39;weather_factor&#39;] *= 1.25<br>df.loc[df[&#39;temp_extreme_cold&#39;] == 1, &#39;weather_factor&#39;] *= 1.20<br>df[&#39;patients&#39;] *= df[&#39;weather_factor&#39;]</pre><p>Each factor has a measurable and verifiable uplift in the final data as quantified in the chart below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ZE_m3Ahp7CokNJIgcCzRBg.png" /><figcaption>Impact of Key Factors on Patient Volume</figcaption></figure><p>Each panel compares mean patient count with and without the factor active. Flu peak (+54%), public holidays (+40%), and day-after-public-holiday (+22%) dominate. Hayfever season has a small but consistent lift (+5%). By the way, not every hay fever day leads to asthma, but across the full two-month window, the effect stacks up.</p><p>To make the synthetic demand less perfectly behaved, I added Gaussian noise with a standard deviation equal to 12% of expected daily traffic, a 2% outlier rate where traffic spikes to 1.5–2.5× normal levels, and a small daily growth trend of about 0.01% to reflect a slowly expanding catchment population. The final values are clipped between 30 and 500 patients/day, which is a realistic operating range for a single walk-in clinic. Remember, this is simulated data and should NOT be used as a replacement for real clinic records.</p><p>The resulting distribution has the right-skewed shape typical of event-driven demand. Most days cluster around 100–150 patients, with a long tail of flu-season weekdays and public-holiday spikes that push above 250.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wgrRzFnXgqZIvF8pnSxxVg.png" /><figcaption>Right-skewed distribution of raw patient counts</figcaption></figure><p>The histogram shows skewness ≈ 1.14, with the mean sitting above the median and a visible right tail of high-volume days.</p><h4><strong>3. Feature Engineering</strong></h4><p>I’ve made the same point in previous articles and will keep making it: raw data gets you baseline predictions while feature-engineered data gets you good ones. Starting from 31 raw columns, I engineered 87 features grouped into eight categories:</p><pre>Feature groups:<br>  Temporal    :  17<br>  Calendar    :   3<br>  Season      :   4<br>  Weather     :  11<br>  Epidemio    :   5<br>  Interaction :   6<br>  Lag         :  14<br>  Rolling     :  27<br>  Total       :  87</pre><p>Most of the techniques here are around cyclical encoding, lag features, and rolling stats, so I’ll briefly touch on those and linger on the ones that are specific to the urgent care domain.</p><p><strong>a) Cyclical Encoding — <em>Why Day 0 and Day 6 Should Be “Close”</em></strong></p><p>Here’s the subtle problem with temporal features again. If you feed <em>day_of_week</em> directly into a model as values 0 to 6, then the model will treat Monday (0) and Sunday (6) as being far apart as much as they’re adjacent. The same applies to months. December (12) and January (1) are close and not opposites.</p><p>Cyclical encoding using <em>sin/cos </em>transforms is the best workaround for this:</p><pre>df[&#39;dow_sin&#39;] = np.sin(2 * np.pi * df[&#39;day_of_week&#39;]/7)<br>df[&#39;dow_cos&#39;] = np.cos(2 * np.pi * df[&#39;day_of_week&#39;]/7)<br>df[&#39;month_sin&#39;] = np.sin(2 * np.pi * (df[&#39;month&#39;]-1)/12)<br>df[&#39;month_cos&#39;] = np.cos(2 * np.pi * (df[&#39;month&#39;]-1)/12)<br>df[&#39;doy&#39;] = df[&#39;date&#39;].dt.dayofyear<br>df[&#39;doy_sin&#39;] = np.sin(2 * np.pi * df[&#39;doy&#39;]/365)<br>df[&#39;doy_cos&#39;] = np.cos(2 * np.pi * df[&#39;doy&#39;]/365)</pre><p>The best way to think around this is to assume that you’re placing each day on the face of a clock. Monday and Sunday end up right next to each other, and December sits beside January. Both <em>sin </em>and <em>cos </em>components are needed because <em>sin</em> alone can’t distinguish between two days that map to the same y-coordinate (e.g., Tuesday and Friday might have the same <em>sin </em>value but different <em>cos </em>values).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*voK-_49j1FPp-wfD6JP05Q.png" /><figcaption>Cyclical Encoding</figcaption></figure><p>On the left, days of the week are mapped onto a circle via <em>sin/cos</em>. On the right, months on the same circular encoding. Both components are needed to uniquely identify each position on the circle.</p><p>I applied this to<em> day-of-week, month, day-of-year, </em>and <em>week-of-year</em>.</p><p><strong>b) Lag Features—&quot;Yesterday<em> Predicts Tomorrow”</em></strong></p><p>Recent history turned out to be one of the strongest predictive signals in this dataset. If 150 patients walked in yesterday, today’s number is more likely to sit near 150 than near 80. I replicated this in a way that intervals correspond to real rhythms: short-term momentum, weekly cycle, fortnightly, monthly, and seasonal.</p><pre>for lag in [1, 2, 3, 7, 14, 21, 28, 60, 90]:<br>    df[f&#39;patients_lag_{lag}&#39;] = df[&#39;patients&#39;].shift(lag)<br><br># Same-weekday average over the last 4 occurrences<br>df[&#39;mean_last_4_same_dow&#39;] = df[[&#39;lag_7_same_dow&#39;,&#39;lag_14_same_dow&#39;,<br>                                 &#39;lag_21_same_dow&#39;,&#39;lag_28_same_dow&#39;]].mean(axis=1)</pre><p>Why these specific intervals, if you may ask? Lags 1–3 capture short-term momentum. Lag 7 captures the same-day-last-week pattern (which matters enormously here, since Mondays and Sundays behave very differently). Lags 14, 21, and 28 capture fortnightly and monthly rhythms. Lags 60 and 90 capture seasonal trends. The same-weekday average is particularly powerful in urgent care because it answers the clinically grounded question, &quot;<em>How many patients typically walk in on a Monday?&quot;</em> which is a very different question from “<em>How many typically walk in on a Sunday?&quot;</em> Mondays in this dataset run ~50% above Sundays.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*83U14uyBdmEJD_LjM-EsIg.png" /><figcaption>Autocorrelation Function</figcaption></figure><p>The ACF shows significant autocorrelation at multiples of 7 days (the weekly cycle) and a slower decay that persists out past 60 days (the seasonal structure). The PACF highlights lags 1 and 7 as carrying the most independent predictive information, exactly the lags prioritised in the feature set.</p><p><strong>c) Rolling Window Statistics</strong></p><p>Beyond point lags, I computed rolling means, standard deviations, max, and min across multiple windows, plus exponentially weighted moving averages:</p><pre>for window in [3, 7, 14, 30, 60, 90]:<br>    df[f&#39;rolling_mean_{window}d&#39;] = df[&#39;patients&#39;].shift(1).rolling(window).mean()<br>    df[f&#39;rolling_std_{window}d&#39;]  = df[&#39;patients&#39;].shift(1).rolling(window).std()<br>    df[f&#39;rolling_max_{window}d&#39;]  = df[&#39;patients&#39;].shift(1).rolling(window).max()<br>    df[f&#39;rolling_min_{window}d&#39;]  = df[&#39;patients&#39;].shift(1).rolling(window).min()<br><br>#  Exponentially Weighted Moving Average(EWMA) — more weight on recent days<br>for span in [7, 14, 30]:<br>    df[f&#39;ewma_{span}d&#39;] = df[&#39;patients&#39;].shift(1).ewm(span=span, adjust=False).mean()</pre><p>The shift(1) is critical as it prevents data leakage by ensuring that we only use information available <em>before</em> the prediction date. If you get this wrong, then you’ll have leaked future data into training, and test metrics will look amazing until the model hits production and falls apart. The EWMA variants give extra weight to recent observations, which is useful because patterns can shift gradually (think of the ramp-up at the start of flu season).</p><p><strong>d) Interaction Features</strong></p><p>In most cases, urgent care demand is driven by <em>combinations </em>and not individual factors. A <em>rainy Monday</em> during <em>flu peak</em> is a fundamentally different day from <em>a sunny Tuesday</em> in <em>February</em>, even if you break down each feature independently. I captured the most clinically plausible combinations as follows, and of course this can be extended to your liking:</p><pre>df[&#39;ph_x_monday&#39;]=df[&#39;is_public_holiday&#39;]*df[&#39;is_monday&#39;]<br>df[&#39;flu_peak_x_rainy&#39;]= df[&#39;is_flu_peak&#39;]*(df[&#39;weather_type&#39;]==&#39;Rainy&#39;).astype(int)<br>df[&#39;hayfever_x_high_pollen&#39;]= df[&#39;is_hayfever_season&#39;]*(df[&#39;pollen_index&#39;]&gt;=7).astype(int)<br>df[&#39;school_holiday_x_weekend&#39;]= df[&#39;is_school_holiday&#39;]*df[&#39;is_weekend&#39;]<br>df[&#39;extreme_cold_x_flu_season&#39;]= df[&#39;temp_extreme_cold&#39;]*df[&#39;is_flu_season&#39;]<br>df[&#39;day_after_ph_x_monday&#39;]= df[&#39;is_day_after_public_holiday&#39;]*df[&#39;is_monday&#39;]<br><br># Count of concurrent illness drivers<br>df[&#39;illness_driver_count&#39;]=(df[&#39;is_flu_season&#39;]+df[&#39;is_hayfever_season&#39;]<br>                              +df[&#39;is_day_after_public_holiday&#39;]<br>                              +df[&#39;temp_extreme_hot&#39;]+df[&#39;temp_extreme_cold&#39;])</pre><p>The illness_driver_count feature captures a simple idea, which is <em>how many illness-related factors are active on a given day</em>. For example, a day with three active drivers is fundamentally different from a day with zero, and the stacking is nearly monotonic.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/889/1*LIn6kbaMUfU1De5muuLpmw.png" /><figcaption>Average Patients by Illness Driver Count</figcaption></figure><p>The plot above shows a clear stacking effect. Days with no illness drivers average about 111 patients across 605 days, while those with one driver average about 153 patients across 475 days. Days with two concurrent drivers are rare, appearing only 16 times in the three-year window, but they average about 175 patients.</p><p><strong>e) Melbourne Seasons</strong></p><p>This matters because Melbourne’s seasonal pattern is flipped relative to Northern Hemisphere cities. December is summer, not winter, and if the model gets that wrong, its seasonal assumptions will definitelypoint in the wrong direction.</p><pre>def get_season(month):<br>    if month in [12, 1, 2]: return &#39;Summer&#39;<br>    if month in [3, 4, 5]:  return &#39;Autumn&#39;<br>    if month in [6, 7, 8]:  return &#39;Winter&#39;<br>    return &#39;Spring&#39;</pre><p>In urgent care, seasonal alignment matters because Melbourne’s winter months, June to August, map directly to flu season. If the model encodes seasonality incorrectly, then the flu-related demand signals get diluted across the wrong part of the calendar.</p><p><strong>f) Log-Transform of the Target</strong></p><p>Patient counts as in the original data are right-skewed (skewness ≈ 1.14), and log1pbrings the distribution much closer to normal, which helps gradient-based models converge and stops them from overweighting outlier days at the expense of typical ones:</p><pre>df[&#39;patients_log&#39;] = np.log1p(df[&#39;patients&#39;])</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vtdFIPWQqwoKLSydkwgZMw.png" /><figcaption>Log Transformation on Patient Counts</figcaption></figure><p>The left plot shows the raw target distribution, with a skewness of 1.14 and a clear right tail from high-volume days. After applying log1p, skewness drops to 0.30, and the distribution becomes much closer to normal. The models train on this transformed target, then predictions are converted back to patient counts using np.expm1() before being returned. I’ll revisit this in Part 2 when discussing the production API.</p><p>With all 87 features engineered, the next step was to see which ones actually carried signal against the target.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/889/1*MGuOKmDLP-cBklZzhlWTpQ.png" /><figcaption>Top 30 Features by Correlation with Patient Count</figcaption></figure><p>The rolling 7/14-day averages cluster at the top (correlations between 0.61 and about 0.69), followed by the ‘<em>rolling_mean7d</em>’.</p><p>The longest lag feature looked back 90 days, so the first 90 rows had to be removed after feature engineering. That left about 1,006 usable observations, enough for model training and a separate 90-day holdout set.</p><h4><strong>4. Model Training &amp; Comparison</strong></h4><p>With the features ready, I trained and compared four modeling approaches. Before that, I needed to get the split strategy right.</p><p><strong>Train/Test Split — Respecting Time</strong></p><p>With time series data, random train/test splits do not really work well. If you shuffle the dates, the model can learn from future patterns while being evaluated on earlier ones. That is data leakage, and it makes the metrics look better than they really are.</p><p>I kept the split chronological, where the final 90 days became the holdout test set, and everything before that was used for training.</p><pre>Train: 915 rows (2023–04–01 → 2025–10–02)<br>Test : 90 rows (2025–10–03 → 2025–12–31)</pre><p>For cross-validation, I used TimeSeriesSplit with five expanding-window folds. Each fold trained only on past data and validated on future data, keeping the evaluation aligned with how the model would be used in practice.</p><p><strong>The Models</strong></p><p>Before reaching for more sophisticated algorithms, I started with naive baselines. These are simple prediction strategies that rely on arithmetic rather than learned patterns. If a more complex model cannot beat them, then it&#39;s not earning its complexity.</p><p>I used three naive baselines, each slightly more informed than the last.</p><ul><li><strong>Global mean: This</strong> predicts every day as the training set average, which was about 130 patients. It ignores everything, including day of week, weather, flu season, and public holidays. It is the “shrug” prediction, but it anchors the lower end of the performance range.</li><li><strong>Last week’s count (lag 7)</strong>: This predicts today’s patient volume using the count from the same day last week. It captures a weekly rhythm, but nothing else. That means it cannot adapt to flu peaks, weather changes, school holidays, or public holidays.</li><li><strong>7-day moving average</strong>: This predicts today’s volume as the average of the previous seven days. It is smoother than the lag-7 baseline, but still blind to why demand is changing. It follows recent drift, but it does not understand the drivers behind it.</li></ul><p>The shared limitation of all three is that they use a single signal and can’t combine information. <em>A sunny Saturday during school holidays with a thunderstorm-asthma alert</em> is nothing like a <em>rainy Tuesday in summer</em>, and none of these baselines can tell the difference. That’s the gap ML fills.</p><p>I used three modeling algorithms for this problem.</p><ol><li><strong>Random Forest</strong> used 300 decision trees, with each tree trained on a random slice of the data and a random subset of features using max_features=&#39;sqrt&#39;. Individual trees can learn rules such as <em>&quot;If it is Monday, flu activity is high, and temperature is below 10°C, expect higher patient volume.” </em>The final prediction averages the output from all 300 trees, which smooths out the quirks of any single tree. I also set min_samples_leaf=5 to stop the trees from memorising small pockets of noise. Like the other tree-based models, Random Forest was trained on the log-transformed target.</li><li><strong>XGBoost </strong>works differently. Instead of growing trees independently, it builds them in sequence. Each new tree focuses on the errors left behind by the previous trees. In simple terms, Random Forest averages many independent trees, while XGBoost builds trees one after another, with each tree chasing the remaining residuals. I used 500 trees and a learning rate of 0.05, meaning each tree only made a small correction. That usually requires more trees, but it produces a more stable model. I trained XGBoost on log1p(patients) so that high-volume days did not dominate the loss function.</li><li><strong>Prophet </strong>(Facebook’s time series model) takes a different angle. Instead of relying on the hand-built features, it decomposes the time series into trend, seasonality, and holiday effects, then fits those components directly. I used multiplicative seasonality so the seasonal swings could scale with the baseline level of demand. I also added external regressors for public holidays, flu peak, pollen index, and weather.</li></ol><p>Here is the XGBoost training block.</p><pre>xgb_model = xgb.XGBRegressor(<br>    n_estimators=500,<br>    learning_rate=0.05,<br>    max_depth=6,<br>    subsample=0.8,<br>    colsample_bytree=0.8,<br>    min_child_weight=5,<br>    reg_alpha=0.1,<br>    reg_lambda=1.0,<br>    random_state=42,<br>    n_jobs=-1,<br>    verbosity=0,<br>)<br># Train on log-transformed target<br>xgb_model.fit(X_train, y_train_log)<br># Predictions need inverse transform<br>xgb_preds = np.expm1(xgb_model.predict(X_test))</pre><p>The code for the other two approaches can be found <a href="https://github.com/wandabwa2004/urgent_care_forecast/blob/main/notebooks/04_modeling.ipynb">HERE</a>.</p><h4><strong>Results</strong></h4><p>XGBoost produced the best results across all metrics. Its MAPE of 11.89% means that, on an average day of about 130 patients, the forecast error is roughly 15 patients. The point is not that the model predicts every day exactly, but that it provides a reliable enough signal to support staffing decisions.</p><pre>| Model                | MAE   | RMSE  | R²    | MAPE   |<br>|----------------------|-------|-------|-------|--------|<br>| XGBoost              | 17.04 | 27.69 |  0.30 | 11.89% |<br>| Prophet              | 17.98 | 28.98 |  0.23 | 12.84% |<br>| Random Forest        | 18.34 | 29.35 |  0.21 | 12.91% |<br>| Baseline (7d MA)     | 22.93 | 32.36 |  0.04 | 17.18% |<br>| Baseline (mean)      | 24.04 | 33.04 | -0.00 | 18.06% |<br>| Baseline (last week) | 24.09 | 35.87 | -0.18 | 17.22% |</pre><p>The improvement over baselines is substantial. The best baseline (7-day moving average) has an MAE of 22.93 patients, while XGBoost achieves 17.04, a 26% reduction in error. MAPE drops from 17.2% to 11.9%, where this gap is the difference between useful predictions and educated guesses.</p><p>Two things worth calling out on these numbers:</p><ul><li>The negative<strong> R² </strong>for the last-week baseline is not a bug<strong>.</strong> It just means that baseline actually performs <em>worse</em> than predicting the constant mean on this test set. Baselines aren’t guaranteed to beat constant-mean, and that’s part of why they’re useful as they calibrate expectations.</li><li>An R² of about 0.30 looks modest, but that is partly a signal-to-noise issue. Daily urgent care volumes have a lot of short-term variation that calendar, weather, and epidemiological features cannot fully explain. MAPE is more useful here because it measures the typical percentage error in the forecast. At 11.9%, it gives a clearer view of whether the model is useful for planning staff.</li></ul><p>On the test data, actuals vs predicted numbers are as below:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*oOoSo7HNtY8gbj-poGsMvQ.png" /><figcaption>Actual vs Predicted Patient Counts</figcaption></figure><p>XGBoost tracks the observed series most closely, capturing both the weekly cycle and the late-year flu ramp. Random Forest follows the overall shape but underestimates peak demand. Prophet captures the broad seasonal movement but smooths over short-term variation because it relies more on decomposed trend and seasonality than on the engineered feature set used by the tree-based models.</p><h4><strong>5. Model Evaluation &amp; SHAP Explainability</strong></h4><p>Raw metrics tell you <em>how well</em> the model performs while evaluation tells you <em>where it struggles and why</em>, and that’s the part that actually matters when you’re deciding whether to trust it.</p><p><strong>Residual Analysis</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*TtB7qRfWZRnXfUSnAq5eRg.png" /><figcaption>Residual Analysis</figcaption></figure><p>Overall, the residuals look sensible. They are roughly centred around zero, with no obvious time-based drift across the holdout period. The predicted-versus-actual plot tracks the diagonal reasonably well, although the model still under-predicts some of the highest-volume days. Most percentage errors fall within about ±20%, with a slight negative skew.</p><p>The model has a small negative bias of about 6.5 patients per day, meaning it tends to under-predict demand slightly, which is fundamental in rostering. Understaffing is riskier than modest overstaffing, so the production pipeline should not treat the point forecast as the final staffing number. A safer approach is to roster closer to the upper bound of the 80% prediction interval.</p><p><strong>SHapley Additive exPlanations (SHAP) Explainability</strong></p><p>I used SHAP to understand what was driving the model’s predictions. This mattered for two reasons. First, it helped check whether the model was relying on clinically sensible signals rather than spurious patterns. Second, it gave me a concrete way to explain the forecasts to clinical directors and rostering managers when they ask why the model expects demand to rise or fall.</p><pre>explainer = shap.TreeExplainer(model)<br>shap_values = explainer.shap_values(X_shap)<br>shap.summary_plot(shap_values, X_shap, plot_type=&#39;dot&#39;, max_display=20)</pre><p>The top features line up with what a clinician would expect:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/939/1*otHwKyAlyFsdTSP1Ffhahw.png" /><figcaption>SHAP Values</figcaption></figure><p>The figure above is a SHAP beeswarm plot, showing how each feature pushes predictions up or down across the holdout set. Each dot represents one observation for that feature, coloured by the feature value, with red indicating higher values and blue indicating lower values. is_flu_peak, illness_driver_count, and patients_lag_7 dominate the top of the ranking, suggesting the model relies heavily on epidemiological pressure and recent demand. Lag and rolling-window features also contribute meaningfully, but they sit further down the ranking.</p><p>I also generated a waterfall plot for the single worst prediction day. This is the kind of plot you want ready when a clinical director asks, <em>“Why did the model get yesterday so wrong?”</em> It breaks the forecast into feature-level contributions, showing which signals pushed the prediction up or down and by how much.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/802/1*owkvx9i-ENDfAAhiesDQ5w.png" /><figcaption>Single prediction explanation for the worst prediction day</figcaption></figure><p>That matters because not every bad prediction has the same cause. Sometimes the model misses because something happened outside the feature set, such as a nearby GP closure, a road accident, or another local disruption. Other times, the model may be facing a rare combination of feature values it has not seen often enough during training. The waterfall plot helps separate those two cases.</p><p><strong>Prediction Intervals</strong></p><p>A point prediction alone isn’t enough. Rostering managers need to know the <em>range</em> of uncertainty, which is the reason I computed bootstrap prediction intervals using training residuals:</p><pre>np.random.seed(42)<br>n_bootstrap = 500<br>base_pred = test[&#39;pred&#39;].values<br>bootstrap_preds = np.array([<br>    base_pred + np.random.choice(cv_resid, size=len(base_pred), replace=True)<br>    for _ in range(n_bootstrap)<br>])<br>lower_80 = np.percentile(bootstrap_preds, 10, axis=0)<br>upper_80 = np.percentile(bootstrap_preds, 90, axis=0)<br>lower_95 = np.percentile(bootstrap_preds, 2.5, axis=0)<br>upper_95 = np.percentile(bootstrap_preds, 97.5, axis=0)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*P7vZNI8sXbI5bcjCRy5Y1w.png" /><figcaption>Predictions with Bootstrap Intervals</figcaption></figure><p>The red line shows the model’s prediction, while the black line shows the actual patient count. The darker band represents the 80% prediction interval, and the lighter band represents the 95% interval. The actual values fall within these bands at roughly the expected rates, which suggests the uncertainty estimates are reasonably well calibrated.</p><p>For a rostering manager, this is more useful than a single number. Instead of saying, <em>“We predict 145 patients tomorrow,”</em> the model can say,<em> “We predict 145 patients tomorrow, with an 80% prediction interval of 113 to 177.”</em> That gives the manager a planning range rather than a false sense of precision.</p><p>There is one caveat, though. This bootstrap method assumes the residuals are drawn from the same error distribution across all conditions, which is unlikely to be true. Public holidays, flu peaks, and unusual demand days probably have wider errors than quiet weekdays. A more refined version would estimate condition-specific intervals, but for a first deployment, this gives a useful and honest uncertainty range.</p><h4><strong>6. Stakeholders Engagement</strong></h4><p>From experience, MAE and RMSE do not mean much to clinical directors or rostering managers on their own. They are useful modeling metrics, but they do not directly answer the operational question:<em> How many people should we roster tomorrow?</em></p><p>So I framed the model output in two more practical ways.</p><p><strong>Staffing Tier Classification</strong></p><p>Instead of only returning a raw patient-count forecast, the model can also classify each day into a staffing tier that maps directly to roster planning. The exact thresholds should be defined by the clinical operations team, but a simple example could look like this:</p><pre><strong>Low </strong>(&lt;80 patients): 2 doctors, 3 nurses, standard roster <br><strong>Medium </strong>(80–150): 3 doctors, 4 nurses, extra triage nurse <br><strong>High </strong>(&gt;150): 4+ doctors, surge protocol, ED overflow coordination</pre><p>This makes the forecast easier to act on. Each tier has a concrete staffing implication, a cost implication, and a clinical risk implication. The rostering manager no longer has to interpret patient-count deltas in isolation. They can use the model output as a planning signal tied directly to staffing decisions.</p><p><strong>Operational Impact</strong></p><p>The real value is not just lower MAE. It is better roster decisions. If the model gets the demand tier right, the clinic avoids wasting money on unnecessary staff during quiet days and reduces the risk of understaffing during demand spikes.</p><p>That is the version a clinical director can use. Instead of saying, <em>“The model has an MAE of 17 patients,”</em> the better message is, &quot;We<em> can identify the correct staffing tier about 8 out of 10 days, and we can give you an 80% prediction interval before the roster is locked in.”</em></p><p>That’s Part 1 done, and I’m sorry to admit that I ran it long. I promise (just like I’ve done in the past), Part 2 will be way shorter and more intuitive. We’ve gone from breaking down the business problem through data generation, feature engineering, model training, evaluation, and stakeholder framing. The XGBoost model with 87 engineered features hits a MAPE of 11.9% and produces well-calibrated prediction intervals that can be used for rostering decisions. I’ve basically provided for you a framework that you can plug your code and data into and forecast results.</p><p>In Part 2, I’ll take this trained model and deploy it as a full-stack application: a <em>FastAPI</em> backend serving predictions through a <em>REST API</em>, a <em>React + Tailwind</em> dashboard for stakeholders, and possibly a Supabase database for logging predictions against actuals so we can backfill real outcomes and monitor drift. I’ll also cover deployment considerations and what I’d do differently with real clinic data. If you’d like an article on optimising predicted numbers to actual staffing levels, then let me know.</p><p>As always, all code is open-sourced <a href="https://github.com/wandabwa2004/urgent_care_forecast/tree/main/notebooks">HERE</a>. Feel free to clone it and adapt it for your own use case. If you found this useful, please clap, leave a comment, or share it with someone working on forecasting or operational AI. You can always find my other articles on my <a href="https://medium.com/@hermanwandabwa">profile</a>, and I’m always happy to connect via <a href="https://www.linkedin.com/in/wandabwaherman/">LinkedIn</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d4ca30007991" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science-collective/predicting-daily-patient-volume-for-a-melbourne-urgent-care-clinic-d4ca30007991">Predicting Daily Patient Volume for a Melbourne Urgent Care Clinic</a> was originally published in <a href="https://medium.com/data-science-collective">Data Science Collective</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[From Theory to Code — Building a SIM Replacement Agent in LangGraph (Part 2)]]></title>
            <link>https://medium.com/data-science-collective/part-2-from-theory-to-code-building-a-sim-swap-agent-in-langgraph-7290f5699023?source=rss-58e995b6d0e3------2</link>
            <guid isPermaLink="false">https://medium.com/p/7290f5699023</guid>
            <category><![CDATA[generative-ai-use-cases]]></category>
            <category><![CDATA[ai-agent]]></category>
            <category><![CDATA[llm-applications]]></category>
            <category><![CDATA[agentic-ai]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Herman Wandabwa]]></dc:creator>
            <pubDate>Mon, 27 Apr 2026 21:35:40 GMT</pubDate>
            <atom:updated>2026-05-07T09:56:05.053Z</atom:updated>
            <content:encoded><![CDATA[<h4>How State, Tools, Nodes, and Edges Power an AI Agent</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ouRPz9aNqoR9P80gvSizRQ.png" /><figcaption>Image generated by the author</figcaption></figure><p><em>Stuck behind a firewall? Read the article for FREE </em><a href="https://hermanwandabwa.medium.com/7290f5699023?source=friends_link&amp;sk=69220bfdbde4ff83250de1435b40f495"><strong><em>here</em></strong></a><em>.</em></p><p>In <a href="https://medium.com/data-science-collective/what-agentic-ai-actually-is-a-sim-replacement-use-case-part-1-b9f1672d68b5">Part 1</a>, I covered what agentic AI actually means and why a SIM card replacement workflow is a good way to test whether an application is genuinely agentic. I introduced the four building blocks that every agent should ideally have:<strong><em> state, tools, nodes,</em></strong> and <strong><em>edges</em></strong>. If you haven’t read the <a href="https://medium.com/data-science-collective/what-agentic-ai-actually-is-a-sim-replacement-use-case-part-1-b9f1672d68b5">first one</a> yet, I’d suggest starting there because this article builds directly on it.</p><p>The goal in this part is to translate the theoretical concepts in Part 1 to a practical and working agentic prototype. I’ll stick to <a href="https://www.langchain.com/langgraph">LangGraph </a>, an orchestration framework to build, manage and deploy stateful AI agents. The good thing with LangGraph is that it allows for better design and customisation of sub-agents and related artefacts. My hope is that you’ll have a good and practical grasp of agentic AI through a working agentic application that can run the full SIM swap flow for a telcom company. This ranges from identity verification all the way to customer notification about the final swap. The representative flowchart, just like what is in Part 1 is below:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*fwsawTk6wxb2w9oa.png" /><figcaption>SIM card replacement process</figcaption></figure><p>As always, all the code is open-source and can be found <a href="https://github.com/wandabwa2004/safaricom_agentic_ai">here</a>, so you can clone and run it yourself.</p><p>So let’s get into the interesting bits : -</p><h4><strong>1. The State Object</strong></h4><p>Remember the notebook analogy from <a href="https://medium.com/data-science-collective/what-agentic-ai-actually-is-a-sim-replacement-use-case-part-1-b9f1672d68b5">Part 1</a>? <strong><em>State </em></strong>is what the agent carries from step to step which is everything it knows at any given moment. In LangGraph, state can be defined as a Python TypedDict.</p><p>The choice of TypedDictover a class is deliberate as it serialises cleanly, it’s straightforward to checkpoint and restore, and LangGraph can easily track incremental state changes without any custom logic. Here’s the full state for the SIM swap agent:</p><pre>from typing import Annotated, TypedDict<br>from langgraph.graph.message import add_messages<br><br>class SimSwapState(TypedDict):<br>    # Subscriber context<br>    subscriber_id: str<br>    subscriber_name: str<br>    msisdn: str<br>    alternate_msisdn: str<br>    id_number: str<br><br>    # Request context<br>    request_id: str<br>    request_reason: str  # &quot;lost&quot; | &quot;stolen&quot; | &quot;damaged&quot; | &quot;upgrade&quot;<br>    request_timestamp: str<br>    channel: str    # &quot;shop&quot; | &quot;call_centre&quot; | &quot;ussd&quot;<br><br>    # Identity verification - Knowledge-Based Authentication(KBA)<br>    id_verification_status: str  # &quot;pending&quot; | &quot;passed&quot; | &quot;failed&quot;<br>    id_verification_attempts: int<br>    id_max_retries: int<br>    kba_questions_correct: int<br><br>    # Line status checks<br>    line_status: str  # &quot;active&quot; | &quot;suspended&quot; | &quot;barred&quot; | &quot;churned&quot; | &quot;not_found&quot;<br>    fraud_flag: bool<br>    pending_swap: bool<br>    line_check_passed: bool<br><br>    # New SIM capture<br>    new_iccid: str<br>    new_imsi: str<br>    sim_validation_status: str # &quot;pending&quot; | &quot;valid&quot; | &quot;invalid&quot;<br>    sim_validation_attempts: int<br>    sim_max_retries: int<br>    sim_validation_errors: list[str]<br><br>    # Current (to-be-blocked) SIM<br>    old_iccid: str<br>    old_imsi: str<br>    old_sim_blocked: bool<br><br>    # Provisioning<br>    new_sim_provisioned: bool<br><br>    # M-PESA<br>    mpesa_registered: bool<br>    mpesa_balance: float<br>    mpesa_reactivated: bool<br>    mpesa_temporary_pin_sent: bool<br><br>    # Network<br>    network_activated: bool<br><br>    # Notifications<br>    confirmation_sms_sent: bool<br><br>    # Process metadata<br>    current_step: str<br>    audit_log: list[dict]<br>    error_messages: list[str]<br>    process_status: str  # &quot;in_progress&quot; | &quot;completed&quot; | &quot;rejected&quot; | &quot;halted&quot;<br><br>    # Conversation<br>    messages: Annotated[list, add_messages]</pre><p>That looks like a lot of fields, but if you read it in groups like in the above code, then it will be easy to make sense of. The fields in the <em>subscriber context </em>are the details that a customer would need to have before they are served. Their MSISDN (phone number), ID number, and an alternate number to send the final confirmation SMS to etc. Thereafter, the <em>request context</em> has fields that capture the reason for the interaction with the system.</p><p>From there, every block maps directly to a stage in the workflow. KBA tracking holds the attempt counts, while the line status checks whether fraud or a pending swap was found. The SIM capture block tracks validation retries with pipeline fields (blocked, provisioned, M-PESA, network) flipping from <em>False</em> to <em>True</em> as each step completes. The metadata block at the very end gives the audit log, error accumulation, and an overall process status.</p><p>Most state fields follow a simple last-write-wins rule. If a node returns a new value for fraud_flag, the old one is overwritten. The exception field is messages, which uses Annotated[list, add_messages] to tell LangGraph to merge new messages into the existing list rather than replace it. That&#39;s how the agent accumulates a full conversation history as it moves through the steps.</p><p>One thing worth noting is that the graph itself is <em>stateless </em>as it has no memory between runs. Basically, the state object carries everything, including every business rule in the workflow, from retry limits to fraud flags to M-PESA reactivation checks.</p><h4><strong>2. Tools</strong></h4><p>If the state object is the notebook, the tools are what the agent uses to fill it in. In this implementation, each tool gets its own service classand they are all defined <a href="https://github.com/wandabwa2004/safaricom_agentic_ai/tree/dev/tools">here</a>:</p><pre>|<strong> Tool                          | What it simulates </strong><br>|-------------------------------|---------------------------------------------|<br>| IdentityVerificationService   | KBA challenge: ID number, frequently called numbers, recent M-PESA transaction, last top-up amount <br>| LineStatusService             | HLR lookup: line status (active/suspended/barred) + fraud flag + pending swap check <br>| SimValidationService          | Validates ICCID format (19–20 digits, `89254…` Kenya prefix) and IMSI uniqueness <br>| SimManagementService          | Blocks the old SIM, provisions the new one (binds IMSI to MSISDN in the HLR) <br>| MpesaService                  | Restores the M-PESA wallet on the new SIM and resets the PIN <br>| NetworkActivationService      | Activates voice, SMS, data, and USSD on the new SIM <br>| NotificationService           | Sends the confirmation SMS to the alternate number </pre><p>I mocked all seven tools in this example. They use configurable success rates and a short asyncio.sleep to simulate real-world latency. Here&#39;s what IdentityVerificationService looks like:</p><pre>class IdentityVerificationService:<br>    def __init__(self, success_rate: float | None = None):<br>        self.success_rate = (<br>            success_rate if success_rate is not None<br>            else config.IDENTITY_VERIFICATION_SUCCESS_RATE<br>        )<br><br>    async def verify(self, subscriber_id: str) -&gt; dict:<br>        await asyncio.sleep(random.uniform(0.6, 1.4))<br><br>        questions = [&quot;id_number&quot;, &quot;frequently_called&quot;, &quot;recent_mpesa&quot;, &quot;last_topup&quot;]<br>        per_question: dict[str, bool] = {<br>            q: random.random() &lt; self.success_rate for q in questions<br>        }<br>        correct = sum(per_question.values())<br>        passed = correct &gt;= config.KBA_QUESTIONS_REQUIRED<br><br>        return {<br>            &quot;verification_id&quot;: str(uuid.uuid4()),<br>            &quot;status&quot;: &quot;verified&quot; if passed else &quot;failed&quot;,<br>            &quot;questions_total&quot;: config.KBA_QUESTIONS_TOTAL,<br>            &quot;questions_required&quot;: config.KBA_QUESTIONS_REQUIRED,<br>            &quot;questions_correct&quot;: correct,<br>            &quot;per_question&quot;: per_question,<br>            ...<br>        }</pre><p>The KBA challenge asks the customer four independent questions. They need to get at least three right to pass, which is configurable in <a href="https://github.com/wandabwa2004/safaricom_agentic_ai/blob/dev/config.py">config.py</a>:</p><pre>KBA_QUESTIONS_TOTAL = 4<br>KBA_QUESTIONS_REQUIRED = 3<br>MAX_IDENTITY_RETRIES = 3<br>MAX_SIM_SERIAL_RETRIES = 3</pre><p>You might be wondering why I used mock APIs. The answer is that what is being built and tested here is the agent’s decision-making logic, including the retries, routing, state transitions, and fraud handling. That logic is completely separate from any actual Safaricom API. Mocking the services isolates the agent logic so I can run dozens of deterministic scenarios without needing a live network, and the mock classes can be swapped for real HTTP clients whenever the system is ready for production.</p><h4><strong>3. The Nodes</strong></h4><p><strong><em>Nodes </em></strong>are the individual steps of the workflow, and in LangGraph they follow a simple contract where each node is an async function that takes the full state as input and returns a dictionary of just the fields it changed. LangGraph merges the partial update back into the state automatically.</p><p>Here’s the first node, initiate_request which fires when a customer starts the process:</p><pre>async def initiate_request(state: dict) -&gt; dict:<br>    request_id = f&quot;SWP-{uuid.uuid4().hex[:8].upper()}&quot;<br>    now = datetime.now().isoformat()<br><br>    msg = (<br>        f&quot;Karibu Safaricom. I&#39;m here to help you replace your SIM for a &quot;<br>        f&quot;**{state.get(&#39;request_reason&#39;, &#39;lost&#39;)}** line.\n\n&quot;<br>        f&quot;Your request reference is **{request_id}**. &quot;<br>        f&quot;I&#39;ll take you through the swap step by step.\n\n&quot;<br>        f&quot;**Step 1:** Let&#39;s first verify your identity.&quot;<br>    )<br><br>    audit_log = _log(state, &quot;request_initiated&quot;, {<br>        &quot;request_id&quot;: request_id,<br>        &quot;request_reason&quot;: state.get(&quot;request_reason&quot;, &quot;lost&quot;),<br>        &quot;subscriber_id&quot;: state.get(&quot;subscriber_id&quot;, &quot;&quot;),<br>        &quot;channel&quot;: state.get(&quot;channel&quot;, &quot;shop&quot;),<br>    })<br><br>    return {<br>        &quot;request_id&quot;: request_id,<br>        &quot;request_timestamp&quot;: now,<br>        &quot;current_step&quot;: &quot;initiate_request&quot;,<br>        &quot;process_status&quot;: &quot;in_progress&quot;,<br>        &quot;id_verification_status&quot;: &quot;pending&quot;,<br>        &quot;id_verification_attempts&quot;: 0,<br>        &quot;id_max_retries&quot;: config.MAX_IDENTITY_RETRIES,<br>        &quot;sim_validation_status&quot;: &quot;pending&quot;,<br>        &quot;sim_validation_attempts&quot;: 0,<br>        &quot;sim_max_retries&quot;: config.MAX_SIM_SERIAL_RETRIES,<br>        &quot;old_sim_blocked&quot;: False,<br>        &quot;new_sim_provisioned&quot;: False,<br>        ...<br>        &quot;audit_log&quot;: audit_log,<br>        &quot;messages&quot;: [AIMessage(content=msg)],<br>    }</pre><p>This node does three things. It <em>generates a unique request reference</em>, <em>sets every counter and flag to its initial value,</em> and <em>appends the opening message</em>. That last point is intentional as all initialisation lives here and not scattered across downstream nodes.</p><p>Now let’s look at verify_identity, because this is the first node where things can actually go wrong:</p><pre>async def verify_identity(state: dict) -&gt; dict:<br>    attempts = state.get(&quot;id_verification_attempts&quot;, 0) + 1<br>    result = await identity_service.verify(state[&quot;subscriber_id&quot;])<br><br>    correct = result[&quot;questions_correct&quot;]<br>    required = result[&quot;questions_required&quot;]<br>    total = result[&quot;questions_total&quot;]<br><br>    if result[&quot;status&quot;] == &quot;verified&quot;:<br>        msg = (<br>            f&quot;Identity verified — you answered **{correct}/{total}** questions &quot;<br>            f&quot;correctly (minimum {required} required). Thank you!\n\n&quot;<br>            f&quot;**Step 2:** I&#39;ll now check the status of your line.&quot;<br>        )<br>        status = &quot;passed&quot;<br>    else:<br>        remaining = state.get(&quot;id_max_retries&quot;, config.MAX_IDENTITY_RETRIES) - attempts<br>        reasons = &quot;; &quot;.join(result.get(&quot;failure_reasons&quot;, []))<br>        if remaining &gt; 0:<br>            msg = (<br>                f&quot;I got **{correct}/{total}** correct — I need at least **{required}**. &quot;<br>                f&quot;Specifically: {reasons}.\n\n&quot;<br>                f&quot;Let&#39;s try again. You have **{remaining}** attempt(s) remaining.&quot;<br>            )<br>        else:<br>            msg = (<br>                f&quot;Identity verification failed after &quot;<br>                f&quot;{state.get(&#39;id_max_retries&#39;, config.MAX_IDENTITY_RETRIES)} attempts. &quot;<br>                f&quot;For your security, I cannot proceed with the SIM swap.&quot;<br>            )<br>        status = &quot;failed&quot;<br><br>    return {<br>        &quot;current_step&quot;: &quot;verify_identity&quot;,<br>        &quot;id_verification_status&quot;: status,<br>        &quot;id_verification_attempts&quot;: attempts,<br>        &quot;kba_questions_correct&quot;: correct,<br>        &quot;audit_log&quot;: _log(state, &quot;identity_verification&quot;, {...}),<br>        &quot;messages&quot;: [AIMessage(content=msg)],<br>    }</pre><p>The node increments id_verification_attempts itself, and produces a contextual message depending on the outcome. If retries remain, the customer is told how many are left. If they are exhausted, the node delivers the rejection message. Most importantly though, the node does not decide what happens next. It simply sets id_verification_status to &quot;passed&quot; or &quot;failed&quot; and returns. The decision about whether to retry or reject lives elsewhere.</p><p>The other nodes follow the same pattern. check_line_statusqueries the Home Location Register (HLR) and sets <em>fraud_flag</em>, <em>pending_swap</em>, and <em>line_check_passed</em>. capture_new_sim validates the new SIM’s ICCID and IMSI. Thereafter, the core pipeline runs sequentially: block_old_sim, provision_new_sim, reactivate_mpesa, activate_on_network, and finally send_confirmation_sms, which assembles the full swap summary table for the customer.</p><p>There’s also a shared log helper used by every node:</p><pre>def _log(state: dict, action: str, details: dict) -&gt; list[dict]:<br>    entry = {<br>        &quot;timestamp&quot;: datetime.now().isoformat(),<br>        &quot;step&quot;: state.get(&quot;current_step&quot;, &quot;unknown&quot;),<br>        &quot;action&quot;: action,<br>        &quot;request_id&quot;: state.get(&quot;request_id&quot;, &quot;&quot;),<br>        **details,<br>    }<br>    return state.get(&quot;audit_log&quot;, []) + [entry]</pre><p>It appends one structured entry per node execution and returns the full updated list. Because audit_log is just another field in the state, it travels with the swap from start to finish, building up a complete record of every step without needing any external logging system.</p><h4><strong>4. Edges — Where the Decisions Actually Live</strong></h4><p>This is the most important component of the agentic framework. If nodes are the checklist items then edges are the logic between them. In LangGraph, an edge is a function that reads the current state and returns the name of the next node to run.</p><p>There are two kinds. <em>Linear edges</em> connect nodes that always run in sequence, no matter what. The core swap pipeline is entirely linear once the gate checks pass:</p><pre>graph.add_edge(&quot;block_old_sim&quot;, &quot;provision_new_sim&quot;)<br>graph.add_edge(&quot;provision_new_sim&quot;, &quot;reactivate_mpesa&quot;)<br>graph.add_edge(&quot;reactivate_mpesa&quot;, &quot;activate_on_network&quot;)<br>graph.add_edge(&quot;activate_on_network&quot;, &quot;send_confirmation_sms&quot;)</pre><p>The linear logic in the above code is that you cannot provision before blocking and activate before provisioning. The order is non-negotiable and the linear edges encode that directly.</p><p>On the other hand,<strong> </strong><em>Conditional edges</em><strong> </strong>are where the agentic behaviour actually comes from and they are three of them. Let’s look at the one after identity verification:</p><pre>def route_after_identity(state: SimSwapState) -&gt; str:<br>    if state.get(&quot;id_verification_status&quot;) == &quot;passed&quot;:<br>        return &quot;check_line_status&quot;<br>    if state.get(&quot;id_verification_attempts&quot;, 0) &gt;= state.get(<br>        &quot;id_max_retries&quot;, config.MAX_IDENTITY_RETRIES<br>    ):<br>        return &quot;reject_request&quot;<br>    return &quot;verify_identity&quot;</pre><p>Three possible outcomes are here. If KBA passed, move on to the line check. If it failed but retries remain, return verify_identityof the same node that just ran. If attempts are exhausted, send to reject_request. That last return value, routing a node back to itself, is the retry loop. A chatbot doesn’t have this. It can’t decide to revisit a step based on what just happened. This function is five lines and it’s actually the difference between a pipeline and an agent.</p><p>The line check router is deliberately simpler:</p><pre>def route_after_line_check(state: SimSwapState) -&gt; str:<br>    if state.get(&quot;line_check_passed&quot;):<br>        return &quot;capture_new_sim&quot;<br>    return &quot;reject_request&quot;</pre><p>It has two outcomes and no retry. Fraud flags, suspended lines, and pending swaps all terminate immediately because none of those are recoverable through self-service. There’s no loop here by design as the edge just encodes the business rule.</p><p>The SIM validation router mirrors the identity one, with pass, retry up to the limit, or reject as the three possible outcomes:</p><pre>def route_after_sim_validation(state: SimSwapState) -&gt; str:<br>    if state.get(&quot;sim_validation_status&quot;) == &quot;valid&quot;:<br>        return &quot;block_old_sim&quot;<br>    if state.get(&quot;sim_validation_attempts&quot;, 0) &gt;= state.get(<br>        &quot;sim_max_retries&quot;, config.MAX_SIM_SERIAL_RETRIES<br>    ):<br>        return &quot;reject_request&quot;<br>    return &quot;capture_new_sim&quot;</pre><p>The value of keeping routing logic in edges rather than nodes is so that you can unit test each router as a pure function of state, pass it a dict with the relevant fields and <em>assert </em>on the returned string. You don’t need to wire up the full graph, call any services, or manage <em>async</em>. In addition, if a business rule changes, say the fraud policy becomes a soft warning rather than a hard stop, then you update one function in <a href="https://github.com/wandabwa2004/safaricom_agentic_ai/blob/dev/graph/edges.py">edges.py</a> and nothing else in the system needs to change.</p><h4><strong>Putting It All Together</strong></h4><p>The graph assembly lives in <a href="https://github.com/wandabwa2004/safaricom_agentic_ai/blob/dev/graph/builder.py">builder.py</a>. It is a straightforward sequence that follows this path: <em>create the graph, register each node</em>, <em>connect them with edges,</em> and <em>compile</em>:</p><pre>from langgraph.graph import StateGraph, END<br><br>def build_graph():<br>    graph = StateGraph(SimSwapState)<br><br>    # Register nodes<br>    graph.add_node(&quot;initiate_request&quot;, initiate_request)<br>    graph.add_node(&quot;verify_identity&quot;, verify_identity)<br>    graph.add_node(&quot;check_line_status&quot;, check_line_status)<br>    graph.add_node(&quot;capture_new_sim&quot;, capture_new_sim)<br>    graph.add_node(&quot;block_old_sim&quot;, block_old_sim)<br>    graph.add_node(&quot;provision_new_sim&quot;, provision_new_sim)<br>    graph.add_node(&quot;reactivate_mpesa&quot;, reactivate_mpesa)<br>    graph.add_node(&quot;activate_on_network&quot;, activate_on_network)<br>    graph.add_node(&quot;send_confirmation_sms&quot;, send_confirmation_sms)<br>    graph.add_node(&quot;reject_request&quot;, reject_request)<br><br>    # Entry point<br>    graph.set_entry_point(&quot;initiate_request&quot;)<br><br>    # Linear edge: entry flows into first gate<br>    graph.add_edge(&quot;initiate_request&quot;, &quot;verify_identity&quot;)<br><br>    # Conditional routing at the three decision points<br>    graph.add_conditional_edges(&quot;verify_identity&quot;, route_after_identity, {<br>        &quot;check_line_status&quot;: &quot;check_line_status&quot;,<br>        &quot;reject_request&quot;: &quot;reject_request&quot;,<br>        &quot;verify_identity&quot;: &quot;verify_identity&quot;,   # the retry self-loop<br>    })<br>    graph.add_conditional_edges(&quot;check_line_status&quot;, route_after_line_check, {<br>        &quot;capture_new_sim&quot;: &quot;capture_new_sim&quot;,<br>        &quot;reject_request&quot;: &quot;reject_request&quot;,<br>    })<br>    graph.add_conditional_edges(&quot;capture_new_sim&quot;, route_after_sim_validation, {<br>        &quot;block_old_sim&quot;: &quot;block_old_sim&quot;,<br>        &quot;capture_new_sim&quot;: &quot;capture_new_sim&quot;,   # retry self-loop<br>        &quot;reject_request&quot;: &quot;reject_request&quot;,<br>    })<br><br>    # Linear pipeline: core swap<br>    graph.add_edge(&quot;block_old_sim&quot;, &quot;provision_new_sim&quot;)<br>    graph.add_edge(&quot;provision_new_sim&quot;, &quot;reactivate_mpesa&quot;)<br>    graph.add_edge(&quot;reactivate_mpesa&quot;, &quot;activate_on_network&quot;)<br>    graph.add_edge(&quot;activate_on_network&quot;, &quot;send_confirmation_sms&quot;)<br><br>    # Terminal nodes<br>    graph.add_edge(&quot;send_confirmation_sms&quot;, END)<br>    graph.add_edge(&quot;reject_request&quot;, END)<br><br>    return graph.compile()</pre><p>One thing I like about LangGraph is that the compiled graph can export itself as a Mermaid diagram for a graphical view as follows:</p><pre>def get_graph_mermaid() -&gt; str:<br>    compiled = build_graph()<br>    return compiled.get_graph().draw_mermaid()</pre><p>The diagram is generated from the same code that runs in production, which means it can never go stale. Change a routing rule and the diagram reflects it on the next export. Anyone who has ever maintained a system architecture diagram in a slide deck, only to watch it become outdated within a week, will appreciate why that matters.</p><h4><strong>Five Scenarios, Five Paths Through the Same Graph</strong></h4><p>I added a <a href="https://github.com/wandabwa2004/safaricom_agentic_ai/tree/dev/demo">demo </a>that you can <a href="https://github.com/wandabwa2004/safaricom_agentic_ai/blob/dev/README.md">setup </a>locally and run. The demo comes with five built-in scenarios, each using a different subscriber from the seed data. You run them like this:</p><pre>python -m demo.cli happy_path<br>python -m demo.cli identity_retry<br>python -m demo.cli fraud_flag<br>python -m demo.cli bad_iccid<br>python -m demo.cli full_failure</pre><p>Here’s what each one exercises:</p><pre>| Scenario       | Subscriber                     | What actually happens |<br>|----------------|--------------------------------|---------------------------|<br>| happy_path     | Wanjiku Kamau (+254712345678)  | All gates pass on the first attempt, full pipeline runs, swap completes |<br>| identity_retry | —                              | KBA fails once, the self-loop fires, passes on the second attempt |<br>| fraud_flag     | Njoroge Mwangi (+254745678901) | Line check finds a fraud flag, `route_after_line_check` returns `reject_request` immediately |<br>| bad_iccid      | —                              | SIM validation fails on the first attempt, retries, eventually rejects |<br>| full_failure   | —                              | Identity verification exhausted after three attempts, process rejected |</pre><p>The important thing to observe is that all five paths run through the same compiled graph. The agent doesn’t have a separate code path for fraud versus normal flow. The fraud flag is a field in the state that is read by the edge, and is followed by routing. That’s all there is to it.</p><p>Below is an example of a happy path that I ran. It produces an output that looks roughly like this:</p><pre>Agent: Karibu Safaricom. I&#39;m here to help you replace your SIM for a lost line.<br><br>Your request reference is SWP-3F8A21DC. I&#39;ll take you through the swap step by step.<br><br>Step 1: Let&#39;s first verify your identity.<br><br>Agent: Identity verified — you answered 3/4 questions correctly (minimum 3 required). Thank you!<br><br>Step 2: I&#39;ll now check the status of your line.<br><br>Agent: Line status confirmed.<br>- MSISDN: +254712345678<br>- Name: Wanjiku Kamau<br>- Status: Active<br>- Current SIM serial: 89254…5678<br>- M-PESA: Registered (balance Ksh 4,320.00)<br><br>Step 3: Please hand me the new SIM so I can capture its serial (ICCID) and IMSI.<br><br>...<br><br>Agent: Your new SIM is now LIVE on the Safaricom network — voice, SMS, data and USSD are all enabled.<br><br>SIM Swap Summary<br>| Request Ref       | SWP-3F8A21DC          |<br>| MSISDN            | +254712345678         |<br>| Old SIM           | 89254…1234 (Blocked)  |<br>| New SIM           | 89254…5678 (Active)   |<br>| M-PESA            | Restored (Ksh 4,320)  |<br>| Confirmation SMS  | Delivered             |<br><br>AUDIT LOG (9 entries)<br>  [2026-04-25T10:12:03] request_initiated<br>  [2026-04-25T10:12:04] identity_verification<br>  [2026-04-25T10:12:05] line_status_check<br>  [2026-04-25T10:12:06] sim_validation<br>  [2026-04-25T10:12:07] block_old_sim<br>  [2026-04-25T10:12:08] provision_new_sim<br>  [2026-04-25T10:12:09] mpesa_reactivate<br>  [2026-04-25T10:12:10] network_activation<br>  [2026-04-25T10:12:11] confirmation_sms<br><br>Final status: completed</pre><p>The above shows nine audit entries, one per node, each with a timestamp and a request ID. The full state at any point in that run can be checkpointed and replayed. That’s not something you get out of any conventional chatbot.</p><h4><strong>What You Actually Have Now</strong></h4><p>If you run this locally, here’s what you’ve actually built:</p><ul><li><strong>The graph is the documentation: </strong>The Mermaid export is generated from live code. The retry loops, the fraud halt, and the M-PESA skip for unregistered lines are all visible in the diagram and come directly from the routing functions. They’re all visible in the diagram and they all come directly from the routing functions. Nobody has to keep a separate flowchart up to date.</li><li><strong>The state is the audit trail: </strong>Every step appends one entry to audit_log. By the time the process reaches its terminal node, you have a complete, timestamped record of exactly what the agent did and why. That record lives in the same object as the result, no separate logging infrastructure required for the prototype.</li><li><strong>The edges are the policy: </strong>If Safaricom changes the KBA requirement from three-out-of-four to four-out-of-four, that’s a one-line change in config.py. If fraud policy changes from immediate termination to a human-escalation path, you update route_after_line_check and add a new node. The rest of the graph is not really affected.</li></ul><h4><strong>What’s Still Missing</strong></h4><p>What I’ve tried put together here is a solid prototype. But there’s a meaningful gap between prototype and production, and I’ll try cover this gap in Part 3.</p><p>Remember I haven’t connected this to anything external yet. There’s a FastAPI wrapper that exposes the agent as an OpenAI-compatible endpoint so Open WebUI can talk to it, but I haven’t walked through that here. I’ve also not added observability. Right now, if something fails mid-run, you would need to dig through the state manually. There is also no human-in-the-loop interrupt, meaning an operator cannot pause the flow and review it before the old SIM gets blocked. For a production SIM swap system, that last one is not optional.</p><p>I promise to cover most of these in Part 3: the FastAPI service, hooking into Langfuse for tracing, and adding interrupt-based human review at the fraud flag step. The code will be extended in the same repo.</p><p>The full source code and Readme on how to run it is available <a href="https://github.com/wandabwa2004/safaricom_agentic_ai"><strong>HERE</strong></a>.</p><p>I hope this was useful. If you found it helpful, then please follow me for more of such, leave a comment, clap for me and of course let me know what else you’d like to see in Part 3 . You can also find all my other articles on my <a href="https://medium.com/@hermanwandabwa">profile </a>and connect with me on <a href="https://www.linkedin.com/in/wandabwaherman/">LinkedIn</a>. Remember all my articles are FREE to read.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7290f5699023" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science-collective/part-2-from-theory-to-code-building-a-sim-swap-agent-in-langgraph-7290f5699023">From Theory to Code — Building a SIM Replacement Agent in LangGraph (Part 2)</a> was originally published in <a href="https://medium.com/data-science-collective">Data Science Collective</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[What Agentic AI Actually Is: A SIM Replacement Use Case (Part 1)]]></title>
            <link>https://medium.com/data-science-collective/what-agentic-ai-actually-is-a-sim-replacement-use-case-part-1-b9f1672d68b5?source=rss-58e995b6d0e3------2</link>
            <guid isPermaLink="false">https://medium.com/p/b9f1672d68b5</guid>
            <category><![CDATA[agentic-ai]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[data-analytics]]></category>
            <category><![CDATA[large-language-models]]></category>
            <dc:creator><![CDATA[Herman Wandabwa]]></dc:creator>
            <pubDate>Mon, 20 Apr 2026 10:51:16 GMT</pubDate>
            <atom:updated>2026-04-20T10:52:53.664Z</atom:updated>
            <content:encoded><![CDATA[<h4>How a real, multi-step telco workflow exposes the architectural gap between chatbots and agents</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*47418695aOkrc-BzqnldBg.png" /><figcaption>Image generated by the author</figcaption></figure><p><strong>Stuck behind a firewall? Read the article for FREE </strong><a href="https://hermanwandabwa.medium.com/what-agentic-ai-actually-is-a-sim-replacement-use-case-part-1-b9f1672d68b5?source=friends_link&amp;sk=ba7b37ebf34944a94c9af14e2ced55ee"><strong>here</strong></a></p><p>It’s been quite some time since I started writing “seriously” about <a href="https://hermanwandabwa.medium.com/list/data-science-and-analytics-af1e961b7a0f">data analytics and science</a>, <a href="https://hermanwandabwa.medium.com/list/optimisation-13c618431392">optimisation </a>and, most recently, <a href="https://hermanwandabwa.medium.com/list/agentic-ai-and-llms-6d358df6e86c">generative AI</a>. In the Gen AI space, I’ve tried covering topics around RAG, finetuning, agentic AI, and a fair bit in between. If you’ve read any of my articles, I’m sure you know I lean towards not only a simple but also a practical angle rather than theoretical ones. I also share all links on my <a href="https://hermanwandabwa.medium.com/">profile,</a> and all are FREE to read.</p><p>Agentic AI is THE buzzword right now, and I’m happy to join the bandwagon with a series on it. After all, I’m just human 😂.</p><p>But here’s what actually pushed me to write this. I kept reading articles that promised to explain “agentic AI” and ended up describing a chatbot with a slightly longer prompt. Most of them are wrappers around an LLM (Large Language Model) with one or two API calls bolted on, dressed up as something brand new. Simply put, agentic AI is an AI system that can accomplish a specific goal with limited human supervision, exhibiting real autonomy, goal-driven behaviour, and adaptability rather than just operating within predefined constraints. I don’t buy the watered-down versions, and of course I’m NOT going to claim I’ve built something wildly novel either.</p><p>But I did build one that actually is agentic. An agent that runs the full SIM card replacement process for a mobile network on its own, from identity verification all the way to customer notification, with no human pushing it along at each step. I built it as a reference implementation around the <a href="https://www.safaricom.co.ke/">Safaricom</a> Kenya SIM swap flow just in case a customer needs it done.</p><p>This is Part 1 of a three-part series. Here I’ll cover what agentic AI actually means, why a SIM replacement workflow is a good example of it, and the four mental building blocks every agent needs. Part 2 turns everything here into working open-source Python code with LangGraph. Part 3 is about what it takes to ship one of these into production. So don’t expect any code in this part, just the concepts you’ll need to make sense of the rest of the articles.</p><h3>What Makes an AI “Agentic”?</h3><p>Let me clear something up, because this word gets abused a lot.</p><p>A standard chatbot is reactive. You say X, it says Y. It has no memory of what it did five messages ago. It cannot call external systems on your behalf. And critically, it cannot decide what to do next based on what just happened. Every turn is independent. Have you ever found yourself typing “talk to a human” or “speak to an agent” into a customer service chatbot just to get transferred to a real person? The bot did not figure out on its own that you were frustrated or that your issue was beyond it. You had to spell it out. That is exactly the limitation we are talking about.</p><p>An agentic AI is different. It decides which action to take next based on the current situation. It calls real tools like APIs, databases, and provisioning systems. It handles failures gracefully, retrying or escalating as needed. It maintains its own running state across many steps. Lastly, it works towards a goal without a human directing every move unless fully specified to be so in its design.</p><p>Think of the difference between a customer service rep reading from a script versus an experienced retail agent who knows the process, handles edge cases, and uses their own judgment at each step. That’s the gap we’re bridging. The <em>script-reader</em> is your chatbot, while the <em>experienced agent</em> is your agent.</p><p>The SIM replacement use case is a good testbed for this because it is genuinely multi-step, genuinely branching, and genuinely fails in interesting ways. Identity verification can fail several times before the system gives up. The new SIM serial may not match what the customer reads out. M-PESA reactivation can stall. A line flagged for fraud terminates the process entirely. A simple chatbot collapses at the first unexpected branch while a properly configured agent handles all of it.</p><h3>The Four Building Blocks of Every Agent</h3><p>Before we get into the specifics of the SIM replacement workflow, here’s the mental model that makes everything else click. Every agent, regardless of framework, comes down to four concepts.</p><pre>Concept   | Question it answers                    | Analogy<br>----------|----------------------------------------|--------------------------------<br>State     | What does the agent know right now?    | A notebook the agent carries<br>Tools     | What can the agent do?                 | The agent&#39;s hands<br>Nodes     | How does each step work?               | Individual tasks on a checklist<br>Edges     | What should the agent do next?         | The decision logic between tasks</pre><p>Let me walk through each one in plain language.</p><ol><li><strong>State</strong> is the agent’s running memory. It’s everything the agent knows at any moment, including who the customer is, what’s been verified, what’s failed, and what’s still pending. In the SIM replacement scenario, <em>state </em>holds the customer’s MSISDN (their phone number), their ID details, how many verification attempts have been used, the new SIM’s serial number, whether the old line has been blocked, whether the new SIM has been activated, whether M-PESA has been re-linked, and so on. Think of it as a notebook the agent carries from step to step. Every step reads from it and writes to it.</li><li><strong>Tools</strong> are what the agent can actually do in the real world. APIs to call, databases to query, provisioning systems to update, SMS gateways to message. In the SIM replacement example, the tools include an identity verification service, a line status check, the SIM provisioning system that blocks the old IMSI and binds the new one, an M-PESA reactivation service, and a notification service. Without tools, an agent is just talk. With tools, it can take action.</li><li><strong>Nodes</strong> are the individual steps of the workflow. Each node has one job. In this example, it could be to verify the customer’s identity, check the line status, block the old SIM, activate the new one, re-link M-PESA or even send the confirmation SMS. Each step does its work, usually by calling one or more tools, then updates the state. Keeping each node focused on a single job is what makes the whole system testable and debuggable later.</li><li><strong>Edges</strong> are the decision logic between steps. After a node finishes, an edge looks at the current state and decides where to go next. This is where the “agentic” behaviour actually lives. For example, if identity verification passes, then go to the line status check. If it failed but retries remain, loop back and try again, and if retries are exhausted, terminate the request. Edges are how an agent decides, dynamically, what to do next based on what just happened. A chatbot doesn’t have edges. It just responds.</li></ol><p>That’s the entire mental model.<strong> State, tools, nodes</strong>, and <strong>edges</strong>. Once you have these four ideas in your head, every agent you ever look at will make sense.</p><h3>The Problem We’re Actually Solving</h3><p>When a Safaricom customer loses their SIM card, the process at an M-PESA agent or retail shop looks roughly like this, and this is just one path that I’m keen to automate:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*z1GrMnWZR7XeMK6gDMT5Lg.png" /><figcaption>SIM card replacement process</figcaption></figure><p>Each step depends on the previous one. Some steps, like identity verification, can loop within a limit while some are terminal. For example, a fraud flag ends the process immediately while others adapt to circumstances. If the customer has forgotten their original PIN, the workflow takes a different verification path using deeper account history questions instead of failing outright. This is exactly the kind of workflow that exposes whether your “agent” is real and works or is simply a marketing gimmick.</p><p>It will be hard for a traditional chatbot to do this. It can answer “what happens when I lose my SIM” with a script. However, it cannot actually run the process, handle the branching, retry failed steps, or decide between standard verification and the deeper PIN-recovery path based on what the customer remembers. An agent can and that’s the whole point.</p><h3>Why This Matters Now</h3><p>The framing I keep coming back to is this: an agent is not a smarter chatbot. It’s a different architecture entirely. You stop thinking about responses and start thinking about processes. You stop thinking about prompts and start thinking about state. That shift in mental model is what separates the people building real agentic systems from the people slapping the label on a chatbot and hoping no one notices.</p><p>There’s a practical reason this matters now. The cost of LLM calls has dropped enough that running a multi-step workflow is no longer prohibitive. Frameworks like LangGraph have matured to the point where the plumbing is no longer the hard part. The constraint has shifted from “can we build this” to “do we understand what we’re building.” For organisations sitting on workflows that look like the SIM replacement process, and most telcos, utilities, and service providers have dozens of them, the door is open.</p><p>The interesting question is no longer whether agents work. It’s which workflows are worth turning into agents and what you need to know to build them properly.</p><h3>What Comes Next</h3><p>That’s Part 1, and I intended to keep it as theoretical as possible. I covered what agentic AI actually means, why a SIM replacement workflow is a good test of whether something is genuinely agentic, and the four mental building blocks (state, tools, nodes, edges) that every agent shares.</p><p>In Part 2, I’ll translate all of this into working Python code using LangGraph. We’ll build the actual state object, wire up the tools, write the nodes, and define the edges that handle the retry loops, the fraud flag termination, and the alternative verification paths. Everything in this article will be there in the code that will be open for anyone to customise.</p><p>In Part 3, I’ll cover deployment. This will largely be what it takes to move from a working prototype locally to a production system: observability, human-in-the-loop interrupts, the FastAPI service, etc.</p><p>I hope this walkthrough was useful. Don’t forget to follow me, clap for me, and leave a comment. Let me know if you’d like me to cover anything specific in Part 2 or Part 3, whether that’s a particular LangGraph pattern, a deployment detail, or something else entirely. If you want to check out my other articles, you can find them on my <a href="https://medium.com/@hermanwandabwa"><strong>profile</strong></a>. I’m also happy to connect via <a href="https://www.linkedin.com/in/wandabwaherman/"><strong>LinkedIn</strong></a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b9f1672d68b5" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science-collective/what-agentic-ai-actually-is-a-sim-replacement-use-case-part-1-b9f1672d68b5">What Agentic AI Actually Is: A SIM Replacement Use Case (Part 1)</a> was originally published in <a href="https://medium.com/data-science-collective">Data Science Collective</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[From Jupyter Notebook to Full-Stack ML App with FastAPI, React, and Supabase]]></title>
            <link>https://medium.com/data-science-collective/stop-guessing-staffing-needs-how-id-predict-daily-museum-visitors-before-they-arrive-part-2-760498e03c4f?source=rss-58e995b6d0e3------2</link>
            <guid isPermaLink="false">https://medium.com/p/760498e03c4f</guid>
            <category><![CDATA[museum-analytics]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[visitor-forecasting]]></category>
            <category><![CDATA[full-stack-development]]></category>
            <dc:creator><![CDATA[Herman Wandabwa]]></dc:creator>
            <pubDate>Mon, 13 Apr 2026 00:50:47 GMT</pubDate>
            <atom:updated>2026-04-25T03:23:50.890Z</atom:updated>
            <content:encoded><![CDATA[<h4>A hands-on guide to deploying an XGBoost model as a REST API and interactive dashboard for daily visitor forecasting</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0A9bW6ePT23GSEC_ml02VQ.png" /><figcaption>Image generated by the author</figcaption></figure><p><strong>Author’s note:</strong> <em>This article uses independently created code and synthetic data for illustration. The methods described are standard forecasting and machine learning techniques commonly used across many application areas.</em></p><p><strong>Stuck behind a firewall? Read the article for FREE </strong><a href="https://hermanwandabwa.medium.com/760498e03c4f?source=friends_link&amp;sk=91968c50229a25008617d04de471b7ce"><strong>here</strong></a></p><p>In <a href="https://medium.com/data-science-collective/stop-guessing-staffing-needs-predicting-daily-museum-visitors-before-they-arrive-part-1-336ac0ca4f60">Part 1</a>, I covered the data science workflow behind this project. This included the process of generating synthetic museum visitor data, engineering 80+ features from 26 raw columns, training and comparing multiple models, and evaluating the winner. XGBoost took the top spot with a MAPE of 14.0%. I also detailed the process of framing model performance in business terms that museum directors actually care about.</p><p>But a model sitting in a Jupyter notebook is completely useless to a museum operations manager who needs to plan staffing for next Tuesday. The real value comes from making predictions accessible to the people who need them through a working application with a user interface.</p><p>In this part, I’ll take the trained model and deploy it as a full-stack application: a FastAPI backend that exposes the model as an API, a React dashboard for the frontend, and a Supabase database for logging predictions. By the end of this article, you’ll understand not just <em>what</em> I built, but <em>why</em> each architectural decision was made.</p><p>I recommend once again that you start with <a href="https://medium.com/data-science-collective/stop-guessing-staffing-needs-predicting-daily-museum-visitors-before-they-arrive-part-1-336ac0ca4f60">Part 1</a> if you haven’t read it yet as it literally covers the <em>why</em> and <em>what</em> behind the model. On the other hand, this article covers the <em>how</em> of getting it into production. As always, all code is open-sourced <a href="https://github.com/wandabwa2004/museum-visitor-prediction">here</a>.</p><h3><strong>From a Notebook to Production — The Full Stack</strong></h3><ol><li><strong>Why We Need a Backend at All</strong></li></ol><p>Before diving into code, let’s establish why a backend in a production setting exists in the first place. The trained XGBoost model is basically a Python object that lives in a .json file on disk and can only be used by Python code that knows how to load it.</p><p>On the other hand, a React web dashboard is JavaScript running in a browser that cannot directly load a Python model. Therefore, the solution is a <strong>backend server</strong> which is just a Python application that loads the model and exposes it over HTTP. Any client e.g., a browser, mobile app, or spreadsheet plugin can then send it a request and receive a prediction in return.</p><p>This pattern is called a <a href="https://www.redhat.com/en/topics/api/what-is-a-rest-api"><strong>REST API</strong></a>(Representational State Transfer Application Programming Interface). Think of it like a restaurant: you (the frontend) don’t walk into the kitchen and cook your own food. You hand a request to a waiter (the API), the kitchen (the backend) prepares the response, and the waiter brings it back to you in a standard format. You never need to know how the kitchen works.</p><p><strong>a) Backend — FastAPI</strong></p><p>I used a <a href="https://fastapi.tiangolo.com/">FastAPI </a>framework that is a modern Python one for building APIs quickly and correctly. Two things make it particularly good for ML serving:</p><ol><li><strong>Automatic data validation —</strong> You just define the shape of your request and response using Python type hints, and FastAPI automatically validates incoming requests and rejects malformed ones before they can crash your model.</li><li><strong>Auto-generated documentation —</strong> FastAPI produces a live /docs page that lists every endpoint, what it accepts, and what it returns. This is invaluable when frontend developers (or your future self) need to understand how to call the API.</li></ol><p>The backend in this application exposes four endpoints, each serving a distinct purpose:</p><pre>- `POST /api/predict` - The core endpoint. Accepts a date and optional context parameters (weather, holidays, special events), runs them through the model, and returns a prediction with confidence intervals and a traffic tier label.<br>- `GET /api/historical` - Returns paginated historical visitor data with optional date filters. This powers the charts in the frontend&#39;s Historical tab.<br>- `GET /api/insights` - Returns model performance metrics, feature importance rankings, and a comparison table of all the models trained in Part 1. This powers the Insights tab.<br>- `GET /api/date-info` - A smart helper endpoint: given a date, it automatically detects Victorian public holidays and fetches weather forecasts. The frontend calls this whenever the user picks a date, so the prediction form populates itself without requiring manual input.</pre><p><strong>Loading the Model Once — <em>The Lifespan Pattern</em></strong></p><p>One of the first problems you’ll encounter when deploying an ML model is the thought of deciding when the model is to be loaded.<em> </em>The naive approach is to load the model at the top of your script as a global variable. This works, but it has a problem in production where web servers typically run multiple worker processes to handle concurrent requests. A global variable in one process is not shared with another process meaning you’d be loading the model independently in each worker, which wastes memory.</p><p>A better approach and I stand to be corrected is FastAPI’s lifespan context manager. This is a special function that runs once when the application starts up, before it begins accepting any requests:</p><pre>@asynccontextmanager<br>async def lifespan(app: FastAPI):<br>    ModelService.load()<br>    yield<br><br>app = FastAPI(<br>    title=&quot;Museum Visitor Prediction API&quot;,<br>    description=&quot;Predict daily visitor numbers for the Melbourne Museum of Migration&quot;,<br>    version=&quot;1.0.0&quot;,<br>    lifespan=lifespan,<br>)</pre><p>The yield keyword is what makes this a context manager. Everything before yieldruns at startup and everything after yield (if you add cleanup code) runs at shutdown. The ModelService.load() call loads the XGBoost model from disk, reads the feature list, loads historical data, and computes residual statistics. This is the heavy one-time work that should not be repeated on every request.</p><p><strong>The ModelService Singleton</strong></p><p>The ModelService class is designed as a singleton, which is just a<strong> </strong>software pattern where a class has at most one instance, and that instance is shared across the whole application.</p><p>Rather than using Python’s normal instance creation model = ModelService(), all the methods are marked as @classmethod. This means they operate on the class itself, not on an instance. The model, feature list, historical data, and residual stats are stored as class-level variables, not instance variables. The effect is straightforward; there’s exactly one copy of the model in memory no matter how many API calls come in simultaneously, and every call reads from the same shared state.</p><pre>Normal class: ModelService() → creates new object every time<br>Singleton pattern: ModelService.predict(…) → always uses the same shared state</pre><p>This matters because XGBoost models can be large (tens or hundreds of megabytes). Loading one per request would exhaust memory within seconds under real traffic. Therefore loading one at startup and sharing it is the correct production pattern. Don’t worry if the above doesn’t make much sense as I’ll dig deeper in the interesting parts of the code.</p><p>Before running any of these steps, please follow this <a href="https://github.com/wandabwa2004/museum-visitor-prediction/blob/dev/article/setup_guide.md">setup guide</a> to run the backend processes first. You’d normally get an output like the following if the setup worked correctly:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/620/1*smLXTXkpV8oqPTDRhQzvIw.png" /><figcaption>A working backend</figcaption></figure><p>Remember are all these are setup for a local deployment.</p><p>When a user clicks the <strong>Predict </strong>button<strong> </strong>in the frontend that I’ll describe later, this is what really happens:</p><ul><li><strong>Request arrives — </strong>The frontend sends an HTTP POST request to /api/predictwith a JSON body containing the date and any contextual parameters the user provided or the auto-fetch populated (weather, holidays, events).</li><li><strong>Feature construction — </strong>The raw request has only a handful of fields. But the model was trained on 81 features. The <em>build_feature_row()</em><strong> </strong>function bridges this gap. It takes the date and context, looks up recent visitor history (needed for lag features), applies the same cyclical encodings and interaction terms that were built during feature engineering in <a href="https://medium.com/data-science-collective/stop-guessing-staffing-needs-predicting-daily-museum-visitors-before-they-arrive-part-1-336ac0ca4f60">Part 1</a>, and returns a single-row DataFrame with all 81 features in the exact order the model expects. For context, this is one of the most common failure points in ML deployments where a <strong>training-serving skew </strong>happens. This is where if the feature engineering code in the backend differs even slightly from what was done during training, the model will silently produce wrong predictions. I stored the feature list in a features.json(the same file produced by the training notebook), so the backend reads that file and uses it as the authoritative column ordering.</li><li><strong>Model inference and inverse transformation — </strong>There’s something interesting that’s happening in the code below. In Part 1, the XGBoost model was trained on log1p(visitors) rather than visitorsdirectly. If you remember, it was because visitor counts were right-skewed. This meant that on most days the museum received around 200–400 visitors, but some peak days spiked to approximately 1,200. This skew can confuse tree-based models that try to minimise squared error , meaning they’ll <strong><em>underpredict </em></strong>peaks to avoid large penalties on the majority of ordinary days. Taking the log then compresses this scale in the lines of log1p(200)≈5.3 and log1p(1200)≈7.1. The model now sees a more uniform distribution and learns better. But the prediction comes back in log-space, so we have to invert it. np.expm1(raw_prediction)converts the log-space output back to visitor counts. The max(0,point)call in the code clips any negative predictions (which are physically impossible for visitor counts) to zero.</li></ul><pre>@classmethod<br>def predict(cls, feature_row: pd.DataFrame) -&gt; dict:<br>    raw = cls._model.predict(feature_row)<br>    point = float(np.expm1(raw[0] + cls._xgb_log_offset)) \<br>        if cls._best_model_name == &quot;XGBoost&quot; else float(raw[0])<br>    point = max(0, point)</pre><ul><li><strong>Prediction intervals — </strong>A single number e.g., 347 visitors as the prediction is less useful than say a range (<em>“between 290 and 405 visitors with 80% confidence”</em>). Normally, prediction intervals communicate uncertainty, which is critical for staffing decisions such that if the interval is wide, schedule conservatively etc.</li></ul><pre>std = cls._residual_std or 50.0<br>lower_80 = max(0, int(point - 1.28 * std))<br>upper_80 = int(point + 1.28 * std)<br>lower_95 = max(0, int(point - 1.96 * std))<br>upper_95 = int(point + 1.96 * std)</pre><p>The residual_std above is the standard deviation of the model’s prediction errors on the training data. To simplify it, residuals are just the differences between what the model predicted and what actually happened. For example, if the model predicted 350 but 410 came in, then the residual is 60. Computing the standard deviation of all residuals gives us a single number that summarises how spread out the errors typically are.</p><p>The multipliers 1.28 and 1.96 come from the <a href="https://www.gcp-service.com/what-is-the-normal-distribution"><strong>normal distribution</strong></a><strong> </strong>whereby under a normal distribution:</p><pre>- 80% of values fall within ±1.28 standard deviations of the mean<br>- 95% of values fall within ±1.96 standard deviations of the mean</pre><p>Therefore, <em>point ± 1.28 * std</em> gives us the 80% prediction interval whereby in 8 out of 10 days, actual visitors should fall inside this range. The 95% interval is wider as it needs to capture more of the possible outcomes. This is called a <a href="https://www.sciencedirect.com/topics/computer-science/gaussian-approximation"><strong><em>Gaussian approximation</em></strong></a> of the prediction interval. It’s not perfect (residuals are rarely perfectly normal), but it’s a practical and interpretable first approximation that works well in production.</p><ul><li><strong>Traffic tier classification and response</strong></li></ul><pre>if point &lt; 250:   tier = &quot;Low&quot;<br>elif point &lt; 450: tier = &quot;Medium&quot;<br>else:              tier = &quot;High&quot;<br><br>return {<br>    &quot;predicted_visitors&quot;: int(round(point)),<br>    &quot;lower_80&quot;: lower_80, &quot;upper_80&quot;: upper_80,<br>    &quot;lower_95&quot;: lower_95, &quot;upper_95&quot;: upper_95,<br>    &quot;traffic_tier&quot;: tier,<br>    &quot;confidence&quot;: round(max(0.0, min(1.0, float(r2))), 3),<br>    &quot;model_used&quot;: cls._best_model_name,<br>}</pre><p>The traffic tier translates the raw number into an operational instruction. A museum operations manager doesn’t just need a number, they need an action. This could be something like <em>call in extra staff, run normal operations, or reduce floor coverage</em>. <strong>Low/Medium/High</strong> maps directly to those three staffing modes. This is an example of designing the output around the decision it needs to support.</p><p><strong>The Date Info Endpoint — Smart Defaults That Reduce Friction</strong></p><p>The /api/date-info endpoint demonstrates something I think is under-appreciated in data product design: reducing the cognitive load of correct usage.</p><p>The prediction form needs several inputs beyond just the date: <em>Is it a public holiday? What will the weather be? Is there a school holiday?</em> A naive form would present empty text boxes and leave it to the user to look all this up. That creates friction and introduces human error. For example, the user might not know that Anzac Day (April 25th) is a Victorian public holiday, or might enter the wrong temperature.</p><p>Instead, when the user selects a date, the frontend automatically calls /api/date-info which:</p><ol><li><strong>Checks the Victorian public holiday calendar </strong>— a hardcoded list of known dates plus rules for floating holidays (like Easter).</li><li><strong>Fetches weather from the Open-Meteo API </strong>— a free weather API that provides both historical data and 7-day forecasts. For dates in the forecast window, it returns the forecast. For past dates, it returns historical actuals. For dates beyond the forecast window (more than 7 days out), it falls back to seasonal defaults based on Melbourne’s average climate by month.</li></ol><p>The user sees the prediction form pre-populated with contextually appropriate values, each labelled with a badge showing its source: <strong>Auto</strong>, <strong>Forecast</strong>, <strong>Historical</strong>, or <strong>Seasonal Default</strong>. If they want to override any of these, they can as the form switches to manual mode and stops auto-fetching that field. This preserves user intent while still providing smart defaults.</p><p>This is an example of UX design thinking applied to a data product. The goal is not just for the model to be accurate, but for the application to be <em>usable by someone who is not a data scientist</em>.</p><h4><strong>Frontend — React + Tailwind Dashboard</strong></h4><p>The frontend is a React 18 application built with Vite and styled with Tailwind CSS as shown below and can also be accessed <a href="https://museum-visitor-prediction.vercel.app/"><strong>HERE</strong></a>. It has three tabs, each serving a distinct purpose for a different type of user question.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gC3OnWPijyJWt1Dj-8bbug.png" /><figcaption>Front-end Dashboard</figcaption></figure><p><strong>Why React? </strong>React is a JavaScript library that makes it practical to build UIs where the data and the visual representation stay in sync. For example, when a user selects a new date, the form fields, weather badges, and prediction result all need to update together. React’s component model makes this manageable where each piece of the UI is a self-contained component that re-renders when its data changes, instead of requiring you to manually update the DOM for every change.</p><p><strong>Why Vite?</strong> Vite on the other hand is the tool that bundles and serves the React application. For development, it provides instant reloading such that when you save a file, the browser updates in under a second without losing the current UI state. For production, it produces an optimised static bundle (HTML, CSS, JavaScript files) that can be hosted on any static file host like Vercel or Netlify.</p><p><strong>Why Tailwind CSS?</strong> Tailwind is a utility-first CSS framework. Instead of writing custom CSS classes, you apply small single-purpose utility classes directly in your JSX markup (`<em>text-lg font-bold text-blue-600</em>`). The result looks verbose but produces consistent, responsive designs without the overhead of managing a separate stylesheet.</p><p><strong>Predict Tab</strong></p><p>This is the primary view. It presents a form where the user selects a date. It then auto-populates the weather and holiday fields appear immediately after date selection via the /api/date-info call. The user can override any field if they have better information (e.g., they know a special event is happening that day).</p><p>When users submit, the prediction result panel appears alongside the form as below:</p><ul><li>The large point estimate (e.g., “538 visitors”)</li><li>The 80% and 95% confidence intervals displayed as a range bar</li><li>The traffic tier badge (Low / Medium / High) in a colour-coded chip</li><li>Estimated revenue that is computed by multiplying the predicted visitor count by an assumed average spend per visitor. This can be customised.</li><li>The model name and its R² score, so the user understands which model produced this result</li></ul><p>Displaying confidence intervals rather than just a point estimate is a deliberate choice. A museum director who sees “<em>Low: 526/ High: 549”</em> immediately understands there is uncertainty in the prediction and can plan for the upper bound when in doubt. However, if the person only sees “538” might staff for exactly 538 and be caught out on a day that comes in at say 600.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/958/1*f6KsEHEjl0ZOx70nEZOr2g.png" /><figcaption>Predictions Tab</figcaption></figure><p><strong>Historical Tab</strong></p><p>This tab shows interactive charts of historical visitor patterns over the past year, built with <a href="https://recharts.github.io/"><strong>Recharts</strong> </a>— a charting library for React that renders responsive SVG charts.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/952/1*IxQRFq0M0E0Bl99tIxGUGg.png" /><figcaption>Historical Visitor Numbers</figcaption></figure><p>The primary value here is context. Before making a staffing decision based on a prediction, a manager might want to know something like: <em>“Is next Saturday typically busy at this time of year? Has that changed recently?”</em> The historical chart lets them visually scan for seasonal patterns, spot anomalies, and build intuition about the data the model was trained on.</p><p>The data is fetched from the GET /api/historicalendpoint, which supports date-range filtering and pagination so large date ranges don’t overwhelm the browser.</p><p><strong>Insights Tab</strong></p><p>This tab is aimed at a more technically curious user. Perhaps an internal analytics team or a curious manager who wants to understand <em>why</em> the model is making certain predictions.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/872/1*TCL_ZjNv85w4e3ZzFGm2aA.png" /><figcaption>Insights Tab</figcaption></figure><p>It shows:</p><ul><li><strong>Model performance metrics </strong>— MAE, RMSE, R², MAPE for the best model, displayed as summary stat cards. I described some of them in Part 1.</li><li><strong>Feature importance rankings</strong> — the top features the model relies on, pulled from the SHAP analysis done in Part 1. This answers questions like <em>“Is the model mostly driven by day-of-week, or does weather matter a lot?”</em></li><li><strong>Model comparison table</strong> — shows all the models trained (Random Forest, XGBoost, Prophet) side by side, so the user can see how much better XGBoost is compared to the alternatives and understand the trade-offs</li></ul><p>The summary stat cards at the top of the whole dashboard i.e., the average daily visitors, peak day visitors, days of history, best model R² etc., give any user immediate orientation before they start exploring. These numbers answer the first questions anyone asks when they encounter a new dataset: <em>“What are we dealing with here?”</em></p><h4><strong>Deployment Considerations</strong></h4><p>As is, the frontend can be deployed to <a href="https://vercel.com/"><strong>Vercel</strong> </a>(static files, CDN-served) and the backend to <a href="https://railway.com/"><strong>Railway</strong></a> (Python process, model artifacts bundled). Keeping them separate means you can redeploy a CSS fix without touching the model server, and scale each independently. If you want to add a database layer like I did for prediction logging, the repo includes a <a href="https://supabase.com/">Supabase </a>setup — see <a href="https://github.com/wandabwa2004/museum-visitor-prediction/blob/dev/article/setup_guide.md">setup_guide.md</a>.</p><p>Remember to set environment variables to keep secrets out of source code:</p><pre># Backend<br>APP_ENV=production<br># Frontend<br>VITE_API_URL=https://your-backend.railway.app</pre><p>One security note to consider before going live: the backend currently has allow_origins=[&quot;*”], which lets any website call your API from a browser. Remember to restrict it to your frontend domain in production as follows:</p><pre>allow_origins=[&quot;https://museum-dashboard.vercel.app&quot;]</pre><h4><strong>Would I do it differently with real data?</strong></h4><p>Building on synthetic data is a great way to develop and validate a system end-to-end, but I want to be honest about where the assumptions break down.</p><ol><li><strong>Data quality is the hard part — </strong>Synthetic data is clean by construction. Every day has exactly one record, all fields are populated, and the relationships between variables are consistent because I programmed them to be. Real visitor logs are messy. A real data pipeline would encounter missing days (scanners were down, the museum was closed for maintenance, or records simply weren’t captured), duplicate entries (the same group scanned through multiple times), scanner malfunctions (a door counter stuck reporting 12 visitors per hour regardless of actual traffic etc. These systematic errors are hard to detect without cross-validation against staff headcounts or revenue data), and inconsistent recording (different staff members classifying “school group” visits differently across years, making that feature unreliable as a historical signal). Budget significant time, often more than modelling itself for data cleaning as a sophisticated model cannot compensate for unreliable input data.</li><li><strong>Features you cannot simulate — </strong>Real venues have data signals I couldn’t credibly replicate in my experimentation such as:</li></ol><ul><li>Social media mentions — For example a viral Instagram post of a new exhibition can drive a spike in visitors 3–5 days later. Social listening APIs can quantify this.</li><li>Google search trends — search volumes for the museum’s name are a leading indicator of visit intent, available from Google Trends.</li><li>Nearby parking occupancy and public transport ridership — visitors can’t come if they can’t park or get there. Another example is the free transport in Victoria that could spike the numbers.</li><li>Competitor events — a major festival at a nearby venue can either draw visitors away (substitution) or increase foot traffic to the whole area (complementarity).</li></ul><p>Each of these would require integration work but could meaningfully improve predictions. The XGBoost model we built is good at learning from tabular signals. Therefore, adding better signals is often more valuable than tuning the model itself.</p><p><strong>3. Model retraining — </strong>A model trained today will degrade over time, a concept called <a href="https://medium.com/@anicomanesh/model-drift-identifying-and-monitoring-for-model-drift-in-machine-learning-engineering-and-0f74b2aa2fb0">model drift</a>. Normally, the patterns in the world change, but the model’s learned parameters don’t. For a museum, drift sources include a new exhibition type that attracts a different visitor demographic, a pricing change that affects visit frequency, construction nearby that reduces foot traffic for several months, or a global event (a pandemic, an economic recession) that fundamentally shifts cultural activity patterns etc.</p><p>The practical solution to such changes is a scheduled retraining pipeline. Monthly or quarterly, retrain the model on accumulated historical data (including recent data not available at initial training time), compare it against the current production model on a holdout period, and promote the new model if it performs better. A prediction log table that records inputs, outputs, and eventually actual visitor counts is the foundation for this as it gives you a growing dataset of prediction errors that reveals whether the model is drifting.</p><p><strong>4. Shadow mode before going live — </strong>Before using model predictions to make actual staffing decisions, run the model in shadow mode for several weeks. The model generates predictions silently in the background and logs them but they’re not shown to the operations team, who continue making decisions based on their existing process.</p><p>After a few weeks, compare the model’s predictions against actual visitor counts alongside the team’s existing estimates against actual visitor counts.</p><p>This serves two purposes. First, it validates that the model is actually better than the human baseline before it influences real decisions . If its not, then you need to understand why before deploying it. Second, it builds trust with the operations team. When they can see side-by-side that the model’s numbers were closer to reality on 73% of days, they stop treating it as a black box and start treating it as a useful tool.</p><p>Skipping shadow mode and deploying directly to operations is a common and costly mistake. The model might be good, but if the team doesn’t trust it, they’ll ignore it. If it makes a bad call on week one (which it will, eventually), they’ll dismiss it entirely and go back to manual estimates. Shadow mode lets you demonstrate value before you ask people to change their behaviour.</p><h4><strong>Conclusion</strong></h4><p>That’s it for now. Across these two articles, I’ve taken you through the entire process of building a visitor prediction system, from simulating realistic museum data, through feature engineering with 80+ features, training and comparing multiple models, evaluating with SHAP explainability, and deploying as a full-stack application with a FastAPI backend and React dashboard.</p><p>A few things worth noting from the two articles (Part 1 is <a href="https://medium.com/data-science-collective/stop-guessing-staffing-needs-predicting-daily-museum-visitors-before-they-arrive-part-1-336ac0ca4f60">here</a>). XGBoost wins with a MAPE of 14.0%, which is production-ready for operations planning. A 14% error on a day with 350 expected visitors means being off by about 49 people which is manageable for staffing adjustments. Seasonality and context dominate: <em>day-of-year, day-of-week</em>, and <em>contextual factors (boost count, weather, holidays)</em> outrank individual lag features in SHAP importance. The model is essentially learning that <em>“Saturdays in summer are busy; rainy Tuesdays are not.”</em> Feature engineering matters more than model selection as most of the performance gain comes from them. A well-engineered feature set with a simple model often beats a poorly-engineered one with a sophisticated model. In addition, business framing is everything. For example, traffic tier classification and revenue estimates speak louder than R² scores. An R² of 0.89 means nothing to a museum director. However, identifying a <em>“Low/Medium/High</em> staffing tier” is immediately actionable.</p><p>If you combine this temporal prediction with the spatial analytics from my <a href="https://medium.com/data-science-collective/finding-the-perfect-spot-wifi-analytics-for-premium-space-advertising-b8470dd9181b">WiFi article</a>, you have a complete visitor intelligence system. You’ll have the power to predict <em>how many </em>will come, then optimise <em>where</em> they go within the venue. This sounds like a million-dollar startup idea 🚀.</p><p>As always, all the code is open-sourced <a href="https://github.com/wandabwa2004/museum-visitor-prediction">here</a>. Clone the repo, run the notebooks in order (01 through 05), spin up the <em>backend</em> with uvicorn app.main:app --reload, and the <em>frontend </em>with npm run dev . Please feel free to adapt it for your own venue, whether it’s a museum, gallery, shopping mall, or event space.</p><p>I hope this walkthrough was useful. Don’t forget to follow me, clap for me, and leave a comment. Let me know if you’d like me to extend this to real-time prediction with streaming data or visitor segmentation (families vs solo visitors vs groups). If you want to check out my other articles, you can find them on my <a href="https://medium.com/@hermanwandabwa">profile</a>. I’m also happy to connect via <a href="https://www.linkedin.com/in/wandabwaherman/">LinkedIn</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=760498e03c4f" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science-collective/stop-guessing-staffing-needs-how-id-predict-daily-museum-visitors-before-they-arrive-part-2-760498e03c4f">From Jupyter Notebook to Full-Stack ML App with FastAPI, React, and Supabase</a> was originally published in <a href="https://medium.com/data-science-collective">Data Science Collective</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Stop Guessing Staffing Needs: Predicting Daily Museum Visitors Before They Arrive (Part 1)]]></title>
            <link>https://medium.com/data-science-collective/stop-guessing-staffing-needs-predicting-daily-museum-visitors-before-they-arrive-part-1-336ac0ca4f60?source=rss-58e995b6d0e3------2</link>
            <guid isPermaLink="false">https://medium.com/p/336ac0ca4f60</guid>
            <category><![CDATA[data-analysis]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[business]]></category>
            <dc:creator><![CDATA[Herman Wandabwa]]></dc:creator>
            <pubDate>Mon, 06 Apr 2026 01:55:46 GMT</pubDate>
            <atom:updated>2026-04-25T02:34:18.651Z</atom:updated>
            <content:encoded><![CDATA[<h3>Predicting Daily Museum Visitors for Better Staffing Decisions</h3><h4>How I built an XGBoost forecasting pipeline to turn visitor demand into staffing insight for a Melbourne museum.</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*NE_6oLcOKfSGKkwOr1CV3A.png" /><figcaption>Image generated by the author</figcaption></figure><p><strong>Author’s note:</strong> <em>This article uses independently created code and synthetic data for illustration. The methods described are standard forecasting and machine learning techniques commonly used across many application areas.</em></p><p><strong>Stuck behind a firewall. Read the article for free </strong><a href="https://hermanwandabwa.medium.com/stop-guessing-staffing-needs-predicting-daily-museum-visitors-before-they-arrive-part-1-336ac0ca4f60?source=friends_link&amp;sk=bf7b550e648beb7d5690552680531038"><strong>HERE</strong></a></p><p>It’s been a while since I wrote my last article on <a href="https://hermanwandabwa.medium.com/finding-the-perfect-spot-wifi-analytics-for-premium-space-advertising-b8470dd9181b?sk=79f7744a1237aa3f5eb63826a2e2b82e">WiFi Analytics</a>. For context, I outlined a simple process of analyzing foot traffic patterns to identify the best locations for premium advertisements or exhibitions in malls using WiFi pings. It was a graph-inspired analysis to rank the locations by reach, dwell time, and flow. Please read it here for <a href="https://hermanwandabwa.medium.com/finding-the-perfect-spot-wifi-analytics-for-premium-space-advertising-b8470dd9181b?sk=79f7744a1237aa3f5eb63826a2e2b82e">free </a>to get the whole idea. As usual, I also <a href="https://github.com/wandabwa2004">shared all the code</a> for reuse.</p><p>But here’s the thing: knowing where people go is only half the puzzle. The other half is knowing how many are likely to show up in the first place. I’m sure every mall/museum/exhibition operations manager has sleepless nights when the question <em>“How many visitors should we expect today?</em>” crosses their mind.</p><p>This is what led me to writing this article. It is a natural progression from the WiFi analytics piece and is split in two parts. Whereas the WiFi piece focused on spatial patterns, this one (Part 1) focuses on temporal prediction of visitor numbers. This is not a simple process contrary to what some people might think. I’ll try to provide a picture of how prediction of visitor numbers happens in such venues.</p><p>I also pivoted a little from the shopping/malls use cases and actually focused on the museum space and specifically the <strong>Immigration</strong> <strong>Museum </strong>in <strong>Melbourne, Australia</strong>. This venue actually exists and I chose it based on several factors. Melbourne’s Southern Hemisphere seasons, Victorian public holidays, and unpredictable weather make for an interesting prediction challenge as these are all features that will be packaged in the modelling process. In addition, and this is personal, it’s the only museum that I pass by when travelling into the city of Melbourne. Maybe that’s why the name and interest stuck.</p><p>As outlined earlier, Part 1 covers the synthetic data generation process through to feature engineering, model training and evaluation. Part 2 will cover the deployment part with a production-ready FastAPI backend and a React dashboard. All together, they form an end-to-end pipeline that anyone can leverage when building a related solution. As always, all code is open-sourced <a href="https://github.com/wandabwa2004/museum-visitor-prediction/tree/dev/notebooks">here</a>. Please feel free to clone the repo and adapt it for your own venue.</p><h4><strong>1. The Problem and Why Prediction Matters</strong></h4><p>Museums and cultural venues face daily operational challenges whereby if you staff too many people on a quiet Tuesday, then you&#39;re burning budget. On the contrary, if you staff too few on a public holiday or weekend then visitor experience suffers. The same logic applies to other related areas like ticketing, catering, exhibit rotations, etc.</p><p>A properly thought-through predictive model captures all of these factors simultaneously and outputs a number along with a confidence interval and even a traffic tier that the operations team can actually rely and act on. Therefore, with such a model in place, operations managers in such venues are likely to ask something like <em>&quot;Do we need the Low, Medium, or High staffing plan today?&quot;</em> instead of <em>&quot;how many visitors will we get?&quot;</em>. Enough of the theory. Let&#39;s jump to the interesting bits.</p><h4><strong>2. Dataset Design &amp; Simulation</strong></h4><p>The first challenge in any predictive modelling project is data. I didn’t have access to real museum visitor logs, so I opted to generate a synthetic dataset that largely reflects realistic museum visitation patterns. You can also replicate this and especially if you’re wanting to control the signal-to-noise ratio and demonstrate some methodology before deploying on messy real-world data.</p><p>I simulated a visitations dataset spanning three years (January 2023 to December 2025). This amounted to about 1,096 daily records.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vjqGBQDWbFx2bQMblOtwIg.png" /><figcaption>Visitor Counts over time</figcaption></figure><p>The greyish line in the plot is the raw daily count; the red line is the 30-day moving average that reveals the seasonal rhythm e.g., summer peaks, winter troughs, and a slight upward trend over the three years.</p><p>Each record captures the following:</p><ul><li><strong>Temporal features</strong>: <em>day of week, month, quarter, week of year</em></li><li><strong>Weather</strong>: <em>temperature (Melbourne-specific ranges), precipitation, weather type (Sunny/Partly Cloudy/Cloudy/Rainy)</em></li><li><strong>Calendar events</strong>: <em>Victorian public holidays (Australia Day, Anzac Day, Melbourne Cup Day, etc.) and school holidays</em></li><li><strong>Venue events</strong>: <em>special exhibitions (9 across 3 years, each lasting 2–3 months), local events (F1 Grand Prix, Arts Festival), marketing campaigns, ticket promotions</em></li></ul><p>This approach might present some deviation from the real data but the key here is “realism”. For example, Melbourne’s climate is specific where January averages about 20°C while July averages 9.5°C (Southern Hemisphere, remember). Victorian public holidays like Melbourne Cup Day are usually held on the first Tuesday of November and are unique to this state.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*OavpSIGnRPi0pVypMDef-g.png" /><figcaption>Weather Impact on Visitor Numbers</figcaption></figure><p>From the data and the plot above, sunny days average noticeably higher visitor counts than rainy ones, and there’s a clear positive trend between temperature and visitor numbers. Anyone who’s experienced a Melbourne winter knows outdoor plans take a hit, and this is exactly what you’d expect for an outdoor-accessible venue in Melbourne.</p><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*ijRXrsgFiIGg5h3TbhUpPA.png" /><figcaption>Temporal Patterns in Visitor Traffic</figcaption></figure><p>Alternatively, the above plot has four views of the same temporal story: weekends (Saturday and Sunday, highlighted in red) drawing 30 to 45% more visitors than midweek; December and January peak at ~40% above average thanks to summer holidays; Q1 and Q4 dominate quarterly traffic. The heatmap in the bottom-right reveals a deeper interaction whereby a Saturday in December is the busiest cell in the entire grid.</p><p>In the simulation process, the visitor count itself is generated using <em>multiplicative factors</em> applied to a base of 250 (arbitrary number that can be changed) visitors/day as follows:</p><pre># Day of week effect<br>day_multiplier = {0: 0.90, 1: 0.85, 2: 0.88, 3: 0.92, 4: 1.00, 5: 1.30, 6: 1.25}<br>df[&#39;visitors&#39;] *= df[&#39;day_of_week&#39;].map(day_multiplier)<br><br># Seasonal effect (Southern Hemisphere)<br>season_multiplier = {<br>    1: 1.40, 2: 1.35, 3: 1.15, 4: 1.05, 5: 0.90, 6: 0.80,<br>    7: 0.75, 8: 0.85, 9: 1.00, 10: 1.10, 11: 1.20, 12: 1.45<br>}<br>df[&#39;visitors&#39;] *= df[&#39;month&#39;].map(season_multiplier)<br><br># Holiday effects<br>df.loc[df[&#39;is_public_holiday&#39;] == 1, &#39;visitors&#39;] *= 1.50<br>df.loc[df[&#39;is_school_holiday&#39;] == 1, &#39;visitors&#39;] *= 1.30<br><br># Weather impact<br>df.loc[df[&#39;weather_type&#39;] == &#39;Sunny&#39;, &#39;weather_factor&#39;] = 1.20<br>df.loc[df[&#39;weather_type&#39;] == &#39;Rainy&#39;, &#39;weather_factor&#39;] = 0.70<br>df[&#39;visitors&#39;] *= df[&#39;weather_factor&#39;] * df[&#39;temp_factor&#39;]<br><br># Event and promotion effects<br>df.loc[df[&#39;special_exhibition&#39;] == 1, &#39;visitors&#39;] *= 1.25<br>df.loc[df[&#39;local_event&#39;] == 1, &#39;visitors&#39;] *= 1.40</pre><p>There is a high chance that the base number will be different but I hope you get the point. Beyond the core multipliers, each factor has a measurable impact on visitor numbers. The chart below shows the average uplift from each binary factor. Public holidays boost attendance by ~50%, local events by ~40%, and even marketing campaigns deliver a measurable ~15% lift. Remember all these factors are hypothetical but I believe they largely represent the reality.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ucOaOCj_Yd-H9IBrFt8rGw.png" /><figcaption>Impact of Key Factors on Visitor Numbers</figcaption></figure><p>Each panel in the above plot compares average daily visitors with and without that factor active. The percentage labels show the relative uplift. Public holidays and local events dominate, but the smaller effects (marketing, promotions) are still meaningful when they stack together.</p><p>I also added 12% Gaussian noise, a 2% outlier rate (days with 1.5 to 2.5x normal traffic), and a slow growth trend (~0.01% daily). The final values are clipped between 50 and 1,200 visitors. These choices create a dataset that’s statistically rich enough to demonstrate the methodology while being reproducible. Remember, this is a simulated dataset and should not be used as a replacement for real data.</p><p>Looking at the year-over-year monthly averages, you can see the seasonal pattern is consistent across all three years, with 2025 sitting slightly above 2023 thanks to that slow growth trend baked into the simulation.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*iqJjVIncI4IwZbGEpx7_sA.png" /><figcaption>Year-over-Year Monthly Comparison</figcaption></figure><p>All three years follow the same seasonal arc. A summer peak in December to January, a winter period between June and July, and a gentle upward shift each year reflecting the long-term growth factor.</p><p>The resulting distribution shows the right-skewed shape typical of real visitor data: most days cluster between 200 and 400 visitors, with a long tail stretching past 800 on peak days.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vQIgkR7n4wwuDn_CQ52nDA.png" /><figcaption>Visitor Count Distribution</figcaption></figure><p>The histogram on the left shows the characteristic right skew (skewness ≈ 0.9), with the mean sitting above the median and a handful of high-traffic outlier days pulling the average upward. The Q-Q plot confirms the departure from normality in the tails, motivating the log transform we’ll apply during feature engineering.</p><h4><strong>3. Feature Engineering</strong></h4><p>I’ve always insisted in my previous articles that raw data can get you baseline predictions. However, feature-engineered ones are more likely to get you good predictions. Starting from ~26 raw columns, I engineered 81 features grouped into eight categories:</p><pre>Feature groups:<br>  Temporal     : 15<br>  Calendar     :  2<br>  Events       :  5<br>  Weather      :  7<br>  Season       :  4<br>  Interaction  :  7<br>  Lag          : 14<br>  Rolling      : 27</pre><p>Here are the most important ones:</p><p><strong>a) Cyclical Encoding</strong> — Why Day 0 and Day 6 Should Be “Close”</p><p>Here’s a subtle problem with temporal features. If you feed day_of_week directly into a model as values 0 to 6, the model treats Monday (0) and Sunday (6) as being far apart. In reality, they&#39;re adjacent days. The same applies to months. December (12) and January (1) are close, not opposites.</p><p>The fix is cyclical encoding using sin/cos transforms:</p><pre>df[&#39;dow_sin&#39;] = np.sin(2 * np.pi * df[&#39;day_of_week&#39;] / 7)<br>df[&#39;dow_cos&#39;] = np.cos(2 * np.pi * df[&#39;day_of_week&#39;] / 7)<br><br># Month: 1-12 -&gt; cyclical<br>df[&#39;month_sin&#39;] = np.sin(2 * np.pi * (df[&#39;month&#39;] - 1) / 12)<br>df[&#39;month_cos&#39;] = np.cos(2 * np.pi * (df[&#39;month&#39;] - 1) / 12)<br><br># Day of year: 1-365 -&gt; cyclical<br>df[&#39;doy&#39;] = df[&#39;date&#39;].dt.dayofyear<br>df[&#39;doy_sin&#39;] = np.sin(2 * np.pi * df[&#39;doy&#39;] / 365)<br>df[&#39;doy_cos&#39;] = np.cos(2 * np.pi * df[&#39;doy&#39;] / 365)</pre><p>Think of it this way: you’re placing each day on the face of a clock. Monday and Sunday end up right next to each other, and December sits beside January. Both sin and cos components are needed because sin alone can’t distinguish between two days that map to the same y-coordinate (e.g., Tuesday and Friday might have the same sinvalue, but different cos values).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/707/1*3YizfvBOmkq9Pw1lsEee6w.png" /><figcaption>Cyclical Encoding</figcaption></figure><p>On the left, days of the week are mapped onto a circle via <em>sin/cos</em>. On the right, months on the same circular encoding. Both <em>sin </em>and <em>cos</em> components are needed to uniquely identify each position on the circle.</p><p>I applied this to day-of-week, month, day-of-year, and week-of-year.</p><p><strong>b) Lag Features —</strong> “Yesterday Predicts Tomorrow”</p><p>Recent history turned out to be one of the most important predictive signals in this dataset. For example, if 400 people visited yesterday, chances are today will be closer to 400 than to 200. Therefore, I replicated the same lags at strategic intervals:</p><pre>for lag in [1, 2, 3, 14, 21, 28, 60, 90]:<br>    df[f&#39;visitors_lag_{lag}&#39;] = df[&#39;visitors&#39;].shift(lag)<br><br># Same weekday average (last 4 occurrences)<br>df[&#39;mean_last_4_same_dow&#39;] = df[[&#39;lag_7_same_dow&#39;,&#39;lag_14_same_dow&#39;,<br>                                  &#39;lag_21_same_dow&#39;,&#39;lag_28_same_dow&#39;]].mean(axis=1)</pre><p>Why these specific intervals? Lags 1 to 3 capture short-term momentum. Lag 7 captures the same-day-last-week pattern (crucial for museums where weekdays and weekends behave very differently). Lags 14, 21, and 28 capture fortnightly and monthly rhythms. Lags 60 and 90 capture seasonal trends. The same-weekday average of the last 4 weeks is particularly powerful because it answers the question: <em>“How many people typically visit on a Wednesday?”</em></p><p>The autocorrelation plot from the EDA confirms why these lag choices matter. There’s a clear spike at lag 7 (the weekly cycle) and a gradual decay that persists out to 60+ days, reflecting the seasonal structure.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-XcsVAgm8BA1pucuKFV9Qg.png" /><figcaption>Autocorrelation Function</figcaption></figure><p>The ACF shows significant autocorrelation at multiples of 7 days (the weekly rhythm), while the PACF reveals that lags 1 and 7 carry the most independent predictive information, exactly the lags we prioritised in feature engineering.</p><p><strong>c) Rolling Window Statistics</strong></p><p>Beyond point lags, I computed rolling means, standard deviations, maximums, and minimums across multiple windows:</p><pre>for window in [3, 7, 14, 30, 60, 90]:<br>    df[f&#39;rolling_mean_{window}d&#39;] = df[&#39;visitors&#39;].shift(1).rolling(window).mean()<br>    df[f&#39;rolling_std_{window}d&#39;]  = df[&#39;visitors&#39;].shift(1).rolling(window).std()<br>    df[f&#39;rolling_max_{window}d&#39;]  = df[&#39;visitors&#39;].shift(1).rolling(window).max()<br>    df[f&#39;rolling_min_{window}d&#39;]  = df[&#39;visitors&#39;].shift(1).rolling(window).min()<br><br># Exponentially Weighted Moving Average (EWMA) - (more weight on recent days)<br>df[&#39;ewma_7d&#39;]  = df[&#39;visitors&#39;].shift(1).ewm(span=7, adjust=False).mean()<br>df[&#39;ewma_14d&#39;] = df[&#39;visitors&#39;].shift(1).ewm(span=14, adjust=False).mean()<br>df[&#39;ewma_30d&#39;] = df[&#39;visitors&#39;].shift(1).ewm(span=30, adjust=False).mean()</pre><p>The shift(1) is critical here as it prevents data leakage by ensuring we only use information available before the prediction date. The EWMA variants give extra weight to recent observations, which is also useful because visitor patterns can shift gradually (e.g., after a new exhibition opens).</p><p><strong>d) Interaction Features </strong>— When Combinations Matter</p><p>Some effects compound. For example, a sunny weekend during school holidays isn’t just “sunny” + “weekend” + “school holiday”. It’s a perfect storm that drives numbers far above what any individual factor predicts. I captured this with interaction features:</p><pre>df[&#39;weekend_x_public_holiday&#39;] = df[&#39;is_weekend&#39;] * df[&#39;is_public_holiday&#39;]<br>df[&#39;exhibition_x_weekend&#39;] = df[&#39;special_exhibition&#39;] * df[&#39;is_weekend&#39;]<br>df[&#39;sunny_weekend&#39;] = ((df[&#39;weather_type&#39;] == &#39;Sunny&#39;) &amp; (df[&#39;is_weekend&#39;] == 1)).astype(int)<br>df[&#39;marketing_x_exhibition&#39;] = df[&#39;marketing_campaign&#39;] * df[&#39;special_exhibition&#39;]<br><br># Count of active boost factors<br>df[&#39;boost_count&#39;] = (<br>    df[&#39;is_public_holiday&#39;] + df[&#39;is_school_holiday&#39;] +<br>    df[&#39;special_exhibition&#39;] + df[&#39;local_event&#39;] +<br>    df[&#39;marketing_campaign&#39;] + df[&#39;ticket_promotion&#39;]<br>)</pre><p>The <em>boost_count </em>feature is a favourite of mine. It answers a simple question: <em>“How many things are happening today that attract visitors?”</em> Basically, a day with three active boost factors will almost certainly see more visitors than a day with one. The chart below quantifies this stacking effect.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/712/1*2ycNjM8aIMnnKS9ybpSqUA.png" /><figcaption>Average Visitors by Boost Count</figcaption></figure><p>The relationship is nearly monotonic in that each additional active factor adds roughly 50 to 100 visitors on average. Days with zero boost factors average around 300 visitors; days with 3+ factors routinely exceed 500.</p><p><strong>e) Melbourne Seasons (Southern Hemisphere)</strong></p><p>This might seem trivial and personal to me, but it’s the kind of thing that trips up models if you’re deriving your thought process from, say, the Northern Hemisphere. December in Melbourne is summer, not winter:</p><pre>def get_season(month):<br>    if month in [12, 1, 2]: return &#39;Summer&#39;<br>    elif month in [3, 4, 5]: return &#39;Autumn&#39;<br>    elif month in [6, 7, 8]: return &#39;Winter&#39;<br>    else: return &#39;Spring&#39;</pre><p>January visitors run ~40% above average (summer holidays), while July drops to ~25% below average (winter). Getting this wrong would confuse the model completely.</p><p><strong>f) Log Transform of the Target</strong></p><p>The visitor count distribution is right-skewed whereby most days cluster between 200 and 400, but outlier days push above 800. A log transform (<em>log1p</em>) makes this more normally distributed, which helps gradient-based models converge:</p><pre>df[&#39;visitors_log&#39;] = np.log1p(df[&#39;visitors&#39;])</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/1*0TDAPdZgtw4i8XP86IOL4Q.png" /><figcaption>Log Transformation on Visitor Numbers</figcaption></figure><p><strong>Left</strong>: the original distribution with a skewness of ~0.9 and the long right tail of peak days. <strong>Right</strong>: after <em>log1p</em>, skewness drops dramatically and the distribution becomes much closer to normal. This is the version that the models will be trained on.</p><p>This means the model trains on the log-transformed target, and predictions need to be inverse-transformed using np.expm1(). I’ll come back to this when we discuss more details on modelling and production API.</p><p>After engineering all 81 features, I checked which ones correlated most strongly with the target. Unsurprisingly, the lag and rolling features dominate the top of the list, with recent history being the single best predictor of tomorrow’s visitors as shown in the below feature correlation plot.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/712/1*aYb_ydbPFTCDgopIfLk7Qg.png" /><figcaption>Top 30 Features by Correlation with Visitor Count</figcaption></figure><p>The rolling means and EWMA features cluster at the top with correlations close to 0.6, followed by the raw lag features. Temporal and calendar features contribute meaningfully but sit in the 0.2 to 0.4 range. This hierarchy directly informed which features the models would lean on most.</p><p>After feature engineering, the longest lag (90 days) means I drop the first 90 rows of data, leaving 1,006 usable rows (still plenty for training and evaluation).</p><p><strong>4. Model Training &amp; Comparison</strong></p><p>With features ready, I trained and compared four approaches. But first, the split strategy.</p><p><strong>Train/Test Split — Respecting Time</strong></p><p>This is a crucial one. With time series data, you cannot use random train/test splits. If you randomly shuffle dates, the model will see future data during training and that’s data leakage, and your metrics are likely to lie to you.</p><p>Instead, I used the last 90 days as a hold-out test set, with everything before as training data. Something like the following:</p><pre>Train: 916 rows  (2023-04-01 → 2025-10-02)<br>Test :  90 rows  (2025-10-03 → 2025-12-31)</pre><p>For cross-validation during training, I used <em>TimeSeriesSplit </em>with 5 folds, which creates expanding windows that always train on past data and validate on future data. At least no cheating here.</p><p><strong>The Models</strong></p><p>Before reaching for sophisticated algorithms, I needed to answer a basic question: <em>How good can one get without any machine learning at all?</em> That’s what baselines are for and is a good practice. This is the same thought process that I applied here. They’re deliberately simple prediction strategies that use just arithmetics and no learned patterns. If your fancy model can’t beat these, then it’s not earning its complexity.</p><p>For the <strong>Baselines, </strong>I used three naive approaches, each slightly more informed than the last:</p><ul><li><strong>Global mean:</strong> predict every day as the average of the training set (~370 visitors). This ignores everything: day of week, weather, holidays. It’s the “shrug” prediction, but it anchors the bottom of our performance range.</li><li><strong>Last week’s visitors (lag 7):</strong> This predicts today’s visitors as whatever happened on the same day last week. This captures the weekly rhythm (Saturdays are busy, Tuesdays are quiet) but nothing else and it can’t adapt to weather changes, holidays, or trends.</li><li><strong>7-day moving average:</strong> This predicts today as the average of the past 7 days. This approach is slightly smoother than the lag-7 approach because it not only dampens single-day spikes, but also has no awareness of why visitor counts change. Basically, it just follows the recent trend.</li></ul><p>The key limitation of all three baselines is that they use a single signal. They can’t combine information. For example, they don’t know that a sunny Saturday during school holidays with a special exhibition is a fundamentally different day from a rainy Tuesday in winter.</p><p>That’s where the ML models come in. They learn to weigh dozens of features simultaneously and capture non-linear interactions between them. I made use of the following algorithms:</p><ol><li><strong>Random Forest </strong>— an ensemble of 300 decision trees, each trained on a random subset of features (max_features=&#39;sqrt&#39;) and data. Each tree independently learns rules like <em>&quot;if it&#39;s a weekend AND there&#39;s a special exhibition AND temperature &gt; 20°C, expect high traffic.&quot;</em> The final prediction averages across all 300 trees, which smooths out individual tree quirks. The min_samples_leaf=5 constraint prevents trees from memorising noise in the training data. Like XGBoost, it was trained on the log-transformed target.</li><li><strong>XGBoost </strong>with 500 sequential estimators where each new tree specifically targets the errors the previous trees got wrong (gradient boosting). Whereas Random Forest builds trees independently and averages them, XGBoost builds them one at a time, with each tree focusing on the residual mistakes. The learning rate of 0.05 keeps each tree’s contribution small, which requires more trees but produces a more robust model. I trained it on log1p(visitors) rather than raw visitor counts. This compresses the scale so the model doesn&#39;t disproportionately chase high-traffic outlier days at the expense of typical ones.</li><li><strong>Prophet</strong> — This is Facebook’s time series model that takes a fundamentally different approach. Instead of learning from engineered features, it decomposes the time series into <em>trend + seasonality + holiday effects</em> and fits each component separately. I configured it with <em>multiplicative seasonality </em>(because seasonal swings scale with the baseline level) and added external regressors for holidays, exhibitions, and weather. It’s purpose-built for time series but less flexible at capturing the kind of complex feature interactions that tree-based models handle naturally.</li></ol><p>Here’s the XGBoost training code:</p><pre>xgb_model = xgb.XGBRegressor(<br>    n_estimators=500,<br>    learning_rate=0.05,<br>    max_depth=6,<br>    subsample=0.8,<br>    colsample_bytree=0.8,<br>    min_child_weight=5,<br>    reg_alpha=0.1,<br>    reg_lambda=1.0,<br>    random_state=42,<br>    n_jobs=-1,<br>    verbosity=0<br>)<br># Train on log-transformed target<br>xgb_model.fit(X_train, y_train_log)<br># Predictions need inverse transform<br>xgb_preds = np.expm1(xgb_model.predict(X_test))</pre><p>The code for the other two approaches can be found <a href="https://github.com/wandabwa2004/museum-visitor-prediction/blob/dev/notebooks/04_modeling.ipynb">here</a>.</p><h4><strong>Results</strong></h4><p>Below are the results:</p><pre>| Model               | MAE   | RMSE  |   R²  |  MAPE |<br>|---------------------|-------|-------|------ |-------|<br>| XGBoost             | 69.9  | 90.7  | 0.62  | 14.0% |<br>| Prophet             | 74.3  | 92.7  | 0.60  | 16.3% |<br>| Random Forest       | 78.4  | 103.7 | 0.50  | 16.3% |<br>| Baseline (7d MA)    | 119.3 | 152.7 | -0.09 | 26.5% |<br>| Baseline (last week)| 125.0 | 162.5 | -0.24 | 27.4% |<br>| Baseline (mean)     | 143.9 | 189.8 | -0.68 | 26.2% |</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*03Vs-lO5AXMPUhPXmySUBw.png" /><figcaption>Model Performance Comparison on Test Set</figcaption></figure><p>From the results above, XGBoost was the winner across all metrics. A MAPE of 14.0% means the model is typically off by about 14%. What this typically means is that for a museum expecting 350 visitors, that’s roughly plus or minus 50 people. This is well within the range that operations teams can plan around.</p><p>Note that the negative R² values for the baselines don’t indicate a bug. A negative R² simply means the model performs worse than just predicting the mean of the test set, which confirms these baselines are too simplistic to be useful.</p><p>The improvement over baselines is substantial. The best baseline (7-day moving average) has an MAE of 119.3 visitors, while XGBoost achieves 69.9, a 41% reduction in error. That gap is the difference between useful predictions and educated guesses.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6Mt6CcvmsM3tVLvx1w6vJg.png" /><figcaption>Actual Vs Predicted Visitor Numbers</figcaption></figure><p>XGBoost tracks the actual visitor pattern most closely, capturing both the weekly rhythm and the December holiday surge. Random Forest follows the shape but tends to undershoot peaks. Prophet captures the broad seasonal trend but smooths over the day-to-day variability that the tree-based models pick up. It performs surprisingly well given it doesn’t use the full feature set. Its strength is in capturing the yearly and weekly seasonality automatically. However, it can’t leverage the 81 engineered features as effectively as XGBoost.</p><p><strong>5. Model Evaluation &amp; SHAP Explainability</strong></p><p>Raw metrics tell you how well the model performs. However, evaluation tells you where it struggles and why.</p><p><strong>Residual Analysis</strong></p><p>The four-panel residual analysis below gives a comprehensive picture of the model’s error behaviour.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qpJVUQSuvTMeWBOoBbOWAw.png" /><figcaption>Residual Analysis</figcaption></figure><p>Top-left: residuals over time, the green and red shading shows where the model over- and under-predicts. Top-right: the residual distribution is roughly normal and centred near zero. Bottom-left: the predicted-vs-actual scatter follows the diagonal reasonably well, though the model underestimates the highest-traffic days. Bottom-right: percentage errors cluster within plus or minus 20%, with a slight negative skew.</p><p>The model has a slight negative bias of about -25 visitors/day, meaning it tends to under-predict on average. This is conservative and it’s actually better to under-promise and over-deliver when it comes to staffing. The residual distribution is roughly normal and centred near zero, which is what we want.</p><p><strong>Error Breakdown</strong></p><p>The model doesn’t perform equally well across all conditions:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GfEnPJpQGHmHhX2kLkntPA.png" /><figcaption>Error Breakdown Analysis</figcaption></figure><p>Top row: MAE by day of week (weekends show higher error), month, and season. Bottom row: MAE split by public holidays, special exhibitions, and weekends. Public holidays show the highest MAE as they are the outlier days with unpredictable spikes.</p><ul><li><strong>Weekdays vs weekends</strong>: Higher error on weekends (more variable traffic)</li><li><strong>Public holidays</strong>: The highest MAE. These are outlier days with unpredictable spikes</li><li><strong>Special exhibitions</strong>: Also higher error, but less than public holidays</li><li><strong>Seasons</strong>: Summer and spring (high-traffic seasons) have larger absolute errors, though percentage errors remain similar</li></ul><p>This is expected. Extreme days are harder to predict because they’re driven by unique combinations of factors. A model that’s <em>“off by 70 visitors”</em> on a 300-visitor day is performing differently than one that’s <em>“off by 70”</em> on a 900-visitor day.</p><p><strong>SHAP Explainability</strong></p><p>I used <strong>SHAP </strong>(SHapley Additive exPlanations) to understand what drives the model’s predictions. This is crucial for two reasons: (1) <em>it validates that the model is learning sensible patterns</em>, and (2) <em>it gives you ammunition for explaining predictions to non-technical stakeholders</em>.</p><pre>explainer = shap.TreeExplainer(model)<br>shap_values = explainer.shap_values(X_shap)<br><br>shap.summary_plot(shap_values, X_shap, plot_type=&#39;dot&#39;, max_display=20)</pre><p>The results confirm what you’d expect:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/763/1*_i0T7B5pFxOEud_Q43IlOA.png" /><figcaption>SHAP Values</figcaption></figure><p>The beeswarm plot shows each feature’s impact on predictions. Each dot is a single prediction with colour representing the feature value (red = high, blue = low). Seasonality and day-of-week features dominate the top, followed by contextual factors like precipitation, temperature, and boost_count. Lag and rolling features contribute meaningfully but sit in the middle of the ranking.</p><p>I also generated a waterfall plot for the single worst prediction day. This is the day where the model’s prediction was furthest from the actual visitor count. The waterfall plot as shown below breaks down each feature’s contribution, showing which ones pushed the prediction up or down and by how much. This lets you diagnose why the model got it wrong: did it miss because something happened that day that no feature could capture (e.g., a surprise road closure), or because the feature values were an unusual combination it hadn’t seen during training (e.g., a sunny public holiday in winter)?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/868/1*pdavteBSqN4JGASj5js7pQ.png" /><figcaption>Single prediction explanation for the worst prediction day</figcaption></figure><p><strong>Prediction Intervals</strong></p><p>A point prediction alone isn’t enough. Operations teams need to know the range of uncertainty. Therefore, I computed bootstrap prediction intervals using training residuals:</p><pre># Bootstrap prediction intervals<br>n_bootstrap = 500<br>bootstrap_preds = np.array([<br>    preds + np.random.choice(train_residuals, size=len(preds), replace=True)<br>    for _ in range(n_bootstrap)<br>])<br><br>lower_80 = np.percentile(bootstrap_preds, 10, axis=0)<br>upper_80 = np.percentile(bootstrap_preds, 90, axis=0)<br>lower_95 = np.percentile(bootstrap_preds, 2.5, axis=0)<br>upper_95 = np.percentile(bootstrap_preds, 97.5, axis=0)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qSozWI70tws0Pj5aLtO7Gg.png" /><figcaption>Predictions with Bootstrap Intervals</figcaption></figure><p>The blue line tracks the model’s predictions against the actual visitor counts (black). The darker shaded band is the 80% interval; the lighter band is 95%. The actual values fall within these bands at close to the expected rate and this is what well-calibrated uncertainty looks like.</p><p>The intervals are well-calibrated whereby the 80% interval captures approximately 80% of actual values, and the 95% interval captures approximately 95%. This is the kind of result that builds trust with stakeholders: <em>“We predict 380 visitors, with 80% confidence it’ll be between 310 and 450.”</em></p><p>It’s worth noting that this bootstrap method assumes residuals are identically distributed across all conditions. In practice, as we saw in the error breakdown, weekends and public holidays have higher error than quiet weekdays. A more refined approach would compute condition-specific intervals, but for an initial deployment this provides a useful and honest range.</p><p><strong>6. Stakeholders Engagement</strong></p><p>I know from experience that metrics like MAE and RMSE don’t resonate with stakeholders who may be the museum directors or the day-to-day operations people in this context. What resonates is business impact. I framed the model’s performance in two ways.</p><p><strong>Traffic Tier Classification</strong></p><p>Instead of reporting raw numbers, the model classifies each day into a traffic tier:</p><pre>- Low: &lt; 250 visitors<br>- Medium: 250–450 visitors<br>- High: &gt; 450 visitors</pre><p>Each tier could easily map to a staffing plan and security/safety deployment.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*VyExuam35iqVl90KZq0J0Q.png" /><figcaption>Business Impact Analysis</figcaption></figure><p>Left: the confusion matrix shows how often the model’s predicted tier matches reality. The diagonal dominance means operations teams can trust the signal. Right: revenue prediction errors (at $25/ticket) cluster tightly around zero, with most days falling within plus or minus $2,000 AUD.</p><p>The model’s tier accuracy gives the operations team a reliable signal: if the model says “High traffic,” they can confidently deploy the larger team.</p><p><strong>Revenue Forecasting</strong></p><p>At say $25 AUD per ticket, though most days the museum is free, every visitor prediction translates directly to revenue. The model’s mean daily revenue error gives the finance team a concrete number to work with for budgeting and forecasting. For a data scientist, it’s worth celebrating MAE = 69.9 visitors. However, a museum director will celebrate when they hear: <strong><em>“we can predict tomorrow’s ticket revenue within plus or minus $1,750 AUD.”</em></strong></p><p>That wraps up Part 1 and I admit it was quite long. I promise to shorten Part 2. We’ve gone from the business problem through data generation, feature engineering, model training, evaluation, and business framing. The XGBoost model with 81 engineered features achieves a MAPE of 14.0%, production-ready for operations planning.</p><p>In <a href="https://hermanwandabwa.medium.com/stop-guessing-staffing-needs-how-id-predict-daily-museum-visitors-before-they-arrive-part-2-760498e03c4f">Part 2</a>, I’ll take this trained model and deploy it as a full-stack application: a FastAPI backend that serves predictions through a REST API, a React + Tailwind dashboard for the operations team, and a Supabase database for prediction logging. I’ll also cover deployment considerations and what I’d do differently with real data. This is something that I have never written about so I’m also very excited.</p><p>As always, all code is open-sourced <a href="https://github.com/wandabwa2004/museum-visitor-prediction/tree/dev/notebooks">here</a>. Please feel free to clone it and adapt it for your own venue. If you found this useful, a clap or comment goes a long way. You can find my other articles on my <a href="https://medium.com/@hermanwandabwa">profile</a>, and I’m always happy to connect via <a href="https://www.linkedin.com/in/wandabwaherman/">LinkedIn</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=336ac0ca4f60" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science-collective/stop-guessing-staffing-needs-predicting-daily-museum-visitors-before-they-arrive-part-1-336ac0ca4f60">Stop Guessing Staffing Needs: Predicting Daily Museum Visitors Before They Arrive (Part 1)</a> was originally published in <a href="https://medium.com/data-science-collective">Data Science Collective</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Finding the Perfect Spot: WiFi Analytics for “Premium” Space Advertising]]></title>
            <link>https://medium.com/data-science-collective/finding-the-perfect-spot-wifi-analytics-for-premium-space-advertising-b8470dd9181b?source=rss-58e995b6d0e3------2</link>
            <guid isPermaLink="false">https://medium.com/p/b8470dd9181b</guid>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[analytics]]></category>
            <category><![CDATA[spatial-computing]]></category>
            <category><![CDATA[data-analysis]]></category>
            <category><![CDATA[advertising]]></category>
            <dc:creator><![CDATA[Herman Wandabwa]]></dc:creator>
            <pubDate>Fri, 02 Jan 2026 01:49:55 GMT</pubDate>
            <atom:updated>2026-01-04T23:12:44.785Z</atom:updated>
            <content:encoded><![CDATA[<h4>A graph-inspired approach to identifying high-value advertising locations using anonymous foot traffic data</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8dxFyhr32Ve5w5K8ZrDVog.png" /><figcaption>Image generated by author</figcaption></figure><p><a href="https://hermanwandabwa.medium.com/finding-the-perfect-spot-wifi-analytics-for-premium-space-advertising-b8470dd9181b?source=friends_link&amp;sk=79f7744a1237aa3f5eb63826a2e2b82e"><strong>Stuck behind a firewall. Read the article for free here</strong></a></p><p>It’s my fourth New Year here in Australia, a day I cherish greatly. Normally, I take this time to relax and reflect on the events of the previous year. However, this time around, I found myself reflecting deeply on my former professional life. Before working in the finance space, I actually worked in spatial analytics for a company that predicted visitor numbers in museums and exhibitions based on various factors. It was one of those roles where you spend half your time explaining to curators (at least the product owners did) why ‘the spot near the bathrooms’ in a mall or museum isn’t actually premium real estate, despite the foot traffic. The other half was spent building models that some clients didn’t fully trust until they saw the results in action. It was fascinating work, and I genuinely enjoyed it.</p><p>Fast forward to today, and I realized I hadn’t touched spatial analytics in such a long time. I’ll still attribute this to being busy riding the Generative AI (Gen AI) wave, plus the usual modeling work that fills my days. I managed to write a few Gen AI articles and some predictive modeling pieces to justify the time spent on it. Many of them can be found <a href="https://medium.com/@hermanwandabwa">here</a>. Please read them for free and share your feedback. As always, all code accompanying each article is open-sourced, so feel free to re-implement it for your own problems.</p><p>For this reason, I decided to revisit spatial analytics once again, but with a twist. For those who may not be familiar with this use case, it’s simply the process of analyzing foot traffic patterns to identify the best locations for premium advertisements or exhibitions in malls and museums. For example, a jewelry shop or museum wanting to place a high-value exhibition needs to understand which locations actually deliver value. To accomplish this, they must understand not just how many people pass through a location, but how they move, where they linger, and which paths they take. Therefore, this will always be venue-specific and cannot be generalized. Additionally, they should have a way of measuring the success of their placement decisions. Don’t worry if this sounds complicated. I’ll break it down for you in the simplest terms possible with a relevant example.</p><h3>The Problem with “Premium” Space</h3><p>Here’s the uncomfortable truth: most “premium advertising space” decisions in malls/museums/exhibitions are still made the old-fashioned way. Someone senior walks the venue, points at a corner, and says, “This feels busy.” And to be fair, sometimes that intuition is correct. But it’s also the reason why premium placements become political. The leasing team thinks one spot is best, the tenants think another spot is best, and the marketing team is somewhere in the middle trying to justify the budget.</p><p>Now, if your venue offers free WiFi and (anonymously) tracks movement, you already have what many businesses dream of: behavioral data at scale. The trick is using it in a way that is statistically defensible, explainable to non-technical stakeholders, and respectful of privacy.</p><h3>Why This Approach is Different</h3><p>I’ll be honest: when I worked in spatial analytics, most of the analysis I saw treated locations as independent silos. Basically, all that was done was to count visitors, measure dwell time and call it a day. But that approach misses something fundamental. The real value of a location depends on:</p><ol><li>The <strong>network structure</strong> — the zones that connect to it</li><li><strong>Flow patterns</strong> — whether it’s a destination or a pass-through</li><li><strong>Movement dynamics </strong>— how people actually navigate the space</li><li><strong>Temporal patterns </strong>— when traffic peaks occur</li></ol><p>This is why I built my solution around a graph-inspired process rather than simple aggregation. Here, the venue e.g., museum or mall is modeled as a network of connected zones, and visitor movement is analyzed as paths through this network. This approach captures something that simple traffic counts miss: a location’s value isn’t just about how many people stop there, but also about being on high-traffic paths and serving as a hub.</p><p>Think of it this way: a corridor with low dwell time but massive throughflow might be more valuable than a quiet corner where people linger. Both matter, but for different reasons and to different advertisers.</p><h3>Dataset Structure &amp; The WiFi Pings Reality</h3><p>I came up with the below approach that would generally be used by venues with WiFi tracking capabilities. However, the implementation is not exhaustive and should <strong>NOT</strong> be adopted as is in production environments. Its subdivided as follows:</p><h4>a) Venue Graph Structure</h4><p>The first step is modeling the physical space as a graph where nodes represent zones (entrances, corridors, food courts, galleries etc.) and edges represent walkable connections between the zones. Therefore, each zone has coordinates (x, y) for spatial visualization. This is fundamentally different from treating each location independently.</p><p>Below is the <strong>exact code I used</strong> to generate a mock venue graph (zones + edges) and simulate WiFi pings.</p><pre>import numpy as np<br>import pandas as pd<br><br>def generate_mock_venue(seed=7):<br>    &quot;&quot;&quot;<br>    Creates:<br>      - zones (a simplified mall/museum graph)<br>      - wifi pings: device_id, timestamp, zone_id, x, y, rssi(received signal strength indicator)<br>      - optional POIs / candidate ad locations<br>    &quot;&quot;&quot;<br>    rng = np.random.default_rng(seed)<br><br>    # ----- Venue zones (nodes) -----<br>    # Think of each zone as a &quot;Wi-Fi localization cell&quot; or area (corridor section / gallery / food court).<br>    zones = pd.DataFrame([<br>        {&quot;zone_id&quot;: &quot;ENT_N&quot;, &quot;name&quot;: &quot;North Entrance&quot;, &quot;type&quot;: &quot;entrance&quot;, &quot;x&quot;: 10, &quot;y&quot;: 90},<br>        {&quot;zone_id&quot;: &quot;ENT_S&quot;, &quot;name&quot;: &quot;South Entrance&quot;, &quot;type&quot;: &quot;entrance&quot;, &quot;x&quot;: 10, &quot;y&quot;: 10},<br>        {&quot;zone_id&quot;: &quot;COR_1&quot;, &quot;name&quot;: &quot;Main Corridor 1&quot;, &quot;type&quot;: &quot;corridor&quot;, &quot;x&quot;: 30, &quot;y&quot;: 70},<br>        {&quot;zone_id&quot;: &quot;COR_2&quot;, &quot;name&quot;: &quot;Main Corridor 2&quot;, &quot;type&quot;: &quot;corridor&quot;, &quot;x&quot;: 50, &quot;y&quot;: 50},<br>        {&quot;zone_id&quot;: &quot;COR_3&quot;, &quot;name&quot;: &quot;Main Corridor 3&quot;, &quot;type&quot;: &quot;corridor&quot;, &quot;x&quot;: 70, &quot;y&quot;: 30},<br>        {&quot;zone_id&quot;: &quot;FOOD&quot;,  &quot;name&quot;: &quot;Food Court&quot;, &quot;type&quot;: &quot;food&quot;, &quot;x&quot;: 80, &quot;y&quot;: 80},<br>        {&quot;zone_id&quot;: &quot;ANCH_A&quot;,&quot;name&quot;: &quot;Anchor Store A&quot;, &quot;type&quot;: &quot;anchor&quot;, &quot;x&quot;: 90, &quot;y&quot;: 60},<br>        {&quot;zone_id&quot;: &quot;ANCH_B&quot;,&quot;name&quot;: &quot;Anchor Store B&quot;, &quot;type&quot;: &quot;anchor&quot;, &quot;x&quot;: 90, &quot;y&quot;: 20},<br>        {&quot;zone_id&quot;: &quot;GALL_1&quot;,&quot;name&quot;: &quot;Gallery 1&quot;, &quot;type&quot;: &quot;gallery&quot;, &quot;x&quot;: 40, &quot;y&quot;: 85},<br>        {&quot;zone_id&quot;: &quot;GALL_2&quot;,&quot;name&quot;: &quot;Gallery 2&quot;, &quot;type&quot;: &quot;gallery&quot;, &quot;x&quot;: 60, &quot;y&quot;: 85},<br>        {&quot;zone_id&quot;: &quot;REST&quot;,  &quot;name&quot;: &quot;Restrooms&quot;, &quot;type&quot;: &quot;utility&quot;, &quot;x&quot;: 65, &quot;y&quot;: 10},<br>        {&quot;zone_id&quot;: &quot;SEAT&quot;,  &quot;name&quot;: &quot;Seating / Atrium&quot;, &quot;type&quot;: &quot;amenity&quot;, &quot;x&quot;: 55, &quot;y&quot;: 55},<br>    ])<br><br> <br>    # zone adjacency edges - think of this as your &quot;walkable graph&quot; approximation. Remember this is hypothetical!<br>    edges = [<br>        (&quot;ENT_N&quot;, &quot;COR_1&quot;), (&quot;ENT_S&quot;, &quot;COR_3&quot;),<br>        (&quot;COR_1&quot;, &quot;COR_2&quot;), (&quot;COR_2&quot;, &quot;COR_3&quot;),<br>        (&quot;COR_2&quot;, &quot;SEAT&quot;),<br>        (&quot;COR_1&quot;, &quot;GALL_1&quot;), (&quot;GALL_1&quot;, &quot;GALL_2&quot;), (&quot;GALL_2&quot;, &quot;FOOD&quot;),<br>        (&quot;COR_2&quot;, &quot;ANCH_A&quot;), (&quot;COR_3&quot;, &quot;ANCH_B&quot;),<br>        (&quot;COR_3&quot;, &quot;REST&quot;),<br>        (&quot;FOOD&quot;, &quot;ANCH_A&quot;),<br>    ]<br><br>    # Convert to adjacency map for simulation<br>    adj = {}<br>    for a, b in edges:<br>        adj.setdefault(a, []).append(b)<br>        adj.setdefault(b, []).append(a)<br><br>    # ----- Simulate device journeys -----<br>    n_devices = 1500<br>    # sessions per device (some repeat visitors)<br>    sessions_per_device = rng.choice([1, 2, 3], size=n_devices, p=[0.72, 0.22, 0.06])<br><br>    # Campaign day timeline<br>    start = pd.Timestamp(&quot;2025-12-01 09:00:00&quot;)<br>    end   = pd.Timestamp(&quot;2025-12-01 21:00:00&quot;)<br>    total_minutes = int((end - start).total_seconds() // 60)<br><br>    # Entry zone bias: more people use south entrance<br>    entry_zones = [&quot;ENT_N&quot;, &quot;ENT_S&quot;]<br>    entry_p = [0.4, 0.6]<br><br>    # Zone &quot;stickiness&quot; (higher -&gt; more dwell)<br>    stickiness = {<br>        &quot;FOOD&quot;: 0.65, &quot;SEAT&quot;: 0.55, &quot;GALL_1&quot;: 0.45, &quot;GALL_2&quot;: 0.45,<br>        &quot;ANCH_A&quot;: 0.35, &quot;ANCH_B&quot;: 0.35,<br>        &quot;REST&quot;: 0.25,<br>        &quot;COR_1&quot;: 0.15, &quot;COR_2&quot;: 0.15, &quot;COR_3&quot;: 0.15,<br>        &quot;ENT_N&quot;: 0.10, &quot;ENT_S&quot;: 0.10<br>    }<br><br>    def simulate_session(device_id: str):<br>        # start time<br>        t0 = start + pd.Timedelta(minutes=int(rng.integers(0, total_minutes)))<br>        # session length (minutes)<br>        duration = int(rng.normal(75, 25))<br>        duration = int(np.clip(duration, 20, 180))<br>        t1 = t0 + pd.Timedelta(minutes=duration)<br><br>        # pick entry<br>        zone = rng.choice(entry_zones, p=entry_p)<br><br>        # simulate movement in 1-minute steps, later we can downsample to &quot;pings&quot;<br>        times = pd.date_range(t0, t1, freq=&quot;1min&quot;)<br>        zones_path = []<br>        for _ in times:<br>            zones_path.append(zone)<br>            # decide whether to stay<br>            if rng.random() &lt; stickiness.get(zone, 0.2):<br>                continue<br>            # else move to a neighbor<br>            nbrs = adj.get(zone, [])<br>            if not nbrs:<br>                continue<br>            # slight bias toward anchors/food places - chances are that stays are longer in such locations <br>            weights = []<br>            for n in nbrs:<br>                w = 1.0<br>                if n in [&quot;FOOD&quot;, &quot;ANCH_A&quot;, &quot;ANCH_B&quot;]:<br>                    w *= 1.35<br>                if n in [&quot;GALL_1&quot;, &quot;GALL_2&quot;]:<br>                    w *= 1.15<br>                weights.append(w)<br>            weights = np.array(weights) / np.sum(weights)<br>            zone = rng.choice(nbrs, p=weights)<br><br>        df = pd.DataFrame({&quot;device_id&quot;: device_id, &quot;timestamp&quot;: times, &quot;zone_id&quot;: zones_path})<br><br>        # add (x,y) with noise around zone centers<br>        zmap = zones.set_index(&quot;zone_id&quot;)[[&quot;x&quot;, &quot;y&quot;]].to_dict(&quot;index&quot;)<br>        df[&quot;x&quot;] = df[&quot;zone_id&quot;].map(lambda z: zmap[z][&quot;x&quot;] + rng.normal(0, 2.5))<br>        df[&quot;y&quot;] = df[&quot;zone_id&quot;].map(lambda z: zmap[z][&quot;y&quot;] + rng.normal(0, 2.5))<br><br>        # add RSSI proxy (more negative is weaker)<br>        # assume closer to AP in zone -&gt; stronger; we just simulate plausible values<br>        df[&quot;rssi&quot;] = -35 - (np.abs(rng.normal(0, 7, size=len(df))) + rng.uniform(0, 10, size=len(df)))<br>        return df<br><br>    all_sessions = []<br>    for i in range(n_devices):<br>        device_id = f&quot;d_{i:05d}&quot;<br>        for s in range(sessions_per_device[i]):<br>            all_sessions.append(simulate_session(device_id))<br><br>    traces = pd.concat(all_sessions, ignore_index=True)<br><br>    # ----- Downsample to &quot;wifi pings&quot; -----<br>    # Real Wi-Fi localization is irregular; simulate that by sampling each device ~ every 2-5 minutes.<br>    traces[&quot;minute&quot;] = traces[&quot;timestamp&quot;].dt.floor(&quot;min&quot;)<br>    # keep ping with probability dependent on random interval<br>    keep = np.zeros(len(traces), dtype=bool)<br>    for dev, g in traces.groupby(&quot;device_id&quot;, sort=False):<br>        step = int(rng.integers(2, 6))<br>        keep_idx = g.iloc[::step].index<br>        keep[keep_idx] = True<br>    pings = traces.loc[keep].drop(columns=[&quot;minute&quot;]).reset_index(drop=True)<br><br>    # Candidate ad/exhibit locations: pick zones that can host placements<br>    candidates = zones[zones[&quot;type&quot;].isin([&quot;corridor&quot;,&quot;food&quot;,&quot;amenity&quot;,&quot;gallery&quot;])].copy()<br>    candidates[&quot;is_candidate&quot;] = True<br><br>    return zones, pd.DataFrame(edges, columns=[&quot;from_zone&quot;,&quot;to_zone&quot;]), pings, candidates<br><br>zones, edges, pings, candidates = generate_mock_venue(seed=7)<br><br>print(&quot;zones:&quot;, zones.shape)<br>print(&quot;edges:&quot;, edges.shape)<br>print(&quot;pings:&quot;, pings.shape)<br>print(&quot;candidates:&quot;, candidates.shape)<br><br>pings.head()</pre><p>By encoding the graph structure, we can measure connectivity and flow, not just presence. This is crucial for understanding how zones relate to each other.</p><h4>b) WiFi Ping Data</h4><p>Real WiFi tracking data is irregular and sparse, making it quite messy to deal with. Devices don’t ping at consistent intervals. Connection quality varies. People also turn WiFi on and off.</p><p>In the code above, I simulated movement in 1-minute steps and then <strong>downsampled</strong> it into irregular pings (every ~2–5 minutes per device). That last part is important because it mimics what WiFi data is like under normal circumstances.</p><p>The “stickiness” is another crucial parameter. Food courts naturally have higher stickiness (people sit and eat). On the other hand, places like corridors have low stickiness (people pass through). This creates realistic heterogeneity in the data and yes, it’s based on patterns I observed in museums and exhibition spaces.</p><h4>c) Session Reconstruction from Irregular Pings</h4><p>As mentioned above, WiFi pings are irregular, so I needed to reconstruct them to coherent sessions. I used time gaps to identify when a device has left and returned. This is how actual WiFi analytics systems work.</p><p>Here’s the exact sessionisation function I used:</p><pre>def build_sessions(pings: pd.DataFrame, inactivity_gap_minutes=15):<br>    &quot;&quot;&quot;<br>    Convert irregular pings into sessions per device based on time gaps.<br>    &quot;&quot;&quot;<br>    df = pings.sort_values([&quot;device_id&quot;,&quot;timestamp&quot;]).copy()<br>    df[&quot;prev_ts&quot;] = df.groupby(&quot;device_id&quot;)[&quot;timestamp&quot;].shift(1)<br>    df[&quot;gap_min&quot;] = (df[&quot;timestamp&quot;] - df[&quot;prev_ts&quot;]).dt.total_seconds() / 60.0<br>    # new session if first ping or big inactivity gap<br>    df[&quot;new_sess&quot;] = (df[&quot;prev_ts&quot;].isna()) | (df[&quot;gap_min&quot;] &gt; inactivity_gap_minutes)<br>    df[&quot;session_id&quot;] = df.groupby(&quot;device_id&quot;)[&quot;new_sess&quot;].cumsum()<br>    df[&quot;session_key&quot;] = df[&quot;device_id&quot;] + &quot;_s&quot; + df[&quot;session_id&quot;].astype(str)<br>    return df.drop(columns=[&quot;prev_ts&quot;,&quot;gap_min&quot;,&quot;new_sess&quot;,&quot;session_id&quot;])</pre><p>You can’t assume continuous tracking and sessions need to be inferred from sparse observations. Therefore, I added a 15-minute inactivity threshold as a reasonable default. However, you should tune this value based on your venue’s typical visit duration.</p><h3>The Metrics That Actually Matter</h3><p>This is where this approach gets a bit interesting. I put together four categories of metrics that overall paint a complete picture of each location’s advertising value. These aren’t random choices but what advertisers actually care about when evaluating placement. To keep it clean, I calculate most zone metrics in one function, then calculate flow metrics separately (because that depends on transitions).</p><h4>1. Reach Metrics</h4><p>This is basic but <strong>essential</strong>. More eyeballs equals more potential impressions. This is the foundation of any advertising value calculation.</p><p>Here’s the exact metric function I used (reach + dwell + repeat + peakiness):</p><pre>def compute_zone_metrics(sessionized: pd.DataFrame, zones: pd.DataFrame, time_bucket=&quot;15min&quot;):<br>    &quot;&quot;&quot;<br>    Computes:<br>      - unique_reach: unique devices per zone<br>      - dwell_minutes: approx dwell per device-zone using time deltas<br>      - repeat_rate: how often devices return to the zone within a session<br>      - peakiness: how concentrated the traffic is in time (good for timed campaigns)<br>    &quot;&quot;&quot;<br>    df = sessionized.sort_values([&quot;session_key&quot;,&quot;timestamp&quot;]).copy()<br><br>    # time delta to next ping within session; cap to avoid huge dwell from sparse pings<br>    df[&quot;next_ts&quot;] = df.groupby(&quot;session_key&quot;)[&quot;timestamp&quot;].shift(-1)<br>    df[&quot;dt_min&quot;] = (df[&quot;next_ts&quot;] - df[&quot;timestamp&quot;]).dt.total_seconds() / 60.0<br>    df[&quot;dt_min&quot;] = df[&quot;dt_min&quot;].clip(lower=0, upper=8)  # cap dwell contribution<br><br>    # reach<br>    reach = df.groupby(&quot;zone_id&quot;)[&quot;device_id&quot;].nunique().rename(&quot;unique_reach&quot;)<br><br>    # dwell time per zone (sum of dt_min)<br>    dwell = df.groupby(&quot;zone_id&quot;)[&quot;dt_min&quot;].sum().rename(&quot;dwell_minutes&quot;)<br><br>    # average dwell per device (attention proxy)<br>    dwell_per_device = (df.groupby([&quot;zone_id&quot;,&quot;device_id&quot;])[&quot;dt_min&quot;].sum()<br>                          .groupby(&quot;zone_id&quot;).mean()<br>                          .rename(&quot;avg_dwell_per_device_min&quot;))<br><br>    # repeat exposure in-session: count visits to zone within a session<br>    visits = (df.groupby([&quot;session_key&quot;,&quot;zone_id&quot;]).size()<br>                .rename(&quot;pings_in_zone&quot;)<br>                .reset_index())<br>    repeat = visits.groupby(&quot;zone_id&quot;)[&quot;pings_in_zone&quot;].apply(lambda x: (x &gt; 1).mean()).rename(&quot;repeat_rate&quot;)<br><br>    # peakiness: bucket traffic in time and compute coefficient of variation<br>    df[&quot;bucket&quot;] = df[&quot;timestamp&quot;].dt.floor(time_bucket)<br>    bucket_counts = df.groupby([&quot;zone_id&quot;,&quot;bucket&quot;])[&quot;device_id&quot;].nunique().reset_index(name=&quot;u&quot;)<br>    peakiness = bucket_counts.groupby(&quot;zone_id&quot;)[&quot;u&quot;].agg([&quot;mean&quot;,&quot;std&quot;])<br>    peakiness[&quot;peakiness_cv&quot;] = (peakiness[&quot;std&quot;] / peakiness[&quot;mean&quot;]).replace([np.inf, -np.inf], np.nan).fillna(0)<br>    peakiness = peakiness[&quot;peakiness_cv&quot;]<br><br>    out = pd.concat([reach, dwell, dwell_per_device, repeat, peakiness], axis=1).fillna(0).reset_index()<br>    out = out.merge(zones[[&quot;zone_id&quot;,&quot;name&quot;,&quot;type&quot;]], on=&quot;zone_id&quot;, how=&quot;left&quot;)<br>    return out</pre><h4>2. Engagement Metrics (Dwell Time)</h4><p>The 8-minute cap is important. Sparse pings can create artificially long dwell times. Someone sitting in the food court for 45 minutes might only appear in the data twice, creating a huge apparent gap. The cap prevents these outliers from dominating the analysis. This is one of those things you learn the hard way when working with real WiFi data. As you can see in the code above, dwell is derived from the dt_min deltas and clipped to 8 minutes per ping interval.</p><h4>3. Flow Metrics</h4><p>This is where the graph-inspired approach pays off. I measured <em>throughflow </em>which is just how many transitions involve each zone. This captures something simple traffic counts is likely to miss.</p><p>Below is the exact flow function I used. It returns both the transition table T and per-zone flow metrics:</p><pre>def compute_flow_metrics(sessionized: pd.DataFrame, edges: pd.DataFrame):<br>    &quot;&quot;&quot;<br>    Build directed transitions between zones and compute:<br>      - throughflow: total transitions in + out for each zone<br>      - connectivity: number of distinct zones people come from and go to (observed)<br>    Returns:<br>      - T: transition table (from_zone, to_zone, n)<br>      - flow: per-zone flow metrics (zone_id, inflow, outflow, throughflow, connectivity)<br>    &quot;&quot;&quot;<br>    df = sessionized.sort_values([&quot;session_key&quot;, &quot;timestamp&quot;]).copy()<br>    df[&quot;prev_zone&quot;] = df.groupby(&quot;session_key&quot;)[&quot;zone_id&quot;].shift(1)<br><br>    trans = df.dropna(subset=[&quot;prev_zone&quot;]).copy()<br>    trans = trans[trans[&quot;prev_zone&quot;] != trans[&quot;zone_id&quot;]]<br><br>    # Transition counts<br>    T = (<br>        trans.groupby([&quot;prev_zone&quot;, &quot;zone_id&quot;]).size()<br>        .rename(&quot;n&quot;)<br>        .reset_index()<br>        .rename(columns={&quot;prev_zone&quot;: &quot;from_zone&quot;, &quot;zone_id&quot;: &quot;to_zone&quot;})<br>    )<br><br>    # Inflow/outflow/throughflow<br>    outflow = T.groupby(&quot;from_zone&quot;)[&quot;n&quot;].sum().reset_index().rename(<br>        columns={&quot;from_zone&quot;: &quot;zone_id&quot;, &quot;n&quot;: &quot;outflow&quot;}<br>    )<br>    inflow = T.groupby(&quot;to_zone&quot;)[&quot;n&quot;].sum().reset_index().rename(<br>        columns={&quot;to_zone&quot;: &quot;zone_id&quot;, &quot;n&quot;: &quot;inflow&quot;}<br>    )<br><br>    flow = pd.merge(inflow, outflow, on=&quot;zone_id&quot;, how=&quot;outer&quot;).fillna(0)<br>    flow[&quot;throughflow&quot;] = flow[&quot;inflow&quot;] + flow[&quot;outflow&quot;]<br><br>    # Connectivity proxy: distinct previous + distinct next (observed)<br>    distinct_next = T.groupby(&quot;from_zone&quot;)[&quot;to_zone&quot;].nunique().reset_index().rename(<br>        columns={&quot;from_zone&quot;: &quot;zone_id&quot;, &quot;to_zone&quot;: &quot;distinct_next_zones&quot;}<br>    )<br>    distinct_prev = T.groupby(&quot;to_zone&quot;)[&quot;from_zone&quot;].nunique().reset_index().rename(<br>        columns={&quot;to_zone&quot;: &quot;zone_id&quot;, &quot;from_zone&quot;: &quot;distinct_prev_zones&quot;}<br>    )<br><br>    conn = pd.merge(distinct_prev, distinct_next, on=&quot;zone_id&quot;, how=&quot;outer&quot;).fillna(0)<br>    conn[&quot;connectivity&quot;] = conn[&quot;distinct_prev_zones&quot;] + conn[&quot;distinct_next_zones&quot;]<br><br>    flow = pd.merge(flow, conn[[&quot;zone_id&quot;, &quot;connectivity&quot;]], on=&quot;zone_id&quot;, how=&quot;left&quot;).fillna(0)<br><br>    return T, flow</pre><p><strong><em>Why this matters:</em></strong> A corridor might have <em>low dwell time</em> but massive <em>throughflow</em>. For instance, if 5,000 people walk through it daily, that’s 5,000 advertising impressions, even if each person only spends 30 seconds there. This is the difference between a destination and a pathway, and both have value.</p><h4>4. Repeat Exposure</h4><p>High repeat rate means it’s a hub people return to, or it’s unavoidable (like a main corridor). This is good for brand reinforcement. If someone passes your ad three times during their visit, that’s more valuable than one pass as they are likely to have an idea of what is being advertised. Repeat rate is also computed in compute_zone_metrics() above via “pings per session per zone”.</p><h4>5. Temporal Peakiness</h4><p>Coefficient of variation tells you if traffic is concentrated (high CV) or spread out (low CV). High peakiness is great for timed campaigns or events. Low peakiness means consistent exposure throughout the day. This is also included in compute_zone_metrics() above via peakiness_cv.</p><h4>The Scoring Formula</h4><p>With all these metrics, I combine them into a single “<strong>premium score</strong>” (for lack of a better term) using weighted aggregation. This is where art meets science, because the weights reflect what different stakeholders care about.</p><p>Before scoring, I first combine everything into one dataset (sessionize → compute metrics → merge flow). Here’s exactly how I do that:</p><pre>sessionized = build_sessions(pings, inactivity_gap_minutes=15)<br>zone_metrics = compute_zone_metrics(sessionized, zones)<br>T, flow_metrics = compute_flow_metrics(sessionized, edges)<br><br>zone_metrics = zone_metrics.merge(flow_metrics[[&quot;zone_id&quot;,&quot;throughflow&quot;,&quot;connectivity&quot;]], on=&quot;zone_id&quot;, how=&quot;left&quot;).fillna(0)<br><br>zone_metrics.sort_values(&quot;unique_reach&quot;, ascending=False).head(10)</pre><p>Now for the scoring.</p><h4>Min-Max Normalization</h4><p>Without normalization, raw numbers would completely dominate dwell times (in minutes). Normalization brings everything to the same scale [0, 1], so the weights actually mean something.</p><p>Here’s the exact normalization function I used:</p><pre>def minmax(s: pd.Series):<br>    if s.max() == s.min():<br>        return pd.Series(np.zeros(len(s)), index=s.index)<br>    return (s - s.min()) / (s.max() - s.min())</pre><h4>Weighted Scoring + Ranking</h4><p>I used this ranking function that includes the justification string so you can defend the output with stakeholders.</p><pre>def rank_candidates(zone_metrics: pd.DataFrame, candidates: pd.DataFrame,<br>                    weights=None):<br>    if weights is None:<br>        weights = {<br>            &quot;unique_reach&quot;: 0.35,<br>            &quot;avg_dwell_per_device_min&quot;: 0.30,<br>            &quot;throughflow&quot;: 0.25,<br>            &quot;repeat_rate&quot;: 0.10<br>        }<br><br>    df = zone_metrics.merge(candidates[[&quot;zone_id&quot;,&quot;is_candidate&quot;]], on=&quot;zone_id&quot;, how=&quot;left&quot;)<br>    df = df[df[&quot;is_candidate&quot;] == True].copy()<br><br>    # Normalize each component so scales don&#39;t dominate<br>    for k in weights.keys():<br>        df[f&quot;{k}_norm&quot;] = minmax(df[k].astype(float))<br><br>    df[&quot;premium_score&quot;] = 0.0<br>    for k, w in weights.items():<br>        df[&quot;premium_score&quot;] += w * df[f&quot;{k}_norm&quot;]<br><br>    # Justification strings for each zone (top drivers)<br>    def explain_row(r):<br>        parts = []<br>        parts.append(f&quot;Reach={int(r[&#39;unique_reach&#39;])} devices&quot;)<br>        parts.append(f&quot;Avg dwell={r[&#39;avg_dwell_per_device_min&#39;]:.2f} min/device&quot;)<br>        parts.append(f&quot;Flow={int(r[&#39;throughflow&#39;])} transitions&quot;)<br>        parts.append(f&quot;Repeat={r[&#39;repeat_rate&#39;]:.2%}&quot;)<br>        return &quot; | &quot;.join(parts)<br><br>    df[&quot;justification&quot;] = df.apply(explain_row, axis=1)<br>    return df.sort_values(&quot;premium_score&quot;, ascending=False).reset_index(drop=True), weights<br><br>ranked, used_weights = rank_candidates(zone_metrics, candidates)<br>ranked[[&quot;zone_id&quot;,&quot;name&quot;,&quot;type&quot;,&quot;premium_score&quot;,&quot;justification&quot;]].head(10)</pre><p>Why these weights?</p><ul><li><strong>Reach (35%)</strong>: More people equals more value. This is the foundation.</li><li><strong>Dwell time (30%)</strong>: Longer exposure equals better ad absorption. Quality matters.</li><li><strong>Throughflow (25%)</strong>: Being on high-traffic paths matters even without long dwell.</li><li><strong>Repeat (10%)</strong>: Nice bonus for reinforcement, but secondary to the above.</li></ul><p>These weights are not arbitrary. They reflect what advertisers actually care about: impressions (reach), attention (dwell), and position (flow). You can adjust them based on your specific use case. For a quick-impact ad campaign (think movie posters), you might increase throughflow weight. For detailed product displays, increase dwell time weight.</p><h4>Results and Justifications</h4><p>Running this on the simulated dataset, the top locations emerge with clear justifications.</p><p>Typical outputs are likely to look like this:</p><pre>zone_id  name              premium_score  justification<br>COR_2    Main Corridor 2   0.89          Reach=1247 devices | Avg dwell=2.30 min/device | Flow=8234 transitions | Repeat=45%<br>FOOD     Food Court        0.85          Reach=1089 devices | Avg dwell=5.80 min/device | Flow=6421 transitions | Repeat=38%<br>SEAT     Seating/Atrium    0.82          Reach=1156 devices | Avg dwell=4.20 min/device | Flow=5892 transitions | Repeat=52%<br>COR_1    Main Corridor 1   0.79          Reach=1198 devices | Avg dwell=1.80 min/device | Flow=7456 transitions | Repeat=41%<br>GALL_1   Gallery 1         0.74          Reach=892 devices | Avg dwell=3.90 min/device | Flow=4231 transitions | Repeat=29%</pre><p><strong>What this tells us:</strong></p><ol><li>Main corridors dominate despite low dwell time. Throughflow is massive. These are your “billboard on the highway” locations.</li><li>Food Court <strong>balances reach and engagement</strong>. People linger here. This is perfect for detailed product displays or interactive kiosks.</li><li>Atrium/Seating is a hub with good<strong> repeat exposure</strong>. People return to rest, wait for friends, etc.</li><li>Galleries have engaged audiences but lower overall traffic. They could be great for niche products or high-value exhibitions.</li></ol><p>This is the kind of insight you can bring to a leasing meeting. Instead of “this corner feels busy,” you can say something like <em>“Main Corridor 2 has 8,234 daily transitions with 1,247 unique visitors and 45% repeat exposure within sessions.”</em></p><h4>Visualization: Making It Tangible</h4><p>One thing I learned from my museum analytics days is that business stakeholders love spatial visualizations. Numbers are abstract. Maps are the real deal to them.</p><p>You can adopt this simple function for the plots:</p><pre>import matplotlib.pyplot as plt<br><br>def plot_zone_scores(zones, ranked, top_n=8):<br>    df = zones.merge(ranked[[&quot;zone_id&quot;,&quot;premium_score&quot;]], on=&quot;zone_id&quot;, how=&quot;left&quot;).fillna(0)<br><br>    plt.figure(figsize=(8,6))<br>    plt.scatter(df[&quot;x&quot;], df[&quot;y&quot;], s=200, alpha=0.6)<br>    for _, r in df.iterrows():<br>        plt.text(r[&quot;x&quot;]+0.8, r[&quot;y&quot;]+0.8, r[&quot;zone_id&quot;], fontsize=9)<br><br>    # highlight top<br>    top = ranked.head(top_n).merge(zones, on=&quot;zone_id&quot;, how=&quot;left&quot;)<br>    plt.scatter(top[&quot;x&quot;], top[&quot;y&quot;], s=600, alpha=0.35)<br><br>    plt.title(&quot;Venue zones + highlighted top premium placement candidates&quot;)<br>    plt.xlabel(&quot;x&quot;)<br>    plt.ylabel(&quot;y&quot;)<br>    plt.show()<br><br>plot_zone_scores(zones, ranked, top_n=6)</pre><p>The below spatial plot immediately shows which zones are physically clustered and which are isolated.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/1*jmdvCqvs5i3PZXKV8V4xQg.png" /><figcaption>Plot of zones + premium spaces</figcaption></figure><p>Combined with the scores, it helps identify strategic placement patterns. For example, you might notice that all your top zones are in one wing of the mall. This is useful information for planning advertising campaigns or tenant mix.</p><h4>Privacy and Ethics</h4><p>Before we go further, let me be very clear about privacy. This is not optional. Do it wrong and you’ll have lawsuits, not insights.</p><p><strong>What you must do:</strong></p><ul><li>Only track anonymous device IDs (MAC addresses, hashed)</li><li>Aggregate data before analysis (no individual tracking)</li><li>Clear signage about WiFi analytics</li><li>Compliance with GDPR/local privacy laws</li><li>Data retention policies (delete raw pings after aggregation)</li></ul><p><strong>What you MUST NEVER do:</strong></p><ul><li>Track specific individuals</li><li>Link WiFi data to payment data or personal info</li><li>Share raw movement data with third parties</li><li>Use data for purposes beyond what was disclosed</li></ul><p><strong>The general rule:</strong> <em>if you can’t explain your data collection to a concerned parent or privacy advocate, you shouldn’t be doing it.</em></p><h3>Future Directions</h3><p>This approach is solid, but there’s always room for improvement. Here are extensions I’d consider in future iterations:</p><h4>1. Demographic Proxies</h4><p>Device types can serve as demographic indicators. For example:</p><ul><li>&gt; iPhone 15 Pro → likely affluent (I could be biased on this).</li><li>Older Android models → budget-conscious</li><li>Tablets → families with children</li></ul><p>You can’t know for certain, but probabilistic inference adds value.</p><h4>2. Path Analysis</h4><p>Instead of just zone-level metrics, analyze common journeys:</p><ul><li>Entry → Corridor → Food Court → Exit</li><li>Entry → Gallery 1 → Gallery 2 → Exit</li></ul><p>This reveals which advertising sequences work (billboard in corridor → detailed display in gallery).</p><h4>3. Time-of-Day Patterns</h4><p>Different hours have different audiences:</p><ul><li>Morning: commuters, coffee runs</li><li>Midday: lunch crowd, families</li><li>Evening: diners, entertainment seekers</li></ul><p>Adjust placement recommendations by time of day.</p><h4>4. A/B Testing Zones</h4><p>Actually place ads and measure conversion:</p><ul><li>QR code scans</li><li>App downloads</li><li>Coupon redemptions</li></ul><p>Feed this back into your model. Real performance beats all predictions.</p><h4><strong>5. Causal Modeling</strong></h4><p>What if an ad was placed here? This is uplift thinking applied to spatial analytics. Estimate the causal effect of placement on engagement, not just correlation. May be its time you revisit my earlier uplift modelling article <a href="https://medium.com/data-science-collective/stop-guessing-what-works-how-id-use-uplift-modelling-to-target-churn-interventions-without-fancy-3e4230de2575">here</a>.</p><h3>Conclusion</h3><p>That’s it for now. I’ve taken you through the entire process of building a graph-inspired WiFi analytics system for identifying premium advertising locations in any space. We’ve moved from simple traffic counting to sophisticated network analysis, capturing not just where people are, but how they move, why it matters, and what it’s worth.</p><p><strong>The key insight: </strong>location value is about more than traffic volume. It’s about the intersection of reach, engagement, flow, and connectivity. A low-traffic zone on a critical path can be more valuable than a high-traffic dead end. Both matter, but for different reasons and to different advertisers.</p><p>I hope this walkthrough was also useful. Don’t forget to follow me, clap for me, and leave a comment. Let me know if you’d like me to extend this to causal analysis (what-if scenarios for different layouts) or multi-venue comparisons. If you want to check out my other articles, you can find them on my <a href="https://medium.com/@hermanwandabwa">profile</a>. I’m also happy to connect via <a href="https://www.linkedin.com/in/wandabwaherman/">LinkedIn</a>.</p><p><strong>Technical Note:</strong> All data is simulated for educational purposes. Real deployments require proper WiFi infrastructure, privacy compliance, and ground-truth validation.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b8470dd9181b" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science-collective/finding-the-perfect-spot-wifi-analytics-for-premium-space-advertising-b8470dd9181b">Finding the Perfect Spot: WiFi Analytics for “Premium” Space Advertising</a> was originally published in <a href="https://medium.com/data-science-collective">Data Science Collective</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>