<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Manoj Ramani on Medium]]></title>
        <description><![CDATA[Stories by Manoj Ramani on Medium]]></description>
        <link>https://medium.com/@manoj.ramani03?source=rss-2da6dbdbde2a------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*0XYWdnfUD8qhC7A1.jpg</url>
            <title>Stories by Manoj Ramani on Medium</title>
            <link>https://medium.com/@manoj.ramani03?source=rss-2da6dbdbde2a------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sun, 22 Nov 2026 14:04:43 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@manoj.ramani03/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Stop Asking Your LLM to Do Math: How We Split Work Between GenAI and ML]]></title>
            <link>https://medium.com/@manoj.ramani03/stop-asking-your-llm-to-do-math-how-we-split-work-between-genai-and-ml-2bc089e36267?source=rss-2da6dbdbde2a------2</link>
            <guid isPermaLink="false">https://medium.com/p/2bc089e36267</guid>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[advertising-technology]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[agentic-ai]]></category>
            <dc:creator><![CDATA[Manoj Ramani]]></dc:creator>
            <pubDate>Fri, 28 Aug 2026 22:16:55 GMT</pubDate>
            <atom:updated>2026-08-28T22:16:55.511Z</atom:updated>
            <content:encoded><![CDATA[<p><em>LLMs hallucinate numbers. ML models can’t hold a conversation. We built an agent that uses each for what it’s good at — and cut an 8-hour manual workflow to 15 seconds.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2uo1x90TNfQEV8NxSSjPMA.png" /><figcaption>Before: 8-hour blind spots and spreadsheet triage. After: AI agent diagnoses and fixes campaigns in 15 seconds... Photo Credit: Google Gemini</figcaption></figure><h3>Part 1: The Problem That Makes This Worth Building</h3><p>Let’s ground this in a concrete scenario. Nexalith Ads is a fictitious ad-tech company we created for this POC, modeled on patterns we see across the industry. It manages over <strong>60,000 active advertising campaigns</strong> every day. These are programmatic ad campaigns — automotive brands, retailers, healthcare companies buying connected TV and digital ad inventory across the country.</p><p>The people who manage these campaigns are called media traders. Think of them as portfolio managers, but for ad spend instead of stocks. Each trader oversees roughly 200 campaigns. Their job: make sure every campaign delivers the impressions the client paid for, on time and on budget.</p><p>Here’s the problem. Traders get pacing reports <strong>three times a day</strong> — 8am, noon, 4pm. That means there’s up to an <strong>8-hour window</strong> where a campaign can be quietly failing, and nobody knows.</p><p>When a campaign falls behind, the manual workflow looks like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*pXq8simZVjDmxG-6S4qo4Q.png" /></figure><p>That cycle takes <strong>30–60 minutes per campaign</strong> when done well. And it repeats. Multiply that across hundreds of at-risk campaigns and you have a scaling problem that hiring can’t solve. You can’t add traders 1-for-1 with campaign growth.</p><p>What if an AI agent could do the monitoring, diagnosis, and recommendation in <strong>15 seconds</strong>?</p><p>That’s what we built. The rest of this post shows you how — from the agent that orchestrates the work, to the ML pipeline that classifies root causes and generates optimized recommendations, to the deployment on AWS. <em>(Want to follow along in the code? The full working implementation is in the </em><a href="https://github.com/aws-samples/sample-agentic-campaign-mgmt"><em>companion GitHub repo</em></a><em>.)</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*iIrtSF3wMJAcK-Ro4gasHQ.png" /><figcaption>The trader’s dashboard. Red = at-risk campaigns that need attention. The agent surfaces these automatically.</figcaption></figure><h3>Part 2: The Orchestrator Agent — The Brain of the System</h3><p>The core of the system is a <strong>Campaign Optimization Agent</strong> — a Claude-powered orchestrator running on <a href="https://github.com/aws-samples/sample-agentic-campaign-mgmt/blob/main/FAQ.md#faq-4-what-is-amazon-bedrock-agentcore">Amazon Bedrock AgentCore</a>. This POC uses Claude Sonnet 3.5 on Bedrock (current as of March 2026) — the architecture is model-agnostic, so swap in any model with tool-calling support. When a trader types “What should I do about campaign 4782?”, the agent figures out what data it needs, which tools to call, and how to assemble a coherent answer.</p><p>Here’s the architecture:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*sqZw4LU_5bOSSUYPDnPrfw.png" /></figure><p>The agent doesn’t follow a hard-coded workflow. It uses the <strong>ReAct pattern</strong> — Reason, Act, Observe, Repeat — to figure out what to do step by step.</p><h4>Walking Through a Real Query</h4><p>Let’s trace what happens when a trader asks: <strong>“What should I do about campaign 4782?”</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*uoEkVvXJ56xdJ0YeFqvQ_Q.png" /></figure><p>Here’s what the trader actually sees — a structured response assembled from four tool calls:</p><pre>Campaign #4782 is 23% behind pace — bid is below the market floor.<br><br>KEY METRICS<br>• Current Bid:        $4.20 CPM<br>• Market Floor:       $5.10 CPM (73rd percentile, automotive/Chicago)<br>• Win Rate:           8% (benchmark: 22%)<br>• Delivery:           29% (expected: 52% at this point in flight)<br>• Competitors:        12 active (up from 6 yesterday)<br><br>DIAGNOSIS<br>Root cause: bid_too_low (confidence: 99.7%)<br>Your bid ($4.20) is 18% below the current market floor ($5.10).<br>The model identified bid-to-floor ratio and competitor surge (+6 in 24h)<br>as the primary drivers.<br><br>RECOMMENDATION<br>Raise bid to $5.25 CPM<br>• Projected win rate: 24% (from 8%)<br>• Projected delivery recovery: on-pace within 36 hours<br>• Confidence: 85%, based on 17 similar automotive/Chicago campaigns<br>  in the last 90 days<br><br>NEXT STEP<br>→ Accept &amp; Apply | Modify | Reject</pre><p>Every number traces back to a tool call — nothing is fabricated. The agent fires independent calls in parallel, chains dependent calls sequentially, and synthesizes everything into a clear decision point.</p><h4>The 10 Tools</h4><p>The agent has access to 10 tools, grouped by function. Each tool is registered via <a href="https://github.com/aws-samples/sample-agentic-campaign-mgmt/blob/main/FAQ.md#faq-3-what-is-mcp-model-context-protocol">MCP (Model Context Protocol)</a> through AgentCore Gateway:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/850/1*sdfgzaWqVvpTVkAfffclHg.png" /></figure><p>The agent decides which tools to call based on the trader’s question — not a script. A metrics question calls one tool. A “what should I do?” question chains four. The tool descriptions tell the agent <em>what each tool does and when to call it</em>. The system prompt tells the agent <em>how to behave</em> — guardrails, output format, and one critical rule: <strong>never apply a change without trader approval</strong>.</p><h4>Human in the Loop</h4><p>Every recommendation surfaces three options: <strong>Accept &amp; Apply</strong>, <strong>Modify</strong>, or <strong>Reject</strong>. The agent is an advisor, not an autopilot. The trader stays in control.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kDdO50PN5WLndDfTg14oOQ.png" /><figcaption>The agent’s diagnosis: root cause, confidence score, and evidence — all from the ML pipeline.</figcaption></figure><h3>Part 3: GenAI + Traditional ML — Why You Need Both</h3><p>Here’s the key architectural insight: <strong>GenAI alone doesn’t solve this problem.</strong> You need two types of AI working together.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xtMT9tVsCWSmT-xHsiyVwQ.png" /></figure><p><strong>Traditional ML</strong> handles the structured, repeatable work through a three-stage pipeline:</p><ul><li><strong>Stage 1 — XGBoost diagnosis</strong> (15 features → 6 issue types): answers “what’s wrong?” Given features like bid_to_floor_ratio, win_rate, competitor_change_24h, it outputs a probability distribution across 6 root-cause issue types. Fast (under 10ms), interpretable, cheap to host on a small CPU instance.</li><li><strong>Stage 2 — RandomForest classification</strong> (19 features → 5 action types): answers “what to do?” Given the campaign metrics plus the diagnosed issue, it classifies the appropriate corrective action (e.g., bid_adjustment, creative_refresh).</li><li><strong>Stage 3 — GradientBoosting regression</strong> (25 features → optimal parameter value): answers “how much?” Per-action regressors predict the optimal numeric value for the chosen action (e.g., the recommended bid in CPM dollars).</li></ul><p><strong>Confirmed smoke test results (Campaign 4782):</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wmyjfihktBp7vrILlXxsGQ.png" /></figure><p><strong>GenAI</strong> (Claude LLM on AgentCore) handles the unstructured, reasoning work. It understands trader questions in natural language. It decides which tools to call and in what order. It synthesizes the ML pipeline’s structured output into a human-readable recommendation with context.</p><p>Neither one can do the other’s job well.</p><blockquote>The agent (Claude LLM on AgentCore) reasons through the problem and calls ML tools when it needs structured answers. GenAI decides what to do next. The ML pipeline tells it what’s wrong, what action to take, and by exactly how much.</blockquote><h3>Part 4: Building the ML Diagnosis Model</h3><p>The diagnose_campaign_issue tool calls an XGBoost classifier trained to answer one question: <strong>Why is this campaign underdelivering?</strong></p><p>We chose XGBoost because it’s interpretable, fast, handles mixed feature types, and runs on a small CPU instance. For 15 features and 6 classes, it’s the right tool.</p><h4>The 6 Issue Types</h4><p>The model classifies each campaign into one of these root causes:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/850/1*wnRY05NLyhexYIVXR4HraA.png" /></figure><h4>The Feature Importance Surprise</h4><p>We feed the model 15 features. Here’s what the model considers most important:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/850/1*Awm6iRoWuIsJWEocZSlOaQ.png" /><figcaption><em>Gain measures how much each feature improves the model’s predictions when used in a decision split — higher gain means the feature is more informative.</em></figcaption></figure><p>The biggest surprise: <strong>bid_to_floor_ratio ranked 7th, not 1st.</strong> The original rule-based system treated bid as the <em>only</em> signal. The ML model discovered that market dynamics -- how many competitors entered in the last 24 hours, click-through rate trends, delivery variance -- are equally or more important.</p><p>That’s the whole point of ML over rules. Rules encode what you already know. ML finds what you don’t.</p><blockquote><strong>Why not just use if/else rules?</strong> The rule-based approach we started with had one rule: if bid &lt; market floor, it’s bid_too_low. It caught that case well. But it couldn&#39;t distinguish between competitive pressure and inventory shortage when the bid was fine. It missed creative fatigue entirely. The ML model catches all six -- and shows its confidence level for each.</blockquote><h4>Training and Testing</h4><p>We trained on 1,200 synthetic rows (200 per class, balanced). Synthetic data is fine for a POC — it validates the full pipeline end-to-end before real labeled data exists. When adapting this for your own use case, you’d retrain on real labeled data from your domain.</p><p>The code pipeline is three files:</p><ol><li>generate_training_data.py -- creates 1,200 labeled examples with domain-realistic feature distributions</li><li>train_model.py -- trains the XGBoost classifier, saves diagnosis_model.pkl + label_encoder.pkl</li><li>diagnose_campaign_ml.py -- the inference module with a predict(features) function</li></ol><p>We wrote <strong>14 unit tests</strong> that exercise predict() directly with raw feature dictionaries -- no Lambda, no SageMaker, no network calls. The model was tested in isolation before it touched any infrastructure.</p><h3>Part 4b: The Recommendation Pipeline — Classify, Then Size</h3><p>Diagnosis tells you <em>what’s wrong</em>. The recommendation pipeline answers <em>what to do about it</em> and <em>by exactly how much</em>. It’s two ML steps chained together:</p><p><strong>Stage 2: RandomForest classifier</strong> (19 features → 5 action types). Given the diagnosis plus campaign context, it picks the corrective action. The key insight: diagnosis-to-action isn’t 1:1. bid_too_low maps to bid_adjustment ~70% of the time, but to budget_reallocation or targeting_expansion the other 30%, depending on budget headroom and geo constraints. Those edge cases are exactly where ML earns its keep over a lookup table.</p><p><strong>Stage 3: GradientBoosting regressors</strong> (25 features → optimal parameter value). Five separate models, one per action type, each predicting in its own units (CPM dollars for bids, multipliers for pacing, fractions for rotation). We chose separate models over one multi-output model because the targets have different units, different dominant features, and need independent tuning.</p><h3>Part 5: Deploying the Models — Lambda + SageMaker</h3><p>With trained models, the next question is: where do they run?</p><h4>Three Options, One Choice</h4><p>There are three plausible deployment paths:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/850/1*AkpUltdydGRzdDn1GUFl6Q.png" /></figure><p>All three are valid depending on your use case — they differ in cost, latency, and operational overhead. Evaluate case-by-case. For this POC, we went with <strong>SageMaker endpoint invoked by Lambda</strong>: the Lambda stays thin and the models get a purpose-built hosting environment with built-in health checks and model versioning.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*X7410hM0n4sgRLLxzvxt_Q.png" /></figure><p>The Lambda stays thin — just handler.py + boto3 (pre-installed in the Lambda runtime). The heavy ML inference runs on SageMaker&#39;s dedicated ml.t2.medium instance.</p><h4>Deploy and Smoke Test</h4><p>Both deploy scripts are <strong>pure Python</strong> — no bash, no PowerShell. They work on Windows and Mac without path conversion issues.</p><pre># Deploy the Lambda function<br>uv run python deploy/deploy_lambda_zip.py<br><br># Deploy the SageMaker endpoint + wire Lambda<br>uv run python deploy/deploy_sagemaker_diagnosis.py<br><br># Smoke test<br>uv run python lambda/smoke_test.py</pre><p>The smoke test invokes the Lambda, which calls the SageMaker endpoints, which run inference. Result: campaign 4782 diagnosed as bid_too_low with <strong>99.7% confidence</strong> (bid 4.20belowmarketfloor4.20<em>belowmarketfloor</em>5.10), action classified as bid_adjustment, recommended bid of <strong>$5.25 CPM</strong>.</p><blockquote><strong>Lesson learned:</strong> Make deploy scripts in the same language as your project. We started with bash, hit path and zip issues on Windows. Rewrote in PowerShell, hit string escaping issues. Rewrote in Python. Should have started there.</blockquote><h3>Part 6: Architectural Decisions — The Choices We Made and Why</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/850/1*4cx8xmFH4ymUyEhFOVmBTA.png" /></figure><h3>Part 7: What We Learned — Practical Takeaways</h3><p>Six things that would have saved us time if we’d known them on day one:</p><p><strong>1. GenAI and ML are not interchangeable.</strong> Use ML for structured classification and regression. Use GenAI for reasoning and synthesis. The agent orchestrates both. Trying to use GenAI for numeric classification leads to hallucinated numbers. Trying to use ML for natural language understanding leads to a brittle keyword parser.</p><p><strong>2. Start with the feature vector, not the model.</strong> The features you choose determine your ceiling. Our model’s biggest insight came from including competitor_change_24h -- a feature the rule-based system never considered. Spend your time on feature engineering, not hyperparameter tuning.</p><p><strong>3. Unit test the ML model in isolation.</strong> We wrote 14 tests against predict() directly -- raw feature dictionaries in, classification results out. No Lambda, no SageMaker, no network. We found and fixed issues before any infrastructure existed. The model was working locally before we deployed anything.</p><p><strong>4. Deploy scripts should be cross-platform from day one.</strong> We started with bash (hit zip not found, cygpath issues, python3 not in PATH on Windows). Rewrote in PowerShell (hit string escaping issues in inline Python). Finally rewrote in Python with boto3 and zipfile. Works everywhere. Should have started there.</p><p><strong>5. The system prompt and tool descriptions have different jobs.</strong> The system prompt owns cross-cutting behavior: guardrails, output format, autonomy boundaries. Tool descriptions own per-tool instructions: what it does, when to call it, required parameters, dependency ordering. Putting both in the system prompt creates maintenance drift — when you rename a tool, you have to update two places instead of one.</p><p><strong>6. Diagnosis ≠ Action — don’t skip the classification stage.</strong> It’s tempting to map diagnoses directly to actions with a lookup table. In our data, that works ~70% of the time. The remaining 30% — where budget constraints, geo locks, or market conditions change the optimal action — are exactly the high-value cases where the classifier earns its keep. The three-stage pipeline (diagnose → classify → size) keeps each decision independently testable and explainable.</p><h3>Part 8: Where This Goes Next</h3><p>What we’ve built is a working POC that validates the full pipeline. Here’s what we’d explore next:</p><p><strong>Retrain on real data.</strong> The synthetic training data validates the pipeline. The real gains come from retraining on actual trader feedback — when a trader accepts or rejects a recommendation, that’s a labeled training example. Over time, the models learn patterns specific to this market.</p><p><strong>Event-driven detection.</strong> The current POC waits for the trader to ask. A natural next step is to flip this model entirely: event-driven systems capture business events as they happen, automatically diagnose issues, generate recommendations, and notify traders for immediate action. Instead of traders hunting for problems, the system pulls them in — broadcasting diagnosed issues and suggested fixes to Slack (or any channel) so traders take action in minutes, not hours:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*d37AW_gMjd5TW6N8LKT2bA.png" /></figure><p>Campaign goes at-risk, detection fires, the agent diagnoses and recommends, the trader gets a Slack message with a specific action and a confidence score. <strong>15 seconds, not 8 hours.</strong></p><p><strong>Expand to 13 issue types.</strong> The current model classifies 6. The full label taxonomy includes budget exhaustion, frequency cap blocking, daypart restrictions, publisher exclusions, and more.</p><h3>Taking This to Your Use Case</h3><p>This POC validates the full pipeline: agent orchestration, ML classification and regression, SageMaker deployment, and tool integration via MCP. If you’re adapting this pattern for your own domain, here’s what to consider before going live:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/850/1*-m55dD00eYc130b02C04Bw.png" /></figure><p>The architecture and patterns transfer directly — the GenAI + ML split, the MCP tool registration, the ReAct orchestration, the env-var routing. What changes is the hardening around it.</p><p><a href="https://github.com/aws-samples/sample-agentic-campaign-mgmt/blob/main/FAQ.md"><strong>FAQ — Deep-Dive Reference</strong></a><strong>:</strong> 11 questions on the technical concepts behind this post.</p><blockquote><strong>This is a proof of concept (POC).</strong> The code and architecture in this post demonstrate the art of the possible — they validate the full pipeline end-to-end. They are not production-ready as-is. If you’re adapting this for your own use case, see Taking This to Your Use Case for guidance on what to harden before going live.</blockquote><p><strong>Get the code:</strong> The full source for this project, including the agent, ML pipeline, deployment scripts, evals, and the React prototype UI, is in the <a href="https://github.com/aws-samples/sample-agentic-campaign-mgmt">companion GitHub repo</a>. Clone it, run the prototype locally with zero AWS dependencies, then progressively light up the real agent and deployed stack as you go deeper. The README walks you through it in three phases.</p><p><strong><em>Disclaimer:</em></strong> <em>I am a Sr Applied AI Solutions Architect working for AWS. While the ideas and suggestions in this post are drawn from my personal experience working with many different AWS customers, all views and opinions expressed are my own and do not reflect the positions of my employer, Amazon.</em></p><p><em>If you enjoyed this article, feel free to connect with me on </em><a href="https://www.linkedin.com/in/manojramani/"><em>LinkedIn</em></a><em>.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2bc089e36267" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Notes from the Field: Before You Deploy Agents, Mine the Mess]]></title>
            <link>https://medium.com/@manoj.ramani03/notes-from-the-field-before-you-deploy-agents-mine-the-mess-b3a52dc5b99b?source=rss-2da6dbdbde2a------2</link>
            <guid isPermaLink="false">https://medium.com/p/b3a52dc5b99b</guid>
            <category><![CDATA[process-automation]]></category>
            <category><![CDATA[digital-transformation]]></category>
            <category><![CDATA[ai-agent]]></category>
            <category><![CDATA[enterprise-ai]]></category>
            <category><![CDATA[process-mining]]></category>
            <dc:creator><![CDATA[Manoj Ramani]]></dc:creator>
            <pubDate>Wed, 26 Aug 2026 01:30:01 GMT</pubDate>
            <atom:updated>2026-08-27T21:20:12.637Z</atom:updated>
            <content:encoded><![CDATA[<p><em>A practitioner’s guide to task mining, process mining, and when you need both</em></p><figure><img alt="*”If I had an hour to solve a problem, I’d spend 55 minutes understanding it and 5 minutes solving it.”* *Image credit: Google Gemini Nanobanana*" src="https://cdn-images-1.medium.com/max/1024/1*9fDdDywMLp7c9w5ouus1Iw.jpeg" /><figcaption>Whats really happening in my enterprise .. <em>Image credits: Google Gemini Nano Banana</em></figcaption></figure><blockquote>If I had an hour to solve a problem, I’d spend 55 minutes understanding it and 5 minutes solving it.</blockquote><h4>Task Mining &amp; AI-Powered Process Discovery</h4><p>Enter any large enterprise in healthcare, insurance, or marketing and one thing is universal: there are people looking to automate and simplify. With the rise of agentic AI, the pitch is seductive. Just “watch” what people do and replicate it with agents. That appeal drove an explosion of task mining tools in 2025–2026.</p><p><strong><em>But the real world is rarely that clean.</em></strong></p><p>Think about claims adjudication, procure-to-pay, merchandising, new product introduction. These are end-to-end business flows that developed organically over years, span ERPs, home-grown custom apps, and everyone’s favorite: an Excel or Access database file living under someone’s desk. The actual process is buried beneath layers of silos and executes across multiple business domains that no single system of record captures.</p><p>When we approach a customer, we start with one objective: mine what the actual process is under the hood, then overlay it with the real work activity people carry out day-to-day. That combination gives you the true end-to-end view of what’s actually going on.</p><blockquote>You need to go slow to go fast. The temptation to bring agents in without sufficient understanding runs the risk of “running a poorly designed process, faster,” which doesn’t deliver the gains customers should expect from agentic technology.</blockquote><p>This article is built on what we’ve seen in the field: how task mining, process mining, and agentic AI fit together for discovery and resolution. The tool listings are accurate as of today. Given the pace of development in this space, expect alternatives to multiply, both open-source and build-your-own variants.</p><p><strong>So what are these two disciplines?</strong></p><p>Task mining watches <strong>how people actually work</strong> at the desktop level: clicks, keystrokes, screen content, app switches. It converts those observations into process maps, SOPs, and automation candidates. Process mining reads system event logs to show how cases flow end-to-end. Task mining fills the gap between those logs and the messy human reality that happens <em>between</em> systems.</p><blockquote><strong>Real-life analogy</strong><br> Process mining reads the flight logs. Task mining watches the pilot in the cockpit. One tells you the route was flown; the other shows you the workarounds, the tribal knowledge, and the steps nobody documented.</blockquote><h4>Where it sits in the stack</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*eTIiTTtrbudFUldGcyloCg.png" /><figcaption><em>Fig 1: Task mining and process mining feed different signals into a unified Process Intelligence view</em></figcaption></figure><h4>Task Mining vs Process Mining — The Distinction</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*m-UNm3wcmEybotwuvEKsyQ.png" /><figcaption><em>Fig 2: Task mining vs process mining — different data sources, different insights, complementary roles</em></figcaption></figure><h4>Customer Requirement to Capability Mapping</h4><p>Listen for the customer’s framing. Their words tell you which bucket they need.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*QzTp_O07pe6GOwpbSWInwQ.png" /><figcaption><em>Fig 3: What the customer says maps directly to which mining capability they need</em></figcaption></figure><h3>Why This Matters Now (Aug 2026)</h3><p>Four things happened in 2025–2026 that turned task mining from a niche procurement line item into something strategic:</p><ol><li>Gartner renamed the category. The May 2026 Magic Quadrant dropped “Process Mining” for “Process Intelligence Platforms,” and a separate <em>Market Guide for Task Mining Tools</em> gave task mining its own category for the first time. (<a href="https://www.gartner.com/en/documents/7837281">Gartner</a>, May 2026)</li><li>Process mining + agentic AI started merging. Celonis launched a Context Model and acquired Ikigai Labs (<a href="https://www.celonis.com/news/press/celonis-launches-the-context-model-to-eliminate-enterprise-ais-operational-blind-spots-agrees-to-acquire-ai-decision-intelligence-leader-ikigai-labs">Celonis</a>, May 2026). Salesforce bought Apromore to feed Agentforce (<a href="https://www.salesforce.com/news/stories/salesforce-signs-definitive-agreement-to-acquire-apromore/">Salesforce</a>, Nov 2025). ServiceNow integrated task mining with closed-loop AI Agent creation (<a href="https://www.servicenow.com/community/process-mining-blog/what-s-new-for-process-and-task-mining-in-the-australia-release/ba-p/3508013">ServiceNow</a>, Mar 2026).</li><li>AI-native entrants showed up. WIQ (from Bardeen’s co-founder) and Screenpipe (YC S26) skip the “make a report” step and go straight from observation to deployed agent. The distance between “discover the process” and “automate it” is shrinking to zero.</li><li>Most AI pilots still fail. Teams automate processes they never properly understood. You end up running a bad process faster, not fixing it. Task mining is the insurance policy: you can’t hand work to agents if you haven’t first watched what “good” actually looks like. (<a href="https://www.kognitos.com/blog/process-mining-vs-agentic-ai-2026-guide/">Kognitos</a>, Jun 2026)</li></ol><h4>Enterprise vs accessible tools: two speeds</h4><p>Enterprise tools (Celonis, Skan, UiPath) take weeks-to-months and custom contracts. Accessible tools (Screenpipe, Skill Recorder, Grok Bot) give you first insight in minutes-to-days at known price points. The pragmatic move: start accessible, capture 10–20 high-value workflows, show results, then justify the enterprise spend.</p><h3>The Tools (as of Aug 2026)</h3><h4>Accessible / AI-Native Tools</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*D-Nw_BbQDNIHwmKc6dVFCA.png" /><figcaption><em>Fig 4: Accessible and AI-native task mining tools you can start with today</em></figcaption></figure><h4>Where Tools Actually Sit on the Spectrum</h4><p>Very few tools are purely one thing anymore. Everyone’s converging toward “process intelligence.” But each tool still has a center of gravity:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*C_8HLakAjQGZEPHFGU2YbA.png" /><figcaption><em>Fig 5: The spectrum from pure task mining to pure process mining — most tools are converging toward the hybrid middle</em></figcaption></figure><h4>Build your own: pm4py + OCEL 2.0 on AWS</h4><p>Besides the commercial products above, there’s a viable build-your-own path. <a href="http://ocel-standard.org/">OCEL 2.0</a> (the open standard for object-centric event logs) and <a href="https://pm4py.fit.fraunhofer.de/">pm4py</a> (the open-source process mining library from Fraunhofer) give you the algorithmic engine and data model without vendor lock-in.</p><p>This combination has proven successful with a large healthcare customer. The architecture:</p><ul><li><strong>Ingest </strong>multi-object enterprise logs into Amazon S3 using OCEL 2.0 as the data standard</li><li><strong>Process </strong>via AWS Glue or Amazon ECS pipelines running pm4py for discovery, conformance checking, and variant analysis</li><li><strong>Query </strong>complex process relationships using Amazon Athena for ad-hoc analysis and dashboarding</li></ul><p>The result is a completely custom, serverless process mining platform. You own the data, control the algorithms, and pay only for what you compute. For organizations already on AWS, this avoids six-figure enterprise licensing while giving you flexibility no SaaS tool offers, particularly when your event logs span multiple systems with complex object relationships (the exact scenario OCEL 2.0 was designed for).</p><h4>What to actually ask when picking a tool</h4><p>Skip “is this task mining or process mining?” That distinction is blurring. Ask these instead:</p><ul><li>“<strong><em>Do we have system event logs to analyze?</em></strong>” — Start process-mining-gravity (Celonis, Signavio, Power Automate)</li><li>“<strong><em>Is all the work in browsers/Excel/email with no system of record?</em></strong>” — Start task-mining-gravity (Screenpipe, Skan, WIQ)</li><li>“<strong><em>Do we need discovery AND execution in one tool?</em></strong>” — Look at hybrids (UiPath, WIQ, Grok Bot)</li><li>“<strong><em>Do we just need to capture one workflow fast?</em></strong>” — Pure task, lightweight (Skill Recorder, Screenpipe)</li><li>“<strong><em>Do we need enterprise governance, PII masking, audit trails?</em></strong>” — Enterprise tier (Skan, UiPath, Celonis) regardless of origin</li></ul><h3>What To Actually Do</h3><h4>Recommended approach by tool</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*NikSLeTa8kHAL48R60DuUA.png" /><figcaption><em>Fig 6: Match your scenario to the right tool</em></figcaption></figure><h4>The strategic sequence</h4><blockquote>Process mining without agentic AI produces beautiful diagnostic reports the organization cannot fix. Agentic AI without process mining produces fast automation of processes that should have been redesigned first.<br> — paraphrased from <a href="https://www.kognitos.com/blog/process-mining-vs-agentic-ai-2026-guide/">Kognitos</a>, Jun 2026</blockquote><p>The order matters:</p><ol><li><strong>Discover</strong> with task/process mining</li><li><strong>Redesign</strong> using process intelligence (kill the waste before you automate it)</li><li><strong>Execute</strong> with agentic AI</li><li><strong>Monitor</strong>, loop back to mining, repeat</li></ol><p>The department-wide Excel file or Access database under someone’s desk isn’t going anywhere. Neither is the tribal knowledge living in one person’s head, or the workaround nobody documented. But now you know how to find it, map it, and decide whether an agent or a human should handle it next.</p><blockquote>Go slow to go fast. Mine the mess. Then automate what deserves to be automated using agents.</blockquote><p><em>Disclaimer:</em> <em>I am a Sr Applied AI Solutions Architect working for AWS. While the ideas and suggestions in this post are drawn from my personal experience working with many different AWS customers, all views and opinions expressed are my own and do not reflect the positions of my employer, Amazon.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b3a52dc5b99b" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[You Already Understand ML. You Just Don’t Know It Yet.]]></title>
            <link>https://medium.com/@manoj.ramani03/you-already-understand-ml-you-just-dont-know-it-yet-bed4654e838f?source=rss-2da6dbdbde2a------2</link>
            <guid isPermaLink="false">https://medium.com/p/bed4654e838f</guid>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[generative-ai-solution]]></category>
            <category><![CDATA[learning]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Manoj Ramani]]></dc:creator>
            <pubDate>Mon, 03 Aug 2026 18:44:41 GMT</pubDate>
            <atom:updated>2026-08-03T22:46:52.811Z</atom:updated>
            <content:encoded><![CDATA[<p><em>A cloud engineer’s shortcut to machine learning fluency. No PhD required.</em></p><figure><img alt="An isometric illustration showing cloud infrastructure on the left morphing into neural network architecture on the right, connected by a bridge" src="https://cdn-images-1.medium.com/max/1024/1*wgtdGJ11m2X0nHvnVOweOQ.png" /><figcaption>From cloud infra to neural networks: the same systems, different vocabulary. (Image: Gemini 3.1 Pro)</figcaption></figure><h3>The Problem</h3><p>You’ve deployed Kubernetes clusters, designed event-driven architectures, and debugged distributed systems at 2 AM. You think in APIs, load balancers, and service meshes.</p><p>Then someone drops a machine learning paper on your desk. Suddenly you’re drowning in “stochastic gradient descent,” “backpropagation through time,” and “variational inference.” You feel like a beginner again.</p><p>But you’re not. You already understand 80% of ML concepts. They’re just wearing unfamiliar names.</p><p>This post kicks off a series called <strong>LHTL (Learning How To Learn)</strong>, where I help cloud technologists build ML/AI fluency by connecting new concepts to the mental models they already own.</p><p>No hand-waving. No dumbing down. Just translation.</p><h3>The Insight: ML Is Just Systems Engineering With Different Vocabulary</h3><p>I had a breakthrough while studying <a href="https://arxiv.org/pdf/2603.19415">a research paper on prompt routing</a> (dynamically selecting which LLM handles each request). The paper was dense: mixture-of-experts, quality estimation adapters, graph-based clustering, iterative Leiden community detection.</p><p>Intimidating? Sure. Until I realized:</p><ul><li>The “mixture-of-experts” was just <strong>a service mesh with specialized handlers</strong></li><li>“Training” was <strong>auto-tuning a load balancer by replaying traffic logs</strong></li><li>“Backpropagation” was <strong>root cause analysis through a call chain</strong></li><li>The whole system was <strong>a smart request router with health-check scoring</strong></li></ul><p>I wasn’t learning something new. I was translating something familiar.</p><p>That’s the method. And it works for every ML concept I’ve encountered since.</p><h3>The Translation Table</h3><p>Here’s the core mapping. Bookmark this. Come back to it every time you read a paper, a blog post, or a Bedrock documentation page.</p><h4>Fundamentals</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/905/1*1z0ajM6WIaWRRR9DyEK6bw.png" /></figure><h4>Training</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/889/1*ocEy3TQOZPu8LSZFpbIP0w.png" /></figure><h4>Architecture Patterns</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/893/1*9mjXcYh5vv9CEN2kw3BrCg.png" /></figure><h4>Deployment &amp; Operations</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/878/1*j559BCe45X3seHVWk_WnRg.png" /></figure><h4>Evaluation</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/880/1*MBXJ3Wc87h5AVLP4acWMCQ.png" /></figure><h4>The Method: How to Actually Learn ML/AI From Here</h4><p>The translation table gets you reading fluency. But fluency isn’t mastery. Here’s the full path I recommend for cloud engineers building ML competence:</p><h4>Level 1: Read (1 week)</h4><p><strong>Goal:</strong> Understand what ML systems do, not how to build them from scratch.</p><ul><li>Use the translation table above as a live reference while reading any ML content</li><li>When you hit an unfamiliar term, ask: “What system component is this replacing?”</li><li>Read paper abstracts and conclusions first (skip the math initially)</li><li>Look at Figure 1 of any paper — it nearly always shows the full architecture as a system diagram</li></ul><p><strong>Milestone:</strong> You can explain an ML paper’s architecture using only systems terminology in a 5-minute whiteboard sketch.</p><h4>Level 2: Map (1–2 weeks)</h4><p><strong>Goal:</strong> Connect ML architectures to AWS services you already deploy.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/864/1*ducId69mihkUFhYBOMwjnQ.png" /></figure><p><strong>Milestone:</strong> Given any ML system diagram, you can sketch the AWS architecture that would deploy it.</p><h4>Level 3: Build (2–4 weeks)</h4><p><strong>Goal:</strong> Build a toy ML system using tools you already know.</p><p>You don’t need PyTorch. Start with:</p><pre>from sklearn.neural_network import MLPRegressor<br>from sentence_transformers import SentenceTransformer<br><br># Encode prompts<br>encoder = SentenceTransformer(&#39;all-MiniLM-L6-v2&#39;)<br>embeddings = encoder.encode(prompts)<br><br># Train one &quot;adapter&quot; per model (like the paper!)<br>router_claude = MLPRegressor(hidden_layer_sizes=(128,)).fit(embeddings, claude_scores)<br>router_llama = MLPRegressor(hidden_layer_sizes=(128,)).fit(embeddings, llama_scores)<br><br># Route: pick highest-scoring model<br>scores = [router_claude.predict(new_emb), router_llama.predict(new_emb)]<br>best_model = [&quot;claude&quot;, &quot;llama&quot;][argmax(scores)]</pre><p>50 lines of Python. No GPU. No PhD. And you just built a prompt router that implements the core idea from a research paper.</p><p><strong>Milestone:</strong> You’ve built something that works, and you understand <em>why</em> each line matters because it maps to a concept from the table above.</p><h4>Level 4: Explain (Ongoing)</h4><p><strong>Goal:</strong> Teach what you’ve learned.</p><p><a href="https://fs.blog/feynman-technique/">The Feynman Technique</a>: if you can’t explain it simply, you don’t understand it. Test yourself:</p><blockquote>“Think of prompt routing like a smart load balancer. Instead of round-robin, it classifies each request by type and routes to the backend that historically performs best for that category. Training is just the phase where you replay traffic through all backends to build the routing table.”</blockquote><p>If a customer, a colleague, or a junior engineer nods along, you’ve mastered it.</p><h3>A Worked Example: Reading an ML Paper</h3><p>Let me show the method in action. Here’s a sentence from a real research paper:</p><blockquote>“Stage 2 employs a mixture-of-experts quality estimation architecture where task-specific prediction heads are invoked based on the assigned task type.”</blockquote><p>A year ago, I would’ve glazed over. Now I translate in real-time:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/868/1*Bf1L8e3RnTsq3qac-TAJ5w.png" /></figure><p>Full translation:</p><blockquote>“Stage 2 is a service mesh where specialized health-check functions score each backend. The request classifier from Stage 1 determines which specialized functions to invoke.”</blockquote><p>Same meaning. Zero intimidation. And now I can reason about it (latency, scaling, failure modes) in the language I think in.</p><h3>Why This Works (The Learning Science)</h3><p>This isn’t a hack. It’s grounded in how expertise actually transfers:</p><ol><li><strong>Schema theory.</strong> New knowledge sticks when it attaches to existing mental frameworks. You already have rich schemas for distributed systems. ML concepts snap onto them like Lego.</li><li><strong>Analogical reasoning.</strong> Experts in one domain learn faster in adjacent domains because they transfer structural patterns, not surface details. A load balancer and a neural network router share deep structure.</li><li><strong>Reduced cognitive load.</strong> When terminology is the barrier (not the concepts), translation removes the bottleneck entirely. You’re not learning from zero; you’re relabeling.</li></ol><h3>What’s Next in This Series</h3><p>This is Part 1 of the <strong>LHTL (Learning How To Learn)</strong> series. I have a few ideas for where to go next:</p><ul><li><strong>The Transformer, Explained as a Message Bus.</strong> Self-attention, multi-head attention, and positional encoding mapped to event-driven architecture patterns.</li><li><strong>Training a Model = Running a Load Test.</strong> Loss functions, optimizers, and convergence as operational metrics you already monitor.</li></ul><p>But I’d rather write what’s actually tripping you up. What ML/AI concept makes you feel like a beginner again? Is it RAG architectures? Fine-tuning vs. prompt engineering tradeoffs? Reading those dense arxiv papers without glazing over? Something else entirely?</p><p>Drop a comment or reach out. The next post in this series will be shaped by what you tell me.</p><h3>Your Next Step</h3><p>Pick one ML paper that’s relevant to your work. Any paper. Read the abstract.</p><p>For every term that feels foreign, find it in the translation table above. Sketch the architecture on a whiteboard using only systems language. You’ll be surprised how much you already know.</p><h3>References</h3><ul><li>Zhang et al., <a href="https://arxiv.org/abs/2603.19415">“Scalable Prompt Routing via Fine-Grained Latent Task Discovery”</a> (2026). The paper that sparked this series.</li></ul><p><strong><em>Disclaimer:</em></strong><em> I am a Sr Applied AI Solutions Architect working for AWS. While the ideas and suggestions in this post are drawn from my personal experience working with many different AWS customers, all views and opinions expressed are my own and do not reflect the positions of my employer, Amazon.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=bed4654e838f" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>