<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Puspanjali Sarma on Medium]]></title>
        <description><![CDATA[Stories by Puspanjali Sarma on Medium]]></description>
        <link>https://medium.com/@puspanjalisarma23?source=rss-657cc3b2578c------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*Hj71HdIjcXQmdn3h-j8BIw.jpeg</url>
            <title>Stories by Puspanjali Sarma on Medium</title>
            <link>https://medium.com/@puspanjalisarma23?source=rss-657cc3b2578c------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 22:58:26 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@puspanjalisarma23/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Beyond “Vibes”: The Engineering Reality of Measuring LLM Reliability]]></title>
            <link>https://medium.com/@puspanjalisarma23/beyond-vibes-the-engineering-reality-of-measuring-llm-reliability-cf041639a019?source=rss-657cc3b2578c------2</link>
            <guid isPermaLink="false">https://medium.com/p/cf041639a019</guid>
            <category><![CDATA[ai-engineering]]></category>
            <category><![CDATA[llm-evaluation]]></category>
            <category><![CDATA[llmops]]></category>
            <category><![CDATA[llm]]></category>
            <dc:creator><![CDATA[Puspanjali Sarma]]></dc:creator>
            <pubDate>Thu, 06 Aug 2026 18:16:28 GMT</pubDate>
            <atom:updated>2026-08-06T18:16:28.959Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*YTrugkbf-bVhNiAZCWFVkA.png" /><figcaption>LLM Evaluation</figcaption></figure><p><strong>Building production AI isn’t about writing better prompts. It’s about building better evaluation systems.</strong></p><p>The AI space is currently drowning in noise. Spend five minutes on LinkedIn or X, and you will encounter an army of self-proclaimed <strong>“AI Agents Experts”</strong> and <strong>“LLM Whisperers”</strong> whose entire methodology consists of typing a prompt into a chat interface, looking at the output, and declaring, <strong><em>“Looks good to me!”</em></strong></p><p>This is what we call <strong>vibe-based evaluation</strong>. And if you are building production-grade AI systems, vibes are not a metric.</p><p>When a traditional software engineer writes a function, they write a unit test. You pass an input, you assert an expected output, and the test passes deterministically. Large Language Models (LLMs) destroy this paradigm. LLMs operate on probability distributions, meaning the exact same prompt can yield a completely different response on every run due to settings like “<strong>temperature</strong>”. Furthermore, LLM quality is inherently fuzzy and multidimensional. There is rarely a single “correct” string to assert against; there are only ranges of acceptable answers.</p><p>Tweaking a prompt to fix a specific hallucination without a systematic evaluation framework is just a blind bet — a silent regression waiting to happen.</p><p>If you want to cut through the hype and build LLM applications you can actually trust, you need to transition from a debugging mindset to a rigorous <strong>measurement framework</strong>. Here is how real engineers measure LLM reliability beyond unit tests.</p><h3>1. The Core Primitives of an “Eval Stack”</h3><p>To measure subjective quality, you must operationalize it into data. You cannot automate a test for “helpfulness,” but you can test a strict rubric. A robust evaluation system requires three primitives:</p><ol><li><strong>Criteria &amp; Quality Dimensions:</strong> You must define exactly what matters for your specific use case. Are you measuring <em>Relevance</em> (did it answer the prompt?), <em>Coherence</em> (does it flow logically?), <em>Factuality</em>, or <em>Safety</em>?.</li><li><strong>Rubrics:</strong> A rubric breaks vague criteria into strict, scorable questions (e.g., “Is the response under 200 words? Does it contain valid JSON?”).</li><li><strong>The Golden Set:</strong> This is your ground truth. A version-controlled dataset of hundreds of test cases (inputs and ideal outputs). Crucially, a golden set cannot be synthetic data you hallucinated in a conference room. It must be seeded with <strong>real, anonymized production queries</strong> reflecting how users actually try to break your system.</li></ol><h3>2. The Illusion of Traditional Metrics</h3><p>Once you have your Golden Set, how do you score the model’s outputs? The initial instinct is to use established Natural Language Processing (NLP) metrics like ROUGE or BLEU, which measure n-gram (word) overlap between the generated text and a reference answer.</p><p>The problem? <strong>They are often fundamentally broken for factual accuracy.</strong></p><p>Recent studies have exposed that metrics like ROUGE exhibit alarmingly low precision for identifying actual factual errors. Because they rely on lexical overlap, a model can generate a factually catastrophic answer that uses similar vocabulary to the reference, and ROUGE will score it highly.</p><p>Worse, these metrics suffer from severe <strong>Length Bias</strong>. Models often attempt to maintain coherence while hallucinating, leading to a “snowball effect” where initial errors compound into long, verbose, incorrect explanations. Paradoxically, researchers have found that simple heuristics — like checking the raw length or the standard deviation of an answer’s length — can sometimes match or exceed the performance of sophisticated hallucination detectors.</p><h3>3. “LLM-as-a-Judge” and its Hidden Biases</h3><p>To evaluate complex logic and tone at scale, the industry standard has shifted to the <strong>LLM-as-a-Judge</strong> paradigm. You prompt a highly capable model (like GPT-4o or Claude 3.5 Sonnet) with your Golden Set input, the response to evaluate, and your strict rubric, asking it to output a score and a rationale.</p><p>While highly scalable, LLM judges are not objective arbiters of truth. They inherit systematic biases that “AI experts” routinely fail to account for:</p><ul><li><strong>Position Bias:</strong> When comparing two responses side-by-side, the judge often favors whichever response is placed in the first (or last) slot of the prompt, regardless of quality.</li><li><strong>Self-Preference Bias:</strong> Models demonstrate a narcissistic tendency to disproportionately favor outputs generated by their own model family.</li><li><strong>Rubric Order &amp; Score ID Bias:</strong> Shockingly, judge models are highly sensitive to superficial prompt formatting. Research shows that an LLM’s score will change based on whether the rubric is presented in ascending order (1 to 5) or descending order (5 to 1). Furthermore, using Roman numerals (i, ii, iii) or letter grades (A, B, C) as Score IDs instead of standard Arabic numerals can drastically shift a judge’s accuracy.</li></ul><p>To build a reliable LLM-as-a-Judge system, you must actively engineer around these biases by swapping positions, tweaking score IDs, and injecting “perfect score” reference answers to anchor the judge’s logic.</p><h3>4. Evaluating RAG and Autonomous Agents</h3><p>If your application connects to a database — Retrieval-Augmented Generation (RAG) — evaluating the final answer alone is useless. You must evaluate the pipeline via the <strong>RAG Triad</strong>:</p><ol><li><strong>Context Precision:</strong> Did the retrieval system fetch the right documents?</li><li><strong>Faithfulness:</strong> Did the model stick strictly to the retrieved context, or did it invent facts (hallucinate)?</li><li><strong>Answer Relevance:</strong> Did the generated response actually address the user’s intent?</li></ol><p>As we transition from simple RAG chatbots to <strong>Autonomous Agents</strong>, the evaluation demands grow exponentially. A chatbot finds a needle in a haystack; an agent must map the entire haystack. This introduces the <strong>Comprehensiveness Gap</strong>.</p><p>To evaluate an agent tasked with deep research (e.g., “Identify all clinical trials for mRNA vaccines initiated in 2024”), standard metrics fail. Agents must be evaluated on <em>Systematic Collation</em> (synthesizing hundreds of sources without dropping data), <em>Entity Resolution</em> (de-duplicating facts), and crucially, <em>Stopping Criteria</em> — the epistemic reasoning to know when to stop searching and realize that an answer simply does not exist on the web.</p><h3>5. The Dirty Secret of AI Leaderboards: Contamination &amp; Aging</h3><p>Finally, be incredibly skeptical of “experts” touting their model’s performance on public benchmarks like MMLU or HumanEval.</p><p>The industry is currently facing a massive <strong>Data Contamination</strong> crisis. Because LLMs are trained on internet-scale web scraping, the test questions from these benchmarks inevitably leak into their training data. When a model scores 90% on a benchmark, it is often a measure of <em>memorization</em>, not genuine reasoning or generalization capability.</p><p>Furthermore, static benchmarks suffer from <strong>Temporal Alignment Decay</strong>. The real world updates, but static datasets do not. For example, if a benchmark from 2023 lists “China” as the most populated country in the world, an LLM correctly answering “India” in 2025 will be penalized for being factually accurate. A significant portion of evaluation failures today occur simply because our models have outgrown the aging datasets we use to test them.</p><h3>Conclusion</h3><p>Building an AI feature in an afternoon is easy. Proving that it works consistently at scale is hard engineering.</p><p>The next time an <strong>“AI Expert”</strong> tries to sell you on a tool or a prompt because it “passed their tests,” ask them about their Golden Set coverage. Ask them how they mitigated Rubric Order Bias in their <strong>LLM-as-a-Judge </strong>pipeline. Ask them how they measure Faithfulness against Context Precision.</p><p>If you want to deploy AI systems that survive contact with reality, ditch the vibes. Start treating evaluation as your core measurement infrastructure.</p><h3>Let’s Connect</h3><p>If this perspective on AI-native enterprises resonated with you:</p><ul><li>Follow me on <a href="https://www.linkedin.com/in/puspanjalisarma/"><strong>Linkedin</strong></a> , <a href="https://github.com/puspanjalis"><strong>Github</strong></a><strong> </strong>and <a href="https://medium.com/@puspanjalisarma23"><strong>Medium</strong></a><strong> </strong>for practical insights on AI engineering, agentic systems, and enterprise-scale AI</li><li>Subscribe to my newsletter <em>“</em><a href="https://www.linkedin.com/build-relation/newsletter-follow?entityUrn=7412051308393787392"><strong><em>Scaling AI Without Hype</em></strong></a><em>”</em> for real-world frameworks and no-nonsense playbooks</li><li>Reach out if you’re building or scaling AI platforms — open to meaningful discussions and collaboration</li></ul><p>I’d love to hear how your organization approaches evaluation, observability, and production AI engineering.👇🏻</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=cf041639a019" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Beyond Prompt Engineering: Moving
from Context to Loops and Harnesses]]></title>
            <link>https://medium.com/@puspanjalisarma23/beyond-prompt-engineering-moving-from-context-to-loops-and-harnesses-533f709ed2a4?source=rss-657cc3b2578c------2</link>
            <guid isPermaLink="false">https://medium.com/p/533f709ed2a4</guid>
            <category><![CDATA[agents]]></category>
            <category><![CDATA[context-engineering]]></category>
            <category><![CDATA[prompt-engineering]]></category>
            <category><![CDATA[loop-engineering]]></category>
            <category><![CDATA[harness-engineering]]></category>
            <dc:creator><![CDATA[Puspanjali Sarma]]></dc:creator>
            <pubDate>Wed, 10 Jun 2026 10:47:24 GMT</pubDate>
            <atom:updated>2026-06-10T10:47:24.630Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="loop engineering vs context engineering vs harness engineering" src="https://cdn-images-1.medium.com/max/1024/1*6WfHdmJodGMd8cEUW84_6Q.png" /><figcaption>Beyond Prompt Engineering</figcaption></figure><p>If I see one more enterprise AI architecture diagram that resolves entirely to a single RAG pipeline hitting an API endpoint, I’m going to lose my mind. Over the past two years, the industry has suffered from a collective failure of imagination. We’ve treated <strong>Large Language Models</strong> like<br>traditional deterministic functions: feed in the correct input (<strong>Prompt Engineering</strong>), attach a vector database (Context Engineering), and pray the output doesn’t confidently misinform your biggest client.</p><p>It’s a fragile paradigm, and it’s hitting a wall. In my engineering leadership roles, when teams come to me complaining that their LLM systems work beautifully in a proof-of-concept but degrade into unpredictable chaos under production loads, the root cause is almost always the same. They are treating a systemic architecture problem as a prompting problem.</p><p>If you want to move beyond toys and build production-grade AI systems that can automate actual workflows with deterministic guardrails, you need to understand the structural divide between three distinct disciplines: <strong>Context Engineering</strong>, <strong>Loop Engineering</strong>, and <strong>Harness Engineering</strong>. They<br>are not interchangeable terms. They are separate layers of the modern enterprise cognitive stack.</p><pre>┌────────────────────────────────────────────────────────┐<br>│  HARNESS ENGINEERING (The Environment: Rules, Tools, Perms)  │<br>│  ┌──────────────────────────────────────────────────┐  │<br>│  │  LOOP ENGINEERING (The Flow: Self-Correction Loop) │  │<br>│  │  ┌────────────────────────────────────────────┐  │  │<br>│  │  │  CONTEXT ENGINEERING (The Input: Data &amp; Window)│  │  │<br>│  │  └────────────────────────────────────────────┘  │  │<br>│  └──────────────────────────────────────────────────┘  │<br>└────────────────────────────────────────────────────────┘</pre><h4>The Three Layers of the Cognitive Stack</h4><p>To contextualize where we are, we have to look at how enterprise adoption patterns have shifted. In 2023, everyone was obsessed with context windows and vector retrieval. In 2024, the focus shifted to multi-agent frameworks. Now, we’re realizing that agents without runtime<br>guarantees are just expensive ways to generate infinite loops and run up cloud bills.</p><p><strong>1. Context Engineering: The Data Horizon<br>Context Engineering</strong> is about state management and information retrieval. It answers a simple question: <em>What does the model need to know right now to perform this specific action?</em></p><ul><li><strong>Scope: </strong>Bounds single-turn input and output windows.</li><li><strong>Core Elements: </strong>Hybrid keyword/semantic search, graph relationships, and metadata filtering.</li><li><strong>Economics: </strong>Bound by the cost equation of input token density versus retrieval precision.</li></ul><p><strong>2. Loop Engineering: The Execution Horizon</strong><br><strong>Loop Engineering </strong>introduces time, iteration, and state transitions. It moves the model from a stateless transaction to a stateful, iterative machine.</p><ul><li><strong>Scope: </strong>Bounded execution pathways where the model systematically evaluates and revises its own output until it clears a defined bar.</li><li><strong>Core Elements: </strong>Plan-execute-verify topologies, state-machine tracking, and Mem0 memory-first loop designs.</li><li><strong>Risks: </strong>Infinite recursion, context drift, and compounding logical errors</li></ul><p><strong>3. Harness Engineering: The Operational Horizon<br></strong>Coined as an industry standard following paradigms popularized by <a href="https://milvus.io/blog/harness-engineering-ai-agents.md">Mitchell Hashimoto’s Agent Environment Framework</a>, <strong>Harness Engineering</strong> designs the infrastructure that isolates, evaluates, and regulates the loop. The core tenant, supported by <a href="https://martinfowler.com/articles/harness-engineering.html">Martin Fowler’s Outer Harness Architecture</a>, treats the AI loop as untrusted code running within a highly trusted border.</p><ul><li><strong>Scope: </strong>The complete external environment, operating system, and runtime boundaries.</li><li><strong>Core Elements:</strong> Model-Context Protocol (MCP) servers, system permissions manifests, and automated verification gates</li><li><strong>Risks:</strong> Sandbox escape or misconfiguration leading to security breaches, system exploitation, and severe execution latency.</li></ul><p><strong>Harness Engineering</strong> is the infrastructure that isolates, evaluates, and controls the loop. It is the sandboxed environment, the continuous integration pipeline, the security proxy, and the deterministic runtime that makes sure an autonomous agent doesn’t execute a destructive<br>terminal command on your database or leak proprietary source code.</p><p>The harness treats the AI loop as untrusted code running within a highly trusted border.</p><blockquote><strong>“Context feeds the model. The Loop focuses the model. The Harness protects the enterprise from the model.”</strong></blockquote><h4>The Core Thesis: Why Prompting is a Dead End</h4><p>Relying on prompt engineering to fix system behavioral issues is like trying to fix a memory leak by rewriting the documentation. Prompts are soft constraints. In an enterprise environment requiring five-nines reliability, soft constraints are liabilities.</p><p>When you shift the burden of reliability from the prompt to the <strong>Harness</strong> and the <strong>Loop</strong>, something interesting happens: you can actually use smaller, cheaper, open-weights models. You don’t need a massive, closed-source model to handle everything if your loop handles the self-<br>correction and your harness handles the environment enforcement.</p><p>This structural shift is where we analyze how changing the system topology matters far more than tweaking adjectives in a system prompt.</p><h4>The Tri-Axis Cognitive Reliability Framework (TCR)</h4><p>To evaluate whether your AI systems are ready for enterprise deployment, I use a framework that measures system health across three distinct axes rather than looking at raw accuracy metrics:</p><ul><li><strong>Context Efficiency (Η_c):</strong> The ratio of task-relevant tokens delivered to total tokens processed. If you are stuffing 100k tokens of raw documentation into a context window to extract a three-word answer, your efficiency is abysmal, and your architecture is economically unsustainable.</li><li><strong>Convergence Rate (Γ_l): </strong>The mathematical probability that an iterative self-correcting loop will arrive at a deterministic stop state within N iterations, rather than diverging into infinite repetition or state decay.</li><li><strong>Isolation Threshold (Ω_h): </strong>The percentage of untrusted actions (e.g., code execution, API mutations) that are perfectly sandboxed and intercepted by the runtime harness before hitting external stateful systems.</li></ul><h4>Real-World Failure Modes</h4><p>Let’s talk about how these systems break in production when this framework is ignored.</p><ul><li><strong>The Infinite Loop Degeneracy: </strong>In a multi-agent system without strict <strong>Loop Engineering</strong>, Agent A generates a report. Agent B finds a minor formatting error and rejects it. Agent A regenerates it, changing something else. Agent B rejects it again. Your system spins in a circle, consuming thousands of dollars in API credits over the weekend until a budget alert kills the server.</li><li><strong>Context Drift and Hallucination Accumulation:</strong> In long-running conversational systems, each iteration appends new data to the history. If you do not actively curate, prune, and summarize that context, the model’s attention matrix degrades. The noise-to-signal ratio crosses a critical threshold, and the model starts hallucinating based on its own previous misstatements.</li></ul><h4>A Code-Level Deep Dive: The Self-Correcting Loop</h4><p>To see <strong>Loop Engineering</strong> in action, let’s look at a concrete implementation.</p><p>This is a Python-coded pattern for a self-correcting engine designed to generate valid, safe JSON metadata configurations. It doesn’t just ask the model for JSON; it evaluates the output deterministically and passes stack traces back into the execution loop for auto-correction.</p><pre>#self_correcting_loop.py<br>import json<br>import re<br>from typing import Dict, Any, Tuple<br>class SelfCorrectingLoop:<br>    def __init__(self, llm_client, max_retries: int = 3):<br>        self.client = llm_client<br>        self.max_retries = max_retries<br>    def _deterministic_validator(self, raw_output: str) -&gt; Tuple[bool, Any]:<br>        try:<br>            # Context cleaning: Remove markdown block wrappers if the LLM includes them<br>            cleaned = re.sub(r&quot;```json\s*|```&quot;, &quot;&quot;, raw_output).strip()<br>            data = json.loads(cleaned)<br>            <br>            # Validation Layer: Check for structural keys<br>            required_keys = [&quot;transaction_id&quot;, &quot;risk_score&quot;, &quot;routing_target&quot;]<br>            for key in required_keys:<br>                if key not in data:<br>                    return False, f&quot;Missing mandatory structural key: &#39;{key}&#39;&quot;<br>            <br>            # Validation Layer: Check business logic boundaries<br>            if not (0 &lt;= float(data[&quot;risk_score&quot;]) &lt;= 1.0):<br>                return False, &quot;Value out of bounds: &#39;risk_score&#39; must be between 0.0 and 1.0&quot;<br>                <br>            # FIX: Changed &#39;result_data&#39; to &#39;data&#39; to resolve the NameError crash<br>            return True, data<br>            <br>        except json.JSONDecodeError as e:<br>            return False, f&quot;Invalid JSON format: {str(e)}&quot;<br>        except Exception as e:<br>            return False, f&quot;Unexpected validation failure: {str(e)}&quot;<br>    def execute(self, payload: Dict[str, Any]) -&gt; Dict[str, Any]:<br>        context_prompt = f&quot;Generate routing payload for: {json.dumps(payload)}. Return ONLY valid JSON.&quot;<br>        current_prompt = context_prompt<br>        error_feedback = &quot;&quot;<br>        <br>        for attempt in range(1, self.max_retries + 1):<br>            if error_feedback:<br>                current_prompt = f&quot;{context_prompt}\nFailure on previous attempt: {error_feedback}&quot;<br>            <br>            # Execute model call<br>            raw_response = self.client.generate(current_prompt)<br>            success, result = self._deterministic_validator(raw_response)<br>            <br>            if success:<br>                return {&quot;status&quot;: &quot;converged&quot;, &quot;attempts&quot;: attempt, &quot;data&quot;: result}<br>            <br>            # Save error signal for the next loop iteration context window<br>            error_feedback = str(result)<br>            <br>        raise RuntimeError(f&quot;Loop failed to converge within {self.max_retries} retries.&quot;)</pre><p>Notice what is happening here: <em>the loop handles the correction, not a more descriptive prompt. The model is given a chance to inspect its own stack trace. If it fails after three attempts, the system intentionally throws a runtime exception, triggering an upstream safety fallback. This is the definition of defensive AI design.</em></p><h4>The Architectural Blueprint: Harness Engineering with Docker</h4><p><em>But what happens if the loop needs to run untrusted code generated by the model?</em></p><p>You cannot run that code on your application server. You need isolation.</p><p><em>This is where Harness Engineering steps in.</em></p><pre>┌────────────────────────────────────────────────────────┐<br>│                      CONTROL PLANE                     │<br>│  Orchestrates workflows &amp; evaluates execution logs     │<br>└───────────────────────────┬────────────────────────────┘<br>                            │ Serializes Task via API<br>                            ▼ whitelisted endpoints<br>┌────────────────────────────────────────────────────────┐<br>│                    ISOLATED HARNESS                    │<br>│  Ephemeral Docker Container (No Intranet, Egress Caps) │<br>└────────────────────────────────────────────────────────┘</pre><p>A resilient runtime harness isolates the execution loop within an ephemeral, resource-constrained container environment. The architecture separates cleanly into a secure<strong> Control Plane</strong> and an <strong>Isolated Execution </strong>Domain :</p><ol><li>The <strong>Control Plane</strong> orchestrates state workflows . When the loop triggers code evaluation or a terminal payload, it posts the task to an isolated Execution API.</li><li>The <strong>Execution API </strong>spins up a temporary Docker container with total networking restrictions (no egress to internal corporate networks, whitelisted internet proxies) and a strict execution timeout limit.</li><li>The <strong>Container</strong> captures standard output (stdout) and error traces (stderr), signs the data contract back to the Control Plane, and instantly self-destructs . If malicious logic or an infinite loop runs, it kills nothing but that isolated instance.</li></ol><h4>Strategic Leadership Trade-Offs</h4><p>As an engineering executive, your role is to manage technical debt and unit economics. Moving up the stack from <strong>Context</strong> to <strong>Harnessing</strong> is not free. It introduces significant trade-offs that you must actively manage:</p><ul><li><strong>Latency vs. Autonomy: </strong>Every step in a self-correcting loop adds sequential LLM round-trips. A single-shot prompt takes 800ms. A 3-turn self-correcting loop running code inside a Docker container can take 6 seconds. You must decide where your application sits on the latency-vs-<br>certainty spectrum. For a user-facing chatbot, loop-driven latency is a product killer; for a back- office financial reconciliation agent, it’s completely acceptable.</li><li><strong>Operational Complexity: </strong>Building a Dockerized execution harness means your AI engineering team is now building distributed infrastructure. You need container orchestration, logging aggregation for untrusted code execution, and rigorous secret-management boundaries. If your engineering team does not have deep DevOps maturity, building a sophisticated harness will become a massive operational bottleneck.</li></ul><h4>The Next 36 Months: Cognitive Operating Systems</h4><p>We are rapidly moving toward a world where we stop thinking about LLMs as standalone applications and start viewing them as the central processing unit of a new type of computer.</p><p>In this paradigm, <strong>Context Engineering</strong> is your virtual memory architecture, <strong>Loop Engineering </strong>is your instruction scheduling, and <strong>Harness Engineering</strong> is your kernel space and user space<br>separation.</p><p>The companies that win this next era won’t be the ones who write the cleverest prompts. They will be the ones who build the most resilient, secure, and cost-efficient execution environments.</p><p>They will build systems that are robust enough to allow unpredictable neural networks to perform highly predictable business operations.</p><blockquote><strong>Stop optimizing your adjectives. Start engineering your boundaries.</strong></blockquote><h4>Connect &amp; Collaborate</h4><p>If you’re interested in building predictable systems with unpredictable models, consider subscribing to my weekly newsletter, <strong>Scaling AI Without Hype</strong>, where I break down production post-mortems and architecture models without the marketing fluff.</p><p>You can also connect with me on <a href="https://www.linkedin.com/in/puspanjalisarma/"><strong>LinkedIn</strong></a> and check out my open-source architecture experiments on <a href="https://github.com/puspanjalis"><strong>GitHub</strong></a>.</p><p>For a deeper strategic roadmap on structuring enterprise engineering organizations for this shift, look out for my book, <strong>Strategic AI Leadership Through Data</strong>.</p><p>🛡️ Move Beyond the Prompt Window</p><p>Is your engineering team struggling to transition AI models out of basic prototyping and into highly predictable, sandboxed production workflows?</p><p>At <a href="https://www.criterionaiproductlab.com/"><strong>Criterion AI Product Lab</strong></a>, we design robust runtime harnesses and deterministic self-correcting workflows that bring architectural stability to enterprise-grade AI applications. Stop letting unpredictable agents dictate your token budgets and cloud infrastructure bills.</p><p>👉 Follow our <a href="https://www.linkedin.com/company/criterion-ai-product-lab"><strong>Official LinkedIn Company Page </strong></a>for weekly architectural deep-dives, production software patterns, and open-source infrastructure insights.</p><p>👉 Looking to secure your <strong>AI cognitive stack</strong>? Connect with us to schedule an enterprise architecture audit.</p><p>A question for the comments: <strong><em>Where is your team spending the majority of its engineering hours right now — refining prompts, building data pipelines (Context), or building execution guardrails (Harness)?</em></strong></p><p>Let’s discuss below.👇🏻</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=533f709ed2a4" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Why does the same prompt give different answers at different times, days, or environments?]]></title>
            <link>https://blog.gopenai.com/why-does-the-same-prompt-give-different-answers-at-different-times-days-or-environments-f39475c0c9af?source=rss-657cc3b2578c------2</link>
            <guid isPermaLink="false">https://medium.com/p/f39475c0c9af</guid>
            <category><![CDATA[tokenization]]></category>
            <category><![CDATA[ai-platform]]></category>
            <category><![CDATA[ai-engineering]]></category>
            <category><![CDATA[ai-reliability]]></category>
            <category><![CDATA[ai-agent]]></category>
            <dc:creator><![CDATA[Puspanjali Sarma]]></dc:creator>
            <pubDate>Sun, 17 May 2026 19:46:00 GMT</pubDate>
            <atom:updated>2026-05-17T19:46:00.657Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ni_2KRIJROyZxwO1PETikQ.png" /></figure><p>I keep getting this question in conferences, leadership roundtables, and architecture discussions:</p><p><strong>“If I use the exact same prompt, why does the AI give different answers on different days, environments, or even time zones?”</strong></p><p>Most people immediately point to:</p><ul><li>temperature</li><li>randomness</li><li>model updates</li></ul><p>Those matter.</p><p><strong>But in enterprise AI systems, the deeper issue is usually much bigger:</strong></p><p><strong><em>The system itself is changing underneath the prompt.</em></strong></p><p>One underrated contributor to this behavior is <strong>tokenization drift</strong>.</p><p><em>And as </em><strong><em>agents</em></strong><em> become more autonomous, this problem becomes much more serious.</em></p><h3>What is tokenization drift?</h3><p>At a simple level, <strong>tokenization</strong> is how an LLM converts text into machine-readable chunks (“tokens”).</p><p>Humans see:</p><p><strong><em>“Restart the failed payment service.”</em></strong></p><p>The model sees:</p><ul><li><strong><em>token IDs</em></strong></li><li><strong><em>token boundaries</em></strong></li><li><strong><em>embeddings</em></strong></li><li><strong><em>positional relationships</em></strong></li></ul><p>That translation layer matters more than most teams realize.</p><p>Even tiny differences in:</p><ul><li>whitespace</li><li>formatting</li><li>punctuation</li><li>unicode normalization</li><li>tokenizer versions</li><li>model routing</li><li>hidden system prompts</li></ul><p>can change:</p><ul><li>token boundaries</li><li>context windows</li><li>attention behavior</li><li>retrieval ranking</li><li>reasoning paths</li></ul><p>So technically, the <strong>“same prompt”</strong> is often not the same input anymore.</p><p><em>This is where many teams misunderstand the problem.</em></p><p>They assume prompts are <strong>deterministic</strong> strings.</p><p>In reality, enterprise <strong>AI systems</strong> behave more like distributed runtime environments with probabilistic execution paths.</p><h3>Why answers change over time</h3><p>In production systems, several invisible layers evolve continuously.</p><h3>1. Model version drift</h3><p>Many hosted AI platforms silently:</p><ul><li>update model checkpoints</li><li>optimize inference paths</li><li>patch safety layers</li><li>modify routing logic</li></ul><p><a href="https://openai.com/"><strong>OpenAI</strong></a><strong>, </strong><a href="https://www.google.com/search?q=Anthropic&amp;rlz=1C5GCEM_enIN1203IN1204&amp;oq=Anthropic&amp;gs_lcrp=EgZjaHJvbWUyBggAEEUYOdIBBzE3OWowajmoAgawAgHxBXkeoOp5ZI3x&amp;sourceid=chrome&amp;ie=UTF-8#:~:text=Home%20%5C%20Anthropic,www.anthropic.com"><strong>Anthropic</strong></a><strong>, </strong><a href="https://www.google.com/"><strong>Google</strong></a>, and others continuously optimize serving infrastructure.</p><p>So the exact same request today may hit:</p><ul><li>a different inference cluster</li><li>a different quantized model variant</li><li>a different routing path</li><li>updated safety tuning layers</li></ul><p>This is why <strong>reproducibility</strong> becomes difficult without strict version pinning.</p><p>And most enterprises are not version-pinning nearly as aggressively as they think they are.</p><h3>2. Tokenizer changes</h3><p>This is the subtle one.</p><p>Different tokenizer versions may:</p><ul><li>split words differently</li><li>compress tokens differently</li><li>interpret special characters differently</li></ul><p>This directly affects:</p><ul><li>embedding generation</li><li>retrieval quality</li><li>chunking behavior</li><li>truncation boundaries</li><li>reasoning chains</li></ul><p>And this matters more than people think.</p><p>For example:</p><ul><li><strong>OpenAI</strong> uses <strong>BPE-style tokenization</strong></li><li><strong>Llama</strong>-family models use <strong>SentencePiece</strong> variants</li><li><a href="https://www.google.com/search?q=Mistral&amp;rlz=1C5GCEM_enIN1203IN1204&amp;oq=Mistral&amp;gs_lcrp=EgZjaHJvbWUyBggAEEUYOdIBBzE1NWowajmoAgawAgHxBbtBKgwfKS8O&amp;sourceid=chrome&amp;ie=UTF-8#:~:text=Mistral%20AI%3A%20Frontier,mistral.ai%20%E2%80%BA%20..."><strong>Mistral</strong></a> and <a href="https://gemini.google.com/?hl=en-IN"><strong>Gemini</strong></a> optimize differently for multilingual token efficiency</li></ul><p>The same sentence can produce very different token structures across models.</p><p>That changes how attention is distributed internally.</p><p>The problem is not just model quality.</p><p>The problem is that token structure influences how the system “<strong>experiences</strong>” context.</p><h3>3. RAG retrieval inconsistency</h3><p>In enterprise systems, prompts rarely go directly to the model anymore.</p><p>They flow through:</p><ul><li>vector databases</li><li>embedding models</li><li>rerankers</li><li>memory systems</li><li>orchestration frameworks</li><li>guardrail engines</li></ul><p>A small tokenization shift can retrieve:</p><ul><li>different chunks</li><li>different memories</li><li>different ranking orders</li></ul><p><em>Now the reasoning chain changes entirely.</em></p><p>This is one reason why many enterprise RAG systems appear “unstable” even when prompts are unchanged.</p><p>And this is where a commonly accepted belief starts breaking down:</p><p>Most teams think <strong>prompt engineering</strong> is the primary reliability layer.<br>In production systems, <strong>retrieval determinism</strong> matters far more than prompt wording.</p><p><em>A beautifully engineered prompt cannot compensate for inconsistent retrieval paths.</em></p><h3>4. Multi-region deployment variance</h3><p>This is extremely common in large organizations.</p><p>One region may run:</p><ul><li>different tokenizer libraries</li><li>different orchestration services</li><li>different model containers</li><li>different cache layers</li><li>different safety middleware</li></ul><p>Teams think they are testing <strong>“the same system.”</strong></p><blockquote><strong>They are not.</strong></blockquote><p>This is classic distributed systems drift appearing inside <strong>AI infrastructure</strong>.</p><p>The industry often talks about “<strong>AI variability</strong>” as if it is purely model behavior.</p><p>In many enterprise environments, the variability is architectural.</p><h3>Why this becomes dangerous in AI agents</h3><p>For chatbots, inconsistency is annoying.</p><p>For agents, inconsistency becomes operational risk.</p><p>Because agents:</p><ul><li>plan</li><li>reason</li><li>call tools</li><li>maintain memory</li><li>execute workflows</li><li>chain decisions autonomously</li></ul><p><strong>Small tokenization</strong> drift can lead to:</p><ul><li>different planning paths</li><li>incorrect tool selection</li><li>unsafe parameter generation</li><li>inconsistent memory recall</li><li>broken workflows</li></ul><p>Example:</p><p>An incident-resolution agent receives:</p><blockquote><strong>“Rollback the last deployment if CPU stays above 85% for 10 minutes.”</strong></blockquote><p>A small retrieval or token interpretation change may:</p><ul><li>reference the wrong deployment</li><li>alter escalation behavior</li><li>skip approval logic</li><li>trigger rollback prematurely</li></ul><p>Now the issue is no longer “AI variability.”</p><p>It becomes:</p><ul><li>governance risk</li><li>audit risk</li><li>reliability risk</li><li>operational instability</li></ul><h3>This is fundamentally a distributed systems problem</h3><p>They treat agents like smarter <strong>chatbots</strong>.</p><p>But agents are runtime systems with execution authority.</p><p>That changes everything.</p><p><strong>This Is Fundamentally a Distributed Systems Problem!</strong></p><p>This is the deeper shift happening underneath enterprise AI.</p><p><strong>Production AI </strong>behaves less like a standalone model and more like:</p><ul><li>distributed systems</li><li>probabilistic infrastructure</li><li>stateful orchestration platforms</li></ul><p>The problem is not just the model.</p><p>It’s:</p><ul><li>orchestration</li><li>memory consistency</li><li>retrieval determinism</li><li>policy enforcement</li><li>runtime state synchronization</li></ul><p>In many ways, modern agent architectures are rediscovering old distributed systems problems:</p><ul><li>eventual consistency</li><li>cascading failures</li><li>retry storms</li><li>state drift</li><li>synchronization gaps</li></ul><p>just wrapped in <strong>AI terminology</strong>.</p><p>In my experience, organizations that succeed with AI at scale usually have strong platform engineering discipline long before they have sophisticated prompting strategies.</p><p>Because eventually every AI problem becomes an infrastructure problem.</p><p><strong>The Industry Is Already Responding!</strong></p><p>This is exactly why newer enterprise AI standards and frameworks are increasingly focused on:</p><ul><li>observability</li><li>governance</li><li>traceability</li><li>reproducibility</li></ul><p>We are seeing this across the ecosystem:</p><h3>OWASP Agentic AI Security Project</h3><p>OWASP’s emerging guidance for agentic applications highlights:</p><ul><li>goal hijacking</li><li>memory poisoning</li><li>cascading failures</li><li>tool abuse</li><li>agent identity misuse</li></ul><p>Many of these risks become harder to debug under tokenization and orchestration drift.</p><h3>NIST AI RMF (AI Risk Management Framework)</h3><p>NIST increasingly emphasizes:</p><ul><li>continuous monitoring</li><li>runtime governance</li><li>explainability</li><li>traceability</li><li>system reliability</li></ul><p>Not just model accuracy.</p><p>Because enterprise AI risk is now operational risk.</p><h3>ISO/IEC 42001</h3><p>The new AI management system standard pushes organizations toward:</p><ul><li>AI governance controls</li><li>lifecycle monitoring</li><li>accountability mechanisms</li><li>risk management processes</li></ul><p>Again, the focus is shifting from models to systems.</p><h3>OpenTelemetry and AI Observability</h3><p>One of the biggest shifts in the last 12–18 months has been the rise of AI observability.</p><p>Teams are now instrumenting:</p><p>A major trend in the last 12–18 months is AI observability.</p><p>Teams are instrumenting:</p><ul><li>prompts</li><li>token usage</li><li>retrieval traces</li><li>agent workflows</li><li>tool calls</li><li>memory state</li><li>latency paths</li></ul><p>because debugging AI systems without traces is becoming impossible.</p><p>This is why platforms like:</p><ul><li><a href="https://smith.langchain.com/">LangSmith</a></li><li><a href="https://openlit.io/">OpenLIT</a></li><li><a href="https://arize.com/">Arize</a></li><li><a href="https://wandb.ai/site/weave/">Weights &amp; Biases Weave</a></li><li><a href="https://www.google.com/search?q=Helicone&amp;rlz=1C5GCEM_enIN1203IN1204&amp;oq=Helicone&amp;gs_lcrp=EgZjaHJvbWUyBggAEEUYOdIBBzQwMGowajeoAgCwAgA&amp;sourceid=chrome&amp;ie=UTF-8#:~:text=Helicone%20/%20AI%20Gateway,www.helicone.ai">Helicone</a></li><li><a href="https://www.datadoghq.com/product/llm-observability/">Datadog</a> LLM Observability</li></ul><p>are growing rapidly.</p><h3>What mature teams do differently</h3><p>In my experience, teams that successfully scale AI systems focus heavily on reproducibility engineering.</p><p>Not just prompting.</p><h3>Practical controls that actually help</h3><h3>Lock tokenizer + model versions</h3><ul><li>Treat tokenizer changes like schema migrations.</li><li>Never update silently in production.</li></ul><h3>Standardize preprocessing</h3><p>Normalize:</p><ul><li>whitespace</li><li>Unicode</li><li>timestamps</li><li>formatting</li><li>metadata injection</li></ul><p>before inference.</p><h3>Log token-level traces</h3><p>Most teams log prompts.</p><p>Very few log:</p><ul><li>token boundaries</li><li>token counts</li><li>retrieved chunks</li><li>ranking scores</li><li>orchestration paths</li><li>memory retrieval decisions</li></ul><p>Without that, debugging becomes guesswork.</p><h3>Run continuous regression evaluations</h3><p>Not once.</p><p>Continuously.</p><p>Run the same prompts across:</p><ul><li>regions</li><li>time windows</li><li>orchestration paths</li><li>model versions</li></ul><p>Track behavioral drift over time.</p><p>This is becoming a core AI platform engineering capability.</p><h3>Build bounded autonomy</h3><p>Agents should never have unrestricted execution paths.</p><p>Use:</p><ul><li>policy engines</li><li>approval gates</li><li>rollback controls</li><li>deterministic tool schemas</li><li>scoped permissions</li></ul><p>especially for state-changing operations.</p><p>The future of enterprise AI will not be defined by maximum autonomy.</p><p>It will be defined by <strong>controlled autonomy</strong>.</p><h3>A bigger shift is happening</h3><p>As AI moves from:</p><p><strong>“generate text” to “execute workflows” — d<em>eterminism and governance become core engineering problems.</em></strong></p><p>This is why the next generation of <strong>AI engineering</strong> will not be won by:</p><ul><li>bigger prompts</li><li>larger context windows</li><li>more autonomous agents</li></ul><p>It will be won by:</p><ul><li>orchestration reliability</li><li>runtime governance</li><li>observability</li><li>reproducibility</li><li>systems discipline</li></ul><p>Exactly the same principles that scaled cloud platforms, microservices, and distributed systems.</p><p>I recently explored related themes around runtime controls, governance, and operational guardrails in my newsletter <strong><em>Scaling AI Without Hype</em></strong>.</p><p>Because increasingly, the hardest AI problems are no longer model problems.</p><blockquote><strong>They are systems problems.</strong></blockquote><h3>Let’s Connect</h3><p>If this perspective on AI-native enterprises resonated with you:</p><ul><li>Follow me on <a href="https://www.linkedin.com/in/puspanjalisarma/"><strong>Linkedin</strong></a> , <a href="https://github.com/puspanjalis"><strong>Github</strong></a><strong> </strong>and <a href="https://medium.com/@puspanjalisarma23"><strong>Medium</strong></a><strong> </strong>for practical insights on AI engineering, agentic systems, and enterprise-scale AI</li><li>Subscribe to my newsletter <em>“</em><a href="https://www.linkedin.com/build-relation/newsletter-follow?entityUrn=7412051308393787392"><strong><em>Scaling AI Without Hype</em></strong></a><em>”</em> for real-world frameworks and no-nonsense playbooks</li><li>Share your perspective — <strong><em>how is your organization evolving toward AI-native?</em></strong></li><li>Reach out if you’re building or scaling AI platforms — open to meaningful discussions and collaboration</li></ul><h3>Go Deeper</h3><p>For a structured, leadership-level approach to building AI-native organizations:</p><ul><li>Refer to my book <strong><em>“</em></strong><a href="https://www.amazon.in/Strategic-Leadership-Through-Data-transformation/dp/9365895758"><strong><em>Strategic AI Leadership Through Data</em></strong></a><strong><em>”</em></strong> — a practical guide to scaling AI, designing platforms, and aligning data strategy with business impact</li></ul><h3>Join the Conversation</h3><p>I am curious to know..</p><ul><li><em>Have you seen “same prompt, different behavior” issues in production AI systems?</em></li><li><em>And were you able to trace the real root cause?</em></li></ul><p>Add your thoughts below.👇</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f39475c0c9af" width="1" height="1" alt=""><hr><p><a href="https://blog.gopenai.com/why-does-the-same-prompt-give-different-answers-at-different-times-days-or-environments-f39475c0c9af">Why does the same prompt give different answers at different times, days, or environments?</a> was originally published in <a href="https://blog.gopenai.com/">GoPenAI</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Building a Robust Synthetic Data Platform on Snowflake]]></title>
            <link>https://medium.com/@puspanjalisarma23/building-a-robust-synthetic-data-platform-on-snowflake-29ea9061551a?source=rss-657cc3b2578c------2</link>
            <guid isPermaLink="false">https://medium.com/p/29ea9061551a</guid>
            <category><![CDATA[generative-ai-tools]]></category>
            <category><![CDATA[synthetic-data]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[snowflake]]></category>
            <dc:creator><![CDATA[Puspanjali Sarma]]></dc:creator>
            <pubDate>Thu, 14 May 2026 10:56:36 GMT</pubDate>
            <atom:updated>2026-05-14T10:56:36.019Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*OdJ5TB1hCSFLL8Khiu8RSA.png" /></figure><p>Synthetic data unlocks faster experimentation, safer collaboration, and cheaper iteration — <strong>without touching sensitive production rows</strong>. This article walks you through a complete, production-grade solution on Snowflake that combines:</p><ul><li><strong>Snowflake’s native generator</strong>: SNOWFLAKE.DATA_PRIVACY.GENERATE_SYNTHETIC_DATA</li><li>A <strong>fault-tolerant fallback</strong>: an <strong>empirical</strong> (distribution-aware) Python Snowpark stored procedure</li><li>A <strong>Streamlit</strong> app (running inside Snowflake) that lets users select objects and generate synthetic tables with <strong>_SYN</strong> suffix, previewing rows inline</li></ul><p>You’ll get <strong>ready-to-paste code</strong>, a suggested <a href="https://github.com/puspanjalis/snowflake_synthetic_data_platform">GitHub</a> repo layout, deploy steps, and a clean UI.</p><h3>What you’ll build</h3><p>A <strong>wrapper procedure</strong> that:</p><ul><li>Pre-validates inputs (≥20 distinct rows, ≤100 cols, supported types)</li><li>Calls the <strong>native</strong> generator in batches of 5 objects</li><li>Falls back to the <strong>empirical</strong> generator per object on error or when native is unavailable,</li><li>Optionally validates referential integrity for base tables.</li><li>Returns a structured JSON report (batches, processed, skipped, errors)</li></ul><p>An <strong>empirical generator</strong> SP (drop-in replacement for naive Faker):</p><ul><li>Learns distributions from a sample of the source table/view,</li><li>Preserves categories/labels, numeric ranges, null rates, and date patterns.</li><li>Coerces data to target Snowflake types before loading.</li><li>Writes results to &lt;DB&gt;.&lt;SCHEMA&gt;.&lt;TABLE&gt;_SYN</li></ul><p>A <strong>Streamlit</strong> app:</p><ul><li>Radio to pick <strong>Native Wrapper</strong> or <strong>Empirical Fallback.</strong></li><li>Dropdown of schema objects (tables + views)</li><li>“Rows to generate” input (used by empirical path).</li><li>One-click <strong>Run</strong>, with <strong>10-row previews</strong> of generated _SYN tables</li></ul><h3>Repo structure</h3><pre>snowflake-synth/<br>├─ README.md<br>├─ LICENSE<br>├─ snowflake/<br>│  ├─ procedures/<br>│  │  ├─ sp_fake_synthetic_data.sql<br>│  │  └─ generate_and_validate_synthetic_data_audit.sql<br>│  └─ streamlit/<br>│     └─ app.py<br>└─ docs/<br>   └─ troubleshooting.md</pre><p>You can copy each file below into this layout.</p><h3>Prerequisites</h3><p>A Snowflake account with a role that can:</p><ul><li>USAGE on target database &amp; schema</li><li>CREATE TABLE, INSERT on target schema(s)</li><li>USAGE on SNOWFLAKE database and SNOWFLAKE.DATA_PRIVACY schema (for native generator)</li><li>Compute warehouse for the Streamlit app and procedures</li><li>Streamlit-in-Snowflake enabled</li></ul><blockquote><strong><em>Native generator availability</em></strong><em> varies by account/edition/region. If it’s not present, the solution automatically falls back to the empirical SP.</em></blockquote><h3>1) Empirical generator SP (fallback)</h3><p><strong>File:</strong> snowflake/procedures/sp_fake_synthetic_data.sql</p><pre>CREATE OR REPLACE PROCEDURE SP_FAKE_SYNTHETIC_DATA(<br>  INPUT_DB STRING,<br>  INPUT_SCHEMA STRING,<br>  INPUT_TABLE STRING,<br>  OUTPUT_SCHEMA STRING,<br>  ROW_LIMIT INT<br>)<br>RETURNS STRING<br>LANGUAGE PYTHON<br>RUNTIME_VERSION = &#39;3.11&#39;<br>PACKAGES = (&#39;snowflake-snowpark-python&#39;,&#39;pandas&#39;,&#39;numpy&#39;)<br>HANDLER = &#39;run&#39;<br>EXECUTE AS CALLER<br>AS<br>$$<br>def run(session, INPUT_DB, INPUT_SCHEMA, INPUT_TABLE, OUTPUT_SCHEMA, ROW_LIMIT):<br>    import json<br>    import pandas as pd, numpy as np, math<br><br>    source_fq = f&quot;{INPUT_DB}.{INPUT_SCHEMA}.{INPUT_TABLE}&quot;<br>    target_fq = f&quot;{INPUT_DB}.{OUTPUT_SCHEMA}.{INPUT_TABLE}_SYN&quot;<br>    RESULT = {&quot;status&quot;:&quot;OK&quot;,&quot;target&quot;:target_fq,&quot;generated&quot;:0,&quot;staged&quot;:0,&quot;inserted&quot;:0,&quot;error&quot;:&quot;&quot;}<br><br>    try:<br>        # Create target with identical structure (works for tables or views)<br>        session.sql(f&quot;CREATE OR REPLACE TABLE {target_fq} AS SELECT * FROM {source_fq} WHERE 1=0&quot;).collect()<br><br>        # Sample source to learn distributions<br>        src_df = session.table(source_fq).limit(100000)<br>        if src_df.count() == 0:<br>            RESULT.update(status=&quot;ERROR&quot;, error=f&quot;{source_fq} has no rows&quot;)<br>            return json.dumps(RESULT)<br><br>        pdf = src_df.to_pandas()<br>        if pdf.empty:<br>            RESULT.update(status=&quot;ERROR&quot;, error=f&quot;{source_fq} sample empty&quot;)<br>            return json.dumps(RESULT)<br><br>        pdf.columns = [str(c).upper() for c in pdf.columns]<br>        rng = np.random.default_rng()<br><br>        def sample_with_nulls(series, n):<br>            null_rate = series.isna().mean() if len(series) else 0.0<br>            nonnull = series.dropna()<br>            out = np.empty(n, dtype=object)<br>            out[:] = rng.choice(nonnull.values, size=n, replace=True) if not nonnull.empty else None<br>            if null_rate &gt; 0:<br>                out[rng.random(n) &lt; null_rate] = None<br>            return out<br><br>        # Build synthetic data per column (empirical)<br>        syn_cols = {}<br>        for col in pdf.columns:<br>            s = pdf[col]<br>            if s.dtype == &#39;object&#39;:<br>                syn_cols[col] = sample_with_nulls(s, ROW_LIMIT)<br><br>            elif pd.api.types.is_integer_dtype(s) or pd.api.types.is_float_dtype(s):<br>                nonnull = pd.to_numeric(s, errors=&quot;coerce&quot;).dropna()<br>                if nonnull.empty:<br>                    syn_cols[col] = np.array([None]*ROW_LIMIT, dtype=object); continue<br>                mu = float(nonnull.mean())<br>                sd_raw = float(nonnull.std()) if not math.isnan(nonnull.std()) else 0.0<br>                sd = sd_raw if sd_raw &gt; 0 else max(abs(mu)*0.05, 1.0)<br>                draws_emp  = rng.choice(nonnull.values, size=ROW_LIMIT, replace=True).astype(float)<br>                draws_gaus = rng.normal(loc=mu, scale=sd, size=ROW_LIMIT)<br>                mix_mask   = rng.random(ROW_LIMIT) &lt; 0.30<br>                mixed = draws_emp<br>                mixed[mix_mask] = draws_gaus[mix_mask]<br>                lo, hi = float(nonnull.min()), float(nonnull.max())<br>                mixed = np.clip(mixed, lo, hi)<br>                mixed = np.round(mixed).astype(object) if pd.api.types.is_integer_dtype(s) \<br>                        else np.round(mixed, 6).astype(object)<br>                null_rate = s.isna().mean()<br>                if null_rate &gt; 0: mixed[rng.random(ROW_LIMIT) &lt; null_rate] = None<br>                syn_cols[col] = mixed<br><br>            elif pd.api.types.is_datetime64_any_dtype(s):<br>                syn_cols[col] = sample_with_nulls(s.astype(&#39;datetime64[ns]&#39;), ROW_LIMIT)<br><br>            elif pd.api.types.is_bool_dtype(s):<br>                nonnull = s.dropna()<br>                p_true = float(nonnull.mean()) if not nonnull.empty else 0.5<br>                vals = (rng.random(ROW_LIMIT) &lt; p_true).astype(object)<br>                null_rate = s.isna().mean()<br>                if null_rate &gt; 0: vals[rng.random(ROW_LIMIT) &lt; null_rate] = None<br>                syn_cols[col] = vals<br><br>            else:<br>                syn_cols[col] = sample_with_nulls(s.astype(object), ROW_LIMIT)<br><br>        syn_pdf = pd.DataFrame(syn_cols)<br>        syn_pdf.columns = [str(c).upper() for c in syn_pdf.columns]<br>        RESULT[&quot;generated&quot;] = len(syn_pdf)<br><br>        # Target DDL &amp; types<br>        cols_meta = session.sql(f&quot;&quot;&quot;<br>            SELECT COLUMN_NAME, DATA_TYPE<br>            FROM {INPUT_DB}.INFORMATION_SCHEMA.COLUMNS<br>            WHERE TABLE_SCHEMA = &#39;{OUTPUT_SCHEMA}&#39; AND TABLE_NAME = &#39;{INPUT_TABLE}_SYN&#39;<br>            ORDER BY ORDINAL_POSITION<br>        &quot;&quot;&quot;).collect()<br>        target_cols  = [r[0] for r in cols_meta]<br>        target_types = {r[0]: r[1].upper() for r in cols_meta}<br><br>        # Coerce pandas dtypes to Snowflake target types to avoid NUMBER→TIMESTAMP errors, etc.<br>        for c in target_cols:<br>            if c not in syn_pdf.columns: syn_pdf[c] = None<br>            dt = target_types[c]<br>            try:<br>                if dt.startswith(&#39;TIMESTAMP&#39;):<br>                    syn_pdf[c] = pd.to_datetime(syn_pdf[c], errors=&#39;coerce&#39;)<br>                elif dt == &#39;DATE&#39;:<br>                    syn_pdf[c] = pd.to_datetime(syn_pdf[c], errors=&#39;coerce&#39;).dt.date<br>                elif dt.startswith((&#39;NUMBER&#39;,&#39;DECIMAL&#39;,&#39;FLOAT&#39;,&#39;DOUBLE&#39;)):<br>                    syn_pdf[c] = pd.to_numeric(syn_pdf[c], errors=&#39;coerce&#39;)<br>                elif dt == &#39;BOOLEAN&#39;:<br>                    syn_pdf[c] = syn_pdf[c].astype(&#39;boolean&#39;)<br>                else:<br>                    syn_pdf[c] = syn_pdf[c].astype(object)<br>            except Exception:<br>                pass<br><br>        syn_pdf = syn_pdf[target_cols]<br><br>        # Stage → Target using Snowpark (robust inside SPs)<br>        stage_name = f&quot;{INPUT_TABLE}_SYN_STAGE_{abs(np.random.default_rng().integers(1,1_000_000))}&quot;<br>        stage_fq   = f&quot;{INPUT_DB}.{OUTPUT_SCHEMA}.{stage_name}&quot;<br>        session.sql(f&quot;CREATE OR REPLACE TABLE {stage_fq} LIKE {target_fq}&quot;).collect()<br>        session.sql(f&quot;TRUNCATE TABLE {stage_fq}&quot;).collect()<br><br>        spdf = session.create_dataframe(syn_pdf).to_df(*syn_pdf.columns.tolist())<br>        spdf.write.mode(&quot;append&quot;).save_as_table(stage_fq)<br>        RESULT[&quot;staged&quot;] = session.sql(f&quot;SELECT COUNT(*) C FROM {stage_fq}&quot;).collect()[0][&#39;C&#39;]<br><br>        session.sql(f&quot;TRUNCATE TABLE {target_fq}&quot;).collect()<br>        col_list = &quot;, &quot;.join(target_cols)<br>        session.sql(f&quot;INSERT INTO {target_fq} ({col_list}) SELECT {col_list} FROM {stage_fq}&quot;).collect()<br>        RESULT[&quot;inserted&quot;] = session.sql(f&quot;SELECT COUNT(*) C FROM {target_fq}&quot;).collect()[0][&#39;C&#39;]<br><br>        session.sql(f&quot;DROP TABLE {stage_fq}&quot;).collect()<br>        return json.dumps(RESULT)<br><br>    except Exception as e:<br>        RESULT.update(status=&quot;ERROR&quot;, error=str(e))<br>        return json.dumps(RESULT)<br>$$;    source_fq = f&quot;{INPUT_DB}.{INPUT_SCHEMA}.{INPUT_TABLE}&quot;<br>    target_fq = f&quot;{INPUT_DB}.{OUTPUT_SCHEMA}.{INPUT_TABLE}_SYN&quot;<br>    RESULT = {&quot;status&quot;:&quot;OK&quot;,&quot;target&quot;:target_fq,&quot;generated&quot;:0,&quot;staged&quot;:0,&quot;inserted&quot;:0,&quot;error&quot;:&quot;&quot;}    try:<br>        # Create target with identical structure (works for tables or views)<br>        session.sql(f&quot;CREATE OR REPLACE TABLE {target_fq} AS SELECT * FROM {source_fq} WHERE 1=0&quot;).collect()</pre><h3>2) Wrapper SP (native + fallback + batching + validation)</h3><p><strong>File:</strong> snowflake/procedures/generate_and_validate_synthetic_data_audit.sql</p><pre>CREATE OR REPLACE PROCEDURE GENERATE_AND_VALIDATE_SYNTHETIC_DATA_AUDIT(<br>    IN_DB STRING,<br>    IN_SCHEMA STRING,<br>    IN_OBJECTS ARRAY      -- NULL/empty =&gt; process all eligible objects in schema<br>)<br>RETURNS VARIANT<br>LANGUAGE PYTHON<br>RUNTIME_VERSION = &#39;3.11&#39;<br>PACKAGES = (&#39;snowflake-snowpark-python&#39;,&#39;pandas&#39;,&#39;numpy&#39;)<br>HANDLER = &#39;run&#39;<br>EXECUTE AS CALLER<br>AS<br>$$<br>def run(session, IN_DB: str, IN_SCHEMA: str, IN_OBJECTS):<br>    import json, math<br>    import pandas as pd<br>    import numpy as np<br><br>    RESULT = {<br>        &quot;input&quot;: {&quot;db&quot;: IN_DB, &quot;schema&quot;: IN_SCHEMA},<br>        &quot;objects_considered&quot;: [],<br>        &quot;objects_skipped&quot;: [],<br>        &quot;objects_processed&quot;: [],<br>        &quot;batches&quot;: [],<br>        &quot;fk_validation&quot;: [],<br>        &quot;native_available&quot;: False,<br>        &quot;status&quot;: &quot;SUCCESS&quot;,<br>        &quot;error&quot;: &quot;&quot;<br>    }<br><br>    def sql_df(q): return session.sql(q).to_pandas()<br>    def obj_fq(name): return f&quot;{IN_DB}.{IN_SCHEMA}.{name}&quot;<br>    def output_name(name): return f&quot;{name}_SYN&quot;<br><br>    def native_available():<br>        try:<br>            rows = session.sql(&quot;&quot;&quot;<br>              SHOW PROCEDURES LIKE &#39;GENERATE_SYNTHETIC_DATA&#39; IN SCHEMA SNOWFLAKE.DATA_PRIVACY<br>            &quot;&quot;&quot;).collect()<br>            return len(rows) &gt; 0<br>        except Exception:<br>            return False<br><br>    def eligible_objects():<br>        q = f&quot;&quot;&quot;<br>        SELECT TABLE_NAME, TABLE_TYPE<br>        FROM &quot;{IN_DB}&quot;.INFORMATION_SCHEMA.TABLES<br>        WHERE UPPER(TABLE_SCHEMA) = UPPER(&#39;{IN_SCHEMA}&#39;)<br>          AND TABLE_TYPE IN (&#39;BASE TABLE&#39;,&#39;VIEW&#39;,&#39;MATERIALIZED VIEW&#39;)<br>        ORDER BY TABLE_NAME<br>        &quot;&quot;&quot;<br>        pdf = sql_df(q)<br>        if IN_OBJECTS is not None and len(IN_OBJECTS) &gt; 0:<br>            want = set([str(x) for x in IN_OBJECTS])<br>            pdf = pdf[pdf[&#39;TABLE_NAME&#39;].isin(want)]<br>        return pdf<br><br>    def validate_input_object(name):<br>        cols = sql_df(f&quot;&quot;&quot;<br>          SELECT COLUMN_NAME, DATA_TYPE<br>          FROM &quot;{IN_DB}&quot;.INFORMATION_SCHEMA.COLUMNS<br>          WHERE UPPER(TABLE_SCHEMA) = UPPER(&#39;{IN_SCHEMA}&#39;) AND TABLE_NAME = &#39;{name}&#39;<br>          ORDER BY ORDINAL_POSITION<br>        &quot;&quot;&quot;)<br>        if cols.empty: return (False, &quot;No columns resolved&quot;)<br>        if cols.shape[0] &gt; 100: return (False, f&quot;Too many columns ({cols.shape[0]} &gt; 100)&quot;)<br>        unsupported = set([&quot;VARIANT&quot;,&quot;ARRAY&quot;,&quot;OBJECT&quot;,&quot;TIMESTAMP_TZ&quot;])<br>        bad = cols[cols[&#39;DATA_TYPE&#39;].str.upper().isin(unsupported)]<br>        if not bad.empty:<br>            return (False, f&quot;Unsupported column types: {&#39;, &#39;.join(sorted(set(bad[&#39;DATA_TYPE&#39;].str.upper())))}&quot;)<br>        try:<br>            cnt = session.sql(f&quot;&quot;&quot;<br>              SELECT COUNT(DISTINCT SHA1(TO_VARIANT(OBJECT_CONSTRUCT(*)))) AS CNT<br>              FROM {obj_fq(name)}<br>            &quot;&quot;&quot;).collect()[0][&quot;CNT&quot;]<br>        except Exception as e:<br>            return (False, f&quot;Row count check failed: {e}&quot;)<br>        if cnt is None or int(cnt) &lt; 20:<br>            return (False, f&quot;Less than 20 distinct rows ({cnt})&quot;)<br>        return (True, &quot;OK&quot;)<br><br>    def call_native(batch):<br>        payload = {<br>            &quot;datasets&quot;: [{<br>                &quot;input_table&quot;: obj_fq(n),<br>                &quot;output_table&quot;: f&quot;{IN_DB}.{IN_SCHEMA}.{output_name(n)}&quot;<br>            } for n in batch],<br>            &quot;replace_output_tables&quot;: True,<br>            &quot;similarity_filter&quot;: False<br>        }<br>        session.sql(f&quot;&quot;&quot;<br>          CALL SNOWFLAKE.DATA_PRIVACY.GENERATE_SYNTHETIC_DATA(<br>            PARSE_JSON(&#39;{json.dumps(payload)}&#39;)<br>          )<br>        &quot;&quot;&quot;).collect()<br><br>    def call_fallback(obj_name):<br>        return session.call(&quot;SP_FAKE_SYNTHETIC_DATA&quot;, IN_DB, IN_SCHEMA, obj_name, IN_SCHEMA, 500)<br><br>    # Discover + pre-validate<br>    try:<br>        objs_pdf = eligible_objects()<br>        if objs_pdf.empty:<br>            RESULT[&quot;status&quot;] = &quot;NO_OBJECTS&quot;<br>            RESULT[&quot;error&quot;] = f&quot;No tables/views found in {IN_DB}.{IN_SCHEMA}&quot;<br>            return RESULT<br><br>        candidate_names = [str(x) for x in (IN_OBJECTS if IN_OBJECTS else [])]<br>        RESULT[&quot;objects_considered&quot;] = [] if candidate_names else objs_pdf[&quot;TABLE_NAME&quot;].tolist()<br><br>        if candidate_names:<br>            missing = sorted(list(set(candidate_names) - set(objs_pdf[&quot;TABLE_NAME&quot;].tolist())))<br>            for m in missing:<br>                RESULT[&quot;objects_skipped&quot;].append({&quot;name&quot;: m, &quot;reason&quot;: &quot;Not found in schema&quot;})<br>            objs_pdf = objs_pdf[objs_pdf[&quot;TABLE_NAME&quot;].isin(candidate_names)]<br><br>        eligible = []<br>        for _, row in objs_pdf.iterrows():<br>            name = row[&quot;TABLE_NAME&quot;]<br>            ok, reason = validate_input_object(name)<br>            if ok:<br>                eligible.append((name, row[&quot;TABLE_TYPE&quot;]))<br>            else:<br>                RESULT[&quot;objects_skipped&quot;].append({&quot;name&quot;: name, &quot;reason&quot;: reason})<br><br>        if not eligible:<br>            RESULT[&quot;status&quot;] = &quot;NO_ELIGIBLE_OBJECTS&quot;<br>            RESULT[&quot;error&quot;] = &quot;All requested objects failed pre-validation.&quot;<br>            return RESULT<br><br>        RESULT[&quot;native_available&quot;] = native_available()<br>    except Exception as e:<br>        RESULT[&quot;status&quot;] = &quot;FAILED&quot;<br>        RESULT[&quot;error&quot;] = f&quot;Discovery/validation failed: {e}&quot;<br>        return RESULT<br><br>    # Generate in batches (&lt;=5 per native call)<br>    names_only = [n for (n, t) in eligible]<br>    batch_size = 5<br>    for i in range(0, len(names_only), batch_size):<br>        batch = names_only[i:i+batch_size]<br>        batch_report = {&quot;batch&quot;: f&quot;{i//batch_size+1}/{math.ceil(len(names_only)/batch_size)}&quot;, &quot;objects&quot;: batch, &quot;mode&quot;: &quot;&quot;, &quot;result&quot;: &quot;SUCCESS&quot;, &quot;errors&quot;: []}<br>        try:<br>            if RESULT[&quot;native_available&quot;]:<br>                try:<br>                    call_native(batch)<br>                    batch_report[&quot;mode&quot;] = &quot;NATIVE&quot;<br>                    RESULT[&quot;objects_processed&quot;] += [{&quot;name&quot;: n, &quot;output&quot;: output_name(n), &quot;mode&quot;: &quot;NATIVE&quot;} for n in batch]<br>                except Exception as ne:<br>                    batch_report[&quot;mode&quot;] = &quot;FALLBACK_PER_OBJECT&quot;<br>                    for n in batch:<br>                        try:<br>                            msg = call_fallback(n)<br>                            RESULT[&quot;objects_processed&quot;].append({&quot;name&quot;: n, &quot;output&quot;: output_name(n), &quot;mode&quot;: &quot;FALLBACK&quot;, &quot;detail&quot;: str(msg)})<br>                        except Exception as fe:<br>                            batch_report[&quot;errors&quot;].append(f&quot;{n}: {fe}&quot;)<br>                            RESULT[&quot;objects_skipped&quot;].append({&quot;name&quot;: n, &quot;reason&quot;: f&quot;Native+Fallback failed: {fe}&quot;})<br>            else:<br>                batch_report[&quot;mode&quot;] = &quot;FALLBACK_PER_OBJECT&quot;<br>                for n in batch:<br>                    try:<br>                        msg = call_fallback(n)<br>                        RESULT[&quot;objects_processed&quot;].append({&quot;name&quot;: n, &quot;output&quot;: output_name(n), &quot;mode&quot;: &quot;FALLBACK&quot;, &quot;detail&quot;: str(msg)})<br>                    except Exception as fe:<br>                        batch_report[&quot;errors&quot;].append(f&quot;{n}: {fe}&quot;)<br>                        RESULT[&quot;objects_skipped&quot;].append({&quot;name&quot;: n, &quot;reason&quot;: f&quot;Fallback failed: {fe}&quot;})<br>        except Exception as e:<br>            batch_report[&quot;result&quot;] = &quot;FAILED&quot;<br>            batch_report[&quot;errors&quot;].append(str(e))<br>        RESULT[&quot;batches&quot;].append(batch_report)<br><br>    # Optional FK validation (base tables only). If INFORMATION_SCHEMA isn’t accessible, we record the error and continue.<br>    try:<br>        fk_q = f&quot;&quot;&quot;<br>        SELECT<br>          kcu.TABLE_NAME      AS CHILD_TABLE,<br>          kcu.COLUMN_NAME     AS CHILD_COLUMN,<br>          uk.TABLE_NAME       AS PARENT_TABLE,<br>          uk.COLUMN_NAME      AS PARENT_COLUMN<br>        FROM &quot;{IN_DB}&quot;.INFORMATION_SCHEMA.REFERENTIAL_CONSTRAINTS rc<br>        JOIN &quot;{IN_DB}&quot;.INFORMATION_SCHEMA.KEY_COLUMN_USAGE kcu<br>          ON rc.CONSTRAINT_NAME = kcu.CONSTRAINT_NAME AND UPPER(kcu.TABLE_SCHEMA) = UPPER(&#39;{IN_SCHEMA}&#39;)<br>        JOIN &quot;{IN_DB}&quot;.INFORMATION_SCHEMA.KEY_COLUMN_USAGE uk<br>          ON rc.UNIQUE_CONSTRAINT_NAME = uk.CONSTRAINT_NAME AND UPPER(uk.TABLE_SCHEMA) = UPPER(&#39;{IN_SCHEMA}&#39;)<br>        WHERE UPPER(rc.CONSTRAINT_SCHEMA) = UPPER(&#39;{IN_SCHEMA}&#39;)<br>        &quot;&quot;&quot;<br>        fk_pdf = sql_df(fk_q)<br>        if not fk_pdf.empty:<br>            base_tables = sql_df(f&quot;&quot;&quot;<br>              SELECT TABLE_NAME FROM &quot;{IN_DB}&quot;.INFORMATION_SCHEMA.TABLES<br>              WHERE UPPER(TABLE_SCHEMA) = UPPER(&#39;{IN_SCHEMA}&#39;) AND TABLE_TYPE = &#39;BASE TABLE&#39;<br>            &quot;&quot;&quot;)[&quot;TABLE_NAME&quot;].tolist()<br>            for _, r in fk_pdf.iterrows():<br>                ct, cc, pt, pc = r.CHILD_TABLE, r.CHILD_COLUMN, r.PARENT_TABLE, r.PARENT_COLUMN<br>                if ct in base_tables and pt in base_tables:<br>                    syn_child = obj_fq(output_name(ct))<br>                    syn_parent = obj_fq(output_name(pt))<br>                    try:<br>                        q = f&quot;&quot;&quot;<br>                        SELECT COUNT(*) AS BROKEN<br>                        FROM {syn_child} c<br>                        LEFT JOIN {syn_parent} p<br>                          ON c.{cc} = p.{pc}<br>                        WHERE p.{pc} IS NULL<br>                        &quot;&quot;&quot;<br>                        broken = session.sql(q).collect()[0][&quot;BROKEN&quot;]<br>                        RESULT[&quot;fk_validation&quot;].append({<br>                            &quot;relationship&quot;: f&quot;{ct}.{cc} -&gt; {pt}.{pc}&quot;,<br>                            &quot;broken_links&quot;: int(broken)<br>                        })<br>                    except Exception as ve:<br>                        RESULT[&quot;fk_validation&quot;].append({<br>                            &quot;relationship&quot;: f&quot;{ct}.{cc} -&gt; {pt}.{pc}&quot;,<br>                            &quot;error&quot;: str(ve)<br>                        })<br>    except Exception as e:<br>        RESULT[&quot;fk_validation&quot;].append({&quot;error&quot;: f&quot;FK validation failed: {e}&quot;})<br><br>    any_failed = any(b.get(&quot;errors&quot;) for b in RESULT[&quot;batches&quot;])<br>    if any_failed and not RESULT[&quot;objects_processed&quot;]:<br>        RESULT[&quot;status&quot;] = &quot;FAILED&quot;<br>    elif any_failed:<br>        RESULT[&quot;status&quot;] = &quot;PARTIAL_SUCCESS&quot;<br><br>    return RESULT<br>$$;</pre><h3>3) Streamlit app (radio + dropdown + preview)</h3><p><strong>File:</strong> snowflake/streamlit/app.py</p><pre>import json<br>import pandas as pd<br>import streamlit as st<br>from snowflake.snowpark.context import get_active_session<br><br>session = get_active_session()<br><br>st.title(&quot;Synthetic Data Generator&quot;)<br><br># ========= Inputs =========<br>with st.container():<br>    c1, c2 = st.columns(2)<br>    with c1:<br>        db_name = st.text_input(&quot;Database&quot;, value=&quot;&quot;, placeholder=&quot;e.g., EDW&quot;)<br>    with c2:<br>        schema_name = st.text_input(&quot;Schema&quot;, value=&quot;&quot;, placeholder=&quot;e.g., SALES&quot;)<br><br># ========= Helpers =========<br>def fetch_object_names(db: str, schema: str):<br>    try:<br>        q = f&quot;&quot;&quot;<br>        SELECT TABLE_NAME<br>        FROM &quot;{db}&quot;.INFORMATION_SCHEMA.TABLES<br>        WHERE UPPER(TABLE_SCHEMA) = UPPER(&#39;{schema}&#39;)<br>          AND TABLE_TYPE IN (&#39;BASE TABLE&#39;,&#39;VIEW&#39;,&#39;MATERIALIZED VIEW&#39;)<br>        ORDER BY TABLE_NAME<br>        &quot;&quot;&quot;<br>        df = session.sql(q).to_pandas()<br>        return df[&quot;TABLE_NAME&quot;].tolist()<br>    except Exception as e:<br>        st.error(f&quot;Error fetching objects from {db}.{schema}: {e}&quot;)<br>        return []<br><br>def array_construct_sql(names):<br>    if not names:<br>        return &quot;NULL&quot;<br>    esc = []<br>    for n in names:<br>        n_escaped = n.replace(&quot;&#39;&quot;, &quot;&#39;&#39;&quot;)<br>        esc.append(f&quot;&#39;{n_escaped}&#39;&quot;)<br>    return f&quot;ARRAY_CONSTRUCT({&#39;,&#39;.join(esc)})&quot;<br><br>def variant_row_to_json(row_obj):<br>    try:<br>        cell0 = row_obj[0]<br>    except Exception:<br>        cell0 = row_obj<br>    if isinstance(cell0, dict):<br>        return cell0<br>    if isinstance(cell0, str):<br>        try:<br>            return json.loads(cell0)<br>        except Exception:<br>            return {&quot;status&quot;:&quot;ERROR&quot;,&quot;error&quot;:f&quot;Unparseable JSON from SP: {cell0}&quot;}<br>    try:<br>        d = row_obj.asDict()<br>        if d:<br>            first_val = next(iter(d.values()))<br>            if isinstance(first_val, dict):<br>                return first_val<br>            if isinstance(first_val, str):<br>                try:<br>                    return json.loads(first_val)<br>                except Exception:<br>                    return {&quot;status&quot;:&quot;ERROR&quot;,&quot;error&quot;:f&quot;Unparseable JSON from SP: {first_val}&quot;}<br>    except Exception:<br>        pass<br>    return {&quot;status&quot;:&quot;ERROR&quot;,&quot;error&quot;:f&quot;Unexpected SP result type: {type(row_obj)}&quot;}<br><br>def preview_table(fq: str, limit: int = 10):<br>    try:<br>        df = session.sql(f&quot;SELECT * FROM {fq} LIMIT {limit}&quot;).to_pandas()<br>        st.dataframe(df, use_container_width=True)<br>    except Exception as e:<br>        st.error(f&quot;Preview failed for {fq}: {e}&quot;)<br><br># ========= UI controls =========<br>mode = st.radio(<br>    &quot;Select generation mode&quot;,<br>    [&quot;Generate via Wrapper (Recommended)&quot;, &quot;Debug / Direct Fallback (Empirical Generator)&quot;],<br>    index=0,<br>    horizontal=True<br>)<br><br>selected_objects = []<br>selected_objects_single = &quot;&quot;<br>all_objects = fetch_object_names(db_name, schema_name) if (db_name and schema_name) else []<br><br>if mode == &quot;Generate via Wrapper (Recommended)&quot;:<br>    selected_objects = st.multiselect(<br>        &quot;Select Tables/Views (leave empty to process all eligible)&quot;,<br>        options=all_objects,<br>        placeholder=&quot;Start typing to search…&quot;,<br>    )<br>else:<br>    selected_objects_single = st.selectbox(<br>        &quot;Select one Table/View&quot;,<br>        options=[&quot;&quot;] + all_objects,<br>        index=0,<br>        placeholder=&quot;Choose a single object&quot;,<br>    )<br><br>rows_to_generate = st.number_input(<br>    &quot;Rows to generate (used by Empirical Generator)&quot;,<br>    min_value=10, max_value=200000, value=500, step=100<br>)<br>show_preview = st.checkbox(&quot;Show 10-row preview after generation&quot;, value=True)<br><br>st.divider()<br><br># ========= Run =========<br>if st.button(&quot;Run&quot;):<br>    if not db_name or not schema_name:<br>        st.warning(&quot;Please provide both Database and Schema.&quot;)<br>        st.stop()<br><br>    try:<br>        if mode == &quot;Generate via Wrapper (Recommended)&quot;:<br>            arr_sql = array_construct_sql(selected_objects)<br>            with st.spinner(&quot;Generating synthetic data via wrapper…&quot;):<br>                row = session.sql(<br>                    f&quot;CALL GENERATE_AND_VALIDATE_SYNTHETIC_DATA_AUDIT(&#39;{db_name}&#39;,&#39;{schema_name}&#39;, {arr_sql})&quot;<br>                ).collect()[0]<br>            res = variant_row_to_json(row)<br><br>            st.write(f&quot;Overall status: {res.get(&#39;status&#39;)}&quot;)<br>            if res.get(&quot;error&quot;):<br>                st.error(f&quot;Top-level error: {res.get(&#39;error&#39;)}&quot;)<br><br>            processed = res.get(&quot;objects_processed&quot;, [])<br>            if processed:<br>                st.subheader(&quot;Synthetic objects created&quot;)<br>                for obj in processed:<br>                    out_name = obj.get(&quot;output&quot;)<br>                    if out_name:<br>                        fq = f&quot;{db_name}.{schema_name}.{out_name}&quot;<br>                        st.write(fq)<br>                        if show_preview:<br>                            preview_table(fq, limit=10)<br><br>            skipped = res.get(&quot;objects_skipped&quot;, [])<br>            if skipped:<br>                st.subheader(&quot;Skipped / Failed&quot;)<br>                st.dataframe(pd.DataFrame(skipped), use_container_width=True)<br><br>            batches = res.get(&quot;batches&quot;, [])<br>            if batches:<br>                with st.expander(&quot;Batch results&quot;):<br>                    rows = []<br>                    for b in batches:<br>                        rows.append({<br>                            &quot;batch&quot;: b.get(&quot;batch&quot;),<br>                            &quot;mode&quot;: b.get(&quot;mode&quot;),<br>                            &quot;result&quot;: b.get(&quot;result&quot;),<br>                            &quot;objects&quot;: &quot;, &quot;.join(b.get(&quot;objects&quot;, [])),<br>                            &quot;errors&quot;: &quot;; &quot;.join(b.get(&quot;errors&quot;) or [])<br>                        })<br>                    st.dataframe(pd.DataFrame(rows), use_container_width=True)<br><br>        else:<br>            if not (db_name and schema_name and selected_objects_single):<br>                st.warning(&quot;Provide Database, Schema, and choose a single object.&quot;)<br>                st.stop()<br><br>            with st.spinner(&quot;Generating synthetic data via empirical fallback…&quot;):<br>                resp = session.call(<br>                    &quot;SP_FAKE_SYNTHETIC_DATA&quot;,<br>                    db_name, schema_name, selected_objects_single, schema_name, int(rows_to_generate)<br>                )<br><br>            try:<br>                rj = json.loads(resp) if isinstance(resp, str) else resp<br>            except Exception:<br>                st.error(f&quot;Non-JSON response from SP_FAKE_SYNTHETIC_DATA: {resp}&quot;)<br>                rj = {&quot;status&quot;: &quot;ERROR&quot;, &quot;error&quot;: str(resp)}<br><br>            if rj.get(&quot;status&quot;) == &quot;OK&quot;:<br>                target = rj.get(&quot;target&quot;)<br>                inserted = rj.get(&quot;inserted&quot;, 0)<br>                st.subheader(&quot;Synthetic object created&quot;)<br>                st.write(target or &quot;(unknown)&quot;)<br>                st.write(f&quot;Inserted rows: {inserted}&quot;)<br>                if show_preview and target:<br>                    preview_table(target, limit=10)<br>            else:<br>                st.subheader(&quot;Error&quot;)<br>                st.error(rj.get(&quot;error&quot;))<br>                st.json(rj)<br><br>    except Exception as e:<br>        st.error(f&quot;Run failed: {e}&quot;)</pre><h3>Deployment</h3><ol><li><strong>Run the SQL files</strong> in your target DB/Schema (role with proper privileges):</li></ol><pre>-- Optional: verify native generator existence<br>SHOW PROCEDURES LIKE &#39;GENERATE_SYNTHETIC_DATA&#39; IN SCHEMA SNOWFLAKE.DATA_PRIVACY;<br><br>-- Create fallback SP<br>-- paste contents of sp_fake_synthetic_data.sql<br>-- Create wrapper SP<br>-- paste contents of generate_and_validate_synthetic_data_audit.sql</pre><ol><li><strong>Grants</strong> (example):</li></ol><pre>GRANT USAGE ON DATABASE SNOWFLAKE TO ROLE &lt;ROLE&gt;;<br>GRANT USAGE ON SCHEMA SNOWFLAKE.DATA_PRIVACY TO ROLE &lt;ROLE&gt;;<br><br>GRANT USAGE ON DATABASE &lt;DB&gt; TO ROLE &lt;ROLE&gt;;<br>GRANT USAGE ON SCHEMA &lt;DB&gt;.&lt;SCHEMA&gt; TO ROLE &lt;ROLE&gt;;<br>GRANT CREATE TABLE, INSERT, SELECT ON SCHEMA &lt;DB&gt;.&lt;SCHEMA&gt; TO ROLE &lt;ROLE&gt;;</pre><ol><li><strong>Streamlit</strong>: create an app in Snowflake UI and paste app.py into it (uses get_active_session(), so no connector secrets required).</li></ol><h3>Usage</h3><ul><li>Launch the Streamlit app.</li><li>Enter <strong>Database</strong> and <strong>Schema</strong>.</li><li>Pick <strong>Generate via Wrapper</strong> (recommended) or <strong>Empirical</strong>.</li><li>Select objects (multi-select for wrapper; single for empirical).</li><li>Click <strong>Run</strong>.</li><li>Review the <strong>status</strong>, <strong>errors (if any)</strong>, and <strong>10-row previews</strong> of _SYN outputs.</li></ul><h3>Troubleshooting (common)</h3><ul><li><strong>“Unknown function GENERATE_SYNTHETIC_DATA”</strong><br> You must call SNOWFLAKE.DATA_PRIVACY.GENERATE_SYNTHETIC_DATA and have USAGE on SNOWFLAKE DB and SNOWFLAKE.DATA_PRIVACY schema. If the procedure isn’t listed by SHOW PROCEDURES, it’s not enabled in your account. The wrapper will fall back automatically.</li><li><strong>“Number of input tables exceeds the allowed limit”</strong><br> Native SP allows <strong>max 5</strong> inputs per call. The wrapper batches automatically; if you call it directly, also batch on the client.</li><li><strong>“Expression type does not match column data type … expecting TIMESTAMP_NTZ … but got NUMBER …”</strong><br> Use the <strong>type-coercion</strong> logic in SP_FAKE_SYNTHETIC_DATA (included above).</li><li><strong>Views not generating</strong><br> The wrapper pre-validates and the fallback SP creates CREATE TABLE … AS SELECT * FROM &lt;view&gt; WHERE 1=0, so views are supported. Ensure ≥20 distinct rows and supported column types.</li><li><strong>FK validation errors</strong><br> If you lack access to INFORMATION_SCHEMA.KEY_COLUMN_USAGE in your DB, the wrapper logs this under fk_validation and still returns success for generation.</li></ul><h3>Why native + fallback?</h3><p><strong>Native</strong> generation provides <strong>privacy-aware synthesis</strong> with built-in controls and is optimized by Snowflake; but it may be unavailable or fail on specific objects due to requirements.</p><p>The <strong>empirical fallback</strong> is deterministic and <strong>data-shape preserving</strong>:</p><ul><li>Samples real categories/labels</li><li>Preserves numeric ranges and null rates</li><li>Coerces types to the target DDL</li><li>Works on both tables and views</li></ul><p>Together they give you <strong>reliability + realism</strong>.</p><h3>License &amp; contributions</h3><ul><li>Use MIT for the repo.</li><li>PRs welcome: add column-level rules (e.g., force STRICT sampling on *_ID), or pluggable redaction policies.</li></ul><h3>Let’s Connect</h3><p>If this perspective on AI-native enterprises resonated with you:</p><ul><li>Follow me on <a href="https://www.linkedin.com/in/puspanjalisarma/"><strong>Linkedin</strong></a> , <a href="https://github.com/puspanjalis"><strong>Github</strong></a><strong> </strong>and <a href="https://medium.com/@puspanjalisarma23"><strong>Medium</strong></a><strong> </strong>for practical insights on AI engineering, agentic systems, and enterprise-scale AI</li><li>Subscribe to my newsletter <em>“</em><a href="https://www.linkedin.com/build-relation/newsletter-follow?entityUrn=7412051308393787392"><strong><em>Scaling AI Without Hype</em></strong></a><em>”</em> for real-world frameworks and no-nonsense playbooks</li><li>Share your perspective — <strong><em>how is your organization evolving toward AI-native?</em></strong></li><li>Reach out if you’re building or scaling AI platforms — open to meaningful discussions and collaboration</li></ul><h3>Go Deeper</h3><p>For a structured, leadership-level approach to building AI-native organizations:</p><ul><li>Refer to my book <strong><em>“</em></strong><a href="https://www.amazon.in/Strategic-Leadership-Through-Data-transformation/dp/9365895758"><strong><em>Strategic AI Leadership Through Data</em></strong></a><strong><em>”</em></strong> — a practical guide to scaling AI, designing platforms, and aligning data strategy with business impact</li></ul><h3>Join the Conversation</h3><p>I am curious to know…</p><ul><li>Have you faced challenges with synthetic data quality in enterprise environments?</li><li>Has Snowflake’s native synthetic data generation worked reliably for your workloads?</li><li>What approaches have worked best for preserving statistical realism while protecting sensitive data?</li><li>Are you using rule-based masking, synthetic generation, differential privacy, or hybrid approaches?</li></ul><p>Add your thoughts below.👇</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=29ea9061551a" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[AI-Native Enterprises: The Next Operating Model for Digital Organizations]]></title>
            <link>https://medium.com/@puspanjalisarma23/ai-native-enterprises-the-next-operating-model-for-digital-organizations-35292d873c82?source=rss-657cc3b2578c------2</link>
            <guid isPermaLink="false">https://medium.com/p/35292d873c82</guid>
            <category><![CDATA[digital-transformation]]></category>
            <category><![CDATA[enterprise-ai-strategy]]></category>
            <category><![CDATA[business-models]]></category>
            <category><![CDATA[gen-ai-revolution]]></category>
            <category><![CDATA[ai-native]]></category>
            <dc:creator><![CDATA[Puspanjali Sarma]]></dc:creator>
            <pubDate>Sun, 29 Mar 2026 11:49:18 GMT</pubDate>
            <atom:updated>2026-03-29T11:49:18.759Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="AI-Native Enterprises: The Next Operating Model for Digital Organizations" src="https://cdn-images-1.medium.com/max/1024/1*NyYTKN8LDYj7rQEM60EtTA.png" /><figcaption>AI-Native Enterprises: The Next Operating Model for Digital Organizations</figcaption></figure><p>Artificial Intelligence is no longer a layer added to existing systems — it is becoming the foundation upon which modern enterprises are built. The concept of <strong>AI-native</strong> organizations represents a fundamental shift from traditional digital transformation to a new paradigm where AI is embedded into the core operating model, decision-making processes, and product experiences.</p><p><strong><em>This shift is not incremental — it is architectural, cultural, and strategic.</em></strong></p><h3>What Does “AI-Native” Really Mean?</h3><p>An AI-native organization is one where:</p><ul><li>AI is embedded into <strong>every core workflow</strong>, not just specific use cases</li><li>Systems are designed to be <strong>learning systems</strong>, continuously improving with data</li><li>Decision-making is <strong>augmented or automated</strong> using models, not intuition alone</li><li>Products are <strong>intelligent by design</strong>, not retrofitted with AI features</li></ul><p>Unlike AI-enabled organizations (which add AI capabilities), AI-native companies are <strong>built around AI as a first principle</strong>.</p><h3>The Industry Shift: From Digital-First to AI-First</h3><p>Over the last decade, enterprises focused on becoming <em>digital-first</em>. Today, that is table stakes.</p><p>The current shift is toward:</p><ul><li><strong>AI-first architectures</strong> replacing rule-based systems</li><li><strong>Agentic workflows</strong> replacing static automation</li><li><strong>Real-time decision systems</strong> replacing batch analytics</li></ul><p>According to multiple industry analyses (McKinsey, Gartner, and IDC):</p><ul><li>Over <strong>70% of organizations</strong> are experimenting with Generative AI</li><li>Nearly <strong>30% are moving toward production-scale deployments</strong></li><li>AI-driven automation is expected to contribute <strong>trillions of dollars in economic value annually by 2030</strong></li></ul><p>However, only a small percentage are truly AI-native.</p><h3>Core Characteristics of AI-Native Enterprises</h3><h3>1. Data as a Product</h3><p>AI-native companies treat data as a <strong>first-class product</strong>, not a byproduct of systems.</p><ul><li>Strong data governance and lineage</li><li>Real-time data pipelines</li><li>Feature stores and reusable data assets</li></ul><h3>2. Model-Centric Architecture</h3><p>Applications are built around models rather than hard-coded logic.</p><ul><li>Continuous training and evaluation loops</li><li>Model versioning and observability</li><li>Experimentation-first mindset</li></ul><h3>3. Agentic Systems and Autonomous Workflows</h3><p>The rise of AI agents is transforming enterprise workflows.</p><ul><li>Multi-step reasoning and task execution</li><li>Tool integration (APIs, databases, enterprise systems)</li><li>Human-in-the-loop for governance</li></ul><h3>4. AI Governance by Design</h3><p>AI-native systems embed governance at runtime, not as an afterthought.</p><ul><li>Policy enforcement</li><li>Auditability and traceability</li><li>Bias and fairness monitoring</li></ul><h3>5. Platform Thinking</h3><p>Successful organizations build <strong>AI platforms</strong>, not isolated solutions.</p><ul><li>Reusable components (LLM gateways, vector stores, evaluation frameworks)</li><li>Standardized tooling for teams</li><li>Internal developer ecosystems</li></ul><h3>Technology Stack of AI-Native Enterprises</h3><p>A typical AI-native stack includes:</p><ul><li><strong>Data Layer</strong>: Real-time ingestion, streaming, lakehouse architectures</li><li><strong>Model Layer</strong>: Foundation models, fine-tuned models, embeddings</li><li><strong>Orchestration Layer</strong>: Agent frameworks, workflow engines</li><li><strong>Evaluation Layer</strong>: Metrics, guardrails, benchmarking tools</li><li><strong>Observability Layer</strong>: Tracing, logging, performance monitoring</li><li><strong>Governance Layer</strong>: Policy engines, access control, compliance frameworks</li></ul><p>The key difference is not the tools themselves — but how tightly integrated and continuously evolving the system is.</p><h3>Real-World Trends Shaping AI-Native Adoption</h3><h3>1. Rise of Generative AI in Enterprise Workflows</h3><p>Generative AI is moving beyond copilots into <strong>autonomous execution systems</strong>.</p><ul><li>Code generation and validation</li><li>Automated testing and QA</li><li>Customer support automation with reasoning</li></ul><h3>2. Shift Toward Agentic AI</h3><p>Organizations are moving from single-model applications to <strong>multi-agent systems</strong>.</p><ul><li>Planning + execution loops</li><li>Tool calling and orchestration</li><li>Task decomposition across agents</li></ul><h3>3. Evaluation Becomes a First-Class Discipline</h3><p>One of the biggest gaps in AI adoption is <strong>evaluation and reliability</strong>.</p><p>Enterprises are investing in:</p><ul><li>Continuous evaluation pipelines</li><li>Scenario-based testing</li><li>Risk-tiered guardrails</li></ul><h3>4. AI + Observability Convergence</h3><p>Observability is extending beyond infrastructure into AI systems.</p><ul><li>Prompt tracing</li><li>Model performance monitoring</li><li>End-to-end workflow visibility</li></ul><h3>5. Regulatory and Responsible AI Pressure</h3><p>Global regulations are pushing enterprises toward <strong>responsible AI practices</strong>.</p><ul><li>Transparency requirements</li><li>Data privacy constraints</li><li>Explainability standards</li></ul><h3>Challenges in Becoming AI-Native</h3><p>Despite the momentum, most organizations struggle due to:</p><h3>1. Fragmented Architectures</h3><p>Disconnected tools and pipelines prevent scalability.</p><h3>2. Lack of Evaluation Frameworks</h3><p>Many teams deploy AI without robust validation mechanisms.</p><h3>3. Talent and Skill Gaps</h3><p>AI-native requires cross-functional expertise:</p><ul><li>Data engineering</li><li>ML engineering</li><li>Product thinking</li><li>AI governance</li></ul><h3>4. Cultural Resistance</h3><p>Transitioning from deterministic systems to probabilistic systems requires a mindset shift.</p><h3>A Practical Path to Becoming AI-Native</h3><p>Organizations can adopt a phased approach:</p><h3>Phase 1: AI Enablement</h3><ul><li>Identify high-impact use cases</li><li>Introduce foundational models</li><li>Build initial pipelines</li></ul><h3>Phase 2: AI Integration</h3><ul><li>Embed AI into core workflow</li><li>Standardize tooling and platforms</li><li>Introduce evaluation and monitoring</li></ul><h3>Phase 3: AI-Native Transformation</h3><ul><li>Redesign systems around AI</li><li>Deploy agentic workflows</li><li>Implement governance at scale</li></ul><h3>Strategic Implications for Leaders</h3><p>For leaders, becoming AI-native is not a technology decision — it is a <strong>business transformation strategy</strong>.</p><p>Key focus areas:</p><ul><li>Align AI initiatives with business outcomes</li><li>Invest in platform capabilities over point solutions</li><li>Build governance frameworks early</li><li>Enable teams with the right tools and autonomy</li></ul><h3>Closing Perspective</h3><p>AI-native enterprises will define the next generation of market leaders. The competitive advantage will not come from access to models alone, but from <strong>how effectively organizations design, operate, and govern AI systems at scale</strong>.</p><p>This is where architecture, evaluation, and leadership converge.</p><p>For readers looking to go deeper into building and scaling such systems, you can refer to my book <strong><em>“</em></strong><a href="https://www.amazon.in/Strategic-Leadership-Through-Data-transformation/dp/9365895758"><strong><em>Strategic AI Leadership Through Data</em></strong></a><strong><em>”</em></strong>, which provides a practical framework for designing AI-driven organizations, scaling AI platforms, and aligning data strategy with business impact.</p><h3>Final Thought</h3><p>The question is no longer whether organizations should adopt AI.</p><p>The real question is:</p><p><strong><em>Are you building AI capabilities — or becoming an AI-native enterprise?</em></strong></p><h3>Let’s Connect</h3><p>If this perspective on AI-native enterprises resonated with you:</p><ul><li>Follow me on <a href="https://www.linkedin.com/in/puspanjalisarma/"><strong>Linkedin</strong></a> , <a href="https://github.com/puspanjalis"><strong>Github</strong></a><strong> </strong>and <a href="https://medium.com/@puspanjalisarma23"><strong>Medium</strong></a><strong> </strong>for practical insights on AI engineering, agentic systems, and enterprise-scale AI</li><li>Subscribe to my newsletter <em>“</em><a href="https://www.linkedin.com/build-relation/newsletter-follow?entityUrn=7412051308393787392"><strong><em>Scaling AI Without Hype</em></strong></a><em>”</em> for real-world frameworks and no-nonsense playbooks</li><li>Share your perspective — <strong><em>how is your organization evolving toward AI-native?</em></strong></li><li>Reach out if you’re building or scaling AI platforms — open to meaningful discussions and collaboration</li></ul><h3>Go Deeper</h3><p>For a structured, leadership-level approach to building AI-native organizations:</p><ul><li>Refer to my book <strong><em>“</em></strong><a href="https://www.amazon.in/Strategic-Leadership-Through-Data-transformation/dp/9365895758"><strong><em>Strategic AI Leadership Through Data</em></strong></a><strong><em>”</em></strong> — a practical guide to scaling AI, designing platforms, and aligning data strategy with business impact</li></ul><h3>Join the Conversation</h3><ul><li><em>What’s the biggest barrier in your AI journey today?</em></li><li><em>Are you experimenting with AI — or redesigning your organization around it?</em></li></ul><p>Add your thoughts below.👇</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=35292d873c82" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Discover AI Agents with Google ADK in 20 Minutes — Build a GitHub Trend Recommender]]></title>
            <link>https://medium.com/@puspanjalisarma23/discover-ai-agents-with-google-adk-in-20-minutes-build-a-github-trend-recommender-abbd33abe93b?source=rss-657cc3b2578c------2</link>
            <guid isPermaLink="false">https://medium.com/p/abbd33abe93b</guid>
            <category><![CDATA[generative-ai-use-cases]]></category>
            <category><![CDATA[recommender]]></category>
            <category><![CDATA[ai-agents-in-action]]></category>
            <category><![CDATA[google-agent-development]]></category>
            <category><![CDATA[ai-agent]]></category>
            <dc:creator><![CDATA[Puspanjali Sarma]]></dc:creator>
            <pubDate>Fri, 04 Jul 2025 11:38:27 GMT</pubDate>
            <atom:updated>2025-07-04T11:38:27.455Z</atom:updated>
            <content:encoded><![CDATA[<h3>Discover AI Agents with Google ADK in 20 Minutes — Build a GitHub Trend Recommender</h3><p>Imagine having an <strong>AI assistant</strong> that doesn’t just fetch information, but actively curates trending <a href="https://github.com/"><strong>GitHub</strong></a><strong> projects</strong> based on your unique interests — like a knowledgeable friend who anticipates what excites you in tech.</p><p>This is now possible with <a href="https://developers.googleblog.com/en/agent-development-kit-easy-to-build-multi-agent-applications/"><strong>Google’s Agent Development Kit (ADK)</strong></a>, an innovative toolkit designed for building <strong>intelligent agents</strong> that can understand <strong>natural language queries</strong>, <strong>plan multi-step tasks</strong>, and leverage memory to provide <strong>thoughtful</strong>, <strong>context-aware responses</strong>.</p><p>The power of<a href="https://developers.googleblog.com/en/agent-development-kit-easy-to-build-multi-agent-applications/"> Google ADK</a> lies in its modular design. You can integrate <strong>Large Language Models (LLMs)</strong> with planning components, memory storage (like vector databases), and connectors to external APIs — in this case, <a href="https://github.com/">GitHub</a>’s trending repositories API.</p><p><strong>Here’s how such an agent operates:</strong></p><ul><li><strong>Intelligent Query Understanding:</strong> When you ask for trending AI projects in a specific area, the agent breaks down your request, recognizing not just keywords but the real-world context and your likely interests.</li><li><strong>Real-Time Data Gathering:</strong> Leveraging APIs or web scraping, the agent pulls the latest repository data from GitHub, capturing project descriptions, activity metrics, and contributor trends.</li><li><strong>Natural Language Embeddings: </strong>Using Gemini’s embeddings, the agent encodes each repository’s description into a high-dimensional vector — a mathematical fingerprint of its content and focus. This makes semantic matching far more powerful than simple keyword search.</li><li><strong>Memory and Personalization: </strong>ADK enables your agent to remember your previous queries, starred projects, and feedback, learning over time to fine-tune its recommendations to your evolving tastes.</li><li><strong>Adaptive Planning:</strong> The agent can break down complex requests into a series of logical steps — filtering, ranking, and even justifying why certain projects are most relevant to you. This multi-step planning ensures that recommendations are not only accurate but tailored to the nuances of your request.</li></ul><p>What truly sets this approach apart is the seamless blend of sophisticated AI with user-centric design. Rather than being a reactive search tool, your agent becomes a proactive technology scout, offering curated suggestions, surfacing up-and-coming innovators, and even predicting which trends might catch your attention next.</p><p>With the combined strengths of <a href="https://gemini.google.com/">Gemini</a> and <a href="https://developers.googleblog.com/en/agent-development-kit-easy-to-build-multi-agent-applications/">ADK</a>, developers can build <strong>AI agents </strong>that aren’t just smart — they’re attentive, adaptive, and genuinely helpful in navigating the fast-paced world of open-source software.</p><p>This means your agent can interpret user instructions (such as <em>“</em><strong><em>Show me the top open-source projects in computer vision”</em></strong>), search for relevant projects, analyze repository metadata (stars, forks, recent activity, topics), and return tailored recommendations.</p><p>But the capabilities don’t end there. With memory modules, the agent remembers your previous queries and preferences, refining its suggestions over time. Planning components enable it to break down complex requests (<em>“Find projects on neural networks published in the last month with more than 500 stars”</em>) into actionable steps: querying, filtering, ranking, and summarizing results.</p><p>Furthermore, this approach isn’t limited to <a href="https://github.com/">GitHub</a>. The ADK’s flexible architecture allows you to swap in new data sources or connect to other platforms, making it suitable for news aggregation, research, job searches, or even personalized content feeds.</p><p>In this tutorial, we’ll build a <a href="https://github.com/puspanjalis/puspanjali-sarma-github-gemini-agent"><strong>GitHub AI Agent</strong></a> in less than 20 minutes. Think of it as your tech-savvy buddy that:</p><ul><li>Understands your project interest (like “AI fairness” or “robotics”)</li><li>Searches GitHub for trending projects</li><li>Embeds and stores metadata</li><li>Recommends repos using LLM-powered reasoning</li></ul><h4><strong><em>Let’s dive in!</em></strong></h4><h3>Use Case: GitHub Recommender Agent</h3><p>Imagine delving into a new area of artificial intelligence — perhaps reinforcement learning or AI ethics. Instead of spending hours manually browsing GitHub repositories, you could simply pose a natural-language query like,</p><p><strong><em>“Show me the latest AI research projects on reinforcement learning,”</em></strong></p><p>and instantly receive curated, context-aware recommendations.</p><p>That’s what we’re building today: <strong><em>an LLM-powered AI agent that recommends trending GitHub repos using planning, memory, and vector search — all powered by Google ADK.</em></strong></p><p>The agent interprets your intent, considers your specific interests (maybe you prefer repositories with active communities or cutting-edge research topics), and leverages its memory to recall your previous preferences. This means every suggestion grows sharper and more aligned with your evolving needs over time.</p><p>This agent uses a sophisticated synergy of planning modules and vector database search. When you make a request, it autonomously breaks your question down into actionable tasks:</p><p>· First, it issues GitHub API calls to gather a broad set of trending or recently updated repositories in your target domain.</p><p>· Next, it analyzes metadata such as star count, recent commit activity, contributor engagement, and even repository topics or README content, embedding this data to measure relevance.</p><p>· It filters and ranks results, not just by popularity, but by nuanced criteria — such as recency, documentation quality, or alignment with your previously expressed interests.</p><p>· Finally, it summarizes its findings, providing not just a list, but context: “These three repositories have had the highest growth in contributors in the last month and focus specifically on multi-agent reinforcement learning.”</p><p>By using an adaptive learning loop, the agent also remembers your feedback — so if you often select repositories with comprehensive tutorials, it learns to favor those in future recommendations.</p><p>Beyond convenience, this approach fundamentally enhances discovery. You’re empowered to uncover promising, lesser-known projects that might otherwise be buried under a sea of popular but less relevant repositories. For researchers, students, and developers alike, this means staying at the forefront of innovation without being overwhelmed by information overload.</p><p>In short, building this AI-powered GitHub recommender agent transforms your search from a tedious manual process into a dynamic, interactive experience. It adapts to your goals, curates results with intelligence, and ensures you never miss out on the latest breakthroughs in your fields of interest.</p><h3>What You’ll Need</h3><p>Tool Purpose</p><p><strong>Google Gemini API</strong> LLM + Embeddings</p><p><strong>Google ADK</strong> Agent planning, memory, tools</p><p><strong>GitHub REST API</strong> Fetch trending projects</p><p><strong>Python</strong> Glue code</p><p><strong>Local Vector Search</strong> No 3rd-party DBs or APIs</p><h3>Step 1: Project Scaffold</h3><p>Here’s the layout:</p><pre>github-ai-agent/<br>├── agent.py              # Main ADK agent<br>├── github_fetcher.py     # Trending repo scraper<br>├── retriever.py          # Embedding + vector search<br>├── config.py             # API key loader<br>├── .env                  # Local secrets<br>├── requirements.txt      # Python deps<br>└── README.md</pre><p>Install dependencies:</p><pre>pip install -r requirements.txt</pre><p>requirements.txt</p><pre>google-adk<br>openai<br>chromadb<br>requests<br>python-dotenv</pre><h3>Step 2: Configure Secrets</h3><p>Create a .env file like:</p><pre>GEMINI_API_KEY=your_google_api_key</pre><p>Then load in config.py:</p><pre>from dotenv import load_dotenv<br>import os<br><br>load_dotenv()<br>GEMINI_API_KEY = os.getenv(&quot;GEMINI_API_KEY&quot;)</pre><h3>Step 3: Scrape GitHub Trending</h3><pre># github_fetcher.py<br><br>import requests<br><br>def get_trending_repos():<br>    res = requests.get(&quot;https://ghapi.huchen.dev/repositories&quot;)<br>    res.raise_for_status()<br>    return [{&quot;name&quot;: r[&quot;name&quot;], &quot;desc&quot;: r.get(&quot;description&quot;, &quot;&quot;), &quot;url&quot;: r[&quot;url&quot;]} for r in res.json()]</pre><h3>Step 4: Gemini Embeddings + Local Vector Search</h3><pre># vector_store.py<br><br>import google.generativeai as genai<br>import numpy as np<br>from config import GEMINI_API_KEY<br><br>genai.configure(api_key=GEMINI_API_KEY)<br><br>class LocalVectorDB:<br>    def __init__(self):<br>        self.items = []<br><br>    def embed(self, text):<br>        model = genai.get_model(&#39;embedding-001&#39;)<br>        return np.array(model.embed_content(content=text)[&quot;embedding&quot;])<br><br>    def add(self, name, desc, url):<br>        vector = self.embed(desc)<br>        self.items.append((name, desc, url, vector))<br><br>    def search(self, query, top_k=3):<br>        q_vec = self.embed(query)<br>        ranked = sorted(self.items, key=lambda x: -np.dot(x[3], q_vec))<br>        return [(name, url) for name, _, url, _ in ranked[:top_k]]</pre><h3>Step 5: Gemini + ADK Agent</h3><pre># agent.py<br><br>from adk.agent import Agent<br>from adk.planner import GeminiPlanner<br>from adk.memory import SimpleMemory<br>from adk.tools import Tool, Toolset<br><br>from github_fetcher import get_trending_repos<br>from vector_store import LocalVectorDB<br>from config import GEMINI_API_KEY<br><br># Embed trending repos<br>db = LocalVectorDB()<br>repos = get_trending_repos()<br>for repo in repos:<br>    db.add(repo[&quot;name&quot;], repo[&quot;desc&quot;], repo[&quot;url&quot;])<br><br># Define a custom tool<br>class GitHubRecommendTool(Tool):<br>    name = &quot;github_recommender&quot;<br>    description = &quot;Recommend AI projects from GitHub&quot;<br><br>    def invoke(self, query: str) -&gt; str:<br>        results = db.search(query)<br>        return &quot;\n&quot;.join([f&quot;{name}: {url}&quot; for name, url in results])<br><br># Assemble agent<br>planner = GeminiPlanner(api_key=GEMINI_API_KEY)<br>memory = SimpleMemory()<br>toolset = Toolset([GitHubRecommendTool()])<br><br>agent = Agent(name=&quot;github-expert&quot;, planner=planner, memory=memory, toolset=toolset)<br><br>def main():<br>    q = input(&quot;What kind of AI project are you looking for? &quot;)<br>    print(agent.run(q))<br><br>if __name__ == &quot;__main__&quot;:<br>    main()</pre><h3>Sample Run</h3><ul><li><strong>Run the app:</strong></li></ul><pre>python agent.py</pre><ul><li><strong>Prompt:</strong></li></ul><p><strong><em>“I want trending computer vision projects.”</em></strong></p><ul><li><strong>Agent Response:</strong></li></ul><pre>Ultralytics YOLOv8: https://github.com/ultralytics/ultralytics  <br>Detectron2: https://github.com/facebookresearch/detectron2  <br>MMDetection: https://github.com/open-mmlab/mmdetectio</pre><p><strong>Prompt:</strong></p><blockquote><em>“What are the best computer vision projects on GitHub right now?”</em></blockquote><p><strong>Response:</strong></p><pre>1. https://github.com/facebookresearch/detectron2  <br>2. https://github.com/open-mmlab/mmdetection  <br>3. https://github.com/ultralytics/ultralytics</pre><h3>Deploy on GitHub</h3><ol><li>Create a new repo (e.g., github-ai-agent)</li><li>Push the full code base</li><li>Add a README with usage and installation</li><li>Include a sample .env.example</li></ol><p>Link to GitHub repo template <a href="https://github.com/puspanjalis/puspanjali-sarma-github-gemini-agent">here</a></p><h3>What’s Happening Behind the Scenes?</h3><p>Component Role</p><ul><li><strong>Planner : </strong>Uses LLM to interpret your query</li><li><strong>Memory: </strong>Stores past interaction context</li><li><strong>Toolset: </strong>Runs your GitHub search &amp; retrieval</li><li><strong>Embedding: </strong>Transforms text → vector for similarity search</li><li><strong>ChromaDB: </strong>Lightweight vector storage for metadata</li></ul><h3>What’s Next?</h3><ul><li>Add LangChain or MongoDB Atlas for memory</li><li>Deploy as a web UI with Streamlit or FastAPI</li><li>Build a team of agents for niche queries (LLM repos, NLP, RL, etc.)</li></ul><h3>Want to Customize This?</h3><ul><li>Filter repos by language or topic in github_fetcher.py</li><li>Swap vector scoring method in vector_store.py</li><li>Add user profiles with SQLite or Firebase</li><li>Add a feedback loop: learn what repos users click on</li></ul><h3>What Makes This Special?</h3><p>Unlike typical search bots, this agent is:</p><ul><li><strong>Truly intelligent</strong>: Uses embeddings, not keywords</li><li><strong>Modular</strong>: Easy to extend with more tools or memories</li><li><strong>100% Google-native</strong>: No OpenAI, Pinecone, or MongoDB</li></ul><blockquote>This demo reflects my growing work in agent-based design and LLM tooling in enterprise AI environments.</blockquote><h3>Final Thoughts</h3><p>Using Google ADK and Gemini, I built this agent from scratch to demonstrate:</p><ul><li>Modular agent planning</li><li>Gemini-based vector reasoning</li><li>Real-world data integration (GitHub)</li></ul><p>This work shows the practical potential of intelligent AI agents beyond chatbots — into the world of true automation, insight, and technical enablement.</p><blockquote>Stay curious. Keep building!</blockquote><h3>Let’s Connect</h3><p>If you found this useful, consider:</p><ul><li>Following me here on <a href="https://www.linkedin.com/in/puspanjalisarma/">Linkedin</a> , <a href="https://github.com/puspanjalis">Github</a> and <a href="https://medium.com/@puspanjalisarma23">Medium</a>!</li><li>Starring the <a href="https://github.com/puspanjalis/LLM-SQL-Accuracy-Evaluator">GitHub repo</a></li><li>Sharing this with your data and ML teams</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=abbd33abe93b" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Building an AI-First Mindset Across the Enterprise]]></title>
            <link>https://medium.com/@puspanjalisarma23/building-an-ai-first-mindset-across-the-enterprise-635f6cc76382?source=rss-657cc3b2578c------2</link>
            <guid isPermaLink="false">https://medium.com/p/635f6cc76382</guid>
            <category><![CDATA[ai-innovation]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[generative-ai-tools]]></category>
            <category><![CDATA[leadership]]></category>
            <category><![CDATA[ai]]></category>
            <dc:creator><![CDATA[Puspanjali Sarma]]></dc:creator>
            <pubDate>Mon, 23 Jun 2025 17:59:36 GMT</pubDate>
            <atom:updated>2025-09-20T08:26:54.080Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*NTeh9-Thi4FUCnS-" /><figcaption>Photo by <a href="https://unsplash.com/@steve_j?utm_source=medium&amp;utm_medium=referral">Steve Johnson</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>We’re at an inflection point. AI is no longer a niche tool relegated to innovation labs — it is becoming the operating system of modern enterprises. But for AI to truly transform a business, it must go beyond proof-of-concepts, past dashboards and data pipelines. It must be embraced as a mindset.</p><p>A true <strong><em>AI-first mindset</em></strong> is not about deploying more models. It is about creating an enterprise where AI is considered a core strategic lever, integrated into every product, process, and decision.</p><p>Organizations that succeed here actively cultivate curiosity and experimentation at every level. Leaders encourage teams to question established routines, seeking opportunities for AI to unlock new forms of value. Instead of waiting for perfect data or fully mature technologies, they foster a culture where iterative prototyping and continuous learning are the norm.</p><p>Moreover, an AI-first mindset requires cross-functional collaboration. IT, business, and operational leaders must break down silos and work side-by-side, ensuring that AI initiatives are both technically robust and deeply aligned to real-world business outcomes. This means rethinking talent: training employees not just to use AI, but to creatively envision new possibilities it can enable.</p><p>Finally, ethical considerations and responsible AI must be embedded from the outset. Enterprises committed to an AI-first approach recognize that trust, transparency, and fairness are fundamental for sustainable transformation. In this context, governance frameworks and risk assessments are not afterthoughts but critical enablers of innovation.</p><p>Having led AI innovation and scaled data-driven transformation across global organizations, I’ve come to believe that mindset — not models — is the real frontier of enterprise AI success.</p><h3>What Does “AI-First” Really Mean?</h3><p>An AI-first approach demands a fundamental shift in perspective: it’s not about layering AI onto legacy systems, but about architecting systems, experiences, and even business models around AI’s unique strengths. This requires organizations to embrace agility and resilience, as AI’s rapid evolution will continually reshape what’s possible.</p><p>Being “AI-first” means you don’t just use AI to optimize current processes — you <strong>reimagine</strong> those processes with AI at the center.</p><p>Instead of:<br><em> “How can we automate this report?”<br> Ask: “What decisions should this report be making for us?”</em></p><p>Instead of:<br><em> “How do we add AI to our product?”<br> Ask: “What would this product look like if it was powered by intelligence from day one?”</em></p><p>In practice, this means cultivating a workforce adept at both critical thinking and technical fluency — people who can identify where human judgment is essential and where machine intelligence can drive exponential value. Collaboration between disciplines becomes paramount; AI-first enterprises blur the boundaries between technology, business, and operations, fostering cross-pollination of ideas.</p><p>Furthermore, ethical considerations grow in complexity as AI systems scale. Becoming AI-first means weaving responsible use, bias mitigation, and transparency into the fabric of every project from inception. Mature AI organizations establish feedback loops — not just for model performance, but for monitoring societal and business impact over time.</p><p>Ultimately, <strong><em>“AI-first”</em></strong> is a declaration of intent. It’s about envisioning value creation beyond efficiency, unlocking new capabilities, and empowering people to do what they do best — supported, but not supplanted, by intelligent systems. The journey requires courage, curiosity, and a relentless focus on outcomes that matter.</p><h3>Why Enterprises Need an AI-First Mindset Now</h3><p>As the pace of technological advancement accelerates, organizations that thrive are those willing to reimagine their foundations, not merely upgrade their tools. An AI-first mindset is more than a strategic pivot — it’s a cultural transformation that shapes how businesses sense, decide, and act in an era defined by data and intelligence.</p><blockquote><em>“In the past, companies competed on scale. In the future, they will compete on intelligence.” — Fei-Fei Li</em></blockquote><h4>The Strategic Imperative of AI-First Thinking</h4><p>Adopting an AI-first approach compels leaders to ask: how can we create value in fundamentally new ways, rather than simply optimizing what already exists? This means evaluating every business process, customer interaction, and operational workflow for its potential to be redefined by intelligence — automated where possible, augmented where necessary, and reinvented where opportunity beckons.</p><p>Insight: Early adopters have found that AI-first initiatives foster greater agility, allowing enterprises to pivot in response to emerging trends and competitors more rapidly. What distinguishes the AI-first leaders is their willingness to invest in robust data infrastructure and empower multidisciplinary teams to experiment fearlessly.</p><h4>Cultivating Organizational Readiness</h4><p>Transitioning to an AI-first model requires more than technical upgrades. It’s about fostering a culture where curiosity, continuous learning, and collaboration are central. Training programs that blend AI literacy with domain expertise become essential. Importantly, organizations must also put systems in place for responsible AI governance, ensuring that ethical considerations are not addressed as afterthoughts, but baked into every step of the innovation lifecycle.</p><h4>Case Study: The AI-First Transformation of Ping An Insurance</h4><p><a href="https://group.pingan.com/media/perspectives/2025/driving-innovation-with-generative-ai.html">Ping An</a>, one of China’s largest insurers, offers a compelling example of AI-first transformation. Recognizing the disruptive power of AI, Ping An invested heavily in machine learning, natural language processing, and computer vision. Today, over 90% of its customer service is powered by AI chatbots, reducing response times from hours to seconds and significantly improving customer satisfaction.</p><p>But Ping An’s AI-first journey did not stop with automation. The company used AI to create entirely new business lines, such as smart healthcare diagnostics and AI-driven asset management platforms. By embedding AI into every facet of its operations, Ping An achieved a reduction in claims processing time by 78%, a 30% increase in productivity, and a measurable impact on profitability and market competitiveness.</p><p><strong><em>Insight:</em></strong><em> Ping An’s success underscores the importance of executive sponsorship, relentless experimentation, and a long-term commitment to reskilling employees alongside technological investment.</em></p><h4>Charting the Path Forward</h4><p>Enterprises must recognize that AI-first is not a destination but a journey — one characterized by constant evolution. The most successful organizations are those that recognize the inseparability of technology and talent, and that use AI not just to do things better, but to do better things.</p><p>By reimagining value creation, building AI-fluent teams, and learning from pioneers like Ping An, organizations can unlock the transformative potential of AI and position themselves for enduring advantage in the intelligence-driven economy.</p><h4>Reality Check:</h4><ul><li><strong>Only 21%</strong> of enterprises have embedded AI into multiple business units (<a href="https://www.accenture.com/content/dam/accenture/final/a-com-migration/thought-leadership-assets/accenture-built-to-scale-pdf-report.pdf">Accenture, 2023</a>).</li><li><strong>84%</strong> of C-suite leaders agree that AI is essential to competitiveness, yet <strong>76%</strong> struggle to move beyond experimentation (<a href="https://www.pwc.com/gx/en/issues/c-suite-insights/ceo-survey.html">PwC AI Survey</a>).</li><li>Companies that have adopted AI successfully across units report <strong>up to 50% faster decision-making</strong>, <strong>20–40% productivity gains</strong>, and <strong>revenue increases</strong> of 6–10% (<a href="https://www.mckinsey.com/capabilities/quantumblack/our-insights/the-state-of-ai">McKinsey, 2024)</a>.</li></ul><h3>The Six Dimensions of an AI-First Mindset</h3><h4>1. Leadership Vision: Sponsor Transformation, Not Just Tools</h4><p>A successful AI-first transformation is ignited and sustained by visionary leadership. It is not enough for executives to simply endorse AI initiatives in theory; they must actively champion and embody the change they wish to see. This means articulating a compelling narrative about AI’s potential, aligning the organization’s strategy around that vision, and setting clear expectations that AI is integral to future competitiveness.</p><p>An AI-first transformation must be led from the top. This means:</p><ul><li>CEOs publicly championing AI’s strategic role.</li><li>CHROs investing in AI literacy.</li><li>CFOs framing AI as a capital investment — not just an expense.</li></ul><blockquote>“Every leader is now a technology leader.” — Julie Sweet, CEO, Accenture</blockquote><p>Leaders pave the way by fostering accountability, breaking down silos, and ensuring that AI adoption is not viewed as an isolated IT project but as a core business driver. Their support signals to the entire organization that AI is a strategic imperative, not a fleeting trend. This top-down commitment encourages others to participate, experiment, and innovate with confidence.</p><p>Moreover, effective leadership in this context involves empowering others throughout the organization. It’s about enabling teams to take calculated risks, providing resources for education and experimentation, and celebrating early wins to build momentum. Leaders who visibly invest in AI literacy, prioritize cross-functional collaboration, and demonstrate adaptability in the face of change set the standard for the entire enterprise.</p><p>Organizations that have leaders at every level advocating for AI — not just the C-suite, but also department heads and project managers — are far more likely to overcome resistance and scale their AI ambitions. By positioning themselves as sponsors of transformation, these leaders help create a culture in which AI-driven experimentation and data-informed decision-making become the norm, paving the way for sustainable and meaningful progress.</p><p>At <a href="https://www.dbs.com/india/newsroom_media/how-ai-is-shaping-the-employee-experience-at-dbs.page">DBS Bank</a>, the CEO personally sponsors AI literacy programs, resulting in over 16,000 employees being trained in data-driven thinking, leading to a 60% increase in AI-powered workflows.</p><h4>2. Culture of Digital Curiosity and Trust</h4><p>A culture that fears automation will resist AI. A culture that sees AI as a collaborator will accelerate it.</p><p>Foster a culture where:</p><ul><li>Teams feel empowered to test AI tools.</li><li>Failures are seen as learning, not liability.</li><li>Leaders model transparency and ethical use of AI.</li></ul><p>Building a culture of digital curiosity and trust is foundational for any organization aiming to unlock the true potential of AI. This begins by framing AI not as a threat to jobs or established processes, but as an enabler — empowering employees at all levels to amplify their contributions and focus on higher-value work. Management should clearly communicate that AI is here to augment, not replace, human ingenuity.</p><p>To foster curiosity, organizations can create “sandbox” environments where employees safely experiment with AI tools and solutions without fear of repercussions. Hosting regular hackathons, innovation workshops, and AI learning labs enables staff from all backgrounds — not just technical teams — to engage hands-on with new technologies. These initiatives spark excitement and reduce apprehension, turning the unknown into opportunity.</p><p>Furthermore, a psychologically safe environment is crucial. Leaders should openly share their own experiences with AI — successes and failures alike — demonstrating that experimentation is encouraged and setbacks are seen as valuable learning moments, not liabilities. Establishing forums for cross-functional teams to share AI use cases and lessons learned cultivates organizational learning and encourages a growth mindset.</p><p>Transparency and ethical use of AI must also be modeled from the top down. This includes communicating how AI systems make decisions, openly discussing limitations and biases, and involving employees in conversations about responsible AI deployment. Regular training on topics such as data privacy, algorithmic fairness, and digital ethics builds trust and equips teams to use AI responsibly.</p><p>Companies like <a href="https://blog.adobe.com/en/publish/2021/08/04/adobe-is-a-best-workplace-for-innovators"><strong>Adobe</strong></a> have successfully instilled this culture by inviting employees to submit AI ideas via internal innovation platforms, resulting in dozens of practical, employee-driven tools. At Intuit, employees are incentivized to participate in <strong><em>“AI Days”</em></strong> where they propose and build prototypes, helping demystify AI and promote widespread adoption.</p><p>Ultimately, organizations that prioritize digital curiosity and psychological safety, while weaving trust and transparency into their AI initiatives, are best positioned to harness AI’s transformative power. This environment empowers teams to move beyond skepticism, embrace continuous learning, and champion AI-driven innovation for lasting impact.</p><h4>3. Data and Infrastructure Readiness</h4><p>AI without clean, governed, and accessible data is just hype — a promise that rarely translates into measurable impact. For organizations seeking to genuinely leverage AI, establishing a robust data and infrastructure foundation is not optional; it is the critical enabler for all subsequent innovation.</p><p>To begin, building a modern data platform — often leveraging cloud-native solutions such as <strong>Snowflake</strong>, <strong>Databricks</strong>, or <strong>AWS Lake Formation</strong> — allows organizations to efficiently store, process, and scale vast volumes of diverse data types. However, raw storage is just the start; the true value emerges from giving teams frictionless, self-serve access to high-quality, trusted data.</p><p>Standardizing metadata and data lineage practices ensures that every data asset can be traced, audited, and understood from origin to application. This transparency supports compliance, enables faster troubleshooting, and builds trust in AI outputs. Organizations should implement data cataloging tools and enforce data governance policies across business units to avoid silos and duplication.</p><p>Deploying data observability tools provides real-time monitoring of data pipelines and datasets, flagging anomalies or data drift before they become systemic issues. This proactive approach is essential for maintaining model accuracy and reliability, especially as data volumes and sources scale.</p><p>A central <strong>Feature Store</strong> is another strategic investment. By cataloguing and sharing reusable machine learning features across teams and projects, Feature Stores speed up experimentation, reduce redundancy, and improve consistency between training and production environments.</p><p>Data quality cannot be overstated. According to <strong>McKinsey</strong>, poor data quality and fragmented data sources account for <strong>40% of AI project delays</strong>. Ensuring data is accurate, complete, and up-to-date should be a cross-functional responsibility, supported by continuous data profiling and cleansing routines.</p><p>From an operational perspective, organizations should establish robust <strong>MLOps (Machine Learning Operations)</strong> frameworks. These govern the end-to-end lifecycle of AI models — including versioning, automated testing, deployment, and monitoring — reducing friction between data scientists and IT while ensuring traceability and compliance.</p><p>A practical example: By investing in a cross-domain data marketplace and instituting clear MLOps governance, one organization was able to make critical datasets easily discoverable and reusable across the business, reducing <strong>AI model cycle time by 35%.</strong> This enabled faster time to insight, broader democratization of AI, and greater trust in model outcomes.</p><p>Finally, cultivating a data-driven mindset is as important as the technical stack. Leaders should encourage all teams, not just data scientists, to frame business challenges in terms of data and measurable outcomes. Regular data literacy workshops, open data office hours, and cross-functional data councils can help embed this culture, ensuring that data and infrastructure readiness becomes a shared organizational asset rather than a technical afterthought.</p><h4>4. Democratizing AI Literacy and Ownership</h4><p>To truly unlock AI’s organizational potential, it is imperative to make AI literacy and ownership accessible well beyond traditional technical teams. Democratizing AI is not simply about deploying new tools; it’s about fostering a pervasive culture in which every team member understands, engages with, and takes responsibility for the way AI shapes business outcomes.</p><ul><li><strong>Inclusive Literacy Across Functions: </strong>AI should not be the exclusive domain of specialists. Product owners must learn to articulate business problems as data questions, analysts need fluency in model interpretability and limitations, and functions like HR, finance, and marketing should grasp the ethical frameworks, bias risks, and operational impacts of AI-driven decisions. By integrating AI concepts into onboarding, ongoing learning, and job-specific training, organizations can level the playing field and empower informed participation at every level.</li><li><strong>Cross-Functional Engagement:</strong> Companies such as Intuit, Google, and Unilever have pioneered AI Simulation workshops — interactive sessions in which employees from various departments role-play different scenarios with AI-driven recommendations. These simulations surface ethical dilemmas, illuminate model blind spots, and develop practical skills in scrutinizing outputs. Such cross-functional exercises build a common language and shared accountability for AI outcomes.</li><li><strong>Practical Upskilling Initiatives: </strong>Building a robust internal AI community involves more than a few training sessions. Consider regular “AI Literacy Bootcamps” that demystify AI concepts, introduce key terminology, and connect technical advances to real business use cases. “GenAI Prompt Engineering Workshops” can help non-technical staff experiment with generative AI tools, enabling them to craft effective prompts and understand their limitations. “Responsible AI Awareness Sessions” can guide teams through regulatory landscapes, bias mitigation strategies, and the societal impacts of AI adoption.</li><li><strong>Establishing a Community of Practice: </strong>An internal network — spanning data scientists, engineers, business analysts, and non-technical roles — serves as a forum for sharing best practices and lessons learned. Regular meetups, lunch-and-learns, and open Q&amp;A hours create a psychologically safe environment to ask questions, troubleshoot challenges, and celebrate wins. Peer-to-peer mentoring further reinforces ongoing skill development and knowledge transfer.</li><li><strong>Making AI Accessible Through Self-Service: </strong>To nurture ownership, organizations should provide low-code/no-code AI tools and dashboards. Empowering business users to experiment with data, build simple models, or simulate outcomes without heavy IT involvement encourages curiosity, accelerates adoption, and closes the gap between business needs and technical execution.</li><li><strong>Measuring Impact and Progress: </strong>Track engagement with AI learning resources, the diversity of teams building AI solutions, and improvements in AI-driven decision-making. Use surveys and case studies to surface success stories and identify knowledge gaps. This ongoing feedback loop ensures that AI democratization is a living, evolving goal rather than a one-off initiative.</li></ul><p>By embracing these practices, organizations create a thriving culture where AI becomes everyone’s business — not just the purview of a select few. This not only drives better outcomes but also builds resilience, ethical awareness, and sustainable innovation across the enterprise.</p><h4><em>Key tools</em>:</h4><ul><li>AI Literacy Bootcamps</li><li>GenAI Prompt Engineering Workshops</li><li>Responsible AI Awareness Sessions</li><li>Internal AI Community of Practice</li></ul><h4>5. Scaling Through Platforms and Guardrails</h4><p>Successfully scaling AI initiatives across an organization demands far more than isolated tools or ad hoc deployments. At its core, scaling requires robust, standardized platforms that serve as the foundation for reproducibility, governance, and operational excellence — while also embedding guardrails that ensure security, compliance, and ethical responsibility at every stage.</p><ul><li><strong>Reusable Pipelines and Modular Workflows : </strong>Modern AI platforms such as <strong>MLOps</strong> (Machine Learning Operations) and <strong>LLMOps</strong> (Large Language Model Operations) frameworks empower organizations to automate the end-to-end lifecycle of AI models. This includes data collection, preprocessing, feature engineering, training, validation, deployment, monitoring, and continuous improvement. By building reusable pipelines, teams avoid redundant effort, accelerate time to value, and ensure consistency across projects. Modular workflows also make it easier to onboard new teams: rather than reinventing the wheel, they can tap into proven templates and practices.</li><li><strong>Centralized Monitoring and Drift Detection: </strong>When AI solutions operate at scale, model performance can degrade due to data drift, concept drift, or changing business contexts. Centralized monitoring platforms allow teams to proactively detect anomalies, performance drops, or fairness issues. Early detection means faster interventions, preventing business disruptions or harm to end users. Implementing real-time dashboards and alerting systems also encourages a culture of shared responsibility for AI outcomes.</li><li><strong>Model Lineage, Auditability, and Impact Tracking: </strong>Traceability is vital in regulated industries and for public trust. Platforms that track model lineage — recording the data, algorithms, parameters, and deployment history behind each model — make it possible to audit decisions, reproduce results, and understand the impact of AI on business processes. This is essential for incident response, compliance checks, and fostering transparency for both internal and external stakeholders.</li></ul><h4><strong>Guardrails: Security, Privacy, and Ethical Safeguards</strong></h4><p>With great power comes great responsibility. Scaling AI safely means embedding guardrails directly into the platform level:</p><ul><li><strong>Access Controls: </strong>Ensure only authorized users can deploy, modify, or access sensitive models and datasets.</li><li><strong>Automated Compliance Checks: </strong>Integrate regulatory checks (e.g., GDPR, HIPAA, financial regulations) into the development lifecycle so that policies are enforced by default, not as an afterthought.</li><li><strong>Bias and Fairness Audits: </strong>Incorporate automated fairness testing and flagging mechanisms to surface demographic or outcome imbalances before models impact real users.</li><li><strong>Explainability and Documentation: </strong>Require model documentation, rationale, and explainability artifacts at deployment, supporting downstream trust and accountability.</li></ul><h4><strong>Case Studies: Platforms in Action</strong></h4><p>Consider <a href="https://www.uber.com/en-IN/blog/michelangelo-machine-learning-platform/">Uber’s Michelangelo </a>and <a href="https://www.meritshot.com/airbnb-case-study/">Airbnb’s Bighead</a>: these internal machine learning platforms empower hundreds of teams to develop, test, and deploy AI solutions using standardized workflows, built-in monitoring, and compliance guardrails. This consistency not only accelerates innovation but also ensures that every team, regardless of technical background, adheres to the organization’s best practices and ethical standards. The result is a scalable AI ecosystem where experimentation thrives within the boundaries of safety and compliance.</p><h4>Fostering Collaboration and Continuous Improvement</h4><p>Scaling through platforms and guardrails isn’t only a technical matter — it’s cultural. Successful organizations invest in cross-functional communities, shared resource libraries, and regular review cycles. They encourage teams to contribute improvements back to the platform, fostering a sense of ownership and collective progress. Platforms evolve, guardrails adapt, and the entire enterprise benefits from a virtuous cycle of learning and refinement.</p><p>In summary, scaling AI through robust platforms and thoughtfully designed guardrails transforms isolated successes into enterprise-wide impact — enabling organizations to innovate boldly, act responsibly, and adapt swiftly as AI capabilities continue to advance.</p><blockquote><strong><em>“Scale with safety, automate with accountability.” </em></strong><em>— </em>My personal mantra when leading ML practice transformations.</blockquote><h4>6. Embedding Responsible AI and Ethics</h4><p>Embedding Responsible AI and Ethics goes far beyond implementing technical guardrails — it signifies a holistic commitment to integrity, transparency, and societal well-being at every stage of the AI lifecycle.</p><p>AI-first must also mean <strong>risk-first</strong>. Organizations need:</p><ul><li>Ethics review boards</li><li>Bias detection pipelines</li><li>Differential privacy standards</li><li>Human-in-the-loop systems for sensitive use cases</li></ul><p>First, organizations should establish multidisciplinary ethics review boards that include not just data scientists and engineers, but also ethicists, legal experts, frontline employees, and representatives from impacted communities. These boards regularly assess AI projects for unintended consequences, potential harms, and alignment with organizational values. For example, they can review model training data for unintentional biases or evaluate the societal impact of automated decision-making systems.</p><p>Developing robust bias detection pipelines is also essential. These pipelines integrate fairness metrics and auditing tools directly into model development workflows, automatically flagging disparities in model predictions across demographic groups. This allows for early intervention — teams can retrain, rebalance, or adjust algorithms before deployment. Some organizations go further, publishing fairness dashboards and bias audit results to increase accountability and public trust.</p><p>Differential privacy standards should be integrated into every data pipeline. By employing techniques such as data anonymization, synthetic data generation, or noise injection, companies can maintain individual privacy while still extracting meaningful insights for model training. This is especially critical in sectors with sensitive data, such as healthcare or finance, where regulatory requirements are strict and the risks of misuse are high.</p><p>Human-in-the-loop systems are another key pillar of responsible AI. For sensitive use cases — such as hiring, lending, or healthcare recommendations — AI should augment, not replace, human judgment. These systems enable experts to review, override, or provide feedback on AI-generated decisions, ensuring that accountability remains clear and that nuanced, context-specific insights are not lost in automation.</p><p>Transparency and communication form the glue holding these efforts together. Organizations should publish concise, plain-language documents that explain how and why AI models make decisions. These explanations should be accessible to customers, regulators, and internal stakeholders alike, helping demystify complex algorithms and fostering a culture of trust. According to a Deloitte study, 63% of consumers are more likely to trust companies that openly explain their use of AI — a powerful incentive for transparency.</p><p>Finally, <strong>“Red Flag AI”</strong> protocols can act as an automated early warning system. By continuously monitoring deployed models for shifts in fairness metrics or unexpected data changes, these systems can alert compliance and ethics leads as soon as risks emerge. This proactive stance not only mitigates potential harm but also demonstrates a public commitment to ethical leadership.</p><p>In summary, embedding <strong>Responsible AI and Ethics </strong>means weaving ethical considerations into the very fabric of organizational processes — from concept and design to deployment and continuous monitoring. It transforms regulatory compliance from a checkbox exercise into a living, adaptive practice that earns stakeholder trust and secures sustainable AI success.</p><p><em>Here is the ready-to-use AI-Maturity Checklist in my </em><a href="https://github.com/puspanjalis/AI-First-Enterprise-Maturity-Checklist/tree/main"><em>Github</em></a><em>!</em></p><h3>Case Study: What It Looks Like in Practice</h3><p>Let’s take an enterprise scenario — <strong>Customer Support Automation</strong>:</p><p>One of the most visible frontiers for AI-first thinking is <strong>customer support transformation</strong>. Traditional approaches have relied heavily on scripts, tiered escalation, and retrospective analytics. But an AI-first mindset asks: <em>What if support could be anticipatory, emotional, and real-time?</em></p><p>When the enterprise shifts its <em>mindset</em> from “automate tasks” to “amplify intelligence,” the results go from cost savings to <strong>business advantage</strong>.</p><h4>Real-World Spotlight: Vodafone</h4><p>Vodafone, a global telecom giant, deployed an AI virtual assistant called <strong>TOBi</strong> across its customer touchpoints. Initially used for simple inquiries, TOBi now:</p><ul><li>Handles over <strong>60%</strong> of customer queries end-to-end without human intervention</li><li>Contributed to a <strong>15-point increase</strong> in Net Promoter Score (NPS)</li><li>Reduced human agent workload significantly, enabling them to focus on complex and emotionally sensitive cases</li></ul><p>This isn’t just chatbot automation — it’s <strong>multi-intent prediction</strong>, <strong>context preservation</strong>, and <strong>language style adaptation</strong> tuned to customer personas.</p><h3>Metrics That Matter</h3><p>From industry benchmarks and client implementations, here’s what an AI-first support model can deliver:</p><ul><li><strong>40% drop in support call volume</strong> when AI triages tickets preemptively (<a href="https://www2.deloitte.com/content/dam/Deloitte/us/Documents/consulting/us-using-generative-ai-to-reduce-it-ticket-volume-2.pdf">Deloitte, 2023</a>)</li><li><strong>30% faster resolution time</strong> due to predictive classification and auto-recommendations (<a href="https://www.servicenow.com/workflow/learn/how-cx-ai-improving-customer-satisfaction.html">ServiceNow AI in CX Report</a>)</li><li><strong>22% increase in satisfaction scores</strong> when empathetic LLM-generated replies were used for upset customers (<a href="https://www.salesforce.com/in/news/stories/ai-customer-research/">Salesforce Research, 2024</a>)</li><li><strong>Up to 50% cost savings</strong> on Level 1 support when AI agents handle first contact resolution (<a href="https://www.mckinsey.com/capabilities/operations/our-insights/the-next-frontier-of-customer-engagement-ai-enabled-customer-service">McKinsey, 2023</a>)</li></ul><h3>Strategic Mindset Shift</h3><p>Most companies approach AI in support as a way to “cut costs.” But organizations that thrive view AI as a <strong>customer experience amplifier</strong>.</p><p>The transformation starts when leaders ask:</p><blockquote>“How can we make support smarter, not just cheaper?”</blockquote><p>This is the core of the AI-first mindset: moving from automation of tasks to <strong>augmentation of decisions</strong> and <strong>anticipation of needs</strong>.</p><h3>From AI-First to Future-Ready</h3><p><em>“The real AI transformation happens when companies stop asking how to adopt AI — and start asking how AI can redefine their business.” — Puspanjali Sarma</em></p><p>Embracing an AI-first mindset means more than simply implementing new technologies; it requires a fundamental reimagining of how organizations function at every level. Embedding intelligence into your organizational DNA involves weaving AI-driven thinking into daily processes, strategic decision-making, and company culture.</p><p>This shift starts with leadership. Visionary leaders champion the integration of AI not just as a tool but as a catalyst for continuous learning and innovation. They encourage teams to approach challenges with curiosity, to question assumptions, and to leverage data as a source of truth. This mindset fosters a culture where experimentation is rewarded, and failure is viewed as a step toward insight, not a setback.</p><p>Embedding intelligence also means operationalizing AI beyond isolated initiatives. Instead of confining AI to a single department or project, future-ready organizations infuse intelligence across all functions — from customer support to supply chain, HR to product development. For example, teams might use machine learning to predict customer churn, optimize inventory, or personalize employee development plans. This cross-functional approach unlocks new efficiencies and enables proactive, rather than reactive, decision-making.</p><p>Another key aspect is ethical stewardship. As organizations become more reliant on automated decision-making, establishing frameworks for transparency, fairness, and accountability is essential. Regular audits, cross-disciplinary ethics committees, and clear communication with stakeholders can help ensure AI systems align with organizational values and societal expectations.</p><p>Finally, building an intelligence-driven enterprise means investing in people as much as in platforms. Upskilling employees to work alongside AI — and fostering digital literacy at all levels — empowers teams to harness new technologies creatively and responsibly. When AI literacy permeates every layer of the organization, innovation becomes democratized, and adaptability becomes second nature.</p><p>Let’s not just implement AI. Let’s build enterprises that think like AI systems — curious, data-driven, ethical, and always learning. This is the blueprint for future-ready organizations, prepared to thrive in a landscape where intelligence is the new currency of success.</p><h4>Reflection Questions for Enterprise Leaders:</h4><ul><li><em>What would your business model look like if intelligence was embedded into every process?</em></li><li><em>Are your teams incentivized to experiment with AI or avoid risk?</em></li><li><em>How would your customers feel about your use of AI — empowered or exploited?</em></li><li><em>Are you building AI as a tool — or as a new way of thinking?</em></li></ul><h3>Next Steps</h3><p>If you’re a business or technology leader on this journey:</p><ul><li>Start small, but <strong>think big</strong>.</li><li>Invest in people as much as platforms.</li><li>Treat AI not as a trend — but as your next transformation.</li></ul><p>Feel free to connect on <a href="https://www.linkedin.com/in/puspanjalisarma/">Linkedin</a> , <a href="https://github.com/puspanjalis">Github</a> and <a href="https://medium.com/@puspanjalisarma23">Medium</a>!— I’m always keen to collaborate on designing <strong>AI-first, human-centered enterprises</strong>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=635f6cc76382" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to Calculate the Accuracy Percentage of a SQL Query Generated by an LLM]]></title>
            <link>https://medium.com/@puspanjalisarma23/how-to-calculate-the-accuracy-percentage-of-a-sql-query-generated-by-an-llm-4cbd998b2e03?source=rss-657cc3b2578c------2</link>
            <guid isPermaLink="false">https://medium.com/p/4cbd998b2e03</guid>
            <category><![CDATA[snowflake]]></category>
            <category><![CDATA[agentic-applications]]></category>
            <category><![CDATA[agentic-ai]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[llm]]></category>
            <dc:creator><![CDATA[Puspanjali Sarma]]></dc:creator>
            <pubDate>Thu, 05 Jun 2025 11:26:56 GMT</pubDate>
            <atom:updated>2025-06-05T11:26:56.033Z</atom:updated>
            <content:encoded><![CDATA[<h4>Test and improve model outputs through automated evaluation framework</h4><p>Large Language Models (LLMs) like ChatGPT, Snowflake Cortext AI, Claude, and Gemini are now generating SQL queries from natural language prompts with remarkable fluency. This has revolutionized how we interact with databases — making SQL more accessible to analysts, product managers, and business users alike.</p><p>But here’s the challenge: <strong><em>How do we know if the generated SQL is actually correct?</em></strong></p><p>It’s not enough that the query runs — it must return the <em>right</em> data.</p><p>In this article, you’ll learn how to <strong>evaluate the accuracy</strong> of LLM-generated SQL queries using:</p><ul><li>Ground truth comparisons</li><li>Result-set validation</li><li>Optional semantic assessment using LLM-as-a-Judge</li></ul><p>And <em>yes</em> — there’s a GitHub-ready Python project included that you can clone and run today.</p><h3>Why Accuracy Evaluation Matters</h3><p>Natural language interfaces to databases are no longer futuristic. Tools like ChatGPT, GitHub Copilot, and custom enterprise agents are enabling analysts and business users to write complex queries with just a prompt.</p><p>Relying on LLM-generated SQL without validation is like letting a junior analyst run production queries blindfolded. Incorrect queries can:</p><ul><li>Undermine trust in AI systems</li><li>Return incomplete, incorrect, or misleading data</li><li>Fail silently in mission-critical pipelines</li></ul><p>But LLMs are probabilistic, not deterministic. This means they might generate plausible-looking SQL that fails:</p><ul><li>to meet the full business requirement.</li><li>to match the expected schema.</li><li>or worse, to return incorrect or incomplete results.</li></ul><p><strong><em>Evaluating SQL accuracy is key</em></strong><em> to building reliable, production-grade systems that use LLMs for data analysis.</em></p><p>Evaluating the accuracy percentage ensures that:</p><ul><li>You quantify how “correct” each query is</li><li>You identify edge cases and model blind spots</li><li>You track improvements in LLM SQL generation performance</li></ul><h3>Evaluation Workflow</h3><p>Here’s a high-level view of how SQL query accuracy is evaluated:</p><h4>Evaluation Modes</h4><ul><li><strong>Exact Match</strong>: Rows and columns must match 100%.</li><li><strong>Subset Match</strong>: Checks if the generated result is a subset of the golden result.</li><li><strong>LLM-as-a-Judge</strong>: Use GPT to rate query relevance.Let’s walk through each step with implementable Python code.</li></ul><p>Now, let’s walk through each step with implementable Python code used in the evaluation framework.</p><h3>Create a Project Structure</h3><pre>llm_sql_accuracy_evaluator/<br>├── data/<br>│   └── db.sqlite             # Sample SQLite database<br>├── golden_queries/<br>│   └── benchmark.json        # JSON file with questions and ground-truth SQL<br>├── src/<br>│   ├── evaluator.py          # Core logic for executing and comparing queries<br>│   ├── sql_generator.py      # Placeholder for LLM SQL generation logic<br>│   └── judge.py              # Optional: LLM-as-a-judge evaluation<br>├── notebook/<br>│   └── example_usage.ipynb   # Sample usage in Jupyter Notebook<br>├── requirements.txt<br>└── README.md</pre><h3>Step 1: Setup and Golden Queries</h3><p>We assume a simple SQLite database with a table like:</p><pre>CREATE TABLE employees (<br>    id INTEGER,<br>    employee_name TEXT,<br>    department TEXT,<br>    salary INTEGER<br>);</pre><p>Let’s define our ground truth — the “correct” SQL for a natural language question:</p><p>📄 golden_queries/benchmark.json</p><pre>{<br>  &quot;Q1&quot;: {<br>    &quot;question&quot;: &quot;Get all employees with salary greater than 50000&quot;,<br>    &quot;sql&quot;: &quot;SELECT * FROM employees WHERE salary &gt; 50000&quot;<br>  },<br>  &quot;Q2&quot;: {<br>    &quot;question&quot;: &quot;List names of employees in HR&quot;,<br>    &quot;sql&quot;: &quot;SELECT employee_name FROM employees WHERE department = &#39;HR&#39;&quot;<br>  }<br>}</pre><p>This golden SQL serves as the benchmark to evaluate the generated SQL against.</p><h3>Step 2: Generate SQL Using an LLM</h3><p>In a real-world app, this would come from OpenAI, Claude, etc. Here’s a mock generator for testing:</p><p>📄 src/sql_generator.py</p><pre>def mock_llm_sql_generator(question):<br>    if &quot;salary&quot; in question:<br>        return &quot;SELECT employee_name, employee_id FROM employees WHERE salary &gt; 50000&quot;<br>    return &quot;SELECT * FROM employees&quot;</pre><p>Notice how it simplifies the output — returning only two columns instead of all.</p><h3>Step 3: Execute Both Queries</h3><p>We execute the golden and generated queries on the same database:</p><p>📄 src/evaluator.py</p><pre>import sqlite3<br>import pandas as pd<br><br>DB_PATH = &#39;data/db.sqlite&#39;<br>def run_query(query):<br>    conn = sqlite3.connect(DB_PATH)<br>    return pd.read_sql_query(query, conn)</pre><p>This gives us two DataFrames to compare.</p><h3>Step 4: Compare Results</h3><h4>Column Match Score</h4><pre>def column_match_score(df1, df2):<br>    cols_1 = set(df1.columns)<br>    cols_2 = set(df2.columns)<br>    return len(cols_1 &amp; cols_2) / len(cols_1 | cols_2) * 100</pre><h4>Row Match Score</h4><pre>def row_match_score(df1, df2):<br>    set_1 = set(df1.apply(tuple, axis=1))<br>    set_2 = set(df2.apply(tuple, axis=1))<br>    return len(set_1 &amp; set_2) / len(set_1 | set_2) * 100</pre><h3>Final Accuracy Score</h3><pre>def compute_accuracy(df_gold, df_gen):<br>    row_score = row_match_score(df_gold, df_gen)<br>    col_score = column_match_score(df_gold, df_gen)<br>    return round((row_score + col_score) / 2, 2)</pre><p>You can now use this function to evaluate any query.</p><h3>Example</h3><pre>from src.evaluator import run_query, compute_accuracy<br>from src.sql_generator import mock_llm_sql_generator<br>import json<br><br>with open(&#39;golden_queries/benchmark.json&#39;) as f:<br>    gold = json.load(f)<br>    question = gold[&#39;Q1&#39;][&#39;question&#39;]<br>    golden_sql = gold[&#39;Q1&#39;][&#39;sql&#39;]<br>generated_sql = mock_llm_sql_generator(question)<br>df_gold = run_query(golden_sql)<br>df_gen = run_query(generated_sql)<br>accuracy = compute_accuracy(df_gold, df_gen)<br>print(f&quot;Accuracy: {accuracy}%&quot;)</pre><h3>Sample Output:</h3><pre>Accuracy: 75.0%</pre><p>Explanation: Full row match (100%), partial column match (50%) → average = 75%</p><h3>Bonus: LLM-as-a-Judge (Semantic Score)</h3><p>For more nuanced evaluation, use an LLM to <em>rate</em> the query:</p><p>📄 src/judge.py</p><pre>import openai<br><br>def llm_judge_score(question, generated_sql, golden_sql):<br>    prompt = f&quot;&quot;&quot;<br>You are an expert SQL reviewer.<br>Question: {question}<br>Golden SQL: {golden_sql}<br>Generated SQL: {generated_sql}<br>Rate the accuracy of the generated SQL from 0 to 100 based on how well it meets the intent.<br>&quot;&quot;&quot;<br>    response = openai.ChatCompletion.create(<br>        model=&quot;gpt-4&quot;,<br>        messages=[{&quot;role&quot;: &quot;user&quot;, &quot;content&quot;: prompt}]<br>    )<br>    return float(response.choices[0].message[&#39;content&#39;].strip())</pre><p>Use this when result-set comparisons aren’t enough — especially for subjective or schema-free environments.</p><p>If you want to expand this frame work to other data engineering frameworks, here are the enhancements</p><h3>1. Support for Snowflake and Multiple Backends</h3><h4>Enhancement</h4><p>Abstract the database layer to support multiple engines like Snowflake, Postgres, BigQuery, etc.</p><h4>Implementation</h4><p>Create a db_connector.py to switch between backends:</p><pre>def run_query(query, engine=&#39;sqlite&#39;, conn_params={}):<br>    if engine == &#39;sqlite&#39;:<br>        import sqlite3<br>        conn = sqlite3.connect(conn_params.get(&quot;path&quot;, &quot;data/db.sqlite&quot;))<br>    elif engine == &#39;snowflake&#39;:<br>        import snowflake.connector<br>        conn = snowflake.connector.connect(**conn_params)<br>    else:<br>        raise ValueError(&quot;Unsupported engine&quot;)<br><br>df = pd.read_sql(query, conn)<br>    return df</pre><p>You can extend conn_params with Snowflake fields:</p><pre>{<br>  &quot;user&quot;: &quot;...&quot;,<br>  &quot;password&quot;: &quot;...&quot;,<br>  &quot;account&quot;: &quot;...&quot;,<br>  &quot;warehouse&quot;: &quot;...&quot;,<br>  &quot;database&quot;: &quot;...&quot;,<br>  &quot;schema&quot;: &quot;...&quot;<br>}</pre><h3>2. Test Suite for LLM Query Evaluations</h3><h4>Enhancement</h4><p>Support batch evaluation of multiple query pairs (golden + generated).</p><h4>Implementation</h4><p>Create test_cases.json:</p><pre>[<br>  {<br>    &quot;question_id&quot;: &quot;Q1&quot;,<br>    &quot;generated_sql&quot;: &quot;SELECT id FROM employees WHERE salary &gt; 50000&quot;<br>  },<br>  {<br>    &quot;question_id&quot;: &quot;Q2&quot;,<br>    &quot;generated_sql&quot;: &quot;SELECT * FROM employees WHERE department = &#39;HR&#39;&quot;<br>  }<br></pre><p>Then create a batch evaluator:</p><pre>def evaluate_batch(test_cases_path):<br>    with open(test_cases_path) as f:<br>        cases = json.load(f)<br>    results = []<br>    for case in cases:<br>        accuracy = evaluate_query_accuracy(case[&#39;question_id&#39;], case[&#39;generated_sql&#39;])<br>        results.append({**case, &quot;accuracy&quot;: accuracy})<br>    return pd.DataFrame(results)</pre><h3>3. Schema and Semantic Comparison</h3><h4>Enhancement</h4><p>Check for structural similarity, not just result content.</p><h4>Implementation Ideas:</h4><ul><li>Use SQL parsers like sqlparse or mo_sql_parsing to extract selected fields, conditions, joins.</li><li>Compare query components: SELECT, FROM, WHERE, GROUP BY, etc.</li><li>Apply token-based similarity (e.g., Levenshtein, Jaccard) between ASTs.</li></ul><h3>4. Metrics Dashboard (Optional Streamlit App)</h3><h4>Enhancement</h4><p>Visualize accuracy results interactively.</p><h4>Implementation</h4><p>Create a Streamlit dashboard:</p><pre>import streamlit as st<br>df = evaluate_batch(&quot;test_cases.json&quot;)<br>st.dataframe(df)<br>st.bar_chart(df.set_index(&quot;question_id&quot;)[&quot;accuracy&quot;])</pre><p>This makes it useful for reviewing LLM performance over time.</p><h3>5. Advanced LLM-as-a-Judge Configurations</h3><h4>Enhancement</h4><p>Let users choose between scoring styles:</p><ul><li>“Strict” (logic + structure)</li><li>“Lenient” (intent match only)</li><li>“Explainable” (also returns explanation)</li></ul><h4>Implementation</h4><p>Use an optional config file:</p><pre>{<br>  &quot;llm_judge_mode&quot;: &quot;explainable&quot;,<br>  &quot;model&quot;: &quot;gpt-4&quot;,<br>  &quot;temperature&quot;: 0.2<br>}</pre><p>Update judge.py to read from it and vary response format based on mode.</p><h3>6. Confidence Scoring + Logging for Auditability</h3><h4>Enhancement</h4><p>Add metadata to each evaluation:</p><ul><li>Query execution success/failure</li><li>Runtime</li><li>Errors (if any)</li><li>Confidence (from judge or model uncertainty)</li></ul><h4>Implementation</h4><p>Return evaluation metadata like:</p><pre>{<br>  &quot;question_id&quot;: &quot;Q1&quot;,<br>  &quot;accuracy&quot;: 75.0,<br>  &quot;row_score&quot;: 100.0,<br>  &quot;column_score&quot;: 50.0,<br>  &quot;execution_success&quot;: true,<br>  &quot;llm_confidence&quot;: 0.87,<br>  &quot;errors&quot;: null<br>}</pre><p>Store these logs to a local file or Snowflake table for QA workflows.</p><h3>7. CI/CD &amp; GitHub Actions Integration</h3><h4>Enhancement</h4><p>Run SQL accuracy tests in pull requests using GitHub Actions.</p><h4>Implementation</h4><p>Create .github/workflows/sql_eval.yml:</p><pre>jobs:<br>  test:<br>    runs-on: ubuntu-latest<br>    steps:<br>    - uses: actions/checkout@v2<br>    - name: Install dependencies<br>      run: pip install -r requirements.txt<br>    - name: Run evaluator<br>      run: python run_batch_eval.py</pre><h3>8. Snowflake-specific Enhancements</h3><ul><li>Handle fully-qualified object names (DB.SCHEMA.TABLE)</li><li>Validate role-based access control (RBAC) on query execution</li><li>Use Snowflake’s result caching to reduce cost</li><li>Add retry and timeout logic for long-running queries</li></ul><h3>9. CI/CD Integration Highlights</h3><ul><li>GitHub Actions triggers on every push or pull request to the main branch</li><li>Installs dependencies from requirements.txt</li><li>Runs the evaluate_batch() function to validate generated SQL against golden queries</li><li>Outputs accuracy scores directly in the CI logs</li></ul><h4>Key CI File</h4><p>.github/workflows/sql_eval.yml</p><p>You can now:</p><ul><li>Upload the project to a GitHub repository.</li><li>GitHub Actions will automatically run on PRs to validate SQL accuracy.</li><li>Extend this with Slack alerts, PR comments, or test thresholds if needed.</li></ul><h3>GitHub Repository</h3><p>Want to try it hands-on?</p><p>👉<a href="https://github.com/puspanjalis/LLM-SQL-Accuracy-Evaluator"> GitHub: LLM SQL Accuracy Evaluator</a></p><p>The repo includes:</p><ul><li>Sample SQLite database</li><li>Golden benchmark file</li><li>Python evaluator module</li><li>Jupyter notebook for testing</li><li>Optional OpenAI API integration</li></ul><p>Clone it, test it, and adapt it to your own data environment.</p><h4>Steps to deploy in GitHub, follow these steps:</h4><ol><li><strong>Create a New Repository (without forking)</strong>:</li></ol><ul><li>Navigate to <a href="https://github.com/">GitHub</a> and log in to your account.</li><li>Click on the “+” icon in the upper-right corner and select “New repository”.</li><li>Name your repository (e.g., llm-sql-accuracy-evaluator) and provide a description.</li><li>Choose the visibility (public or private) and click “Create repository”.</li></ul><p><strong>2. Initialize Local Repository</strong>:</p><ul><li>On your local machine, open a terminal and navigate to your project directory.</li><li>Initialize Git:</li></ul><pre>git init</pre><ul><li>Add all files:</li></ul><pre>git add .</pre><ul><li>Commit the changes:</li></ul><pre>git commit -m &quot;Initial commit&quot;</pre><p><strong>3. Link to GitHub Repository</strong>:</p><ul><li>Add the remote repository:</li></ul><pre>git remote add origin https://github.com/yourusername/llm-sql-accuracy-evaluator.gitp</pre><ul><li>Push the changes:</li></ul><pre>git push -u origin master</pre><p><strong>4. Set Up GitHub Pages (Optional for Documentation)</strong>:</p><ul><li>If you want to host documentation or a static site, you can use GitHub Pages:</li><li>Go to your repository on GitHub.</li><li>Click on “Settings” &gt; “Pages”.</li><li>Under “Source”, select the branch and folder (e.g., /docs) to publish.</li><li>Click “Save”.</li></ul><p><strong>5. Configure GitHub Actions (Optional for CI/CD)</strong>:</p><p>To automate tasks like testing or deployment:</p><ul><li>In your repository, click on “Actions”.</li><li>Choose a workflow template or set up a custom workflow.</li><li>Commit the workflow file to the .github/workflows/ directory.</li></ul><p>For more detailed guidance on deploying with GitHub Actions, refer to GitHub’s official documentation: <a href="https://docs.github.com/en/actions/use-cases-and-examples/deploying/deploying-with-github-actions">Deploying with GitHub Actions</a>.</p><h3>🤝 Contributions</h3><p>Pull requests welcome! Please open an issue to discuss improvements or bugs.</p><pre>---<br>## 📄 src/evaluator.py<br><br>import sqlite3<br>import pandas as pd<br>import json<br>from difflib import SequenceMatcher<br><br>DB_PATH = &#39;data/db.sqlite&#39;<br>GOLDEN_QUERY_PATH = &#39;golden_queries/benchmark.json&#39;<br>def load_golden_query(question_id):<br>    with open(GOLDEN_QUERY_PATH) as f:<br>        data = json.load(f)<br>    return data[question_id][&quot;sql&quot;]<br>def run_query(query):<br>    conn = sqlite3.connect(DB_PATH)<br>    return pd.read_sql_query(query, conn)<br>def row_match_score(df1, df2):<br>    df1_set = set(df1.apply(tuple, axis=1))<br>    df2_set = set(df2.apply(tuple, axis=1))<br>    match_count = len(df1_set.intersection(df2_set))<br>    total_count = len(df1_set.union(df2_set))<br>    return (match_count / total_count) * 100 if total_count &gt; 0 else 0<br>def column_match_score(df1, df2):<br>    common_cols = set(df1.columns).intersection(set(df2.columns))<br>    total_cols = set(df1.columns).union(set(df2.columns))<br>    return (len(common_cols) / len(total_cols)) * 100 if total_cols else 0<br>def evaluate_query_accuracy(question_id, generated_query):<br>    golden_query = load_golden_query(question_id)<br>    try:<br>        df_golden = run_query(golden_query)<br>        df_generated = run_query(generated_query)<br>    except Exception as e:<br>        return 0.0<br>    row_score = row_match_score(df_golden, df_generated)<br>    column_score = column_match_score(df_golden, df_generated)<br>    return round((row_score + column_score) / 2, 2)</pre><p>If you need assistance with specific configurations or have further questions, feel free to ask!</p><h3>Final Thoughts</h3><p>Evaluating SQL queries generated by LLMs is more than syntax checking. It’s about ensuring <strong>data integrity</strong>, <strong>intent alignment</strong>, and <strong>business correctness</strong>.</p><p>By combining:</p><ul><li>Exact result comparison</li><li>Subset logic for flexibility</li><li>LLM-driven semantic evaluation</li></ul><p>— you get a robust framework for validating SQL quality in real-time systems.</p><p>If you’re building agentic AI workflows, BI copilots, or internal query assistants, this framework is for you.</p><h3>Tools &amp; References</h3><ul><li><a href="https://github.com/defog-ai/sql-eval">Defog SQL Eval (GitHub)</a></li><li><a href="https://github.com/IBM/sql-eval">IBM Developer SQL-Eval</a></li><li><a href="https://platform.openai.com/docs/guides/evals">OpenAI Eval Framework</a></li></ul><h3><strong>Let’s Connect</strong></h3><p>If you found this useful, consider:</p><ul><li>Following me here on <a href="https://www.linkedin.com/in/puspanjalisarma/">Linkedin</a> , <a href="https://github.com/puspanjalis">Github</a> and <a href="https://medium.com/@puspanjalisarma23">Medium</a>!</li><li>Starring the <a href="https://github.com/puspanjalis/LLM-SQL-Accuracy-Evaluator">GitHub repo</a></li><li>Sharing this with your data and ML teams</li></ul><blockquote>Thanks for reading — and may your queries always return the truth!</blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4cbd998b2e03" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Building Agentic AI Workflows with Snowflake: A Comprehensive, Hands-On Guide]]></title>
            <link>https://medium.com/@puspanjalisarma23/building-agentic-ai-workflows-with-snowflake-a-comprehensive-hands-on-guide-057290fe8ff7?source=rss-657cc3b2578c------2</link>
            <guid isPermaLink="false">https://medium.com/p/057290fe8ff7</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[innovation]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[ai-agents-in-action]]></category>
            <category><![CDATA[data]]></category>
            <dc:creator><![CDATA[Puspanjali Sarma]]></dc:creator>
            <pubDate>Tue, 27 May 2025 15:57:08 GMT</pubDate>
            <atom:updated>2025-06-05T10:31:14.174Z</atom:updated>
            <content:encoded><![CDATA[<h4>Unlocking Autonomous Decision-Making with Advanced AI and Data Integration</h4><p>The convergence of Agentic AI and Snowflake heralds a transformative era for enterprises seeking to automate processes and enable autonomous decision-making. This guide explores how Snowflake integrates with Agentic AI technologies to create intelligent workflows, offering actionable insights and practical strategies for implementation.</p><h3>Overview of Agentic AI</h3><p>Agentic AI represents a paradigm shift in artificial intelligence. Unlike traditional AI systems that generate outputs for human interpretation, Agentic AI autonomously analyzes and acts on data, optimizing performance without requiring constant intervention. Key features include:</p><ul><li>Autonomous Decision-Making: Performing tasks and making decisions based on real-time data analysis.</li><li>Dynamic Optimization: Adapting actions to varied environments and conditions using reinforcement learning and adaptive algorithms.</li><li>Minimal Human Intervention: Handling predefined or evolving tasks autonomously.</li></ul><h3>Snowflake Cortex and Semantic Models</h3><p><a href="https://www.snowflake.com/en/product/features/cortex/">Snowflake Cortex</a> is a suite of AI-powered services designed to bring generative AI and machine learning directly into the Snowflake ecosystem. It includes components like <a href="https://docs.snowflake.com/en/user-guide/snowflake-cortex/cortex-analyst">Cortex Analyst</a>, <a href="https://docs.snowflake.com/en/user-guide/snowflake-cortex/cortex-agents">Cortex Agent</a>, and <a href="https://docs.snowflake.com/en/user-guide/snowflake-cortex/cortex-search/cortex-search-overview">Cortex Search</a>, each catering to different AI-driven use cases.</p><h3>Snowflake’s BI assistant — Cortex Analyst</h3><p>Cortex Analyst is part of Snowflake Cortex. It functions as an AI-powered BI assistant, translating natural language into optimized SQL queries. This capability relies on pre-trained large language models to interpret user intent, semantic models to ensure accurate query generation, and direct integration with Snowflake for secure, governed data access.</p><figure><img alt="Cortex Analyst, Image by Snowflake" src="https://cdn-images-1.medium.com/max/468/1*8beLBEcI9h3QM2Jqo6cPqA.png" /><figcaption>Cortex Analyst, Image by <a href="https://www.youtube.com/watch?v=DB281MC9aU4">Snowflake</a></figcaption></figure><h3>Snowflake’s Search Engine — Cortex Search</h3><figure><img alt="Cortex Search, Image by Snowflake" src="https://cdn-images-1.medium.com/max/468/1*Ef0WUS7hKTsm_qop38mmng.png" /><figcaption>Cortex Search, Image by <a href="https://www.youtube.com/watch?v=DB281MC9aU4">Snowflake</a></figcaption></figure><p>Cortex Search is a search engine that powers customized LLM apps in Snowflake the Hybrid search engine leverages vector embeddings for semantic similarity plus keyword search for lexical similarity, achieving state-of-the-art retrieval quality.</p><p>It is Fully managed in that it automatically updates with your underlying data, secure and governed in Snowflake, and exposed via simple APIs with no operational overhead.</p><p><strong>Why use it?</strong></p><ul><li>RAG engine for LLM apps: Cortex Search as a RAG engine for “chat with your unstructured data” apps. You can leverage semantic search for customized and contextualized responses.</li><li>Enterprise search: Cortex Search can also be used as a backend for a powerful, performant search bar in your internally or externally-facing application.</li></ul><h3>Snowflake’s Agentic AI Solution: Cortex Agents</h3><p>Snowflake introduces Cortex Agents, a managed service that empowers developers to build applications capable of autonomous decision-making. Cortex Agents utilize Snowflake’s robust data platform to process both structured and unstructured data, enabling intelligent systems to act independently. Core functionalities include:</p><ul><li>Training agents using proprietary enterprise data and large language models.</li><li>Automating insights extraction and task execution within applications.</li><li>Real-time adaptability based on new data inputs.</li></ul><figure><img alt="Cortex Agents framework, Image by AIM" src="https://cdn-images-1.medium.com/max/936/1*trmIzrryFg4WjfFVKJ3bGw.png" /><figcaption>Cortex Agents framework, Image by <a href="https://analyticsindiamag.com/ai-news-updates/snowflakes-new-cortex-agents-bridge-ai-and-enterprise-data-like-never-before/">AIM</a></figcaption></figure><h3>Snowflake for Agentic AI and Machine Learning</h3><p>Snowflake has evolved into a comprehensive platform supporting the entire lifecycle of Agentic AI systems. Its architecture provides:</p><ul><li>Unified Data Repository: Centralized storage of structured, semi-structured, and unstructured data.</li><li>Scalable Compute Resources: Dynamic scaling of resources to meet the demands of AI workloads.</li><li>Seamless Data Sharing: Facilitating collaboration across teams and organizations.</li></ul><p>Snowpark, Snowflake’s developer framework, enhances this capability by integrating with AI tools like TensorFlow, PyTorch, and Scikit-learn, allowing data scientists to build models directly within Snowflake.</p><h3>Architecture and Capabilities</h3><p>Snowflake’s architecture is designed to support complex Agentic AI workloads. Key features include:</p><ul><li>Multi-cluster Virtual Warehouses to optimize resources for specific tasks.</li><li>Metadata Layer which catalogues decision models, features, and actions.</li><li>Cloud Services Layer to allocate resources efficiently for autonomous operations.</li><li>Persistent Storage that maintains secure logs for auditing and model retraining.</li></ul><p>This architecture ensures seamless handling of diverse data sources, empowering AI systems to act intelligently and efficiently.</p><h3>How Snowflake Supports Agentic AI Workloads</h3><p>Snowflake has been purpose-built to address the sophisticated demands of agentic AI workloads. By centralizing data, optimizing resource allocation, and enabling seamless integration with leading AI and machine learning frameworks, Snowflake empowers developers to create systems capable of autonomous decision-making and real-time adaptability.</p><h3>Optimized Architecture for Agentic AI</h3><p>Snowflake’s architecture is meticulously crafted to ensure efficient handling of diverse data sources and operations:</p><p>· Multi-cluster Virtual Warehouses: Tailors resource allocation to specific tasks, ensuring optimal performance for AI-driven workloads.</p><ul><li>Metadata Layer: Organizes decision models, features, and actions for streamlined AI operations.</li><li>Cloud Services Layer: Dynamically allocates resources, enabling autonomous and efficient system operations.</li><li>Persistent Storage: Provides secure logs for auditing and model retraining, ensuring reliability and accountability.</li></ul><p>This robust foundation ensures that agentic AI systems can intelligently manage data while maintaining high efficiency and security.</p><h3>Integration with AI Frameworks</h3><p>Snowflake seamlessly integrates with industry-leading AI frameworks, making it a versatile hub for developing rich, end-to-end workflows. Key features include:</p><ul><li>Snowpark API: Executes Python or Scala code directly within Snowflake, facilitating data processing and model creation close to the data source.</li><li>AI Services Connections: Establishes links to AWS SageMaker, Azure ML, and GCP Vertex AI, enabling developers to train models or leverage third-party services without friction.</li></ul><p>These integrations allow developers to combine advanced computation with centralized data security, enhancing the development process for autonomous AI systems.</p><h3>Streamlined Data Preparation</h3><p>Preparing data is a cornerstone of building effective agentic AI workflows, and Snowflake simplifies this critical step through intuitive tools and processes:</p><ul><li>Data Transformation: Aggregates and normalizes raw data into formats optimized for AI models.</li><li>Feature Engineering: Enables the extraction of embeddings and the construction of features directly within Snowflake, enhancing model performance.</li></ul><h3>Efficient AI Development Using Snowpark</h3><p>Snowpark revolutionizes AI development workflows by bringing computation closer to the data, allowing for streamlined operations:</p><ul><li>DataFrame API: Integrates seamlessly with AI frameworks to enable efficient data manipulation.</li><li>User-Defined Functions (UDFs): Packages logic for autonomous model inference, fostering intelligent decision-making.</li><li>Stored Procedures: Facilitates complex workflows, supporting real-time automation for agentic AI systems.</li></ul><h3>Building and Deploying AI Systems</h3><p>Snowflake provides a unified solution for training, deploying, and managing agentic AI systems. By blending real-time data processing capabilities with scalable compute resources, it ensures that AI agents operate accurately and adaptively across various data environments. This unified platform centralizes structured, semi-structured, and unstructured data, improving decision-making precision while maintaining compliance and security.</p><p>With Snowflake’s advanced capabilities, developers have the tools to unlock the full potential of agentic AI, enabling systems to act autonomously in real-time and drive meaningful outcomes across industries.</p><h3>Building Agentic AI Workflows</h3><h4>Data Preparation</h4><p>Preparing data is a critical step in building Agentic AI workflows. Snowflake simplifies this process with tools like <a href="https://docs.snowflake.com/en/user-guide/data-load-snowpipe-intro">Snowpipe</a> for real-time ingestion and SQL for transformations. Key steps include:</p><ul><li>Data Transformation: Aggregating and normalizing data to make it suitable for AI models.</li><li>Feature Engineering: Extracting embeddings and constructing features directly within Snowflake.</li></ul><h4>Using Snowpark for Development</h4><p><a href="https://docs.snowflake.com/en/developer-guide/snowpark/overview">Snowpark</a> revolutionizes Agentic AI development by enabling computation close to the data. Features include:</p><p>· DataFrame API: Aligns with AI frameworks for efficient data manipulation.</p><p>· <a href="https://docs.snowflake.com/en/sql-reference/udf-python">User-Defined Functions (UDFs)</a>: Packages logic for autonomous model inference.</p><p>· Stored Procedures: Supports complex workflows for real-time automation.</p><h4>Training and Deployment</h4><p>Snowflake provides scalable resources for training and deploying Agentic AI models:</p><ul><li>Dynamic Compute Scaling: Optimizes resources for training and inference tasks.</li><li>Model Deployment: Enables SQL-based inference or external calls for real-time action-taking.</li></ul><h4>Optimizing Workflows</h4><p>Optimizing Agentic AI workflows in Snowflake involves:</p><ul><li>Batch vs. Real-Time Inference: Using Snowpipe for live inputs or scheduled queries for batch processing.</li><li>Query Performance: Leveraging clustering keys, materialized views, and caching to speed up operations.</li></ul><p>These optimizations ensure efficient and reliable AI operations.</p><h4>Security and Governance</h4><p>Ensuring the security of Agentic AI workflows is paramount. Snowflake offers:</p><ul><li>Data Privacy: Compliance with GDPR, CCPA, and HIPAA regulations.</li><li><a href="https://docs.snowflake.com/en/user-guide/security-access-control-overview">Access Control</a>: Role-based permissions to safeguard data and intellectual property.</li><li>Monitoring: Audit logs for transparency and accountability.</li></ul><h4>Real-World Use Cases</h4><p>Agentic AI workflows powered by Snowflake are transforming industries:</p><ul><li>Retail: Autonomous recommendation systems to enhance customer engagement.</li><li>Media: Automated content generation at scale.</li><li>Finance: Real-time fraud detection and anomaly analysis.</li></ul><h3>Future Trends and Innovations</h3><p>Agentic AI is rapidly evolving, with trends like multimodal models and energy-efficient systems shaping its future. Snowflake is positioned to support these advancements, offering enhanced computing capabilities and seamless integrations with generative AI frameworks.</p><h3>In short….</h3><p>Snowflake provides a robust platform for building and deploying Agentic AI workflows. Its advanced architecture, seamless framework integrations, and strong security measures make it an ideal choice for enterprises looking to harness the power of autonomous AI systems. By leveraging Snowflake, organizations can unlock transformative possibilities, from real-time insights to scalable decision-making operations.</p><h3>Let’s build your first Agentic AI Workflows with Snowflake</h3><p>This tutorial will provide you with a step-by-step guide to building Agentic AI workflows using Snowflake.</p><p><strong>Quickstart Guides:</strong></p><ol><li>AI Agents on Snowflake with <a href="https://quickstarts.snowflake.com/guide/create_ai_agents_on_snowflake_with_lang_ai/index.html#0">Lang.ai</a></li><li>Build Agentic Workflows with <a href="https://quickstarts.snowflake.com/guide/build-agentic-workflows-with-huggingface-smolagents-in-snowflake/index.html#0">Hugging Face Smolagents</a> in Snowflake</li></ol><h3>Step 1: Setting Up Your Snowflake Environment</h3><p>Before building your workflow, the first step is to prepare your Snowflake environment.</p><h4>1.1 Create a Snowflake Account</h4><ul><li>Visit Snowflake’s official site.</li><li>Sign up for an account or log into your existing account.</li></ul><h4>1.2 Configure Your Data Warehouse</h4><ul><li>Set up a virtual warehouse to enable computing tasks. Choose the appropriate size based on your workload.</li><li>Partition and structure your data for efficient processing.</li></ul><h4>1.3 Connect Your Data Sources</h4><ul><li>Integrate data sources like databases, APIs, or third-party platforms into Snowflake.</li><li>Use Snowflake connectors for seamless data ingestion.</li></ul><h3>Step 2: Preparing Data for Agentic AI</h3><p>Data preparation is essential for building effective AI workflows. Snowflake provides robust tools to clean, transform, and structure your data.</p><h4>2.1 Data Cleaning</h4><ul><li>Identify and remove duplicates, incomplete records, and outliers.</li><li>Use SQL commands within Snowflake to automate cleansing processes.</li></ul><h4>2.2 Data Transformation</h4><ul><li>Apply normalization techniques to ensure consistency across datasets.</li><li>Transform data into a compatible format for machine learning models (e.g., CSV, JSON).</li></ul><h4>2.3 Feature Engineering</h4><ul><li>Generate derived features that could enhance model performance.</li><li>Leverage Snowflake’s computing capabilities for real-time feature generation.</li></ul><h3>Step 3: Integrating Generative AI Frameworks</h3><p>Agentic AI workflows heavily rely on generative AI frameworks, such as GPT-like models and multimodal systems. Snowflake simplifies integration with these frameworks.</p><h4>3.1 Choose Your Framework</h4><ul><li>Select an AI framework that meets your requirements (e.g., OpenAI, Hugging Face).</li><li>Ensure compatibility with Snowflake’s environment.</li></ul><h4>3.2 Framework Integration</h4><ul><li>Use Snowflake’s external functions feature to connect with APIs of generative AI frameworks.</li><li>Develop connectors that allow seamless data exchange between Snowflake and the framework.</li></ul><h4>3.3 Model Deployment</h4><ul><li>Deploy models within Snowflake’s infrastructure or link them to external deployment services.</li><li>Ensure efficient computing for real-time predictions and insights.</li></ul><h3>Step 4: Workflow Implementation</h3><p>This step involves building the actual steps of your Agentic AI workflow.</p><h4>4.1 Design Logical Steps</h4><ul><li>Identify key stages — data input, processing, decision-making, and output generation.</li><li>Map each stage to Snowflake features (e.g., compute functions, data pipelines).</li></ul><h4>4.2 Develop SQL Queries</h4><ul><li>Write SQL queries to process data dynamically within your workflow.</li><li>Test and optimize queries for performance and accuracy.</li></ul><h4>4.3 Automate Workflow Execution</h4><ul><li>Use <a href="https://docs.snowflake.com/en/user-guide/tasks-overview">Snowflake’s task</a> feature to schedule and automate workflow stages.</li><li>Set triggers based on predefined conditions for workflow execution.</li></ul><h3>Step 5: Ensuring Security and Governance</h3><p>Security is crucial for protecting data and the integrity of Agentic AI workflows.</p><h4>5.1 Role-Based Access Control</h4><ul><li>Assign permissions based on roles to safeguard sensitive data.</li><li>Restrict access to workflow components for unauthorized users.</li></ul><h4>5.2 Data Privacy Compliance</h4><ul><li>Ensure compliance with regulations like GDPR and HIPAA.</li><li>Implement anonymization techniques for sensitive data.</li></ul><h4>5.3 Monitoring and Auditing</h4><ul><li>Enable audit logs to track workflow activities and ensure accountability.</li><li>Use <a href="https://docs.snowflake.com/en/sql-reference/account-usage">monitoring tools</a> to identify anomalies or potential risks.</li></ul><h3>Step 6: Optimizing Performance</h3><p>Performance optimization ensures the scalability and efficiency of your workflows.</p><h4>6.1 Scalability Planning</h4><ul><li>Scale your virtual warehouse resources based on workload demands.</li><li>Optimize memory and storage usage for large-scale AI applications.</li></ul><h4>6.2 Energy-Efficient Computing</h4><ul><li>Explore Snowflake’s support for energy-efficient solutions.</li><li>Reduce computational redundancy in workflow processes.</li></ul><h4>6.3 Real-Time Enhancements</h4><ul><li>Implement real-time fraud detection modules using Snowflake’s compute capabilities.</li><li>Leverage multimodal models for dynamic decision-making.</li></ul><h3>Step 7: Testing and Deployment</h3><p>After building the workflow, testing and deployment are critical.</p><h4>7.1 Workflow Validation</h4><ul><li>Run test cases to validate each stage of your workflow.</li><li>Ensure data accuracy and model reliability.</li></ul><h4>7.2 Deployment</h4><ul><li>Deploy the workflow in a production environment.</li><li>Provide user access and training to interact with the deployed system.</li></ul><h4>7.3 Continuous Iteration</h4><ul><li>Monitor workflow performance regularly to identify improvement areas.</li><li>Iterate and refine models for enhanced results.</li></ul><h3>Conclusion</h3><p>Building Agentic AI workflows with Snowflake empowers organizations to unlock transformative possibilities. By following these steps, you can develop secure, efficient, and scalable workflows that harness the potential of autonomous AI systems. Snowflake’s robust infrastructure ensures seamless integration, powerful computing, and enhanced security, making it the ideal platform for your AI projects.</p><h3>RESOURCES:</h3><h4>General Snowflake Learning:</h4><ul><li><a href="https://docs.snowflake.com/"><strong>Snowflake Documentation:</strong></a><strong> </strong>Provides comprehensive documentation on all aspects of the platform, including SQL queries, data loading, and advanced features.</li><li><a href="https://www.snowflake.com/en/resources/learn/training/"><strong>Snowflake Education</strong></a><strong>: </strong>Offers a variety of courses, from introductory to advanced, covering Snowflake fundamentals, data engineering, and more. They also offer Essentials Workshops, which are free and self-paced.</li><li><a href="https://www.datacamp.com/blog/how-to-learn-snowflake"><strong>DataCamp</strong></a><strong>:</strong> Provides guides and tutorials on learning Snowflake, including prerequisites like basic SQL knowledge.</li><li><a href="https://analyticsindiamag.com/ai-trends/top-7-free-resources-to-learn-snowflake/"><strong>Analytics India Magazine</strong></a><strong>:</strong> Features articles on using Snowflake for different applications, including data apps and AI.</li></ul><h4>Snowflake Agentic AI:</h4><ul><li><a href="https://www.xenonstack.com/blog/agentic-ai-snowflake"><strong>XenonStack</strong></a><strong>:</strong> Provides insights into how Snowflake can be used with agentic AI, including integration with AWS, Azure, and GCP platforms.</li><li><a href="https://quickstarts.snowflake.com/guide/create_ai_agents_on_snowflake_with_lang_ai/index.html"><strong>Snowflake Quickstarts</strong></a><strong>: </strong>Offers guides on building AI agents within Snowflake using tools like Lang.ai.</li><li><a href="https://www.snowflake.com/en/blog/ai-data-agents-snowflake-cortex/"><strong>Snowflake Blog (AI Data Agents)</strong></a>: Explores how Snowflake’s Cortex AI can be used to build and deploy agentic applications.</li><li><a href="https://medium.com/@prathamesh.nimkar/snowflake-agentic-workflows-160a6b83b688"><strong>Medium</strong></a><strong>: </strong>Features articles on Snowflake’s agentic workflows, including examples using Cortex Analyst and Search.</li><li><a href="https://www.snowflake.com/en/developers/solutions-center/building-an-ai-agent-for-healthcare-in-snowflake/"><strong>Snowflake Developers (AI Agent for Healthcare)</strong></a><strong>:</strong> Provides a case study on building an AI agent for healthcare using Snowflake.</li><li><a href="https://www.bigdatawire.com/2024/11/12/snowflake-dips-into-agentic-ai-with-snowflake-intelligence/"><strong>BigDATAwire</strong></a><strong>: </strong>Discusses Snowflake’s new generative AI capabilities, including Snowflake Intelligence for agentic AI applications.</li></ul><h3><strong>Let’s Connect</strong></h3><p>If you found this useful, consider:</p><p>Following me here on <a href="https://www.linkedin.com/in/puspanjalisarma/">Linkedin</a> , <a href="https://github.com/puspanjalis">Github</a> and <a href="https://medium.com/@puspanjalisarma23">Medium</a></p><blockquote>Thanks for reading — and may your queries always return the truth!</blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=057290fe8ff7" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Bridging the gap with Alternative Data]]></title>
            <link>https://medium.com/@puspanjalisarma23/bridging-the-gap-with-alternative-data-1bf8463d9e29?source=rss-657cc3b2578c------2</link>
            <guid isPermaLink="false">https://medium.com/p/1bf8463d9e29</guid>
            <dc:creator><![CDATA[Puspanjali Sarma]]></dc:creator>
            <pubDate>Mon, 27 Apr 2020 17:23:00 GMT</pubDate>
            <atom:updated>2025-06-22T16:48:23.955Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="Credit Score" src="https://cdn-images-1.medium.com/max/1024/1*BFkdsE_eO50CGMZD3ZBJwg.jpeg" /></figure><p>Today, more than<a href="https://economictimes.indiatimes.com/wealth/borrow/most-indians-not-aware-of-credit-score-heres-what-you-need-to-know/articleshow/55258213.cms?from=mdr"> 70% of Indian populations</a> do not have a credit score. This includes people who are <em>credit invisible</em> (those who completely lack a credit file) and the other segment with credit files may be too <em>thin </em>or <em>outdated</em>. Without a credit file, lenders are not able to conduct an adequate cost-benefit analysis for borrowers, so may choose to forgo lending to this population segment altogether.</p><p>But an “<em>unscorable</em>” individual or business is not necessarily a high credit risk rather they are an <em>unknown credit risk</em>. Many of these individuals pay their bills on time each month or have excellent financial performance records such as sales, revenue, IT returns, GST data, cash flow statements etc and could be great candidates for traditional credit. It is just that this segment of the population doesn’t have a credit history yet.</p><p>So, how do lenders gain more visibility to a consumer’s creditworthiness? <em>Alternative credit </em>data has surfaced as a solution providing a more holistic view of all consumers belonging to both segments of population i.e those on the traditional file and those who are credit invisible and emerging.</p><p>The traditional credit score has ruled the financial services space for decades but the landscape is changing as an immense amount of data generated worldwide within the last few years bringing an evolution in <em>credit scoring</em> which will include information from a variety of open and closed sources. This abundance of <em>“unused”</em> data will provide greater depth in understanding consumers’ risk profile and serve the historically under-served.</p><p><em>The emergence of alternative data as a key enabler in expanding credit scoring and financial decision-making process is distinctive.</em></p><p>Today’s consumers are utilizing different types of credit via various channels like fin-techs, short-term loans, cash-checking services etc. With an increased focus on alternative data, Credit Risk Management has come a long way from the days when banks used only one credit score for the underwriting process to assess loan applications. As the understanding of consumers’ payment behaviour deepens through alternative data, the <em>scorable population</em> has increased, equipping financial institutions to embark on an acquisition spree based on better assessments.</p><p>Today, lenders have the opportunity to gather data such as the size and reach of social networks, messages and data with sentiments, geo-location information, the volume and variety of available information with associated red flag indicators and have these data analyzed by <em>advanced artificial intelligence</em>, <em>machine learning</em> and <em>data analytics</em> to produce a rich, dynamic trust profile of the <em>consumer segment</em> which would previously be considered a thin or no-file borrower. In recent years, numerous alternative data providers have emerged on the credit scoring scene giving lenders the ability to create a comprehensive view of their potential borrowers.</p><p>Like traditional data, alternative data is susceptible to inaccuracies as well. Consumers may not be able to readily review and correct alternative data although the standards governing it are constantly changing and evolving to meet customer and regulatory expectations.</p><p>In today’s era of <em>digitization</em>, the most pressing issue with alternative data usage in credit scoring is the difficulty inherent in obtaining the data in the first place due to privacy and security rules, regulatory and compliance norms. Traditionally, credit scoring data is reported by lenders to the three major credit bureaus — Equifax, Experian, and TransUnion for creditworthiness assessments of borrowers.</p><p>In the space of alternative data credit scoring, the most common are social media data, psychometric data, telco data, and device data. Among all the alternative data sources, device data is the most reliable behavioural data, easiest to integrate, and the least invasive (data is anonymised).</p><p>Are Alternative Scoring Models Effective?</p><p>Beyond obtaining alternative data, the biggest question raised on whether alternative scoring models actually work to accurately predict consumer credit risk. While we have decades of data on the accuracy of various credit scoring models, alternative credit scoring has had significantly less research attention, making the validity of the models doubtful.</p><p>Long-term studies of how well alternative scoring models really evaluate consumer risk are required to determine their actual effectiveness — and not just for the scoring agencies. One of the biggest hurdles alternative scoring advocates face is simply getting lenders to use alternative scoring models as part of their decision-making process.</p><p>While we talk about adaption of alternative data for decision-making, debates are turning up seeking insights into both the benefits and risks of so-called “alternative data” and the techniques used to compile and analyze it. With growing speculations on alternative data usage, an existing gap in the availability of funds to various sectors is observed which remained largely under-served by Financial Institutions.</p><p>According to an <a href="http://economictimes.indiatimes.com/articleshow/67298802.cms?from=mdr&amp;utm_source=contentofinterest&amp;utm_medium=text&amp;utm_campaign=cppst">article </a>published by Economic Times, MSME (Micro, Small, and Medium Enterprises) sector comprising of 60 million businesses in India, has a huge unmet demand for credit estimated at approximately $200 billion. This funding challenges for SMEs arises due to choosing a good CIBIL score, detailed audited financial statements or collateral only to assess their creditworthiness by banks and financial institutions.</p><p>For India’s growing economy, a renewed approach with alternative credit scoring based on digital data to assess borrowers’ creditworthiness and associated risks for credit underwriting will play a pivotal role in <em>bridging the gap </em>in a new technology-driven financial decision making landscape.</p><h3>Let’s Connect</h3><p>If you found this useful, consider:</p><ul><li>Following me here on <a href="https://www.linkedin.com/in/puspanjalisarma/">Linkedin</a> , <a href="https://github.com/puspanjalis">Github</a> and <a href="https://medium.com/@puspanjalisarma23">Medium</a>!</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1bf8463d9e29" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>