<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Kiran Jilla on Medium]]></title>
        <description><![CDATA[Stories by Kiran Jilla on Medium]]></description>
        <link>https://medium.com/@kiran.jilla?source=rss-ec2ba42995a3------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*IFgROqvzHGkvh3fO</url>
            <title>Stories by Kiran Jilla on Medium</title>
            <link>https://medium.com/@kiran.jilla?source=rss-ec2ba42995a3------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sun, 22 Nov 2026 10:09:58 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@kiran.jilla/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Building an Efficient AI Software Factory with Snowflake — Part 3: Optimization]]></title>
            <link>https://medium.com/@kiran.jilla/building-an-efficient-ai-software-factory-with-snowflake-part-3-optimization-84ee8572cecf?source=rss-ec2ba42995a3------2</link>
            <guid isPermaLink="false">https://medium.com/p/84ee8572cecf</guid>
            <dc:creator><![CDATA[Kiran Jilla]]></dc:creator>
            <pubDate>Thu, 10 Sep 2026 16:12:20 GMT</pubDate>
            <atom:updated>2026-09-10T16:12:20.334Z</atom:updated>
            <content:encoded><![CDATA[<p><em>Tags: Snowflake, AI Agents, Cortex, MCP, Cost Optimization, Software Factory</em></p><p><em>This is Part 3 of a 3-part series. </em><a href="https://medium.com/@kiran.jilla/building-an-efficient-ai-software-factory-with-snowflake-part-1-overview-70de14db6d96"><em>Part 1: Overview</em></a><em> | </em><a href="https://medium.com/@kiran.jilla/building-an-efficient-ai-software-factory-with-snowflake-part-2-measurement-571b7e63af3d"><em>Part 2: Measurement</em></a></p><p>In <a href="https://medium.com/@kiran.jilla/building-an-efficient-ai-software-factory-with-snowflake-part-1-overview-70de14db6d96">Part 1</a>, we defined the cost equation. In <a href="https://medium.com/@kiran.jilla/building-an-efficient-ai-software-factory-with-snowflake-part-2-measurement-571b7e63af3d">Part 2</a>, we measured every term with SQL. Now we optimize.</p><p>Each term in the cost equation maps to specific Snowflake optimization levers:</p><p><strong>Price / Token</strong></p><ul><li>Dynamic Model Routing (advisor pattern)</li><li>Model catalog selection</li><li>Default model policies</li></ul><p><strong>Tokens / Request</strong></p><ul><li>Semantic Views for typed access</li><li>AI_FILTER to reduce rows</li><li>AI_EXTRACT to reduce payload</li><li>Prompt caching</li></ul><p><strong>Requests / Turn</strong></p><ul><li>Business Ontology for grounded context</li><li>Cortex Agents tool selection</li></ul><p><strong>Visibility &amp; Education</strong></p><ul><li>AI Cost Dashboard</li><li>Resource Budgets</li><li>Per-user limits</li><li>Slack alerts</li></ul><h3>Optimizing Price / Token</h3><h3>Dynamic Model Routing: The Advisor Pattern</h3><p>Announced August 18, 2026, Dynamic Model Routing lives inside the <strong>Cortex AI Gateway</strong>, Snowflake’s unified foundation for governing agent connections. The gateway evaluates each task against admin-defined policies and real-world cost/performance data, then selects the most affordable model that can confidently complete it.</p><p>The mechanism is the <strong>advisor pattern</strong>: a small, cheap model attempts the task first. If it cannot complete it confidently, the request escalates to a larger model as a fallback. The frontier model is only invoked when needed.</p><pre>Agent request → Cortex AI Gateway → Router evaluates complexity<br>                                       │<br>                           ┌───────────┴───────────┐<br>                           ▼                       ▼<br>                    Small model (cheap)      Large model (expensive)<br>                    tries the task            used only as fallback<br>                           │<br>                    Can complete?<br>                     /        \<br>                   Yes          No<br>                    │            │<br>              Return result   Escalate to large model<br>              (low cost)      (higher cost, but correct)</pre><p>You define the tradeoff: which models are approved, the cost threshold, and the quality floor. Snowflake handles the routing, telemetry, and fallback logic so you get Pareto-optimal model selection without building a custom harness.</p><h3>Model Catalog</h3><p>Snowflake hosts frontier and open models from multiple providers under one roof, all accessible from Cortex AI:</p><ul><li><strong>Anthropic</strong>: Claude Sonnet 4–6, Claude Opus 5, Claude Opus 4–8</li><li><strong>OpenAI</strong>: GPT-5.2</li><li><strong>Google</strong>: Gemini</li><li><strong>Meta</strong>: Llama 3.3–70b, Llama 3–8b</li><li><strong>Mistral AI</strong>: Mistral 7b</li><li><strong>SpaceXAI</strong>: Grok</li><li><strong>DeepSeek</strong>: DeepSeek-V4-Flash 0731</li><li><strong>Z.ai</strong>: GLM-5.3</li></ul><p>All routed through the same Cortex AI Gateway with the same governance, data residency, and agent identity controls. You pick the model per task; the gateway enforces your policies.</p><pre>-- Specify model per Cortex Agent based on task complexity<br>CREATE OR REPLACE CORTEX AGENT simple_qa_agent<br>  MODEL = &#39;mistral-7b&#39;  -- cheap, fast for simple classification<br>  TOOLS = (CORTEX_SEARCH SERVICE my_search);</pre><pre>CREATE OR REPLACE CORTEX AGENT deep_research_agent<br>  MODEL = &#39;claude-opus-5&#39;  -- frontier for complex reasoning<br>  TOOLS = (<br>    CORTEX_SEARCH SERVICE my_search,<br>    CORTEX_ANALYST SEMANTIC_VIEW my_semantic_view<br>  );</pre><h3>Optimizing Tokens / Request</h3><p>Every turn re-sends the full conversation history, tool schemas, and prior results. Anything that reduces the per-request payload compounds across the session.</p><h3>Semantic Views: Typed, Structured Access</h3><p>Instead of an agent issuing raw SQL and retrieving untyped results (which bloats the context), Cortex Analyst uses <strong>Semantic Views</strong> to generate precise SQL from natural language. The agent gets typed, minimal results, not a raw dump.</p><pre>-- A semantic view defines metrics, dimensions, and relationships<br>-- so Cortex Analyst generates precise SQL, not exploratory queries<br>CREATE OR REPLACE SEMANTIC VIEW sales_semantic_view AS<br>SELECT<br>    revenue.total_amount AS revenue,<br>    orders.order_count,<br>    customers.region,<br>    products.category,<br>    time.date<br>FROM fact_revenue revenue<br>JOIN dim_orders orders ON revenue.order_id = orders.order_id<br>JOIN dim_customers customers ON orders.customer_id = customers.customer_id<br>-- ... relationships and verified queries here<br>;</pre><p>The agent’s context stays small: one NL question in, one precise result set out.</p><h3>Prompt Caching Economics</h3><p>Every turn re-sends the full conversation history, tool schemas, and system prompt. Prompt caching avoids paying the full cost of re-processing that prefix on every turn. Subsequent reads cost a fraction of the standard input token rate.</p><p>Snowflake’s Cortex AI Gateway handles prompt caching automatically for supported models. The economics are straightforward: the optimal cache TTL depends on the gap between turns in your workload.</p><ul><li><strong>Interactive agent (human in loop)</strong>: Typical gap 30s to 5min idle. Use short TTL (5 min). Engineer reads the response before next prompt; cache stays warm.</li><li><strong>Interactive agent with long pauses</strong>: Typical gap 5 to 30min idle. Use long TTL (1 hour). Engineer steps away; 5-min cache expires and forces full-price rebuild.</li><li><strong>Autonomous managed agent</strong>: Typical gap under 5s between turns. Use short TTL (5 min). Agent fires turns rapidly; no benefit from 1-hour premium.</li><li><strong>Batch processing</strong>: Typical gap hours between sessions. Use no cache or 1-hour. Workload too sparse to benefit; don’t pay the write premium.</li></ul><p>If your average session has more than 5 turns and gaps longer than 5 minutes, long-TTL caching saves more than it costs. For rapid-fire autonomous agents, short TTL is cheaper because you never pay the higher write premium.</p><h3>AI_FILTER: Reduce Rows Before They Reach the Agent</h3><p>Filter unstructured text at the SQL layer before the agent ever sees it. Keep intermediate work out of the model’s context window:</p><pre>-- Only pass relevant rows to the agent<br>SELECT ticket_id, ticket_text<br>FROM support_tickets<br>WHERE AI_FILTER(<br>    ticket_text,<br>    &#39;Is this ticket about a billing dispute or refund request?&#39;<br>);</pre><h3>AI_EXTRACT: Reduce Payload to Structured Fields</h3><p>Instead of sending full documents to the agent, extract only the fields you need:</p><pre>-- Extract structured data from contracts; agent sees only the fields<br>SELECT<br>    contract_id,<br>    AI_EXTRACT(<br>        contract_text,<br>        {<br>            &#39;counterparty&#39;: &#39;string&#39;,<br>            &#39;effective_date&#39;: &#39;date&#39;,<br>            &#39;total_value&#39;: &#39;number&#39;,<br>            &#39;termination_clause&#39;: &#39;string&#39;<br>        }<br>    ) AS extracted_fields<br>FROM contracts<br>WHERE contract_status = &#39;ACTIVE&#39;;</pre><h3>Managing MCP Tool Schema Bloat</h3><p>This is one of the highest-impact optimizations for agent cost. Standard MCP loads all tool schemas directly into every session, regardless of whether the agent will ever invoke them. With 100+ tools installed, that pre-loading adds 50K to 70K tokens of schema overhead to the initial prompt, re-sent on every context turn.</p><p>Snowflake’s MCP Server has a hard limit of 50 tools per server object. That’s a guardrail, not a solution. At 50 tools the schema overhead is still meaningful, and most enterprises need far more than 50 tools across their data estate.</p><p>The pattern that works:</p><p><strong>1. Split tools across multiple MCP Server objects by domain.</strong> Don’t put sales tools and HR tools in the same server. The agent only loads the schema for the server it connects to.</p><pre>-- Sales domain server<br>CREATE OR REPLACE MCP SERVER sales_mcp_server FROM SPECIFICATION<br>$$<br>  tools:<br>  - name: &quot;revenue_query&quot; ...<br>  - name: &quot;pipeline_search&quot; ...<br>  - name: &quot;forecast_analyst&quot; ...<br>$$;</pre><pre>-- HR domain server (separate schema, separate RBAC)<br>CREATE OR REPLACE MCP SERVER hr_mcp_server FROM SPECIFICATION<br>$$<br>  tools:<br>  - name: &quot;headcount_analyst&quot; ...<br>  - name: &quot;policy_search&quot; ...<br>$$;</pre><p><strong>2. Use Cortex Agents tool selection to avoid loading every tool.</strong> When an agent runs inside Snowflake (Layer 2), it doesn’t need all tools loaded into context at once. Cortex Agents handles tool selection as part of orchestration. The agent’s LLM sees the task, decides which tool is relevant, and only that tool’s schema is invoked.</p><p><strong>3. Use </strong><strong>SYSTEM_EXECUTE_SQL for ad-hoc queries instead of predefining every query as a tool.</strong> If you have hundreds of possible analytical queries, don&#39;t create a tool for each. Create one SQL execution tool and let the agent generate the SQL via Cortex Analyst:</p><pre>CREATE OR REPLACE MCP SERVER minimal_mcp_server FROM SPECIFICATION<br>$$<br>  tools:<br>  - name: &quot;sql_execute&quot;<br>    type: &quot;SYSTEM_EXECUTE_SQL&quot;<br>    description: &quot;Execute SQL against any table in the analytics schema&quot;<br>  - name: &quot;schema_search&quot;<br>    type: &quot;CORTEX_SEARCH_SERVICE_QUERY&quot;<br>    identifier: &quot;ANALYTICS.META.SCHEMA_SEARCH&quot;<br>    description: &quot;Search the data dictionary to find the right table&quot;<br>$$;</pre><p>Two tools instead of 50. The agent uses the schema search to find the right table, then uses SQL execution to query it. Schema overhead drops from ~35K tokens to ~3K.</p><p><strong>4. Route external SaaS tools through a single gateway.</strong> If your agents need tools from Jira, Slack, GitHub, or other SaaS platforms, route them through the Cortex AI Gateway rather than loading each vendor’s MCP server schema into the agent context. The gateway aggregates the tools behind one entry point with centralized auth.</p><h3>Optimizing Requests / Turn</h3><p>An ungrounded agent fails slowly rather than cheaply, repeatedly sending an expanding context to search one more location. Providing richer context upfront is the single most powerful lever to reduce search overhead.</p><h3>Business Ontology: Snowflake’s Context Graph</h3><p>Snowflake’s <strong>Business Ontology</strong> is the platform’s native context graph, a unified semantic layer mapping your tables, metrics, dimensions, relationships, and verified queries that any Cortex Agent can query. It’s the equivalent of building a 24M-node, 80M-edge knowledge graph across 30+ internal systems, but done natively on your existing Snowflake data.</p><p>Instead of an agent spending turns exploring schema, the ontology gives it grounded context on turn one: which tables exist, how they relate, which metrics are certified, which queries are verified.</p><pre>Without ontology:  Agent explores schema → 8 turns → finds the right table → queries it<br>With ontology:     Agent queries ontology → 1 turn → gets certified table + query → runs it</pre><p>This directly reduces requests per turn. The agent reaches the answer faster because it starts with structured knowledge of your data estate.</p><h3>A Concrete Before/After</h3><p>Consider a real question an analyst might ask a Cortex Agent: <em>“Which 5 product categories drove the biggest revenue increase in the Pacific region last quarter, and how does that compare to the same quarter last year?”</em></p><p><strong>Without Business Ontology</strong>, the agent has to discover the schema first:</p><ul><li>Turn 1: Lists tables in the database -&gt; 3,200 tokens</li><li>Turn 2: Describes fact_sales to find revenue columns -&gt; 4,800 tokens</li><li>Turn 3: Describes dim_products to find category -&gt; 3,500 tokens</li><li>Turn 4: Describes dim_geography to find Pacific region -&gt; 3,100 tokens</li><li>Turn 5: Writes and runs an exploratory query (wrong join) -&gt; 6,200 tokens</li><li>Turn 6: Corrects the join after seeing an error -&gt; 5,400 tokens</li><li>Turn 7: Runs the corrected current-quarter query -&gt; 2,800 tokens</li><li>Turn 8: Runs the year-ago comparison query -&gt; 2,800 tokens</li><li><strong>Total: 8 turns, ~31,800 tokens</strong></li></ul><p><strong>With Business Ontology</strong>, the agent starts with certified context:</p><ul><li>Turn 1: Queries ontology for “revenue by category, Pacific, QoQ” -&gt; gets certified table + verified query -&gt; 4,200 tokens</li><li>Turn 2: Runs the verified query (already correct) -&gt; 2,800 tokens</li><li><strong>Total: 2 turns, ~7,000 tokens</strong></li></ul><p>Same answer. 4x fewer turns. ~4.5x fewer tokens. The ontology pays for itself the first time an agent avoids a wrong join.</p><h3>Code-Mode Equivalent: Batching Tool Calls in Snowpark</h3><p>A proven optimization pattern batches multiple chatty tool calls into a single Python script, keeping intermediate polling out of the model’s active context. The savings are dramatic: 55% to ~100% depending on result size.</p><p>The Snowflake equivalent: instead of an agent issuing 5 separate SQL queries in 5 turns (each returning intermediate results into the context window), write a single Snowpark Python stored procedure that does all the work and returns only the final summary.</p><p><strong>Without batching (agent issues queries one at a time):</strong></p><pre># Turn 1: Get total revenue<br>result1 = agent.run(&quot;SELECT SUM(amount) FROM sales WHERE quarter = &#39;Q2&#39;&quot;)<br># Turn 2: Get revenue by category<br>result2 = agent.run(&quot;SELECT category, SUM(amount) FROM sales GROUP BY category&quot;)<br># Turn 3: Get top 5 categories<br>result3 = agent.run(&quot;SELECT category FROM (...) ORDER BY revenue DESC LIMIT 5&quot;)<br># Turn 4: Get year-over-year comparison<br>result4 = agent.run(&quot;SELECT ... compare to last year&quot;)<br># Turn 5: Format the final answer<br># Each intermediate result lands in the context window - 5 round trips, 5 payloads</pre><p><strong>With batching (single Snowpark stored procedure):</strong></p><pre>import snowpark<br>from snowflake.snowpark.functions import col, sum as sum_</pre><pre>def top_categories_yoy(session):<br>    # All work happens server-side; only the final summary returns<br>    sales = session.table(&#39;sales&#39;)</pre><pre>    current_q = (sales<br>        .filter(col(&#39;quarter&#39;) == &#39;Q2-2026&#39;)<br>        .groupBy(&#39;category&#39;)<br>        .agg(sum_(&#39;amount&#39;).alias(&#39;current_revenue&#39;)))</pre><pre>    prior_q = (sales<br>        .filter(col(&#39;quarter&#39;) == &#39;Q2-2025&#39;)<br>        .groupBy(&#39;category&#39;)<br>        .agg(sum_(&#39;amount&#39;).alias(&#39;prior_revenue&#39;)))</pre><pre>    result = (current_q<br>        .join(prior_q, &#39;category&#39;)<br>        .with_column(&#39;growth&#39;, col(&#39;current_revenue&#39;) - col(&#39;prior_revenue&#39;))<br>        .order_by(col(&#39;growth&#39;).desc())<br>        .limit(5))</pre><pre>    # Only the 5-row summary returns to the agent - not 5 separate payloads<br>    return result.collect()</pre><pre># Register as a tool the agent can call in ONE turn<br>session.sproc.register(<br>    &quot;top_categories_yoy&quot;,<br>    top_categories_yoy,<br>    return_type=...,<br>    input_types=[...],<br>    replace=True<br>)</pre><p>One tool call, one turn, one small result set. The agent never sees the intermediate queries, the full table scans, or the polling overhead. Push the loop to the server, return only the summary.</p><p>Register the stored procedure as a GENERIC tool in your MCP Server so any external agent can call it:</p><pre>CREATE OR REPLACE MCP SERVER analytics_mcp_server FROM SPECIFICATION<br>$$<br>  tools:<br>  - name: &quot;top_categories_yoy&quot;<br>    type: &quot;GENERIC&quot;<br>    identifier: &quot;ANALYTICS.PUBLIC.TOP_CATEGORIES_YOY&quot;<br>    description: &quot;Get top 5 product categories by YoY revenue growth&quot;<br>$$;</pre><h3>Visibility &amp; Education</h3><h3>The AI Cost Dashboard</h3><p>Snowflake ships a ready-built <strong>Streamlit dashboard</strong> that aggregates usage from all Cortex AI service views into local tables, computes dollar costs using a service-tier and region-based pricing model, and visualizes it with an integrated Cortex Analyst chatbot.</p><pre># Deploy the AI Cost Dashboard<br>git clone https://github.com/Snowflake-Labs/sfguide-ai-cost-dashboard.git<br>cd sfguide-ai-cost-dashboard<br># In Cortex Code: &quot;Use the skill to deploy the ai cost dashboard&quot;</pre><p>The dashboard surfaces:</p><ul><li>Total spend by service type (Agents, Functions, Search, CoWork, Code)</li><li>Per-user and per-agent attribution</li><li>Model cost share and Pareto frontier analysis</li><li>Trend lines for cost-per-request and cost-per-session</li></ul><h3>Session Anti-Patterns the Dashboard Catches</h3><p>The dashboard’s value is not in aggregate metrics. It’s in flagging specific waste patterns in individual sessions. Here are the anti-patterns to look for, with the SQL to detect each:</p><p><strong>1. Overpowered model for a simple task.</strong> A multi-turn classification session routed to Claude Opus when Mistral 7B would suffice.</p><pre>-- Find sessions where a frontier model was used for low-credit tasks<br>SELECT user_name, model_name, request_count, total_credits<br>FROM (<br>    SELECT<br>        u.name AS user_name,<br>        h.model_name,<br>        COUNT(*) AS request_count,<br>        SUM(h.credits) AS total_credits<br>    FROM SNOWFLAKE.ACCOUNT_USAGE.CORTEX_AI_FUNCTIONS_USAGE_HISTORY h<br>    JOIN SNOWFLAKE.ACCOUNT_USAGE.USERS u ON h.user_id = u.user_id<br>    WHERE h.start_time &gt;= DATEADD(&#39;day&#39;, -7, CURRENT_TIMESTAMP())<br>      AND h.model_name IN (&#39;claude-opus-5&#39;, &#39;claude-opus-4-8&#39;)<br>    GROUP BY u.name, h.model_name<br>)<br>ORDER BY total_credits DESC;</pre><p><strong>2. Context window bloat from large tool responses.</strong> A 40KB MCP tool response persists in context and gets re-billed on every subsequent turn.</p><pre>-- Find functions with high per-request credit consumption (large responses bloat context)<br>SELECT<br>    function_name,<br>    model_name,<br>    request_count,<br>    total_credits,<br>    credits_per_request<br>FROM (<br>    SELECT<br>        function_name,<br>        model_name,<br>        COUNT(*) AS request_count,<br>        SUM(credits) AS total_credits,<br>        SUM(credits) / COUNT(*) AS credits_per_request<br>    FROM SNOWFLAKE.ACCOUNT_USAGE.CORTEX_AI_FUNCTIONS_USAGE_HISTORY<br>    WHERE start_time &gt;= DATEADD(&#39;day&#39;, -7, CURRENT_TIMESTAMP())<br>    GROUP BY function_name, model_name<br>)<br>WHERE credits_per_request &gt; 0.01<br>ORDER BY credits_per_request DESC;</pre><p><strong>3. Cache expiration forcing full-price rebuilds.</strong> Sessions resumed after the prompt cache expired, paying full input-token cost to reprocess the prefix.</p><p><strong>4. Repeated failed tool calls.</strong> An agent calling the same tool 3+ times in a session, each time getting an error, each time re-sending the full context.</p><pre>-- Find users with high call counts on the same function (possible retries/errors)<br>SELECT<br>    u.name AS user_name,<br>    h.function_name,<br>    COUNT(*) AS call_count,<br>    SUM(h.credits) AS total_credits<br>FROM SNOWFLAKE.ACCOUNT_USAGE.CORTEX_AI_FUNCTIONS_USAGE_HISTORY h<br>JOIN SNOWFLAKE.ACCOUNT_USAGE.USERS u ON h.user_id = u.user_id<br>WHERE h.start_time &gt;= DATEADD(&#39;day&#39;, -7, CURRENT_TIMESTAMP())<br>GROUP BY u.name, h.function_name<br>HAVING COUNT(*) &gt; 50<br>ORDER BY call_count DESC;</pre><p><strong>5. Exploratory query storms.</strong> An agent issuing many small schema-discovery queries instead of one ontology-grounded query, the exact pattern Business Ontology exists to prevent.</p><pre>-- Find users with high AI function call counts (possible schema exploration pattern)<br>SELECT<br>    u.name AS user_name,<br>    COUNT(*) AS function_calls,<br>    SUM(h.credits) AS total_credits<br>FROM SNOWFLAKE.ACCOUNT_USAGE.CORTEX_AI_FUNCTIONS_USAGE_HISTORY h<br>JOIN SNOWFLAKE.ACCOUNT_USAGE.USERS u ON h.user_id = u.user_id<br>WHERE h.start_time &gt;= DATEADD(&#39;day&#39;, -1, CURRENT_TIMESTAMP())<br>GROUP BY u.name<br>HAVING COUNT(*) &gt; 20<br>ORDER BY function_calls DESC;</pre><p>Each of these patterns has a targeted remediation: route simple tasks to a cheaper model, use AI_EXTRACT to shrink tool responses, set a longer cache TTL, fix the failing tool&#39;s schema, or point the agent at a Semantic View instead of raw SQL.</p><h3>Resource Budgets for Cortex Agents</h3><p>GA April 10, 2026. Tag Cortex Agent objects and set monthly credit limits. Spend is evaluated against the budget periodically; you get alerts when spending approaches or exceeds thresholds.</p><pre>-- Create a budget for a specific Cortex Agent<br>CREATE BUDGET my_agent_budget<br>  MONTHLY_CREDIT_LIMIT = 500;  -- 500 AI Credits / month</pre><pre>-- Tag the Cortex Agent so its spend rolls up to the budget<br>ALTER CORTEX AGENT my_analyst_agent<br>  SET TAG budget = &#39;my_agent_budget&#39;;</pre><pre>-- Check current spend against the budget<br>CALL my_agent_budget!GET_SERVICE_TYPE_USAGE_V2(&#39;2026-06&#39;, &#39;2026-06&#39;);<br>-- Result:<br>-- SERVICE_TYPE   | NAME              | CREDITS_USED<br>-- CORTEX_AGENT   | MY_ANALYST_AGENT  | 0.3456<br>-- AI_SERVICES    | -                 | 0.1234</pre><p>This is the equivalent of spend tiers applied to specific agent objects, not just users.</p><h3>Per-User Spending Limits</h3><p>Automatically revoke or restore AI function access based on configurable monthly credit limits:</p><pre>-- Access control table: tracks who has access and their monthly budget<br>CREATE TABLE IF NOT EXISTS ai_function_access (<br>    user_name STRING,<br>    monthly_credit_limit NUMBER,<br>    current_month_credits NUMBER DEFAULT 0,<br>    access_revoked BOOLEAN DEFAULT FALSE,<br>    last_revoked TIMESTAMP<br>);</pre><pre>-- Hourly task: check spend and revoke if over limit<br>CREATE OR REPLACE TASK enforce_user_limits<br>  WAREHOUSE = COMPUTE_WH<br>  SCHEDULE = &#39;USING CRON 0 * * * * UTC&#39;  -- every hour<br>AS<br>  MERGE INTO ai_function_access AS target<br>  USING (<br>    SELECT<br>      u.name AS user_name,<br>      SUM(h.credits_used) AS current_month_credits<br>    FROM SNOWFLAKE.ACCOUNT_USAGE.CORTEX_AI_FUNCTIONS_USAGE_HISTORY h<br>    JOIN SNOWFLAKE.ACCOUNT_USAGE.USERS u ON h.user_id = u.user_id<br>    WHERE h.start_time &gt;= DATE_TRUNC(&#39;month&#39;, CURRENT_DATE())<br>    GROUP BY u.name<br>  ) AS source<br>  ON target.user_name = source.user_name<br>  WHEN MATCHED AND source.current_month_credits &gt; target.monthly_credit_limit THEN<br>    UPDATE SET<br>      current_month_credits = source.current_month_credits,<br>      access_revoked = TRUE,<br>      last_revoked = CURRENT_TIMESTAMP()<br>  WHEN MATCHED THEN<br>    UPDATE SET current_month_credits = source.current_month_credits;</pre><h3>Slack Alerts at Spend Tiers</h3><p>Rather than hard caps that block work, implement tiered nudges that give engineers time to plan:</p><ul><li><strong>50%</strong>: User hits half their monthly limit. Slack nudge: “You’re at 50% of your AI budget”</li><li><strong>80%</strong>: User hits 80%. Slack nudge + manager CC’d: “Approaching limit”</li><li><strong>100%</strong>: User exceeds limit. Auto-revoke AI function access + email to user and manager</li><li><strong>Approval</strong>: User requests tier upgrade. Manager sign-off via a quick approval flow</li></ul><pre>-- Alert when a user hits 80% of their monthly limit<br>CREATE OR REPLACE TASK alert_user_spend_tier<br>  WAREHOUSE = COMPUTE_WH<br>  SCHEDULE = &#39;USING CRON 0 9 * * * UTC&#39;  -- daily at 9am UTC<br>AS<br>  SELECT SYSTEM$SEND_EMAIL(<br>    &#39;ops-team@company.com&#39;,<br>    &#39;AI spend alert: user approaching monthly limit&#39;,<br>    &#39;User &#39; || user_name || &#39; has used &#39; ||<br>      ROUND(current_month_credits, 1) || &#39; of &#39; ||<br>      monthly_credit_limit || &#39; credits this month.&#39;<br>  )<br>  FROM ai_function_access<br>  WHERE current_month_credits &gt;= monthly_credit_limit * 0.8<br>    AND access_revoked = FALSE;</pre><p>The goal is visibility and feedback loops, not strict caps. Engineers evaluate task ROI independently while mitigating runaway expenses.</p><h3>Deep Dive: MCP as the Unified Gateway</h3><p>Organizations running large agent fleets often route hundreds or thousands of MCP servers through a single gateway to centralize authentication and policy enforcement. Snowflake gives you this pattern natively.</p><p>The <strong>Cortex AI Gateway</strong> (announced July 2026 at Black Hat) is Snowflake’s unified entry point for governing agent connections. Combined with the <strong>Managed MCP Server</strong>, it replaces the need to manage dozens of external MCP servers:</p><pre>AI Agents (any framework)<br>       │<br>       │  MCP tool calls<br>       ▼<br>  ┌─────────────────────────────────┐<br>  │   Snowflake Cortex AI Gateway    │<br>  │  · Governed model list           │<br>  │  · Data residency rules          │<br>  │  · Agent identity controls       │<br>  │  · Data exfiltration prevention  │<br>  │  · Dynamic model routing         │<br>  └────────────────┬────────────────┘<br>                   │<br>                   ▼<br>  ┌─────────────────────────────────┐<br>  │   Snowflake Data + Tools         │<br>  │  · Cortex Search                 │<br>  │  · Cortex Analyst                │<br>  │  · Cortex Agents                 │<br>  │  · Custom UDFs / stored procs     │<br>  │  · Your tables &amp; semantic views  │<br>  └─────────────────────────────────┘</pre><h3>Bring Your Own MCP Servers via SPCS</h3><p>For tools that don’t fit inside Snowflake’s native MCP Server, deploy custom MCP servers on <strong>Snowpark Container Services (SPCS)</strong> and integrate them with Snowflake Intelligence or Cortex Agents:</p><pre>-- Create a Cortex Agent that uses both native tools AND a custom MCP server<br>CREATE OR REPLACE CORTEX AGENT hybrid_agent<br>  MODEL = &#39;claude-sonnet-4-6&#39;<br>  TOOLS = (<br>    CORTEX_SEARCH SERVICE my_search,<br>    CORTEX_ANALYST SEMANTIC_VIEW my_semantic_view,<br>    MCP SERVER my_custom_mcp_server  -- running on SPCS<br>  );</pre><p>This is the Snowflake-native path to the “1,000 MCP servers behind one gateway” pattern. Your custom tools run inside Snowflake’s security perimeter, governed by the same RBAC and network policies as your data.</p><h3>MCP Governance</h3><p>Announced at Black Hat 2026, Snowflake added enterprise-grade MCP governance:</p><ul><li><strong>Agent identity controls</strong>: which agents can invoke which tools</li><li><strong>Data exfiltration prevention</strong>: block tool calls that would send governed data to unapproved destinations</li><li><strong>Automated threat scanners</strong>: built into the Snowflake Trust Center</li></ul><p>These give you centralized policy enforcement over which agents can access which data, the governance layer that ad-hoc MCP deployments lack.</p><h3>What’s Next</h3><p>The software factory is not static. Current Snowflake initiatives in progress:</p><ul><li><strong>Intelligence Efficiency metric</strong>: Snowflake’s term for “how effectively companies turn compute, models, data, and context into business impact.” This is the north-star metric, cost per completed task, not just cost per token. It moves the conversation from “how much did we spend” to “how much value did each credit produce.”</li><li><strong>Evolving session analytics into real-time guidance</strong>: shifting from periodic batch detection of anti-patterns to continuous trace monitoring, delivering personalized efficiency recommendations directly to engineers.</li><li><strong>Expanding dynamic model routing</strong>: broader benchmark coverage across diverse programming languages, code repositories, and agent modalities. The router gets smarter as it sees more workload data.</li><li><strong>Continuous skill improvement</strong>: recording papercuts from agent skill executions and auto-generating skill updates from collected traces.</li><li><strong>Deepening Business Ontology integration</strong>: unlocking graph query capabilities across a wider selection of autonomous agents, so agents start with grounded context on turn one.</li></ul><h3>Conclusion</h3><p>Managing and curbing rising AI expenses is a tractable engineering challenge. By eliminating wasted token consumption and routing to the right model for each task, rather than relying solely on lower unit prices or downgrading tooling, you can scale usage while simultaneously reducing unit costs and maintaining output quality.</p><p>The core shift is moving from ad-hoc AI tooling to a governed, measured, optimized software factory. Snowflake provides the backbone: a unified data platform with native MCP governance, dynamic model routing, per-agent budgets, and usage-tracking views that let you decompose spend down to the user, model, and request.</p><p>The methodology: build a benchmark out of real work, measure every term in the cost equation, route to the Pareto-optimal model, and keep moving as the frontier shifts. On Snowflake, the data, the agents, the governance, and the cost tracking all live in one platform.</p><p><strong>Resources</strong></p><ul><li><a href="https://www.snowflake.com/en/blog/enterprise-ai-security-agentic-mcp-governance/">Snowflake Cortex AI Gateway announcement (Black Hat 2026)</a></li><li><a href="https://www.snowflake.com/en/blog/ai-intelligence-efficiency-dynamic-model-routing/">Dynamic Model Routing announcement</a></li><li><a href="https://docs.snowflake.com/en/user-guide/snowflake-cortex/ai-func-cost-management">Managing Cortex AI Function costs with Account Usage</a></li><li><a href="https://docs.snowflake.com/en/user-guide/snowflake-cortex/governance-and-availability/ai-cost-management-and-governance">AI cost management and governance</a></li><li><a href="https://www.snowflake.com/en/developers/guides/ai-cost-dashboard/">Cortex AI Cost Dashboard quickstart</a></li><li><a href="https://docs.snowflake.com/en/user-guide/snowflake-cortex/cortex-agents-mcp">Snowflake Managed MCP Server documentation</a></li><li><a href="https://docs.snowflake.com/en/user-guide/snowflake-cortex/aisql-regional-availability">Models and regional availability for Cortex AI Functions</a></li></ul><p><em>Have questions or want to share what you built? Drop a comment below.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=84ee8572cecf" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Building an Efficient AI Software Factory with Snowflake — Part 2: Measurement]]></title>
            <link>https://medium.com/@kiran.jilla/building-an-efficient-ai-software-factory-with-snowflake-part-2-measurement-571b7e63af3d?source=rss-ec2ba42995a3------2</link>
            <guid isPermaLink="false">https://medium.com/p/571b7e63af3d</guid>
            <dc:creator><![CDATA[Kiran Jilla]]></dc:creator>
            <pubDate>Tue, 08 Sep 2026 04:51:58 GMT</pubDate>
            <atom:updated>2026-09-10T16:21:22.447Z</atom:updated>
            <content:encoded><![CDATA[<p><em>Tags: Snowflake, AI Agents, Cortex, Cost Optimization, Software Factory</em></p><p><em>This is Part 2 of a 3-part series. </em><a href="https://medium.com/@kiran.jilla/building-an-efficient-ai-software-factory-with-snowflake-part-1-overview-70de14db6d96"><em>Part 1: Overview</em></a><em> | </em><a href="https://medium.com/@kiran.jilla/building-an-efficient-ai-software-factory-with-snowflake-part-3-optimization-84ee8572cecf"><em>Part 3: Optimization</em></a></p><p>In <a href="https://medium.com/@kiran.jilla/building-an-efficient-ai-software-factory-with-snowflake-part-1-overview-70de14db6d96">Part 1</a>, we covered the four layers of agent usage and the cost equation that decomposes AI spend into measurable terms. Now we need to actually measure each term.</p><p>Snowflake exposes every data point you need through SNOWFLAKE.ACCOUNT_USAGE views. This part covers the full metric set with the SQL to compute each one: portfolio spend, unit economics, per-agent attribution, per-user attribution, model economics, driver decomposition, and agent outcomes.</p><h3>How We Measure</h3><h3>Portfolio: Where the money is going</h3><pre>-- Total AI spend by service type, last 30 days<br>SELECT<br>    service_type,<br>    SUM(credits_used) AS total_credits,<br>    SUM(credits_used) * 2.00 AS estimated_cost_usd  -- global routing<br>FROM SNOWFLAKE.ACCOUNT_USAGE.METERING_HISTORY<br>WHERE start_time &gt;= DATEADD(&#39;day&#39;, -30, CURRENT_TIMESTAMP())<br>  AND service_type IN (<br>    &#39;AI_SERVICES&#39;, &#39;CORTEX_AGENT&#39;, &#39;SNOWFLAKE_INTELLIGENCE&#39;,<br>    &#39;CORTEX_SEARCH&#39;, &#39;CORTEX_CODE&#39;<br>  )<br>GROUP BY service_type<br>ORDER BY total_credits DESC;</pre><h3>Unit economics: per-function spend</h3><pre>-- Cost per AI function, per model, last 7 days<br>SELECT<br>    function_name,<br>    model_name,<br>    COUNT(*) AS request_count,<br>    SUM(credits) AS total_credits,<br>    SUM(credits) / COUNT(*) AS credits_per_request<br>FROM SNOWFLAKE.ACCOUNT_USAGE.CORTEX_AI_FUNCTIONS_USAGE_HISTORY<br>WHERE start_time &gt;= DATEADD(&#39;day&#39;, -7, CURRENT_TIMESTAMP())<br>GROUP BY function_name, model_name<br>ORDER BY total_credits DESC;</pre><h3>Per-agent spend attribution</h3><pre>-- Cost per Cortex Agent object<br>SELECT<br>    agent_name,<br>    COUNT(*) AS session_count,<br>    SUM(tokens) AS total_tokens,<br>    SUM(token_credits) AS total_credits,<br>    SUM(token_credits) / COUNT(*) AS credits_per_session<br>FROM SNOWFLAKE.ACCOUNT_USAGE.CORTEX_AGENT_USAGE_HISTORY<br>WHERE start_time &gt;= DATEADD(&#39;day&#39;, -30, CURRENT_TIMESTAMP())<br>  AND agent_name IS NOT NULL<br>GROUP BY agent_name<br>ORDER BY total_credits DESC;</pre><h3>Per-user attribution</h3><p>The usage views for AI Functions expose user_id, not the human-readable name. Join to USERS to see who is spending. (The Agent usage view already includes user_name directly.)</p><pre>-- Per-user AI Function spend (requires join to USERS)<br>SELECT<br>    u.name AS user_name,<br>    h.function_name,<br>    h.model_name,<br>    SUM(h.credits) AS total_credits<br>FROM SNOWFLAKE.ACCOUNT_USAGE.CORTEX_AI_FUNCTIONS_USAGE_HISTORY h<br>JOIN SNOWFLAKE.ACCOUNT_USAGE.USERS u<br>  ON h.user_id = u.user_id<br>WHERE h.start_time &gt;= DATEADD(&#39;day&#39;, -7, CURRENT_TIMESTAMP())<br>GROUP BY u.name, h.function_name, h.model_name<br>ORDER BY total_credits DESC<br>LIMIT 50;</pre><pre>-- Per-user Agent spend (user_name is available directly)<br>SELECT<br>    user_name,<br>    agent_name,<br>    SUM(token_credits) AS total_credits,<br>    SUM(tokens) AS total_tokens<br>FROM SNOWFLAKE.ACCOUNT_USAGE.CORTEX_AGENT_USAGE_HISTORY<br>WHERE start_time &gt;= DATEADD(&#39;day&#39;, -7, CURRENT_TIMESTAMP())<br>  AND user_name IS NOT NULL<br>GROUP BY user_name, agent_name<br>ORDER BY total_credits DESC<br>LIMIT 50;</pre><h3>Model economics: which model actually moved the bill</h3><pre>-- Cost and request share by model<br>SELECT<br>    model_name,<br>    COUNT(*) AS requests,<br>    SUM(credits) AS total_credits,<br>    SUM(credits) * 100.0 /<br>      SUM(SUM(credits)) OVER () AS pct_of_total_spend,<br>    SUM(credits) / COUNT(*) AS credits_per_request<br>FROM SNOWFLAKE.ACCOUNT_USAGE.CORTEX_AI_FUNCTIONS_USAGE_HISTORY<br>WHERE start_time &gt;= DATEADD(&#39;day&#39;, -30, CURRENT_TIMESTAMP())<br>GROUP BY model_name<br>ORDER BY total_credits DESC;</pre><h3>Driver decomposition: why the number moved</h3><p>Break a cost change into its component drivers (adoption, engagement, input workload, output workload) with nothing left in an unexplained residual:</p><pre>WITH this_month AS (<br>    SELECT<br>        COUNT(DISTINCT user_id) AS users,<br>        COUNT(*) AS requests,<br>        SUM(credits) AS total_credits<br>    FROM SNOWFLAKE.ACCOUNT_USAGE.CORTEX_AI_FUNCTIONS_USAGE_HISTORY<br>    WHERE start_time BETWEEN &#39;2026-06-01&#39; AND &#39;2026-06-30&#39;<br>),<br>last_month AS (<br>    SELECT<br>        COUNT(DISTINCT user_id) AS users,<br>        COUNT(*) AS requests,<br>        SUM(credits) AS total_credits<br>    FROM SNOWFLAKE.ACCOUNT_USAGE.CORTEX_AI_FUNCTIONS_USAGE_HISTORY<br>    WHERE start_time BETWEEN &#39;2026-05-01&#39; AND &#39;2026-05-31&#39;<br>)<br>SELECT<br>    this_month.users AS current_users,<br>    last_month.users AS prior_users,<br>    this_month.requests AS current_requests,<br>    last_month.requests AS prior_requests,<br>    this_month.total_credits AS current_credits,<br>    last_month.total_credits AS prior_credits<br>FROM this_month, last_month;</pre><h3>Agent outcomes: cost per unit of value</h3><p>For managed agents (Layer 2 and 3), track outcome-denominated cost, not just spend:</p><pre>-- Cost per Cortex Agent, grouped by agent<br>SELECT<br>    agent_name,<br>    COUNT(*) AS total_sessions,<br>    SUM(token_credits) AS total_credits,<br>    SUM(tokens) AS total_tokens,<br>    SUM(token_credits) / COUNT(*) AS credits_per_session<br>FROM SNOWFLAKE.ACCOUNT_USAGE.CORTEX_AGENT_USAGE_HISTORY<br>WHERE start_time &gt;= DATEADD(&#39;day&#39;, -30, CURRENT_TIMESTAMP())<br>  AND agent_name IS NOT NULL<br>GROUP BY agent_name<br>ORDER BY credits_per_session DESC;</pre><h3>What’s Next</h3><p>In <a href="https://medium.com/@kiran.jilla/building-an-efficient-ai-software-factory-with-snowflake-part-3-optimization-84ee8572cecf">Part 3: Optimization</a>, we cover the optimization levers that act on each term in the cost equation: dynamic model routing, semantic views, AI_FILTER, AI_EXTRACT, MCP tool schema management, business ontology, code-mode batching, and visibility tools (AI Cost Dashboard, resource budgets, per-user limits, spend tiers).</p><p><strong>Resources</strong></p><ul><li><a href="https://docs.snowflake.com/en/user-guide/snowflake-cortex/ai-func-cost-management">Managing Cortex AI Function costs with Account Usage</a></li><li><a href="https://docs.snowflake.com/en/user-guide/snowflake-cortex/governance-and-availability/ai-cost-management-and-governance">AI cost management and governance</a></li><li><a href="https://www.snowflake.com/en/developers/guides/ai-cost-dashboard/">Cortex AI Cost Dashboard quickstart</a></li></ul><p><em>Have questions or want to share what you built? Drop a comment below.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=571b7e63af3d" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Building an Efficient AI Software Factory with Snowflake — Part 1: Overview]]></title>
            <link>https://medium.com/@kiran.jilla/building-an-efficient-ai-software-factory-with-snowflake-part-1-overview-70de14db6d96?source=rss-ec2ba42995a3------2</link>
            <guid isPermaLink="false">https://medium.com/p/70de14db6d96</guid>
            <dc:creator><![CDATA[Kiran Jilla]]></dc:creator>
            <pubDate>Sat, 29 Aug 2026 17:37:55 GMT</pubDate>
            <atom:updated>2026-09-10T16:22:30.037Z</atom:updated>
            <content:encoded><![CDATA[<p><em>Tags: Snowflake, AI Agents, Cortex, MCP, Cost Optimization, Software Factory</em></p><p>AI agents are now embedded in every phase of the data and engineering lifecycle. The question is no longer <em>whether</em> to deploy them — it’s <em>how to deploy them efficiently</em>. Without governance, model routing, and cost tracking, AI spend explodes while business value plateaus.</p><p>The pattern is familiar to anyone who has watched a platform scale. Usage grows while costs stabilize only because of optimization work happening underneath. In one <a href="https://www.uber.com/us/en/blog/efficient-software-factory/">widely-cited case</a>, an engineering organization built over 3,600 agent skills and executed more than 30K agent skill executions per day, with weekly active users growing 7x over six months and weekly agentic requests growing 9.4x. Total AI spend stabilized because cost per 1,000 model requests dropped 34% from its peak, and cost per session dropped 52% from its June peak — with the model held constant.</p><p>That same dynamic is now hitting every organization running Snowflake. Cortex Agents, CoWork, Cortex Code, and AI Functions are all GA. The Managed MCP Server lets any external agent — Claude, Cursor, Copilot, LangChain — connect to your governed data. Usage is growing fast. The question is whether you can hold unit costs down while it does.</p><p>Snowflake’s 2026 platform — Cortex AI Gateway, Dynamic Model Routing, Managed MCP Server, Resource Budgets, and a full suite of usage-tracking views — gives you the building blocks to run a governed, measured, optimized AI software factory on top of your governed data. This series walks through the architecture: the four layers agents run in, the cost equation, how to measure each term with real SQL, and the optimization levers Snowflake provides.</p><p>This is a practitioner’s guide. Every section includes the SQL, config, or setup you need to implement it.</p><h3>The Four Layers of Agent Usage</h3><p>We organize AI usage into four layers, from the most specialized to the most general. The higher the layer, the more control you have over cost, quality, and model selection.</p><pre>┌─────────────────────────────────────────────────────────┐<br>│  Layer 1: AISQL Functions                                │  Most specialized<br>│  AI_COMPLETE · AI_CLASSIFY · AI_EXTRACT · AI_FILTER    │  Lowest cost<br>├─────────────────────────────────────────────────────────┤<br>│  Layer 2: Cortex Agents                                  │<br>│  Orchestrate across Cortex Search + Analyst              │<br>├─────────────────────────────────────────────────────────┤<br>│  Layer 3: Snowflake CoWork                              │<br>│  Managed work agent for every knowledge worker          │<br>├─────────────────────────────────────────────────────────┤<br>│  Layer 4: External Agents via MCP                        │  Most general<br>│  Claude · Cursor · Copilot · LangChain · OpenAI SDK     │  Any framework<br>└─────────────────────────────────────────────────────────┘</pre><h3>Layer 1: AISQL Functions</h3><p>The most specialized and cheapest layer. Single-purpose AI operations called directly from SQL — classify a support ticket, extract fields from a contract, summarize a document, filter rows by semantic meaning. Billed per million tokens; a well-scoped AI_CLASSIFY call costs fractions of a cent.</p><pre>-- Classify 10,000 support tickets in a single query<br>SELECT<br>    ticket_id,<br>    AI_CLASSIFY(<br>        ticket_text,<br>        [&#39;billing&#39;, &#39;technical&#39;, &#39;account&#39;, &#39;other&#39;]<br>    ) AS category<br>FROM support_tickets<br>WHERE created_at &gt; &#39;2026-06-01&#39;;</pre><p>Best when: the task is a single transformation, you know the exact input shape, and you want the cost predictability of SQL.</p><h3>Layer 2: Cortex Agents</h3><p>Agents that orchestrate across Cortex Search (unstructured) and Cortex Analyst (structured SQL via natural language). The agent decides which tool to call, in what order, and synthesizes a response. Billed in AI Credits per million tokens — cost compounds across every underlying service the agent invokes.</p><pre>CREATE OR REPLACE CORTEX AGENT my_analyst_agent<br>  MODEL = &#39;claude-sonnet-4-6&#39;<br>  TOOLS = (<br>    CORTEX_SEARCH SERVICE my_search,<br>    CORTEX_ANALYST SEMANTIC_VIEW my_semantic_view<br>  );</pre><p>Best when: the task requires reasoning across both structured and unstructured data, and you want Snowflake to handle the orchestration.</p><h3>Layer 3: Snowflake CoWork</h3><p>The managed work agent — rebranded from Snowflake Intelligence at Summit 2026. A personal agent for every knowledge worker that connects to your data, runs Cortex Agents under the hood, and takes actions (not just answers questions). Billed in AI Credits per session.</p><p>Best when: you want to give non-technical users an AI work assistant without building a custom app. CoWork handles the agent orchestration, tool selection, and memory.</p><h3>Layer 4: External Agents via MCP</h3><p>The most general layer. Any AI agent platform — Claude Desktop, Cursor, Microsoft Copilot, LangChain, OpenAI’s Agents SDK — connects to Snowflake through the Managed MCP Server. You bring your own agent framework; Snowflake provides governed data access via MCP tools.</p><p>Best when: you already have an agent framework in use across your organization, or you need capabilities Snowflake’s native agents don’t provide (custom tool calls, external integrations, long-running autonomous workflows).</p><h3>The Cost Equation</h3><p>Across any of the four layers, the cost of an agentic session breaks down into a <strong>chain of multipliers</strong> — each term feeds into the next, like a funnel. Reduce any term in the chain and the entire cost below it shrinks proportionally.</p><pre>Total AI Spend =<br>    Users                    ← how many people use AI agents<br>  × Sessions per user        ← conversations per user per month  (adoption - grow this)<br>  × Turns per session         ← back-and-forth exchanges per conversation<br>  × Requests per turn         ← API calls per exchange (e.g. one search + one SQL)<br>  × Tokens per request        ← input + output tokens per API call<br>  × Price per token           ← cost per token for the model used  (optimize this)<br>  + Warehouse compute         ← SQL execution credits (fixed-ish)<br>  + Cortex Search serving      ← always-on search index cost (fixed-ish)<br>  + AI Credits overhead        ← gateway routing cost per request</pre><h3>A Concrete Example</h3><p>Say a small team of 10 analysts uses a Cortex Agent. Each analyst has 5 conversations per week, each conversation has 8 back-and-forth turns, each turn makes 2 API calls (one Cortex Search + one Cortex Analyst), and each call processes 3,000 tokens:</p><pre>10 users × 5 sessions × 8 turns × 2 requests × 3,000 tokens<br>= 2.4 million tokens per week</pre><p>Now here’s where the multiplier effect matters. If you use Semantic Views to return smaller, typed results — cutting tokens per request from 3,000 to 1,000 — the entire weekly total drops to 800K tokens. <strong>A 3x reduction in one term produces a 3x reduction in the total</strong>, because every term above it multiplies that savings.</p><p>Or if Business Ontology helps the agent find the right table in 2 turns instead of 8, the weekly total drops from 2.4M to 600K tokens — a 4x reduction, just by cutting turns per session.</p><p>This is the core insight: because the terms multiply, shrinking waste at any point compounds across everything downstream. You don’t need to negotiate a lower price per token or limit adoption — you just need to make each turn and request more efficient.</p><h3>The Snowflake Billing Reality</h3><p>Since April 1, 2026, Snowflake bills AI usage in a dedicated currency called <strong>AI Credits</strong>, independent of your Snowflake edition:</p><ul><li><strong>$2.00 per AI Credit</strong> with global routing (cross-region inference)</li><li><strong>$2.20 per AI Credit</strong> with regional routing (data residency pinned)</li></ul><p>AI Credits apply to Cortex Agents, Cortex Code, Snowflake CoWork, AI Functions, and Cortex Search. They are separate from Platform Credits (warehouse compute, storage). Your monthly bill shows both currencies.</p><p>This matters for the cost equation because your optimization levers reduce AI Credits directly — a cheaper model routing decision shows up as fewer AI Credits consumed, not just a lower token count.</p><h3>What’s Next</h3><p>In <a href="https://medium.com/@kiran.jilla/building-an-efficient-ai-software-factory-with-snowflake-part-2-measurement-571b7e63af3d">Part 2: Measurement</a>, we cover the full metric set for tracking AI spend with the SQL to compute each one: portfolio, unit economics, per-agent, per-user, model economics, driver decomposition, and agent outcomes.</p><p>In <a href="https://medium.com/@kiran.jilla/building-an-efficient-ai-software-factory-with-snowflake-part-3-optimization-84ee8572cecf">Part 3: Optimization</a>, we cover the optimization levers: dynamic model routing, semantic views, AI_FILTER, AI_EXTRACT, MCP tool schema management, business ontology, and visibility tools.</p><p><em>Have questions or want to share what you built? Drop a comment below.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=70de14db6d96" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>