<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Omid Karami on Medium]]></title>
        <description><![CDATA[Stories by Omid Karami on Medium]]></description>
        <link>https://medium.com/@omid-karami?source=rss-26d5566e003e------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*spp4w_xyvf93f3v24UGXJg.png</url>
            <title>Stories by Omid Karami on Medium</title>
            <link>https://medium.com/@omid-karami?source=rss-26d5566e003e------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 05:25:30 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@omid-karami/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[From Prompts to Management Reports: Snowflake HTML Reports with CoCo]]></title>
            <link>https://medium.com/snowflake/from-prompts-to-management-reports-snowflake-html-reports-with-coco-2febecfc5e8c?source=rss-26d5566e003e------2</link>
            <guid isPermaLink="false">https://medium.com/p/2febecfc5e8c</guid>
            <category><![CDATA[data-engineering]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[snowflake-coco]]></category>
            <category><![CDATA[finops]]></category>
            <category><![CDATA[snowflake]]></category>
            <dc:creator><![CDATA[Omid Karami]]></dc:creator>
            <pubDate>Wed, 18 Nov 2026 15:01:03 GMT</pubDate>
            <atom:updated>2026-11-18T15:01:03.404Z</atom:updated>
            <content:encoded><![CDATA[<p><em>Exploring Snowflake’s new </em><strong><em>HTML Reports</em></strong><em> capability through a practical cost consumption report — and how </em><strong><em>AI-assisted reporting</em></strong><em> can support </em><strong><em>FinOps</em></strong><em>, cost visibility, and better platform decisions.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*IqDDXh40sE3dPh2wR9GRhQ.png" /><figcaption>AI-generated image created using ChatGPT by OpenAI.</figcaption></figure><h3>TL;DR</h3><p>Snowflake <strong>HTML Reports</strong>, introduced in <strong>Public Preview</strong> in November 2026, let practitioners use <strong>Snowflake CoCo </strong>to turn query results into shareable reports. This case study demonstrates a <strong>Snowflake Cost Consumption Report</strong> generated using the bundled cost-intelligence skill. It includes the original prompt, executed SQL, report screenshots, and validation findings. The experiment illustrates how AI-assisted reporting can support management decisions and <strong>FinOps</strong>, while highlighting the importance of data completeness, SQL verification, and transparent limitations.</p><h3>1. From Data to Decisions: The Management Reporting Challenge</h3><p><strong>“Where are our Snowflake credits going?”</strong></p><p>Answering this question often requires examining several Account Usage views. Service metering, warehouse consumption, query attribution, storage, and AI-assisted development activity each provide different perspectives.</p><p>For Data Platform Teams, the challenge extends beyond retrieving metrics. Technical results must be organized into reports that stakeholders can understand, question, and use to prioritize investigations.</p><p>I chose to explore whether CoCo and HTML Reports could bridge the gap between technical data exploration and management reporting. As a data engineer working with Snowflake, I see value in making platform consumption understandable beyond engineering teams.</p><p>Cost consumption was a practical starting point because Snowflake already provides detailed Account Usage data. This allowed me to evaluate two things together: the AI-assisted reporting experience and the accuracy of the generated results.</p><p>The objective was not to demonstrate automatic cost optimization or claim financial savings. It was to investigate whether an AI-assisted workflow could produce a useful management reporting artifact while preserving the technical evidence needed to trust it.</p><h3>2. Introducing Snowflake HTML Reports and CoCo</h3><p>Snowflake introduced HTML Reports in <strong>Public Preview on November 4, 2026</strong>.</p><p>HTML Reports support self-contained reporting artifacts containing saved query results, visualizations, tables, and explanatory content. CoCo can assist with querying data and generating the report from natural-language instructions.</p><p>For this experiment, I requested <strong>inline CSS and SVG only</strong>, without external scripts or visualization libraries. The resulting report used KPI cards, charts, tables, and observations.</p><h4>Reports are published snapshots</h4><p>An HTML Report is not automatically a continuously refreshed dashboard.</p><p>The generated artifact contains saved query results. Refreshing the report in the workspace does not automatically update its published version; the updated artifact must be republished.</p><p>During Public Preview, scheduled report refresh and delivery are not supported.</p><h4>Governance matters</h4><p>Published reports embed data that authorized report viewers can access without necessarily having privileges on the original source tables.</p><p>Report authors must therefore review sensitive identifiers, embedded query results, and audience permissions before publication.</p><p>These characteristics make HTML Reports useful for curated technical and management reporting, while keeping validation and governance responsibilities with the producing team.</p><h3>3. The Use Case: Building a Snowflake Cost Consumption Report</h3><p>The implementation used CoCo in Snowsight to generate a self-contained HTML report from SNOWFLAKE.ACCOUNT_USAGE.</p><p>The requested reporting period was 30 days. The report covered service consumption, daily activity, warehouses, query attribution, database storage, and CoCo usage.</p><h4>Report overview</h4><p>The generated report includes four KPIs: total metered credits, peak daily consumption, largest service, and database storage availability.</p><p>A donut chart presents service consumption, while daily bars show the observed reporting period.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*prXAEfppU4xiQQivXBuhXQ.png" /><figcaption><em>Figure 1. Snowflake Cost Consumption Report generated using CoCo and HTML Reports, showing account-level consumption KPIs, service distribution, and daily credits for November 9–10, 2026.</em></figcaption></figure><p>The report’s principal measurements included:</p><p><strong>Observed Consumption Summary</strong></p><p>Total metered consumption: 13.5678 credits<br>Peak day: November 9, 2026–12.919 credits<br>Largest service: CoCo in Snowsight — 5.1573 credits (38.0%)<br>Database storage: No data returned</p><p>Although the prompt requested 30 days, the observed consumption covered only <strong>November 9–10, 2026</strong>.</p><p>What stood out to me was how a structured prompt could produce a report combining several consumption perspectives. However, the visualizations were only part of the story.</p><p>I noticed the two-day limitation while reviewing the generated report. This reinforced an important lesson: <strong>a polished report can look complete even when its underlying observation period is limited</strong>.</p><h4>CoCo model and token consumption</h4><p>The report also examined CoCo usage from SNOWFLAKE_COCO_USAGE_HISTORY, filtered to the Snowsight interface.</p><p>It displayed model-level credits and input, output, cache-read, and cache-write tokens.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zZLono8C95vveZlEfaMDAw.png" /><figcaption><em>Figure 2. CoCo model-level credit consumption and token usage from </em><em>SNOWFLAKE_COCO_USAGE_HISTORY, including the unresolved difference from service-level metering.</em></figcaption></figure><p>The model-level breakdown totaled <strong>5.3893 credits and 14,665,184 tokens across three models</strong>.</p><p>These measurements provide additional visibility into AI-assisted development consumption, but they should not be assumed equivalent to service-level metering.</p><h3>4. From Natural-Language Prompt to HTML Report</h3><p>The implementation began with a structured prompt in CoCo in Snowsight.</p><p>It explicitly requested Snowflake’s bundled cost-intelligence skill, defined the Account Usage sources, specified the reporting layout, and required transparent handling of missing or inconsistent data.</p><h4>The original CoCo prompt</h4><p>The following is the complete original prompt used for this experiment, preserved without changing its instructions.</p><pre>Using the cost-intelligence skill, build a self-contained HTML cost consumption report for this account and save it to my workspace.<br><br>Data rules:<br>- Query live data from SNOWFLAKE.ACCOUNT_USAGE for the last 30 days. Do not reuse numbers from any earlier session or hardcode values I didn&#39;t get from a query you ran in this session.<br>- Use these sources:<br>  - SNOWFLAKE.ACCOUNT_USAGE.METERING_HISTORY: credits by SERVICE_TYPE and by day.<br>  - SNOWFLAKE.ACCOUNT_USAGE.WAREHOUSE_METERING_HISTORY: credits by warehouse, top 15.<br>  - SNOWFLAKE.ACCOUNT_USAGE.QUERY_ATTRIBUTION_HISTORY: top users by credits_attributed_compute plus credits_used_query_acceleration, and the 10 most expensive queries.<br>  - SNOWFLAKE.ACCOUNT_USAGE.DATABASE_STORAGE_USAGE_HISTORY: average storage by database in GB (database plus failsafe bytes), top 10.<br>  - SNOWFLAKE.ACCOUNT_USAGE.SNOWFLAKE_COCO_USAGE_HISTORY, filtered to LOWER(INTERFACE) = &#39;snowsight&#39;: credits and tokens per model. Flatten CREDITS_GRANULAR and TOKENS_GRANULAR (keys input, output, cache_read_input, cache_write_input) and join them by model name.<br>- If a source returns no rows, keep its section and show a short &quot;No data yet&quot; note. Do not invent data.<br>- Use the real date range of the data for the period line, and say so if all usage falls on a single day.<br>Report layout (inline CSS and inline SVG only, no external libraries or scripts):<br>1. Header: &quot;Snowflake Cost Consumption Report&quot;, with a line &quot;Period: &lt;start&gt; – &lt;end&gt; (last 30 days) · Source: SNOWFLAKE.ACCOUNT_USAGE · Units: credits&quot;.<br>2. Four KPI cards: total credits, peak day credits (with its date), largest service and its % of total, largest database storage (GB, or &quot;n/a&quot; if no data).<br>3. Two columns: a donut chart of credits by service type with a legend, and a vertical bar chart of daily credits.<br>4. Full-width section &quot;Snowflake CoCo (Snowsight): Models &amp; Tokens&quot;: a credits-by-model bar chart plus a table (model, input, output, cache read, cache write, total tokens, credits) and a note with totals.<br>5. Two columns: credits by warehouse (horizontal bars), and top users with the most expensive queries (or a &quot;No data yet&quot; note).<br>6. Storage by database (avg GB).<br>7. Observations: 3 to 5 factual bullets drawn only from the data. If two numbers come from different views and differ, state both and say the cause is unverified.<br>Style:<br>Light background (#f4f7fb), white rounded cards with a subtle shadow, Snowflake blue (#29b5e8) as the primary color, and a purple/pink/orange palette for the donut slices.<br>Do not leave helper scripts in the workspace.<br>When finished, tell me which queries you ran and anything you could not verify.</pre><p>The most important prompting decisions were explicit data provenance, handling empty results without fabrication, and disclosing differences between metrics instead of inventing explanations.</p><h4>The original executed SQL</h4><p>The following three statements were recovered from Snowflake Query History. They are reproduced in their original compact form.</p><p><strong>Warehouse-level metering</strong></p><pre>SELECT WAREHOUSE_NAME, ROUND(SUM(CREDITS_USED),4) CR FROM SNOWFLAKE.ACCOUNT_USAGE.WAREHOUSE_METERING_HISTORY WHERE START_TIME &gt;= DATEADD(day,-30,CURRENT_TIMESTAMP()) GROUP BY 1 ORDER BY 2 DESC LIMIT 15</pre><p><strong>Service-level metering</strong></p><pre>SELECT SERVICE_TYPE, ROUND(SUM(CREDITS_USED),4) CR, MIN(START_TIME::DATE) MN, MAX(START_TIME::DATE) MX FROM SNOWFLAKE.ACCOUNT_USAGE.METERING_HISTORY WHERE START_TIME &gt;= DATEADD(day,-30,CURRENT_TIMESTAMP()) GROUP BY 1 ORDER BY 2 DESC</pre><p><strong>CoCo model-level credits and tokens</strong></p><pre>WITH c AS (SELECT f.key m, SUM(COALESCE(f.value:input::FLOAT,0)+COALESCE(f.value:output::FLOAT,0)+COALESCE(f.value:cache_read_input::FLOAT,0)+COALESCE(f.value:cache_write_input::FLOAT,0)) cr FROM SNOWFLAKE.ACCOUNT_USAGE.SNOWFLAKE_COCO_USAGE_HISTORY h, LATERAL FLATTEN(input=&gt;h.CREDITS_GRANULAR) f WHERE h.USAGE_TIME &gt;= DATEADD(day,-30,CURRENT_TIMESTAMP()) AND LOWER(h.INTERFACE)=&#39;snowsight&#39; GROUP BY 1),<br>t AS (SELECT f.key m, SUM(COALESCE(f.value:input::FLOAT,0)) i, SUM(COALESCE(f.value:output::FLOAT,0)) o, SUM(COALESCE(f.value:cache_read_input::FLOAT,0)) cr_, SUM(COALESCE(f.value:cache_write_input::FLOAT,0)) cw FROM SNOWFLAKE.ACCOUNT_USAGE.SNOWFLAKE_COCO_USAGE_HISTORY h, LATERAL FLATTEN(input=&gt;h.TOKENS_GRANULAR) f WHERE h.USAGE_TIME &gt;= DATEADD(day,-30,CURRENT_TIMESTAMP()) AND LOWER(h.INTERFACE)=&#39;snowsight&#39; GROUP BY 1)<br>SELECT COALESCE(c.m,t.m) model, ROUND(c.cr,4) credits, t.i, t.o, t.cr_ cache_read, t.cw cache_write FROM c FULL JOIN t ON c.m=t.m ORDER BY c.cr DESC NULLS LAST</pre><h4>Why the granular aggregation matters</h4><p>My main concern during validation was whether the generated SQL preserved the meaning of the underlying consumption metrics.</p><p>I paid particular attention to CREDITS_GRANULAR and TOKENS_GRANULAR, because flattening multiple granular structures can introduce duplicate combinations if their expanded rows are joined incorrectly.</p><p>Reviewing the executed SQL showed that the two measurements were aggregated independently by model:</p><ul><li>CTE c flattens and aggregates granular credits.</li><li>CTE t independently aggregates token categories.</li><li>The final FULL JOIN combines the aggregated model results.</li></ul><p>Reviewing the executed SQL confirmed that credits and tokens were aggregated independently by model before joining. This addressed my concern about join fanout in the generated query, although the differences between service-level and model-level credits remained unresolved.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ERM9H6RVeleIUhS1CxQoLg.png" /><figcaption>Figure 3. Snowflake Query History showing the successful execution of the CoCo granular usage query, including independent aggregation of credits and tokens by model and the resulting measurements.</figcaption></figure><p>This validation does not establish complete financial reconciliation or independent verification of every report metric.</p><h4>Reproducing and validating the workflow</h4><p>Practitioners can adapt the original prompt after confirming CoCo and cost-intelligence availability, Account Usage access, and appropriate permissions.</p><p>The practical workflow is to execute the prompt, inspect the generated SQL and returned results, validate important calculations, review the HTML artifact, and approve its content before publishing.</p><p>The key engineering principle is straightforward: <strong>AI-assisted generation can accelerate report construction, but technical validation remains the responsibility of the practitioner.</strong></p><h3>5. Beyond Visualization: Turning Consumption into Management Insights</h3><p>A report becomes useful when metrics lead to informed investigations rather than automatic conclusions.</p><p>A practical reasoning sequence is:</p><p><strong>Consumption Metric → Management Question → Investigation → Potential Decision</strong></p><p>For example:</p><p><strong>From Metrics to Management Decisions</strong></p><p><strong>Service credits</strong><br>Management question: Where is consumption concentrated?<br>Potential investigation: Identify activities and owners.</p><p><strong>Warehouse credits</strong><br>Management question: Which compute resources deserve attention?<br>Potential investigation: Review sizing, utilization, and workload patterns.</p><p><strong>CoCo credits</strong><br>Management question: What drives AI-assisted development usage?<br>Potential investigation: Examine model activity and operational context.</p><p>The report showed <strong>5.1573 CoCo Snowsight credits</strong>, approximately <strong>38%</strong> of the 13.5678 metered credits observed.</p><p>I found that proportion interesting, but I would not immediately interpret it as excessive consumption.</p><p>Given the short observation period, my first questions would concern the activities behind that usage, the value those activities delivered, and whether the pattern continued over time.</p><p>For me, the purpose of reporting is to identify questions worth investigating, not automatically label the largest consumption category as a problem.</p><p>Warehouse metering and query attribution likewise provide different perspectives; query-attributed compute does not account for every warehouse credit.</p><p>The management value comes from connecting these technical measurements with workload importance, operational requirements, and responsible owners.</p><h3>6. From Reporting to FinOps and Cost Optimization</h3><p>The report can support FinOps by connecting consumption visibility with investigation, accountability, and measurable decisions.</p><p>The FinOps framework describes three iterative phases: <strong>Inform, Optimize, and Operate</strong>.</p><h4>Inform: Make consumption understandable</h4><p>The report brings service, warehouse, query, and AI-related measurements into one presentation.</p><p>This helps platform engineers, workload owners, and managers establish a common understanding of where consumption occurs and what requires further investigation.</p><h4>Optimize: Test improvement hypotheses</h4><p>High consumption does not automatically mean waste.</p><p>A warehouse with substantial credit usage may warrant examination of sizing, scheduling, auto-suspend settings, and query performance. Any proposed change should be evaluated against workload requirements and reliability.</p><p>Similarly, CoCo usage should be investigated in the context of development activities before recommending restrictions or configuration changes.</p><h4>Operate: Establish an ongoing review</h4><p>I believe cost visibility becomes more valuable when connected to clear ownership and a repeatable decision-making process.</p><p>That is why I propose the <strong>Prepare → Review → Assign → Evaluate → Measure</strong> cycle as a practical adaptation for Data Platform Teams.</p><p>It connects reporting with informed action while involving workload owners and business stakeholders. This is my proposed operating approach, not an official Snowflake-prescribed workflow.</p><p>Proposed <strong>FinOps</strong> Operating Cycle</p><p><strong>1. PREPARE</strong><br>Activity: Refresh, validate, and republish the approved report when required.<br>Outcome: Reliable snapshot</p><p><strong>2. REVIEW</strong><br>Activity: Discuss patterns and open questions.<br>Outcome: Investigation priorities</p><p><strong>3. ASSIGN</strong><br>Activity: Identify owners and required evidence.<br>Outcome: Accountability</p><p><strong>4. EVALUATE</strong><br>Activity: Assess opportunities against technical and business requirements.<br>Outcome: Informed decisions</p><p><strong>5. MEASURE</strong><br>Activity: Compare outcomes with an agreed baseline.<br>Outcome: Documented results</p><p>Platform engineers own technical investigations; workload owners provide business and operational context; FinOps practitioners support financial interpretation; management prioritizes decisions.</p><p>The goal is not simply to reduce credits. It is to make better decisions about platform efficiency, reliability, and the value delivered by data workloads.</p><h4>Measuring outcomes responsibly</h4><p>Credit consumption, monetary cost, financial allocation, and verified savings are distinct.</p><p>A credible optimization assessment needs comparable before-and-after workload measurements, performance and reliability checks, and appropriate financial reconciliation.</p><p>Lower credit consumption alone does not establish an equivalent reduction in billed costs.</p><p>This experiment demonstrates reporting and investigation opportunities — not verified cost savings.</p><h3>7. What Practitioners Should Know Before Using HTML Reports</h3><p>Before adopting HTML Reports for management reporting, practitioners should account for several limitations:</p><ul><li><strong>Feature maturity:</strong> HTML Reports were introduced in Public Preview; evaluate current capabilities and restrictions.</li><li><strong>Snapshot behavior:</strong> Reports embed saved results. Updating a published artifact requires refreshing and republishing.</li><li><strong>Governance:</strong> Authorized viewers can see embedded data without direct source-table permissions. Sanitize screenshots and published artifacts.</li><li><strong>Freshness:</strong> Account Usage sources have different latencies and measurement scopes. Validate dates, completeness, and SQL.</li></ul><p>This case study also retains four important evidence boundaries:</p><ul><li>Only <strong>November 9–10, 2026</strong> contained observed consumption within the requested 30-day window.</li><li>Database storage returned no rows.</li><li>CoCo service-level metering reported <strong>5.1573 credits</strong>, compared with <strong>5.3893</strong> model-level credits.</li><li>Service-level warehouse metering reported <strong>3.4295 credits</strong>, compared with <strong>3.5604</strong> from WAREHOUSE_METERING_HISTORY.</li></ul><p>The recovered SQL supports the reported values, but the differences between measurements remain unexplained.</p><p>They should not be attributed to billing errors, aggregation defects, or inefficiency without additional evidence.</p><h3>8. Conclusion: A New Approach to Management Reporting</h3><p>Snowflake HTML Reports and CoCo provide a practical approach to turning account usage data into structured, shareable reporting artifacts.</p><p>This experiment demonstrates the workflow with a Snowflake Cost Consumption Report, an unchanged natural-language prompt, and executed SQL that was subsequently reviewed.</p><p>The most important lesson is that generating a convincing report is not the same as validating its evidence.</p><p>Technical review, transparent limitations, and appropriate governance remain essential.</p><p>The broader value comes from connecting that evidence with management questions and a repeatable FinOps operating process.</p><p>HTML Reports can support that connection by making technical findings more accessible. Decisions, optimization, and verified financial outcomes still depend on the people and processes using the report.</p><p><strong>The value is not simply in generating a report. It is in creating a clearer connection between technical evidence and the decisions that follow.</strong></p><h3>References</h3><ol><li><a href="https://docs.snowflake.com/en/user-guide/ui-snowsight/reports">Snowflake HTML Reports — documentation</a></li><li><a href="https://docs.snowflake.com/en/release-notes/2026/other/2026-11-04-html-reports-preview">HTML Reports — November 4, 2026 Public Preview announcement</a></li><li><a href="https://docs.snowflake.com/en/user-guide/cortex-code/bundled-skills">Cortex Code bundled skills</a></li><li><a href="https://docs.snowflake.com/en/user-guide/cortex-code/cortex-code-snowsight/coco-mode">Cortex Code in Snowsight</a></li><li><a href="https://docs.snowflake.com/en/sql-reference/account-usage/snowflake_coco_usage_history">SNOWFLAKE_COCO_USAGE_HISTORY — Account Usage reference</a></li><li><a href="https://docs.snowflake.com/en/sql-reference/account-usage">Snowflake Account Usage — reference and latency</a></li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2febecfc5e8c" width="1" height="1" alt=""><hr><p><a href="https://medium.com/snowflake/from-prompts-to-management-reports-snowflake-html-reports-with-coco-2febecfc5e8c">From Prompts to Management Reports: Snowflake HTML Reports with CoCo</a> was originally published in <a href="https://medium.com/snowflake">Snowflake Builders Blog: Data Engineers, App Developers, AI, &amp; Data Science</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Snowflake Expedition 2026: The Architecture Behind the Agentic Enterprise]]></title>
            <link>https://omid-karami.medium.com/snowflake-expedition-2026-the-architecture-behind-the-agentic-enterprise-9efbbc371946?source=rss-26d5566e003e------2</link>
            <guid isPermaLink="false">https://medium.com/p/9efbbc371946</guid>
            <category><![CDATA[agentic-ai]]></category>
            <category><![CDATA[data-architecture]]></category>
            <category><![CDATA[snowflake]]></category>
            <category><![CDATA[snowflake-data-cloud]]></category>
            <category><![CDATA[data-superhero]]></category>
            <dc:creator><![CDATA[Omid Karami]]></dc:creator>
            <pubDate>Mon, 09 Nov 2026 07:29:47 GMT</pubDate>
            <atom:updated>2026-11-09T07:29:47.907Z</atom:updated>
            <content:encoded><![CDATA[<p><em>From Business Context and AI Governance to Autonomous Data Engineering and Real-Time Intelligence</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ieL-AMUu--Zfm6yDhd-APA.png" /></figure><h3>From AI Features to Agentic Architecture</h3><p>Enterprise AI is moving beyond answering questions toward performing work.</p><p>An assistant might generate SQL, explain a dashboard, or summarize a failed pipeline. An agent can go further: investigate the failure, select tools, propose a change, and potentially execute it.</p><p>That shift changes the engineering problem.</p><p>An incorrect answer can misinform a decision. An incorrect action can modify data, disrupt a pipeline, or trigger an unintended operation. As agents gain operational capabilities, their reliability depends on more than model quality.</p><p><strong>The central challenge is no longer simply making agents more capable. It is engineering the architecture that makes their actions trustworthy.</strong></p><p>Snowflake Expedition 2026 provides an interesting lens through which to examine this transition. Across its announcements and existing platform capabilities, four architectural responsibilities stand out: <strong>Business Context, Governed AI Control, Agentic Execution, and Trusted/Fresh Data</strong>.</p><p>These are not four mandatory layers of a single Snowflake runtime. They represent complementary capabilities with different maturity levels, integration boundaries, and operational responsibilities.</p><p>The interesting question is not whether Snowflake has every component. It is what enterprise data teams must engineer between them.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*M1t3V8GogHzcWqdLRxkg3g.png" /></figure><p><em>Figure 1. An independent conceptual interpretation of the architecture behind enterprise AI agents. The four areas represent complementary responsibilities — not an official Snowflake reference architecture or a single integrated execution pipeline.</em></p><h3>Business Context: Agents Need Meaning, Not Just Data</h3><p>Imagine asking an enterprise agent to calculate net revenue.</p><p>Finance might define it as recognized revenue after refunds. Sales might use a bookings-based calculation. Both definitions may be technically valid and available in governed datasets.</p><p>The problem is not necessarily incorrect SQL. It is deciding which business meaning applies.</p><p>This distinction becomes more important when agents operate without the informal organizational knowledge that experienced employees accumulate.</p><p>Snowflake’s Semantic Views address part of this challenge by defining analytical relationships, dimensions, metrics, and verified queries. Cortex Agents can use Cortex Analyst to query structured data through these semantic definitions.</p><p>But analytical semantics and business agreement are not identical.</p><p>Snowflake’s <strong>Business Ontology</strong>, part of Horizon Context, extends the discussion toward steward-approved business concepts, ownership, relationships, and connections to underlying data assets.</p><p>As of November 3, Business Ontology was in <strong>Private Preview</strong>, with Public Preview announced as forthcoming. Cortex Sense, intended to help discover and infer context, was also announced for a forthcoming Public Preview.</p><p>The distinction between discovered and approved context is particularly valuable. Automated discovery can identify candidate definitions, but organizational accountability determines which meaning is authoritative.</p><p>There is an important integration boundary here. Snowflake documents Cortex Agents using Cortex Analyst with Semantic Views. Business Ontology can connect approved concepts to those assets, but this should not be interpreted as a universal, production-ready ontology integration across every agent runtime.</p><p><strong>My architectural interpretation is that enterprise context needs two complementary forms of governance: computational consistency and business accountability.</strong> An agent may execute a metric correctly while still choosing the wrong metric for the decision.</p><p>That is why trusted business meaning must become an explicit architectural concern rather than an assumption embedded in prompts.</p><p><strong><em>Official references:</em></strong><em> </em><a href="https://www.snowflake.com/en/blog/engineering/business-ontology-context/"><em>Business Ontology engineering deep dive</em></a><em> · </em><a href="https://www.snowflake.com/en/blog/business-ontology-context/"><em>Business Ontology announcement</em></a><em> · </em><a href="https://docs.snowflake.com/en/user-guide/snowflake-cortex/cortex-agents"><em>Cortex Agents documentation</em></a></p><h3>Governed AI Control: The Boundary Matters More Than the Label</h3><p>As agents gain access to models, tools, and enterprise systems, governance becomes a problem of controlling execution paths.</p><p>Snowflake’s <strong>Cortex AI Gateway</strong> is an important development in this area. Its core inference, observability, and cost-management capabilities reached <strong>General Availability on November 3, 2026</strong>.</p><p>The Gateway provides a Snowflake-managed endpoint for supported AI clients, with model access controls, usage visibility, and cost attribution.</p><p>However, the architectural significance lies in understanding precisely what it governs.</p><p><strong>Gateway-specific controls apply to supported traffic routed through the Gateway. They do not automatically cover every Cortex Agent, CoCo operation, or AI request executed elsewhere in Snowflake.</strong></p><p>This is not a minor implementation detail. It determines where an enterprise can actually enforce policy and observe behavior.</p><p>For example, granting USAGE on the Gateway permits a role to send supported requests through that endpoint, but it does not grant additional model entitlements. MONITOR is a separate privilege. Gateway USAGE is granted to PUBLIC by default, so organizations requiring restricted access must review that grant.</p><p>Observability also depends on configuration. Gateway request tracing requires logging to be enabled, while capturing prompts and responses requires an additional setting and raises sensitive-data handling considerations.</p><p>Capabilities such as Gateway Tools and MCP integration have separate Preview scopes and regional restrictions; they should not be conflated with the GA core.</p><p>For enterprise architects, this suggests a broader control model: platform RBAC and data policies govern underlying resources, Gateway controls govern supported routed model traffic, and application-specific safeguards govern consequential actions.</p><p><strong>The independent architectural lesson is that governance follows actual execution paths, not product boundaries or marketing terminology.</strong></p><p>Before claiming centralized AI governance, teams should map which requests pass through which control points — and identify the paths that do not.</p><p><strong><em>Official references:</em></strong><em> </em><a href="https://docs.snowflake.com/en/user-guide/snowflake-cortex/cortex-ai-gateway"><em>Cortex AI Gateway documentation</em></a><em> · </em><a href="https://docs.snowflake.com/en/release-notes/new-features-2026"><em>Snowflake 2026 release notes</em></a></p><h3>Agentic Execution: Autonomy Changes the Security Model</h3><p>Moving from assistance to autonomous execution introduces a different class of operational risk.</p><p>Snowflake’s Cortex Agents and CoCo illustrate complementary approaches.</p><p>Cortex Agents orchestrate analytical and application-oriented workflows through supported tools, including Cortex Analyst, Cortex Search, code execution, and custom integrations.</p><p>CoCo focuses on engineering work, helping developers investigate environments, generate code, and perform supported development and operational tasks.</p><p>The distinction matters because they are not interchangeable execution environments.</p><p><strong>CoCo Automations</strong>, which reached General Availability on November 2, allow scheduled, unattended agent execution in Snowflake-managed environments. Runs can be inspected through their execution history and associated Cortex threads.</p><p>The most interesting engineering detail is not scheduling. It is <strong>execution identity</strong>.</p><p>Snowflake documents that an automation executes as the user who created it, using that user’s default primary role and default secondary roles — not necessarily the role active when the automation was configured.</p><p>This creates a subtle least-privilege risk.</p><p>An engineer may configure an automation while working under a restricted role, yet the unattended execution could have broader access if the user’s default secondary roles expose additional privileges.</p><p>Furthermore, interactive tool-approval prompts are disabled for unattended runs. Tools available to the automation can execute without waiting for a human response.</p><p>The resulting security model demands deliberate design: restrict the execution identity’s effective privileges, limit attached tools and secrets, and introduce approval boundaries for consequential changes.</p><p>GitHub-based workflows can provide a useful separation between an agent proposing a change and that change reaching production. But branch protection, pull-request review, and deployment permissions must be enforced by the engineering workflow itself.</p><p>There is also a reliability distinction worth preserving: a successful automation run means the agent task completed, not necessarily that every intended external action succeeded.</p><p><strong>Autonomy shifts the control problem from approving individual interactions to governing identities, permissions, side effects, and outcomes.</strong></p><p>That is a more consequential architectural change than simply running an assistant on a schedule.</p><p><strong><em>Official references:</em></strong><em> </em><a href="https://docs.snowflake.com/en/user-guide/cortex-code/cortex-code-automations"><em>CoCo Automations documentation</em></a><em> · </em><a href="https://docs.snowflake.com/en/release-notes/2026/other/2026-11-02-cortex-code-automations-ga"><em>CoCo Automations GA release</em></a><em> · </em><a href="https://docs.snowflake.com/en/user-guide/snowflake-cortex/cortex-agents"><em>Cortex Agents documentation</em></a></p><h3>Trusted and Fresh Data: Real-Time Is Not the Same as Reliable</h3><p>Enterprise agents need data appropriate to the decisions they make.</p><p>Some analytical workflows can tolerate yesterday’s snapshot. Operational workflows may require information reflecting changes from seconds ago.</p><p>Snowflake’s developments across Snowpipe Streaming, Interactive Analytics, Apache Iceberg, and Spark interoperability support different aspects of this foundation.</p><p>But a faster data pipeline does not automatically produce more trustworthy decisions.</p><p>Consider Snowpipe Streaming.</p><p><strong>Elastic Channels</strong>, generally available since September 15, provide server-managed ingestion with <strong>at-least-once delivery</strong>. Retries after ambiguous failures can produce duplicates, and ordering is not guaranteed.</p><p><strong>Named Channels</strong> support ordered, exactly-once ingestion within each channel when applications correctly coordinate source offsets, retain replayable source data, and recover from the last committed offset.</p><p>This distinction matters when agents consume operational data. An agent acting on duplicated transactions or incomplete updates can make an incorrect decision even when the data appears fresh.</p><p>The same principle applies to interoperability. Iceberg and Spark capabilities offer flexibility in how data is stored, processed, and accessed, but their availability and operational guarantees must be evaluated feature by feature.</p><p><strong>Freshness is a latency property; trustworthiness is an end-to-end data engineering property.</strong></p><p>For agentic systems, data quality, delivery guarantees, consistency, and lineage are as important as the speed at which information becomes queryable.</p><p><strong><em>Official references:</em></strong><em> </em><a href="https://docs.snowflake.com/en/user-guide/snowpipe-streaming/snowpipe-streaming-choosing-channel-type"><em>Choosing a Snowpipe Streaming channel</em></a><em> · </em><a href="https://docs.snowflake.com/en/release-notes/2026/other/2026-09-15-snowpipe-streaming-elastic-channels-ga"><em>Elastic Channels GA release</em></a><em> · </em><a href="https://docs.snowflake.com/en/user-guide/snowpipe-streaming/snowpipe-streaming-channels"><em>Named Channels and exactly-once delivery</em></a></p><h3>Production Reality: GA Components Do Not Equal a GA Architecture</h3><p>The architectural direction is promising, but product maturity must be assessed at the capability level.</p><p>A generally available Gateway endpoint, a Private Preview ontology, and an autonomous engineering workflow do not automatically combine into a production-ready enterprise architecture.</p><p>Each component has its own access model, supported integrations, operational behavior, and regional availability.</p><p>AI model availability varies across clouds and regions. Cross-region inference can introduce additional data-processing considerations. Some Gateway capabilities have narrower availability than its GA core, and Preview functionality may change before broader release.</p><p>The same caution applies to reliability.</p><p>An agent can use an authorized tool and still perform an unintended operation. It can query a correctly defined metric but select the wrong business interpretation. It can complete a task successfully while leaving an external system unchanged.</p><p>These are not necessarily failures of individual Snowflake services. They are failures at the boundaries between systems, definitions, permissions, and outcomes.</p><p><strong>Production readiness is an end-to-end property of the implemented workflow, not the sum of individual GA labels.</strong></p><p>The enterprise architect’s responsibility is to make those boundaries explicit and test them under realistic conditions.</p><p><strong><em>Official references:</em></strong><em> </em><a href="https://docs.snowflake.com/en/user-guide/snowflake-cortex/governance-and-availability"><em>Snowflake AI governance and availability</em></a><em> · </em><a href="https://docs.snowflake.com/en/user-guide/snowflake-cortex/cortex-ai-gateway"><em>Cortex AI Gateway documentation</em></a></p><h3>What Enterprise Data Teams Should Do Next</h3><p>The practical response is not to redesign an entire data platform around autonomous agents.</p><p>A better starting point is one bounded workflow with a measurable outcome and clearly understood consequences.</p><p>Consider an agent that investigates a failed data pipeline, gathers relevant query and task history, and proposes a corrective change through a pull request.</p><p>Its value can be evaluated without immediately granting permission to modify production.</p><p>The team must first define which operational evidence the agent can trust, which identity it executes under, which actions are permitted, and where human approval is required. Monitoring should capture not only agent completion but also the correctness of proposed changes, actual outcomes, failures, and cost.</p><p>This creates a controlled progression from assistance to limited autonomy, supported by evidence rather than assumptions.</p><p>Snowflake Expedition 2026 highlights meaningful progress toward this direction. Business context can improve interpretation, AI controls can govern supported execution paths, engineering agents can perform increasingly complex work, and the data platform can provide a reliable foundation.</p><p>Yet the most important work remains architectural.</p><p><strong>The future of enterprise AI will depend not only on how intelligent agents become, but on how deliberately we engineer the systems that allow them to act.</strong></p><p>That is where Data Engineering, Data Architecture, Governance, and AI Platform Engineering increasingly converge.</p><p><em>Technical note: This article presents an independent architectural interpretation based on Snowflake documentation and announcements available as of November 9, 2026. Product availability varies by capability, cloud, and region. The conceptual architecture is not an official Snowflake reference implementation.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9efbbc371946" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Protecting Sensitive Clinical Data in Snowflake: A Governance-Led Approach]]></title>
            <link>https://omid-karami.medium.com/protecting-sensitive-clinical-data-in-snowflake-a-governance-led-approach-4072e344ccb1?source=rss-26d5566e003e------2</link>
            <guid isPermaLink="false">https://medium.com/p/4072e344ccb1</guid>
            <category><![CDATA[healthcare]]></category>
            <category><![CDATA[data-governance]]></category>
            <category><![CDATA[snowflake]]></category>
            <dc:creator><![CDATA[Omid Karami]]></dc:creator>
            <pubDate>Tue, 06 Oct 2026 07:48:41 GMT</pubDate>
            <atom:updated>2026-10-06T07:48:41.233Z</atom:updated>
            <content:encoded><![CDATA[<p><em>From manual classification and tagging to masking and protection that follows the data</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gl2VvFf3piNvNtz9IlG4xw.png" /><figcaption><em>AI-generated with ChatGPT by OpenAI.</em></figcaption></figure><p><strong>Healthcare data</strong> has enormous potential. It can support <strong>clinical research</strong>, improve <strong>products</strong>, generate better <strong>insights</strong>, and enable more informed decisions. But the same data can also be <strong>highly sensitive</strong>.</p><p>A participant identifier, date of birth, clinical measurement, device identifier, or combination of otherwise ordinary attributes may reveal information about an individual. Protecting <strong>clinical data</strong> cannot be treated as a final security check after a platform has been built. It needs to be part of the platform’s design from the beginning.</p><p>At <a href="https://www.molnlycke.com/en-gb/">Mölnlycke Health Care</a>, we are building a governed <strong>clinical and digital product data platform</strong> with Snowflake. One of the central questions behind this work is simple:</p><blockquote><em>How can we make valuable data available for trusted use while maintaining strong control over sensitive information?</em></blockquote><p>The answer starts with understanding the data through collaboration between technical teams, data owners, clinical experts, privacy specialists, and security stakeholders.</p><h3>Sensitive data requires business and clinical context</h3><p>Some sensitive columns are easy to recognize. Names such as PATIENT_NAME, EMAIL_ADDRESS, or DATE_OF_BIRTH strongly indicate what the data contains.</p><p>Real data platforms, however, are rarely that consistent. The same type of information might appear across different systems under names such as SUBJECT_ID, PARTICIPANT_REF, or PERSON_KEY. A column may not identify someone on its own, but it could become identifying when combined with location, age, treatment date, or another attribute.</p><p>This is why classification cannot depend only on technical pattern matching. Clinical and business knowledge is essential. The people who understand how data is collected, combined, and used are often best placed to explain its real sensitivity.</p><h3>Start with a manual, governance-led process</h3><p>For a <strong>clinical data platform</strong>, a controlled manual approach can provide a strong foundation. The goal is not simply to label columns. It is to agree on what the data means, why it needs protection, and who should be allowed to use it.</p><p>A practical process can include:</p><ol><li><strong>Inventory the data.</strong> Identify relevant datasets, tables, views, and columns across the platform.</li><li><strong>Review the business meaning.</strong> Work with data owners, clinical experts, and source-system specialists to understand each field’s content and purpose.</li><li><strong>Assess sensitivity.</strong> Determine whether a field is a direct identifier, quasi-identifier, sensitive clinical attribute, or non-sensitive information.</li><li><strong>Assign governance tags.</strong> Apply an agreed organizational tag and sensitivity value to relevant columns.</li><li><strong>Connect tags to protection policies.</strong> Use masking policies to control what different roles can see.</li><li><strong>Validate the result.</strong> Test the experience from authorized and unauthorized roles before releasing the data product.</li><li><strong>Review it over time.</strong> Update classifications and controls when schemas, data sources, regulations, or business purposes change.</li></ol><p>This process may require more effort initially, but it creates shared understanding and clear accountability. It also establishes a trusted baseline before controls are applied more widely.</p><h3>Classification is not the same as protection</h3><p>Classification tells us what the data is. A tag records that knowledge. A protection policy determines what a user is allowed to see.</p><p>For example, an organization might define a user-managed tag such as:</p><p>DATA_SENSITIVITY = HIGHLY_SENSITIVE</p><p>That tag can be manually assigned to participant identifiers or other sensitive clinical columns after review. A <a href="https://docs.snowflake.com/en/sql-reference/sql/create-masking-policy"><strong>masking policy </strong></a>can then be attached to the tag.</p><p>When a user queries a protected column, Snowflake evaluates the masking policy at query time. An approved clinical role may see the original value, while another role receives a masked or transformed value.</p><p>This creates a clear governance chain:</p><p><strong>Understand → Classify → Tag → Protect → Propagate → Validate → Monitor</strong></p><h3>A simplified clinical data platform example</h3><p>Consider an illustrative dataset arriving from a digital health application. It contains:</p><ul><li>A participant reference</li><li>Age range and geographic region</li><li>Wound measurements recorded over time</li><li>Treatment and assessment dates</li><li>Product and device information</li></ul><p>The participant reference might be treated as a direct identifier. Clinical measurements would likely be <strong>sensitive health data</strong>. Age range or region could be quasi-identifiers: they might not identify a person independently, but they can increase re-identification risk when combined with other attributes.</p><p>During <strong>governance review</strong>, the team can document these decisions and apply the appropriate tags manually. Direct identifiers may receive a HIGHLY_SENSITIVE value, while clinical measurements may receive a SENSITIVE value. The precise categories should follow the organization’s agreed classification model and privacy requirements.</p><p>Those <strong>tags</strong> can then activate protection rules. Direct identifiers can be masked for most users. Sensitive clinical attributes can be visible only to approved roles. Quasi-identifiers can require additional review before a data product is shared more widely. Data owners and stewards can approve access based on purpose.</p><p>This allows an analyst to use the clinical measurements needed for an approved use case without automatically receiving access to every identifying attribute.</p><p>Access should be based on purpose, not simply on the fact that data is technically available.</p><h3>Protection should follow the data</h3><p>Clinical data rarely remains in one table. It moves through ingestion, transformation, modelling, and consumption layers. A sensitive source column might later appear in a curated table, view, secure view, dynamic table, or reusable data product.</p><p>Manually classifying and tagging the source is only the first step. If the tag disappears during a downstream transformation, the protection model becomes incomplete.</p><p>Snowflake’s automatic tag propagation for user-defined tags can carry tags to supported downstream objects. Propagation can follow:</p><ul><li><strong>Object dependencies</strong>, such as views, secure views, materialized views, and dynamic tables created from a tagged source.</li><li><strong>Data movement</strong>, including operations such as CREATE TABLE AS SELECT, INSERT, MERGE, UPDATE, and COPY INTO.</li><li><strong>Both</strong>, when a tag is configured with ON_DEPENDENCY_AND_DATA_MOVEMENT.</li></ul><p>Dependency-based propagation is continuous: when a source tag is added, changed, or removed, the dependent object is updated. Tags propagated through data movement are not continuously synchronized. A later source-tag change is reflected only after another supported data-movement operation. A manually applied tag on the target takes precedence over a propagated value.</p><p>Conflict handling also needs to be planned. If multiple sources with different sensitivity values feed the same target, ON_CONFLICT = ALLOWED_VALUES_SEQUENCE uses the order of the tag’s allowed values to resolve the conflict. With the order shown in the SQL example below, the first value is treated as the highest sensitivity. Keep that ordering deliberate: changing it changes how future conflicts are resolved.</p><p><strong>Materialized view note:</strong> Tag propagation supports materialized views, but tag-based masking has a related limitation. A materialized view cannot be created over a table protected by a tag-based masking policy; adding the policy later can also make an existing materialized view unqueryable. Validate this design before using materialized views in a protected data flow.</p><h3>From governance decision to technical control</h3><p>The following simplified SQL demonstrates a manual pattern. Names are illustrative and do not represent Mölnlycke’s internal configuration. Tag propagation and masking policies require <a href="https://www.snowflake.com/en/">Snowflake</a> Enterprise Edition or higher.</p><p>This example assumes the CLINICAL database and GOVERNANCE and CURATED schemas exist, the table and column exist, PARTICIPANT_REFERENCE is a STRING, and the CLINICAL_DATA_AUTHORIZED role has been created and granted appropriately.</p><pre>-- Illustrative example only — not a production configuration.<br><br>-- Create the tag with values ordered from highest to lowest sensitivity.<br>CREATE TAG IF NOT EXISTS clinical.governance.data_sensitivity<br>  ALLOWED_VALUES &#39;HIGHLY_SENSITIVE&#39;, &#39;SENSITIVE&#39;, &#39;INTERNAL&#39;, &#39;PUBLIC&#39;<br>  PROPAGATE = ON_DEPENDENCY_AND_DATA_MOVEMENT<br>  ON_CONFLICT = ALLOWED_VALUES_SEQUENCE;<br><br>-- Create a masking policy for STRING columns.<br>CREATE MASKING POLICY IF NOT EXISTS clinical.governance.mask_sensitive_string<br>  AS (value STRING) RETURNS STRING -&gt;<br>    CASE<br>      WHEN IS_ROLE_IN_SESSION(&#39;CLINICAL_DATA_AUTHORIZED&#39;) THEN value<br>      ELSE &#39;********&#39;<br>    END;<br><br>-- Attach the STRING masking policy to the tag.<br>ALTER TAG clinical.governance.data_sensitivity<br>  SET MASKING POLICY clinical.governance.mask_sensitive_string;<br><br>-- Apply the agreed classification after governance review.<br>ALTER TABLE clinical.curated.participant_observations<br>  MODIFY COLUMN participant_reference<br>  SET TAG clinical.governance.data_sensitivity = &#39;HIGHLY_SENSITIVE&#39;;</pre><p>This SQL demonstrates masking for a STRING identifier only. The policy checks the authorized role; every value of this tag that it protects follows that same role-based rule. A broader implementation may require separate masking policies for other data types and additional policy logic for different sensitivity values.</p><p>If the tag already exists, CREATE TAG IF NOT EXISTS leaves its existing settings unchanged. Review and update the existing tag explicitly before relying on its propagation and conflict-resolution configuration.</p><h3>Monitoring is part of the control</h3><p>Governance does not end when a tag or policy is applied. Teams should be able to answer:</p><ul><li>Which objects and columns carry a sensitivity tag?</li><li>Who approved the classification?</li><li>Which masking policies are active?</li><li>Did a tag fail to propagate?</li><li>Was propagation skipped because a target had a manually assigned tag?</li><li>Did conflicting tag values arrive from different sources?</li><li>Do authorized and unauthorized roles still receive the expected results?</li></ul><p>Snowflake provides <strong>tag</strong> and <strong>policy</strong> reference views for inspecting technical assignments. Tag-propagation events can also be collected in an event table. To collect them, an administrator must enable ENABLE_TAG_PROPAGATION_EVENT_LOGGINGan event table must also be configured for the tag’s database, or the default event table is used.</p><p>Governance records, periodic reviews, and clear ownership should complement these technical controls.</p><h3>Building trust into the platform</h3><p>The purpose of <strong>data governance</strong> is not to make data inaccessible. It is to make responsible use possible.</p><p>When sensitive data is understood, deliberately <strong>classified</strong>, <strong>tagged</strong>, <strong>protected</strong>, and <strong>monitored</strong> as it moves through a platform, governance becomes part of the architecture rather than an afterthought.</p><p>For a <strong>clinical data platform</strong>, this provides a clearer path to trusted analytics and responsible innovation. <strong>Analysts</strong> can access data appropriate to their purpose, <strong>platform teams</strong> can apply controls consistently, and <strong>data owners</strong> can make decisions with better context.</p><p>The most important question is not simply whether an organization can detect sensitive data. It is whether people understand the data well enough to make the right governance decisions — and whether those decisions continue to protect it wherever it is used.</p><p>That is how classification becomes more than a label, and how trust becomes a platform capability.</p><p><em>The clinical example in this article is simplified and illustrative. It does not describe Mölnlycke’s internal schemas, patient data, security configuration, or production implementation.</em></p><h3>Further reading</h3><ul><li><a href="https://docs.snowflake.com/en/user-guide/classify-intro">Snowflake: Introduction to sensitive data classification</a></li><li><a href="https://docs.snowflake.com/en/user-guide/object-tagging/introduction">Snowflake: Introduction to object tagging</a></li><li><a href="https://docs.snowflake.com/en/user-guide/tag-based-masking-policies">Snowflake: Tag-based masking policies</a></li><li><a href="https://docs.snowflake.com/en/user-guide/object-tagging/propagation">Snowflake: Automatic tag propagation with user-defined tags</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4072e344ccb1" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[From dbt Core v1 to dbt v2 on Snowflake: A Practical Migration Guide]]></title>
            <link>https://medium.com/snowflake/from-dbt-core-v1-to-dbt-v2-on-snowflake-a-practical-migration-guide-4eb137573a1f?source=rss-26d5566e003e------2</link>
            <guid isPermaLink="false">https://medium.com/p/4eb137573a1f</guid>
            <category><![CDATA[dbt]]></category>
            <category><![CDATA[data-engineering]]></category>
            <category><![CDATA[sql]]></category>
            <category><![CDATA[snowflake]]></category>
            <category><![CDATA[dbt-core]]></category>
            <dc:creator><![CDATA[Omid Karami]]></dc:creator>
            <pubDate>Thu, 24 Sep 2026 14:01:04 GMT</pubDate>
            <atom:updated>2026-09-24T14:01:04.939Z</atom:updated>
            <content:encoded><![CDATA[<h4>A practical guide to preparing an existing Snowflake dbt project for v2, validating compatibility, and handling static-analysis gaps.</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4uxMnFM1tnVxa730Uj6n1w.png" /><figcaption>AI-generated image created using ChatGPT by OpenAI.</figcaption></figure><p><strong>dbt v2</strong> became generally available on September 16, 2026 (<a href="https://docs.getdbt.com/blog/dbt-v2-is-ga">dbt Labs’ announcement</a>), bringing the engine developed under the “Fusion” codename into the productized version of dbt.</p><p>When I started looking into what it would take to move an existing <strong>Snowflake dbt project</strong> to v2, I expected the engine change itself to be the difficult part. What became more important was the preparation: getting the project ready on v1 first, checking compatibility before switching, and separating documented migration guidance from the smaller environment and SQL issues you may discover along the way.</p><p>This post focuses on that practical migration path.</p><p><strong>By the end of this post, you’ll know:</strong></p><ul><li>The core architectural change between dbt v1 and dbt v2</li><li>What dbt v2 adds, including static analysis and performance improvements</li><li>dbt Labs’ recommended migration sequence — prepare on v1, then switch, then validate</li><li>One Snowflake SQL compatibility gap worth checking after switching</li></ul><h3>TL;DR</h3><p>For an existing Snowflake dbt project, the migration path is:</p><p><strong>Upgrade to dbt v1.12 → resolve deprecations and check package compatibility → run</strong> <strong>dbt parse --use-v2-parser</strong> <strong>→ switch to dbt v2 → compile and test the project.</strong></p><p>The key is to use v1.12 as the preparation step rather than treating the migration as a direct package replacement. After switching, static analysis may also surface warehouse-specific SQL that v1 did not flag, so validate the full project and distinguish parser limitations from actual Snowflake runtime issues.</p><h3>Migration path at a glance</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*AeQ-uD3VMSFDCoGZ0EKPWQ.png" /><figcaption>AI-generated image created using ChatGPT by OpenAI.</figcaption></figure><h3>What actually changes</h3><p>In dbt v1, the toolchain is split into two independently versioned packages: dbt-core (the Python engine — Jinja rendering, DAG resolution, the CLI) and a separate adapter package per warehouse, like dbt-snowflake. These ship on their own release cadences and coordinate through a defined adapter interface, so compatibility between the core engine and the warehouse adapter needs to be managed as part of the v1 setup.</p><p>dbt v2 collapses that into a single package. Per dbt Labs, “all v2 adapters connect to data warehouses via the <a href="https://arrow.apache.org/adbc/">Arrow Database Connectivity (ADBC)</a> standard instead of Python-based adapter libraries” (<a href="https://docs.getdbt.com/docs/dbt-versions/dbt-upgrade/upgrading-to-v2">Upgrading to v2</a>). For Snowflake, that means no separate adapter package to install or version-match:</p><pre>- dbt-core&gt;=1.10,&lt;1.11<br>- dbt-snowflake&gt;=1.10,&lt;1.11<br>+ dbt&gt;=2.0,&lt;3.0</pre><p>dbt v2 also adds a second compilation phase beyond Jinja rendering: static analysis, which builds and validates a logical query plan for your SQL before anything runs against the warehouse. That’s what unlocks column-level lineage and stronger validation — and it’s the source of the compatibility note near the end of this post.</p><h3>Why move to dbt v2?</h3><p>The migration work brings meaningful improvements. dbt v2 introduces local SQL compilation and static analysis, allowing more SQL errors and downstream impacts to be identified before queries reach Snowflake. Because dbt understands the project graph as well as the SQL itself, this analysis can also support capabilities such as column-level lineage and help surface the downstream impact of changes earlier in the development cycle.</p><p>The <strong>Rust-based</strong> engine is also designed to improve parsing and compilation performance, which can make a noticeable difference in larger or highly customized dbt projects. dbt v2 also integrates with <a href="https://docs.getdbt.com/docs/deploy/dbt-state-about?version=2">dbt State</a>, helping reduce unnecessary execution by identifying models that can be skipped or, where applicable, cloned instead of rebuilt.</p><p>Together, these changes can shorten the development feedback loop and catch more problems before warehouse execution — increasingly useful as both developers and coding agents generate SQL more quickly.</p><h3>The recommended migration path</h3><p>dbt Labs documents a specific sequence for moving an existing v1 project to v2, and it’s worth following in order rather than jumping straight to installing v2.</p><h4>1. Prepare while still on v1</h4><p>The starting point isn’t your current v1 version — it’s the latest one. dbt Labs is explicit about this:</p><blockquote><em>“The best way to prepare to migrate is to first upgrade to dbt v1.12 (or the ‘Latest’ release track in the dbt platform).” — </em><a href="https://docs.getdbt.com/blog/dbt-core-v2-is-here"><em>dbt Core v2 is here</em></a></blockquote><p>v1.12 matters specifically because it enforces many of the behavior changes that become mandatory in v2, and it ships with an opt-in flag that lets you test v2’s parser without actually switching engines yet.</p><p>Once you’re on v1.12, three things are worth doing before you touch v2 at all:</p><ul><li><strong>Resolve deprecation warnings.</strong> The upgrade guide treats this as a precondition: “All deprecation warnings must be resolved before upgrading to the new engine.” Many of these are auto-fixable with <a href="https://github.com/dbt-labs/dbt-autofix">dbt-autofix</a>, a tool dbt Labs publishes specifically for this.</li><li><strong>Check third-party package compatibility</strong> — via each package’s require-dbt-version setting or the v2-compatible badge on the dbt package hub.</li><li><strong>Test the v2 parser without switching engines</strong>, using the opt-in flag available on v1.12:</li></ul><pre>dbt parse --use-v2-parser</pre><p>This “delegates parsing to the v2 parser without changing any other behavior, making it a low-risk way to catch compatibility issues early” (Upgrading to v2). Fix whatever it surfaces while you’re still safely on v1.</p><h4>2. Switch to v2</h4><p>dbt Labs’ own signal for when you’re ready is straightforward: once that parser check passes cleanly, install v2.</p><blockquote><em>“Run </em><em>dbt parse --use-v2-parser to try it out. If your project succeeds, you’re good to go, and should check out the installation guide to switch over.” — </em><a href="https://docs.getdbt.com/blog/dbt-core-v2-is-here"><em>dbt Core v2 is here</em></a></blockquote><p>For a fresh environment, <a href="https://docs.getdbt.com/docs/local/install-dbt?version=2">install dbt</a> via pip, Homebrew, or a curl-based install. Homebrew/curl install a self-contained binary outside of any Python environment; pip installs dbt as a Python package into whatever environment is active. dbt Labs’ own preference leans toward the binary install, but pip is fully supported and the right choice if your tooling specifically needs dbt resolvable from a Python environment — for example, some IDE integrations select dbt from a project’s virtual environment rather than from $PATH.</p><p>One practical note for existing pip-based environments: based on our migration experience, we recommend recreating the virtual environment rather than layering dbt v2 onto an environment that already contains the v1 Python packages. This is a practical recommendation rather than a documented dbt Labs migration requirement, and it gives the project a clean, reproducible environment based on its declared dependencies:</p><pre>rm -rf .venv<br>python -m venv .venv<br>source .venv/bin/activate<br>pip install -r requirements.txt</pre><h4>3. Validate on v2</h4><p>After switching, run a full project compile, not just a spot check — v2’s stricter parsing surfaces things v1 never checked, and it’s better to see all of them at once. Then run your actual test suite (dbt build or dbt test) to confirm runtime behavior, since static analysis alone doesn&#39;t execute anything against the warehouse.</p><p>Static analysis defaults to baseline mode, where findings are warnings, not build-breaking errors:</p><blockquote><em>“Your project can continue running even when the compiler flags invalid or problematic SQL.” — </em><a href="https://docs.getdbt.com/docs/build/about-static-analysis?version=2"><em>About static analysis</em></a></blockquote><p>That same page treats unsupported SQL as an expected, normal part of migrating, and asks users to report gaps so parsers can improve.</p><h3>A compatibility gap worth checking for</h3><p>One specific gap we ran into during validation: dbt v2’s parser doesn’t yet fully understand Snowflake’s GET_PRESIGNED_URL() function when the stage argument is passed as a bare identifier. This isn&#39;t just something we noticed — dbt Labs&#39; own <a href="https://docs.getdbt.com/reference/resource-configs/snowflake-function-support">Supported Snowflake functions</a> reference explicitly lists GET_PRESIGNED_URL as unsupported for type-checking today.</p><p>Snowflake’s <a href="https://docs.snowflake.com/en/sql-reference/functions/get_presigned_url">syntax documentation</a> shows the unquoted form as the standard example:</p><pre>GET_PRESIGNED_URL(@my_stage, &#39;path/to/file.csv&#39;, 3600)</pre><p>Quoting the stage name is only documented as necessary when it contains spaces or special characters. In practice, wrapping the stage reference in quotes satisfies dbt v2’s current parser and clears the warning, without changing behavior:</p><pre>-- Compiles cleanly under dbt v2&#39;s current baseline static analysis<br>GET_PRESIGNED_URL(&#39;@my_stage&#39;, &#39;path/to/file.csv&#39;, 3600)</pre><p>In our validation, this behaved as a parser-satisfying change rather than a runtime behavior change: both syntaxes returned identical presigned URLs against Snowflake, and running the actual model and fetching the resulting file confirmed it resolved correctly at runtime.</p><p>Worth framing this precisely: it’s a workaround for where dbt v2’s parser coverage happens to be today, not a Snowflake-documented requirement or an official dbt fix. If you hit something similar, the better long-term move — per dbt Labs’ own guidance — is to <a href="https://github.com/dbt-labs/dbt-fusion/issues">open an issue</a> with the specific SQL that failed to parse.</p><h3>Practical recommendations</h3><ul><li><strong>Don’t skip the v1.12 stepping stone.</strong> Testing with --use-v2-parser while still on v1 is a low-risk way to find compatibility issues, and dbt Labs built it for exactly this purpose.</li><li><strong>Run</strong> <strong>dbt-autofix</strong> <strong>before switching</strong>, not after — it handles many deprecations automatically.</li><li><strong>For existing pip-based setups, consider recreating the virtual environment</strong> rather than layering v2 onto an existing v1 environment. Treat this as a practical environment-management recommendation, not a dbt Labs migration requirement.</li><li><strong>After switching, compile the whole project and run your test suite</strong> — not just a single model — and don’t assume a static-analysis warning means broken SQL. Check whether the function you’re using appears in dbt’s Snowflake function-support reference before spending time on a workaround.</li></ul><h3>Conclusion</h3><p>dbt Labs’ documented path — prepare on the latest v1 release, validate with the v2 parser before switching, then confirm on v2 with a full compile and test run — turns what could be a risky jump into a predictable, low-drama migration. One area worth watching is SQL comprehension coverage for warehouse-specific functions. Under the default baseline static-analysis mode, unsupported or problematic SQL can surface as warnings without necessarily blocking the build, giving teams a chance to investigate parser gaps separately from actual runtime failures.</p><h3>References</h3><ul><li><a href="https://docs.getdbt.com/blog/dbt-v2-is-ga">dbt v2 is GA</a> — dbt Labs’ official GA announcement (September 16, 2026)</li><li><a href="https://docs.getdbt.com/blog/dbt-core-v2-is-here">dbt Core v2 is here: still open source, now rebuilt for what’s next</a> — the recommended v1.12-first migration path and the --use-v2-parser readiness check</li><li><a href="https://docs.getdbt.com/docs/dbt-versions/dbt-upgrade/upgrading-to-v2">Upgrading to v2</a> — official upgrade guide: ADBC adapter architecture, deprecation-resolution requirements, package compatibility</li><li><a href="https://docs.getdbt.com/docs/local/install-dbt?version=2">Install dbt</a> — official install instructions for dbt v2 (pip, Homebrew, curl)</li><li><a href="https://docs.getdbt.com/docs/build/about-static-analysis?version=2">About static analysis</a> — baseline vs. strict static analysis modes and guidance for unsupported SQL</li><li><a href="https://docs.getdbt.com/reference/resource-configs/snowflake-function-support">Supported Snowflake functions in dbt v2</a> — official function-support matrix, refreshed weekly</li><li><a href="https://github.com/dbt-labs/dbt-autofix">dbt-autofix</a> — official dbt Labs tool for automating v1-to-v2 deprecation fixes</li><li><a href="https://github.com/dbt-labs/dbt-fusion/issues">dbt Fusion GitHub Issues</a> — official channel for reporting SQL that dbt v2’s parser doesn’t yet support</li><li><a href="https://docs.snowflake.com/en/sql-reference/functions/get_presigned_url">Snowflake: GET_PRESIGNED_URL</a> — official syntax reference</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4eb137573a1f" width="1" height="1" alt=""><hr><p><a href="https://medium.com/snowflake/from-dbt-core-v1-to-dbt-v2-on-snowflake-a-practical-migration-guide-4eb137573a1f">From dbt Core v1 to dbt v2 on Snowflake: A Practical Migration Guide</a> was originally published in <a href="https://medium.com/snowflake">Snowflake Builders Blog: Data Engineers, App Developers, AI, &amp; Data Science</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Snowflake Cortex Code Cost Management: Monitoring, Usage Limits, and Optimization]]></title>
            <link>https://omid-karami.medium.com/what-i-learned-about-ai-consumption-and-cost-management-in-snowflake-coco-79d3f16499a5?source=rss-26d5566e003e------2</link>
            <guid isPermaLink="false">https://medium.com/p/79d3f16499a5</guid>
            <category><![CDATA[snowflake-data-cloud]]></category>
            <category><![CDATA[snowflake]]></category>
            <category><![CDATA[data-superhero]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[snowflake-cortex]]></category>
            <dc:creator><![CDATA[Omid Karami]]></dc:creator>
            <pubDate>Sat, 15 Aug 2026 20:22:10 GMT</pubDate>
            <atom:updated>2026-11-19T08:30:16.733Z</atom:updated>
            <content:encoded><![CDATA[<p><em>A hands-on guide to tracking </em><strong><em>CoCo</em></strong><em> credit consumption, setting usage limits, analyzing model costs, and building automated monitoring in Snowflake.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*elL9yh4ry2NYcArIAO-WmQ.png" /><figcaption>Snowflake CoCo — AI-generated image created using ChatGPT by OpenAI.</figcaption></figure><h3>Introduction</h3><p>As AI-powered development tools become part of everyday data engineering workflows, understanding their consumption and cost implications is increasingly important.</p><p><strong>Snowflake Cortex Code (CoCo)</strong> introduces new ways to interact with data, write SQL, and accelerate development. But as adoption grows, so does the need for visibility into AI credit consumption, usage limits, and cost management.</p><p>This article provides a practical guide to <strong>monitoring Cortex Code consumption, understanding AI credit usage, configuring daily limits, and analyzing costs</strong> across different interfaces, including Snowsight, Desktop, and CLI.</p><p>I also introduce a <strong>Streamlit Cost Management application</strong> designed to help teams visualize AI credit consumption, identify usage patterns, and better understand the cost implications of Cortex Code within their Snowflake environment.</p><p>This exploration was initially inspired by Snowflake’s free <a href="https://learn.snowflake.com/en/courses/OD-COCOESS/"><strong>CoCo Essentials</strong></a> course on <a href="https://learn.snowflake.com/">learn.snowflake.com</a>, but the focus here is on practical implementation and applying these concepts to real-world cost management.</p><p>If you’re looking to get started with Cortex Code, I highly recommend checking out the course. <strong>It’s free, hands-on, and a great opportunity to build practical skills and strengthen your professional profile!</strong></p><h3>CoCo Is Not Free — It Consumes Cortex AI Credits</h3><p>The first thing to understand: every interaction with Snowflake CoCo — whether you’re asking it to explain code, fix errors, generate SQL, or build a Streamlit app — costs Cortex AI credits. These credits are deducted from your Snowflake account.</p><p>This applies to:</p><ul><li>Chat conversations in the CoCo side panel</li><li>Code explanations (the “Explain” button)</li><li>Error fixes (the “Fix error” button)</li><li>SQL generation and optimization</li><li>Creating and editing notebooks, pipelines, and apps</li><li>CoCo Desktop (the standalone app)</li><li>CoCo CLI (command-line interface)</li></ul><p>The credits are always charged to the <strong>active account and connection</strong>. If you have multiple Snowflake accounts or connections, the consumption is attributed to whichever one you’re currently using.</p><h3>Setting a Daily Credit Limit</h3><p>The course teaches an important safety measure: <strong>you can cap your daily CoCo usage</strong> to prevent runaway costs. This is done with a single SQL command:</p><pre>SET my_user = CURRENT_USER();<br>ALTER USER IDENTIFIER($my_user) <br>  SET CORTEX_CODE_SNOWSIGHT_DAILY_EST_CREDIT_LIMIT_PER_USER = 20;</pre><p>This sets a per-user daily limit of 20 estimated credits. Once you hit the limit, CoCo stops responding until the next day.</p><p><strong>Note:</strong> The daily credit limit shown here applies specifically to CoCo usage in Snowsight. CoCo usage through Desktop and CLI is tracked separately and may require separate monitoring or controls</p><p>For the entire CoCo Essentials course (5 labs), Snowflake estimates no more than <strong>20 credits</strong> total. That gives you a sense of scale — a full day of heavy AI-assisted development might consume 5–10 credits depending on complexity.</p><h3>Deep Dive: The Three Usage History Views</h3><p>Snowflake provides <strong>three dedicated views</strong> under <strong>SNOWFLAKE</strong>.<strong>ACCOUNT_USAGE</strong> to track CoCo consumption across every interface where the AI assistant runs. This is where the real observability lives.</p><h3>The Views</h3><ul><li><strong>CORTEX_CODE_SNOWSIGHT_USAGE_HISTORY</strong> — CoCo usage in the Snowsight web UI</li><li><strong>CORTEX_CODE_DESKTOP_USAGE_HISTORY</strong> — CoCo Desktop (standalone app) usage</li><li><strong>CORTEX_CODE_CLI_USAGE_HISTORY</strong> — CoCo CLI (command-line) usage</li></ul><p>All three views share the same schema:</p><ul><li><strong>USER_ID</strong> — Numeric ID of the user</li><li><strong>USER_NAME</strong> — Name of the user who made the request</li><li><strong>USER_TAGS</strong> — Array of tags assigned to the user (useful for team-based cost allocation)</li><li><strong>REQUEST_ID</strong> — Unique identifier for each CoCo request</li><li><strong>PARENT_REQUEST_ID</strong> — Links follow-up requests to the original (e.g., multi-turn conversations)</li><li><strong>USAGE_TIME</strong> — Timestamp of when the request was made</li><li><strong>TOKEN_CREDITS</strong> — Credits consumed by this request</li><li><strong>TOKENS</strong> — Total tokens (input + output) used</li><li><strong>TOKENS_GRANULAR</strong> — Object breaking down tokens by type (input vs. output)</li><li><strong>CREDITS_GRANULAR</strong> — Object breaking down credits by type</li><li><strong>METADATA</strong> — Additional context about the request</li></ul><h3>Why Three Separate Views?</h3><p>Snowflake separates these because each interface has different usage patterns and potentially different pricing:</p><ul><li><strong>Snowsight</strong> — used during interactive development in the browser</li><li><strong>Desktop</strong> — used by developers who prefer a standalone IDE-like experience</li><li><strong>CLI</strong> — used in automated workflows, scripts, or terminal-based development</li></ul><p>By splitting them, you can answer questions like: <em>“Are our developers burning more credits in the Desktop app than in Snowsight?”</em> or <em>“Is anyone running CoCo from CI/CD pipelines via the CLI?”</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GBf99IQ4HrVp2Z3ptwBp-A.png" /><figcaption><strong>Snowsight Interface</strong></figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*VrGlDMktdMrZUAb3oXDTcg.png" /><figcaption><strong>CoCo Desktop interface</strong></figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*D8dRznon69kyRzDJKo55Dg.png" /><figcaption><strong>CoCo CLI interface</strong></figcaption></figure><h3>Core Monitoring Queries</h3><h3>Query 1: Daily Credit Consumption (Last 30 Days)</h3><p>This is your go-to dashboard query. It shows daily credit burn across all CoCo interfaces combined:</p><pre>SELECT<br>    DATE_TRUNC(&#39;day&#39;, usage_time) AS usage_date,<br>    &#39;Snowsight&#39; AS source,<br>    SUM(token_credits) AS daily_credits,<br>    SUM(tokens) AS daily_tokens,<br>    COUNT(*) AS request_count<br>FROM SNOWFLAKE.ACCOUNT_USAGE.CORTEX_CODE_SNOWSIGHT_USAGE_HISTORY<br>WHERE usage_time &gt;= DATEADD(&#39;day&#39;, -30, CURRENT_TIMESTAMP())<br>GROUP BY 1<br><br>UNION ALL<br><br>SELECT<br>    DATE_TRUNC(&#39;day&#39;, usage_time),<br>    &#39;Desktop&#39;,<br>    SUM(token_credits),<br>    SUM(tokens),<br>    COUNT(*)<br>FROM SNOWFLAKE.ACCOUNT_USAGE.CORTEX_CODE_DESKTOP_USAGE_HISTORY<br>WHERE usage_time &gt;= DATEADD(&#39;day&#39;, -30, CURRENT_TIMESTAMP())<br>GROUP BY 1<br><br>UNION ALL<br><br>SELECT<br>    DATE_TRUNC(&#39;day&#39;, usage_time),<br>    &#39;CLI&#39;,<br>    SUM(token_credits),<br>    SUM(tokens),<br>    COUNT(*)<br>FROM SNOWFLAKE.ACCOUNT_USAGE.CORTEX_CODE_CLI_USAGE_HISTORY<br>WHERE usage_time &gt;= DATEADD(&#39;day&#39;, -30, CURRENT_TIMESTAMP())<br>GROUP BY 1<br><br>ORDER BY 1 DESC, 2;</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*n5DhlBraliICWPynRD1PeQ.png" /><figcaption>Credits by Interface</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*OfZkqm5u1qmj1xZoblUH7A.png" /><figcaption>Daily Credits by Source (Last 30 Days)</figcaption></figure><h3>Query 2: Per-User Credit Consumption</h3><p>Identify who is consuming the most AI credits on your team:</p><pre>-- Top 10 users by CoCo credit usage (last 7 days)<br>SELECT<br>    user_name,<br>    ROUND(SUM(token_credits), 4) AS total_credits,<br>    SUM(tokens) AS total_tokens,<br>    COUNT(*) AS total_requests,<br>    ROUND(SUM(token_credits) / COUNT(*), 6) AS avg_credits_per_request<br>FROM SNOWFLAKE.ACCOUNT_USAGE.CORTEX_CODE_SNOWSIGHT_USAGE_HISTORY<br>WHERE usage_time &gt;= DATEADD(&#39;day&#39;, -7, CURRENT_TIMESTAMP())<br>GROUP BY user_name<br>ORDER BY total_credits DESC<br>LIMIT 10;</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*SHSylqDnSEWfRKNz4lhLZg.png" /><figcaption>Top Users by Credits (Last 7 Days)</figcaption></figure><h3>Query 3: Credit Consumption by Conversation (Multi-Turn Tracking)</h3><p>The PARENT_REQUEST_ID column lets you group related requests into conversations. This reveals which tasks are the most expensive. The version I ended up shipping in the Streamlit app goes a step further than a simple group-by: it flattens TOKENS_GRANULAR per request so you can see the input/output/cache token split for each conversation, and it identifies the <em>dominant model</em> used within that conversation (ranked by total tokens consumed per model):</p><pre>-- Most expensive conversations (grouped by parent request), with token<br>-- breakdown and dominant model per conversation<br>WITH flat AS (<br>    SELECT<br>        COALESCE(parent_request_id, request_id) AS conversation_id,<br>        user_name,<br>        usage_time,<br>        token_credits,<br>        tokens,<br>        f.key AS model,<br>        COALESCE(f.value:&quot;input&quot;::NUMBER, 0) AS input_tok,<br>        COALESCE(f.value:&quot;output&quot;::NUMBER, 0) AS output_tok,<br>        COALESCE(f.value:&quot;cache_read_input&quot;::NUMBER, 0) AS cache_read_tok,<br>        COALESCE(f.value:&quot;cache_write_input&quot;::NUMBER, 0) AS cache_write_tok<br>    FROM SNOWFLAKE.ACCOUNT_USAGE.CORTEX_CODE_SNOWSIGHT_USAGE_HISTORY,<br>         LATERAL FLATTEN(input =&gt; tokens_granular) f<br>    WHERE usage_time &gt;= DATEADD(&#39;day&#39;, -7, CURRENT_TIMESTAMP())<br>),<br>model_rank AS (<br>    SELECT conversation_id, model,<br>           ROW_NUMBER() OVER (<br>               PARTITION BY conversation_id<br>               ORDER BY SUM(input_tok + output_tok + cache_read_tok + cache_write_tok) DESC<br>           ) AS rn<br>    FROM flat<br>    GROUP BY conversation_id, model<br>)<br>SELECT<br>    f.user_name,<br>    MIN(f.usage_time) AS started_at,<br>    COUNT(DISTINCT f.usage_time) AS turns,<br>    MAX(CASE WHEN mr.rn = 1 THEN mr.model END) AS top_model,<br>    SUM(f.input_tok) AS input_tokens,<br>    SUM(f.output_tok) AS output_tokens,<br>    SUM(f.cache_read_tok) AS cache_read_tokens,<br>    SUM(f.cache_write_tok) AS cache_write_tokens,<br>    SUM(f.tokens) AS total_tokens,<br>    ROUND(SUM(f.token_credits), 4) AS total_credits<br>FROM flat f<br>JOIN model_rank mr ON f.conversation_id = mr.conversation_id<br>GROUP BY f.conversation_id, f.user_name<br>ORDER BY total_tokens DESC<br>LIMIT 20;</pre><p>This is powerful — you can identify patterns like “pipeline creation always costs 2–3 credits” vs. “simple SQL fixes cost 0.01 credits,” and now also <em>which model</em> was doing the heavy lifting in each conversation and how much of the cost came from cache reads/writes vs. fresh input/output tokens.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*hErfDcwQ39a7NxYkuT1dug.png" /><figcaption>Most Expensive Conversations (Last 7 Days)</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*VrvOyAqhbwAFoq3UBAvbxA.png" /><figcaption>Top Conversations by Cost</figcaption></figure><h3>Query 4: Combined Dashboard View (All Interfaces)</h3><p>A single query to power a comprehensive cost management dashboard:</p><pre>-- Unified CoCo usage dashboard<br>WITH all_usage AS (<br>    SELECT &#39;Snowsight&#39; AS interface, user_name, usage_time, token_credits, tokens<br>    FROM SNOWFLAKE.ACCOUNT_USAGE.CORTEX_CODE_SNOWSIGHT_USAGE_HISTORY<br>    UNION ALL<br>    SELECT &#39;Desktop&#39;, user_name, usage_time, token_credits, tokens<br>    FROM SNOWFLAKE.ACCOUNT_USAGE.CORTEX_CODE_DESKTOP_USAGE_HISTORY<br>    UNION ALL<br>    SELECT &#39;CLI&#39;, user_name, usage_time, token_credits, tokens<br>    FROM SNOWFLAKE.ACCOUNT_USAGE.CORTEX_CODE_CLI_USAGE_HISTORY<br>)<br>SELECT<br>    interface,<br>    COUNT(DISTINCT user_name) AS active_users,<br>    COUNT(*) AS total_requests,<br>    ROUND(SUM(token_credits), 4) AS total_credits,<br>    ROUND(AVG(token_credits), 6) AS avg_credit_per_request,<br>    ROUND(MAX(token_credits), 4) AS max_single_request<br>FROM all_usage<br>WHERE usage_time &gt;= DATEADD(&#39;day&#39;, -30, CURRENT_TIMESTAMP())<br>GROUP BY interface<br>ORDER BY total_credits DESC;</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*f8i6BOSJ_wEHcuakF0V9pA.png" /><figcaption>Interfaces Details</figcaption></figure><h3>Query 5: Converting Credits to USD</h3><p>Credits alone don’t mean much to a budget owner — they want a dollar figure. The dashboard pulls your account’s actual effective compute rate from ORGANIZATION_USAGE.RATE_SHEET_DAILY and uses it to convert every credit metric to USD on the fly:</p><pre>-- Latest effective credit rate for this account (falls back to $3.00/credit<br>-- in the app if no rate is found)<br>SELECT effective_rate<br>FROM SNOWFLAKE.ORGANIZATION_USAGE.RATE_SHEET_DAILY<br>WHERE account_locator = CURRENT_ACCOUNT()<br>  AND usage_type = &#39;compute&#39;<br>ORDER BY date DESC<br>LIMIT 1;</pre><h3>Query 6: Model Usage Breakdown</h3><p>To see which LLM models are actually driving token (and therefore credit) consumption, flatten TOKENS_GRANULAR and group by model key:</p><pre>-- Requests and tokens by model (last 14 days)<br>SELECT<br>    f.key AS model,<br>    COUNT(*) AS request_count,<br>    SUM(COALESCE(f.value:&quot;input&quot;::NUMBER, 0)<br>        + COALESCE(f.value:&quot;output&quot;::NUMBER, 0)<br>        + COALESCE(f.value:&quot;cache_read_input&quot;::NUMBER, 0)<br>        + COALESCE(f.value:&quot;cache_write_input&quot;::NUMBER, 0)) AS total_tokens<br>FROM SNOWFLAKE.ACCOUNT_USAGE.CORTEX_CODE_SNOWSIGHT_USAGE_HISTORY,<br>     LATERAL FLATTEN(input =&gt; tokens_granular) f<br>WHERE usage_time &gt;= DATEADD(&#39;day&#39;, -14, CURRENT_TIMESTAMP())<br>GROUP BY f.key<br>ORDER BY total_tokens DESC;</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/639/1*G_CT5uOh1pQIIkHjG03JQA.png" /><figcaption>Model Usage (Last 14 Days)</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/639/1*XwNu38l4iaekZ0-KYvVsRA.png" /><figcaption>Requests by Model</figcaption></figure><h3>What Drives AI Credit Consumption?</h3><p>Based on my experience across the 5 labs, here’s what I observed about relative cost:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*eMtJwMGjmy4vUfgbSCf7dw.png" /></figure><p>The more context CoCo needs to gather (reading table schemas, understanding relationships, generating multi-step plans) and the more iterations you go through, the more credits you consume.</p><h3>Practical Tips for Minimizing AI Costs</h3><h3>1. Be Specific in Your Prompts</h3><p>Vague prompts like “Who are the 5 best players?” force CoCo to make assumptions and sometimes generate multiple queries. Instead, be precise:</p><pre>Using @PLAYER_SEASON_STATS, which 5 players have the highest pass yards?</pre><p>The @ mention syntax tells CoCo exactly which table to use, reducing exploration overhead.</p><h3>2. Use Plan Mode for Complex Operations</h3><p>Before creating tables, pipelines, or multi-object setups, enable <strong>Plan Mode</strong>. This lets CoCo show you the full plan before executing anything. You review once, approve once, and avoid costly back-and-forth corrections.</p><h3>3. Start New Chats for New Tasks</h3><p>CoCo carries conversation context, which is helpful for iterative work but can cause confusion (and wasted credits) if you switch topics. Start a fresh chat when beginning a new workflow.</p><h3>The Two Types of Costs to Track</h3><p>Working with CoCo means tracking <strong>two separate cost streams</strong>:</p><ol><li><strong>Cortex AI Credits</strong> — consumed by CoCo interactions (chat, explain, fix, generate)</li><li><strong>Warehouse Credits</strong> — consumed by the SQL queries CoCo generates and executes</li></ol><p>The daily limit setting (CORTEX_CODE_SNOWSIGHT_DAILY_EST_CREDIT_LIMIT_PER_USER) only caps the first type. You still need resource monitors or budgets for warehouse usage. Simple habits help here too — suspend tasks and pipelines when not actively testing, and use small, auto-suspending warehouses for experimentation.</p><h3>Which LLM Model Matters for Cost</h3><p>CoCo offers multiple LLM models (visible in the model selector dropdown). The <strong>Auto</strong> setting picks the best model for your prompt. Different models have different credit costs per token — larger models (like Claude Opus) cost more than smaller ones (like Claude Haiku). Unless you have a specific reason to pick a model, <strong>Auto</strong> is the recommended default for balancing quality and cost.</p><h3>Bonus Queries (For the Curious)</h3><p>Two more queries power the “heatmap” and “token breakdown” views in the Streamlit app but don’t make the core cost-control workflow:</p><p><strong>Hourly usage patterns</strong> — groups by DAYNAME(usage_time) and HOUR(usage_time) against CORTEX_CODE_SNOWSIGHT_USAGE_HISTORY to build a heatmap of when your team uses CoCo most. Useful for capacity planning, not for day-to-day cost control.</p><pre>-- Day-of-week x hour-of-day usage pattern (last 14 days)<br>SELECT<br>    DAYNAME(usage_time) AS day_of_week,<br>    HOUR(usage_time) AS hour_of_day,<br>    COUNT(*) AS request_count,<br>    ROUND(SUM(token_credits), 4) AS credits_used<br>FROM SNOWFLAKE.ACCOUNT_USAGE.CORTEX_CODE_SNOWSIGHT_USAGE_HISTORY<br>WHERE usage_time &gt;= DATEADD(&#39;day&#39;, -14, CURRENT_TIMESTAMP())<br>GROUP BY 1, 2<br>ORDER BY credits_used DESC;</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*w9zueXw4GUgfr-lDvlP10w.png" /><figcaption>Usage Heatmap: Day of Week x Hour</figcaption></figure><p><strong>Input vs. output token breakdown</strong> — flattens TOKENS_GRANULAR per request to see whether credits are going toward long prompts or long generated responses, and tracks the output-token percentage over time. If output tokens dominate, CoCo may be generating more verbose responses than needed.</p><pre>-- Daily input vs. output token split, plus output-token percentage (last 14 days)<br>WITH flattened AS (<br>    SELECT<br>        DATE_TRUNC(&#39;day&#39;, usage_time) AS usage_date,<br>        tokens,<br>        f.value:&quot;input&quot;::NUMBER AS input_tok,<br>        f.value:&quot;output&quot;::NUMBER AS output_tok,<br>        f.value:&quot;cache_read_input&quot;::NUMBER AS cache_read_tok,<br>        f.value:&quot;cache_write_input&quot;::NUMBER AS cache_write_tok<br>    FROM SNOWFLAKE.ACCOUNT_USAGE.CORTEX_CODE_SNOWSIGHT_USAGE_HISTORY,<br>         LATERAL FLATTEN(input =&gt; tokens_granular) f<br>    WHERE usage_time &gt;= DATEADD(&#39;day&#39;, -14, CURRENT_TIMESTAMP())<br>)<br>SELECT<br>    usage_date,<br>    SUM(COALESCE(input_tok, 0) + COALESCE(cache_read_tok, 0) + COALESCE(cache_write_tok, 0)) AS input_tokens,<br>    SUM(COALESCE(output_tok, 0)) AS output_tokens,<br>    ROUND(<br>        SUM(COALESCE(output_tok, 0)) * 100.0 /<br>        NULLIF(SUM(tokens), 0), 1<br>    ) AS output_pct<br>FROM flattened<br>GROUP BY 1<br>ORDER BY 1;</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/987/1*_2Sv_t05zkrSn7DhY56Kgg.png" /><figcaption>Input vs Output Tokens (Last 14 Days)</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/987/1*_bNQHgQ6buDVLUhAIFmZUA.png" /><figcaption>Output Token Percentage Over Time</figcaption></figure><h3>Building a Cost Monitoring Alert</h3><p>The most actionable piece of this whole setup: don’t check a dashboard manually — get notified automatically when usage spikes. Combine the usage views with a native Snowflake alert:</p><pre>CREATE OR REPLACE ALERT coco_daily_usage_alert<br>  WAREHOUSE = COCOLABS_WH<br>  SCHEDULE = &#39;USING CRON 0 9 * * * America/New_York&#39;<br>  IF (EXISTS (<br>    SELECT 1<br>    FROM SNOWFLAKE.ACCOUNT_USAGE.CORTEX_CODE_SNOWSIGHT_USAGE_HISTORY<br>    WHERE usage_time &gt;= DATEADD(&#39;day&#39;, -1, CURRENT_TIMESTAMP())<br>    GROUP BY DATE_TRUNC(&#39;day&#39;, usage_time)<br>    HAVING SUM(token_credits) &gt; 50<br>  ))<br>  THEN<br>    CALL SYSTEM$SEND_EMAIL(<br>      &#39;my_email_integration&#39;,<br>      &#39;admin@company.com&#39;,<br>      &#39;CoCo Usage Alert&#39;,<br>      &#39;Daily CoCo credit usage exceeded 50 credits in the last 24 hours.&#39;<br>    );</pre><p>This runs every morning at 9 am and checks whether the previous day’s credit consumption crossed a threshold (50 credits in this example). If it did, it emails an admin. A few notes if you want to adapt this:</p><ul><li><strong>WAREHOUSE</strong> — alerts need a warehouse to evaluate the condition; a small auto-suspending one is enough</li><li><strong>SCHEDULE</strong> — supports both cron syntax and simple intervals (e.g., &#39;60 MINUTE&#39;) if you want more frequent checks</li><li><strong>The threshold (50 credits)</strong> — set this based on your own baseline. Run Query 1 above for a couple of weeks first to see what “normal” looks like before picking a number.</li><li><strong>SYSTEM$SEND_EMAIL</strong> — requires a notification integration configured for email. You can swap this for a webhook or Slack notification integration if that fits your stack better</li></ul><p>This is the piece that turns cost <em>monitoring</em> into cost <em>management</em> — instead of remembering to check a dashboard, the dashboard checks itself and taps you on the shoulder when something’s off. For a team rolling out CoCo broadly, I’d treat this as non-optional.</p><h3>Conclusion</h3><p>Snowflake’s AI assistant is genuinely useful — across the 5 labs, I built data pipelines, Streamlit dashboards, and optimized SQL without writing most of the code manually. But AI consumption is a real cost that needs active management.</p><p>The key lessons:</p><ul><li><strong>Set daily credit limits</strong> before you start exploring</li><li><strong>Monitor consumption</strong> through the three ACCOUNT_USAGE views</li><li><strong>Track all interfaces</strong> — Snowsight, Desktop, and CLI each have dedicated views</li><li><strong>Use </strong><strong>PARENT_REQUEST_ID</strong> to understand per-conversation costs</li><li><strong>Be specific</strong> in prompts to reduce wasted iterations</li><li><strong>Use Plan Mode</strong> for anything that creates or modifies Snowflake objects</li><li><strong>Understand the dual cost model</strong> — AI credits + warehouse credits</li><li><strong>Build an alert</strong> on top of the usage views so spikes surface automatically instead of relying on manual checks</li></ul><p>The <a href="https://learn.snowflake.com/en/courses/OD-COCOESS/"><strong>CoCo Essentials</strong></a> course is free and takes a few hours. If you’re evaluating Snowflake’s AI capabilities or want to understand the cost model before rolling it out to your team, it’s worth the time. I’ve also built a live Streamlit dashboard on top of these queries.</p><p><em>This article was inspired by the CoCo Essentials course available for free at learn.snowflake.com. The course uses a fictional Professional Football League (PFL) dataset across 5 hands-on labs.</em></p><h3>References</h3><ul><li>Snowflake AI Pricing:<br><a href="https://docs.snowflake.com/en/user-guide/snowflake-cortex/pricing">https://docs.snowflake.com/en/user-guide/snowflake-cortex/pricing</a></li><li>Snowflake Service Consumption Table:<br><a href="https://www.snowflake.com/legal-files/CreditConsumptionTable.pdf">https://www.snowflake.com/legal-files/CreditConsumptionTable.pdf</a></li><li>CoCo Essentials course:<br><a href="https://learn.snowflake.com/en/courses/OD-COCOESS/">https://learn.snowflake.com/en/courses/OD-COCOESS/</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=79d3f16499a5" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[❄️ Snowflake Data Clean Rooms: A Technical Pattern for Privacy-Safe Collaboration in Regulated…]]></title>
            <link>https://omid-karami.medium.com/%EF%B8%8F-snowflake-data-clean-rooms-a-technical-pattern-for-privacy-safe-collaboration-in-regulated-38634059de92?source=rss-26d5566e003e------2</link>
            <guid isPermaLink="false">https://medium.com/p/38634059de92</guid>
            <category><![CDATA[snowflake]]></category>
            <category><![CDATA[snowflake-data-cloud]]></category>
            <category><![CDATA[data-superhero]]></category>
            <dc:creator><![CDATA[Omid Karami]]></dc:creator>
            <pubDate>Wed, 17 Jun 2026 09:35:12 GMT</pubDate>
            <atom:updated>2026-06-17T09:35:12.312Z</atom:updated>
            <content:encoded><![CDATA[<h3>❄️ Snowflake Data Clean Rooms: A Technical Pattern for Privacy-Safe Collaboration in Regulated Industries</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*TElW60NZ-5oGD_KHGLvFCA.jpeg" /></figure><p><em>Snowflake Data Clean Rooms provide a trusted environment for privacy-safe data collaboration.<br>Source: </em><a href="https://www.snowflake.com/en/product/features/data-clean-rooms/"><strong><em>Snowflake</em></strong></a><em>.</em></p><p>From my experience working with data in <strong>healthcare</strong> and <strong>regulated</strong> environments, one of the most important lessons is that data collaboration is never only a technical question.</p><p>It is also a question of <strong>governance</strong>, <strong>privacy</strong>, <strong>compliance</strong>, <strong>auditability</strong>, and <strong>trust</strong>.</p><p><strong>Healthcare</strong> and <strong>life sciences</strong> organizations often need to collaborate across providers, laboratories, research partners, MedTech companies, digital health platforms, and external ecosystems. However, the data involved can be highly sensitive. Clinical data, patient-related information, research data, product usage data, and operational data all require strong controls around privacy, access management, lineage, and compliance.</p><p>This is where <strong>Snowflake Data Clean Rooms</strong> become particularly relevant.</p><p>Snowflake describes a Data Clean Room as a <strong>controlled</strong> environment that enables multiple companies, or different divisions of a company, to securely collaborate on sensitive or regulated data while preserving privacy.</p><p>From a data architecture perspective, this is not only a collaboration feature. It is a pattern for enabling cross-party analytics while reducing unnecessary data exposure.</p><h3>⚠️ Why traditional data sharing creates risk</h3><p>Traditional data collaboration often relies on file-based exchanges, API integrations, data extracts, replicated datasets, or custom point-to-point pipelines between organizations.</p><p>While these approaches are common, they can introduce significant architectural and governance challenges, especially when the underlying data is sensitive, regulated, or commercially valuable.</p><p>Each additional copy of a dataset increases the operational surface area that must be governed. This includes access management, retention policies, security controls, lineage, auditability, consent management, and downstream usage monitoring.</p><p>In regulated environments such as healthcare and life sciences, these considerations become part of the core data architecture. The question is not only how to make data available, but how to make it available within a controlled, compliant, and auditable framework.</p><p>This is where the Data Clean Room pattern becomes relevant.</p><p>Instead of distributing raw datasets across organizational boundaries, a clean room enables governed collaboration through controlled computation. Participants can contribute data, define approved analysis logic, enforce access and privacy rules, and expose only permitted outputs.</p><p>From an architectural point of view, the clean room shifts the collaboration model from data replication to governed data collaboration.</p><p>The objective is not simply to share data faster.</p><p>The objective is to enable cross-party analytics while minimizing unnecessary data movement, reducing exposure of sensitive attributes, and maintaining control over how data is accessed, joined, analyzed, and consumed.</p><h3>🏗️ The technical pattern</h3><p>From a technical perspective, a Data Clean Room acts as a governed collaboration layer between data providers and data consumers.</p><p>The clean room defines the boundaries of collaboration: who can participate, which datasets can be used, what analysis logic is permitted, and what outputs can be shared.</p><p>In this model, participants do not need unrestricted access to each other’s raw data. Instead, collaboration is enabled through approved queries, predefined templates, aggregation rules, privacy controls, and governed output policies.</p><p>This is especially important when multiple parties need to join or compare data across organizational boundaries without exposing sensitive records.</p><p>A clean room architecture can help reduce risks related to:</p><ul><li>Unnecessary data movement</li><li>Dataset duplication</li><li>Uncontrolled downstream usage</li><li>Overexposure of sensitive attributes</li><li>Ambiguous ownership of copied datasets</li><li>Complex partner-to-partner integration patterns</li><li>Limited auditability of shared data assets</li></ul><p>Snowflake highlights this model as a way to collaborate without moving data, while supporting both business and technical users. Business teams can use no-code workflows, analysts can use SQL, and developers can build custom clean room applications.</p><p>For data engineers and architects, this is an important evolution. The platform is not only serving data internally; it is enabling secure, policy-aware collaboration across organizations and ecosystems.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/745/1*xuillBtKbYCU1DNfwvxjIA.png" /></figure><p><em>A data clean room (DCR) provides a controlled environment that allows multiple companies and/or divisions of a company to securely collaborate on sensitive or regulated data while fully preserving that data’s privacy.<br>Source: </em><a href="https://www.snowflake.com/en/product/features/data-clean-rooms/"><strong><em>Snowflake</em></strong></a><em>.</em></p><h3>🏥 A practical collaboration scenario</h3><p>Consider a <strong>MedTech</strong> company collaborating with healthcare providers and research partners to better understand treatment pathways, product usage patterns, or patient journey trends.</p><p>The analytical objective could include understanding how product usage varies across care settings, identifying aggregated patterns in treatment pathways, or combining digital health and operational data with clinical context in a governed way.</p><p>In a traditional model, this type of collaboration may require data extracts, manual anonymization, legal reviews, duplicated datasets, and custom pipelines. Each step can increase operational complexity and governance risk.</p><p>A clean room model provides a more controlled approach.</p><p>The participating organizations can collaborate using approved datasets, governed analysis logic, and privacy-safe outputs, while reducing the need to expose or transfer raw sensitive data.</p><p>For example, instead of giving one party direct access to patient-level or clinical records, the clean room can support approved analysis that returns aggregated or permitted outputs. This allows the collaboration to focus on insight generation while keeping privacy, access control, and compliance boundaries in place.</p><p>Snowflake specifically highlights healthcare and life sciences as an area where Data Clean Rooms can enable secure data analysis between labs and healthcare facilities without exposing sensitive information.</p><h3>🧭 Why this matters for data architecture</h3><p>In regulated environments, building a data platform is not only about ingestion, transformation, modeling, dashboards, and performance optimization.</p><p>It is about designing for trust.</p><p>That means thinking about:</p><ul><li>Data quality</li><li>Access control</li><li>Data governance</li><li>Privacy</li><li>Compliance</li><li>Lineage</li><li>Traceability</li><li>Ownership</li><li>Secure collaboration</li><li>Clear data product boundaries</li></ul><p>These capabilities are not optional layers added at the end. They need to be part of the architecture from the beginning.</p><p>This is why I find Snowflake Data Clean Rooms important. They provide a practical model for enabling collaboration while keeping privacy, governance, and control at the center.</p><h3>👥 Business users, analysts, and developers</h3><p>Another important aspect of Snowflake Data Clean Rooms is that they support different collaboration personas.</p><p><strong>Business users</strong> can collaborate through a no-code interface. <strong>Analysts</strong> can use SQL to run more complex multi-party queries. <strong>Technical teams</strong> can work with ML models, and developers can build and deploy custom clean room solutions.</p><p>This matters because secure data collaboration is not only a technical problem. It is also an adoption problem.</p><p>A strong collaboration model should be accessible to business teams, flexible for analysts and engineers, and governed enough for security, legal, privacy, and compliance stakeholders.</p><p>Snowflake’s approach brings these personas into one collaboration model.</p><h3>🌐 Cross-cloud and ecosystem considerations</h3><p>Real-world collaboration does not happen inside one single technology boundary.</p><p>Partners may operate across different cloud providers, regions, tools, and data maturity levels. Snowflake highlights interoperability across AWS, Microsoft Azure, and Google Cloud, as well as the ability to collaborate with partners even if they are not already using <strong>Snowflake</strong> technology.</p><p>This is important because enterprise collaboration must reflect how organizations actually operate.</p><p>A clean room architecture needs to support collaboration across organizational and technical boundaries, not only within one internal platform.</p><p>Snowflake Marketplace also extends this ecosystem by allowing organizations to bring data, identity, AI, and partner capabilities into the collaboration model.</p><h3>💡 Final thoughts</h3><p>Snowflake Data Clean Rooms represent an important shift in modern data architecture.</p><p>The focus is not only on storing, processing, and serving data. The focus is also on enabling trusted collaboration around data.</p><p>For healthcare, life sciences, financial services, and other regulated industries, this is critical.</p><p>The value of data collaboration is high, but the cost of getting privacy, governance, or compliance wrong is also high.</p><p><strong>Data Clean Rooms</strong> provide a practical architectural pattern for balancing both sides:</p><ul><li>Unlocking value from collaborative data</li><li>Reducing unnecessary data movement</li><li>Preserving privacy</li><li>Supporting business and technical users</li><li>Enforcing governance and compliance</li><li>Building trust across organizations</li></ul><p>For me, the key takeaway is this:</p><p><strong>Trust is becoming part of the data architecture.</strong></p><p>And in regulated industries, that is not optional.</p><h3>📚 References and further reading</h3><p>For more information, see Snowflake’s official page on Data Clean Rooms:</p><p><strong>Snowflake Data Clean Rooms</strong><br><a href="https://www.snowflake.com/en/product/features/data-clean-rooms/">https://www.snowflake.com/en/product/features/data-clean-rooms/</a></p><p><strong>Healthcare and life-sciences examples</strong><br><a href="https://www.snowflake.com/en/solutions/industries/healthcare-and-life-sciences/patient-and-member-360/">https://www.snowflake.com/en/solutions/industries/healthcare-and-life-sciences/patient-and-member-360/</a></p><p><strong>Snowflake</strong> describes Data Clean Rooms as controlled environments where multiple companies, or different divisions of a company, can securely collaborate on sensitive or regulated data while preserving privacy.</p><p>Visuals used in this article are sourced from <strong>Snowflake’s</strong> official <a href="https://www.snowflake.com/en/product/features/data-clean-rooms/">Data Clean Rooms</a> page.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=38634059de92" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[ Snowflake Summit 2026: From Data Cloud to the Agentic Enterprise ❄️]]></title>
            <link>https://omid-karami.medium.com/snowflake-summit-2026-from-data-cloud-to-the-agentic-enterprise-%EF%B8%8F-38a02802480c?source=rss-26d5566e003e------2</link>
            <guid isPermaLink="false">https://medium.com/p/38a02802480c</guid>
            <category><![CDATA[data-superhero]]></category>
            <category><![CDATA[snowflake-computing]]></category>
            <category><![CDATA[snowflake]]></category>
            <category><![CDATA[snowflake-data-cloud]]></category>
            <dc:creator><![CDATA[Omid Karami]]></dc:creator>
            <pubDate>Fri, 05 Jun 2026 14:15:50 GMT</pubDate>
            <atom:updated>2026-06-05T14:15:50.003Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*XaMIU6483V5MfBOi-a4bBw.jpeg" /></figure><p><strong>Snowflake Summit 2026</strong> was packed with announcements, but for me, the most interesting story wasn’t a specific feature.</p><p>It was the direction. A few years ago, <strong>Snowflake</strong> was primarily known as a cloud data warehouse.</p><p>Then it became the Data Cloud. More recently, the AI Data Cloud.</p><p>And after Summit 2026, it’s clear that Snowflake is positioning itself as an <strong>Agentic Enterprise Platform</strong> — a platform where data powers AI, and AI powers intelligent agents.</p><p>Let’s take a closer look at the announcements that stood out the most.</p><h3>❄️ Snowflake CoWork: AI for Knowledge Workers</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Di-CWzKQETtuv7I_3ILfKw.jpeg" /></figure><p>One of the headline announcements was <strong>Snowflake CoWork</strong>, formerly known as Snowflake Intelligence.</p><p><strong>CoWork</strong> is designed to help business users interact with enterprise information through natural language rather than dashboards, reports, and multiple business applications.</p><p>Imagine asking:</p><ul><li>“What were our sales in Europe last quarter?”</li><li>“Why did revenue decrease in Germany?”</li><li>“Show me the latest customer feedback trends.”</li><li>“Create a summary for leadership.”</li></ul><p>Instead of jumping between systems, <strong>CoWork</strong> aims to provide a single AI-powered experience that understands your organization’s data, documents, and business context.</p><h4>Why It Matters</h4><p>For years, organizations have invested heavily in BI tools and dashboards.<br> CoWork represents a shift from <strong>finding information</strong> to <strong>understanding information</strong>, and ultimately to <strong>taking action on information</strong>.</p><p>This may fundamentally change how business users consume analytics.</p><p><strong>Snowflake</strong> also announced two notable additions to the CoWork experience.</p><p>💡 <strong>Cortex Sense </strong>(Coming soon in Private Preview) is designed to proactively surface relevant insights, recommendations, and actions based on a user’s work context, moving beyond reactive question-and-answer interactions.</p><p>📊 <strong>Artifacts</strong> (Coming soon in General Availability) enable users to create and share reusable outputs generated through CoWork, helping transform AI conversations into collaborative business assets such as reports, summaries, analyses, and workflows.</p><p>Together, these capabilities reinforce Snowflake’s vision of CoWork as more than an AI assistant — it is evolving into a personal work agent that can understand context, assist with decision-making, and help users turn insights into action.</p><h4>Official Resources</h4><p><strong>Announcement</strong></p><p><a href="https://www.snowflake.com/en/news/press-releases/snowflake-cowork-powers-the-agentic-enterprise-as-the-personal-agent-for-knowledge-workers-to-work-smarter/?utm_source=chatgpt.com">https://www.snowflake.com/en/news/press-releases/snowflake-cowork-powers-the-agentic-enterprise-as-the-personal-agent-for-knowledge-workers-to-work-smarter/</a></p><p><strong>Product Page</strong></p><p><a href="https://www.snowflake.com/en/product/snowflake-cowork/">https://www.snowflake.com/en/product/snowflake-cowork/</a></p><h3>🤖 Snowflake CoCo &amp; Cortex Code Desktop: AI for Builders</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*U4ulSWduUjF-H7ZeP-I9Tw.jpeg" /></figure><p>For many technical users, this may have been the most exciting announcement from Summit 2026.</p><p>Snowflake expanded <strong>Cortex Code</strong> (CoCo) and introduced <strong>Cortex Code Desktop</strong>, a dedicated AI-powered IDE available for macOS and Windows.</p><h4>Why Cortex Code Desktop Matters</h4><p>Data engineers frequently work across:</p><ul><li>Snowsight</li><li>VS Code</li><li>CLI tools</li><li>dbt projects</li><li>Python environments</li><li>Documentation portals</li></ul><p>This constant context switching reduces productivity.</p><p><strong>Cortex Code Desktop</strong> aims to provide a Snowflake-native development environment where AI assistance is deeply integrated into the workflow.</p><h4>Context-Aware Development</h4><p>Unlike generic coding assistants, CoCo understands Snowflake concepts such as:</p><ul><li>Databases</li><li>Schemas</li><li>Tables</li><li>Views</li><li>SQL</li><li>Snowpark</li><li>Data Pipelines</li></ul><p>This allows developers to ask questions about their environment and receive answers that are aware of the platform context.</p><p>Examples include:</p><ul><li>Explain this SQL query.</li><li>Generate a Snowpark transformation.</li><li>Create a dbt model.</li><li>Optimize this workload.</li><li>Troubleshoot a pipeline.</li></ul><h4>Why I Think This Is Important</h4><p>Many AI coding assistants are code-aware. CoCo aims to become data-platform-aware. That distinction could be significant for organizations heavily invested in Snowflake.</p><h4>Official Resources</h4><p><strong>Download Cortex Code Desktop</strong></p><p><a href="https://www.snowflake.com/en/product/snowflake-coco/downloads/">https://www.snowflake.com/en/product/snowflake-coco/downloads/</a></p><p><strong>Documentation</strong></p><p><a href="https://docs.snowflake.com/en/user-guide/cortex-code/cortex-code-desktop/index">https://docs.snowflake.com/en/user-guide/cortex-code/cortex-code-desktop/index</a></p><p><strong>Product Overview</strong></p><p><a href="https://www.snowflake.com/en/product/features/cortex-code/?utm_source=chatgpt.com">https://www.snowflake.com/en/product/features/cortex-code/</a></p><h3>❄️ Snowflake Openflow: Simplifying Data Movement</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4svlYZ9OZGtxrTvHt7dtdw.jpeg" /></figure><p>Data movement remains one of the most challenging aspects of modern data architectures.</p><p>Organizations often manage dozens of pipelines, connectors, ingestion frameworks, and integration tools simply to move data between systems.</p><p>Snowflake <strong>Openflow</strong> is the company’s latest effort to simplify that landscape.</p><p>Openflow is designed to provide a more seamless approach to ingesting, moving, and preparing data for analytics and AI workloads. The announcement emphasizes interoperability, scalability, and support for real-time and event-driven architectures.</p><p>Rather than focusing solely on data storage and processing, Snowflake is increasingly expanding into the broader data lifecycle.</p><p>This announcement suggests that Snowflake sees data movement as a strategic component of the overall platform rather than a separate problem to be solved by third-party tools.</p><h4>Why It Matters</h4><p>Data engineers frequently spend more time moving data than analyzing it.</p><p>If Openflow succeeds in simplifying data movement, it could reduce operational complexity and accelerate the delivery of data products and AI initiatives.</p><h4>Official Resources</h4><p>Announcement:<br><a href="https://www.snowflake.com/en/news/press-releases/snowflake-openflow-unlocks-full-data-interoperability-accelerating-data-movement-for-ai-innovation/?utm_source=chatgpt.com">https://www.snowflake.com/en/news/press-releases/snowflake-openflow-unlocks-full-data-interoperability-accelerating-data-movement-for-ai-innovation/</a></p><p>Documentation:<br><a href="https://docs.snowflake.com/en/user-guide/data-integration/openflow/about">https://docs.snowflake.com/en/user-guide/data-integration/openflow/about</a></p><h3>🧊 Open Interoperability for Enterprise Data &amp; AI</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DwFBD0lYVpSnw5QPn4_IcQ.jpeg" /></figure><p>One of the most important strategic announcements at Snowflake Summit 2026 was Snowflake’s continued investment in <strong>open interoperability</strong>.</p><p>As organizations modernize their data and AI platforms, they increasingly need the flexibility to work across multiple technologies, cloud providers, storage platforms, and processing engines.</p><p>To address this, Snowflake introduced a new open framework for interoperable enterprise data and AI, built around:</p><ul><li><strong>Apache Iceberg™</strong></li><li><strong>Snowflake Storage for Apache Iceberg Tables</strong></li><li><strong>Horizon Catalog</strong></li><li><strong>Open standards and APIs</strong></li></ul><p>Together, these capabilities help organizations build AI-ready architectures while maintaining flexibility, interoperability, and reduced vendor lock-in.</p><h4>Snowflake Storage for Apache Iceberg Tables</h4><p>A key announcement supporting this vision was the General Availability of Snowflake Storage for Apache Iceberg Tables.</p><p>The goal is simple: provide the benefits of Apache Iceberg interoperability while reducing the complexity of managing external storage infrastructure.</p><p>Snowflake describes this approach as:</p><blockquote><strong><em>“All of the interoperability, none of the complexity.”</em></strong></blockquote><p>This allows organizations to leverage open table formats while continuing to benefit from Snowflake’s governance, security, and performance capabilities.</p><h4>Why It Matters</h4><p>AI applications increasingly rely on data that spans multiple platforms and systems.</p><p>Open and interoperable architectures make it easier to access, share, and govern data across these environments, creating a stronger foundation for analytics and AI initiatives.</p><p>While AI announcements often receive the most attention, interoperable data foundations are equally important for long-term success.</p><h4>Official Resources</h4><p><strong>Snowflake Pioneers New Open Framework for Interoperable Enterprise Data and AI</strong></p><p><a href="https://www.snowflake.com/en/news/press-releases/snowflake-pioneers-new-open-framework-for-interoperable-enterprise-data-and-ai/">https://www.snowflake.com/en/news/press-releases/snowflake-pioneers-new-open-framework-for-interoperable-enterprise-data-and-ai/</a></p><p><strong>Apache Iceberg™ Documentation</strong></p><p><a href="https://docs.snowflake.com/en/user-guide/tables-iceberg">https://docs.snowflake.com/en/user-guide/tables-iceberg</a></p><h3>🏛️ Horizon Catalog &amp; Context for Trusted AI</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*UlGjyb8BFGCBo60vupRuqw.jpeg" /></figure><p>One of the most important announcements at Snowflake Summit 2026 was the expansion of <strong>Snowflake Horizon Catalog</strong>, introducing new capabilities designed to provide governance, security, and business context for AI.</p><p>As organizations move beyond traditional analytics and begin deploying AI assistants and autonomous agents, data alone is no longer enough.</p><p>AI systems need context, not just data. They need to understand:</p><ul><li><strong>Business definitions</strong></li><li><strong>Metadata</strong></li><li><strong>Ownership</strong></li><li><strong>Lineage</strong></li><li><strong>Policies</strong></li><li><strong>Relationships between data assets</strong></li></ul><p>To address this challenge, Snowflake introduced <strong>Horizon Context</strong>, a governed context layer that helps AI systems understand enterprise information in the same way humans do.</p><p>The goal is to ensure that people, applications, dashboards, and AI agents all operate using consistent business definitions and trusted organizational knowledge.</p><h4>Why It Matters</h4><p>One of the biggest challenges facing enterprise AI today is ensuring that AI-generated answers align with business reality.</p><p>Without context, AI can generate responses based solely on data.</p><p>With context, AI can generate responses grounded in trusted definitions, governance policies, and organizational knowledge.</p><p>As AI agents become more common, context may become just as important as the underlying data itself.</p><h4>Official Resources</h4><p><strong>Snowflake Advances Trusted AI with Horizon Catalog</strong></p><p><a href="https://www.snowflake.com/en/news/press-releases/snowflake-advances-trusted-ai-with-snowflake-horizon-catalog-centralizing-governance-context-and-security-across-the-enterprise/">https://www.snowflake.com/en/news/press-releases/snowflake-advances-trusted-ai-with-snowflake-horizon-catalog-centralizing-governance-context-and-security-across-the-enterprise/</a></p><p><strong>Horizon Catalog Product Page</strong></p><p><a href="https://www.snowflake.com/en/product/features/horizon/">https://www.snowflake.com/en/product/features/horizon/</a></p><p><strong>The Governed Context Layer for AI, BI, and Apps</strong></p><p><a href="https://www.snowflake.com/en/blog/horizon-context-governed-context/">https://www.snowflake.com/en/blog/horizon-context-governed-context/</a></p><p><strong>Horizon Documentation</strong></p><p><a href="https://docs.snowflake.com/en/user-guide/snowflake-horizon">Snowflake Horizon Catalog | Snowflake Documentation</a></p><h3>🛡️ Security &amp; Governance for AI Agents</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*CVFD7Rp-hPjuSWiYx7G13g.jpeg" /></figure><p>Every major AI announcement eventually leads to the same question:</p><p>Can we trust it?</p><p><strong>Snowflake</strong> dedicated significant attention at Summit 2026 to security, governance, compliance, and trusted AI.</p><p>As organizations move from experimentation to production-grade AI systems and autonomous agents, governance becomes increasingly important.</p><h4>Why It Matters</h4><p>Enterprise AI adoption depends on trust.</p><p>Organizations need confidence that AI systems operate within approved boundaries, comply with regulations, and align with business requirements.</p><h4>Official Resources</h4><p><strong>Foundation for Trusted AI:</strong><br><a href="https://www.snowflake.com/en/blog/trusted-data-trusted-ai/">https://www.snowflake.com/en/blog/trusted-data-trusted-ai/</a></p><p><strong>Defending Your Enterprise at the Speed of AI:</strong><br><a href="https://www.snowflake.com/en/blog/enterprise-ai-security/">https://www.snowflake.com/en/blog/enterprise-ai-security/</a></p><h3>🎯 Final Thoughts</h3><p><strong>Snowflake Summit 2026</strong> wasn’t simply about introducing new features.</p><p>It was about presenting a broader platform vision for the future of enterprise data and AI.</p><p>- <strong>CoWork</strong> helps business users interact with data and AI through a personal work agent.<br>- <strong>CoCo</strong> helps developers build faster with AI-assisted development experiences.<br>- <strong>Openflow</strong> simplifies how data moves across the enterprise.<br>- <strong>Open Interoperability</strong> enables more flexible and AI-ready data architectures.<br>- <strong>Horizon Catalog &amp; Context</strong> provide the business meaning and context AI systems need.<br>- <strong>Security &amp; Governance</strong> establish the trust required for enterprise AI adoption.</p><p>Together, these capabilities move <strong>Snowflake </strong>❄️ beyond being a Data Cloud Platform and toward becoming an <strong>Agentic Enterprise Platform</strong>.</p><p>Whether this vision becomes the dominant model for enterprise technology remains to be seen, but Summit 2026 provided a clear view of where Snowflake believes the industry is heading.</p><p>For those interested in exploring the announcements in more detail, I highly recommend watching the <strong>Summit keynotes</strong>. They provide additional demonstrations, product insights, and strategic context behind many of the announcements discussed in this article.</p><p>🎥 <strong>Snowflake Summit 2026 Opening Keynote:<br></strong><a href="https://www.youtube.com/watch?v=F34xlRoQ3eQ">https://www.youtube.com/watch?v=F34xlRoQ3eQ</a></p><p>🎥 <strong>Snowflake Summit 2026 Platform Keynote:<br></strong><a href="https://www.youtube.com/watch?v=NFTdHYp9uJo">https://www.youtube.com/watch?v=NFTdHYp9uJo</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=38a02802480c" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Snowflake Now Supports Apache Iceberg™ v3 (General availability)]]></title>
            <link>https://omid-karami.medium.com/snowflake-now-supports-apache-iceberg-v3-general-availability-86dfada637e1?source=rss-26d5566e003e------2</link>
            <guid isPermaLink="false">https://medium.com/p/86dfada637e1</guid>
            <category><![CDATA[snowflake]]></category>
            <category><![CDATA[data-superhero]]></category>
            <category><![CDATA[snowflake-data-cloud]]></category>
            <dc:creator><![CDATA[Omid Karami]]></dc:creator>
            <pubDate>Mon, 25 May 2026 06:05:15 GMT</pubDate>
            <atom:updated>2026-05-25T06:05:15.928Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/380/1*8WWBv_Wam4E7lgCnZXa_2g.png" /><figcaption><strong>Apache Iceberg v3</strong></figcaption></figure><p>On May 7, 2026, <strong>Snowflake</strong> announced the General Availability (GA) of support for <strong>Apache Iceberg™ version 3</strong>.</p><p>This is an important milestone for teams building modern, open, interoperable, and AI-ready data platforms.</p><p>The release introduces support for new Iceberg v3 data types, including:</p><ul><li><strong>geography</strong></li><li><strong>geometry</strong></li><li><strong>nanosecond timestamp</strong></li><li><strong>variant</strong></li></ul><p>And for many modern workloads, the VARIANT support is probably the most exciting addition.</p><h3>Why This Matters</h3><p>Today’s data platforms work with much more than just structured tables.</p><p>JSON payloads, APIs, event streams, IoT data, AI/ML pipelines, logs, digital applications, and clinical systems all generate semi-structured data continuously.</p><p>In many cases, forcing that data into rigid schemas too early creates:</p><ul><li>unnecessary complexity</li><li>extra transformations</li><li>reduced flexibility</li><li>slower ingestion pipelines</li></ul><p>That’s why support for VARIANT inside Apache Iceberg™ v3 is such a significant step forward.</p><p>It enables organizations to work with semi-structured data more natively and openly while still benefiting from Iceberg’s interoperability and open table format architecture.</p><h3>Understanding the Architecture</h3><p>The architecture below shows how Snowflake manages <a href="https://docs.snowflake.com/en/user-guide/tables-iceberg">Apache Iceberg™ tables</a> using external cloud storage while still maintaining centralized metadata management and query capabilities.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/657/1*QbG59cQI9BBbN2X552gjmw.png" /></figure><ul><li>The actual table data and Iceberg metadata files are stored in external cloud storage, typically as Parquet files.</li><li>The External Volume in Snowflake provides secure access to that storage location.</li><li>The Iceberg table itself points to the external storage location.</li><li>Snowflake manages snapshots and metadata through the Iceberg catalog.</li><li>The catalog keeps track of schema evolution, snapshots, partitions, and table state.</li></ul><p>This architecture is powerful because it combines:</p><ul><li>open table formats</li><li>separation of storage and compute</li><li>interoperability across engines</li><li>centralized governance and metadata management</li></ul><p>For organizations building lakehouse architectures, this creates much more flexibility while keeping governance and scalability in focus.</p><h3>How to Create an Iceberg v3 Table in Snowflake</h3><p>Creating an Iceberg table with version 3 is simple.</p><p>Here is a minimal example using the new VARIANT data type and high-precision timestamps:</p><pre>CREATE ICEBERG TABLE my_v3_iceberg_table (<br>  record VARIANT,<br>  event_timestamp TIMESTAMP_LTZ(6)<br>)<br>CATALOG = &#39;SNOWFLAKE&#39;<br>EXTERNAL_VOLUME = &#39;my_external_volume&#39;<br>BASE_LOCATION = &#39;my_iceberg_table&#39;<br>ICEBERG_VERSION = 3;</pre><p>In this example:</p><ul><li>VARIANT enables flexible semi-structured data storage</li><li>TIMESTAMP_LTZ(6) provides high-resolution timestamp precision</li><li>The table metadata is managed through the Snowflake Iceberg catalog</li><li>The actual data remains stored externally in cloud object storage</li></ul><p>This model supports a more open and interoperable architecture while still leveraging Snowflake’s platform capabilities.</p><h3>Why VARIANT Support Is a Big Deal</h3><p>For many teams, VARIANT is already central to modern ELT and AI workflows.</p><p>It is heavily used for:</p><ul><li>JSON ingestion</li><li>event-driven architectures</li><li>API payloads</li><li>telemetry and IoT</li><li>clinical and healthcare data</li><li>AI-ready pipelines</li><li>application metadata</li><li>log analytics</li></ul><p>Previously, open table formats often required additional workarounds or flattening strategies for this type of data.</p><p>Iceberg v3 changes that.</p><p>This opens new opportunities for:</p><ul><li>AI-ready data platforms</li><li>modern lakehouse architectures</li><li>multi-engine interoperability</li><li>scalable semi-structured analytics</li><li>flexible ingestion pipelines</li><li>future-proof data platform design</li></ul><h3>Conclusion</h3><p>The industry continues moving toward more open architectures where interoperability, portability, and governance are increasingly important.</p><p>Apache Iceberg™ v3 support in Snowflake is another strong step in that direction.</p><p>And for teams working heavily with <strong>semi-structured</strong> data, the addition of VARIANT support makes this release particularly exciting.</p><p>The lakehouse ecosystem keeps evolving — and it’s becoming more open, flexible, and AI-ready than ever.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=86dfada637e1" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Snowflake University Platform Skills Badge]]></title>
            <link>https://omid-karami.medium.com/snowflake-university-platform-skills-badge-2b8f6b4d696d?source=rss-26d5566e003e------2</link>
            <guid isPermaLink="false">https://medium.com/p/2b8f6b4d696d</guid>
            <category><![CDATA[snowflake-data-cloud]]></category>
            <category><![CDATA[data-superhero]]></category>
            <category><![CDATA[snowflake]]></category>
            <dc:creator><![CDATA[Omid Karami]]></dc:creator>
            <pubDate>Sun, 24 May 2026 15:48:49 GMT</pubDate>
            <atom:updated>2026-05-24T15:48:49.680Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/400/1*H7AaW0sg4tDFLjzetLzKWQ.png" /><figcaption>Snowflake University Platform Skills Badge</figcaption></figure><p>Understanding the <strong>Snowflake University Platform Skills Badge</strong> — and the Retirement of the <strong>SnowPro Associate: Platform Certification</strong></p><p>The Snowflake learning ecosystem is evolving quickly, especially with the growing focus on AI, hands-on learning, and practical platform experience.</p><p>Recently, I completed the <strong>Snowflake University Platform Skills Badge</strong>, and during the process, I noticed that many people are asking questions such as:</p><ul><li>What exactly is the Platform Skills Badge?</li><li>Is it replacing certifications?</li><li>What happened to the SnowPro Associate: Platform Certification?</li><li>Is it worth taking?</li><li>How can someone enroll?</li></ul><p>In this article, I want to share a practical overview of the badge, explain the recent certification changes, and help anyone starting their Snowflake learning journey.</p><h3>What is the Snowflake University Platform Skills Badge?</h3><p>The Platform Skills Badge is a learning program provided through <a href="https://learn.snowflake.com/en/">Snowflake University</a> that focuses on practical understanding of the Snowflake platform.</p><p>Unlike traditional certification exams that mainly test theoretical knowledge through multiple-choice questions, the Skills Badge focuses more on:</p><ul><li>interactive learning,</li><li>hands-on exercises,</li><li>platform usage,</li><li>and practical workflows.</li></ul><p>The goal is to help learners become familiar with how Snowflake actually works in real environments.</p><p>The badge covers a broad range of platform fundamentals, including:</p><h3>🎯 Core Snowflake Concepts</h3><ul><li>Snowflake architecture</li><li>Storage vs Compute separation</li><li>Cloud Services layer</li><li>Virtual Warehouses</li><li>Databases, schemas, and tables</li></ul><h3>⚙️ Data Engineering Foundations</h3><ul><li>Loading data with COPY INTO</li><li>Stages and file formats</li><li>Structured, semi-structured, and unstructured data</li><li>Working with JSON and VARIANT columns</li><li>Querying and transformations</li></ul><h3>🔐 Security &amp; Governance</h3><ul><li>Roles and permissions</li><li>Access control</li><li>Time Travel</li><li>Data retention</li><li>Secure platform usage</li></ul><h3>🛠️ Snowsight &amp; Development Experience</h3><ul><li>Worksheets</li><li>Query history and monitoring</li><li>Snowflake Notebooks</li><li>SQL and Python notebook workflows</li></ul><h3>🤖ིྀ AI &amp; Modern Data Platform Features</h3><ul><li>Introduction to Snowflake Cortex AI</li><li>Text extraction and classification</li><li>Working with unstructured data</li><li>AI-assisted workflows inside Snowflake</li></ul><p>One thing I appreciated is that the learning path introduces modern platform capabilities beyond traditional SQL warehousing concepts.</p><h3>Why This Badge Matters</h3><p>The data platform landscape is changing rapidly.</p><p>Today, modern data engineers and architects are increasingly expected to understand:</p><ul><li>AI-enabled data platforms,</li><li>governance,</li><li>unstructured data,</li><li>notebooks,</li><li>and integrated analytics workflows.</li></ul><p>The Platform Skills Badge reflects this evolution.</p><p>Instead of focusing only on database theory, it introduces how a modern cloud data platform operates in practice.</p><p>For beginners, this creates a much smoother entry point into Snowflake.</p><p>For experienced professionals, it offers a structured overview of newer platform capabilities such as Cortex AI and Snowflake Notebooks.</p><h3>The Retirement of the SnowPro Associate: Platform Certification</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*fqQGID79fbDTETypT1DtVQ.jpeg" /><figcaption>SnowPro Associate: Platform Certification</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/400/1*Ltrk-FHNPFuvuwcXuYIrbQ.png" /><figcaption>SnowPro Associate: Platform Certification Badge</figcaption></figure><p>One of the most interesting developments is the retirement of the SnowPro Associate: Platform Certification.</p><p>For many years, SnowPro certifications were the primary official validation path for Snowflake skills.</p><p>However, Snowflake appears to be shifting toward a more skills-based learning model through Snowflake University and Skills Badges.</p><p>This reflects a broader trend across the technology industry:</p><ul><li>less emphasis on memorization,</li><li>more emphasis on practical skills,</li><li>hands-on labs,</li><li>and real platform experience.</li></ul><p>The retirement of the Associate-level Platform Certification does not reduce the importance of Snowflake certifications overall.</p><p>Instead, it seems to represent a transition toward:</p><ul><li>practical learning journeys,</li><li>modular badges,</li><li>role-oriented learning,</li><li>and AI-era platform capabilities.</li></ul><p>Advanced certifications and professional-level learning paths remain highly valuable.</p><h3>How to Enroll</h3><p>Getting started is straightforward.</p><h3>Step 1 — Create a Snowflake University Account</h3><p>Visit:</p><p><a href="https://learn.snowflake.com/en/">Snowflake University</a></p><p>and create a free account.</p><h3>Step 2 — Search for “Snowflake Platform Training”</h3><p>Inside the learning portal, search for:<br>“Snowflake Platform Training”</p><p>You will find the learning path and enrollment page.</p><h3>Step 3 — Complete the Learning Modules</h3><p>The modules include:</p><ul><li>videos,</li><li>readings,</li><li>quizzes,</li><li>and hands-on activities.</li></ul><h3>Step 4 — Complete the Assessment</h3><p>After completing the required modules and assessments, you receive the official badge, which you can share on LinkedIn.</p><h3>Who Should Take This Badge?</h3><p>I think this badge is particularly valuable for:</p><ul><li>Data Engineers</li><li>Data Analysts</li><li>BI Developers</li><li>Cloud Engineers</li><li>Analytics Engineers</li><li>Data Architects</li><li>Students entering the data field</li><li>Professionals transitioning into AI and modern data platforms</li></ul><p>It is especially useful for people who want a practical understanding of the Snowflake ecosystem without immediately jumping into advanced certifications.</p><h3>My Personal Thoughts</h3><p>From my perspective, the <strong>Platform Skills Badge</strong> is a smart direction for modern technical learning.</p><p>The platform world is becoming increasingly integrated:</p><ul><li>data engineering,</li><li>analytics,</li><li>governance,</li><li>AI,</li><li>notebooks,</li><li>and unstructured data workflows</li></ul><p>They are now closely connected.</p><p>Learning paths that combine these concepts are much more aligned with real industry workflows than purely theoretical exams.</p><p>I also think the inclusion of Cortex AI concepts is important because AI-native data platforms are becoming the future direction of cloud data ecosystems.</p><h3>Conclusion</h3><p>The <a href="https://learn.snowflake.com/en/courses/OD-SPT/"><strong>Snowflake University Platform Skills Badge</strong></a> is more than just a beginner badge.</p><p>It reflects how modern cloud platforms are evolving toward:</p><ul><li>practical learning,</li><li>integrated AI capabilities,</li><li>governed data products,</li><li>and hands-on platform literacy.</li></ul><p>For anyone starting their Snowflake journey — or wanting to understand where modern data platforms are heading — this is absolutely worth exploring.</p><p>If you are already working in data, analytics, AI, or cloud engineering, I would definitely recommend checking it out.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2b8f6b4d696d" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Understanding Snowflake Database Storage and Table Types]]></title>
            <link>https://omid-karami.medium.com/understanding-snowflake-database-storage-and-table-types-42db33ad53c7?source=rss-26d5566e003e------2</link>
            <guid isPermaLink="false">https://medium.com/p/42db33ad53c7</guid>
            <category><![CDATA[snowflake-data-cloud]]></category>
            <category><![CDATA[snowflake]]></category>
            <category><![CDATA[data-superhero]]></category>
            <dc:creator><![CDATA[Omid Karami]]></dc:creator>
            <pubDate>Wed, 20 May 2026 07:36:23 GMT</pubDate>
            <atom:updated>2026-05-20T07:36:23.322Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Sk0lCanFFEG-TZhAh3KJzA.png" /></figure><p>Snowflake is no longer designed only for traditional analytics workloads. Today’s architecture supports structured, semi-structured, and unstructured data while balancing analytics, operational systems, AI, governance, and interoperability across platforms.</p><p>One of the interesting aspects of it is how its database storage layer has evolved to support these different patterns within a single platform.</p><p>Snowflake supports three primary categories of data:</p><p>• <strong>Structured </strong>data — traditional rows and columns with fixed schemas<br>• <strong>Semi-structured</strong> data — formats such as JSON, XML, Avro, and Parquet<br>• <strong>Unstructured</strong> data — documents, images, audio, and video files</p><p>To support different workloads and architectural approaches, Snowflake provides multiple table types.</p><h3>Snowflake Tables</h3><p>Snowflake Tables are the default and most commonly used table type.</p><p>When data is loaded, Snowflake automatically reorganizes and stores it in an optimized compressed columnar format. Internally, Snowflake manages:</p><ul><li>Compression</li><li>Metadata</li><li>Statistics</li><li>Storage optimization</li><li>Micro-partitioning</li></ul><p>This abstraction removes much of the operational complexity traditionally associated with large-scale data platforms.</p><p>These tables are particularly well-suited for:</p><ul><li>Enterprise analytics</li><li>BI workloads</li><li>Modern cloud data warehouses</li><li>High-performance analytical processing</li></ul><p>The following example creates a <strong>Snowflake table</strong>:</p><pre>--Snowflake Table<br>CREATE OR REPLACE TABLE customers (<br>    customer_id NUMBER,<br>    customer_name STRING,<br>    email STRING,<br>    created_at TIMESTAMP<br>);</pre><h3>Apache Iceberg™ Tables</h3><p>Apache Iceberg™ Tables extend Snowflake capabilities into open lakehouse architectures.</p><p>With Iceberg Tables, data and metadata remain in external cloud storage, such as:</p><ul><li>Amazon S3</li><li>Azure Storage</li><li>Google Cloud Storage</li></ul><p>This model provides interoperability across multiple compute engines while still enabling Snowflake query and governance capabilities.</p><p>Iceberg Tables are especially relevant for:</p><ul><li>Existing data lakes</li><li>Open table format strategies</li><li>Multi-engine ecosystems</li><li>Lakehouse architectures</li><li>Organizations requiring external storage ownership</li></ul><p>The following example creates an <strong>Iceberg table</strong>:</p><pre>--Apache Iceberg™ Table<br>CREATE OR REPLACE ICEBERG TABLE customers_iceberg (<br>    customer_id NUMBER,<br>    customer_name STRING,<br>    email STRING,<br>    created_at TIMESTAMP<br>)<br>EXTERNAL_VOLUME = &#39;my_external_volume&#39;<br>CATALOG = &#39;SNOWFLAKE&#39;<br>BASE_LOCATION = &#39;customers_iceberg/&#39;;</pre><h3>Hybrid Tables</h3><p>Hybrid Tables are designed for workloads that require both transactional and analytical capabilities.</p><p>Unlike traditional analytical tables, Hybrid Tables support:</p><ul><li>Low-latency reads and writes</li><li>Row-level locking</li><li>Referential integrity</li><li>Transactional operations</li></ul><p>This enables operational and analytical workloads to coexist within the same platform architecture.</p><p>The following example creates a <strong>Hybrid table</strong>:</p><pre>--Hybrid Table<br>CREATE OR REPLACE HYBRID TABLE customers_hybrid (<br>    customer_id NUMBER PRIMARY KEY,<br>    customer_name STRING,<br>    email STRING UNIQUE,<br>    created_at TIMESTAMP<br>);</pre><h3>Final Thoughts</h3><p>What makes this evolution particularly interesting is that <strong>Snowflake</strong> is no longer positioned only as a data warehouse platform.</p><p>It is increasingly becoming a unified data platform capable of supporting:</p><ul><li><strong>Warehouses</strong></li><li><strong>Lakehouses</strong></li><li><strong>AI-ready architectures</strong></li><li><strong>Near real-time operational systems</strong></li><li><strong>Transactional + analytical workloads together</strong></li></ul><p>As modern architectures continue to evolve, storage layer design decisions are becoming just as important as compute and AI capabilities.</p><p><strong>Snowflake Official documentation:</strong><br><a href="https://docs.snowflake.com/en/user-guide/intro-key-concepts#database-storage">https://docs.snowflake.com/en/user-guide/intro-key-concepts#database-storage</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=42db33ad53c7" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>