<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Shubham Singh Rao on Medium]]></title>
        <description><![CDATA[Stories by Shubham Singh Rao on Medium]]></description>
        <link>https://medium.com/@shubhamsinghrao?source=rss-f29458f3446d------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/proxy/1*TGH72Nnw24QL3iV9IOm4VA.png</url>
            <title>Stories by Shubham Singh Rao on Medium</title>
            <link>https://medium.com/@shubhamsinghrao?source=rss-f29458f3446d------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sun, 22 Nov 2026 00:54:53 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@shubhamsinghrao/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[SAP Data in Snowflake Without a Single Pipeline: How Zero-Copy Actually Works]]></title>
            <link>https://medium.com/@shubhamsinghrao/sap-data-in-snowflake-without-a-single-pipeline-how-zero-copy-actually-works-0e7e8074f297?source=rss-f29458f3446d------2</link>
            <guid isPermaLink="false">https://medium.com/p/0e7e8074f297</guid>
            <category><![CDATA[sap-hana]]></category>
            <category><![CDATA[sap]]></category>
            <category><![CDATA[snowflake-data-cloud]]></category>
            <category><![CDATA[snowflake]]></category>
            <dc:creator><![CDATA[Shubham Singh Rao]]></dc:creator>
            <pubDate>Wed, 19 Aug 2026 16:55:53 GMT</pubDate>
            <atom:updated>2026-08-19T16:55:53.475Z</atom:updated>
            <content:encoded><![CDATA[<p><em>The integration everyone is talking about, explained by someone who read the docs instead of the press release.</em></p><p>If you have ever been handed the requirement “we need SAP data in Snowflake,” you know how the next six months go.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/790/1*6UGusJQaBU2L1OZevrcHBA.png" /></figure><p>You scope an extractor. You argue about whether to go through a third-party CDC tool or build on top of SLT. You discover that the business logic you actually needed lives inside a calculation view that nobody can explain. You land flat tables in Snowflake, and then you spend the rest of the project rebuilding the semantics — the hierarchies, the relationships, the “revenue means <em>this</em> not <em>that</em>” rules — that were sitting right there in SAP the whole time.</p><p>Then somebody asks why the numbers don’t match the SAP report, and you spend a quarter on reconciliation.</p><p>SAP BDC Connect for Snowflake is aimed squarely at that six months. The claim is that SAP data becomes queryable in Snowflake with no ETL, no replication, and — this is the part people skip past — with the business semantics intact.</p><p>I want to unpack how it actually works, because the marketing version and the engineering version are different documents.</p><h3>The mental model</h3><p>Forget pipelines for a second. The right analogy is Snowflake’s own data sharing.</p><p>When another Snowflake account shares data with you, you don’t receive a copy. You get a pointer. You run SELECT, your warehouse spins up, and it reads storage that lives on the provider&#39;s side. You pay for compute, not for a second copy of the data. Nobody schedules a job.</p><p>SAP BDC Connect for Snowflake extends that idea across the platform boundary. SAP publishes a <strong>data product</strong> in the BDC catalog. Snowflake mounts it as a <strong>catalog-linked database</strong>. Your queries read it in place.</p><p>There is no orchestration layer. There is no landing zone. There is no “did last night’s load finish” Slack message.</p><h3>The three moving parts</h3><h3>1. The connector</h3><p>On the Snowflake side, this is a schema-level object — same as a stage or a pipe:</p><pre>CREATE DATABASE IF NOT EXISTS sap_integration;<br>CREATE SCHEMA   IF NOT EXISTS sap_integration.bdc;</pre><pre>CREATE ZEROCOPY CONNECTOR sap_integration.bdc.sap_connector<br>  PARTNER = SAP_BDC;</pre><p>On its own, this object does nothing. It has to be enrolled with the SAP side using an invitation link that SAP generates when you provision BDC Connect:</p><pre>ALTER ZEROCOPY CONNECTOR sap_integration.bdc.sap_connector<br>  CONNECT WITH CONFIG = (<br>    INVITATION_LINK = &#39;&lt;link from SAP BDC tenant notifications&gt;&#39;<br>  );</pre><pre>DESC ZEROCOPY CONNECTOR sap_integration.bdc.sap_connector;</pre><p>The handshake is asynchronous. The connector moves NEW → CONNECTING → CONNECTED, and DESC is how you watch it. If it fails, a connection_error field tells you why.</p><p>Two gotchas that will cost you an afternoon if you don’t know them:</p><p><strong>The invitation link is single-use.</strong> Enroll it once, and if you need another connector, you go back to SAP for another link. There is no re-reading it.</p><p><strong>Somebody with ORGADMIN has to accept partner terms first.</strong> In Snowsight, under Admin → Terms. It is a one-time action per Snowflake <em>organization</em>, not per account, and you cannot self-revoke it afterwards. In a large enterprise, finding the person who holds ORGADMIN and getting them to click a legal acceptance is genuinely a multi-week risk. Start that conversation in week one, not week five.</p><h3>2. The share</h3><p>Nothing flows until SAP publishes something. In the BDC catalog, you find the data product, hit share, pick the target Snowflake account, and confirm.</p><p>Worth knowing: data products carry an <strong>Active / Inactive</strong> status. An inactive product will happily appear in the catalog and will happily not deliver data. If your mount comes back empty, check this before you start debugging the connector.</p><h3>3. The mount</h3><p>Back in Snowflake, ask what has arrived:</p><pre>SELECT SYSTEM$ZEROCOPY_CONNECTOR_LIST_SHARES(<br>  &#39;sap_integration.bdc.sap_connector&#39;);</pre><p>Then mount it:</p><pre>CREATE DATABASE sap_finance<br>  LINKED_ZEROCOPY_CONNECTOR = (<br>    CONNECTOR_NAME        = &#39;sap_integration.bdc.sap_connector&#39;,<br>    SHARE_NAME            = &#39;&lt;share name&gt;&#39;,<br>    SYNC_INTERVAL_SECONDS = 3600<br>  );</pre><p>One catalog-linked database per data product. Multiple databases can hang off one connector.</p><p>SYNC_INTERVAL_SECONDS is the one parameter people get wrong. It controls how often Snowflake discovers <strong>schema and table changes</strong> — not how often data refreshes, because data isn&#39;t refreshing, it&#39;s being read live. The range is 30 to 86,400 seconds, and the SAP BDC default is 86,400. That&#39;s one day. If you are demoing this to stakeholders and want them to see a structural change propagate, turn it down first.</p><h3>The part that actually matters</h3><p>Here is what separates this from every extractor you have ever built.</p><p>When you mount a data product, Snowflake automatically creates a read-only schema called snowflake$ inside the database. It contains <strong>Semantic Views</strong>, generated from SAP&#39;s Core Schema Notation.</p><p>Open one and you find entity definitions, primary keys, declared relationships between entities, and the comment text carried straight over from SAP.</p><p>Sit with that for a moment. The thing that normally takes a full modelling workstream — reconstructing what the SAP data <em>means</em> — arrives as metadata.</p><p>And because Snowflake Semantic Views are a first-class object, Cortex Analyst can sit on top of them immediately. Natural-language questions over SAP data, with SAP’s own definitions of the entities, without anybody writing a semantic layer.</p><p>If you demo this, demo <em>that</em>. Everyone expects the data to show up. Nobody expects the meaning to show up with it.</p><h3>Where the honesty lives</h3><p>Three things the launch material tends to soften.</p><p><strong>Zero-copy describes one hop, not the whole journey.</strong> No copy is created on the Snowflake side. But data products have to exist in BDC first. If your source is a modern S/4HANA landscape, many exist out of the box. If your source is classic ECC, they don’t — you build a custom data product, which means a real table-level ingest into BDC with a Datasphere wrapper on top. That ingest is a copy. The honest sentence is “no second copy in Snowflake,” not “no copy anywhere.” Say the honest one, because someone in the room will check.</p><p><strong>The published examples are easier than your problem.</strong> Most walkthroughs join two SAP data products to each other. That is a fine demonstration of the plumbing and a poor proxy for effort. The reason you want this integration is almost certainly to join SAP data to something that isn’t SAP — mainframe extracts, IoT telemetry, a warehouse of clickstream. That join needs a key that reliably matches across two systems that were never designed to agree. Confirm the key on sample values before you promise a timeline. “Both tables have a part number column” is not the same as “the part numbers match.”</p><p><strong>Check which implementation you’re reading about.</strong> This shipped first as an Openflow connector in Preview, driven by a parameter context and SYSTEM$SAP_BDC_LIST_SHARES. It has since become the SQL-native Zerocopy Connector shown above. A lot of material still in circulation describes the older path. The concepts are identical, the commands are not — and it is a frustrating way to lose a day.</p><h3>Is it worth it?</h3><p>If you have SAP and you have Snowflake, yes — but for a more specific reason than “less ETL.”</p><p>The pipeline cost was never really the pipeline. It was the semantic drift: the slow divergence between what SAP says a number means and what your warehouse says it means, and the reconciliation work that divergence generates forever.</p><p>Removing the copy removes the drift. That’s the actual win.</p><p>Everything else is a nice side effect.</p><p><em>Working on something similar, or hit a wall I didn’t cover? I’d genuinely like to hear about it — the failure modes are where the useful knowledge is.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=0e7e8074f297" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[I Wasted 3 Years on Data Pipelines. These 6 Snowflake Concepts Changed Everything.]]></title>
            <link>https://medium.com/@shubhamsinghrao/i-wasted-3-years-on-data-pipelines-these-6-snowflake-concepts-changed-everything-15bb4f83b57a?source=rss-f29458f3446d------2</link>
            <guid isPermaLink="false">https://medium.com/p/15bb4f83b57a</guid>
            <category><![CDATA[snowflake]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[data-engineering]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[technology]]></category>
            <dc:creator><![CDATA[Shubham Singh Rao]]></dc:creator>
            <pubDate>Sun, 19 Apr 2026 08:01:54 GMT</pubDate>
            <atom:updated>2026-04-19T08:01:54.298Z</atom:updated>
            <content:encoded><![CDATA[<h3>I Wasted 3 Years Building Data Pipelines. Then I Learned These 6 Snowflake Concepts.</h3><h3>The unfair advantages hiding in plain sight — and why most data engineers skip them.</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*WFWA_bJt0VGmbEiwMo_JPg.jpeg" /></figure><p>My first Snowflake project was a disaster.</p><p>Not because I didn’t know SQL. Not because I lacked data skills. But because I treated Snowflake like it was just another database — and completely ignored the features that make it extraordinary.</p><p>Two years into that job, a senior engineer handed me a cheat sheet. Six concepts. I read it in 10 minutes. Then I spent the next two weeks rewriting nearly everything I’d built.</p><p>Snowflake isn’t a better database. It’s a different category of tool — and most people never unlock what makes it special.</p><p>Here’s the six-concept framework I now use as a mental checklist before every architecture decision.</p><h4><strong>1. Virtual Warehouses — Your Elastic Compute Muscle</strong></h4><p>Stop thinking of a warehouse as “a server.” It’s a dynamically scalable compute cluster.</p><p>Scale up for heavy analytical queries. Scale out for concurrent users. And when no one’s running queries? It suspends automatically. You only pay for compute that actually runs.</p><p>This one change cut our monthly Snowflake bill by 40%.</p><h4><strong>2. Snowpipe — Serverless, Real-Time Ingestion</strong></h4><p>If you’re still writing batch ETL jobs just to land files into Snowflake, meet Snowpipe.</p><p>Drop a file in S3, GCS, or Azure Blob — Snowpipe detects it and loads automatically. Serverless. Event-driven. No scheduler. No cron job. Just data flowing continuously into your tables.</p><p>Key SEO note: Snowpipe is one of the most underused Snowflake features in production environments.</p><h4><strong>3. Stages — The Loading Dock Before Your Table</strong></h4><p>Before data lands in Snowflake, it passes through a Stage.</p><p>Think of it as a holding area backed by your cloud storage (S3, Azure, GCP). You inspect, validate, and transform files here before they ever touch your tables. It gives you a natural quality gate — a step most beginners skip entirely.</p><h4><strong>4. Native File Formats — CSV, JSON, Parquet as First-Class Citizens</strong></h4><p>Snowflake doesn’t force you to normalize everything before querying.</p><p>You can query semi-structured JSON directly inside a VARIANT column. Parse nested data with dot notation. No Spark cluster required for multi-format workloads.</p><p>This is a massive advantage for modern analytics engineering pipelines.</p><h4><strong>5. Time Travel — The Ultimate Undo Button</strong></h4><p>Accidentally dropped a table?<br> Ran a bad UPDATE without a WHERE clause?</p><p>Time Travel lets you query, clone, or restore any table at any point up to 90 days in the past. I’ve used this feature to save production data more times than I’d like to admit. It’s a career-saver disguised as a database feature.</p><h4>6. Zero-Copy Cloning — Full Environments at $0 Extra Cost</h4><p>Want a full production clone for testing? One command.</p><p>No data is physically copied — Snowflake uses metadata pointers under the hood. You get an instant, full-sized staging environment with zero extra storage cost. This fundamentally changes how you approach data development and testing.</p><p>The honest truth: most Snowflake users never go past Virtual Warehouses. They connect their BI tool, run queries, pay the bill, and call it done.</p><p>The engineers who stand out are the ones who know these six concepts cold — and can architect pipelines that are cheaper, faster, and more resilient because of it.</p><p>Which of these is Snowflake’s biggest “unfair advantage”? Drop your answer in the comments — I read every response.</p><p>Tags: #Snowflake #DataEngineering #CloudDataWarehouse #ETL #ModernDataStack #SQL #BigData #AnalyticsEngineering</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=15bb4f83b57a" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to Implement Writeback in Sigma + Snowflake (Step-by-Step Guide)]]></title>
            <link>https://medium.com/@shubhamsinghrao/how-to-implement-writeback-in-sigma-snowflake-step-by-step-guide-994be02923d0?source=rss-f29458f3446d------2</link>
            <guid isPermaLink="false">https://medium.com/p/994be02923d0</guid>
            <category><![CDATA[snowflake]]></category>
            <category><![CDATA[business-intelligence]]></category>
            <category><![CDATA[sigma-computing]]></category>
            <category><![CDATA[write-back]]></category>
            <category><![CDATA[snowflake-data-cloud]]></category>
            <dc:creator><![CDATA[Shubham Singh Rao]]></dc:creator>
            <pubDate>Sun, 05 Apr 2026 05:27:12 GMT</pubDate>
            <atom:updated>2026-04-05T05:27:12.043Z</atom:updated>
            <content:encoded><![CDATA[<h3>Introduction</h3><p>In modern data platforms, dashboards are no longer just for viewing data — they are becoming <strong>interactive systems</strong> where users can take action.</p><p>One of the most powerful use cases is <strong>Writeback</strong>, where users can input or update data directly from a BI tool.</p><p>In this article, I’ll walk you through how to implement <strong>Writeback using Sigma Computing and Snowflake</strong>, based on real-world experience.</p><h3>🔍 What is Writeback?</h3><p>Writeback allows users to:</p><ul><li>Enter data directly from dashboards</li><li>Update business inputs like targets and forecasts</li><li>Push changes back into the database</li></ul><p>👉 Example: A sales allocation tool where users assign targets directly from a dashboard.</p><h3>🧩 Architecture Overview</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*FnbOnVRAVLZ-sFKi.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*syDG-d8RsBeEqXcz.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*hN0tx-bQ0JDWcJp7.png" /></figure><p><strong>A typical Writeback architecture includes:</strong></p><ul><li><strong>Snowflake Target Table</strong> → Stores final data</li><li><strong>Staging Table</strong> → Temporary validation layer</li><li><strong>Sigma Input Table</strong> → User interaction layer</li><li><strong>Merge Logic</strong> → Syncs data into final table</li></ul><h3>🏗️ Step 1: Create Target Table in Snowflake</h3><p>🧾 Step 2: Create Staging Table</p><pre>CREATE OR REPLACE TABLE SALES_ALLOCATION (<br>    SALES_ORG STRING,<br>    SOLD_TO STRING,<br>    PROFIT_CENTER STRING,<br>    BRAND STRING,<br>    ALLOCATOR STRING,<br>    VALUE NUMBER<br>);</pre><h3><strong>🧾 Step 2: Create Staging Table</strong></h3><pre>CREATE OR REPLACE TABLE SALES_ALLOCATION_STAGE LIKE SALES_ALLOCATION;</pre><pre>👉 This table will temporarily store user inputs before final processing.</pre><h3>📊 Step 3: Build Input Table in Sigma</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/552/0*4JfWK3w6M5uRqC-f.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*7VBKnlucqfEmjCUV.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*CQbDMI0iKHUhm9Ok.png" /></figure><h3><strong>Steps:</strong></h3><ol><li>Create a <strong>Workbook in Sigma</strong></li><li>Add an <strong>Input Table</strong></li><li>Map columns to match Snowflake schema</li><li>Enable editing for users</li></ol><p>👉 This is where users will enter or update data.</p><h3>🔄 Step 4: Push Data to Snowflake (Writeback Logic)</h3><p>There are two approaches:</p><h3>✅ Option 1: Direct Insert (Basic)</h3><ul><li>Input table directly writes into Snowflake</li></ul><h3>✅ Option 2: Staging + Merge (Recommended)</h3><ul><li>Input → Staging Table</li><li>Validate → Merge into Target Table</li></ul><h3>⚙️ Step 5: Merge Data into Final Table</h3><pre>MERGE INTO SALES_ALLOCATION T<br>USING SALES_ALLOCATION_STAGE S<br>ON T.SALES_ORG = S.SALES_ORG<br>AND T.SOLD_TO = S.SOLD_TO</pre><pre>WHEN MATCHED THEN<br>UPDATE SET T.VALUE = S.VALUE</pre><pre>WHEN NOT MATCHED THEN<br>INSERT VALUES (<br>    S.SALES_ORG,<br>    S.SOLD_TO,<br>    S.PROFIT_CENTER,<br>    S.BRAND,<br>    S.ALLOCATOR,<br>    S.VALUE<br>);</pre><h3>💼 Real-World Use Cases</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*OUlaQd8sp_Ua2-Z_.jpg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*ww3DyVUmapcrdryS.png" /></figure><ul><li>Budget planning systems</li><li>Sales allocation tools</li><li>Forecast adjustments</li><li>Manual corrections in data</li></ul><h3>⚠️ Common Mistakes</h3><p>❌ Writing directly to production tables<br> ❌ No validation layer<br> ❌ Ignoring duplicate records<br> ❌ No audit tracking</p><h3>💡 Pro Tips</h3><p>✔ Always use a staging table<br> ✔ Add duplicate checks before merge<br> ✔ Track user inputs (created_by, timestamp)<br> ✔ Build rollback strategy<br> ✔ Use Snowflake tasks for automation</p><h3>🎯 Conclusion</h3><p>Writeback transforms dashboards from <strong>passive reporting tools</strong> into <strong>interactive decision-making systems</strong>.</p><p>With <strong>Sigma + Snowflake</strong>, you can build powerful applications that allow users to directly interact with data — without complex engineering.</p><h3>🚀 About the Author</h3><p><strong>Shubham Singh Rao</strong><br> Snowflake (4x Certified) | BI Expert | Sigma • Power BI • Tableau</p><p>Founder of <a href="https://techtown.in/"><strong>TechTown</strong></a><strong> — School Of Learnings</strong></p><p>👉 Follow for practical insights on Snowflake, BI, and real-world analytics</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=994be02923d0" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>