<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Chris Chandler on Medium]]></title>
        <description><![CDATA[Stories by Chris Chandler on Medium]]></description>
        <link>https://medium.com/@chris_omnata?source=rss-3504a00aa687------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*qg86slL9mqgQ2-Lj</url>
            <title>Stories by Chris Chandler on Medium</title>
            <link>https://medium.com/@chris_omnata?source=rss-3504a00aa687------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Wed, 18 Nov 2026 08:16:23 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@chris_omnata/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Snowflake & Salesforce integration: New options for 2024]]></title>
            <link>https://medium.com/@chris_omnata/snowflake-salesforce-integration-new-options-for-2024-237f83b56423?source=rss-3504a00aa687------2</link>
            <guid isPermaLink="false">https://medium.com/p/237f83b56423</guid>
            <category><![CDATA[integration]]></category>
            <category><![CDATA[data-architecture]]></category>
            <category><![CDATA[salesforce]]></category>
            <category><![CDATA[snowflake-data-cloud]]></category>
            <dc:creator><![CDATA[Chris Chandler]]></dc:creator>
            <pubDate>Mon, 15 Jan 2024 05:38:44 GMT</pubDate>
            <atom:updated>2024-01-15T05:38:44.201Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*p0tB3caAGpNe9mBCxU1DxA.png" /></figure><p>Snowflake and Salesforce, the world’s leading SQL data warehouse and leading CRM, is one of the most popular integration combinations. There are a multitude of SaaS middleware options that can move data between the platforms via API. Some platforms are event based like Zapier and Workato, others use replication like Fivetran or Hightouch, and others can do a bit of both like Mulesoft and Informatica.</p><p>At their core, when working with SaaS endpoints, these tools pick up data from one cloud, transit it through its own cloud, then land it in the target cloud. In 2024, this simply doesn’t need happen anymore…</p><h3>The end of the middleware era as native options emerge</h3><p>In this post we’ll compare two new Snowflake and Salesforce native options that will emerge over 2024:</p><ul><li>Snowflake Native Apps — Omnata Sync</li><li>Snowflake &amp; Salesforce Data Cloud native integration</li></ul><h3>Snowflake Native Apps change the game</h3><p>Omnata pioneered ‘no-middleware’ integration between Salesforce and cloud data warehouses with <a href="https://omnata.com/products/connect">Omnata Connect</a>.</p><p>Now, Snowflake enter the realm of native apps and massively increase the value that the data-warehouse can deliver to customers, even bringing into question the whole model of SaaS data tools.</p><h3>Omnata Sync</h3><p>Omnata Sync, which recently launched on the Snowflake Marketplace, is a first of its kind integration native app with no SaaS infrastructure. It enables bi-directional data sync between Snowflake and Salesforce, plus other SaaS endpoints.</p><p>Omnata Sync simplifies the SaaS architecture built around Snowflake; often referred to as the ‘modern data stack’ set of tools. ETL/ELT is ingesting data into Snowflake from Salesforce, and Reverse-ETL is outputting data from Snowflake to Salesforce.</p><h4>Use cases</h4><p>Two-way integration capabilities give you a powerful solution that requires no external services. The use cases fall into three buckets:</p><ul><li>Inbound into Snowflake — pull data from SaaS sources for BI and analytics. Understand how sales are progressing or support cases are being handled.</li><li>Outbound from Snowflake — activate modelled data to SaaS apps where they are consumed by users and processes. Output lead scores, metrics and related enterprise data for CRM users.</li><li>Loops — pull data from an app, model and enrich the data, then push it back to the same app. Enrich your Lead, Contact and Account data using enterprise data or marketplace datasets.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*jXOxZjKmmijNtj5u.png" /></figure><h3>Native apps are faster to deploy and reduce SaaS integration costs</h3><p>Omnata’s novel native integration architecture has two major categories of benefit for enterprises.</p><ol><li>Faster and easier cyber-security approval. <br>Omnata Sync has no data-handling infrastructure, everything resides inside your Snowflake account.</li><li>Reduced costs. <br>Customers can reduce the number of integration tools and Omnata does not charge for data volume or compute due to its native app architecture.</li></ol><h3>How does Omnata Sync compare to Salesforce Data Cloud (CDP) ?</h3><p>The best way to think about Salesforce Data Cloud is by its original name, a CDP (Customer Data Platform) that lives within the Salesforce ecosystem. However, it’s important to understand that Data Cloud is a separately sold product, and underlying infrastructure, from the Salesforce CRM and Marketing Clouds.</p><h3>Salesforce Data Cloud is native, but an extra hop</h3><p>Salesforce Data Cloud is native to the Salesforce ecosystem and provides a near ‘real-time’ integration with Snowflake.</p><p>However, this is only for the first ‘hop’, between Snowflake and Salesforce Data Cloud. To move data from Snowflake to Salesforce CRM Objects via CDP, an internal data sync is required. This sync is subject to the same performance as the regular APIs; 10–15 minute sync latency. In addition, the Salesforce admin must configure data models in CDP which can duplicate work typically done in Snowflake.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*pKl0lOB9mILQzzBw.png" /></figure><p>So overall, the integration architecture via Salesforce Data Cloud is actually: Snowflake → Salesforce Data Cloud → Salesforce CRM Objects, and vice versa. In addition, platform and usage-based costs make this a heavy-weight option for bi-directional data syncs.</p><h3>Omnata Sync connects directly to the Salesforce CRM APIs</h3><p>Built upon Snowflake’s new Native App framework and utilising new platform features like External Access, Omnata Sync pulls and pushes data directly between Snowflake and the Salesforce APIs.</p><p>The result is a single hop integration with benefits like:</p><ul><li>Lower sync latency; as low as 1min.</li><li>Advanced API features; like merge and convert records.</li><li>Simpler configuration; via a no-code native app UI.</li><li>Reduced licensing costs; no platform add-ons required.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ZeqQz51EqIIKp4acOCi0AA.png" /></figure><h3>Try Omnata Sync for Snowflake and Salesforce integration for free</h3><p>You can be up and running in minutes from the Snowflake Marketplace deploying bi-directional Snowflake and Salesforce integration for far less than SaaS middleware with unlimited data volume and sync frequency.</p><p>Start a <a href="https://app.snowflake.com/marketplace/providers/GZSUZ59II4/Omnata">30-day free trial</a> today, or <a href="https://omnata.com/contact-us">get in touch with us</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=237f83b56423" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Best External Databases for Salesforce Connect]]></title>
            <link>https://medium.com/inside-the-salesforce-ecosystem/best-external-databases-salesforce-connect-881a8ced8af0?source=rss-3504a00aa687------2</link>
            <guid isPermaLink="false">https://medium.com/p/881a8ced8af0</guid>
            <category><![CDATA[salesforce]]></category>
            <category><![CDATA[salesforce-tools]]></category>
            <category><![CDATA[apps]]></category>
            <category><![CDATA[appexchange]]></category>
            <category><![CDATA[saas]]></category>
            <dc:creator><![CDATA[Chris Chandler]]></dc:creator>
            <pubDate>Tue, 08 Feb 2022 15:03:06 GMT</pubDate>
            <atom:updated>2022-02-08T15:03:06.605Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="A man wearing a button-down bird shirt writes the word ‘database’ on a mirror with a black marker." src="https://cdn-images-1.medium.com/max/1024/1*A2hIVUWlsVrjIQoaTG6Aig.jpeg" /></figure><p>As data grows many enterprises will need to think differently about how to architect their CRM for large data volumes. With an external database, <a href="https://www.salesforce.com/au/products/platform/products/salesforce-connect/">Salesforce Connect</a> is a great way to integrate outside systems or offload non-core or historical data. Salesforce Connect virtualizes data in external objects which solves the problems of storage or replicating large volumes of data via API.</p><p>With the big cloud vendors you can spin all manner of databases easily and at a relatively low cost, but what database should you choose for Salesforce Connect?</p><p>To answer this, let’s look at the core capabilities of cloud database vendors and offer some recommendations that we use and support.</p><p>The marketing lingo has evolved a lot in the last five years. Cloud databases, cloud data warehouses, data lakes, and the modern data stack all refer to the same trend but with nuances in architecture and capabilities. If you’re a Salesforce expert, this blog will help inform you of the leaders in the cloud data warehouse market.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*OFU0OOgePthMpfSCMXGDzA.png" /></figure><h3>Cloud Database Capabilities to Consider</h3><p>Overall, for Salesforce Connect you need to have speed, scalability, and connectivity. While Salesforce Connect was originally used for OData integrations, connecting live to your legacy application database is generally a bad idea. Not least because of how this workload will disrupt its performance. The significant gains will come from centralizing enterprise data and creating better data processing steps.</p><h3>Storage Costs</h3><p>Salesforce storage costs are very high compared to today’s public cloud options, however, the premium reflects the proximity of storage to the core CRM. Storing related or external data in the CRM is very convenient and therefore costly, whereas using blob storage is super cheap but requires extra services to process it.</p><p>The new era of cloud database vendors has wrapped up these extra services and called them modern cloud data warehouses. As a result of this architecture, cloud vendors like Snowflake sell storage at the same price as the public cloud vendors. Snowflake charge $23/month for 1TB of storage, the same as an AWS S3 bucket.</p><p>Moving large volumes of data between enterprise systems is challenging. A cloud data warehouse allows you to take advantage of ultra-low-cost storage, and when paired with Salesforce Connect, makes access as convenient as on-platform data. It’s the best of both worlds.</p><h3>ETL to ELT</h3><p>The leading cloud data warehouses are architected with a complete separation of compute and storage so huge volumes of data can sit in storage without affecting compute operations. This has led to a new paradigm of data integration, ELT.</p><p><em>Bonus: </em><a href="https://fivetran.com/blog/etl-vs-elt"><em>Read more about ETL vs ELT</em></a></p><p>In summary, instead of limiting the amount of data extracted from source systems, new tools simply replicate and land raw data in the data warehouse. Tools like <a href="https://fivetran.com/">Fivetran</a>, <a href="https://www.stitchdata.com/">Stitch</a>, and <a href="https://airbyte.io/">Airbyte</a> make it incredibly simple and automated to replicate Salesforce data to a cloud data warehouse. Once this has happened, consolidation and modeling of data can be performed within the data warehouse itself using popular frameworks like <a href="https://getdbt.com/">debt</a>.</p><h3>Fast Query Response</h3><p>Salesforce Connect will call out to the external database and return results to the external object. For something like a related list or report, this happens while the page loads. This means your database needs to be able to process the query at this time to ensure users are not waiting for a lagging component or report.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*iMiyWclqTrGzVnOFypeHyQ.png" /></figure><p>There are many databases available in the cloud, only a few vendors have designed new query engines from scratch for the cloud. For relational databases like <a href="https://www.snowflake.com/">Snowflake</a> and <a href="https://cloud.google.com/bigquery">BigQuery</a>, the types of queries run are fairly easy to process and so perform well, but large tables may benefit from some indexing work. On the other hand, <a href="https://rockset.com/">Rockset</a> is a new database that indexes all columns by default and is optimized for millisecond query response. Given the expected Salesforce page load times, you don’t need to optimize for absolute speed, but it must be fast enough for a good user experience.</p><p>The best overall performing external database also needs to be scalable for operational use.</p><h3>Scalable Compute</h3><p>Modern cloud data warehouses have largely solved the scalability problem. When using a data warehouse as external storage, you need a technology that can scale to the workload created by CRM users and processes that run off it. With up to thousands of CRM users, scaling for concurrency is the most important factor.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*fwgOMCK9BMVNRywy5IPv3w.png" /></figure><p>Snowflake allows you to have your Salesforce Connect queries hit a dedicated compute resource. If you have many concurrent CRM users, Snowflake also has the option of <a href="https://www.snowflake.com/blog/auto-scale-snowflake-major-leap-forward-massively-concurrent-enterprise-applications/">auto-scaling</a> horizontally. Say you had surges of queries, like a sales team doing their forecasts at the same time. Snowflake would automatically add compute resources up to the limits you set and then automatically switch them off as demand subsides. Snowflake charges you by the compute time and allows you to micro-manage that time.</p><p>On the other hand, <a href="https://cloud.google.com/blog/products/gcp/understanding-bigquerys-rapid-scaling-and-simple-pricing">BigQuery works in a more automated way</a>. It will auto-scale vertically and horizontally as needed, but the data warehouse admins have less control over exactly how. BigQuery charges you per byte processed in a query and allows you to set limits on what queries will run.</p><p>In either case, these platforms allow you to manage costs, share workloads, and optimize performance for your Salesforce Connect use case.</p><h3>Truly Cloud-Based</h3><p>Before the latest generation of cloud data warehouses, ‘cloud databases’ were simply on-premise databases hosted in a public cloud. This means they live inside a cloud VPC and don’t have an HTTP API endpoint.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Y-oEnqDw12tn9tLYQUUEqw.png" /></figure><p>Platforms like Snowflake, BigQuery, and Rockset are all ‘true cloud services. All customers share the same service, which means they can be queried from a public endpoint. Apps like Omnata can query them directly from another cloud without middleware infrastructure. Middleware-free integration removes the complexity of a third party processing data and significantly reduces the cost of tools, people, and cyber-security overheads.</p><h3>What Have We Left Out?</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*hnJvuDJYKYxIAJsTRu5iaA.png" /></figure><p><a href="https://www.heroku.com/connect">Heroku Postgres</a> and Salesforce have a native integration, called Heroku Connect, that uses OData with external objects. Heroku Postgres is great for developers and recommended when you have an app that bridges both Heroku and Salesforce. However, it’s not as well-suited to being a central data repository. As a transactional database, it requires a lot of close attention to indexing compared to a modern, columnar, cloud data platform. Heavy workloads, large datasets, and many concurrent users could create challenges.</p><p>AWS <a href="https://aws.amazon.com/redshift/">Redshift</a> became popular in the first iteration of cloud data warehouses. It is somewhat less known Redshift was <a href="https://en.wikipedia.org/wiki/Amazon_Redshift">licensed</a> from Actian and is built on top of Postgres. For our purposes, Redshift isn’t truly cloud-based as it runs inside your AWS VPC. While initially groundbreaking in speed and scalability, more recently Snowflake and BigQuery have upped the standard for large datasets and concurrent users.</p><p><a href="https://databricks.com/">Databricks</a> is another heavy hitter in the modern data platform space that is built on Apache Spark. Originally optimized for data processing and machine learning, they have made moves towards more general SQL-like capabilities with Databricks SQL. However, like AWS Redshift, Databricks is deployed inside a customer’s cloud VPC and so its <a href="https://docs.databricks.com/sql/admin/sql-endpoints.html">API endpoint</a> can only be accessed from within the instance. For our purposes, Databricks don’t offer a ‘true cloud service’ that can be queried directly from another cloud, but we’re hopeful that this will be on the roadmap.</p><h3>What do we recommend? Omnata Uses and Supports Snowflake, BigQuery, and Rockset</h3><p>Omnata’s <a href="https://appexchange.salesforce.com/appxListingDetail?listingId=a0N3u00000MBcLmEAL">native Salesforce app</a> lets you integrate enterprise data from the leading cloud data platforms. To do this, you need database technology that can scale to handle large datasets and large numbers of concurrent users. You also need to reduce the costs of middleware and specialized integrators. This is achieved by removing the middleware layer by choosing a true cloud platform that can be queried directly.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*D3avOcQjYhQ9onYrBpk38Q.png" /></figure><p>Overall, many organizations are already centralizing enterprise data on platforms like Snowflake and BigQuery and these are your best bet to use with Salesforce Connect. However, if you have a narrower scope of real-time use cases then Rockset is a compelling option.</p><h3>Does Your Company Already Use One of These Technologies?</h3><p>These new technologies have made it much easier to bring on new teams, datasets, and use cases. You should spark up a conversation with your data team today, or <a href="https://omnata.com/">talk to Omnata</a> about how you can get value from these platforms.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=881a8ced8af0" width="1" height="1" alt=""><hr><p><a href="https://medium.com/inside-the-salesforce-ecosystem/best-external-databases-salesforce-connect-881a8ced8af0">The Best External Databases for Salesforce Connect</a> was originally published in <a href="https://medium.com/inside-the-salesforce-ecosystem">AppExchange and the Salesforce Ecosystem</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Large Data Volumes in Salesforce — a new approach for 2021]]></title>
            <link>https://medium.com/omnata/large-data-volumes-in-salesforce-a-new-approach-for-2021-26032f2babbd?source=rss-3504a00aa687------2</link>
            <guid isPermaLink="false">https://medium.com/p/26032f2babbd</guid>
            <category><![CDATA[salesforce-tools]]></category>
            <category><![CDATA[salesforce]]></category>
            <category><![CDATA[data-integration]]></category>
            <dc:creator><![CDATA[Chris Chandler]]></dc:creator>
            <pubDate>Mon, 10 May 2021 08:10:11 GMT</pubDate>
            <atom:updated>2021-05-10T08:10:34.427Z</atom:updated>
            <content:encoded><![CDATA[<h3>Large Data Volumes in Salesforce — a new approach for 2021</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*fACHxEracila7_Kj.png" /></figure><p><em>Originally published at </em><a href="https://omnata.com/blog"><em>omnata.com/blog</em></a></p><p>Large data volumes in Salesforce can be challenging and there is a bunch of advice circulating on how to handle it best.</p><p>According to Salesforce, ‘large data volumes’ is defined as:<br>“an imprecise, elastic term. If your deployment has tens of thousands of users, tens of millions of records, or hundreds of gigabytes of total record storage, you have a large data volume.”</p><p>By the standards of modern data architectures, ‘tens of millions of records’ would hardly classify as large, but for Salesforce and many cloud applications, storing and processing big data was simply never intended to be a core capability.</p><p>This article will talk about large data volumes from across an enterprise, not data that has grown organically in Salesforce like having lots of customer accounts.</p><h4>Two key architectural trends are happening:</h4><ul><li>Companies are centralising data in cloud data warehouses like Snowflake and BigQuery.</li><li>Platforms like Salesforce are systems of action, but they are no longer the system of record for all customer data.</li></ul><p>In practical terms, if I use Service Cloud for customer support and run a high volume business with a hundred thousand customers, I’m not loading every single customer I’ve ever interacted with into Salesforce. They only become a record once they need support. It’s the system of action for my service agents.</p><p>The system of record has become the data warehouse. Containing a modelled view of each customer that combines all of their touchpoints across systems. This single view is used all across the business by people and applications. My service agent needs access to this to understand things like product licenses, usage, transactions, and contact history with other teams.</p><h3>What does that change?</h3><p>Considering the trends above and the following flow diagram for handling large data volumes in Salesforce, companies with modern data platforms increasingly fall into the ‘NO + NO’ path, which leads us to Salesforce Connect.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/999/0*MVXw_7HbiA1UKgCn.png" /><figcaption>Source: Heidi Lemmetyinen (https://medium.com/big-object/learn-the-strategies-for-managing-big-data-in-salesforce)</figcaption></figure><p>Salesforce Connect is often recommended for org to org connections or creating single-purpose integrations to other systems, usually via Odata or middleware like Mulesoft or Jitterbit. It isn’t a new feature, but it hasn’t been leveraged as an agile data connection before.</p><h3>A new way to use Salesforce Connect</h3><p>Omnata allows you to directly connect Salesforce to cloud data warehouses; Snowflake and BigQuery. These platforms are designed to handle operational scale workloads and using live-queries solves a bunch of problems faced by traditional extract and load integrations. Using Omnata unlocks integration use cases like product data, revenue and transactional data, machine learning outputs, and patient records.</p><h4>No limits on dataset size</h4><p>Omnata allows you to query from production tables in Snowflake or BigQuery that could contain millions or billions of records. Live-query removes the limits of storage and API quota and doesn’t bog down performance.</p><h4>No longer maintain copies in Salesforce</h4><p>The integrity of the copy or handling load and update failures cease to be problems to deal with.</p><h4>Real-time data freshness</h4><p>Live-queries mean data is always up to date.</p><h4>Leverage the source of truth</h4><p>Your data team are already centralizing and modelling all enterprise data so there’s no need to redo this work by integrating point-to-point.</p><h4>Secure connection directly to your warehouse</h4><p>Omnata runs in your Salesforce environment and there is no separately hosted service handling your data or credentials. From a security point of view, this significantly reduces your attack surface and makes it straightforward for initial security reviews and ongoing vendor assurance activities. From a privacy point of view, it removes the need for yet another third party data processor you need to trust.</p><h3>Enabling Salesforce teams to do more</h3><p>Salesforce admins and architects now have the ability to leverage all enterprise data in external objects. The best part of external objects is that data appears natively and can be used in platform features like reporting and automation. On top of this, Omnata makes big improvements to the out-of-the-box functionality of Salesforce Connect so that it’s far more user friendly and flexible. All of this helps the CRM team move faster and create a more valuable and enriched workspace.</p><h4>Automates external object creation</h4><p>Save hours of setup time defining data types and swapping between systems with a user-friendly setup experience.</p><h4>Simplifies relationship management</h4><p>Easily create parent-child relationships, rename fields for usability and create drillable object relationships.</p><h4>Handles underlying schema changes</h4><p>A single purpose integration breaks whenever the underlying schema changes, and this can happen a lot with agile deployments. Omnata gracefully handles changes instead of starting from scratch.</p><h3>Transform Salesforce into a data driven system of action</h3><p>The combination of Omnata and Salesforce Connect can help you rethink your large data volume architecture. It is simple, agile and suits the shift in enterprise data architectures. New data warehouses are fast, scalable, and reliable to support operational workloads. Omnata builds the critical link between your new system of record and primary system of action.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=26032f2babbd" width="1" height="1" alt=""><hr><p><a href="https://medium.com/omnata/large-data-volumes-in-salesforce-a-new-approach-for-2021-26032f2babbd">Large Data Volumes in Salesforce — a new approach for 2021</a> was originally published in <a href="https://medium.com/omnata">Omnata</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>