<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Rinkesh Pati on Medium]]></title>
        <description><![CDATA[Stories by Rinkesh Pati on Medium]]></description>
        <link>https://medium.com/@rinkesh.pati?source=rss-437d3986d319------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*tLzcw6ixxZGAEMVJSWjQVg@2x.jpeg</url>
            <title>Stories by Rinkesh Pati on Medium</title>
            <link>https://medium.com/@rinkesh.pati?source=rss-437d3986d319------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 08:41:23 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@rinkesh.pati/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Whispers of Wisdom!]]></title>
            <link>https://medium.com/weeds-wildflowers/whispers-of-wisdom-4631b2a93661?source=rss-437d3986d319------2</link>
            <guid isPermaLink="false">https://medium.com/p/4631b2a93661</guid>
            <category><![CDATA[life-lessons]]></category>
            <category><![CDATA[spirituality]]></category>
            <category><![CDATA[words-of-wisdom]]></category>
            <category><![CDATA[fatherhood]]></category>
            <category><![CDATA[father-and-son]]></category>
            <dc:creator><![CDATA[Rinkesh Pati]]></dc:creator>
            <pubDate>Fri, 19 Dec 2025 16:58:46 GMT</pubDate>
            <atom:updated>2025-12-19T16:58:46.819Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6h6JftUCGBhmfqwTK3Vqkg@2x.jpeg" /><figcaption>Photo by Elias Maurer on <a href="https://unsplash.com/photos/a-couple-of-people-standing-on-a-beach-under-a-cloudy-sky-rI5q0m07MIk">Unsplash</a></figcaption></figure><p>Hey there, buddy, with your scraped-up knees,</p><p>Running wild like the wind through the trees.</p><p>You’re nine years old, full of fire and dreams,</p><p>But life’s a river, twisting with streams.</p><p>I watch you grow, and my heart fills with pride,</p><p>But time slips away like the evening tide.</p><p>Son, listen close to these words from my soul,</p><p>The world’s big and bright, but it takes its toll.</p><p>Be kind to the broken, stand tall when you fall,</p><p>Chase what you love, give your heart to it all.</p><p>You’re my little man, but one day you’ll see,</p><p>The deepest truths are in you and me.</p><p>Remember, kid, not everything’s fair,</p><p>Some days are storms, some burdens to bear.</p><p>But in every shadow, there’s light you can find,</p><p>Forgive when it’s hard, leave grudges behind.</p><p>The stars up above, they’ve been here so long,</p><p>Teaching us patience, where we belong.</p><p>One day I’ll be gone, but my voice will remain,</p><p>In the choices you make, through joy and through pain.</p><p>Hold onto wonder, question the why,</p><p>Life’s a great mystery under the sky.</p><p>You’re stronger than you know, my boy, believe,</p><p>in the quiet moments, that’s where you’ll achieve.</p><p>Son, listen close to these words from my soul,</p><p>The world’s big and bright, but it takes its toll.</p><p>Be kind to the broken, stand tall when you fall,</p><p>Chase what you love, give your heart to it all.</p><p>You’re my little man, but one day you’ll see,</p><p>The deepest truths are in you and me.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4631b2a93661" width="1" height="1" alt=""><hr><p><a href="https://medium.com/weeds-wildflowers/whispers-of-wisdom-4631b2a93661">Whispers of Wisdom!</a> was originally published in <a href="https://medium.com/weeds-wildflowers">Weeds &amp; Wildflowers</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The prison of mind]]></title>
            <description><![CDATA[<div class="medium-feed-item"><p class="medium-feed-image"><a href="https://medium.com/weeds-wildflowers/the-prison-of-mind-64880927771a?source=rss-437d3986d319------2"><img src="https://cdn-images-1.medium.com/max/2600/0*iCllh9CrOH50nhB4" width="3600"></a></p><p class="medium-feed-snippet">A poem</p><p class="medium-feed-link"><a href="https://medium.com/weeds-wildflowers/the-prison-of-mind-64880927771a?source=rss-437d3986d319------2">Continue reading on Weeds &amp; Wildflowers »</a></p></div>]]></description>
            <link>https://medium.com/weeds-wildflowers/the-prison-of-mind-64880927771a?source=rss-437d3986d319------2</link>
            <guid isPermaLink="false">https://medium.com/p/64880927771a</guid>
            <category><![CDATA[suffering]]></category>
            <category><![CDATA[truth]]></category>
            <category><![CDATA[salvation]]></category>
            <category><![CDATA[mind]]></category>
            <category><![CDATA[life]]></category>
            <dc:creator><![CDATA[Rinkesh Pati]]></dc:creator>
            <pubDate>Mon, 26 Oct 2020 15:52:31 GMT</pubDate>
            <atom:updated>2020-10-26T15:52:31.686Z</atom:updated>
        </item>
        <item>
            <title><![CDATA[Transience]]></title>
            <link>https://medium.com/weeds-wildflowers/transience-8ac6eaa94ae2?source=rss-437d3986d319------2</link>
            <guid isPermaLink="false">https://medium.com/p/8ac6eaa94ae2</guid>
            <category><![CDATA[philosophy]]></category>
            <category><![CDATA[truth]]></category>
            <category><![CDATA[death]]></category>
            <category><![CDATA[life]]></category>
            <category><![CDATA[mortality]]></category>
            <dc:creator><![CDATA[Rinkesh Pati]]></dc:creator>
            <pubDate>Sat, 29 Aug 2020 12:33:06 GMT</pubDate>
            <atom:updated>2020-08-29T12:33:06.778Z</atom:updated>
            <content:encoded><![CDATA[<p>Mortal.. we all are!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*jQ3vu0vXxKxq-XlT" /><figcaption>Photo by <a href="https://unsplash.com/@grakozy?utm_source=medium&amp;utm_medium=referral">Greg Rakozy</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>One fine day<br>I’ll say goodbye</p><p>To all I know<br>Friend and foe<br>The life I lived<br>The world I kissed<br>My failure and success<br>The wins and losses</p><p>The love I got<br>The battles I fought<br>The cheers and accolade<br>The caustic bitter tirade</p><p>One fine day<br>I’ll say goodbye!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8ac6eaa94ae2" width="1" height="1" alt=""><hr><p><a href="https://medium.com/weeds-wildflowers/transience-8ac6eaa94ae2">Transience</a> was originally published in <a href="https://medium.com/weeds-wildflowers">Weeds &amp; Wildflowers</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Awaiting You]]></title>
            <description><![CDATA[<div class="medium-feed-item"><p class="medium-feed-image"><a href="https://medium.com/weeds-wildflowers/awaiting-you-6b07ba4a2b83?source=rss-437d3986d319------2"><img src="https://cdn-images-1.medium.com/max/2600/0*1KN4N7ALnWZ7wELl" width="5362"></a></p><p class="medium-feed-snippet">A father speaking with his to-be-born baby!</p><p class="medium-feed-link"><a href="https://medium.com/weeds-wildflowers/awaiting-you-6b07ba4a2b83?source=rss-437d3986d319------2">Continue reading on Weeds &amp; Wildflowers »</a></p></div>]]></description>
            <link>https://medium.com/weeds-wildflowers/awaiting-you-6b07ba4a2b83?source=rss-437d3986d319------2</link>
            <guid isPermaLink="false">https://medium.com/p/6b07ba4a2b83</guid>
            <category><![CDATA[life]]></category>
            <category><![CDATA[baby]]></category>
            <category><![CDATA[fatherhood]]></category>
            <category><![CDATA[parenthood]]></category>
            <category><![CDATA[love]]></category>
            <dc:creator><![CDATA[Rinkesh Pati]]></dc:creator>
            <pubDate>Sun, 23 Aug 2020 18:21:01 GMT</pubDate>
            <atom:updated>2020-08-23T18:21:01.657Z</atom:updated>
        </item>
        <item>
            <title><![CDATA[The Wounded Monk]]></title>
            <description><![CDATA[<div class="medium-feed-item"><p class="medium-feed-image"><a href="https://medium.com/illumination/the-wounded-monk-75518eace1fb?source=rss-437d3986d319------2"><img src="https://cdn-images-1.medium.com/max/2600/0*7EBieP-S0ZGQvFCr" width="6000"></a></p><p class="medium-feed-snippet">The path to solitude</p><p class="medium-feed-link"><a href="https://medium.com/illumination/the-wounded-monk-75518eace1fb?source=rss-437d3986d319------2">Continue reading on ILLUMINATION »</a></p></div>]]></description>
            <link>https://medium.com/illumination/the-wounded-monk-75518eace1fb?source=rss-437d3986d319------2</link>
            <guid isPermaLink="false">https://medium.com/p/75518eace1fb</guid>
            <category><![CDATA[spirituality]]></category>
            <category><![CDATA[solitude]]></category>
            <category><![CDATA[life]]></category>
            <category><![CDATA[poetry]]></category>
            <category><![CDATA[poem]]></category>
            <dc:creator><![CDATA[Rinkesh Pati]]></dc:creator>
            <pubDate>Sat, 22 Aug 2020 19:00:58 GMT</pubDate>
            <atom:updated>2020-08-22T19:00:58.151Z</atom:updated>
        </item>
        <item>
            <title><![CDATA[The Radiant Dawn]]></title>
            <description><![CDATA[<div class="medium-feed-item"><p class="medium-feed-image"><a href="https://medium.com/weeds-wildflowers/the-radiant-dawn-5b8ec0b951f7?source=rss-437d3986d319------2"><img src="https://cdn-images-1.medium.com/max/2600/0*YyZyj_Wc44sIRF_2" width="2758"></a></p><p class="medium-feed-snippet">As you step into my life</p><p class="medium-feed-link"><a href="https://medium.com/weeds-wildflowers/the-radiant-dawn-5b8ec0b951f7?source=rss-437d3986d319------2">Continue reading on Weeds &amp; Wildflowers »</a></p></div>]]></description>
            <link>https://medium.com/weeds-wildflowers/the-radiant-dawn-5b8ec0b951f7?source=rss-437d3986d319------2</link>
            <guid isPermaLink="false">https://medium.com/p/5b8ec0b951f7</guid>
            <category><![CDATA[love]]></category>
            <category><![CDATA[life]]></category>
            <category><![CDATA[spouse]]></category>
            <category><![CDATA[companion]]></category>
            <category><![CDATA[life-partner]]></category>
            <dc:creator><![CDATA[Rinkesh Pati]]></dc:creator>
            <pubDate>Sun, 16 Aug 2020 22:31:01 GMT</pubDate>
            <atom:updated>2020-08-17T15:22:01.520Z</atom:updated>
        </item>
        <item>
            <title><![CDATA[Salvation]]></title>
            <description><![CDATA[<div class="medium-feed-item"><p class="medium-feed-image"><a href="https://medium.com/weeds-wildflowers/salvation-a1bed53e298f?source=rss-437d3986d319------2"><img src="https://cdn-images-1.medium.com/max/2600/0*qhIJ3zG83gMH5oQz" width="5184"></a></p><p class="medium-feed-snippet">Let your soul stay unruffled</p><p class="medium-feed-link"><a href="https://medium.com/weeds-wildflowers/salvation-a1bed53e298f?source=rss-437d3986d319------2">Continue reading on Weeds &amp; Wildflowers »</a></p></div>]]></description>
            <link>https://medium.com/weeds-wildflowers/salvation-a1bed53e298f?source=rss-437d3986d319------2</link>
            <guid isPermaLink="false">https://medium.com/p/a1bed53e298f</guid>
            <dc:creator><![CDATA[Rinkesh Pati]]></dc:creator>
            <pubDate>Sat, 15 Aug 2020 22:53:02 GMT</pubDate>
            <atom:updated>2020-08-16T12:48:18.311Z</atom:updated>
        </item>
        <item>
            <title><![CDATA[NoSQL database options on the three leading cloud platforms]]></title>
            <link>https://medium.com/@rinkesh.pati/nosql-database-options-on-the-three-leading-cloud-platforms-176200407aa0?source=rss-437d3986d319------2</link>
            <guid isPermaLink="false">https://medium.com/p/176200407aa0</guid>
            <category><![CDATA[google-cloud-datastore]]></category>
            <category><![CDATA[cosmosdb]]></category>
            <category><![CDATA[nosql]]></category>
            <category><![CDATA[dynamodb]]></category>
            <category><![CDATA[cloud-database]]></category>
            <dc:creator><![CDATA[Rinkesh Pati]]></dc:creator>
            <pubDate>Fri, 22 May 2020 14:45:20 GMT</pubDate>
            <atom:updated>2020-08-15T22:54:19.342Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*oB3NPinWUhX3b3Pu" /><figcaption>Photo by <a href="https://unsplash.com/@markusspiske?utm_source=medium&amp;utm_medium=referral">Markus Spiske</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>In the rapidly changing digital world, the emergence of new technology paradigms such as IoT, blockchain and social network among others has drastically transformed the mechanism of data management. NoSQL database technology has evolved as one of the most viable options for managing data in such business use cases.</p><p>While there are numerous non-cloud open source and commercial NoSQL database solutions available in the market, the major cloud providers have also come up with various cloud native NoSQL offerings. As organizations continue to move their web applications and services to cloud, it becomes a natural choice for them to move the underlying NoSQL databases to cloud and get an end to end managed service experience. This article aims to provide a brief overview of different NoSQL database types and highlight the various NoSQL database options available on Google Cloud, AWS and Azure.</p><h3>What is a NoSQL database?</h3><p>NoSQL databases are designed to handle large volumes of rapidly changing data with no fixed schema. They offer the flexibility of storing data in schema-agnostic data models that can change dynamically and can store a wide variety of data including semi-structured and unstructured data.</p><p>In contrast to relational databases which are designed for high consistency, NoSQL databases are optimized for modern applications that require high performance and low latency. They offer a variety of data models and a number of data access patterns, and are suitable for high throughput, low latency use cases. NoSQL serves as the best platform for fuzzy and complex data structures that benefit from BASE modeling rather than ACID modeling. Although some NoSQL databases offer some SQL like querying and basic ACID transaction capabilities, they are generally not designed to handle complex querying and multi-row transactions.</p><h3>Types of NoSQL databases</h3><p>NoSQL databases can be broadly categorized into four categories: column store, key value, document store and graph database.</p><p>Column Store: Column Store NoSQL databases organize and store data in columns. They appear similar to traditional relational databases as they present data in the form of tables, rows and columns. However, they organize data differently by storing data in columns and column families in flexible table structures. They are designed to eliminate sparse data problems and enable faster column operations. Typical use cases for column store NoSQL databases include IoT, time series, and real time data processing. The popular column store databases outside the cloud eco systems are HBase and Cassandra.</p><p>Key Value: Key Value NoSQL databases follow a simple approach of storing data in key-value pairs in which keys work as unique identifiers of data. They are ideal when the schema is relatively simple and there is need for high speed read/write operations and high scalability. Typical use cases include IoT, blockchain, session store and shopping cart type operations. Examples of major non-cloud key value solutions are Redis and Riak. The column store solutions HBase and Cassandra can also be provisioned to be used as key value databases.</p><p>Document Store: Document Store NoSQL databases are designed to store and manage document-oriented data usually in the form of JSON like documents. Typical use cases include product catalogs, user profile information and content management systems. MongoDB and Couchbase are two key players in this field outside the cloud eco-systems.</p><p>Graph Database: Graph databases are designed around data entities and the relationships among them. Entities are represented by vertices and relationships are represented by edges. All vertices as well as edges have attributes associated with them. Graph databases are ideal for social network, fraud analysis and real time recommendation use cases. Examples include Janusgraph and Neo4Js.</p><h3>NoSQL offerings on GCP, AWS and Azure</h3><p>The table below provides an overview of NoSQL database options in the 3 major cloud service platforms. If your organization is currently relying on an on-prem NoSQL database and you wish to migrate to cloud, then this table can serve as a reference to help you choose the correct solution based on the type of your NoSQL database and the cloud platform of your choice.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/961/0*so_W97BS_KZZJFAZ" /></figure><p>GCP:</p><p>In the column store category, GCP offers Cloud Bigtable which is a massively scalable fully managed NoSQL database. Bigtable is ideal for large analytical and operational workloads. Although, it was initially designed to handle terabyte-scale and petabyte-scale workloads, recently Google has expanded Bigtable’s support to handle smaller workloads. Cloud Bigtable HBase client for Java, which is a customized version of Apache HBase client for Java, lets you write java applications that can communicate with Bigtable through the open source HBase API.</p><p>Bigtable can also be used as a key value NoSQL database. In the key value category, the other two options on GCP are Memorystore for Redis and Memorystore for Memcached (Memorystore for Memcached is in beta stage now). Memorystore is Google Cloud’s fully managed in-memory data store service and is compatible with Redis and Memcached. Memorystore for Redis and Memorystore for Memcached are ideal for use cases that require fast, real-time processing of data.</p><p>For document store use cases, Google Cloud offers Cloud Datastore and Cloud Firestore. Cloud Firestore is the next generation of Cloud Datastore and it combines the best of both Google Cloud and the Firebase platform (Firebase is the mobile and web application development platform developed by Firebase Inc., a company that has been acquired by Google). Cloud Firestore is designed to support web and mobile development via its native SDKs. It also supports real time syncing of data.</p><p>Google Cloud offers no native Graph NoSQL database solution at present. However, it supports Janusgraph and Neo4j, two major open source graph database solutions, to be hosted on Kubernetes. For Janusgraph implementation on Kubernetes, Bigtable can be used as the storage backend.</p><p>AWS:</p><p>Amazon does not offer any cloud native column store NoSQL solution. If you want to use HBase on AWS, you can leverage Amazon EMR to provision HBase clusters. (Using the same approach, HBase can be provisioned with Dataproc and HDInsight on GCP and Azure respectively)</p><p>Amazon DynamoDB is Amazon’s cloud native key value NoSQL offering. In addition, like GCP’s Memorystore, AWS also offers an in-memory data store service called Elasticache which is compatible with Redis and Memcached.</p><p>Amazon DocumentDB and Amazon Neptune are Amazon’s cloud native offerings in the document database and Graph database NoSQL categories respectively.</p><p>Azure:</p><p>Azure Cosmos DB is Azure’s multi model NoSQL database offering that can be used to model and access data in any of the 4 approaches, i.e., columnar, key value, document, and graph data models. Cosmos DB offers various APIs such as SQL, Cassandra, MongoDB, Gremlin, and Table APIs. The data in Cosmos DB database can be modeled and accessed in any of the 4 formats by using the appropriate API. You can use Cassandra API for columnar, SQL and MongoDB API for document, Table API for key value, and Gremlin API for graph data modeling.</p><p>Azure also offers a separate key value NoSQL solution called Azure Table Storage. While Cosmos DB is designed for global distribution, high throughput and low latency, Azure Table Storage is designed for high scalability in a single region.</p><p>Like GCP and AWS, Azure also offers an in-memory data store service called Azure Cache. Azure Cache for Redis is a fully managed in-memory data store solution based on Redis and is suitable for fast and real-time data processing.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=176200407aa0" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Cloud analytics at scale with cloud data warehouse: A look at the top players]]></title>
            <link>https://medium.com/@rinkesh.pati/cloud-analytics-at-scale-with-cloud-data-warehouse-a-look-at-the-top-players-cd5590a96a38?source=rss-437d3986d319------2</link>
            <guid isPermaLink="false">https://medium.com/p/cd5590a96a38</guid>
            <category><![CDATA[big-data-analytics]]></category>
            <category><![CDATA[cloud]]></category>
            <category><![CDATA[big-data]]></category>
            <category><![CDATA[analytics]]></category>
            <category><![CDATA[data-warehouse]]></category>
            <dc:creator><![CDATA[Rinkesh Pati]]></dc:creator>
            <pubDate>Fri, 22 May 2020 04:23:53 GMT</pubDate>
            <atom:updated>2020-08-15T22:45:50.437Z</atom:updated>
            <content:encoded><![CDATA[<h3>Cloud Data Warehouse: Snowflake vs BigQuery vs Synapse vs Redshift</h3><h4>Comparing the top players in the cloud data warehouse space</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Mc_H00R8MkmwW13I" /><figcaption>Photo by <a href="https://unsplash.com/@fabioha?utm_source=medium&amp;utm_medium=referral">fabio</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>It is needless to mention the importance of Big Data and Analytics in the modern time when the rate of accumulation of data continues to increase exponentially every year. The volume of data being generated in various business domains has been exploding like never before, and as a result, the IT industry has begun to witness continuous innovations in the effort to effectively manage the growing data challenges.</p><p>As it becomes increasingly more important to deliver real time and accurate analytics on data, businesses tend to focus more on storing and managing data on a robust platform that can effectively accommodate the growing data volumes and stand heavy analytics workloads. Cloud Data Warehousing, a technology that was in its nascent stage only a few years ago, has now become the dominant technology of choice for many organizations that are looking for ways to better manage their data and derive insights from it. A cloud data warehouse is basically a cloud based centralized repository of data that houses data coming from various sources such as operational and transactional databases, business applications, system logs and other internal and external data sources. It is based on MPP architecture that stores data in a distributed fashion and performs parallel processing across multiple servers to gain higher efficiency.</p><p>In this article, we will discuss about Amazon Redshift, Azure Synapse and Google BigQuery, the cloud native data warehouse solutions offered by the 3 major cloud players — Amazon AWS, Microsoft Azure and Google Cloud Platform respectively. We will also discuss about a fourth cloud native data warehouse solution, i.e., Snowflake, which is designed solely for the cloud and can be hosted on any of the above 3 cloud platforms. All these solutions are fully managed services that take the burden of infrastructure management off the customers’ shoulders and let them focus on the data. While each of these solutions is designed to handle massive volume of data and support heavy workloads for analytical processing, there are some differences and similarities in their approaches.</p><p><strong>Architecture and Compute Model</strong></p><ul><li>There are two types of compute models when it comes to querying and accessing data on a cloud data warehouse — Resource Provisioning and Serverless. In the first model, you provision a cluster of nodes based on your compute requirements. You pay for the resources you provision for yourself and you can choose to scale out or scale in as your compute requirement changes over time. On the other hand, in serverless model, you push the operational responsibilities to the cloud provider and the cloud provider takes care of the provisioning of resources under the hood. You pay only for the amount of data processed by your queries.</li><li>Amazon Redshift and Azure Synapse take the first approach where you need to provision a cluster of nodes. A cluster is the core infrastructure element in both these data warehouses. A cluster consists of a number of compute nodes and a leader node (Redshift)/control node (Synapse). Client applications communicate with the leader/control node. The leader/control node runs the MPP engine and passes the queries to the compute nodes for parallel processing. Although Redshift and Synapse have the same approach for compute, they follow different approaches for storage. It is described in detail in the Storage section below.</li><li>Snowflake also takes a similar approach and follows the MPP architecture. The compute unit is called a ‘virtual warehouse’. Each warehouse is composed of one or more compute clusters. The virtual warehouses are independent of each other and each warehouse has its own compute resources.</li><li>Google BigQuery, on the other hand, follows the serverless approach and is offered as DWaaS — data warehouse as a service. You do not have to provision any cluster or any other resources. BigQuery service manages the underlying infrastructure including scalability and high-availability.</li><li>Azure Synapse also recently introduced a serverless offering which is currently in preview. Although Amazon Redshift is not a serverless solution, it provides a feature called Redshift Spectrum which is completely serverless and enables its users to directly query data stored in S3.</li></ul><p><strong>Separation and Independent Scaling of Storage and Compute</strong></p><ul><li>One of the key benefits of a cloud data warehouse is the separation of storage and compute. It is important because storage cost in cloud is much lower than compute cost. The ability to scale storage and compute separately based on one’s need will be more economical than a model where storage and compute are coupled together and require simultaneous scaling.</li><li>As part of their architecture, BigQuery, Synapse and Snowflake separate storage from compute and offer the ability to scale them independently. They come with managed columnar storage solutions for storing data outside the cluster. Since storage is external to the compute environment, it offers the user the ability to scale them independently.</li><li>Redshift, on the other hand, did not separate storage and compute as part of its basic architecture. It stores data locally in the compute nodes in columnar storage. As storage used to be always tightly coupled with compute, it did not offer the ability to scale them independently. Such design by Redshift was basically keeping in mind the opportunity to utilize the throughput of local disks for faster data retrieval and higher processing speed. However, the inability to independently scale storage and compute had been a big lacking feature in Redshift and there had been criticism and customer demands around it. Very recently, Redshift introduced a new type of nodes called RA3 nodes that separate storage from compute and let you scale them independently.</li></ul><p><strong>Automatic Elastic Resizing</strong></p><ul><li>Another key feature of a cloud native data warehouse is elasticity — the ability to avail additional compute resources when your workload increases and downsize the compute power when the workload subsides. All of the concerned solutions (except BigQuery — it is serverless and it manages scalability under the hood) provide you the ability to scale the compute power up or down according to your compute requirements.</li><li>Redshift offers the ability to elastically resize a cluster. If you know what time of day/week to expect heavier workloads and when to expect lighter workloads, you can schedule the cluster resizing operation accordingly. By using the AWS Management Console or programmatically by AWS CLI, you can perform the scheduling so that the cluster can expand/shrink at specific times of day/week.</li><li>Similarly, Synapse also supports elastic scalability and offers the ability to scale out/in the compute power to accommodate changing workloads. The process can be automated with the help of Azure Functions so that the scaling operation can automatically take place at specific times of day/week.</li><li>It would be ideal if the data warehouse could automatically detect changing workloads and add/remove compute resources accordingly. This is called automatic elastic scalability. Snowflake provides this capability of dynamic auto-scaling without any interruption. It automatically detects increase in workload and adds compute resources and shuts down the resources as the workload subsides.</li></ul><p><strong>Ability to Pause and Resume Automatically</strong></p><ul><li>In case of Redshift, Synapse and Snowflake, which require cluster provisioning, you keep getting charged whenever a cluster is up and running. Depending on your compute requirements, you may or may not want the cluster to be running 24X7. So, the ability to bring up the cluster only when you need to execute queries and bring it down for the remaining time will be an ideal scenario to optimize the compute cost.</li><li>Snowflake offers out-of-the-box auto suspension and auto resume features. The virtual warehouses can be set to automatically shut down when there is no query activity for a specified period of time and can be set to resume automatically as soon as new queries are submitted.</li><li>Synapse also offers the ability to pause and resume clusters and this can be automated with the use of Azure Functions. It requires some coding to automate the process. This automation can ensure the cluster pauses or resumes at specific times as specified by the user. However, unlike in the case of Snowflake, the auto pause process in Synapse is not going to be a dynamic process.</li><li>Redshift does not offer the option to pause and resume a cluster. However, the workaround for it is to delete the cluster after taking a snapshot and then restore the cluster from the snapshot. This process will require some coding for it to get automated and the user will need to specify timings for the pause and resume activities. Again, it is not going to be a dynamic process.</li><li>In the case of BigQuery, there is no need to pause any cluster as it is a serverless solution. You incur compute cost only when you execute a query and there is no compute cost incurred when no query is being executed.</li></ul><p>The above is a high-level analysis of how the 4 solutions are designed to operate in few of the key operational areas. It is in no way a detailed assessment of one’s suitability over the other. With the passage of time, all these vendors keep adding new features to their products. The above details are current as of today but may become obsolete in the coming months as the vendors continue to innovate and redesign their products. The solutions are continuously competing with each other to ensure better speed and query performance and the ability to speedily process higher volume of data.</p><p>Besides, these solutions are not designed to be simply a data storage platform. All 4 solutions seamlessly integrate with BI, data science and machine learning platforms to facilitate advanced analytics. In addition, Google BigQuery offers native machine learning capabilities via BigQuery ML that enables users to create and execute machine learning models within BigQuery using standard SQL queries. Synapse offers a native PREDICT statement using which users can score machine learning models trained in Azure Machine Learning, Apache Spark or other frameworks, and derive insights within Synapse without having to perform massive data movement. All 4 solutions support direct streaming ingestion and facilitate users to perform real time analytics on streaming data.</p><p>With the simplicity of managing both structured and semi-structured data of diverse data types, near limitless scalability, faster performance, easy integration with analytics tools, and high security and regulatory compliance standards, cloud data warehouse technology has started to gradually replace on-premise data warehouses and Hadoop based platforms.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=cd5590a96a38" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>