<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Dr. Rabi Prasad Padhy on Medium]]></title>
        <description><![CDATA[Stories by Dr. Rabi Prasad Padhy on Medium]]></description>
        <link>https://medium.com/@rabi.padhy?source=rss-74a38b601f29------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*XXOGOygsyETwNScALws8aA.jpeg</url>
            <title>Stories by Dr. Rabi Prasad Padhy on Medium</title>
            <link>https://medium.com/@rabi.padhy?source=rss-74a38b601f29------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Wed, 18 Nov 2026 00:16:36 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@rabi.padhy/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[AWS Greengrass Machine Learning Inference: Beyond Edge Computing]]></title>
            <link>https://medium.com/@rabi.padhy/aws-greengrass-machine-learning-inference-beyond-edge-computing-56b5ff3125be?source=rss-74a38b601f29------2</link>
            <guid isPermaLink="false">https://medium.com/p/56b5ff3125be</guid>
            <category><![CDATA[aws-iot]]></category>
            <category><![CDATA[internet-of-things]]></category>
            <dc:creator><![CDATA[Dr. Rabi Prasad Padhy]]></dc:creator>
            <pubDate>Thu, 11 Oct 2018 04:48:41 GMT</pubDate>
            <atom:updated>2018-10-11T04:48:41.853Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*w7iIY4mLADoM9A4Q" /></figure><p>Edge computing has become critical for IoT Solutions because of the proliferation of devices and the need for real-time communications. As per various forecast reports, IoT devices are expected to grow from 5.8 billion in 2016 to 13.7 billion by 2021. IoT devices have limited processing power and memory, and they often rely on the cloud for processing and data storage. But transmitting data to the cloud increases latency and requires always-on connectivity. When that’s not available, IoT devices need to perform more tasks at the network edge.</p><h3>IoT Edge Computing current players in market:</h3><p>AWS IoT Greengrass: Its Amazon’s edge offering and currently its the leading IoT Edge computing software in the market.</p><p>Azure IoT Edge: It’s Microsoft’s IoT edge Computing offering and its free to use but it requires MS IoT Hub.</p><p>AWS Greengrass edge computing service which is part of AWS IoT ecosystem is the solution to this problem. AWS Greengrass can be programmed to filter device data and only transmit necessary information back to the cloud i.e a way to reduce traffic in the cloud is to run AWS Greengrass at the edge. Greengrass can intelligently process and filter data locally, eliminating the need to send all device data upstream. Recently AWS added a new feature onto the Greengrass called as “Greengrass machine learning (ML) Inference” which allows machine learning models to be deployed directly to devices. Greengrass ML Inference took the route of providing many pre-built ML frameworks (TensorFlow, MXNet, Chainer, Caffe2, and even Microsoft Cognitive Toolkit). These can make decisions quickly, even when the devices are not connected to the cloud. AWS Greengrass seamlessly extends AWS to devices so they can act locally on the data they generate, while still using the cloud for management, analytics, and durable storage.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*EEnCPaeruO2fUi9n" /></figure><h3>What is Greengrass ML Inference and how it works:</h3><p>AWS Greengrass ML Inference is a feature of AWS Greengrass that makes it easy to perform machine learning inference locally on Greengrass devices using models that are built and trained in the cloud. This feature is available for AWS Greengrass Core v1.5.0 and later.We can train our inference models anywhere and deploy them locally as machine learning resources in a Greengrass group then access them from Greengrass Lambda functions. Let us assume we can build and train deep-learning models in Amazon SageMaker and deploy them to our Greengrass core. Then your Lambda functions can use the local models to perform inference on connected devices and send new training data back to the cloud.</p><p>The below diagram shows the AWS Greengrass ML inference workflow.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/0*PXRwchtKELLvorDD" /></figure><p>Image Source: AWS</p><h3>How AWS ML(Machine Learning) Inference benefits Businesses</h3><p>Increased Operational Reliability: We can locally store, process and derive intelligence from data at the edge , we can build robust IoT Solutions on-premise without warrying about intermittent cloud connectivity.</p><p>Faster Real-Time Predictions: By ruining on-device ML models provides significantly critical IoT applications while ensuring data privacy and confidentiality.</p><p>Increased Security for devices and Data: AWS IoT Edge can process and analyze different types of files locally on edge devices instead of sending the raw data to the cloud and then wait for a response. As this is processing is local to the devices so it addresses specific compliance needs and reduces data privacy risk.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=56b5ff3125be" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Enterprise Data Lake: State of the Art]]></title>
            <link>https://medium.com/@rabi.padhy/enterprise-data-lake-state-of-the-art-d998d7cb50c8?source=rss-74a38b601f29------2</link>
            <guid isPermaLink="false">https://medium.com/p/d998d7cb50c8</guid>
            <category><![CDATA[data-lake]]></category>
            <category><![CDATA[big-data]]></category>
            <dc:creator><![CDATA[Dr. Rabi Prasad Padhy]]></dc:creator>
            <pubDate>Thu, 11 Oct 2018 04:26:07 GMT</pubDate>
            <atom:updated>2018-10-11T04:26:07.255Z</atom:updated>
            <content:encoded><![CDATA[<p>We can define the Enterprise Data Lake is a repository that holds raw data file extracts of all the Enterprise source systems. Its is a new type of cloud-based enterprise architecture. The idea is to have a single store for all of the raw data that anyone in an organization might need to analyze. It develops a Data ecosystem that Stores and Governs the information needs of various Business Functions within our enterprise. We can store all types of data (structured, semi-structured, unstructured) in the Data Lake. The Data Lake act as a single and cheap repository of Enterprise-wide data. It Abstracts our DW / BI Solutions from source systems. It Supports potential Big Data Processing and Analytics.</p><p>I will correlate with the concept of Data-warehouse and Data Lake using an real life example.We definitely visited a Lake near by our towns (Example Chilika Lake — Odisha, largest freshwater lake ) where we will found different activities executing by different people like picnic spot, fishing, Boat Ride, Temple visit and even the lake supplies the purest form of water. So the message we are getting here is the same lake used for multiple purposes by multiple people. We have also used packaged drinking water which is also the purest form of water which we can use for limited use i.e just Drinking purpose. So In real time scenario we can use the Data Lake for Machine-Learning and Analytics purpose where the Data Warehouse more business process focused. The Enterprise Data Lake stores data in the purest form, caters to multiple stakeholders, customers, business units and can also be used to package data in a form that can be consumed by end-users.</p><h3>Capabilities of Data Lake:</h3><p>As we know databases, data warehouses and Data lakes are all designed to store data. A database (SQL/NoSQL/NewSQL) is an organized collection of data and they usually have only the most recent data available. A data mart is used by individual departments or groups A data warehouse is a storage repository that stores data that are extracted, transformed and loaded into the files and folders. A data warehouse only stores structured data from one or more disparate sources that are processed later for the business users. Data extracted from a data warehouse helps the users to make business decisions. Data warehouse is a model to support the flow of data from operational systems to decision systems. Although databases and data warehouses can handle unstructured data but they don’t do so in the most efficient manner but a data lake can do it. A data lake can store vast amounts of raw data in its native format (CSV, logs, XML, JSON, Document, PDFs, Mails, Tables, Images, Audio and Video )</p><p>Businesses across the world use big data to gain valuable insights that can help them make informed business decisions, correctly comprehend the current market trends, and understand the expectations of the customers to gain an edge over their competitors. The main capabilities of a Cloud based Data Lake are scalability and elasticity. Data Lakes capture and store raw data at scale for a low cost. It stores many types of data in the same repository, perform transformations on the data and also defines the structure of the data at the time, it is used, referred to as schema. It can easily keep in track with the data growth of the organization provided the processing and governance. Cloud Data lake analytics eliminates the need for deploying, configuring, and performance tuning.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/802/0*YOQ_scTsjElQsKbG" /></figure><h3>Architecture Components of a Data Lake</h3><p>The Data Lake is an effective data management solution for advanced analytics experts, business users and the senior leadership team. Architecting a successful Data Lake requirs a comprehensive understanding of the technicalx and the business requireemnts to meet organization’s specific needs.</p><p>The Core Layers of a Enterprise Data Lake:</p><p>Data Sources: There are many types of data sources like Databases, Documents, Emails, IoT Devices, Social Media, Mainframe systems etc.</p><p>Data Ingestion: This step generally getting data into the data lake from various sources regardless of the data types. We can leverage ETL tools, custom programs to load. The data could be loaded into the data lake in batches or in real-time.</p><p>Data lakes usually align with an “ELT” strategy which means we can Extract and Load into the data lake in its original format, then Transform later. An “ELT” strategy goes along with the “Schema on Read” technique that is prevalent in the big data world. Traditional data warehousing typically follows at “ETL” strategy in which the Transformation occurs before the Load to the data warehouse — this is referred to a “Schema on Write” because the schema(structure of the data) must be defined before the data is loaded to the data warehouse.</p><p>Data Processing: This Layer runs analytical algorithms and users queries with varying real time, interactive, batch to generate structured data for easier analysis. Before we can load data into a data warehouse, we first need to give it some shape and structure i.e., we need to model it. That’s called schema-on-write. With a data lake, you just load in the raw data, as-is, and then when you’re ready to use the data, that’s when you give it shape and structure. That’s called schema-on-read.</p><p>Data Storage: Storage is where the data lake lives. Usually as files in a cloud bucket such as AWS S3. Obviously the cheaper and easier to access we can make this component the better.</p><p>Data Visualization or Utilization: Data Visualization is consists of the tools that will drill down into the data and getting the Insight of it. Some of the top tools are Microsoft Power BI, Tableau etc. We can also leverage SQL, NoSQL, NewSQL queries, or even excel could be used for data analysis.</p><p>Security &amp; Monitoring: Security needs to be implemented in every layer of the Data lake. It starts with Storage, Unearthing, and Consumption. The basic security requirement is to stop access for unauthorized users. It should support different tools to access data with easy to navigate GUI and Dashboards. Some of the important security features of Data Lake includes Authentication, Accounting, Authorization, Auditing and Data Protection.</p><h3>Data Lake: Data flow between different Layers:</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*BZW7v0rftKxuaT_p" /></figure><h3>Enterprise Data Lake Advantages</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*O_V2ofU7L8mpSHtH" /></figure><p>Public Cloud Service Selection:</p><p>Data Lake Services — ETL, Processing, Storage, Analytics:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/815/0*Ua36w48eYubB-LOS" /></figure><h3>Use Cases:</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/783/0*T00vFIqossLoWjaW" /></figure><h3>Data Lake Technologies &amp; Tools:</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Nzk8wRr707oouxoG" /></figure><h3>Data Warehouse and Data Lake Development Life-cycle</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*lkhsPrgjKaB4fNC9" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d998d7cb50c8" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Alibaba Anti-DDoS: The best Hybrid Cloud-based DDoS Protection Service]]></title>
            <link>https://medium.com/@rabi.padhy/alibaba-anti-ddos-the-best-hybrid-cloud-based-ddos-protection-service-9fc926b0bed0?source=rss-74a38b601f29------2</link>
            <guid isPermaLink="false">https://medium.com/p/9fc926b0bed0</guid>
            <category><![CDATA[cybersecurity]]></category>
            <category><![CDATA[ddos-protection]]></category>
            <dc:creator><![CDATA[Dr. Rabi Prasad Padhy]]></dc:creator>
            <pubDate>Thu, 11 Oct 2018 04:23:58 GMT</pubDate>
            <atom:updated>2018-10-11T04:23:58.251Z</atom:updated>
            <content:encoded><![CDATA[<p>Currently DDoS (Distributed Denial of Service ) attacks are increasing with a higher level of complexity and frequency. These attacks are targeting important resources ( network bandwidth, Web Server Sockets, Web server threads , CPU Utilization and any other compute resources ) &amp; specific markets ( Financial Sector, Government sector websites, Top E-Commerce websites, Universities, Internet Service Providers and any other sectors ) which leads to service downtime and severe financial loss.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/899/0*mvfT3d2IIxppEqsP" /></figure><p>Actually DDoS means a lot of computers from different places around the world send a large number of connection requests to a server and the target server also needs to process all of those requests simultaneously then the server starts running very slow, some cases even becomes unreachable. If we can simulate the DDoS attack to a real life equivalent scenario then it like getting a crowd of so many non-customers in front of a shop that customers can’t even get into the shop’s front door which preventing the shop from doing any business. In the Seven OSI Layers the common DDoS attacks include Network layer/connection layer/session layer and application layer attracts.</p><p>Challenges of DDoS Attacks in Public Cloud</p><p>There are specific features of cloud networking makes it easier for DDoS attacker to cause maximum harm to their targets namely Elasticity nature of cloud services, Compute Service Billing Model and Cloud enabled IoT Devices.</p><p>More Common Types of DDoS Attracts in Public Cloud</p><p>(1) Rate-Based Protocol DDoS Attacks: UDP-based DDoS, Small-packet, high-rate TCP DDOS</p><p>(2) Reflection-Based Attacks</p><p>(3) High-Volume Application Attacks</p><p>(4) Exploiting Application Behavior — Low and Slow Attacks</p><p>Alibaba Anti-DDoS is a cloud-based anti-DDoS Security as a Service (SECaaS) solution and currently its the №1 cloud-based Security as a Service in current public market place. The Anti-DDoS traffic cleaning service is a network security service that defends IP addresses against distributed denial of service (DDoS) attacks. Anti-DDoS monitors traffic directed to specific IP addresses in real time and detects access traffic at network egress points to identify any DDoS attack as soon as possible. It then cleans abnormal traffic according to user-configured defense policies without affecting normal services. In addition, monitoring reports are generated, clearly presenting users with a security view of network traffic. Anti-DDoS service supports traffic cleaning only for the EIP bound with ECS and ELB services.</p><p>Alibaba Cloud Anti-DDoS is available in two tiers: A Basic Version and A Pro Version.</p><p>Alibaba Cloud “Anti-DDoS Pro” designed to defend against all types of DDoS attacks like HTTP flood, WAF protection services and is capable to defend against three to seven layers of DDoS attacks such as SYN flood, UDP flood, ACK flood, ICMP flood, DNS query flood, NTP reply flood, HTTP flood attack, and Web application attacks.</p><p>Alibaba Cloud Anti-DDoS Basic is a cloud-based security service that integrates with Alibaba Cloud ECS instances to safeguard customers data and applications from DDoS attacks. This Basic Service is available to all Alibaba Cloud users free of charge, enabled by default and protect up to 5 Gbps common DDoS attacks. If the data transfer rate exceeds the default black hole threshold then Alibaba cloud will block external network access to the ECS instance. We can also upgrade to Anti-DDoS Pro Plan for additional protection service and its a paid plan that can provide a mitigation capacity of more than 2000 Gbps.</p><h3>Anti-DDoS Pro Service Architecture</h3><p>Once purchasing the Anti-DDoS Pro service then we must resolve our domain name to the Anti-DDoS Pro IP and set forwarding rules to Anti-DDoS Pro. For web service, the domain name needs to be directed to the Anti-DDoS IP and for non-web service, the service IP needs to be replaced with the Anti-DDoS Pro IP. This configuration directs all public network traffic to the Anti-DDoS server room. The user access traffic is forwarded to the origin site IP by using port protocol forwarding. Meanwhile the malicious attack traffic is cleaned and filtered through the Anti-DDoS Pro service then the normal traffic is returned to the origin site IP. This entire process ascertains the stable access to the origin site IP and protects services. The Anti-DDoS Pro version has a wider range of options for attack protection that will guarantee greater security for your web application. It also protects the application and server from the most common attacks, along with the most complex ones. With the Pro version, the mitigation capacity goes from 5 Gbps, which is the capacity defended by Basic, to 2 Tbps.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/876/0*RJ3CEO0x6vz5VDn_" /></figure><p>Image Source: alibabacloud.com</p><p>Use Cases: Anti-DDoS Pro is recommended for large enterprise application web sites and e-commerce websites. The Smaller websites, such as blogs, the Anti-DDoS Basic should be sufficient.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9fc926b0bed0" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[AWS Lambda: A ‘WhatsApp’ Like Movement in Serverless Computing]]></title>
            <link>https://medium.com/@rabi.padhy/aws-lambda-a-whatsapp-like-movement-in-serverless-computing-d227a3ba1d9d?source=rss-74a38b601f29------2</link>
            <guid isPermaLink="false">https://medium.com/p/d227a3ba1d9d</guid>
            <category><![CDATA[ec2]]></category>
            <category><![CDATA[serverless]]></category>
            <category><![CDATA[aws-lambda]]></category>
            <dc:creator><![CDATA[Dr. Rabi Prasad Padhy]]></dc:creator>
            <pubDate>Thu, 11 Oct 2018 04:21:28 GMT</pubDate>
            <atom:updated>2018-10-11T04:21:28.022Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*GunIvVBXIBU6LmJ6" /></figure><p>As we are witnessing a paradigm Shift in IT Architecture Evolution i.e Software Developement and the way of IT Infrastructure Provisioning/Deprovisioning using Infrastructure as Code. We also know the pain of on-boarding a a physical servers and how easy it is now by using of Container technology and Cloud Infrastructure/Platform/Software services (IaaS, PaaS, SaaS). From traditional waterfall model to current agile DevOps model to future NoOps Model gives an opportunity to explore next generation application architecture. So from physical server to Virtual Machine then Container and now Serverless computing acting as a game changer agent for Cloud Computing Space. In this post I have given a big picture view of AWS Lambda Service.</p><p>Serverless computing is a hot topic in the Cloud Computing architecture. The “Big Five” Public cloud vendors namely Amazon, Microsoft, Google, IBM, Alibaba are heavily invested in Serverless. We have also watched various live webinar in YouTube, read plenty of eBooks, GitHub projects, conferences, blogs and vendor documentation dedicated to the subject.</p><p>Serverless architectures consist of two different concepts : Function as a Service (FaaS), Backend as a Service (BaaS).</p><p>This two service models “FaaS &amp; BaaS” coming under Platform as a Service (PaaS).</p><p>AWS Lambda is a FAAS (Function as a service), it lets developers run code without provisioning or managing servers. The Lambda Service is designed to serve “microservices” ( example lightweight web server frameworks like Node.js ) instead of resource intensive frameworks (ie: Apache, Websphere, or .NET). Since the release of AWS Lambda, “Microservice Applications” became a buzzword and starting point or new learning opportunity for developers to be part of the cloud journey as Serverless computing is a cloud-computing execution model.</p><p>Currently Microservice architecture widely spreading in Large Enterprises and are now one of the the driving wave of innovation. Many companies are already using it, some are very actively doing the implementation and some are trying to use it. If we can focus on latest trend on Industry adopting cloud services list then “FaaS” services are drastically increasing i.e IT industry witnessing a WhatsApp like movement in the area of adopting Serverless architecture ( Example: Developing Microservice Applications in Public cloud ).</p><p>If we compare Lambda with Docker containers then both have a place in a modern digital enterprise. Both services used to build microservices but serve different needs. If we want to eliminate application management and don’t care about the architecture then Lambda(Serverless) is the best option. If we want to deploy an application on specified system architecture with having control over it, then Docker(Containers) will be the best option.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*DrMVGiVwPvuRVxPb" /></figure><h3>Monolithic vs SOA vs. Microservice Application Design Approach</h3><p>As we know there are multiple issues can developer face while developing &amp; deploying a monolithic application like storing all application components in one archive (EAR or WAR ) the size of the file will keep growing when application gets enhanced. Every new release there are new functionalists and features are added, so the code-base keep grows. Initially we can neglect but eventually it hampers developer teams productivity as IDE cannot handle large amount of code efficiently. For small change in one module of the application demands deployment of entire application. The below figure is the high level design of a monolithic application.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*A45f5jtcxHnSdJAv" /></figure><p>The Service Oriented Architecture (SOA) is an architectural pattern that guides business solutions to create, organize and reuse its computing components. Adopting SOA will help developers for separate of business process from application and the latest flavor of application development i.e Microservices applications using Serverless computing (microservice architecture) can solve all the problems related to Monolithic &amp; SOA framework i.e application functionalists are broken down into small services which talks to each other using HTTP/REST (synchronous or asynchronous). For example, an application might consist of services such as the invoice management service, the user administration service etc. These services can be developed independently and we can deploy them independently with one to one or one to many mapping between services and database. This will keep things smooth when we touch one module and be relaxed because our other module will still be operational. The below figure is the high level design of a microservice application design.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*SpipWfhGU2AwkzB8" /></figure><h3>Other options in AWS cloud if we do not want to use Lambda Service</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*LzCrZHQb3wpyAkWv" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/993/0*zApnILBy79JCuef9" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*lNbHgLljsVGBtMmR" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d227a3ba1d9d" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Google Spanner:A NewSQL Journey or Beginning of the End of the NoSQL Era]]></title>
            <link>https://medium.com/rabiprasadpadhy/google-spanner-a-newsql-journey-or-beginning-of-the-end-of-the-nosql-era-3785be8e5c38?source=rss-74a38b601f29------2</link>
            <guid isPermaLink="false">https://medium.com/p/3785be8e5c38</guid>
            <category><![CDATA[big-data]]></category>
            <category><![CDATA[newsql]]></category>
            <dc:creator><![CDATA[Dr. Rabi Prasad Padhy]]></dc:creator>
            <pubDate>Thu, 11 Oct 2018 03:56:52 GMT</pubDate>
            <atom:updated>2018-10-11T03:56:52.311Z</atom:updated>
            <content:encoded><![CDATA[<p>In this era of Digital Transformation where a growing numbers of companies move their application data to the Public cloud, NewSQL has become a center of attention due to its key offers like Horizontal scalability, SQL Support and a great way for users to access the data using APIs. Large Enterprises thinking about their long-term architecture should consider the implications on the database layer and the potential for a more distributed and flexible database architecture, which will make NewSQL vendors worth exploring. Cloud enabled new applications require new databases and the NewSQL db engine designed to support these applications. Most of the Customers are moving away from their old data warehouse platforms to “Data Lake” because they don’t have the flexibility they need with the existing Traditional DWH. So the current innovation on the Database field promising the database landscape continuous to shift further and its just a beginning of end of NoSql or I can say its one more opportunity for IT professionals to be part of this NewSQL Journey.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/752/0*SjaSaU5YPd8u0Mr8" /></figure><p>Cloud Database : A cloud database is a database that typically runs on a Public cloud computing platform but not limited to Public cloud but also compatible for Private and Hybrid Cloud Environment.</p><p>There are two types of cloud database: VM (IaaS) and Database As a Service (DBaaS)</p><p>Database as a Service (DBaaS)</p><p>Database as a Service (DBaaS) is a method of storing and managing data in the Cloud Computing environment. DBaaS service providers are hosting your database using appropriate database infrastructure through PaaS on the cloud. Google Cloud Spanner is DBaaS Service Offering Model ( Part of GCP — Google Cloud Platform ) is the best of RDBMS+NoSQL Database.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/927/0*sWJ7yWd_pDP19Mxb" /></figure><p>There are Three types of database technology: Relational Databases (Structured-SQL) ; Non-Relational Databases (Unstructured-NoSQL); NewSQL. The DBaaS Model shifting most of the Database Admin tasks to Service Operator : Configuration, Scalability, Performance Tuning, Backup, Recovery, Monitoring, Security, Access Control etc.</p><p>A database can be highly scalable and distributed (NoSQL approach), or it can be transactionally consistent (RDBMS approach).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*-sXQv6p3LflIWxzX" /></figure><h3>Key Features of Google Cloud Spanner:</h3><p>Google Cloud Spanner designed to achieve all these features. Google Spanner is a automatic Synchronously-Replicated Globally-Distributed Multi-Version Database and the key features are given below:</p><p>99.999% SLA: Google Cloud Spanner uses Google Infrastructure ( private wide area network ) and TruTime Technology , so the data is highly available.</p><p>Externally Consistent: 2-Phase Commits and Custom Paxos</p><p>Fully Managed: Replication and Maintenance are Automatic</p><p>Easily Scalable: The biggest advantage is Spanner supports horizontal scalability concept and designed for horizontal scalability for read &amp; write mode i.e its only database that supports read &amp; write mode scalability or we can say scales-out both writes and reads without any application changes. ( AWS RDS only supports read mode scalability ).</p><p>ACID Transactions: Google Cloud Spanner supports ACID Transactions.</p><p>SQL Query Support: Google Cloud Spanner supports SQL Language.</p><p>Enterprise Grade Security: Data-layer encryption, IAM integration for access and control.</p><h3>Cloud Spanner — A NewSQL Database for “always-on” Enterprises</h3><p>Google Cloud Spanner is a cloud native, enterprise grade and fully managed relational database service offering high availability, horizontally scalability with consistent global ACID transactions. It is the only enterprise-grade NewSQL , globally-distributed and strongly consistent database service built for the cloud specifically to combine the benefits of relational &amp; NoSQL Database. This combination delivers high-performance transactions and strong consistency across rows, regions, and continents with an industry-leading 99.999% availability SLA, no planned downtime, and enterprise-grade security.</p><h3>Spanner Architecture Overview</h3><p>Cloud Spanner was designed to offer horizontal scalability and a developer-friendly SQL interface. Each deployed Spanner instance called Universe (Spanner Deployment) :</p><p>Basic Architecture elements:</p><p>Replica consistency: Using Paxos protocol; Concurrency Control: Using two phase locking Transaction coordination: 2 phase commit; Timestamps for transactions: Multiple timestamped versions of data items maintained</p><p>The Cloud Spanner deployment having the below components</p><p>Universe Master — Master-process, coordinating the work of many Spanner zones. Universe master displays zone status information.</p><p>Spanner Zones : A zone is the unit of administrative deployment and a cluster in each zone contains multiple span servers. Zones are distributed across Data Centers. Each Zone contains one zone-master and 100 to 1000s of spanservers.</p><p>ZoneMaster (one per zone) : Zonemasters assign data to spanservers which server clients.</p><p>Poxos: Paxos is a consensus protocol. The major issue Cloud Spanner addresses is database replication on a global scale and it provides data consistency even if multiple users conduct transactions which required to connect to data centers across globe. Cloud Spanner eliminates that bottleneck by using the Paxos system of protocols, which relies on an algorithm to create consensus among globally distributed nodes to determine which location, largely based on data gravity is the most authoritative to conduct the commit.</p><p>Spanserver: Spanserver serves data to clients and each spanserver is responsible for 100 to 1000 tablets which hold the key-value mappings. Data is replicated across spanservers ( could be multiple data-centers ) in the unit of tablets. To support replication, each spanserver implements a Paxos state machine on top of each tablet.</p><p>Location proxy : Location proxies help clients locate spanservers</p><p>Placement driver: A process that controls the movement of data between different Zone.</p><p>TrueTime: This is an implementation of synchronized clocks using GPS receivers and atomic clocks in every data center. TrueTime gives Cloud Spanner a crucial piece of information for write activity i.e the system can absolutely determine the serial order of writes, even across distributed nodes. Using this information, consistency issues across nodes in a cluster can be determined and prevented. TrueTime API sync</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/820/0*iV6v55YEW2-ZCzfd" /></figure><p>Spanner supports read-write transactions, read-only transactions and snapshot reads. Standalone writes are implemented as read-write transactions; non-snapshot standalone reads are implemented as read-only transactions. A snapshot read is a read in the past. Concurrency control achieving using two phase locking and Transaction coordination using 2 phase commit. Read-write transactions use 2-phase locking and 2-phase commit. The client first issues reads to the leader of the appropriate group which acquires read locks and reads the most recent data. When a client has completed all reads and buffered all writes then it starts 2-phase commit. Read-write transactions can be assigned commit timestamps by the coordinator leader at any time when all locks have been acquired but before any locks have been released. In Cloud Spanner when data is locked during write operations, spanner only has to lock cells ( a cell is particular column in a particular row) rather then lock the entire row which minimizes contention and accelerates transaction commitment.</p><p>Creating a Cloud Spanner database is very easy using Google Cloud Console and just need to setting the number of nodes required and selecting the regional location.</p><h3>Types of NewSQL Databases</h3><p>There are 3 types of NewSQL and there are no competitors to Google’s Spanner database. Spanner relies on a TrueTime implementation. There are no other known implementations of databases which utilize a time system like TrueTime.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*UD6-y2XE1xLebAbP" /></figure><h3>Comparison between SQL , NoSQL and NewSQL Databases</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/717/0*_GabdyDxIt-fjisE" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3785be8e5c38" width="1" height="1" alt=""><hr><p><a href="https://medium.com/rabiprasadpadhy/google-spanner-a-newsql-journey-or-beginning-of-the-end-of-the-nosql-era-3785be8e5c38">Google Spanner:A NewSQL Journey or Beginning of the End of the NoSQL Era</a> was originally published in <a href="https://medium.com/rabiprasadpadhy">Rabi Prasad Padhy</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>