<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Pradnya Patil on Medium]]></title>
        <description><![CDATA[Stories by Pradnya Patil on Medium]]></description>
        <link>https://medium.com/@pradnyapatil29?source=rss-232555faa69e------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*KVOoDaKQ5PN_WeEoZq5KYg.jpeg</url>
            <title>Stories by Pradnya Patil on Medium</title>
            <link>https://medium.com/@pradnyapatil29?source=rss-232555faa69e------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Wed, 18 Nov 2026 07:47:29 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@pradnyapatil29/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Cloud Application: Architectural Styles — Part 5 ( Big Data )]]></title>
            <link>https://pradnyapatil29.medium.com/cloud-application-architectural-styles-part-5-big-data-d335e554e6a5?source=rss-232555faa69e------2</link>
            <guid isPermaLink="false">https://medium.com/p/d335e554e6a5</guid>
            <category><![CDATA[architecturestyle]]></category>
            <category><![CDATA[software-design]]></category>
            <category><![CDATA[big-data]]></category>
            <category><![CDATA[software-architecture]]></category>
            <category><![CDATA[bigdatanalytics]]></category>
            <dc:creator><![CDATA[Pradnya Patil]]></dc:creator>
            <pubDate>Thu, 13 Oct 2022 01:38:08 GMT</pubDate>
            <atom:updated>2022-10-13T01:38:08.526Z</atom:updated>
            <content:encoded><![CDATA[<h3><strong>Cloud Application: Architectural Styles — Part 5 ( Big Data )</strong></h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/961/1*QhaJAb9GYcS2y84ytJgZvA.jpeg" /></figure><p><strong>Introduction</strong></p><p>In this series of articles, we are going to study the cloud application architectural styles, patterns and practical examples. In Part-5 of the series, we are going to have an overview of the “Big Data” architecture style along with its logical diagram, where to use this style, benefits, and challenges.</p><p>We will study further cloud architectural styles and patterns in subsequent articles of this series.</p><p>You can check my articles on <a href="https://medium.com/@pradnyapatil29/cloud-application-architectural-styles-part-1-bbda76f8ad3f">“N-tier”</a>, <a href="https://medium.com/@pradnyapatil29/cloud-application-architectural-styles-part-2-b67469d524fa">“Microservices”</a> , <a href="https://medium.com/@pradnyapatil29/cloud-application-architectural-styles-part-3-web-queue-worker-956a902a6271">“Web — Queue — Worker”</a> and <a href="https://medium.com/@pradnyapatil29/cloud-application-architectural-styles-part-4-event-driven-396130b014a3">“Event Driven”</a> architectural styles.</p><p><strong>Background</strong></p><p>Big Data architecture deals with ingestion, analysis and processing of the data which is too large to be processed by traditional database systems.</p><p><strong>Logical Diagram:</strong></p><p>Big Data solutions typically process these categories of workload:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/679/1*DHxmj9aSLECbpO8L9J4Q9g.png" /></figure><p>Big Data Architecture includes these components:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/961/1*QhaJAb9GYcS2y84ytJgZvA.jpeg" /></figure><p><strong>Data sources (structured / unstructured):</strong></p><p>Data sources for big data architecture include</p><p>o Real time sources like IoT devices</p><p>o Static files</p><p>o Application data sources such as relational databases</p><p><strong>Data storage:</strong></p><p>o Data lake can be used as a data store which is capable of holding large volumes of different file formats.</p><p>e.g. Blob Containers in Azure Storage / Azure Data Lake Store</p><p><strong>Batch processing:</strong></p><p>o Big data solutions must process large files using long running jobs to filter, aggregate and prepare data for analysis.</p><p>o Usually it involves below steps:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/370/1*N5yqNaK0TMYyasCj-_Ieog.png" /></figure><p>o This can be done using tools such as Pig, Hive, U-SQL jobs in Azure Data Lake Analytics, custom Map-Reduce jobs in an HDInsight Hadoop cluster or using Java, Scala, or Python programs in an HDInsight Spark cluster.</p><p><strong>Real time message ingestion:</strong></p><p>o For real time processing, the architecture should support a way to capture and store real time messages for stream-processing.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/583/1*j-BIXAbkoLppLo3fPEPrFg.jpeg" /></figure><p><strong>Complex event processing (CEP):</strong></p><p>The goal of complex event processing is to identify meaningful events (such as opportunities or threats) in real-time situations and respond to them as quickly as possible.</p><p>The series of events are carefully analysed / readings from an embedded device are aggregated over a time window to generate a notification if the moving average crosses a certain threshold.</p><p>I found this hilarious example on <a href="https://en.wikipedia.org/wiki/Complex_event_processing">wikipedia</a>: :-)</p><p>Among thousands of incoming events, a monitoring system may for instance receive the following three from the same source:</p><p>(1) Church bells ringing.</p><p>(2) The appearance of a man in a tuxedo with a woman in a flowing white gown.</p><p>(3) Rice flying through the air.</p><p>From these events the monitoring system may infer a complex event: a wedding.</p><p>Technologies such as Azure Stream Analytics, Apache Storm etc are useful for Complex Event Processing.</p><p><strong>Event stream processing (ESP):</strong></p><p>This <a href="https://www.bmc.com/blogs/event-stream-processing/">blog</a> explains the event stream processing nicely.</p><p>An <em>event</em> is anything that happens at a clearly defined time and that can be specifically recorded. A <em>stream</em> is a constant and continuous flow of event objects that navigate into and around companies from thousands of connected devices, IoT, and any other sensors.</p><p><em>Processing</em> is the final act of analysing all this data.</p><p><em>So, putting all three of these together, you could say that event stream processing is the process of being able to quickly analyse data streaming from one device to another at an almost instantaneous rate after it is created.</em></p><p>It is meant for continuous monitoring and interactions, detecting any interesting patterns, handling issues etc.</p><p>Platforms like Azure IoT Hub or Apache Kafka can be leveraged as a stream processor. This is typically suited for IoT workloads.</p><p>The terms ESP and CEP are often used interchangeably in practice.</p><p><strong>Analytical data store:</strong></p><p>This is how the big data solutions work:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/440/1*TU2xY5npPoB0msYOiW7AeA.jpeg" /></figure><p>Analytical data store can be Kimball-style relational data warehouse</p><p><em>(Initiated by Ralph Kimball, the Kimball data model follows a bottom-up approach to data warehouse architecture design in which data marts are first formed based on the business requirements.)</em></p><p>Data can be presented by noSQL technologies like HBase or Hive etc. HDInsight supports Interactive Hive, HBase, and Spark SQL.</p><p><strong>Analysis and reporting:</strong></p><p>o The purpose of most of the big data solutions is to provide insights into the data by analysing and reporting. It may support business analytics via Microsoft Power BI or Microsoft Excel. The architecture may have a data modelling layer like OLAP cube or tabular data model in Azure analysis services. Azure services support analytical notebooks like Jupyter allowing users to explore data with Python or R. Microsoft R server can be used for large scale data exploration.</p><p><strong>Orchestration</strong></p><p>o Orchestration technology like Azure Data Factory / Apache Oozie / Sqoop can be utilized to automate repetitive data processing operations in the workflow (e.g. data transformation, moving data between different sources, load the data into analytical data store).</p><p><strong>Where to use:</strong></p><p>· Multiple subsystems have to process same events.</p><p>· Respond in real time to changing conditions.</p><p>· Complex event processing like pattern matching, drawing inferences .</p><p>· Analysis of high velocity of data such as IoT / big data.</p><p><strong>Benefits:</strong></p><p>· Distributed and scalable.</p><p>· New consumers can be easily added, point to point integration is avoided.</p><p>· Decoupling: Segregation between producers and consumers.</p><p>· Events can be responded to quickly.</p><p>Applications of event stream processing include but not limited to eCommerce, Fraud Detection, Intelligence and surveillance, Network monitoring etc.</p><p><strong>Challenges:</strong></p><p>· Guaranteed delivery.</p><p>e.g. In IoT scenarios, it’s important to guarantee that events are delivered.</p><p>· Processing events exactly once or in order .</p><p><strong>Other considerations:</strong></p><p>· The amount of data included in the event which affects performance and cost. This is tricky as having all required information in the event can save additional processing. However, putting minimum information in the event (like only ID or Order / Customer) will reduce transport time and cost but will increase the processing time needed to fetch additional information (e.g. Order / Customer Details here).</p><p><strong>Best Practices:</strong></p><p>· No point-to-point integrations</p><p>· High performance and highly scalable systems.</p><p>· Being capable of coupling and decoupling into different networks in response to different events.</p><p><strong>Reference:</strong></p><p><a href="https://docs.microsoft.com/en-us/azure/">https://docs.microsoft.com/en-us/azure/</a></p><p><a href="https://en.wikipedia.org/wiki/Complex_event_processing">https://en.wikipedia.org/wiki/Complex_event_processing</a></p><p><a href="https://www.bmc.com/blogs/event-stream-processing/">https://www.bmc.com/blogs/event-stream-processing/</a></p><p><a href="https://en.wikipedia.org/wiki/Event_stream_processing">https://en.wikipedia.org/wiki/Event_stream_processing</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d335e554e6a5" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Leadership Lessons From Krishna]]></title>
            <link>https://pradnyapatil29.medium.com/leadership-lessons-from-krishna-da318aa01a46?source=rss-232555faa69e------2</link>
            <guid isPermaLink="false">https://medium.com/p/da318aa01a46</guid>
            <category><![CDATA[leadership-skills]]></category>
            <category><![CDATA[leadership-development]]></category>
            <category><![CDATA[leadership-lessons]]></category>
            <category><![CDATA[lord-krishna]]></category>
            <dc:creator><![CDATA[Pradnya Patil]]></dc:creator>
            <pubDate>Mon, 22 Aug 2022 12:59:16 GMT</pubDate>
            <atom:updated>2022-08-22T12:59:16.624Z</atom:updated>
            <content:encoded><![CDATA[<p>I have always been enchanted by the stories of Lord Krishna from Mahabharata (an Indian Epic) that I have been listening from my mom since childhood.</p><p>I admire him as a versatile leader. I thought of writing about what we can learn from him on the auspicious occasion of his birthday (Krishna Janmashtami).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1023/1*U1N0uE_eCSNAsBe0G223Hw.jpeg" /></figure><p><strong>Lessons from Lord Krishna:</strong></p><ul><li>Focusing on process rather than outcome.</li><li>Practicing detachment — being able to easily let go of the role once the purpose is served and moving on to the next.</li><li>Ready to go to any length for the highest good of all.</li><li>Being creative about the solutions.</li><li>Being light hearted and joyous even in the midst of dire situations.</li><li>Pursuing arts and music (hobbies) apart from career.</li><li>Succession planning and knowledge sharing.</li><li>Forming meaningful connections. Being easy going and kind yet powerful and strong.</li><li>Being diplomatic and stragetic with the enemy. Using their weakness to advantage.</li><li>Flexibly demonstrating different types of leadership based on the situation at hand — Empathetic, Strategic, Transformational, Charismatic, Servant, Meantor.</li></ul><p>There are many more which we can write about.</p><p>What do you think are the best leadership qualities of Lord Krisha ?</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=da318aa01a46" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Product Management — Kano Model]]></title>
            <link>https://pradnyapatil29.medium.com/product-management-kano-model-e157deb7aae?source=rss-232555faa69e------2</link>
            <guid isPermaLink="false">https://medium.com/p/e157deb7aae</guid>
            <dc:creator><![CDATA[Pradnya Patil]]></dc:creator>
            <pubDate>Wed, 08 Jun 2022 02:40:49 GMT</pubDate>
            <atom:updated>2022-06-08T02:44:42.374Z</atom:updated>
            <content:encoded><![CDATA[<h3>Product Management — Kano Model</h3><p>Recently I started studying models for assesing the impact of Product features on customer satisfaction and I stumbled upon the “Kano Model”.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/704/1*FAynkCPrBJ7rxaqPiHhtvw.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/865/1*2p_ldmCT5GjhannGnW45BA.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7Mv0ucW_Sv7h8Lw7Jj54eQ.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*J0JFvhm7JsFT6t89Utrfzg.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/747/1*y0bhmgubs20hRfxNb4WoEA.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/681/1*ols3IUbMRgjZ4vuUXzfvLw.jpeg" /></figure><p>Which is your preferred model when it comes to assesing the impact of Product features ?</p><p>Eager to learn from you all.</p><p><strong>References:</strong></p><p><a href="https://www.mindtools.com/pages/article/newCT_97.htm">https://www.mindtools.com/pages/article/newCT_97.htm</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e157deb7aae" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Cloud Application: Architectural Styles — Part 4 ( Event Driven )]]></title>
            <link>https://pradnyapatil29.medium.com/cloud-application-architectural-styles-part-4-event-driven-396130b014a3?source=rss-232555faa69e------2</link>
            <guid isPermaLink="false">https://medium.com/p/396130b014a3</guid>
            <dc:creator><![CDATA[Pradnya Patil]]></dc:creator>
            <pubDate>Wed, 23 Jun 2021 19:49:53 GMT</pubDate>
            <atom:updated>2021-06-23T19:49:53.074Z</atom:updated>
            <content:encoded><![CDATA[<p><strong>Cloud Application: Architectural Styles — Part 4 ( Event Driven )</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*Q5VZVVhgUKci9hA0FpMYFg.png" /></figure><p><strong>Introduction</strong></p><p>In this series of articles, we are going to study the cloud application architectural styles, patterns and practical examples. In Part-4 of the series, we are going to have an overview of the “Event-driven” architecture style along with its logical diagram, where to use this style, benefits, and challenges.</p><p>We will study further cloud architectural styles and patterns in subsequent articles of this series.</p><p>You can check my articles on <a href="https://pradnyapatil29.medium.com/cloud-application-architectural-styles-part-1-bbda76f8ad3f?source=your_stories_page-------------------------------------">“N-tier”</a>,<a href="https://pradnyapatil29.medium.com/cloud-application-architectural-styles-part-2-b67469d524fa?source=your_stories_page-------------------------------------"> “Microservices”</a> and <a href="https://pradnyapatil29.medium.com/cloud-application-architectural-styles-part-3-web-queue-worker-956a902a6271?source=your_stories_page-------------------------------------">“Web — Queue — Worker”</a> architectural styles.</p><p><strong>Background</strong></p><p>The important components of this architecture include “event producers” which produce events and “event consumers” which listen to them.</p><blockquote>Events are produced in almost real time and consumers respond to them as they occur. Producers are separated from consumers and are not aware of the producers. Consumers are also segregated from each other, and each consumer can see all the events.</blockquote><p>This is little different from “Competing Consumers” pattern where consumers pull a message from a queue, and it is processed only once. In systems like IoT, high volume of events must be ingested.</p><p><strong>Logical Diagram:</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/481/1*95JfKvxvB2MgyhyEOHMXGw.jpeg" /></figure><p>Event driven architecture can use “Publisher / Subscriber” model or “Event Stream” model.</p><p><strong>Publisher / Subscriber model:</strong></p><ul><li>Event is sent to each subscriber once it is published.</li><li>The messaging infrastructure tracks the subscriptions.</li><li>Event cannot be replayed once received and new subscribers do not see the event.</li></ul><p><strong>Event Streaming model:</strong></p><ul><li>Events are strictly ordered, durable and written to the log.</li><li>Clients can read from stream, but they don’t subscribe to it.</li><li>Client has to advance the position in the stream. They can replay the events and join any time.</li></ul><p>Here are the different flavors on the consumer side:</p><p><strong>Simple event processing: </strong>An instant action is triggered by the event in the consumer. e.g. Whenever a message is published to Service Bus Topic or Event Grid, an Azure Function is executed.</p><p><strong>Complex event processing (CEP):</strong></p><p>The goal of complex event processing is to identify meaningful events (such as opportunities or threats) in real-time situations and respond to them as quickly as possible.</p><blockquote>The series of events are carefully analyzed / readings from an embedded device are aggregated over a time window to generate a notification if the moving average crosses a certain threshold.</blockquote><p>I found this hilarious example on <a href="https://en.wikipedia.org/wiki/Complex_event_processing">wikipedia</a>:</p><p>Among thousands of incoming events, a monitoring system may for instance receive the following three from the same source:</p><p>(1) Church bells ringing.</p><p>(2) The appearance of a man in a tuxedo with a woman in a flowing white gown.</p><p>(3) Rice flying through the air.</p><p>From these events the monitoring system may infer a complex event: a wedding.</p><p>Technologies such as Azure Stream Analytics, Apache Storm etc are useful for Complex Event Processing.</p><p><strong>Event stream processing (ESP):</strong></p><p>This <a href="https://www.bmc.com/blogs/event-stream-processing/">blog</a> explains the event stream processing nicely.</p><p>An <em>event</em> is anything that happens at a clearly defined time and that can be specifically recorded. A <em>stream</em> is a constant and continuous flow of event objects that navigate into and around companies from thousands of connected devices, IoT, and any other sensors. <em>Processing</em> is the final act of analyzing all this data.</p><blockquote><em>So, putting all three of these together, you could say that </em>event stream processing<em> is the process of being able to quickly analyze data streaming from one device to another at an almost instantaneous rate after it is created.</em></blockquote><p>It is meant for continuous monitoring and interactions, detecting any interesting patterns, handling issues etc.</p><p>Platforms like Azure IoT Hub or Apache Kafka can be leveraged as a stream processor. This is typically suited for IoT workloads.</p><p>The terms ESP and CEP are often used interchangeably in practice.</p><p><strong>Where to use:</strong></p><ul><li>Multiple subsystems have to process same events</li><li>Respond in real time to changing conditions</li><li>Complex event processing like pattern matching, drawing inferences</li><li>Analysis of high velocity of data such as IoT / big data.</li></ul><p><strong>Benefits:</strong></p><ul><li>Distributed and scalable</li><li>New consumers can be easily added, point to point integration is avoided.</li><li>Decoupling: Segregation between producers and consumers.</li><li>Events can be responded to quickly.</li></ul><blockquote>Applications of event stream processing include but not limited to eCommerce, Fraud Detection, Intelligence and surveillance, Network monitoring etc.</blockquote><p><strong>Challenges:</strong></p><ul><li>Guaranteed delivery. e.g. In IoT scenarios, it’s important to guarantee that events are delivered.</li><li>Processing events exactly once or in order .</li></ul><p><strong>Other considerations:</strong></p><blockquote>The amount of data included in the event which affects performance and cost. This is tricky as having all required information in the event can save additional processing. However, putting minimum information in the event (like only ID or Order / Customer) will reduce transport time and cost but will increase the processing time needed to fetch additional information (e.g. Order / Customer Details here).</blockquote><p><strong>Best Practices:</strong></p><ul><li>No point-to-point integrations</li><li>High performance and highly scalable systems.</li><li>Being capable of coupling and decoupling into different networks in response to different events.</li></ul><p><strong>References:</strong></p><p><a href="https://docs.microsoft.com/en-us/azure/">https://docs.microsoft.com/en-us/azure/</a></p><p><a href="https://en.wikipedia.org/wiki/Complex_event_processing">https://en.wikipedia.org/wiki/Complex_event_processing</a></p><p><a href="https://www.bmc.com/blogs/event-stream-processing/">https://www.bmc.com/blogs/event-stream-processing/</a></p><p><a href="https://en.wikipedia.org/wiki/Event_stream_processing">https://en.wikipedia.org/wiki/Event_stream_processing</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=396130b014a3" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Cloud Application: Architectural Styles — Part 3 (Web — Queue — Worker)]]></title>
            <link>https://pradnyapatil29.medium.com/cloud-application-architectural-styles-part-3-web-queue-worker-956a902a6271?source=rss-232555faa69e------2</link>
            <guid isPermaLink="false">https://medium.com/p/956a902a6271</guid>
            <category><![CDATA[cloud]]></category>
            <category><![CDATA[architecture]]></category>
            <category><![CDATA[azure-architecture]]></category>
            <category><![CDATA[software-design-patterns]]></category>
            <category><![CDATA[design]]></category>
            <dc:creator><![CDATA[Pradnya Patil]]></dc:creator>
            <pubDate>Sun, 13 Dec 2020 17:44:42 GMT</pubDate>
            <atom:updated>2020-12-13T17:44:42.821Z</atom:updated>
            <content:encoded><![CDATA[<h3>Cloud Application: Architectural Styles — Part 3 (Web — Queue — Worker)</h3><p><strong>Introduction</strong></p><p>In this series of articles, we are going to study the cloud application architectural styles, patterns and practical examples. In Part-3 of the series, we are going to have an overview of the “Web-Queue-Worker-driven” architecture style along with its logical diagram, where to use this style, benefits and challenges.</p><p>This article also revisits some of the “Microservices” concepts.</p><p>We will study further cloud architectural styles and patterns in subsequent articles of this series.</p><p>You can check the discussion about “N-tier”, “Microservices” architectural styles in “<strong>Cloud Application: Architectural Styles — Part 1 (N-Tier)</strong>”</p><p><a href="https://pradnyapatil29.medium.com/cloud-application-architectural-styles-part-1-bbda76f8ad3f">Cloud Application: Architectural Styles — Part 1</a></p><p>and “Microservices” in in “<strong>Cloud Application: Architectural Styles — Part 2 (Microservices)</strong>”.</p><p><a href="https://pradnyapatil29.medium.com/cloud-application-architectural-styles-part-2-b67469d524fa">Cloud Application: Architectural Styles — Part 2</a></p><p><strong>Background</strong></p><blockquote>The important components of this architecture include a web front end which serves http requests from client and a worker which runs long running / resource intensive tasks or batch processes. The communication between web front end and worker happens through a message queue.</blockquote><h3>Web-Queue-Worker architecture style</h3><p>It can be considered for purely PaaS solutions. The architecture is easy to understand just like N-tier. Worker is an optional component and can be omitted if there are no long running tasks.</p><p><strong>Logical Diagram:</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/303/1*GQiGfR6iU83ly44pC4ogcA.png" /></figure><p>There are some other important components also in the microservice architecture:</p><ul><li>Identity provider for authentication</li><li>CDN for serving static content (images, script files, CSS etc)</li><li>Remote services (e.g. SMS or email)</li><li>One or more databases</li><li>Cache for accessing values faster from database</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/887/1*5AMXlHOwHKWnhPzyqNVAoQ.png" /></figure><p><strong>Management:</strong></p><p>Responsibilities of the management component:</p><ul><li>Placing services on nodes</li><li>Rebalancing services across nodes</li><li>Identifying failures</li></ul><p><strong>Service Discovery</strong></p><p>Responsibilities:</p><ul><li>Enable service lookup to find endpoint of the service</li><li>Maintain a list of services and nodes on which they are located</li></ul><p><strong>API Gateway:</strong></p><p>Responsibilities:</p><ul><li>Acts as an entry point for the clients</li><li>Clients call the API Gateway instead of calling the service which forwards call to the appropriate services on the back end.</li><li>It may aggregate responses from multiple services and return the aggregated response.</li></ul><p><strong>Advantages of API Gateway:</strong></p><ul><li>Services can be versioned and refactored without updating all the clients. It decouples the clients from services.</li><li>Services can use non web friendly messaging protocols like AMQP.</li><li>API Gateway can perform other functions like authentication, logging, load balancing, SSL termination etc.</li></ul><p><strong>Where to use:</strong></p><ul><li>Large applications which need high release velocity</li><li>Complex apps which are required to be highly scalable</li><li>Apps with rich domains and many subdomains</li><li>An organization having small development teams</li></ul><p><strong>Benefits:</strong></p><p><strong>Independent development:</strong> Each service can be built by a small, focused development team which results in continuous innovation and faster release cycle.</p><p><strong>Mixed technology stacks:</strong> We can pick the technology stack that is most suitable for the service.</p><p><strong>Small, focused teams:</strong> The smaller scope of services makes the code easier to understand and easier to ramp up for new developers. The team can focus on one service.</p><p><strong>Independent deployments:</strong> Service can be updated or rolled back without redeploying the entire application. Fixing defects and releasing features becomes more manageable and involves less risk. Frequent updates can be deployed as we can deploy individual services with minimal coordination between teams.</p><p><strong>Granular Scaling: </strong>Services can be scaled independently.</p><p><strong>Fault Isolation: </strong>Even if a service goes down, it won’t stop entire application. The application can become resilient provided you follow resiliency best practices and design patterns.</p><p><strong>Challenges:</strong></p><p><strong>Data integrity:</strong> Since each microservice is responsible for its own data persistence, data consistency can be a challenge. Hence, we need to adopt eventual consistency wherever possible.</p><blockquote>(<em>Eventual consistency: using an event-driven, eventually consistent approach, each service publishes an event whenever it </em>updates<em> its data. Other </em>services<em> subscribe to events. When an event is received, a service updates its data.</em>)</blockquote><p>With each microservice responsible for its own data persistence. As a result, data consistency can be a challenge. Embrace eventual consistency where possible.</p><p><strong>Complexity: </strong>Though each service is simple, the system as a whole becomes more complex since microservices application has more moving parts as compared to the monolithic application.</p><p><strong>Management:</strong> Microservices requires a mature DevOps culture. Correlated logging across services can be challenging. Typically, logging must correlate multiple service calls for a single user operation.</p><p><strong>Development and test: </strong>Testing services dependencies poses a challenge. Existing tools are not necessarily designed to work with service dependencies. Refactoring across service boundaries can be difficult.</p><p><strong>Network congestion and latency:</strong> We need to carefully design the APIs, avoiding overly chatty ones and also think about serialization formats and look for places to use asynchronous communication patterns. Using many, small granular services can cause more interservice communication. Also, if the chain of service dependencies gets too long (service A calls B, which calls C…), the additional latency can become a problem.</p><p><strong>Lack of governance: </strong>The decentralized approach to building microservices has its own Pros and Cons. You may end up with many different languages and frameworks that the system becomes hard to maintain. It may be useful to put some project-wide standards in place without restricting team’s flexibility.</p><p><strong>Skillset:</strong> We need to carefully evaluate if the team has skills and experience to be successful building microservices since they are highly distributed systems.</p><p><strong>Versioning:</strong> Updates to a service must not break services that depend on it. Multiple services could be updated at any given time, so without careful design, you might have problems with backward or forward compatibility.</p><h4><strong>Best Practices:</strong></h4><p>Model services around the business domain.</p><p><strong>Create a Separate Data Store for Each Microservice:</strong></p><p>You end up with the situation where if one team updates a database structure, other services that also use that structure has to be changed too.</p><p><em>(Breaking apart the data can make data management more complicated, because the separate storage systems can more easily get out sync or become inconsistent, and foreign keys can change unexpectedly. You need to add a tool that performs master data management (MDM) by operating in the background to find and fix inconsistencies.)</em></p><p><strong>Keep Code at a Similar Level of Maturity:</strong></p><p>Keep all code in a microservice at a similar level of maturity and stability. In other words, if you need to add or rewrite some of the code in a deployed microservice that’s working well, the best approach is usually to create a new microservice for the new or changed code, leaving the existing microservice in place</p><p>(Immutable infrastructure principle)</p><p><strong>Deploy in Containers:</strong></p><p>Deploying microservices in containers is important because it means you just need just one tool to deploy everything.</p><p>APIs should model the domain, not the internal implementation of the service:</p><p>Services communicate through well-designed APIs. Avoid leaking implementation details.</p><p><strong>Loose coupling and high functional cohesion:</strong></p><p>Functions that are likely to change together should be packaged and deployed together. If they reside in separate services, those services end up being tightly coupled, because a change in one service will require updating the other service. Overly chatty communication between two services may be a symptom of tight coupling and low cohesion.</p><p><strong>Isolate failures:</strong></p><p>Use resiliency strategies to prevent failures within a service from cascading.</p><p>Offload cross-cutting concerns, such as authentication and SSL termination, to the gateway.</p><p><strong>Keep domain knowledge out of the gateway:</strong></p><p>The gateway should handle and route client requests without any knowledge of the business rules or domain logic. Otherwise, the gateway becomes a dependency and can cause coupling between services.</p><p><strong>References:</strong></p><p><a href="https://docs.microsoft.com/en-us/azure/">https://docs.microsoft.com/en-us/azure/</a></p><p><a href="https://martinfowler.com/articles/microservices.html">https://martinfowler.com/articles/microservices.html</a></p><p><a href="https://www.nginx.com/blog/microservices-at-netflix-architectural-best-practices/">https://www.nginx.com/blog/microservices-at-netflix-architectural-best-practices/</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=956a902a6271" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Cloud Application: Architectural Styles — Part 2]]></title>
            <link>https://pradnyapatil29.medium.com/cloud-application-architectural-styles-part-2-b67469d524fa?source=rss-232555faa69e------2</link>
            <guid isPermaLink="false">https://medium.com/p/b67469d524fa</guid>
            <category><![CDATA[design]]></category>
            <category><![CDATA[cloud-computing]]></category>
            <category><![CDATA[microservices]]></category>
            <category><![CDATA[software-architecture]]></category>
            <category><![CDATA[cloud-services]]></category>
            <dc:creator><![CDATA[Pradnya Patil]]></dc:creator>
            <pubDate>Tue, 28 May 2019 07:25:51 GMT</pubDate>
            <atom:updated>2020-12-13T17:13:46.590Z</atom:updated>
            <content:encoded><![CDATA[<h3><strong>Cloud Application: Architectural Styles — Part 2 </strong>(Microservices)</h3><p><strong>Introduction</strong></p><p>In this series of articles, we are going to study the cloud application architectural styles, patterns and practical examples. In Part-2 of the series, we are going to have an overview of the “Microservices” architectural styles along with their logical diagram, where to use these styles, their benefits and challenges.</p><p>We will study further cloud architectural styles and patterns in subsequent articles of this series.</p><p><strong>Background</strong></p><p>I like this description which I found on <a href="https://martinfowler.com/articles/microservices.html">MartinFowler.com</a> site:</p><p><em>The term “Microservice Architecture” has sprung up over the last few years to describe a particular way of designing software applications as suites of independently deployable services. While there is no precise definition of this architectural style, there are certain common characteristics around organization around business capability, automated deployment, intelligence in the endpoints, and decentralized control of languages and data.</em></p><p>You can check the discussion about “N-tier ”, “Web-Queue-Worker” architectural styles in “<strong>Cloud Application: Architectural Styles — Part 1 (N-Tier)</strong>”</p><p><a href="https://medium.com/@pradnyapatil29/cloud-application-architectural-styles-part-1-bbda76f8ad3f">Cloud Application: Architectural Styles — Part 1</a></p><h3>Microservices architecture style</h3><p>A microservices application consists of many small, independent services. Each service implements a single business capability. Services are loosely coupled, communicating through API contracts. (I guess this architectural style deserves a separate article so I will try to limit my scope here.)</p><p>Microservices are like next evolution of service oriented architectures (SOA), but there are differences between microservices and SOA. Here are some characteristics of a microservice:</p><ul><li>Each service has a separate codebase which can be managed by a small development team</li><li>Services are small, independent, and loosely coupled in microservices architecture.</li><li>Existing service can be updated without redeploying or rebuilding entire application. So the deployment is independent.</li><li>Implementation details of the service are hidden from other services. Services communicate with each other by using well defined APIs.</li><li>Services are not required to share same technology stack, frameworks or libraries.</li><li>Services are responsible for persisting their own data or external state in contrast to the traditional model where data persistence is handled by a separate data layer.</li></ul><p><strong>Logical Diagram:</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/469/1*plNosCktr73gTVKOOEqIig.png" /></figure><p>There are some other important components also in the microservice architecture:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/622/1*7zsOjpH34U4UQx6d10K72g.png" /></figure><p><strong>Management:</strong></p><p>Responsibilities of the management component:</p><ul><li>Placing services on nodes</li><li>Rebalancing services across nodes</li><li>Identifying failures</li><li>Service Discovery</li></ul><p><strong>Responsibilities:</strong></p><ul><li>Enable service lookup to find endpoint of the service</li><li>Maintain a list of services and on which nodes they are located</li></ul><p><strong>API Gateway :</strong></p><p><strong>Responsibilities:</strong></p><ul><li>Acts as an entry point for the clients</li><li>Clients call the API Gateway instead of calling the service which forwards call to the appropriate services on the back end.</li><li>It may aggregate responses from multiple services and return the aggregated response.</li></ul><p><strong>Advantages of API Gateway:</strong></p><ul><li>Services can be versioned and refactored without updating all the clients. It decouples the clients from services.</li><li>Services can use non web friendly messaging protocols like AMQP</li><li>API Gateway can perform other functions like authentication, logging, load balancing, SSL termination etc.</li></ul><p><strong>Where to use:</strong></p><ul><li>Large applications which need high release velocity</li><li>Complex apps which are required to be highly scalable</li><li>Apps with rich domains and many subdomains</li><li>An organization having small development teams</li></ul><p><strong>Benefits:</strong></p><p><strong>Independent development:</strong> Each service can be built by a small, focused development team which results in continuous innovation and faster release cycle.</p><p><strong>Mixed technology stacks:</strong> We can pick the technology stack that is most suitable for the service.</p><p><strong>Small, focused teams:</strong> The smaller scope of services makes the code easier to understand and easier to ramp up for new developers. The team can focus on one service.</p><p><strong>Independent deployments:</strong> Service can be updated or rolledback without redeploying the entire application. Fixing defects and releasing features becomes more manageable and invloves less risk. Frequent updates can be deployed as we can deploy individual services with minimal coordination between teams.</p><p><strong>Granular Scaling: </strong>Services can be scaled independently.</p><p><strong>Fault Isolation: </strong>Even if a services goes down, it won’t stop entire application. The application can become resilient provided you follow resiliency best practices and design patterns.</p><p><strong>Challenges:</strong></p><p><strong>Data integrity:</strong> Since each microservice is responsible for its own data persistence, data consistency can be a challenge. Hence we need to adopt eventual consistency wherever possible.</p><p>(<em>Eventual consistency: using an event-driven, eventually consistent approach, each service publishes an event whenever it </em>updates<em> its data. Other </em>services<em> subscribe to events. When an event is received, a service updates its data.</em>)</p><p>With each microservice responsible for its own data persistence. As a result, data consistency can be a challenge. Embrace eventual consistency where possible.</p><p><strong>Complexity:</strong> Though each service is simple, the whole system as a whole becomes more complex since microservices application has more moving parts as compared to the monolithic application.</p><p><strong>Management:</strong> Microservices requires a mature DevOps culture. Correlated logging across services can be challenging. Typically, logging must correlate multiple service calls for a single user operation.</p><p><strong>Development and test:</strong> Testing services dependencies poses a challenge. Existing tools are not necessarily designed to work with service dependencies. Refactoring across service boundaries can be difficult.</p><p><strong>Network congestion and latency:</strong> We need to carefully design the APIs, avoiding overly chatty ones and also think about serialization formats and look for places to use asynchronous communication patterns. Using many, small granular services can cause more interservice communication. Also, if the chain of service dependencies gets too long (service A calls B, which calls C…), the additional latency can become a problem.</p><p><strong>Lack of governance:</strong> The decentralized approach to building microservices has its own Pros and Cons. You may end up with many different languages and frameworks that the system becomes hard to maintain. It may be useful to put some project-wide standards in place without restricting team’s flexibility.</p><p><strong>Skillset:</strong> We need to carefully evaluate if the team has skills and experience to be successful building microservices since they are highly distributed systems.</p><p><strong>Versioning:</strong> Updates to a service must not break services that depend on it. Multiple services could be updated at any given time, so without careful design, you might have problems with backward or forward compatibility.</p><p><strong>Best Practices:</strong></p><p>Model services around the business domain.</p><p><strong>Create a Separate Data Store for Each Microservice:</strong></p><p>You end up with the situation where if one team updates a database structure, other services that also use that structure have to be changed too.</p><p>(Breaking apart the data can make data management more complicated, because the separate storage systems can more easily get out sync or become inconsistent, and foreign keys can change unexpectedly. You need to add a tool that performs master data management (MDM) by operating in the background to find and fix inconsistencies.)</p><p><strong>Keep Code at a Similar Level of Maturity:</strong></p><p>Keep all code in a microservice at a similar level of maturity and stability. In other words, if you need to add or rewrite some of the code in a deployed microservice that’s working well, the best approach is usually to create a new microservice for the new or changed code, leaving the existing microservice in place</p><p>(Immutable infrastructure principle)</p><p><strong>Deploy in Containers:</strong></p><p>Deploying microservices in containers is important because it means you just need just one tool to deploy everything.</p><p><strong>APIs should model the domain, not the internal implementation of the service:</strong></p><p>Services communicate through well-designed APIs. Avoid leaking implementation details.</p><p><strong>Loose coupling and high functional cohesion:</strong></p><p>Functions that are likely to change together should be packaged and deployed together. If they reside in separate services, those services end up being tightly coupled, because a change in one service will require updating the other service. Overly chatty communication between two services may be a symptom of tight coupling and low cohesion.</p><p><strong>Isolate failures:</strong></p><p>Use resiliency strategies to prevent failures within a service from cascading.</p><p>Offload cross-cutting concerns, such as authentication and SSL termination, to the gateway.</p><p><strong>Keep domain knowledge out of the gateway:</strong></p><p>The gateway should handle and route client requests without any knowledge of the business rules or domain logic. Otherwise, the gateway becomes a dependency and can cause coupling between services.</p><p><strong>Reference:</strong></p><p><a href="https://docs.microsoft.com/en-us/azure/">https://docs.microsoft.com/en-us/azure/</a></p><p><a href="https://martinfowler.com/articles/microservices.html">https://martinfowler.com/articles/microservices.html</a></p><p><a href="https://www.nginx.com/blog/microservices-at-netflix-architectural-best-practices/">https://www.nginx.com/blog/microservices-at-netflix-architectural-best-practices/</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b67469d524fa" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Cloud Application: Architectural Styles — Part 1]]></title>
            <link>https://pradnyapatil29.medium.com/cloud-application-architectural-styles-part-1-bbda76f8ad3f?source=rss-232555faa69e------2</link>
            <guid isPermaLink="false">https://medium.com/p/bbda76f8ad3f</guid>
            <category><![CDATA[software-architecture]]></category>
            <category><![CDATA[queue]]></category>
            <category><![CDATA[architecture]]></category>
            <category><![CDATA[cloud-computing]]></category>
            <category><![CDATA[architectural-styles]]></category>
            <dc:creator><![CDATA[Pradnya Patil]]></dc:creator>
            <pubDate>Mon, 27 May 2019 11:53:49 GMT</pubDate>
            <atom:updated>2020-12-13T17:14:45.290Z</atom:updated>
            <content:encoded><![CDATA[<h3><strong>Cloud Application: Architectural Styles — Part 1 (N-Tier)</strong></h3><p><strong>Introduction</strong></p><p>In this series of articles, we are going to study the cloud application architectural styles, patterns and practical examples. In Part-1 of the series, we are going to have an overview of the “N-tier architecture” and “Web-Queue-Worker” architectural styles along with their logical diagram, where to use these styles, their benefits and challenges.</p><p>We will study further cloud architectural styles and patterns in subsequent articles of this series.</p><p><strong>Background</strong></p><p>As cloud is changing the way applications are designed. Instead of monoliths, applications are decomposed into smaller, decentralized services. These services communicate through APIs or by using asynchronous messaging or eventing. Applications scale horizontally, adding new instances as demand requires.</p><p>These trends bring new challenges. Application state is distributed. Operations are done in parallel and asynchronously. The system as a whole must be resilient when failures occur.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tzltIR1aUAZ3SmpMwTdTmA.png" /></figure><p><strong>Architecture styles</strong></p><p>The first decision point is the most fundamental. What kind of architecture are you building? It might be a microservices architecture, a more traditional N-tier application, or a big data solution. We have identified several distinct architecture styles. There are benefits and challenges to each.</p><p>An architectural style defines: a family of systems in terms of a pattern of structural organization; a vocabulary of components and connectors, with constraints on how they can be combined.</p><p>Fox example, N-Tier is a common architectural style. Microservices architecture has become more popular recently. Architectural styles are not specific to technology. However, some technologies are best suited for certain architectures.</p><p>In this article, we will study a set of architectural stypes for cloud application.</p><ul><li>Logical diagram</li><li>When to use this style</li><li>Benefits, challenges, best practices</li><li>Recommended deployment using Azure Services</li></ul><p><strong>N-Tier</strong></p><p>An N-tier architecture divides an application into logical layers and physical tiers.</p><p>Layers are a way to separate responsibilities and manage dependencies. Each layer has a specific responsibility. A higher layer can use services in a lower layer, but not the other way around.</p><p>Tiers are physically separated, running on separate machines. A tier can call to another tier directly, or use asynchronous messaging (message queue). Although each layer might be hosted in its own tier, that’s not required. Several layers might be hosted on the same tier. Physically separating the tiers improves scalability and resiliency, but also adds latency from the additional network communication.</p><p><strong>Logical Diagram:</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/315/1*86z30r0gwbwJs42uTEIRCg.png" /></figure><p>N-tier is a traditional architecture for enterprise applications. Dependencies are managed by dividing the application into layers that perform logical functions, such as presentation, business logic, and data access. A layer can only call into layers that sit below it.</p><p>An N-tier application can have a closed layer architecture or an open layer architecture:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/565/1*QbJ3sXMNiWR3zLxS2gpA4Q.png" /></figure><p><strong>Challenges:</strong></p><ul><li>Due to horizontal layering, it becomes hard to make changes in one part of the application without touching the rest of the application.</li><li>Hence it is difficult to make frequent updates and it takes more time to add a feature.</li></ul><p>· It’s easy to end up with a middle tier that just does CRUD operations on the database, adding extra latency without doing any useful work.</p><p>· Monolithic design prevents independent deployment of features.</p><p>· Managing an IaaS application is more work than an application that uses only managed services.</p><p>· It can be difficult to manage network security in a large system.</p><p><strong>Benefits</strong>:</p><p>Portability between cloud and on-premises, and between cloud platforms.</p><p>·Less learning curve for most developers.</p><p>·Natural evolution from the traditional application model.</p><p>·Open to heterogeneous environment (Windows/Linux)</p><p><strong>Where to use:</strong></p><p>N-tier is a good fit for migrating existing applications that use a layered architecture. Hence it is mostly observed in IaaS (Infrastructure as a service) solutions or mix of IaaS and managed services with each tier running on a separate set of VMs. However, an N-tier application doesn’t need to be pure IaaS. Often, it’s advantageous to use managed services for some parts of the architecture, particularly caching, messaging, and data storage.</p><p>Consider an N-tier architecture for:</p><p>·Simple web applications.</p><p>·Migrating an on-premises application to Azure with minimal refactoring.</p><p>·Unified development of on-premises and cloud applications.</p><p>N-tier architectures are very common in traditional on-premises applications, so it’s a natural fit for migrating existing workloads to Azure.</p><h3>Web-Queue-Worker</h3><p>The application has a web front end that handles HTTP requests and a back-end worker that performs long running CPU-intensive tasks in this style. The front end communicates to the worker through an asynchronous message queue.</p><h3>Logical Diagram</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/303/1*5i7iE7sNqvQdokNtLLAP9w.png" /></figure><p>Other components that are commonly incorporated into this architecture include:</p><p>·One or more databases.</p><p>·A cache to store values from the database for quick reads.</p><p>·CDN to serve static content</p><p>·Remote services, such as email or SMS service. Often these are provided by third parties.</p><p>·Identity provider for authentication.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/887/1*Vxv10P7tmgUvouJiMf5iWQ.png" /></figure><p>The web and worker are both stateless. Session state can be stored in a distributed cache. Any long-running work is done asynchronously by the worker.</p><p>The worker is an optional component. If there are no long-running operations, the worker can be omitted.</p><p>The front end might consist of a web API. On the client side, the web API can be consumed by a single-page application that makes AJAX calls, or by a native client application.</p><p>The Web-Queue-Worker architecture is typically implemented using managed compute services, either Azure App Service or Azure Cloud Services.</p><p><strong>Where to use:</strong></p><p>·For a PaaS ( Platform as a service ) solution</p><p>·Applications with a relatively simple domain.</p><p>· Applications with some long-running workflows or batch operations.</p><p>· When you want to use managed services, rather than infrastructure as a service (IaaS).</p><p><strong>Benefits</strong>:</p><ul><li>The architecture is easy to understand.</li></ul><p>· Easy to deploy and manage.</p><p>· Clear separation of concerns.</p><p>· The front end is decoupled from the worker using asynchronous messaging.</p><p>· The front end and the worker can be scaled independently.</p><p><strong>Challenges</strong>:</p><ul><li>With complex domains, it can be hard to manage dependencies. The front end and worker can easily become large, monolithic components that are hard to maintain and update.</li></ul><p>· There may be hidden dependencies, if the front end and worker share data schemas or code modules.</p><p>Reference:</p><p><a href="https://docs.microsoft.com/en-us/azure/">https://docs.microsoft.com/en-us/azure/</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=bbda76f8ad3f" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>