<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Mayur SSoni on Medium]]></title>
        <description><![CDATA[Stories by Mayur SSoni on Medium]]></description>
        <link>https://medium.com/@mayurssoni2456?source=rss-d8758f4a3d09------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*FlogSF_0aU3_rnmY.jpg</url>
            <title>Stories by Mayur SSoni on Medium</title>
            <link>https://medium.com/@mayurssoni2456?source=rss-d8758f4a3d09------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 08:27:16 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@mayurssoni2456/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Part 2 — Database Transactions — Exploring Concurrency Control and Isolation Levels]]></title>
            <link>https://mayurssoni2456.medium.com/part-2-database-transactions-exploring-concurrency-control-and-isolation-levels-ae0a468a3c63?source=rss-d8758f4a3d09------2</link>
            <guid isPermaLink="false">https://medium.com/p/ae0a468a3c63</guid>
            <category><![CDATA[isolation-level]]></category>
            <category><![CDATA[database-transaction]]></category>
            <category><![CDATA[sql]]></category>
            <category><![CDATA[database-locks]]></category>
            <category><![CDATA[database]]></category>
            <dc:creator><![CDATA[Mayur SSoni]]></dc:creator>
            <pubDate>Sat, 30 Dec 2023 15:36:56 GMT</pubDate>
            <atom:updated>2023-12-30T15:36:56.736Z</atom:updated>
            <content:encoded><![CDATA[<h3>Part 2 — Database Transactions — Exploring Concurrency Control and Isolation Levels</h3><p>Understanding a topic through <strong>intuition is key to learning</strong>.</p><p>In <a href="https://medium.com/@mayurssoni2456/database-transactions-in-a-very-simple-language-55c154cbf690">Part-1</a>, we covered — What is Transactions, and the role of ACID properties in the Database to ensure data consistency.</p><p>Now, let’s understand <strong>Concurrency Control</strong> and <strong>Isolation</strong> <strong>Levels</strong>.</p><h3>Concurrency Control</h3><p>Just as the traffic lights regulate the flow, concurrency control governs the access and interaction of transactions, avoiding collisions and maintaining the smooth operation of the database.</p><p>Concurrency control mechanisms are important in managing simultaneous access to data within a database.</p><ul><li>Lock-Based Concurrency Control</li><li>Two Phase Locking</li></ul><p><strong>Lock-Based Concurrency Control</strong></p><ol><li><strong>Shared (Read) Locks</strong>: These locks allow multiple transactions to read data simultaneously but prevent any transaction from making changes while the data is being read. This type of lock is useful when multiple users need to access the same data concurrently without modifying it.</li><li><strong>Exclusive (Write) Locks</strong>: Exclusive locks restrict access to a single transaction, ensuring that only one transaction can modify the data at any given time. This lock prevents other transactions from reading or writing to the data until the lock is released.</li></ol><p><strong>Two-Phase Locking</strong></p><p>Two-phase locking includes <strong>Acquiring</strong> the lock and <strong>Releasing</strong> the lock.</p><p>In the <strong><em>Acquiring</em></strong> phase — Transactions acquire the locks they need before accessing or modifying the data.</p><p><strong><em>Releasing</em></strong> the lock — After a transaction has acquired all necessary locks and completed the operations, it releases all the locks it acquired. Once the lock is released, other transactions can acquire the lock.</p><p>This will help in avoiding <strong><em>Deadlocks and the problem of Data inconsistency.</em></strong></p><h3>Isolation Levels</h3><p>Imagine a shared kitchen with several chefs. Each chef works on a dish independently (higher isolation levels), avoiding conflicts over ingredients. on the other hand, some chefs freely share ingredients and space (lower isolation levels), risking clashes but allowing collaboration.</p><p>Isolation levels determine the degree of visibility and interaction between the concurrent transactions.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3xxWC2TbR5QZc0J1w4i2-A.png" /><figcaption><a href="https://jlvbcoop.com/en/transaction-isolation-levels-2/">https://jlvbcoop.com/en/transaction-isolation-levels-2/</a></figcaption></figure><ul><li><strong>Read Uncommitted</strong>: This is the lowest isolation level where a transaction can read uncommitted changes made by other transactions. It offers no guarantees about data consistency and can result in dirty reads, non-repeatable reads, and phantom reads.</li><li><strong>Read Committed: </strong>A transaction can only read committed data. It prevents dirty reads, but it might still encounter non-repeatable reads and phantom reads as other transactions commit changes concurrently.</li><li><strong>Repeatable Read</strong>: This ensures that once a record is read within a transaction, subsequent reads of the same record during the transaction will return the same data. It prevents non-repeatable reads but may still face phantom reads.</li><li><strong>Serializable: </strong>This is the highest level of isolation. It ensures strict transaction isolation by preventing concurrent transactions from affecting each other. However, please keep in mind — that strict locking can lead to performance issues.</li></ul><p><strong>In Part 3</strong> — I will try to cover the real-world case studies so that you can relate and understand more in-depth about these concepts<strong>.</strong></p><p><strong>…Stay tuned</strong></p><p>Thanks for reading :)</p><p><em>To learn more about me — </em><a href="https://www.linkedin.com/in/mayurssoni2456/">https://www.linkedin.com/in/mayurssoni2456/</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ae0a468a3c63" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Database Transactions — In a very simple language]]></title>
            <link>https://mayurssoni2456.medium.com/database-transactions-in-a-very-simple-language-55c154cbf690?source=rss-d8758f4a3d09------2</link>
            <guid isPermaLink="false">https://medium.com/p/55c154cbf690</guid>
            <category><![CDATA[sql]]></category>
            <category><![CDATA[database]]></category>
            <category><![CDATA[software-engineering]]></category>
            <category><![CDATA[database-transaction]]></category>
            <dc:creator><![CDATA[Mayur SSoni]]></dc:creator>
            <pubDate>Sun, 24 Dec 2023 16:28:23 GMT</pubDate>
            <atom:updated>2023-12-24T16:28:24.227Z</atom:updated>
            <content:encoded><![CDATA[<h3>Database Transactions — In a very simple language</h3><p>Understanding a topic through <strong>intuition is key to learning</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/590/1*V2VDyKKHZCH9wYB1ssQGhg.jpeg" /></figure><h3>Transactions</h3><p>In simple language, Is a recipe in cooking, where all ingredients need to be added and cooked together to produce a complete dish.</p><p>Let me give you some more concrete examples:</p><ol><li>The act of transferring money between two bank accounts</li><li>Booking a ticket online, etc.</li></ol><p>The key thing to notice in the above examples is those use cases are sequences of actions, that can be either <strong>complete or incomplete.</strong></p><h3>Problem Statement</h3><p>I want to ensure that either my set of actions succeeds or fails.</p><h3>How this can be addressed?</h3><p>By RELIABILITY | CONSISTENCY | INTEGRITY</p><p><strong>Reliability</strong> — Imagine ordering a pizza for delivery. Reliability is like trusting that the pizza will arrive within the promised time, hot and fresh.</p><p>In a database, reliability means ensuring that when you request information (like ordering a pizza), the system consistently delivers the correct and expected results without errors or delays.</p><p><strong>Consistency</strong> — Think of baking cookies where each cookie should have the same shape and taste.</p><p>Similarly, in a database, for instance, if you update your address in one place, the change should reflect everywhere else in the system immediately.</p><p><strong>Integrity — </strong>Just as a librarian ensures no torn pages or missing chapters in a book, database integrity ensures there are no incorrect or missing pieces of data.</p><p><em>Some of the above terms are nothing but foundational principles of the Databases and its termed </em><strong>ACID</strong><em> properties.</em></p><h3>ACID</h3><p>In essence, ACID properties in database management systems guarantee that transactions are executed reliably, maintain consistency, and safeguard the integrity of the data, thereby ensuring the overall correctness and reliability of the database operations.</p><h3>A — Atomicity</h3><p>Think of buying a combo meal at a fast-food restaurant. Just like a combo meal, where all items (burger, fries, drink) are served together or not at all.</p><p>In a database transaction, atomicity ensures that if a part of the transaction fails (like adding money from one account and deducting from another in a fund transfer), the entire transaction is rolled back, keeping the database in its original state.</p><h3><strong>C — Consistency</strong></h3><p>Imagine assembling a puzzle where every piece fits perfectly into place, forming a complete picture.</p><p>Each data entry or modification must align correctly with the predefined rules and constraints, ensuring that the overall database structure remains coherent and accurate.</p><h3>I — Isolation</h3><p>Imagine multiple chefs cooking in the same kitchen but working independently on different recipes without interfering with each other’s ingredients or cooking methods</p><p>In Database — Isolation ensures that multiple transactions can occur simultaneously without affecting each other.</p><h3>D — Durability</h3><p>Think of saving important files on a sturdy external hard drive. Even if the computer crashes, the files remain safe and accessible when the system restarts.</p><p>Durability ensures that once a transaction is committed, its changes are permanent and survive system failures. Committed changes are stored in a way that they can be recovered even in the event of a power outage or system crash.</p><p>In Part 2 — Database Transactions, we will cover <strong>Concurrency Control and Isolation Levels. Stay tuned.</strong></p><p>Thanks for reading :)</p><p><em>To learn more about me — </em><a href="https://www.linkedin.com/in/mayurssoni2456/">https://www.linkedin.com/in/mayurssoni2456/</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=55c154cbf690" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Event Storming — A crucial step in Software Design]]></title>
            <link>https://mayurssoni2456.medium.com/event-storming-a-crucial-step-in-software-design-c172c5b5af6?source=rss-d8758f4a3d09------2</link>
            <guid isPermaLink="false">https://medium.com/p/c172c5b5af6</guid>
            <category><![CDATA[microservices]]></category>
            <category><![CDATA[software-engineering]]></category>
            <category><![CDATA[domain-driven-design]]></category>
            <category><![CDATA[eventstorming]]></category>
            <category><![CDATA[software-design]]></category>
            <dc:creator><![CDATA[Mayur SSoni]]></dc:creator>
            <pubDate>Sun, 27 Feb 2022 19:16:56 GMT</pubDate>
            <atom:updated>2022-02-27T19:16:56.551Z</atom:updated>
            <content:encoded><![CDATA[<p>To achieve Business Goal it is really important to understand the business Domain, Capabilities and how technology can contribute to achieve their strategic goal.</p><h3>Why?</h3><ul><li>There was a major communication gap that happened to exists between Business and Engineering team — <strong>a common language</strong>. Business team does not understand design patterns, database, micro-service or any technical jargons. So, it’s important to communicate in a language that can be understood by everyone.</li><li>From Engineering perspective its very important to understand the Business Domain, Capabilities, relationship between these capabilities, actors, etc.</li></ul><p>Here, comes <strong>Event Storming</strong> — this ensures that the business domain experts and technology experts arrive at a common understanding before constructing a system.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*WScETP7JQk6dc4nJW1gLMQ.jpeg" /><figcaption>source: <a href="http://www.dreamstime.com/">www.dreamstime.com</a></figcaption></figure><h3>Let’s look at key components of Event Storming</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-dg-l5x_u7ngzI6AeAFiSg.jpeg" /></figure><p>Now since, we are familiar with what, why</p><p><em>let’s see how we can </em><strong><em>APPLY</em></strong><em> them…</em></p><h3>Use Case</h3><p>Business ABCLoan who is providing</p><blockquote><strong>Disclaimer</strong> — I am not a domain expert but I will try to add based on my knowledge</blockquote><p>Goal — from Tech company <strong>perspective </strong>is to understand the business better and see how we as a tech company can contribute to achieve business goals.</p><p>1. Understand the domain and list out the sub-domains</p><p>2. For any one sub-domain, we will apply event storming</p><p><strong><em>Domains and Sub-domains</em></strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/843/1*vj0TboABR5aUFqZ5K7PV5A.jpeg" /></figure><h4>We will do an Event Storming for Retail Loans Sub-domain</h4><p>Steps1 — Customer apply for a loan</p><p>Steps2 — Bank will gather more data for a customer</p><p>Steps3 — Validation and Credit Evaluation</p><p>Steps4 — Risk Analysis (How much risk is involved in giving the loan)</p><p>Steps5 — Underwriting — Lender assess the borrower’s ability to repay loan bases on the inputs from above steps</p><p>Step6 — Approval and disbursement of loan</p><h4>Event Storming Step 1</h4><p>In this step people brainstorm and share what they know openly to the team. There will be certain process to refine and merge the events. The outcome would be Domain events.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*NUEdXl00uSitS9C5p_gJnA.jpeg" /></figure><h4>Time Sequencing of Domain Events — Step 2</h4><p>Events are sequenced in ascending order of time. Events happening parallel have been put vertically. This process will discover what happens before the event and after the event in the entire flow.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*LWr_3aPZ0rBOnKEkudffXw.jpeg" /></figure><h4>Identification of Triggers/Commands — Step 3</h4><p>Commands can be initiated by actors, both internal and external to the system. This process will help you to get the broader context in the system in which it operates.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Lcz8ZjFH8J3XJj1usTrhHQ.jpeg" /></figure><h4>Aggregates — Step 4</h4><p>This process will give you the capabilities within the system by grouping command, event, actors.</p><p>For e.g. in above diagram — Customer’s loan request+ Customer initial credit verification+ Documents collection can be combined as <strong>LoanAsses</strong> capability.</p><p>Validation and Risk assessment cab considered as <strong>Operate</strong> capability.</p><p>Initialising the underwriting process, collateral verification, loan processing can be part of <strong>Underwriting</strong> capability.</p><h4>Summary</h4><p>The entire process of event storming gives us the understanding of domain, actors of the domain and give us broader context of the system which can help us to understand the high level capabilities in the business application and segregate the micro-service.</p><p>References:</p><ul><li><a href="https://www.lucidchart.com/blog/ddd-event-storming#:~:text=Event%20storming%20drives%20greater%20understanding,processes%20more%20efficiently%20and%20enjoyably.">https://www.lucidchart.com/blog/ddd-event-storming</a></li><li><a href="https://www.eventstorming.com/">https://www.eventstorming.com/</a></li><li><a href="https://www.agilealliance.org/the-power-of-eventstorming/">https://www.agilealliance.org/the-power-of-eventstorming/</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c172c5b5af6" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[A Decade of journey as a Software Engineer — Mayur SSoni]]></title>
            <link>https://mayurssoni2456.medium.com/a-decade-of-journey-as-a-software-engineer-mayur-ssoni-9e996abb8dbb?source=rss-d8758f4a3d09------2</link>
            <guid isPermaLink="false">https://medium.com/p/9e996abb8dbb</guid>
            <category><![CDATA[software-engineering]]></category>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[developer-stories]]></category>
            <category><![CDATA[full-stack]]></category>
            <category><![CDATA[programming-languages]]></category>
            <dc:creator><![CDATA[Mayur SSoni]]></dc:creator>
            <pubDate>Wed, 16 Jun 2021 12:17:33 GMT</pubDate>
            <atom:updated>2021-06-16T12:17:33.752Z</atom:updated>
            <content:encoded><![CDATA[<h3>A Decade of journey as a Software Engineer — Mayur SSoni</h3><p>In the year <strong>2010</strong>, I earned my graduate degree in computer Applications and had plans for higher studies but Time &amp; Fate had something else on my plate to cater to.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/1*l_dH2LWMpyZ6Z8nxMDa6wQ.jpeg" /><figcaption><a href="https://www.homestratosphere.com/types-of-ladders/">https://www.homestratosphere.com/types-of-ladders/</a></figcaption></figure><p>Every individual has a dream to work in an MNC but a direct entry in the doors is difficult without a master’s that’s what the common thought is, But it’s a myth. With hard work &amp; sheer dedication, one can enter the world of MNCs. Also one should not let their dreams fade as I believe if you dream then only you achieve.</p><p>It was a startup that gave me a start to my career in 2010. While working with Elan Emerging Technologies (My 1st job) I had made my mind clear that I need to prove the caliber within and take on the responsibility of supporting my family on my shoulder.</p><p><strong>iOS Application development</strong> was the start of my journey where I gave in a year and as fortune favours the brave, I got a break to work on the development of a small <strong>Gaming project</strong> (<strong>Marmalade now</strong>) for 3 months’ post which I again switched on iOS development and continued to excel myself for a period of two years.</p><p>Then, there came a twist and an excellent opportunity following my dreams knocked and My dream of working with an MNC seemed to be coming true and I cleared the interview to bag the offer from a giant MNC — <strong>TCS</strong></p><p>The Journey with TCS began and I worked on interesting &amp; skilful projects. It was a very smooth ongoing journey but as we all know there are ups &amp; downs in everybody’s graph of life, due to which I decided to start with a freelance business. The experience taught me many great <strong>LIFE LESSONS</strong> to take away during the 1 year of my freelance career.</p><p>I learned Swift language in tenure of freelancing, but the earnings were not that great by which I could support my family and thus I decided to get back into the job industry again and in 2016 — I happened to step into the company which was in my bucket list — <strong>Deloitte</strong></p><p>Ialways kept the doors open for new learning &amp; accepted the change. Learning <strong>Cross-Platform</strong> technology was on my radar in 2017 and I started learning<strong> React-Native (JavaScript). </strong>Soon we got our first client project for React Native in Deloitte USI. I never looked back since then and delivered multiple projects for almost 3 years.</p><p>What Next?.. Full-stack</p><p>I was ready to take in something new and stepped on the ladder leading towards <strong>a Full-stack Journey</strong>. At <strong>Deloitte,</strong> I have always got the space to explore, learn &amp; work on a new Tech Stack. Since then I have acquired the fundamentals of <strong>AWS, NodeJs, TypeScript.</strong> It has been over a year now; I have been working as a <strong>Backend Engineer.</strong></p><p>And the journey continues to grow and so I.</p><p><strong>Next what..?</strong></p><p>1. AWS Certifications — thankfully completed SA-2021 :)</p><p>2. Solution Architect — This is ongoing and it comes with the experience.</p><p>I will end this with a beautiful quote…</p><blockquote><em>“Every morning of our life bring us some </em><strong><em>CHALLENGES</em></strong><em> and every evening of our life leaves us with </em><strong><em>EXPERIENCES” — </em></strong><em>Gaur Gopal Das</em></blockquote><p>So always keep learning no matter — you will gain some experience.</p><p>Thanks for reading.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9e996abb8dbb" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Understanding System Design]]></title>
            <link>https://mayurssoni2456.medium.com/understanding-system-design-6d01ad6a4a30?source=rss-d8758f4a3d09------2</link>
            <guid isPermaLink="false">https://medium.com/p/6d01ad6a4a30</guid>
            <category><![CDATA[design-systems]]></category>
            <category><![CDATA[system-design-interview]]></category>
            <category><![CDATA[software-system-design]]></category>
            <dc:creator><![CDATA[Mayur SSoni]]></dc:creator>
            <pubDate>Wed, 12 May 2021 07:07:23 GMT</pubDate>
            <atom:updated>2021-05-12T07:07:23.001Z</atom:updated>
            <content:encoded><![CDATA[<p>I am writing this article about my process of learning system design and I will try to include the topics which are related to system design.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*UQ6TANunpkqLMHuCwijjOg.png" /></figure><p>To start with — What is System Design?</p><p>SD defines the architecture, components, modules and data for a system based on the requirements. It the most integral part of Software engineering.</p><p><strong>Key Principles of System Design</strong></p><ol><li>Availability</li><li>Performance</li><li>Reliability</li><li>Scalability</li><li>Manageability</li><li>Cost</li></ol><h3>Availability</h3><p>Design a system to be constantly available and resilient to failure. It’s very <strong>critical</strong> as unavailability of System can lead to bad reputation of the business and a big loss of revenue. Having said that on the other hand system failure is part of software; the key thing is how rapidly it recovers and gracefully.</p><h3>Performance</h3><p>This is again another factor that directly co-relates to revenue and retention. The performance of the application affects usage and user satisfaction. Hence, designing a system that is optimised for fast responses and low latency is important.</p><h3>Reliability</h3><p>System needs to be reliable — Users need to know that if something is written to the system, or stored, it will persist and can be relied on to be in place for future retrieval.</p><h3>Scalability</h3><p>For a large distributed system — scalability is very important. I would not consider only the <strong>SIZE</strong> as an important factor to scale but also the <strong>TIME AND HOW EASY </strong>it is to scale. Scalability parameters could be extending storage size, handling addition traffic, processing more transaction per second, etc.</p><h3>Manageability</h3><p>How <strong>simple the system is to operate</strong> which includes scalability operations, maintenance and updates.</p><h3>Cost</h3><p>Cost is an important factor. This obviously can include hardware and software costs, but it is also important to consider other facets needed to deploy and maintain the system. The amount of developer time the system takes to build, the amount of operational effort required to run the system, and even the amount of training required should all be considered. Cost is the total cost of ownership.</p><p>Each of these principles provides the basis for decisions in designing a distributed system. However, they is also possibility that achieving one objective comes at the cost of another. <strong>e.g.</strong> choosing to address <strong>capacity</strong> by simply adding more servers (<strong>scalability</strong>) can come at the price of <strong>manageability</strong> (you have to operate an additional server) and <strong>cost</strong> (the price of the servers).</p><h3>System Design</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1eDnUnX3TkWfM0kDQPx6Zw.png" /></figure><h3>Building blocks for Large scale system</h3><p><strong>DNS</strong></p><p>In layman terms, It translate name of a person into telephone number(phonebook of the Internet). DNS map a host name to an IP address and vice-versa.</p><p><strong>CDN</strong></p><p>In layman terms, CDN Is a distribution system similar to the post office network. Making your content available at the closest point of presence aka the post office.</p><p>A content delivery network (CDN) is a huge collection of high powered servers that are globally distributed or I should say a collection of servers which work together to provide fast delivery of content through the Internet.</p><p>Why? One of the main reason is to address latency issues — CDN server caches your content. <strong>CDN </strong>does not host a website in its server, but rather it caches from the original web server and provides fast delivery of that content.</p><h4>Load Balancer</h4><p>Load balancing is a core networking solution responsible for distributing incoming traffic among servers of an application. It prevents any application server from becoming a single point of failure, thus improving overall application availability and responsiveness</p><p>Load balancer performs the following functions:</p><ul><li>Distributes client requests or network load efficiently across multiple servers</li><li>Ensures high availability and reliability by sending requests only to servers that are up and running.</li><li>Provides the flexibility to add or subtract servers as demand dictates</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/848/1*QF3Q7DOkAQmUe8sq21YUzg.png" /></figure><h4>Compute Service (Servers)</h4><p>Servers could be dedicated Physical servers on-premise or virtual servers provided by cloud services(AWS, GCP, etc.). It’s part of your infra setup which helps an application for computation workload</p><h4>Data Lake</h4><p>A Data lake is a storage repository that holds a vast amount of raw data. It can store structured, semi-structured, or unstructured data, which means data can be kept in a more flexible format for future use. In essence is nothing but storage part of Big data world that could be <strong>Hadoop, S3 </strong>or any cheap big data storage systems.</p><h4>Data Warehouse</h4><p>Data warehouse is like a neatly arranged wardrobe.</p><p>In simple words, a data warehouse is a central repository containing historical data collected from various source system like (ERP, Finance, HRM, CRM, Sales). Data is extracted periodically and moved to data warehouse. This includes the process of data cleansing, formatting and summarising(Extract, Transform and Load — ETL).</p><p>Ordinary Databases store MBs and GBs of data, which is recently collected. They are not meant to store historic data which could range up to TBs. Due to this reason ordinary Databases don’t provide the facility of analytics. So to store all this large scale historic data, <strong>Data warehouses</strong> were introduced.</p><p>Data warehouses help in insightful and strategic decision making for the business by providing analytical support.</p><p>e.g. Redshift, Teradata,</p><h4>Database</h4><p>I believe — reader is already aware about the database but let me still put it in simple words for those who don’t know. A <strong>database</strong> is an organised collection of <strong>data</strong>. Data is organised into rows, columns and tables, and it is indexed to make it easier to find relevant information. Data gets updated, expanded and deleted as new information is added.</p><p>Database could be Relational or Non-Relational. e.g. Postgres (relational), DynamoDB (non-relational)</p><h4>Caching</h4><p>Here we are talking about in-memory data storage system (e.g. Redis, Memcached). Caching is storing data in a location different than the main data source such that it’s faster to access the data. Generally, caches contain the most recently accessed data as there is a chance that recently requested data is likely to be asked again and again. So, in such scenarios, we need to use caching to minimise data retrieval operation from the database.</p><p>It helps in reducing the source database call and improvement performance as well.</p><blockquote><strong>e.g.</strong> Caching popular Facebook celebrity profile access. Say, a celebrity updates a new post with photos in a Facebook profile. A lot of people are checking that profile updates. If all the users are requesting the same status update and the database has to retrieve the same data every time, it will put huge pressure on the database. In such cases, we can cache the profile of the popular profile and get the data from cache service</blockquote><p>If data is mostly static, caching is an easy way to improve performance. In the case of data that is edited often, the cache is a bit tricky to implement.</p><h4>Distributed Queue System</h4><p>Queues are everywhere in today’s modern distributed systems architecture — adopted across various industries for different use cases.</p><p>A queue provides temporary storage between the <strong>sender</strong> and the <strong>receiver</strong> so that the sender can keep operating without interruption when the destination program is busy or not connected.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*NX_hJ1H0-eiIqY7mm9_2wQ.png" /><figcaption><a href="https://ctoasaservice.org/2019/08/30/the-aws-messaging-stack-sqs-sns-kinesis/">https://ctoasaservice.org/2019/08/30/the-aws-messaging-stack-sqs-sns-kinesis/</a></figcaption></figure><p>There are client applications called <strong>producers</strong> that create messages and deliver them to the message queue. Another application, called a <strong>consumer</strong>, connects to the queue and gets the messages to be processed. Messages placed onto the queue are stored until the consumer retrieves them.</p><p>In <strong><em>Synchronous processing, we</em></strong></p><ol><li>call a service, then</li><li>wait for the service to finish, and then</li><li>move on to the next task.</li></ol><p><strong><em>Asynchronous processing</em></strong><em>.</em> on the other hand, allows a task to call a service, and <em>move on to the next task</em> while the service processes the request at its own pace. That’s why a queue is a beautiful, elegant way to unblock your systems because it puts a layer in front of your services and allows them to tackle the tasks at their own pace.</p><h4>Real-time Streaming Service</h4><p>The process of taking action on data at the time the <strong>data is generated or published</strong>. One of the best examples of a real-time system are those used in the stock market.</p><p>Some of the stream services <strong>Kinesis</strong>, <strong>Kafka</strong> are real-time streaming services.</p><p><em>Thanks for reading. If you enjoyed this article, comment below: </em><strong><em>What all other components would you include which covers almost all use case of designing a Large scale system.</em></strong></p><p><em>Follow me on </em><a href="https://www.linkedin.com/in/mayurssoni2456"><em>LinkedIn</em></a><em>.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6d01ad6a4a30" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>