<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Sujoy Acharya on Medium]]></title>
        <description><![CDATA[Stories by Sujoy Acharya on Medium]]></description>
        <link>https://medium.com/@sujoy.acharya?source=rss-deac8e3b885c------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*tthEnAUvu3RfsQ5E.jpg</url>
            <title>Stories by Sujoy Acharya on Medium</title>
            <link>https://medium.com/@sujoy.acharya?source=rss-deac8e3b885c------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 12:57:26 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@sujoy.acharya/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Free Coffee, Little’s Law and the Strangler Figs]]></title>
            <link>https://medium.com/@sujoy.acharya/free-coffee-littles-law-and-the-strangler-figs-6d6be4defb8c?source=rss-deac8e3b885c------2</link>
            <guid isPermaLink="false">https://medium.com/p/6d6be4defb8c</guid>
            <category><![CDATA[design-patterns]]></category>
            <category><![CDATA[messaging]]></category>
            <category><![CDATA[scalability-in-cloud]]></category>
            <category><![CDATA[cloud-native]]></category>
            <category><![CDATA[distributed-systems]]></category>
            <dc:creator><![CDATA[Sujoy Acharya]]></dc:creator>
            <pubDate>Wed, 19 Oct 2022 12:23:04 GMT</pubDate>
            <atom:updated>2022-10-19T12:56:08.720Z</atom:updated>
            <content:encoded><![CDATA[<p>On a gloomy Monday morning, you decided to do a social experiment with an old Nespresso coffee machine. You started exploring opportunities(pleading to your spouse?) to transform the study room into a self-serving free coffee zone.</p><p>Free coffee rocks! Unexpectedly, it became an instant hit, and people started queuing up in front of your building to grab a cup of special mocha; It felt divine!</p><figure><img alt="Free Coffee" src="https://cdn-images-1.medium.com/max/1024/0*xXpcFfa-AyP7zg2W" /></figure><p>And then your neighbours, who had been drinking free coffee (god forbid), started complaining about the mass gathering outside the building.</p><p>You felt dejected and wanted to solve this problem. Now, you need a bigger space to accommodate everyone so that no one is standing outside, and you are ready to risk your privacy and savings. However, how would you scientifically determine whether the living (not betting the bedroom, right?) room is adequate for the devotees?</p><p>How can you figure it out?</p><h3>Every little thing matters!</h3><p>John Little published a paper in 1954 to help us with this problem. He proposed an equation.</p><h3>L = λ * W</h3><p>Where:</p><p>L — the average number of items in a queuing system</p><p>λ — the average number of items arriving at the system per unit of time</p><p>W — the average waiting time an item spends in a queuing system</p><p>Suppose every coffee aficionado spends 6 minutes at your place(the coffee machine is ageing and needs some cooldown period), and at least thirty visitors brew coffee in an hour.</p><p>As per Little’s law -</p><p>L = 30 * 6/60 ~= 3 persons queuing / minute</p><p>On average, three neighbours are queuing in your coffee corner. Therefore, you need to have a room of at least = 3 * 6 ft (of social distancing space) + 2 ft coffee table = 20 ft long.</p><h3>Microservices and Little&#39;s Law</h3><p>Like our free coffee queue, we can apply this law in cloud-native environments to determine the efficiency of queueing system. A queue decouples publishers/suppliers from the consumers. The publisher(s) can publish, and the consumers can consume messages independently.</p><p>Cloud-native applications often experience considerable demand flux and require to scale instantaneously. Cloud practitioners favour scaling horizontally by spinning additional machines/VMs or application instances to an existing cluster. In Kubernetes, the horizontal pod autoscaler(HPA) can be configured to provision resources automatically to match the demand.</p><p>Cloud-native microservices define KPIs(Key Performance Indicators), and metrics can help businesses determine the ability of their services to meet specific goals. For instance, is my system working effectively during high demand?</p><p>Some of the industry standard metrics are mean time before failure(MTBF), mean time to recover/repair (MTTR) and mean time to acknowledge (MTTA). They also define goals for the performance of the system, such as</p><ol><li>SLA — service level agreement between the service provider and consumers about the metrics like performance, reliability and availability</li><li>SLO — service level objectives. The objective of 99.999 % system availability</li><li>SLI — service level indicator. An actual measurement of the system’s availability.</li></ol><p>How can we optimize the system’s efficiency (meet the SLO) when the demand is high (flux in <strong>λ</strong>),<strong> </strong>and<strong> </strong>publishers stream messages at an exorbitant rate?</p><p>Little’s law says — <strong>L </strong>(Average no. of items in a queuing system)= <strong>λ </strong>(average arriving rate) * <strong>W </strong>(average waiting time)</p><p>The consumer microservices can autoscale, increase the message consumption rate, and reduce the queue’s average waiting time (<strong>W</strong>).</p><p>However, if each message takes time to process, we may need more resources to autoscale and meet the SLO.</p><p>The cloud infrastructure can scale elastically but can’t scale infinitely as cost and QoS are involved. Let’s examine the QoS part.</p><h3>The Strangler Figs</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/676/0*zB5FGy1S5BONC1qV" /></figure><p>Suppose the team applies the <a href="https://martinfowler.com/bliki/StranglerFigApplication.html"><em>Strangler Fig pattern</em></a> for digital transformation, starts with a small set of functionalities, converts them to services and gradually shrinks monoliths.</p><p>The service layer (consuming messages from a request queue) may keep up with the publisher’s speed, autoscale and make calls to the monolith. However, the monolith cannot stretch beyond a limit.</p><p>This hybrid cloud + monolith interaction is an exciting scenario. The following things can happen.</p><ol><li>The monolith doesn’t enforce <em>rate limiting</em>, cannot handle too many requests and may teardown the system (mainly the slowest component like the database). SLO not met.</li><li>The monolith applies <em>rate limiting</em> and rejects excessive requests by returning the TOO_MANY_REQUESTS status code (429). The service layer uses the stability pattern, retries the failures, and creates more churn in the system.</li></ol><p>Therefore, the service layer is scaling, effectively not doing anything and wasting the cloud resources ($$$).</p><p>Shall it drop the message consumption rate and work at a slow pace? Again, numerous things are involved. There’s a limit on the number of queued messages in vanilla MQ/TIBCO EMS type broker. They define max-bytes/memory for each queue so that only one kind of message doesn’t eat up the entire memory space and create the noisy-neighbour problem. Slowing down the processing of messages can create a ripple effect, reach the max-bytes limit sooner, and create an outage.</p><p>Suppose the message broker can scale horizontally like Apache Pulsar, and it can queue as many messages as we want. How would we handle the slow monolith calls?</p><p>Provided the monolith will live longer -</p><ol><li>Can we shift left to apply backpressure and embrace reactive programming?</li><li>Can we enforce outbound rate limiting by leveraging Redis or an OSS library like Bucket4j?</li><li>Can we leverage distributed semaphores and permits?</li></ol><p>Let’s park it for now till we finish our freshly brewed coffee.</p><h4>References</h4><ol><li>Strangler Fig image <a href="https://underthebanyan.files.wordpress.com/2016/10/living_root_bridge.jpg?w=676">https://underthebanyan.files.wordpress.com/2016/10/living_root_bridge.jpg?w=676</a></li><li>Free coffee image — <a href="https://img.freepik.com/free-vector/realistic-coffee-background-with-drawings_79603-603.jpg?w=2000">https://img.freepik.com/free-vector/realistic-coffee-background-with-drawings_79603-603.jpg?w=2000</a></li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6d6be4defb8c" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>