<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Mavitushar on Medium]]></title>
        <description><![CDATA[Stories by Mavitushar on Medium]]></description>
        <link>https://medium.com/@mavitushar16562?source=rss-213d35f91065------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*LW7kxHovAXt6gziD</url>
            <title>Stories by Mavitushar on Medium</title>
            <link>https://medium.com/@mavitushar16562?source=rss-213d35f91065------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Wed, 18 Nov 2026 00:04:04 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@mavitushar16562/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Backpressure: Why Even Scalable Systems Need to Slow Down]]></title>
            <link>https://medium.com/@mavitushar16562/backpressure-why-even-scalable-systems-need-to-slow-down-af0b7569818e?source=rss-213d35f91065------2</link>
            <guid isPermaLink="false">https://medium.com/p/af0b7569818e</guid>
            <dc:creator><![CDATA[Mavitushar]]></dc:creator>
            <pubDate>Sun, 14 Dec 2025 19:01:59 GMT</pubDate>
            <atom:updated>2025-12-14T19:05:33.533Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8gFVowR-sZT2OC39fUfz7g.png" /></figure><p>Modern systems scale faster than ever.<br>Auto-scaling groups, serverless platforms, and managed databases make capacity easier to add.</p><p>So a fair question arises:</p><blockquote><strong><em>If we can scale, why do we need backpressure?</em></strong></blockquote><p>The answer lies in what <strong>does not scale equally</strong>.</p><h3>The Real Constraint Isn’t Compute</h3><p>APIs can scale quickly.<br>But many critical dependencies cannot:</p><ul><li>Databases (locks, IOPS, connections)</li><li>External APIs (rate limits)</li><li>Payment gateways</li><li>Shared caches</li><li>Third-party services</li></ul><p>Scaling the frontend faster <strong>pushes pressure downstream</strong>.</p><p>Backpressure exists to prevent that imbalance from turning into an outage.</p><h3>A Balanced Example</h3><p>Assume:</p><ul><li>API layer auto-scales</li><li>Database safely handles ~200 queries/sec</li><li>Traffic spikes to 2,000 req/sec</li></ul><p>Two possible outcomes:</p><h3>Option 1: Accept Everything</h3><ul><li>DB latency increases</li><li>Timeouts trigger retries</li><li>Retry storms amplify load</li><li>Outage spreads across services</li></ul><h3>Option 2: Apply Backpressure</h3><ul><li>Limit concurrency</li><li>Queue when possible</li><li>Reject excess load early</li><li>Preserve core functionality</li></ul><p>Backpressure does <strong>not</strong> reduce capacity — <br> it <strong>preserves stability</strong>.</p><h3>Backpressure Is Not the Same as Rate Limiting</h3><p>This is a common misconception.</p><ul><li><strong>Rate limiting</strong> protects fairness and abuse</li><li><strong>Backpressure</strong> protects system health</li></ul><p>Most production systems need <strong>both</strong>.</p><h3>Why Autoscaling Alone Is Not Enough</h3><p>Autoscaling reacts <strong>after</strong> pressure appears.<br>Backpressure acts <strong>before</strong> failure occurs.</p><p>By the time scaling kicks in:</p><ul><li>Queues may already be full</li><li>DB connections may be exhausted</li><li>Retries may multiply traffic</li></ul><p>Backpressure prevents the system from entering that state in the first place.</p><h3>Practical, Minimal Backpressure (Node.js Example)</h3><pre>const MAX_IN_FLIGHT = 20;<br>let inFlight = 0;<br>app.post(&quot;/process&quot;, async (req, res) =&gt; {<br>  if (inFlight &gt;= MAX_IN_FLIGHT) {<br>    return res.status(503).send(&quot;Service busy, try again later&quot;);<br>  }<br>  inFlight++;<br>  try {<br>    await heavyDbOperation(req.body);<br>    res.send(&quot;Done&quot;);<br>  } finally {<br>    inFlight--;<br>  }<br>});</pre><p>This simple control:</p><ul><li>Protects the database</li><li>Keeps latency predictable</li><li>Avoids cascading failures</li></ul><h3>Backpressure in the AI Era</h3><p>AI introduces:</p><ul><li>Unpredictable request patterns</li><li>Expensive per-request cost</li><li>Aggressive retries</li></ul><p>In this environment:</p><ul><li>Unlimited acceptance = unlimited cost</li><li>Graceful rejection = controlled spend</li></ul><p>Backpressure becomes a <strong>cost-control mechanism</strong>, not just a reliability tool.</p><h3>The Mature Engineering View</h3><p>Scaling handles growth.</p><p>Backpressure handles stress.</p><p>Reliable systems use <strong>both</strong>.</p><p>Ignoring either is a risk.</p><h3>Final Thought</h3><p>Backpressure is not about saying <em>no</em> to users.<br>It’s about ensuring the system can continue saying <em>yes</em> — <br>consistently, predictably, and safely.</p><p>That’s not pessimism.<br>That’s <strong>professional engineering</strong>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=af0b7569818e" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[ Inbox & Outbox Pattern: The Unsung Hero of Reliable Event-Driven Systems]]></title>
            <link>https://medium.com/@mavitushar16562/inbox-outbox-pattern-the-unsung-hero-of-reliable-event-driven-systems-05af006f292f?source=rss-213d35f91065------2</link>
            <guid isPermaLink="false">https://medium.com/p/05af006f292f</guid>
            <category><![CDATA[backend-development]]></category>
            <category><![CDATA[kafka]]></category>
            <category><![CDATA[software-engineering]]></category>
            <category><![CDATA[system-design-interview]]></category>
            <category><![CDATA[design-systems]]></category>
            <dc:creator><![CDATA[Mavitushar]]></dc:creator>
            <pubDate>Tue, 07 Oct 2025 17:48:14 GMT</pubDate>
            <atom:updated>2025-10-07T17:48:14.136Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DzhRhmSKwC648ns9mCeLoQ.png" /></figure><p>We keep hearing words like <em>Kafka</em>, <em>Event-driven architecture</em>, <em>Microservices</em>, and <em>Async communication</em>.<br>And don’t get me wrong — they’re all amazing concepts that can make systems faster and more scalable.</p><p>But here’s the part that not enough people talk about — <strong>how do you make sure those events are actually reliable?</strong><br>Because if your system updates the database but misses an event… or publishes an event twice… things can go very wrong, very fast.</p><p>That’s where the <strong>Inbox and Outbox pattern</strong> quietly steps in.<br> It’s not glamorous, but it’s one of the most practical patterns every backend engineer should understand — especially when you’re building microservices that depend on asynchronous event flows.</p><h3>💥 The Real Problem Nobody Likes to Talk About</h3><p>Let’s take a simple example.<br>You have a <strong>Payment Service</strong> that updates a payment status in the database and then publishes an event to Kafka so other services (like Ledger or Notification) can react to it.</p><p>Now imagine this flow:</p><ol><li>Your app updates the payment status to SUCCESS.</li><li>It publishes the event PAYMENT_COMPLETED to Kafka.</li><li>But right after the DB commit, the Kafka broker goes down for 2 seconds.</li><li>The event never reaches Kafka.</li></ol><p>Now your database says “Payment completed.” But other services <strong>never got the memo</strong>.</p><p>You just created an <strong>inconsistency</strong> — one of the hardest problems in distributed systems. And when this happens in production, it’s a nightmare to debug and replay safely.</p><h3>🧰 Enter the Outbox Pattern</h3><p>The <strong>Outbox Pattern</strong> solves this problem by keeping your DB write and event publishing in sync — even if Kafka, RabbitMQ, or your network temporarily fail.</p><p>The idea is beautifully simple:</p><ul><li>You don’t publish directly to Kafka when saving data.</li><li>Instead, you write your event to an <strong>Outbox table</strong> in the same database <strong>within the same transaction</strong>.</li></ul><p>This guarantees that if your main operation succeeds, your event is stored too — no matter what.</p><p>Then, a background job or a CDC (Change Data Capture) process picks events from the Outbox and publishes them safely.</p><p>Here’s a simple example in Node.js-style pseudocode:</p><pre>await db.transaction(async (trx) =&gt; {<br>  await trx(&#39;payments&#39;).insert({ id, status: &#39;SUCCESS&#39; });<br>  <br>  await trx(&#39;outbox&#39;).insert({<br>    event_type: &#39;PAYMENT_COMPLETED&#39;,<br>    payload: JSON.stringify({ id, status: &#39;SUCCESS&#39; }),<br>    status: &#39;PENDING&#39;<br>  });<br>});</pre><pre>// Later: background job<br>const events = await db(&#39;outbox&#39;).where({ status: &#39;PENDING&#39; });<br>for (const e of events) {<br>  await kafkaProducer.send(e.payload);<br>  await db(&#39;outbox&#39;).where({ id: e.id }).update({ status: &#39;SENT&#39; });<br>}</pre><p>Simple, right?<br>But this small change makes your event flow <strong>bulletproof</strong>.</p><h3>✅ Benefits:</h3><ul><li><strong>Atomicity:</strong> DB and event are written together.</li><li><strong>Retry safety:</strong> If publishing fails, retry later.</li><li><strong>Audit log:</strong> You can see every event and its status.</li></ul><h3>📥 The Inbox Pattern — The Other Half of Reliability</h3><p>Now let’s look at the other side of the story — the <strong>consumer</strong>.<br>When your service consumes an event, can you guarantee that it’s received only once? Not really. Kafka and most brokers are <em>at-least-once</em> delivery systems — meaning duplicates <em>can</em> happen.</p><p>If you’re not careful, this can lead to weird bugs:</p><ul><li>Double inserts</li><li>Duplicate updates</li><li>Incorrect balances</li></ul><p>That’s where the <strong>Inbox Pattern</strong> helps.</p><p>You store a record for every event you process in an <strong>Inbox table</strong>.<br>If the same event comes again, you simply skip it.</p><p>Example:</p><pre>async function handleEvent(event) {<br>  const exists = await db(&#39;inbox&#39;).where({ event_id: event.id }).first();<br>  if (exists) return; // already processed</pre><pre>  await db.transaction(async (trx) =&gt; {<br>    await trx(&#39;inbox&#39;).insert({ event_id: event.id });<br>    await trx(&#39;orders&#39;)<br>      .where({ id: event.order_id })<br>      .update({ status: &#39;CONFIRMED&#39; });<br>  });<br>}</pre><p>This ensures your consumers are <strong>idempotent</strong> — no matter how many times the same message arrives, your data stays consistent.</p><h3>🧩 Putting It Together — A Reliable Event Flow</h3><p>Here’s how a healthy event-driven microservice setup looks when using both patterns:</p><pre>Service A<br>│<br>├── Writes data to main table<br>├── Writes event to Outbox table (same transaction)<br>│<br>├── Background publisher → Kafka topic<br>│<br>└── Service B consumes event<br>     ├── Checks Inbox table<br>     ├── If not processed → apply logic + insert Inbox record<br>     └── If duplicate → skip</pre><p>It’s simple but powerful. You now have <strong>end-to-end reliability</strong>:</p><ul><li>No lost events</li><li>No duplicates</li><li>Full audit trail</li></ul><h3>🏦 Real Example from a Fintech System</h3><p>In one of our projects (a fintech system handling payouts), we had a service that updated transaction states and triggered events for accounting and reporting systems.</p><p>Initially, we were directly publishing events to Kafka after DB updates.<br>Everything worked fine — until one day Kafka was down for maintenance.</p><p>Some transactions were committed in the database, but the corresponding events never made it out.<br>We had inconsistent states across services, and debugging it later was painful.</p><p>That’s when we implemented the <strong>Outbox Pattern</strong>.</p><p>Now, whenever we update a transaction:</p><ul><li>The event goes into the Outbox within the same DB transaction.</li><li>A background worker picks pending events and publishes them to Kafka.</li><li>If anything fails, we can replay safely from the Outbox.</li></ul><p>On the other side, our consumers use the <strong>Inbox Pattern</strong> to ensure idempotent processing.</p><p>This not only made our system more reliable — it also gave us <strong>clear visibility</strong> of what was sent, what failed, and what was retried.<br>And for audits (which happen a lot in fintech), the Inbox/Outbox tables became an absolute blessing.</p><h3>⚙️ Best Practices and Common Pitfalls</h3><p>A few things I’ve learned while implementing this pattern in production:</p><ol><li><strong>Keep payloads small.</strong><br>Don’t dump massive JSONs in Outbox. Store only essential data or IDs.</li><li><strong>Clean up regularly.</strong><br>Both the Inbox and Outbox tables can grow fast. Archive or delete old records periodically.</li><li><strong>Make publishing idempotent.</strong><br>Even with retries, ensure your producer won’t publish the same event twice.</li><li><strong>Use CDC if possible.</strong><br>Tools like <strong>Debezium</strong> can watch your Outbox table and publish automatically — super efficient and reliable.</li><li><strong>Monitor the Outbox worker.</strong><br>If it stops running, you’ll see pending events pile up. Set alerts or metrics.</li></ol><h3>🧭 The Bigger Lesson — Reliability Is Real Engineering</h3><p>As engineers, we often chase new technologies — Kafka, Redis Streams, gRPC, etc.<br>But real maturity in system design comes when we focus on <em>reliability</em>.</p><p>The Inbox and Outbox patterns may not sound exciting at first, but they quietly make your distributed systems <strong>predictable, debuggable, and resilient</strong>.</p><p>If you’ve ever dealt with missed events or inconsistent data between services, you already know how painful it can be.<br>This pattern doesn’t just fix that — it gives you confidence that your system will behave the same way even when things go wrong.</p><h3>🚀 Final Thoughts</h3><p>The next time you design an event-driven microservice, take a pause and ask:</p><blockquote><em>“What happens if Kafka is down right after my DB commit?”<br> “How do I ensure I don’t process the same event twice?”</em></blockquote><p>If you can answer those questions with <strong>Inbox and Outbox</strong>, you’re already a step ahead of many engineers out there.</p><p>🧩 <strong>Summary:</strong></p><ul><li>Outbox → ensures your events are never lost.</li><li>Inbox → ensures your events are never duplicated.<br> Together, they make your async systems <em>reliable</em> and <em>auditable</em>.</li></ul><p>If this post helped you understand the Inbox/Outbox pattern better, follow me here on Medium — I’ll be writing more about <strong>backend reliability patterns</strong>, <strong>microservice design</strong>, and <strong>real-world system engineering lessons</strong>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=05af006f292f" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[PROBLEM :]]></title>
            <link>https://medium.com/@mavitushar16562/problem-347d857ed708?source=rss-213d35f91065------2</link>
            <guid isPermaLink="false">https://medium.com/p/347d857ed708</guid>
            <category><![CDATA[mechanics]]></category>
            <dc:creator><![CDATA[Mavitushar]]></dc:creator>
            <pubDate>Wed, 30 Nov 2022 17:41:50 GMT</pubDate>
            <atom:updated>2022-11-30T17:41:50.666Z</atom:updated>
            <content:encoded><![CDATA[<blockquote>Mechanicus</blockquote><h3>PROBLEM :</h3><p>The country’s roadways are in poor condition. Finding a repair at that point on those foreign roads is a difficult chore. If a car breaks down in the middle of the road, the trip becomes horrible. In order to deal with this, we came up with a solution that would link them to the local mechanics who were in the region at the time. The major goal of the application is to give two social groups — the first being the service-seeking group, and the other being the service-providing group — a platform through it. This software will provide a platform for meeting these people.</p><h3>Solution</h3><p>An Service based application Mechanicus makes it simple for us to locate mechanics in our area. If our car breaks down while we’re on the road, this app will assist us locate a list of nearby repairs. The User Module and the Mechanic Module would be its two modules. The user’s role is to go to the website, log in, and search for mechanics in the area. After choosing a mechanic, the user can make a request to that particular mechanic. The mechanic may register using all the correct data. Following registration, he has the ability to evaluate requests submitted by users and decide whether to approve or reject them.</p><h3>Sustainability of this project</h3><p>world’s population is growing daily, along with the number of automobiles being manufactured and the number of people purchasing them. and we are aware that the roads are dangerous to drive on. Nowadays, people enjoy taking road trips, but occasionally, a vehicle breakdown makes the trip uncomfortable for everyone involved. As we all know, memories are crucial when we embark on an outing with family or friends. In that scenario, our software will also be useful to individuals who require maintenance on uncharted routes at any moment. Therefore, our Programme will still be useful after so many years because we are aware that this issue would persist.</p><h3>UI of Mechanicus</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/527/1*TfKIFQo6Hhpt1WaJAwhPTg.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/530/1*Ips1hdAumSCGyEACoYjViw.jpeg" /></figure><h3>Research</h3><p>According to a recent check, 70 of the population in the country aren’t suitable to find a job indeed after having the applicable chops in them. Indeed after knowing a particular sphere, they do not know where to showcase their gift to earn a livelihood. Only the bones who are fat are honored indeed if they’ve smaller chops compared to the inadequately professed people. The poor sections of the society substantially workers are always suppressed by the big- scale and educated workers of the society. In the history, the following systems have formerly been erectedwww.gomechanics.inwww.getpitstop.comwww.kavak.com These are the notorious systems erected. Banning these, there are much further operations available in the request but none of the operation/ design available does the task that this operation does i.e., helping the weaker and poor sloggers of the society. Also, taking into account the miserable condition of roads in the country, there are high chances of vehicles getting breaking down in the middle of strange roads. So, the trip experience of numerous trippers gets miserable. In mountain areas during layoffs, trippers occasionally suffer auto battery issues, they fall when you did not anticipate them to. Another quite normal thing is a flat tire and who wants to replace a flat tire on its own when notoriety is traveling solo. There’s a veritably high probability of getting machine fails, the transmission fails, or perhaps your vehicle goes out of gas. To conquer this problem we came up with this operation, which is saving time and energy for our youth along with internal pressure. trippers can fluently communicate the service provider and track whether the service provider is on the way or not. The client will be handed with the identity of the service provider and24/7 client service will be available to the stoner in the future. So that they won’t get fear in such a situation where they got stuck on an strange road far from home and they enjoy the road trip and make good recollections and feel safe always and have faith in humanity.</p><h3>Potential of this App</h3><p>Everyone wants to be able to support themselves, hence this project has an excellent probability of succeeding as a start-up. The wealthy and large-scale corporations have long repressed the small-scale workers, causing them great suffering and making it difficult for them to survive in this cutthroat society. Their existence has long been disregarded in society. Applications that narrow the wealth gap between the rich and the poor are greatly needed. Since there will never be a solution to this issue, once our Programme becomes popular, people will still use it decades from now.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=347d857ed708" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>