<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Sibt-e-ali Baqar on Medium]]></title>
        <description><![CDATA[Stories by Sibt-e-ali Baqar on Medium]]></description>
        <link>https://medium.com/@sibteali786?source=rss-b07c6b861c5d------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*FeCZLku-6pVGDO0noacG_g.jpeg</url>
            <title>Stories by Sibt-e-ali Baqar on Medium</title>
            <link>https://medium.com/@sibteali786?source=rss-b07c6b861c5d------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Wed, 18 Nov 2026 21:34:30 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@sibteali786/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[My API was “fast on average” but users kept complaining. Turns out average was lying to me.]]></title>
            <link>https://medium.com/@sibteali786/my-api-was-fast-on-average-but-users-kept-complaining-turns-out-average-was-lying-to-me-4c6932fa42f5?source=rss-b07c6b861c5d------2</link>
            <guid isPermaLink="false">https://medium.com/p/4c6932fa42f5</guid>
            <category><![CDATA[p99]]></category>
            <category><![CDATA[performance]]></category>
            <category><![CDATA[latency]]></category>
            <category><![CDATA[learning]]></category>
            <category><![CDATA[backend]]></category>
            <dc:creator><![CDATA[Sibt-e-ali Baqar]]></dc:creator>
            <pubDate>Tue, 08 Sep 2026 11:37:56 GMT</pubDate>
            <atom:updated>2026-09-08T11:37:56.538Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/942/1*eWNcJ2IPTIGOSzNsa5lZgg.png" /></figure><p>I said “check the database” when an interviewer asked me how I would find a bottleneck in a system. Their eyebrows went up. Not the reaction I wanted and being interviewing for sometime this single expression destroys your whole confidence..Trust me when i say this.</p><p>That small moment told me there was a gap in my understanding, one that I had actually been carrying into production too, without realizing it. I had been staring at an “average latency: 68ms, looks healthy” dashboard while a chunk of real users were quietly getting 5 second responses when fetching relevant articles. The average was not lying exactly, but it was hiding the one number that actually mattered.</p><p>So instead of just reading a definition of P50 and P99, I reproduced the problem myself against my real Postgres database, dropped an index on purpose, and watched what broke. This article is that practical walkthrough.</p><h3>What P50 and P99 Actually Mean</h3><p>Percentiles describe the distribution of a set of measurements, and in backend engineering that set of measurements is almost always request latency.</p><ul><li><strong>P50 (median)</strong>: the point below which 50% of requests fall. Half your users experience this speed or better.</li><li><strong>P95</strong>: 95% of requests are faster than this. The remaining 5% are already showing friction.</li><li><strong>P99</strong>: 99% of requests are faster than this. Only 1% are slower, but that 1% is where things get ugly.</li><li><strong>P999</strong>: the true worst case. Usually caused by a single flaky node or shard.</li></ul><p>None of these are the average. That distinction matters more than it sounds.</p><h3>Why Not Just Use the Average?</h3><p>Averages are seductive because they compress a distribution into a single number, but they hide the exact thing you need to see: the tail.</p><p>Take 10,000 requests. 9,900 of them complete in 20ms. The remaining 100 take 5,000ms, maybe due to a lock, a cold cache, or a slow disk read.</p><p>The average latency here comes out to roughly 68ms. That number looks perfectly healthy on a dashboard. But the P99 latency is 5,000ms. At real-world scale, where a system serves millions of requests a day, that “1%” is not a rounding error, it is thousands of real users staring at a spinner.</p><p>This is the entire reason P50 and P99 exist as separate metrics: they let you see the shape of user experience, not just its center of mass.</p><h3>Why Interviewers Care About This Specifically</h3><p>When someone asks “how would you find the bottleneck,” a good answer is not a component name, it is a diagnostic lens. “Check the database” identifies a system, not a category of problem. P50 vs P99 is a category of problem. It tells you <em>where to even start looking</em> before you touch a single dashboard.</p><p>The distinction interviewers want to hear is this:</p><blockquote><em>If P50 itself is bad, the problem is structural, something every request pays for. If only P99 is bad while P50 stays fine, the problem is occasional, something a small fraction of requests happen to hit.</em></blockquote><p>That one sentence reframes “find the bottleneck” from a guessing game into a search with a clear first branch.</p><h3>Case 1: When P50 Is Bad, Structural Problems</h3><p>If the <em>typical</em> request is slow, the cause has to be something every request pays for. There is no lucky path to dodge it.</p><p>The classic example is a missing index. If a query filters on a column with no index, Postgres must scan a large portion of the table on every single execution, not just for some unlucky subset of calls. The result: P50, P95, and P99 all shift upward together, and the <em>spread</em> between them (P99 − P50) stays small. That tight spread, all shifted up, is the signature of a structural issue.</p><p>Structural culprits usually include:</p><ul><li>Missing or wrong index</li><li>Bad query plan (sequential scan instead of index scan)</li><li>N+1 queries</li><li>Missing composite index on a multi-column filter</li><li>Serialization/deserialization overhead in the hot path</li></ul><p>The tool to confirm this is EXPLAIN ANALYZE. If you see Seq Scan on a large table where you expected an Index Scan, you have found your structural bottleneck.</p><p>An example below shows a sequential scan.</p><pre>EXPLAIN (ANALYZE, BUFFERS)<br>SELECT count(*) FROM article WHERE region = &#39;US&#39;;<br>                                                                   QUERY PLAN<br>------------------------------------------------------------------------------------------------------------------------------------------------<br> Finalize Aggregate  (cost=221866.77..221866.78 rows=1 width=8) (actual time=1527.707..1542.792 rows=1 loops=1)<br>   Buffers: shared hit=66 read=166104<br>   -&gt;  Gather  (cost=221866.56..221866.77 rows=2 width=8) (actual time=1527.688..1542.775 rows=3 loops=1)<br>         Workers Planned: 2<br>         Workers Launched: 2<br>         Buffers: shared hit=66 read=166104<br>         -&gt;  Partial Aggregate  (cost=220866.56..220866.57 rows=1 width=8) (actual time=1515.379..1515.379 rows=1 loops=3)<br>               Buffers: shared hit=66 read=166104<br>               -&gt;  Parallel Seq Scan on article  (cost=0.00..218253.69 rows=1045146 width=0) (actual time=48.113..1468.311 rows=832986 loops=3)<br>                     Filter: (region = &#39;US&#39;::text)<br>                     Rows Removed by Filter: 2500371<br>                     Buffers: shared hit=66 read=166104<br> Planning:<br>   Buffers: shared hit=6 dirtied=1<br> Planning Time: 0.108 ms<br> JIT:<br>   Functions: 14<br>   Options: Inlining false, Optimization false, Expressions true, Deforming true<br>   Timing: Generation 2.012 ms (Deform 0.734 ms), Inlining 0.000 ms, Optimization 7.542 ms, Emission 135.920 ms, Total 145.474 ms<br> Execution Time: 1912.799 ms<br>(20 rows)</pre><h3>Case 2: When Only P99 Is Bad, Contention and Occasional Failures</h3><p>If P50 is fine but P99 spikes, the cause must be something that only affects <em>some</em> requests, and unpredictably so.</p><ol><li>Cause Why it only hits P99 Garbage collection pause Runs periodically, only requests coinciding with a pause get delayed</li><li>Connection pool exhaustion Pool has a fixed size, a burst of traffic makes some requests wait for a free connection</li><li>Cold cache miss A 99% cache hit rate means the 1% miss rate lines up almost exactly with your tail Slow shard / noisy neighbor.</li><li>Only requests routed to the one bad replica suffer Network retry Only the small percentage that hit a transient failure pay the extra round trip.</li></ol><p>There is a useful mental check here: the percentage of your tail should roughly match the percentage of requests hitting the “unlucky path.” If your cache hit rate is 99%, a bump around P99 is close to mathematically expected. If GC runs every 200 requests or so, this should show up around P99.5, not P50.</p><p>To actually confirm a tail-only issue, you need distributed tracing on the slow requests specifically, correlated against GC logs, connection pool metrics, or per-replica dashboards, not EXPLAIN ANALYZE, since the query plan itself is not the problem.</p><h3>Proving It Against a Real Database</h3><p>Theory is fine, but I wanted to see this happen with real numbers, so I ran an experiment against a Postgres table (article) with a few million rows, containing only primary key indexes to start.</p><p><strong>First attempt , inconclusive by design.</strong> I ran this query with and without a new composite index on (region, id DESC):</p><pre>EXPLAIN ANALYZE<br>SELECT a.id, a.title, a.url, p.name AS &quot;publisherName&quot;,<br>       a.category, a.region, a.published_at AS &quot;publishedAt&quot;<br>FROM article a<br>JOIN publisher p ON p.id = a.publisher_id<br>WHERE a.region = &#39;US&#39; AND a.id &lt; 500000<br>ORDER BY a.id DESC<br>LIMIT 20;</pre><p>The plan used article_pkey both times, ignoring the new index entirely.</p><p>The reason: ORDER BY id DESC LIMIT 20 lets Postgres walk the primary key index backward and stop the moment it collects 20 matching rows, it only had to skip 39 non-matching ones. That is cheap enough that the planner never considered switching. The execution time did drop from 1.14ms to 0.09ms between the two runs, but that had nothing to do with the index, it was Postgres&#39;s buffer cache warming up between runs. That, incidentally, is the exact &quot;cold cache&quot; mechanism from the P99 table above, just observed at the database buffer-pool level instead of an application cache like Redis.</p><p><strong>Second attempt, a fair test.</strong> I removed the LIMIT and the ORDER BY id, forcing the composite index to compete on its own merit for a query that has to touch every matching row:</p><pre>EXPLAIN (ANALYZE, BUFFERS)<br>SELECT count(*) FROM article WHERE region = &#39;US&#39;;</pre><p>With the composite index in place:</p><pre>Parallel Index Only Scan using idx_article_region_id on article<br>Heap Fetches: 0<br>Buffers: shared hit=20 read=9578<br>Execution Time: 213.755 ms</pre><p>After dropping the index:</p><pre>Parallel Seq Scan on article<br>Filter: (region = &#39;US&#39;::text)<br>Rows Removed by Filter: 2500371<br>Buffers: shared hit=66 read=166104<br>Execution Time: 1912.799 ms</pre><p>Dropping one index produced a <strong>~9x slowdown</strong> and a <strong>~17x increase</strong> in disk pages read. The index-backed version was also an <em>Index Only Scan, </em>Heap Fetches: 0 means Postgres never touched the actual table rows at all, because everything the query needed (region, and an implicit count) already lived inside the index structure itself.</p><p>This is the structural signature in its purest form: the problem is uniform, every single request pays it, and there is no “unlucky” subset, it is <em>all</em> of them.</p><h3>The One-Sentence Heuristic</h3><blockquote><em>Uniform problem across the whole distribution → look at the code path itself (query plan, index, algorithm). Badness only in the tail → look at what happens occasionally (contention, cache miss, GC, retries, one bad node).</em></blockquote><h3>Why This Matters Beyond the Interview Room</h3><p>At real scale, this distinction changes what you build. If a query like the sequential scan above (1.9 seconds) sits in a request path serving tens of millions of requests a day, per-request database filtering becomes non-viable regardless of how “correct” it is , this is exactly the kind of measured cost that pushes system designs toward pre-filtered, in-memory caches (like Redis) instead of relying on the database to filter per request.</p><p>The 213ms <em>good</em> case is already uncomfortably close to a typical latency budget for a feed-style endpoint, and that was just a count(*), not a full row fetch with a join.</p><p>Knowing whether you are chasing a structural (P50) problem or a contention (P99) problem is not a trivia fact for interviews. It is the first decision that determines whether you reach for EXPLAIN ANALYZE or for distributed tracing, and getting that first branch right is often the difference between fixing a bottleneck in ten minutes and chasing it for a week.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4c6932fa42f5" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Distributed Systems: Do you even need one ?]]></title>
            <link>https://medium.com/@sibteali786/distributed-systems-do-you-even-need-one-688780e86fd5?source=rss-b07c6b861c5d------2</link>
            <guid isPermaLink="false">https://medium.com/p/688780e86fd5</guid>
            <category><![CDATA[database]]></category>
            <category><![CDATA[distributed-systems]]></category>
            <category><![CDATA[consistency]]></category>
            <category><![CDATA[concurrency]]></category>
            <category><![CDATA[system-design-concepts]]></category>
            <dc:creator><![CDATA[Sibt-e-ali Baqar]]></dc:creator>
            <pubDate>Sat, 29 Aug 2026 20:31:43 GMT</pubDate>
            <atom:updated>2026-08-29T20:31:43.614Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*nzBHgwD9SOpQTZ8ERWp3HA.png" /></figure><blockquote>If you can solve your problem on a single computer, please do that. It’s always simpler.</blockquote><p>That’s not a throwaway line. It’s the first thing Robert Morris tells students on day one of MIT’s legendary 6.824 Distributed Systems course. And it matters, because distributed systems are not simpler, not easier to reason about, and not something one should reach for by default. A distributed system is what you end up building when a single machine, no matter how big, stops being enough.</p><p>So before diving into MapReduce, Raft, or sharded key-value stores, it’s worth sitting with the actual question: <strong>why do we need distributed systems at all? </strong>Because running around throwing servers ( because either your founder has recently raised Seed money or you do not really know what you are doing ) and databases to increase performance is gonna back fire soon.</p><h3>1. Performance: one machine runs out of gas</h3><p>Every computer has a ceiling: a fixed number of CPUs, a fixed amount of memory, a fixed number of disk arms moving data. Some problems are simply bigger than that ceiling.</p><p>Think about indexing the entire web. Even back in 2004, that was tens of terabytes of data. Sorting or analyzing that on a single machine wasn’t a “let it run overnight” problem — it was a “let it run for months” problem. The only way to get that work done in a reasonable time is to split it across hundreds or thousands of machines working in parallel, more CPUs, more disks, more memory, all moving at once.</p><p>This is the appeal of <strong>scalable speedup</strong>: if two machines could reliably do the work of one machine in half the time, you could buy your way to more performance instead of paying engineers to endlessly optimize code. That’s a genuinely powerful trade — money is cheaper than engineering time.</p><p><em>(Spoiler: this scalability story has a catch, and it’s a big one, more on that below.)</em></p><h3>2. Fault tolerance: machines die, work shouldn’t</h3><p>A single server can run for years without a hiccup. But once you have hundreds or thousands of servers, statistics stop being on your side. If each machine has just a 1-in-365 chance of failing on any given day, a fleet of 1,000 machines will see <strong>roughly three failures a da</strong>y, every single day.</p><p>At that scale, failure isn’t a rare event you patch around later, it’s the normal operating condition. Distributed systems exist partly to absorb this: run the same thing on two machines, and if one dies, the other keeps serving without anyone noticing.</p><h3>3. Physical necessity: some systems are already spread out</h3><p>Sometimes distribution isn’t a choice you make for performance , it’s forced on you by geography.</p><p>Picture an Inter-bank transfer: one bank’s system lives in New York, the other’s in Islamabad. There’s no version of this problem where you put both banks’ data on the same machine. The systems are physically separate because the organizations are physically separate, and they simply need a reliable way to talk to each other over a network.</p><h3>4. Security and isolation: not trusting the other side</h3><p>Sometimes you split a computation across machines not because of scale or geography, but because you don’t trust one side of it. Maybe it’s third-party code, maybe it’s a partner’s system that could have bugs or malicious intent. Rather than run everything in one trusted process, you isolate the pieces onto separate machines that only talk to each other through a narrow, well-defined protocol.</p><h3>So why not just do it anyway?</h3><p>Here’s the part that doesn’t get said enough: all four of these reasons come with a tax. Distributed systems introduce two problems that single machines simply don’t have:</p><ul><li><strong>Concurrency everywhere.</strong> Multiple machines doing things at once means all the classic headaches of concurrent programming, race conditions, ordering, timing-dependent bugs , except now spread across a network instead of contained within one process.</li><li><strong>Partial failure.</strong> A single computer basically has two states: working, or crashed. A distributed system can have <em>some</em> pieces working and others not, a server down, a network cable someone tripped over, a switch with a dead fan, while the rest of the system carries on, confused about what’s actually still alive.</li></ul><p>Even the big promise</p><blockquote>just add more machines, get proportionally more performance</blockquote><p>quietly breaks down. A classic example: you scale a website by adding more web servers in front of a single shared database. It works great, right up until the database itself becomes the bottleneck. At that point, adding a 101st web server buys you nothing. Fixing it means redesigning how your data is stored and split up, real engineering work, not a shopping trip.</p><h3>The real trade-off</h3><p>That’s the honest pitch for distributed systems: they’re not something we adopt because they’re elegant. They’re something you’re forced into once a single machine can’t handle the performance, reliability, or physical constraints of your problem, and once you’re forced into it, you inherit concurrency bugs, partial failures, and scalability limits as the price of entry.</p><p>Everything else, MapReduce, replication protocols like Raft, consistency models, sharding, exists to answer one question: <strong>given that we now have many unreliable, independently-failing machines instead of one reliable one, how do we make them behave like a single, well-behaved system anyway?</strong></p><p>That’s the real subject of distributed systems. Not</p><blockquote>“how do we use many computers” but “how do we hide the fact that we’re using many computers”</blockquote><p>from everyone who has to build on top of them.</p><p>I am learning about Distrubuted Systems from scratch so will regularly share about it here if you want to learn more do follow..</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=688780e86fd5" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[I Built a Garbage Collector in C, and It Made Me Understand Python]]></title>
            <link>https://medium.com/@sibteali786/i-built-a-garbage-collector-in-c-and-it-made-me-understand-python-b52007a9ecf1?source=rss-b07c6b861c5d------2</link>
            <guid isPermaLink="false">https://medium.com/p/b52007a9ecf1</guid>
            <category><![CDATA[coding]]></category>
            <category><![CDATA[learning]]></category>
            <category><![CDATA[language-learning]]></category>
            <category><![CDATA[garbage-collection]]></category>
            <dc:creator><![CDATA[Sibt-e-ali Baqar]]></dc:creator>
            <pubDate>Thu, 27 Aug 2026 10:05:05 GMT</pubDate>
            <atom:updated>2026-08-27T10:05:05.274Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*p8NQ_5ql1UjRz8_-YdhKTQ.png" /></figure><p>For the last several weeks I have been building a toy interpreter in C called Sneklang, as part of a “learning in public” project thanks to <a href="https://medium.com/u/2e9384677862">Lane Wagner</a>’s BootDev. Every object in the language, from integers to arrays, has to live somewhere in memory, and something has to decide when that memory gets freed. This week I got to the part where I had to build that something myself. Here is what I learned, in the order I learned it.</p><h3>Every value needs to be the same shape</h3><p>Before you can garbage collect anything, you need a uniform way to represent every kind of value in your language. Sneklang does this with one struct:</p><pre>typedef struct SnekObject {<br>  snek_object_kind_t kind;<br>  snek_object_data_t data;<br>} snek_object_t;</pre><p>kind is an enum (INTEGER, FLOAT, STRING, VECTOR3, ARRAY), and data is a union holding whichever value actually applies. Every function that operates on Sneklang values, addition, length, array access, works generically off this struct by switching on kind.</p><p>This uniformity matters more than it looks like it does. Take VECTOR3:</p><pre>typedef struct {<br>  snek_object_t *x;<br>  snek_object_t *y;<br>  snek_object_t *z;<br>} snek_vector_t;</pre><p>The components are snek_object_t *, not raw floats. That is deliberate. A vector&#39;s x component needs to be a full object in its own right, because it could be an integer or a float, and because it lets functions like snek_add work recursively:</p><pre>if (a-&gt;kind == VECTOR3) {<br>  if (b-&gt;kind != VECTOR3) return NULL;<br>  snek_object_t *x = snek_add(a-&gt;data.v_vector3.x, b-&gt;data.v_vector3.x);<br>  snek_object_t *y = snek_add(a-&gt;data.v_vector3.y, b-&gt;data.v_vector3.y);<br>  snek_object_t *z = snek_add(a-&gt;data.v_vector3.z, b-&gt;data.v_vector3.z);<br>  return new_snek_vector3(x, y, z);<br>}</pre><p>Adding two vectors is just recursively adding their components. That recursion only works because everything, all the way down, is a snek_object_t *. Arrays follow the same idea, they hold snek_object_t **elements, so an array can hold anything, including other arrays.</p><p>The consequence of this design: every single value in the language lives on the heap, referenced by pointer. Which means every single value needs to eventually be freed. Which means I needed a garbage collector.</p><h3>Back to what a Garbage Collector actually is !!</h3><p>Before writing one, I had a fairly hand-wavy idea of what “garbage collector” meant. Turns out the concept is simpler than the name suggests, and the core question it answers is always the same one: is this value still live?</p><blockquote>Live means the value could still possibly be used by the program. If a variable was created inside a function and the function has already returned without anyone else referencing it, that variable is no longer live. Nothing in the program can reach it anymore. It is dead weight, and its memory should go back to the operating system.</blockquote><p>Different languages answer “is this live” in different ways, and where they choose to pay for that answer is one of the more interesting tradeoffs in language design:</p><ul><li><strong>C</strong> makes the programmer answer it, by hand, every time, forever. You call free() when you are sure nothing needs the value anymore. Get it wrong and you leak memory or, worse, free something too early and corrupt your program.</li><li><strong>Rust</strong> has the compiler answer it at compile time, via the borrow checker. You still do a form of manual memory management, but the compiler verifies your lifetimes are correct and inserts the frees for you. The cost is paid during development, as friction with the compiler.</li><li><strong>Garbage collected languages</strong> (Python, JavaScript, Go) have the runtime answer it automatically, while your program is running. This is not free. Some algorithm has to do real work to determine liveness, and that work happens at runtime, which means it costs real CPU cycles your program could otherwise be using.</li></ul><p>That last point undercuts a common misconception: garbage collection is not “no cost,” it is “cost deferred to runtime instead of development time.” You are always paying for memory safety somehow. The question is only when.</p><h3>Reference counting: the algorithm I actually built</h3><p>Sneklang uses the simplest of the common GC algorithms: reference counting. The idea is four rules:</p><ul><li>Every object keeps track of a refcount integer.</li><li>When an object gains a new reference, refcount increments.</li><li>When an object is garbage collected, everything <em>it</em> references gets decremented too.</li><li>When any object’s refcount reaches zero, it gets freed.</li></ul><p>The allocation side needed a small but important change. Instead of using malloc, which leaves memory uninitialized, I switched to calloc:</p><pre>snek_object_t *obj = calloc(1, sizeof(snek_object_t));<br>if (obj == NULL) {<br>  return NULL;<br>}<br>obj-&gt;refcount = 1;<br>return obj;</pre><p>calloc(1, sizeof(snek_object_t)) zeroes every byte of the struct before I touch it, so refcount, kind, and data all start at a known value instead of garbage. malloc would have required manually zeroing each field, which is exactly the kind of step that is easy to forget once and expensive to debug later.</p><p>The decrement side is where the interesting bug lived. Here is refcount_dec and refcount_free:</p><pre>void refcount_dec(snek_object_t *obj) {<br>  if (obj == NULL) {<br>    return;<br>  }<br>  obj-&gt;refcount--;<br>  if (obj-&gt;refcount == 0) {<br>    refcount_free(obj);<br>  }<br>}<br><br>void refcount_free(snek_object_t *obj) {<br>  if (obj-&gt;kind == INTEGER || obj-&gt;kind == FLOAT) {<br>    free(obj);<br>  } else if (obj-&gt;kind == STRING) {<br>    free(obj-&gt;data.v_string);<br>    free(obj);<br>  }<br>}</pre><p>The INTEGER/FLOAT case is one line because the value lives directly inside the object. One allocation, one free.</p><p>The STRING case needs two frees, in a specific order, and this is the part that tripped me up. obj-&gt;data.v_string is a char *, which is a <em>separate</em> heap allocation created back when the string object was built:</p><pre>snek_object_t *new_snek_string(char *value) {<br>  snek_object_t *obj = malloc(sizeof(snek_object_t));<br>  int len = strlen(value);<br>  char *dst = malloc(len + 1);   // separate allocation<br>  strcpy(dst, value);<br>  obj-&gt;data.v_string = dst;<br>  return obj;<br>}</pre><p>A string object is two mallocs, not one: the struct itself, and the character buffer it points to. If you free the struct first, you lose the only pointer to that buffer, and it becomes unreachable, unfreeable memory. A leak with extra steps. So the rule is: free the thing pointed <em>to</em> before you free the pointer that holds it.</p><p>This is a tiny detail, but it is exactly the kind of bug that will not crash your program and will not show up in a quick test run. It just quietly eats memory forever until someone runs valgrind and starts questioning their life choices.</p><h3>Where reference counting falls apart</h3><p>Refcounting has an honest, well known weakness: cycles. If object A references object B, and B references back to A, and nothing else in the program points to either of them, neither refcount will ever reach zero. From the refcount’s perspective, both objects are still “in use,” because they are, just by each other. They leak forever, silently, with no error and no warning.</p><p>This matters for Sneklang specifically. Right now I only have refcounting implemented for INTEGER, FLOAT, and STRING, kinds that cannot reference other objects. The moment I extend it to VECTOR3 and ARRAY, which absolutely can reference each other, I am one poorly-designed data structure away from a cycle that leaks indefinitely.</p><h3>The alternative: Mark and Sweep</h3><p>The other major GC algorithm, mark and sweep, solves the cycle problem by asking a fundamentally different question. Instead of “how many things point at me,” it asks “can I be reached from something that matters.”</p><p>The algorithm, roughly:</p><ul><li>Start from the roots, generally your stack frames and global variables, the things a running program can directly access.</li><li>Trace every reference outward from the roots, marking everything you can reach as alive.</li><li>Anything left unmarked once tracing is done gets swept, freed, and returned to the operating system.</li></ul><p>Two consequences fall out of this naturally. First, cycles stop being special: if A and B reference each other but nothing reachable from a root points into that pair, neither gets marked, and both get swept regardless of how tightly they reference each other. Second, the cost model is completely different. Refcounting pays a small cost on every single reference change, everywhere, all the time. Mark and sweep pays a larger cost, but only occasionally, during a dedicated “GC pause” where the whole program stops while the collector walks the graph.</p><p>Neither algorithm is strictly better. Refcounting frees memory immediately and predictably, which matters for things like large objects or scarce resources, but leaks on cycles and adds overhead to every operation. Mark and sweep handles cycles correctly and has near-zero overhead between pauses, but introduces pause times that can be noticeable in latency sensitive programs, which is why production collectors (V8’s, for instance) tend to use hybrid strategies rather than either algorithm in its pure form.</p><h3>Where this leaves Sneklang</h3><p>Right now, Sneklang is committed to pure reference counting, and for INTEGER, FLOAT, and STRING that is entirely sufficient, none of them can form a cycle. The real decision point is coming up fast: once VECTOR3 and ARRAY get refcounting support, I will have to either accept that cycles are a known limitation of the language, or bolt on some form of cycle detection on top of the refcounting I already have, which is roughly what CPython actually does under the hood.</p><p>Small, seemingly cosmetic bug today, real architectural decision tomorrow. That is a pattern I am learning to expect from systems programming: the thing that looks like a one line fix is usually a preview of a design decision you have not consciously made yet.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b52007a9ecf1" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[My database queries for writing tasks was becoming slow as time progressed ? but why]]></title>
            <link>https://medium.com/@sibteali786/my-database-queries-for-writing-tasks-was-becoming-slow-as-time-progressed-but-why-ef58e14b0ade?source=rss-b07c6b861c5d------2</link>
            <guid isPermaLink="false">https://medium.com/p/ef58e14b0ade</guid>
            <category><![CDATA[database]]></category>
            <category><![CDATA[systems-thinking]]></category>
            <category><![CDATA[learning]]></category>
            <category><![CDATA[postgresql]]></category>
            <dc:creator><![CDATA[Sibt-e-ali Baqar]]></dc:creator>
            <pubDate>Fri, 21 Aug 2026 20:11:55 GMT</pubDate>
            <atom:updated>2026-08-21T20:11:55.941Z</atom:updated>
            <content:encoded><![CDATA[<h3>The Moment of confusion</h3><p>I have been using UUID myself for years in my career as a Software Engineer and honestly until recently i never ever had an idea that they might be killing writes performance in my database and at first as usual i was just confused with no idea to what was happening.</p><p>I started reading about Indexing and how it works to better dig deeper into what was actually happening with writes. I discovered that postgreSQL uses B+trees for storing indexes where leaf nodes are interconnected ( Doubly Linked Lists ) so that a range query does not have move up in the tree levels.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Ls895xiOunCydmYs.png" /></figure><p>B+trees structure</p><p>For reference from <a href="https://medium.com/@akashsdas_dev/b-trees-and-b-trees-682d363df1f7">medium</a> article about difference between B Tree and B+ tree. And it perfectly shows how efficient indexes are really but still if we are using UUID its not gonna matter alot, but why ?</p><ol><li>B+tree or B-tree work on the fact that if items inserted are ordered it will be easy to find their relevant node and fill the page easily</li><li>If its not ordered we might end up splitting the node itself and have perform several IO operations to even store a single new value.</li><li>Now imagine it happening at 1000s of times/sec 💀.</li></ol><h3>Understanding the problem</h3><p>UUID are 128 bit random strings where two UUIDs generated one after another have no order and you cannot tell which one was created first.</p><p>Let’s understand it using a simple example we have some values ( actually UUIDs but for understanding )</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*QdgQC3e3_7KIdyHp.png" /></figure><p>If we try to put these one by one B+tree ( leaf nodes ) in the given page of size 2 ( can hold at most 2 values ) it will go like this</p><p><strong>Step 1 — Insert 30</strong><br>Page A: [30]</p><p><strong>Step 2 — Insert 10</strong><br>10 &lt; 30, fits in same page (still has room).<br>Page A: [10, 30]</p><p><strong>Step 3 — Insert 5</strong><br>5 belongs before 10, but Page A is already full (2/2).<br>→ <strong>Split</strong>: combine [5, 10, 30], split into two pages.<br>Page A: [5, 10] → Page B: [30]<br>(Page A links to Page B)</p><p><strong>Step 4 — Insert 90</strong><br>90 &gt; 30, belongs in Page B, which has room.<br>Page B: [30, 90]</p><p><strong>Step 5 — Insert 40</strong><br>40 belongs between 30 and 90, but Page B is full (2/2).<br>→ <strong>Split</strong>: combine [30, 40, 90], split into two pages.<br>Page B: [30, 40] → Page C: [90]<br>(Page B links to Page C)</p><p><strong>Final state (leaf pages, linked in order):</strong><br>[5, 10] → [30, 40] → [90]</p><p>2 splits occurred out of 5 inserts, for only 5 values.</p><p>Oops !! not looks good honestly and imagine doing this every-time you make a write operation. So what is the solution ?</p><h3>Senpae Shopify to the rescue</h3><p>Similar issue was faced by shopify and guess what they shifted to a different form of unique ids that has randomness and uniquness but its trackable based on a sepcific attrbiyute namely → timestamp.</p><p>Two ULIDs generated one after another will be sequential ( always ) in terms of timestamp so that it can be identified one is created after another.</p><p>So when you have ULIDs its like this and each time you make a write it will be like writing sequentially ( no need to do ninja stuff or moving up, down, left and right the tree for splitting )</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*1eFlBObiDzcY-8jo.png" /></figure><h3>Conclusion</h3><p>This is one of the countless reasons i love backend engineering, everytime i read and learn about such stuff it amazes me and at same time humbles me enough that i know too little and there is a long journey ahead of me still. Hope you enjoyed reading it and learned something new today.</p><h3>Wait, where did my problem went ?</h3><p>Well to be honest, my use-case could still work fine since it had only 100 users 😂 but exploring this issues taught me something that i can definitely use in future when the moment calls for it.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ef58e14b0ade" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How I Fixed the MultiSelect Not Working Inside Dialog — A Radix UI Version Mismatch]]></title>
            <link>https://medium.com/@sibteali786/how-i-fixed-the-multiselect-not-working-inside-dialog-a-radix-ui-version-mismatch-3158104d994a?source=rss-b07c6b861c5d------2</link>
            <guid isPermaLink="false">https://medium.com/p/3158104d994a</guid>
            <category><![CDATA[multiselect]]></category>
            <category><![CDATA[chat]]></category>
            <category><![CDATA[react]]></category>
            <category><![CDATA[shadcn]]></category>
            <dc:creator><![CDATA[Sibt-e-ali Baqar]]></dc:creator>
            <pubDate>Wed, 14 May 2025 11:15:38 GMT</pubDate>
            <atom:updated>2025-05-14T11:15:38.230Z</atom:updated>
            <content:encoded><![CDATA[<h3>How I Fixed the MultiSelect Not Working Inside Dialog — A Radix UI Version Mismatch</h3><p>Have you ever spent hours debugging why your MultiSelect component won’t work inside a Dialog? The dropdown opens, but clicking on items does nothing, and if you click outside the dialog area, it just closes. I’ve been there, and here’s how I finally solved it.</p><h3>The Problem</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DsIg3fj1EqPiQB34n4weBg.png" /></figure><p>I was building a chat application with a “Create Channel” dialog that contained a MultiSelect component for choosing channel members. Everything looked fine visually, but the MultiSelect was completely unresponsive inside the Dialog.</p><p>Here’s what was happening:</p><ul><li>The MultiSelect dropdown would open</li><li>Clicking on items did nothing</li><li>Clicking outside the dialog would close the entire dialog</li><li>The z-index seemed correct, but interactions were broken</li></ul><pre>// The problematic setup<br>&lt;Dialog open={isOpen} onOpenChange={handleOpenChange}&gt;<br>  &lt;DialogContent&gt;<br>    &lt;MultiSelect <br>      options={userOptions}<br>      selected={selectedUsers}<br>      onChange={handleUserSelection}<br>    /&gt;<br>  &lt;/DialogContent&gt;<br>&lt;/Dialog&gt;</pre><h3>The Investigation</h3><p>My first instinct was to check z-index issues. I tried:</p><pre>.popover-content {<br>  z-index: 1000 !important;<br>}</pre><p>No luck. Then I tried event propagation fixes:</p><pre>&lt;PopoverContent <br>  onPointerDownOutside={(e) =&gt; e.preventDefault()}<br>  onInteractOutside={(e) =&gt; e.preventDefault()}<br>&gt;</pre><p>Still nothing. The components looked right, but something deeper was wrong.</p><h3>The Discovery</h3><p>After diving into GitHub issues, I found the culprit: <strong>Radix UI package version mismatches</strong>. Different Radix UI components were using different versions of shared internal dependencies, causing interaction conflicts.</p><p>Here’s what my package.json looked like before:</p><pre>{<br>  &quot;dependencies&quot;: {<br>    &quot;@radix-ui/react-dialog&quot;: &quot;^1.1.13&quot;,<br>    &quot;@radix-ui/react-dropdown-menu&quot;: &quot;^2.1.6&quot;,<br>    &quot;@radix-ui/react-popover&quot;: &quot;^1.1.6&quot;,<br>    &quot;@radix-ui/react-select&quot;: &quot;^2.1.6&quot;<br>  }<br>}</pre><p>The issue? These packages share internal dependencies like @radix-ui/react-dismissable-layer and @radix-ui/react-focus-scope, and version mismatches break their interaction logic.</p><h3>The Solution</h3><p>The fix requires aligning all Radix UI packages to compatible versions:</p><ol><li>First, update your package.json with compatible versions:</li></ol><pre>{<br>  &quot;dependencies&quot;: {<br>    &quot;@radix-ui/react-dialog&quot;: &quot;1.1.13&quot;,<br>    &quot;@radix-ui/react-dropdown-menu&quot;: &quot;2.1.6&quot;,<br>    &quot;@radix-ui/react-select&quot;: &quot;2.1.6&quot;,<br>    &quot;@radix-ui/react-popover&quot;: &quot;1.1.6&quot;,<br>    &quot;@radix-ui/react-alert-dialog&quot;: &quot;1.1.11&quot;,<br>    &quot;cmdk&quot;: &quot;1.0.4&quot;<br>  }<br>}</pre><ol><li>Add overrides to force consistent shared dependencies:</li></ol><pre>{<br>  &quot;overrides&quot;: {<br>    &quot;@radix-ui/react-dismissable-layer&quot;: &quot;1.1.8&quot;,<br>    &quot;@radix-ui/react-focus-scope&quot;: &quot;1.1.2&quot;,<br>    &quot;@radix-ui/react-portal&quot;: &quot;1.1.8&quot;<br>  }<br>}</pre><ol><li>Clean install everything:</li></ol><pre>rm -rf node_modules package-lock.json<br>npm install</pre><h3>Why This Works</h3><p>Radix UI components share several internal packages for handling:</p><ul><li>Focus management (react-focus-scope)</li><li>Click outside detection (react-dismissable-layer)</li><li>Portal rendering (react-portal)</li></ul><p>When these shared dependencies have version conflicts, components can’t communicate properly. The Dialog thinks it owns the interaction layer, while the MultiSelect’s Popover tries to create its own, resulting in broken interactions.</p><h3>Key Takeaways</h3><ol><li><strong>Always check package versions</strong> when Radix UI components don’t interact correctly</li><li><strong>Use exact versions</strong> (not caret ^ ranges) for better consistency</li><li><strong>Add overrides</strong> for shared internal dependencies</li><li><strong>Clean install</strong> after version changes to ensure proper resolution</li></ol><h3>Working Example</h3><p>After the fix, your MultiSelect will work seamlessly inside Dialogs:</p><pre>&lt;Dialog open={isOpen} onOpenChange={setIsOpen}&gt;<br>  &lt;DialogContent className=&quot;sm:max-w-[550px]&quot;&gt;<br>    &lt;DialogHeader&gt;<br>      &lt;DialogTitle&gt;Create New Channel&lt;/DialogTitle&gt;<br>    &lt;/DialogHeader&gt;<br>    <br>    &lt;MultiSelect <br>      options={users.map(user =&gt; ({<br>        label: user.displayName,<br>        value: user._id<br>      }))}<br>      selected={selectedMembers}<br>      onChange={setSelectedMembers}<br>      placeholder=&quot;Select members...&quot;<br>    /&gt;<br>    <br>    &lt;DialogFooter&gt;<br>      &lt;Button onClick={handleCreate}&gt;Create Channel&lt;/Button&gt;<br>    &lt;/DialogFooter&gt;<br>  &lt;/DialogContent&gt;<br>&lt;/Dialog&gt;</pre><p>Now the MultiSelect works perfectly — items are selectable, the dialog stays open, and everything interacts as expected.</p><h3>Conclusion</h3><p>What seemed like a complex z-index or event propagation issue turned out to be a simple version mismatch problem. Sometimes the solution isn’t in your code — it’s in your dependencies. When working with component libraries, especially ones with shared internal packages, keeping versions aligned is crucial.</p><p>Hope this saves you the hours of debugging I went through!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3158104d994a" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Transaction Isolation in Databases]]></title>
            <link>https://medium.com/@sibteali786/transaction-isolation-in-databases-5f40f2995dd3?source=rss-b07c6b861c5d------2</link>
            <guid isPermaLink="false">https://medium.com/p/5f40f2995dd3</guid>
            <category><![CDATA[efficiency]]></category>
            <category><![CDATA[database]]></category>
            <category><![CDATA[postgresql]]></category>
            <dc:creator><![CDATA[Sibt-e-ali Baqar]]></dc:creator>
            <pubDate>Fri, 21 Feb 2025 08:46:14 GMT</pubDate>
            <atom:updated>2025-02-21T08:46:14.494Z</atom:updated>
            <content:encoded><![CDATA[<p>As promised today I&#39;ll share my learnings for the Database Engineering topic and the next important concept in line is Isolation. As usual, the possible things you will discover in this article are</p><ol><li>What is Isolation?</li><li>Why it&#39;s needed?</li><li>Read phenomena and their different Types with examples.</li><li>Isolation Levels</li></ol><h3>What is Isolation</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*1FlAztlIX5Y3XvQ6" /><figcaption>Photo by <a href="https://unsplash.com/@sashafreemind?utm_source=medium&amp;utm_medium=referral">Sasha Freemind</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>Well, the picture is not exactly what I intend to explain here :&gt; anyway</p><blockquote><em>To keep a transaction in a locked, snapshotted state so other on-going or future transactions cannot interfere with the content its dealing with.</em></blockquote><p>Well, it&#39;s the easiest one I could mold into words. Still, it means that when billions of users are making changes to the same data concurrently, how does a database keep things consistent such that no wrong information is conveyed to any one user out of a billion users? <strong><em>Isolation</em></strong> is the simplest and easiest answer to this. <strong><em>Isolation</em></strong> has many underlying concepts and things behind the scenes to make it happen.</p><p>There is a basic question that arises out of <strong><em>Isolation</em></strong></p><p><em>Can my ongoing transaction see changes made by other transactions?</em></p><p>whose answer leads to <em>Read Phemonas </em>which will be explained in detail in upcoming paragraphs in this article.</p><h3>Why is it Needed?</h3><p>We will explain the <strong>why</strong> by explaining and going through each <em>Read Phenomena.</em></p><h3>Read Phenomanas</h3><p>There are different types of <em>Read Phenomena </em>in Databases brought about by a lack of Isolation level implementations. Let’s go through each using the same example of <strong><em>Sales</em></strong> so we can understand how it works.</p><h4>Dirty Reads</h4><blockquote>Reading an entry which has been updated by another transaction but has not been committed to disk yet.</blockquote><p>This happens when we are not using transactions and ultimately no isolation levels are applied. It can lead to inconsistent data because of</p><ol><li>The database crashed before any data was committed to the physical disk.</li><li>The change was rolled back.</li></ol><p>Let&#39;s go over an example with a real-world scenario along with practical coding so you may understand it more clearly.</p><p>Let’s suppose we have a <em>sales </em>table with <em>name</em> and <em>balance</em> columns.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fcarbon.now.sh%2Fembed%3Fbg%3Drgba%2528171%252C%2B184%252C%2B195%252C%2B1%2529%26t%3Dseti%26wt%3Dnone%26l%3Dsql%26width%3D680%26ds%3Dtrue%26dsyoff%3D20px%26dsblur%3D68px%26wc%3Dtrue%26wa%3Dtrue%26pv%3D56px%26ph%3D56px%26ln%3Dfalse%26fl%3D1%26fm%3DHack%26fs%3D14px%26lh%3D133%2525%26si%3Dfalse%26es%3D1x%26wm%3Dfalse%26code%3DCREATE%252520TABLE%252520sales%252520%2528%252520%25250Apid%252520SERIAL%252520PRIMARY%252520KEY%25252C%25250Aprice%252520FLOAT%25252C%25250Aquantity%252520INT%252520%25250A%2529%25253B%25250A%25250AINSERT%252520INTO%252520sales%252520%2528%25250A%252520%252520price%25252C%252520quantity%25250A%2529%252520VALUES%25250A%252810%25252C5%2529%25252C%25250A%252820%25252C4%2529%25253B&amp;display_name=Carbon&amp;url=https%3A%2F%2Fcarbon.now.sh%2F%3Fbg%3Drgba%252528171%25252C%2B184%25252C%2B195%25252C%2B1%252529%26t%3Dseti%26wt%3Dnone%26l%3Dsql%26width%3D680%26ds%3Dtrue%26dsyoff%3D20px%26dsblur%3D68px%26wc%3Dtrue%26wa%3Dtrue%26pv%3D56px%26ph%3D56px%26ln%3Dfalse%26fl%3D1%26fm%3DHack%26fs%3D14px%26lh%3D133%252525%26si%3Dfalse%26es%3D1x%26wm%3Dfalse%26code%3DCREATE%25252520TABLE%25252520sales%25252520%252528%25252520%2525250Apid%25252520SERIAL%25252520PRIMARY%25252520KEY%2525252C%2525250Aprice%25252520FLOAT%2525252C%2525250Aquantity%25252520INT%25252520%2525250A%252529%2525253B%2525250A%2525250AINSERT%25252520INTO%25252520sales%25252520%252528%2525250A%25252520%25252520price%2525252C%25252520quantity%2525250A%252529%25252520VALUES%2525250A%25252810%2525252C5%252529%2525252C%2525250A%25252820%2525252C4%252529%2525253B&amp;image=https%3A%2F%2Fcarbon.now.sh%2Fstatic%2Fbrand%2Fbanner.png&amp;type=text%2Fhtml&amp;scroll=auto&amp;schema=carbon" width="1024" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/c7be6217e988c4fb8580e9b92af979fb/href">https://medium.com/media/c7be6217e988c4fb8580e9b92af979fb/href</a></iframe><p>The operations manager wants to get the <em>total </em>amount of sales made over the history of products.</p><p>Employee<em> </em><strong><em>A</em></strong> starts working on this report in the database and runs a SELECT query for observing results to see the overall values in sales for each product.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fcarbon.now.sh%2Fembed%3Fbg%3Drgba%2528171%252C%2B184%252C%2B195%252C%2B1%2529%26t%3Dseti%26wt%3Dnone%26l%3Dsql%26width%3D680%26ds%3Dtrue%26dsyoff%3D20px%26dsblur%3D68px%26wc%3Dtrue%26wa%3Dtrue%26pv%3D56px%26ph%3D56px%26ln%3Dfalse%26fl%3D1%26fm%3DHack%26fs%3D14px%26lh%3D133%2525%26si%3Dfalse%26es%3D1x%26wm%3Dfalse%26code%3DSELECT%252520pid%25252C%252520quantity%252Aprice%252520FROM%252520sales&amp;display_name=Carbon&amp;url=https%3A%2F%2Fcarbon.now.sh%2F%3Fbg%3Drgba%252528171%25252C%2B184%25252C%2B195%25252C%2B1%252529%26t%3Dseti%26wt%3Dnone%26l%3Dsql%26width%3D680%26ds%3Dtrue%26dsyoff%3D20px%26dsblur%3D68px%26wc%3Dtrue%26wa%3Dtrue%26pv%3D56px%26ph%3D56px%26ln%3Dfalse%26fl%3D1%26fm%3DHack%26fs%3D14px%26lh%3D133%252525%26si%3Dfalse%26es%3D1x%26wm%3Dfalse%26code%3DSELECT%25252520pid%2525252C%25252520quantity%25252Aprice%25252520FROM%25252520sales&amp;image=https%3A%2F%2Fcarbon.now.sh%2Fstatic%2Fbrand%2Fbanner.png&amp;type=text%2Fhtml&amp;scroll=auto&amp;schema=carbon" width="1024" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/d80692b205e6f3a08242a9aed9f3bc63/href">https://medium.com/media/d80692b205e6f3a08242a9aed9f3bc63/href</a></iframe><p>the result is</p><p>pid | ?column?<br> — — -+ — — — — — <br> 1 | 50<br> 2 | 80<br>(2 rows)</p><p>At the same time, Employee <strong><em>B </em></strong>records an update to product quantity and updates the database by running an update query for PID = 1</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fcarbon.now.sh%2Fembed%3Fbg%3Drgba%2528171%252C%2B184%252C%2B195%252C%2B1%2529%26t%3Dseti%26wt%3Dnone%26l%3Dsql%26width%3D680%26ds%3Dtrue%26dsyoff%3D20px%26dsblur%3D68px%26wc%3Dtrue%26wa%3Dtrue%26pv%3D56px%26ph%3D56px%26ln%3Dfalse%26fl%3D1%26fm%3DHack%26fs%3D14px%26lh%3D133%2525%26si%3Dfalse%26es%3D1x%26wm%3Dfalse%26code%3DUPDATE%252520sales%252520SET%252520quantity%25253Dquantity%25252B5%252520WHERE%252520PID%252520%25253D%2525201&amp;display_name=Carbon&amp;url=https%3A%2F%2Fcarbon.now.sh%2F%3Fbg%3Drgba%252528171%25252C%2B184%25252C%2B195%25252C%2B1%252529%26t%3Dseti%26wt%3Dnone%26l%3Dsql%26width%3D680%26ds%3Dtrue%26dsyoff%3D20px%26dsblur%3D68px%26wc%3Dtrue%26wa%3Dtrue%26pv%3D56px%26ph%3D56px%26ln%3Dfalse%26fl%3D1%26fm%3DHack%26fs%3D14px%26lh%3D133%252525%26si%3Dfalse%26es%3D1x%26wm%3Dfalse%26code%3DUPDATE%25252520sales%25252520SET%25252520quantity%2525253Dquantity%2525252B5%25252520WHERE%25252520PID%25252520%2525253D%252525201&amp;image=https%3A%2F%2Fcarbon.now.sh%2Fstatic%2Fbrand%2Fbanner.png&amp;type=text%2Fhtml&amp;scroll=auto&amp;schema=carbon" width="1024" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/1b6b4b651736c0c06e76e3d0bc67e199/href">https://medium.com/media/1b6b4b651736c0c06e76e3d0bc67e199/href</a></iframe><p>This in turn changes the product 1 quantity from 10 to 15.</p><p>Now when Employee <strong><em>A </em></strong>runs the aggregation query on Sales he gets the wrong result of the total amount of sales made based on the SELECT query result he had seen previously.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fcarbon.now.sh%2Fembed%3Fbg%3Drgba%2528171%252C%2B184%252C%2B195%252C%2B1%2529%26t%3Dseti%26wt%3Dnone%26l%3Dsql%26width%3D680%26ds%3Dtrue%26dsyoff%3D20px%26dsblur%3D68px%26wc%3Dtrue%26wa%3Dtrue%26pv%3D56px%26ph%3D56px%26ln%3Dfalse%26fl%3D1%26fm%3DHack%26fs%3D14px%26lh%3D133%2525%26si%3Dfalse%26es%3D1x%26wm%3Dfalse%26code%3DSELECT%252520SUM%2528quanity%252Aprice%2529%252520FROM%252520sales&amp;display_name=Carbon&amp;url=https%3A%2F%2Fcarbon.now.sh%2F%3Fbg%3Drgba%252528171%25252C%2B184%25252C%2B195%25252C%2B1%252529%26t%3Dseti%26wt%3Dnone%26l%3Dsql%26width%3D680%26ds%3Dtrue%26dsyoff%3D20px%26dsblur%3D68px%26wc%3Dtrue%26wa%3Dtrue%26pv%3D56px%26ph%3D56px%26ln%3Dfalse%26fl%3D1%26fm%3DHack%26fs%3D14px%26lh%3D133%252525%26si%3Dfalse%26es%3D1x%26wm%3Dfalse%26code%3DSELECT%25252520SUM%252528quanity%25252Aprice%252529%25252520FROM%25252520sales&amp;image=https%3A%2F%2Fcarbon.now.sh%2Fstatic%2Fbrand%2Fbanner.png&amp;type=text%2Fhtml&amp;scroll=auto&amp;schema=carbon" width="1024" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/d721c33fd86c3012cd511f3567fd3d20/href">https://medium.com/media/d721c33fd86c3012cd511f3567fd3d20/href</a></iframe><p>sum<br> — — -<br> 155<br>(1 row)</p><p>which confuses him/her as previously he/she expected the total sales to be around <strong>$130</strong> but now it&#39;s <strong>$155</strong> because he did not know that employee B made an update to the <strong>Sales</strong> table just when he was running his queries.</p><p>From this scenario it&#39;s pretty clear what issues and disasters lack of proper <strong><em>Isolation</em></strong> implementation can lead to, while this was only an example of a <strong><em>Dirty Read.</em></strong></p><h4>Non-Repeatable Reads</h4><blockquote>Occurs when a transaction re-reads the <strong>same row </strong>and finds that data has been <strong>deleted</strong> or <strong>modified</strong> by another committed transaction.</blockquote><p>This happens because the <strong>row</strong> was <strong>updated</strong> after the initial read. The key causes for this read are <strong>UPDATE</strong> and <strong>DELETE</strong> queries</p><p>Let’s understand it using the same <strong><em>sales</em></strong> example</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fcarbon.now.sh%2Fembed%3Fbg%3Drgba%2528171%252C%2B184%252C%2B195%252C%2B1%2529%26t%3Dseti%26wt%3Dnone%26l%3Dsql%26width%3D680%26ds%3Dtrue%26dsyoff%3D20px%26dsblur%3D68px%26wc%3Dtrue%26wa%3Dtrue%26pv%3D56px%26ph%3D56px%26ln%3Dfalse%26fl%3D1%26fm%3DHack%26fs%3D14px%26lh%3D133%2525%26si%3Dfalse%26es%3D1x%26wm%3Dfalse%26code%3DCREATE%252520TABLE%252520sales%252520%2528%252520%25250Apid%252520SERIAL%252520PRIMARY%252520KEY%25252C%25250Aprice%252520FLOAT%25252C%25250Aquantity%252520INT%252520%25250A%2529%25253B%25250A%25250AINSERT%252520INTO%252520sales%252520%2528%25250A%252520%252520price%25252C%252520quantity%25250A%2529%252520VALUES%252520%25250A%25285%25252C10%2529%25252C%25250A%25284%25252C20%2529%25253B&amp;display_name=Carbon&amp;url=https%3A%2F%2Fcarbon.now.sh%2F%3Fbg%3Drgba%252528171%25252C%2B184%25252C%2B195%25252C%2B1%252529%26t%3Dseti%26wt%3Dnone%26l%3Dsql%26width%3D680%26ds%3Dtrue%26dsyoff%3D20px%26dsblur%3D68px%26wc%3Dtrue%26wa%3Dtrue%26pv%3D56px%26ph%3D56px%26ln%3Dfalse%26fl%3D1%26fm%3DHack%26fs%3D14px%26lh%3D133%252525%26si%3Dfalse%26es%3D1x%26wm%3Dfalse%26code%3DCREATE%25252520TABLE%25252520sales%25252520%252528%25252520%2525250Apid%25252520SERIAL%25252520PRIMARY%25252520KEY%2525252C%2525250Aprice%25252520FLOAT%2525252C%2525250Aquantity%25252520INT%25252520%2525250A%252529%2525253B%2525250A%2525250AINSERT%25252520INTO%25252520sales%25252520%252528%2525250A%25252520%25252520price%2525252C%25252520quantity%2525250A%252529%25252520VALUES%25252520%2525250A%2525285%2525252C10%252529%2525252C%2525250A%2525284%2525252C20%252529%2525253B&amp;image=https%3A%2F%2Fcarbon.now.sh%2Fstatic%2Fbrand%2Fbanner.png&amp;type=text%2Fhtml&amp;scroll=auto&amp;schema=carbon" width="1024" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/06ca7ae5692b0a407bc15ae888630751/href">https://medium.com/media/06ca7ae5692b0a407bc15ae888630751/href</a></iframe><p>Now let&#39;s have two instances of this database running so we can simulate the phenomena</p><p><strong>Terminal No 1</strong></p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fcarbon.now.sh%2Fembed%3Fbg%3Drgba%2528171%252C%2B184%252C%2B195%252C%2B1%2529%26t%3Dseti%26wt%3Dnone%26l%3Dsql%26width%3D680%26ds%3Dtrue%26dsyoff%3D20px%26dsblur%3D68px%26wc%3Dtrue%26wa%3Dtrue%26pv%3D56px%26ph%3D56px%26ln%3Dfalse%26fl%3D1%26fm%3DHack%26fs%3D14px%26lh%3D133%2525%26si%3Dfalse%26es%3D1x%26wm%3Dfalse%26code%3DSELECT%252520pid%25252C%252520quantity%252Aprice%252520From%252520sales&amp;display_name=Carbon&amp;url=https%3A%2F%2Fcarbon.now.sh%2F%3Fbg%3Drgba%252528171%25252C%2B184%25252C%2B195%25252C%2B1%252529%26t%3Dseti%26wt%3Dnone%26l%3Dsql%26width%3D680%26ds%3Dtrue%26dsyoff%3D20px%26dsblur%3D68px%26wc%3Dtrue%26wa%3Dtrue%26pv%3D56px%26ph%3D56px%26ln%3Dfalse%26fl%3D1%26fm%3DHack%26fs%3D14px%26lh%3D133%252525%26si%3Dfalse%26es%3D1x%26wm%3Dfalse%26code%3DSELECT%25252520pid%2525252C%25252520quantity%25252Aprice%25252520From%25252520sales&amp;image=https%3A%2F%2Fcarbon.now.sh%2Fstatic%2Fbrand%2Fbanner.png&amp;type=text%2Fhtml&amp;scroll=auto&amp;schema=carbon" width="1024" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/0f096b5289ad2477b35acf9f25a96e33/href">https://medium.com/media/0f096b5289ad2477b35acf9f25a96e33/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/830/1*ZKM6XvXARXgmupYg85liiA.png" /></figure><p><strong>Terminal No 2</strong></p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fcarbon.now.sh%2Fembed%3Fbg%3Drgba%2528171%252C%2B184%252C%2B195%252C%2B1%2529%26t%3Dseti%26wt%3Dnone%26l%3Dsql%26width%3D680%26ds%3Dtrue%26dsyoff%3D20px%26dsblur%3D68px%26wc%3Dtrue%26wa%3Dtrue%26pv%3D56px%26ph%3D56px%26ln%3Dfalse%26fl%3D1%26fm%3DHack%26fs%3D14px%26lh%3D133%2525%26si%3Dfalse%26es%3D1x%26wm%3Dfalse%26code%3DUPDATE%252520sales%252520SET%252520quantity%25253Dquantity%25252B5%252520WHERE%252520pid%252520%25253D%2525201%25253B&amp;display_name=Carbon&amp;url=https%3A%2F%2Fcarbon.now.sh%2F%3Fbg%3Drgba%252528171%25252C%2B184%25252C%2B195%25252C%2B1%252529%26t%3Dseti%26wt%3Dnone%26l%3Dsql%26width%3D680%26ds%3Dtrue%26dsyoff%3D20px%26dsblur%3D68px%26wc%3Dtrue%26wa%3Dtrue%26pv%3D56px%26ph%3D56px%26ln%3Dfalse%26fl%3D1%26fm%3DHack%26fs%3D14px%26lh%3D133%252525%26si%3Dfalse%26es%3D1x%26wm%3Dfalse%26code%3DUPDATE%25252520sales%25252520SET%25252520quantity%2525253Dquantity%2525252B5%25252520WHERE%25252520pid%25252520%2525253D%252525201%2525253B&amp;image=https%3A%2F%2Fcarbon.now.sh%2Fstatic%2Fbrand%2Fbanner.png&amp;type=text%2Fhtml&amp;scroll=auto&amp;schema=carbon" width="1024" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/cfd0ca9ae430ab7112096049f9cb3c7c/href">https://medium.com/media/cfd0ca9ae430ab7112096049f9cb3c7c/href</a></iframe><p>Ideally, we should see the result of <strong><em>Terminal No 1</em></strong> query to be around 130 but it would be changed to this <strong>Non-Repeatable Read</strong> phenomenon as the change made by <strong><em>Terminal No 2</em></strong> is committed or persisted.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fcarbon.now.sh%2Fembed%3Fbg%3Drgba%2528171%252C%2B184%252C%2B195%252C%2B1%2529%26t%3Dseti%26wt%3Dnone%26l%3Dsql%26width%3D680%26ds%3Dtrue%26dsyoff%3D20px%26dsblur%3D68px%26wc%3Dtrue%26wa%3Dtrue%26pv%3D56px%26ph%3D56px%26ln%3Dfalse%26fl%3D1%26fm%3DHack%26fs%3D14px%26lh%3D133%2525%26si%3Dfalse%26es%3D1x%26wm%3Dfalse%26code%3DSELECT%252520SUM%2528quantity%252Aprice%2529%252520FROM%252520sales%25253B&amp;display_name=Carbon&amp;url=https%3A%2F%2Fcarbon.now.sh%2F%3Fbg%3Drgba%252528171%25252C%2B184%25252C%2B195%25252C%2B1%252529%26t%3Dseti%26wt%3Dnone%26l%3Dsql%26width%3D680%26ds%3Dtrue%26dsyoff%3D20px%26dsblur%3D68px%26wc%3Dtrue%26wa%3Dtrue%26pv%3D56px%26ph%3D56px%26ln%3Dfalse%26fl%3D1%26fm%3DHack%26fs%3D14px%26lh%3D133%252525%26si%3Dfalse%26es%3D1x%26wm%3Dfalse%26code%3DSELECT%25252520SUM%252528quantity%25252Aprice%252529%25252520FROM%25252520sales%2525253B&amp;image=https%3A%2F%2Fcarbon.now.sh%2Fstatic%2Fbrand%2Fbanner.png&amp;type=text%2Fhtml&amp;scroll=auto&amp;schema=carbon" width="1024" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/8879de2e255451252c4ab2ed2b20a020/href">https://medium.com/media/8879de2e255451252c4ab2ed2b20a020/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/556/1*uhEKieOwc4AltvvSDlreZA.png" /></figure><p>This returns result 155 changed from what we initially expected because another committed transaction modified the same rows.</p><h4>Phantom Reads</h4><blockquote>Occurs when a transaction re-executes a query and finds a <strong>new row </strong>(<em>phantom row</em>) that was inserted or existing ones that were deleted by another committed transaction.</blockquote><p>This type of read-phenomena usually occurs as a result of the <strong>INSERT</strong> | <strong>DELETE</strong> operation.</p><p>Let’s use the <strong>sales</strong> example again to understand this read phenomenon</p><p>First, create a sales table with this query and insert some values.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/692/1*mKRNlMKE3uZ3O3_34kVO6A.png" /></figure><p>Then have two terminal sessions open so that we simulate this real phenomenon.</p><p>In <strong>Terminal No 1</strong>, run a simple SELECT query to list down pid and quantity times price values</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fcarbon.now.sh%2Fembed%3Fbg%3Drgba%2528171%252C%2B184%252C%2B195%252C%2B1%2529%26t%3Dseti%26wt%3Dnone%26l%3Dsql%26width%3D680%26ds%3Dtrue%26dsyoff%3D20px%26dsblur%3D68px%26wc%3Dtrue%26wa%3Dtrue%26pv%3D56px%26ph%3D56px%26ln%3Dfalse%26fl%3D1%26fm%3DHack%26fs%3D14px%26lh%3D133%2525%26si%3Dfalse%26es%3D2x%26wm%3Dfalse%26code%3DSELECT%252520pid%25252C%252520quantity%252Aprice%252520FROM%252520sales&amp;display_name=Carbon&amp;url=https%3A%2F%2Fcarbon.now.sh%2F%3Fbg%3Drgba%252528171%25252C%2B184%25252C%2B195%25252C%2B1%252529%26t%3Dseti%26wt%3Dnone%26l%3Dsql%26width%3D680%26ds%3Dtrue%26dsyoff%3D20px%26dsblur%3D68px%26wc%3Dtrue%26wa%3Dtrue%26pv%3D56px%26ph%3D56px%26ln%3Dfalse%26fl%3D1%26fm%3DHack%26fs%3D14px%26lh%3D133%252525%26si%3Dfalse%26es%3D2x%26wm%3Dfalse%26code%3DSELECT%25252520pid%2525252C%25252520quantity%25252Aprice%25252520FROM%25252520sales&amp;image=https%3A%2F%2Fcarbon.now.sh%2Fstatic%2Fbrand%2Fbanner.png&amp;type=text%2Fhtml&amp;scroll=auto&amp;schema=carbon" width="1024" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/dd0b71f9f5c7e5fd1fae746db1d80dc4/href">https://medium.com/media/dd0b71f9f5c7e5fd1fae746db1d80dc4/href</a></iframe><p>Which shows results like</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/534/1*34YVm7ZvXFAZeK_TFqR0sw.png" /></figure><p>Now open <strong>Terminal No. 2</strong> and run an <strong>INSERT</strong> query to add a new record to the <strong>sales</strong> table.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fcarbon.now.sh%2Fembed%3Fbg%3Drgba%2528171%252C%2B184%252C%2B195%252C%2B1%2529%26t%3Dseti%26wt%3Dnone%26l%3Dsql%26width%3D680%26ds%3Dtrue%26dsyoff%3D20px%26dsblur%3D68px%26wc%3Dtrue%26wa%3Dtrue%26pv%3D56px%26ph%3D56px%26ln%3Dfalse%26fl%3D1%26fm%3DHack%26fs%3D14px%26lh%3D133%2525%26si%3Dfalse%26es%3D2x%26wm%3Dfalse%26code%3DINSERT%252520INTO%252520sales%252520%2528price%25252C%252520quantity%2529%252520VALUES%252520%25281%25252C%25252010%2529%25253B&amp;display_name=Carbon&amp;url=https%3A%2F%2Fcarbon.now.sh%2F%3Fbg%3Drgba%252528171%25252C%2B184%25252C%2B195%25252C%2B1%252529%26t%3Dseti%26wt%3Dnone%26l%3Dsql%26width%3D680%26ds%3Dtrue%26dsyoff%3D20px%26dsblur%3D68px%26wc%3Dtrue%26wa%3Dtrue%26pv%3D56px%26ph%3D56px%26ln%3Dfalse%26fl%3D1%26fm%3DHack%26fs%3D14px%26lh%3D133%252525%26si%3Dfalse%26es%3D2x%26wm%3Dfalse%26code%3DINSERT%25252520INTO%25252520sales%25252520%252528price%2525252C%25252520quantity%252529%25252520VALUES%25252520%2525281%2525252C%2525252010%252529%2525253B&amp;image=https%3A%2F%2Fcarbon.now.sh%2Fstatic%2Fbrand%2Fbanner.png&amp;type=text%2Fhtml&amp;scroll=auto&amp;schema=carbon" width="1024" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/fd41ccdfdb3c9c923312215ef9dd84cf/href">https://medium.com/media/fd41ccdfdb3c9c923312215ef9dd84cf/href</a></iframe><p>This would add a new row to the existing <strong>sales</strong> table. Since we have not mentioned any <strong>BEGIN</strong> keyword, in Postgres by default transactions are run this statement will automatically be a transaction and committed to persist.</p><p>Back to <strong>Terminal No. 1</strong>, we run an aggregation query and as per our last query in this transaction, the sum should be around $130. right. Let’s see</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fcarbon.now.sh%2Fembed%3Fbg%3Drgba%2528171%252C%2B184%252C%2B195%252C%2B1%2529%26t%3Dseti%26wt%3Dnone%26l%3Dsql%26width%3D680%26ds%3Dtrue%26dsyoff%3D20px%26dsblur%3D68px%26wc%3Dtrue%26wa%3Dtrue%26pv%3D56px%26ph%3D56px%26ln%3Dfalse%26fl%3D1%26fm%3DHack%26fs%3D14px%26lh%3D133%2525%26si%3Dfalse%26es%3D2x%26wm%3Dfalse%26code%3DSELECT%252520SUM%2528quantity%252Aprice%2529%252520FROM%252520sale&amp;display_name=Carbon&amp;url=https%3A%2F%2Fcarbon.now.sh%2F%3Fbg%3Drgba%252528171%25252C%2B184%25252C%2B195%25252C%2B1%252529%26t%3Dseti%26wt%3Dnone%26l%3Dsql%26width%3D680%26ds%3Dtrue%26dsyoff%3D20px%26dsblur%3D68px%26wc%3Dtrue%26wa%3Dtrue%26pv%3D56px%26ph%3D56px%26ln%3Dfalse%26fl%3D1%26fm%3DHack%26fs%3D14px%26lh%3D133%252525%26si%3Dfalse%26es%3D2x%26wm%3Dfalse%26code%3DSELECT%25252520SUM%252528quantity%25252Aprice%252529%25252520FROM%25252520sale&amp;image=https%3A%2F%2Fcarbon.now.sh%2Fstatic%2Fbrand%2Fbanner.png&amp;type=text%2Fhtml&amp;scroll=auto&amp;schema=carbon" width="1024" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/1a388f73b133818fa7a43260a1f6e326/href">https://medium.com/media/1a388f73b133818fa7a43260a1f6e326/href</a></iframe><p>which results in sum to be $140 instead of $130, why ? because of a phantom row <strong>Terminal No. 2</strong> transaction inserted</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/240/1*hYBWoGpmE-ZIxyq9UwNtRQ.png" /></figure><p>This is how a <strong>phantom-read</strong> phenomenon occurs.</p><h4>Lost Updates</h4><blockquote>Occurs when <strong>two or more transactions</strong> modify the <strong>same data concurrently</strong>, and all of the updates except one are overwritten (lost) because there’s no mechanism to detect or prevent conflicting write.</blockquote><p>It&#39;s mainly caused by <strong>UPDATE</strong> operations without proper <strong>locking and isolation</strong> applied, that is where Transactions come into the picture to save the day.</p><p>Let’s take a real-world example for this as simulating a real-time concurrent transaction is a bit tough.</p><p><strong>Collaborative Document Editing</strong>:</p><p>Consider two users A and B working with a collaborative document like MS Word etc.</p><ul><li><strong>User A</strong> opens a document and edits a sentence.</li><li><strong>User B</strong> opens the same document <em>at the same time</em> and deletes the same sentence.</li><li><strong>User A</strong> saves their changes, and then <strong>User B</strong> saves theirs.</li><li><strong>Result</strong>: User A’s edit is overwritten by User B’s deletion. The update from User A is “lost.”</li></ul><h3>Isolation Levels</h3><p>For all of the above Read-Phenomenon, we have different Isolation levels where each solves a particular read phenomenon problem and some may not even solve any at all.</p><h4>Read Uncommitted</h4><p>This <strong>Isolation Level</strong> is another name for <strong>No Isolation</strong>. So it simply says,</p><blockquote>Show my any change outside of my current transaction.</blockquote><p>Thus it can incur any of the Read Phenomenon we discussed previously.</p><h4>Read Committed</h4><blockquote>Each query in a transaction only sees committed changes by other transactions.</blockquote><p>This <strong>Isolation Level</strong> is mostly the default one for most of the relational database technologies.</p><h4>Repeatable Reads</h4><blockquote>A transaction will make sure that when a query reads a row that row will remain unchanged while transaction is still running.</blockquote><p>Unlike other <strong>Isolation Levels</strong>, we discussed this allows us to avoid Non-Repeatable read phenomena as we can repeat a read which is unlikely possible in the case of a <strong>Non-Repeatable Read</strong>. But it still gets caught by phantom reads so be careful.</p><h4>Snapshot</h4><p>As its name suggests, It’s a transaction in which</p><blockquote>Each query sees changes that have bee committed up to start of on-going transaction.</blockquote><p>It’s simply a snapshot of the database at the moment. This is the one Isolation Level that gets rid of all <strong>Read Phenomena</strong> as we are restricting the transaction ability to read other committed transaction changes and applying locking principles.</p><h4>Serializable</h4><blockquote>Each Transactions rusn as if they are serialized one after another.</blockquote><p>Thus another transaction can&#39;t introduce changes in the current transaction’s database state.</p><h3>Conclusion</h3><p>In this article, we saw how the <strong>Isolation for I in ACID </strong>principle helps keep our data safe and consistent so that we may not run into unknown issues. Though all of that comes at a cost eventually which is speed in execution with advancement in the field of Computation it&#39;s no longer a bigger problem and still billions of CRUD operations in databases are happening concurrently thanks to <strong>ACID</strong> principles.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5f40f2995dd3" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Atomicity in Databases]]></title>
            <link>https://medium.com/@sibteali786/atomicity-in-databases-229e83e1788b?source=rss-b07c6b861c5d------2</link>
            <guid isPermaLink="false">https://medium.com/p/229e83e1788b</guid>
            <category><![CDATA[postgresql]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[database]]></category>
            <category><![CDATA[atomicity]]></category>
            <category><![CDATA[daily-blog]]></category>
            <dc:creator><![CDATA[Sibt-e-ali Baqar]]></dc:creator>
            <pubDate>Tue, 21 Jan 2025 03:16:41 GMT</pubDate>
            <atom:updated>2025-01-21T03:16:41.389Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*mFBmWVXHg5-tIyS2v5C7sg.png" /><figcaption>Atomicity in a Nutshell</figcaption></figure><p>Being a Backend Engineer who recently shited from Frontend Engineering one of many other databases-created concepts s which I had to brush up my skills was the ACID principle. The first and most fundamental concept is called atomicity. Let’s get going and see</p><ul><li>What is Atomicity?</li><li>Why do we need it?</li><li>How does it work?</li></ul><h3>What is Atomicity And How Does it Work</h3><p>In simple words, it is an idea that</p><blockquote>All queries in a transaction must succeed</blockquote><p>OR</p><blockquote>Atomicity is the idea of Transaction being one unit of work and that cannot be split</blockquote><p>Wow! Really? Is that all to Atomicity, practically YES also if you do not know what it is <a href="https://medium.com/@sibteali786/what-is-a-transaction-in-a-database-e4ac3c8d88de">Transaction</a> I have written an article for it so you can check it out.</p><p>Anyway, there are a few things to be kept in mind regarding Atomicity</p><ul><li>If any particular query fails, all prior successful queries should roll back.</li></ul><pre>BEGIN;<br>-- lets say we are transferrign money between accounts<br>UPDATE accounts SET balance = balance + 1000 WHERE user_name = &quot;Alice&quot;;<br><br>-- this query might fail maybe because Bob has not 1000 in his account <br>UPDATE accounts SET balance = balance - 1000 WHERE user_name = &quot;Bob&quot;;<br><br>-- If both queries succeed, commit the changes<br>COMMIT;<br><br>-- If any query fails, all changes are rolled back automatically<br>-- You can also explicitly roll back:<br>ROLLBACK;</pre><ul><li>If the database fails before committing all transactions (successful queries ), it should rollback</li></ul><pre>BEGIN TRANSACTION;<br><br>-- Imagine we&#39;re processing a customer order<br>INSERT INTO orders (order_id, customer_id, order_date)<br>VALUES (&#39;ORD001&#39;, &#39;CUST123&#39;, CURRENT_TIMESTAMP);<br><br>INSERT INTO order_items (order_id, product_id, quantity)<br>VALUES (&#39;ORD001&#39;, &#39;PROD456&#39;, 2);<br><br>UPDATE inventory<br>SET stock_count = stock_count - 2<br>WHERE product_id = &#39;PROD456&#39;;<br><br>-- If the database crashes before this COMMIT, all changes roll back<br>COMMIT;</pre><ul><li>Longer transactions take a longer time to roll back, thus longer transactions are discouraged.</li></ul><pre>-- Example of a long transaction that should be broken down<br>BEGIN TRANSACTION;<br><br>-- Multiple operations that could take significant time<br>UPDATE large_table_1 <br>SET processed = TRUE <br>WHERE created_date &lt; CURRENT_DATE;<br><br>-- Batch processing of records<br>INSERT INTO processed_records <br>SELECT * FROM temporary_records <br>WHERE status = &#39;PENDING&#39;;<br><br>-- More updates that add to transaction time<br>UPDATE customer_statistics <br>SET last_processed = CURRENT_TIMESTAMP<br>WHERE customer_id IN (SELECT DISTINCT customer_id FROM temporary_records);<br><br>-- If a rollback is needed here, it must undo all previous operations<br>COMMIT;</pre><p>Thus it&#39;s better to have a small transaction that can be managed well and if something bad happens it can be rolled back easily.</p><pre>-- Better approach: Break into smaller transactions<br>BEGIN TRANSACTION;<br>UPDATE large_table_1 <br>SET processed = TRUE <br>WHERE created_date &lt; CURRENT_DATE<br>  AND id BETWEEN 1 AND 1000;<br>COMMIT;<br><br>BEGIN TRANSACTION;<br>UPDATE large_table_1 <br>SET processed = TRUE <br>WHERE created_date &lt; CURRENT_DATE<br>  AND id BETWEEN 1001 AND 2000;<br>COMMIT;</pre><h3>Why do we need it?</h3><p>Lack of atomicity leads to many problems, lets discuss a scenario explaining our stance.</p><p>Imagine you’re transferring money between two bank accounts. Let’s say you’re sending $1000 from your savings to your checking account. This seemingly simple transfer involves two operations: deducting money from savings and adding it to checking. Without atomicity, several problematic scenarios could occur:</p><p>If the system deducts <em>$1000</em> from your savings but crashes before adding it to your checking, your money would simply vanish. Or imagine the reverse — if the system adds money to checking but fails to deduct it from savings, you’d essentially create money out of thin air. Both scenarios would be catastrophic for a banking system.</p><p>This is where atomicity comes in. Atomicity ensures that related operations are treated as a single, indivisible unit. Just like an atom was historically considered the smallest indivisible unit of matter, an atomic transaction is indivisible — it either happens completely or not at all.</p><p>Let’s explore the key reasons why atomicity is essential:</p><ol><li>Data Consistency Consider an e-commerce system processing an order. When a customer places an order, several things need to happen:</li></ol><ul><li>The order must be recorded</li><li>The inventory must be reduced</li><li>The customer’s payment must be processed</li><li>A shipping request must be created</li></ul><p>Without atomicity, you might end up with situations where the payment is processed but the inventory isn’t updated, or the inventory is reduced but the order isn’t recorded. This leads to data inconsistency that could cause significant business problems.</p><p>Let&#39;s see one more example</p><p>2. Parallel Processing Modern databases handle many operations simultaneously. Without atomicity, concurrent transactions could interfere with each other in unpredictable ways. Atomicity, along with other ACID properties, helps ensure that parallel transactions don’t create data inconsistencies.</p><h4>Code Example</h4><pre>BEGIN TRANSACTION;<br><br>-- First operation: Deduct from savings<br>UPDATE accounts <br>SET balance = balance - 1000 <br>WHERE account_id = &#39;savings_123&#39;;<br><br>-- Second operation: Add to checking<br>UPDATE accounts <br>SET balance = balance + 1000 <br>WHERE account_id = &#39;checking_456&#39;;<br><br>-- If both operations succeed, make them permanent<br>COMMIT;<br><br>-- If anything fails, undo everything<br>ROLLBACK;</pre><p>In this code, atomicity ensures that both UPDATE operations either succeed or fail together. There’s no possibility of one update happening without the other.</p><h3>What could happen if Atomicity is not there?</h3><p>The implications of not having atomicity would be severe:</p><ul><li>Financial systems could lose track of money</li><li>Inventory systems could show incorrect stock levels</li><li>Customer data could become inconsistent</li><li>Business operations could become unreliable</li><li>Data recovery after system failures would be extremely complex</li></ul><h3>Conclusion</h3><p>Database systems need atomicity to perform complex operations safely. Without it, every transaction would be a dangerous operation that could potentially corrupt your data or maybe turn you from a billionaire to a roadside beggar in one second :&gt;. That is why it&#39;s a crucial part of the ACID principle and together with Consistency, Isolation, and Durability, we are able to make reliable and Robust systems.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=229e83e1788b" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[What is a Transaction in a Database?]]></title>
            <link>https://medium.com/@sibteali786/what-is-a-transaction-in-a-database-e4ac3c8d88de?source=rss-b07c6b861c5d------2</link>
            <guid isPermaLink="false">https://medium.com/p/e4ac3c8d88de</guid>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[transactions]]></category>
            <category><![CDATA[postgresql]]></category>
            <category><![CDATA[database]]></category>
            <dc:creator><![CDATA[Sibt-e-ali Baqar]]></dc:creator>
            <pubDate>Sun, 19 Jan 2025 08:07:31 GMT</pubDate>
            <atom:updated>2025-01-19T08:07:31.393Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*g2Tlnsu3k56EU0xokyGCSg.png" /><figcaption>Graphical way to undertsand Transaction</figcaption></figure><p>I recently went back to grasp a hold of my database skills and fundamentals and as always found myself in a position where I still do not understand fundamental concepts such as <em>Transactions</em>, of course, I had to make sure that I got it right this time. Let&#39;s discover what a Transaction means in Databases.<br>We will discuss the following things</p><ol><li>What is a Transaction</li><li>Why It&#39;s Needed</li><li>How does it work</li><li>Different Parts of a Transaction</li></ol><h3>What is a Transaction?</h3><blockquote>A collection of SQL queries treated as one unit of work.</blockquote><p>This is the simplest and easiest definition or explanation I could honestly think of in a way that describes a transaction unless everything goes fine like all SQL queries are successful or all queries are failed it&#39;s a Transaction.</p><h3>Why it&#39;s needed</h3><p>Without transactions, we will quickly run into many issues which the ACID principle is there to help us avoid.</p><h4>Atomicity</h4><blockquote>All or Nothing</blockquote><p>Either all queries will be written to Disk ( Heap ) or none.</p><pre>BEGIN;<br>-- Transfer $100 from Alice to Bob<br>UPDATE accounts SET balance = balance - 100 WHERE username = &quot;Alice&quot;;<br>UPDATE accounts SET balance = balance + 100 WHERE username = &quot;Bob&quot;;<br><br>-- if any of these fail entire transaction is rolledback<br>-- otherwsie we commit these<br>COMMIT;</pre><h4>Consistency</h4><blockquote>Rules are always enforced</blockquote><p>The transaction allows us to make sure everything is consistent in all the queries to be run, let&#39;s understand via an example</p><pre>BEGIN;<br>-- This enforces a rule that balance can&#39;t be nagative <br>CREATE TABLE accounts (<br>  user_name TEXT,<br>  balance DECIMAL CHECK (balance &gt;= 0)<br>);<br><br>-- This will fail and rollback if rules are violated <br>UPDATE accounts SET balance = balance - 1000 WHERE user_name = &#39;Alice&#39;;<br>COMMIT;</pre><h4>Isolation</h4><blockquote>Transactions do not interfere with other on-going transactions at all.</blockquote><p>If multiple transactions are running in parallel ( using worker threads maybe ) then Isolation will allow each transaction to be unique with its own set of Database table values ( in other words Snapshot ). Let’s try an example to understand it.</p><p><strong>Terminal 1</strong></p><pre>BEGIN;<br><br>UPDATE accounts SET balance = balance - 100 WHERE user_name = &#39;Alice&#39;;<br>-- Changes not visible to other transactions yet</pre><p><strong>Terminal 2</strong></p><pre>BEGIN;<br>SELECT balance FROM accounts WHERE user_name=&#39;Alice&#39;;<br>-- Still sees old balance until Terminal 1 commits</pre><p>There are many types of Isolation which we will discuss in the future for sure, and without any doubt Isolation is an interesting topic in itself.</p><h4>Durability</h4><blockquote>Committed Changes ( Changes written to Disk i.e Heap ) persist.</blockquote><p>When we commit in a transaction, changes are then visible by all other ongoing transactions, also they are written to disk ( Heap ) which makes them persist. It means they stay even if the database crashes or any other natural disaster occurs ( of course one which does not destroy the physical disk :&gt; ).</p><pre>BEGIN;<br>INSERT INTO important_records (data) VALUES (&#39;critical info&#39;);<br>COMMIT;<br>-- Even if database crashes now, this data is safe</pre><h3>How does it work?</h3><p>Let me see, I think I already explained it with all the examples above…kidding, let deep dive further using a complete example that helps us understand it clearly.</p><pre>CREATE TABLE accounts (<br>  user_name TEXT,<br>  balance DECIMAL CHECK (balance &gt;=0)<br>);<br><br>INSERT INTO accounts VALUES<br>(&quot;Alice&quot;, 1000),<br>(&quot;Bob&quot;, 500);<br><br>-- start a trnasaction <br>BEGIN;<br><br>-- Atomicity: Both operations must succeed or none will<br>UPDATE accounts <br>SET balance = balance - 200 <br>WHERE user_name = &#39;Alice&#39;;<br><br>UPDATE accounts <br>SET balance = balance + 200 <br>WHERE user_name = &#39;Bob&#39;;<br><br>-- Consistency: Check will prevent negative balance<br>-- Isolation: Other transactions can&#39;t see these changes yet<br>-- Durability: After commit, changes survive crashes<br>COMMIT;</pre><p>We can verify the changes once they are committed to another terminal ( instance of the database, don’t worry I’ll teach you to do that as well, it&#39;s pretty simple :&lt; That is what they always tell me… lolz )</p><pre>BEGIN;<br>SELECT * FROM accounts;<br>-- Won&#39;t see the transfer until first transaction commits</pre><h3>Post Martum of a Transaction</h3><p>Personally, that is what I call it myself, anyway a Transaction consists of many under-the-hood operations that allow the database to avoid any problems such as we discussed while trying to explain ACID in transactions.</p><p>Let’s go over each in detail, I ll be specifically targeting PostgreSQL here as that is my primary go-to database these days.</p><h3>BEGIN</h3><blockquote>Initiates a Transaction</blockquote><p>Ok, but what does it mean? The word initiates has many layers behind the scenes actually</p><h4>Transaction ID Assignment</h4><p>An ID is assigned to the current transaction, so it can be tracked and distinguished from any other ongoing or upcoming transactions (possibly for reference, isolating stuff, etc ).</p><p>The Id is an ( XID ) i.e. a 32-bit number.</p><p><strong>Benefits</strong></p><ul><li>Ordering i.e. each transaction is ordered. The transaction started at 10:00 am will be assigned xid = 1001, while one afterward will be xid = 1002</li></ul><pre>-- Transaction 1<br>BEGIN;  -- Gets XID 3047<br>UPDATE accounts SET balance = balance - 100;<br><br>-- Transaction 2<br>BEGIN;  -- Gets XID 3048<br>-- This transaction knows it&#39;s &quot;newer&quot; than 3047</pre><ul><li>Helps the database determine the age of a transaction i.e. how old a transaction is. Actually, it allows the database to run VACUM ( kind of maintenance ) .</li></ul><pre>- Check how old transactions are<br>SELECT age(datfrozenxid) FROM pg_database;<br>-- PostgreSQL uses this to know when to run VACUUM</pre><h4>Snapshot Creation</h4><blockquote>The database creates a snapshot of itself so that it can decide what values or states this transaction has access to.</blockquote><p>This also helps in isolating this transaction from other ongoing transactions which may cause in-consistency otherwise.</p><h4>Lock Management System</h4><p>Honestly, this is the one I always forget about, so I ll try to explain it in my own words as easy as i can i make it.</p><blockquote>If two transactions are trying to modify the same resource in the database, lock management makes sure the second one does not take effect unless the first one is either committed or rolled back.</blockquote><h3>COMMIT</h3><blockquote>Write the changes made by queries to DISK to persist the changes to disk.</blockquote><p>When a query runs it only makes changes to Memory ( Buffer Cache ) and to persist them we have to write changes to actual data on Non-volatile memory i.e. Disk or Heap.</p><blockquote>The reason why we do not write directly to disk is simple i.e. it&#39;s expensive and slow buddy, and we as a database make sure we do thing fast but with durability and reliability.</blockquote><p>Let’s look at what exactly happens when we commit.</p><p><strong>Write Ahead Logging</strong></p><ul><li>Before actually writing to disk, the database first writes changes to a log file (WAL).</li><li>This log has all the changes made by the queries</li><li>It&#39;s a safety mechanism: if the database crashes during commit ( ouch is that possible… well technically YES ) database can recover using this WAL</li></ul><p><strong>Memory to Disk Transfer</strong></p><ul><li>The changes that were previously only in memory ( buffer cache ) are written to the actual data files on disk.</li><li>This can involve writing to multiple pages ( that is how data is stored on disk ) as the data written may be part of more than one page, and thus may require more than one IO.</li><li>Indexes are also updated on disk if any indexes were affected by the transaction.</li></ul><p><strong>Locks Release</strong></p><ul><li>Finally, the locks that were put in place by <em>BEGIN</em> are released so that other transactions can see the changes now.</li><li>Other transactions waiting for these locks can now proceed</li></ul><h3>ROLLBACK</h3><p>The final beast or let me say the poor one which has to be responsible for everything if something fails.</p><p>It can be before commit, which is fast as we just have to flush the <em>buffer cache </em>aka Memory.</p><p>If we roll back after the commit, Oops it&#39;s slow and expensive as now it has to write things back from the actual disk which is indeed another <em>compensating</em> transaction. ( Not advised unless a REAL BLUNDER ).</p><h3>Conclusion</h3><p>Transactions are beautiful and much-needed operations in Databases when I was recently working on my backend with Postgres (I am using Prisma ORM ) I did not know if Prism ran any transactions or not but it indeed does and Postgres itself does it by default.</p><p>But we can always try to do so ourselves as well, it helps us gain control over things that involve multiple microservices like uploading an image to S3 and saving metadata to Postgres. It&#39;s a long process and thus requires running a transaction.</p><p>Things to remember about Transaction</p><ol><li>Usually, Transactions are used to modify and change data.</li><li>However, Transactions are not only written operations but read-only as well like running <em>SELECT</em> queries. why it’s something we can ponder over :&gt;</li></ol><p>that&#39;s all for today I will share more about Isolation, Consistency, and Durability in the future.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e4ac3c8d88de" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Debugging Pino Logger Issues in a Next.js]]></title>
            <link>https://medium.com/@sibteali786/debugging-pino-logger-issues-in-a-next-js-4e0c3368ef14?source=rss-b07c6b861c5d------2</link>
            <guid isPermaLink="false">https://medium.com/p/4e0c3368ef14</guid>
            <category><![CDATA[vercel]]></category>
            <category><![CDATA[nodejs]]></category>
            <category><![CDATA[nextjs]]></category>
            <category><![CDATA[pino]]></category>
            <dc:creator><![CDATA[Sibt-e-ali Baqar]]></dc:creator>
            <pubDate>Fri, 10 Jan 2025 08:26:25 GMT</pubDate>
            <atom:updated>2025-01-10T08:26:25.681Z</atom:updated>
            <content:encoded><![CDATA[<h3>Debugging Pino Logger Issues in a Next.js Turbo Monorepo on Vercel: A Journey Through Backend Engineering</h3><p>When building modern web applications, we often encounter unexpected challenges that push us to understand the tools we use better. Today, I’ll share our journey of solving a perplexing issue with Pino logger deployment in a Next.js application within a Turbo monorepo, deployed on Vercel.</p><h3>The Initial Setup</h3><p>Our project started as a typical Turbo monorepo structure using pnpm as our package manager. The architecture looked something like this:</p><p>```<br>.<br>├── apps<br>│ └── web (Next.js application)<br>├── packages<br>│ ├── ui<br>│ ├── eslint-config<br>│ └── typescript-config<br>```</p><p>We were using Pino for logging purposes in our Next.js application, a popular choice for its performance and flexibility. The initial logger setup was straightforward:</p><pre>// utils/logger.ts<br>import pino from &quot;pino&quot;;<br><br>const isProduction = process.env.NODE_ENV === &quot;production&quot;;<br><br>const logger = pino({<br>  level: isProduction ? &quot;info&quot; : &quot;debug&quot;,<br>  browser: {<br>    asObject: true,<br>    write: (o) =&gt; {<br>      console.log(JSON.stringify(o));<br>    }<br>  },<br>  // Configure for both environments without worker threads<br>  transport: {<br>    target: isProduction ? &quot;pino&quot; : &quot;pino-pretty&quot;,<br>    options: {<br>      colorize: !isProduction,<br>      translateTime: &quot;SYS:standard&quot;,<br>      ignore: &quot;pid,hostname&quot;,<br>    },<br>  }<br>});<br><br>export default logger;</pre><h3>The Problem Emerges</h3><p>Everything worked perfectly in local development. However, when we deployed to Vercel, we started seeing cryptic errors:</p><pre>Error: Cannot find module &#39;thread-stream&#39; Require stack:<br>- /var/task/apps/web/.next/server/app/(auth)/signin/page.js</pre><p>This was followed by another error when we added thread-strean as dependency and tried to fix previous error :</p><pre>Error: unable to determine transport target for &quot;pino&quot;</pre><h3>Understanding the Root Cause</h3><p>The issue stemmed from how Next.js handles server-side components and dependencies in production, particularly in serverless environments like Vercel. Pino uses worker threads for performance optimization, which requires special handling in such environments.</p><p>The problem had multiple layers:<br>1. Next.js bundles dependencies for optimization<br>2. Pino needs access to Node.js native features<br>3. Serverless environments have specific constraints<br>4. The monorepo structure added complexity to the package resolution</p><h4>Failed Attempts and Learning</h4><p>We tried several approaches that didn’t quite solve the problem:</p><ol><li>First, we attempted to modify webpack configuration:</li></ol><pre><br>webpack: (config, { isServer }) =&gt; {<br> if (isServer) {<br> config.externals = [<br> …config.externals,<br> &quot;pino-pretty&quot;,<br> &quot;thread-stream&quot;<br> ];<br> }<br> return config;<br>}<br>jav</pre><p>2. Then, we tried adjusting Pino’s configuration to avoid worker threads:</p><pre>const logger = pino({<br> browser: {<br> write: {<br> sync: true<br> }<br> }<br>});</pre><p>Neither approach fully resolved the issue. The problem wasn’t just about external dependencies or configuration — it was about how these packages needed to run in the server environment i.e NodeJs Runtime and not in a NextJs environment.</p><h4>The Solution</h4><p>The breakthrough came when we got tired of LLM suggestions and went old way of Stackoverflow and Github discussions and discovered at <br><a href="https://github.com/pinojs/pino/issues/1964#issuecomment-2299477588">https://github.com/pinojs/pino/issues/1964#issuecomment-2299477588</a></p><p>that Next.js’s experimental feature `serverComponentsExternalPackages`. can help us specify which libraries in Server components specifically need nodejs runtime and Here’s the configuration that finally solved our problem:</p><pre>/** @type {import(&#39;next&#39;).NextConfig} */<br>const nextConfig = {<br>  experimental: {<br>    serverComponentsExternalPackages: [<br>      &quot;thread-stream&quot;,<br>      &quot;pino&quot;,<br>      &quot;pino-worker&quot;,<br>      &quot;pino-file&quot;,<br>      &quot;pino-pretty&quot;,<br>    ],<br>  },<br>//...  <br>webpack: (config) =&gt; {<br>    // Add path alias<br>    config.resolve.alias = {<br>      ...config.resolve.alias,<br>      &quot;@&quot;: &quot;.&quot;,<br>    };<br><br>    return config;<br>  },<br>};<br><br>export default nextConfig;</pre><p>This solution worked because:<br>- It tells Next.js to treat these packages as external in server components<br>- The packages run in the main Node.js environment<br>- They get full access to native Node.js features<br>- It preserves the functionality of worker threads and async logging</p><h3>Lessons Learned</h3><p>This experience taught us several valuable lessons about modern web Frameworks.</p><p>1. Understanding the deployment environment is crucial. Local development and production can behave very differently, especially in serverless environments.</p><p>2. Monorepos adds complexity to dependency management. What works in a traditional setup might need special handling in a monorepo.</p><p>3. When using performance-optimized packages like Pino, we need to consider how they interact with the bundling and deployment process.</p><p>4. Next.js’s experimental features can be powerful solutions for edge cases, though they require careful consideration.</p><h4>Moving Forward</h4><p>While our solution works, there’s a small caveat: we get a Node.js experimental warning about <strong>vm.USE_MAIN_CONTEXT_DEFAULT_LOADER</strong> We can suppress this warning by adding `NODE_NO_WARNINGS=1` to our environment variables in Vercel.</p><p>The warning doesn’t affect functionality, but it reminds us that we’re using experimental features that might change in future versions of Next.js.</p><h3>Conclusion</h3><p>This journey highlights the complexity of modern web development, where we’re often integrating multiple tools and platforms. Understanding how these pieces interact is crucial for solving deployment issues.</p><p>For teams facing similar issues, remember to:<br>- Consider the full stack when debugging — from package management to deployment environment<br>- Don’t be afraid to dive into experimental features when standard solutions fall short<br>- Keep an eye on the broader ecosystem as these tools continue to evolve</p><p>Have you encountered similar issues with logging in serverless environments? How did you solve them? Share your experiences in the comments below.</p><p>You can always subscribe to my medium profile i usually share my learnings, hacks, Tips, and tricks regarding programming.<br>Github</p><p><a href="https://github.com/sibteali786">sibteali786 - Overview</a></p><p>LinkedIn</p><p><a href="https://www.linkedin.com/in/syed-sibteali-baqar/">https://www.linkedin.com/in/syed-sibteali-baqar/</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4e0c3368ef14" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Parallel Routes In NextJs and Tricky parts]]></title>
            <link>https://medium.com/@sibteali786/parallel-routes-in-nextjs-and-tricky-parts-61ee39c9a312?source=rss-b07c6b861c5d------2</link>
            <guid isPermaLink="false">https://medium.com/p/61ee39c9a312</guid>
            <category><![CDATA[performance]]></category>
            <category><![CDATA[react]]></category>
            <category><![CDATA[nextjs]]></category>
            <category><![CDATA[production]]></category>
            <dc:creator><![CDATA[Sibt-e-ali Baqar]]></dc:creator>
            <pubDate>Mon, 16 Dec 2024 08:18:45 GMT</pubDate>
            <atom:updated>2024-12-16T08:18:45.255Z</atom:updated>
            <content:encoded><![CDATA[<h3>Parallel Routes In NextJs and Tricky Parts</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*mHnyLavjK1ZP6hlg" /></figure><h3>Navigating the Challenges of Next.js 14: Role-Based Parallel Routes</h3><p>It has been a journey working with <strong>Next.js 14</strong> and exploring its new features. While adopting the latest tools is always exciting, this experience has come with its share of challenges and frustrations. Here, I want to share one particularly tricky problem I encountered — and how I resolved it — hoping it might help others facing a similar issue.</p><h4>Use Case: Role-Based Parallel Routes</h4><p>In our project, we needed to implement <strong>Parallel Routes (Slots) </strong>to cater to a <strong>multi-user Role-Based Access</strong> system. The goal was straightforward: ensure that users with specific roles only see content tailored to their needs. For example, a user with a <em>CLIENT</em> role should only access content meant for clients.</p><h4>The Problem: Unintended Route Rendering</h4><p>Initially, everything seemed to work fine. However, upon testing, I noticed something odd. My <em>Pino</em> loggers started returning logs from <strong>Parallel Routes</strong> that weren’t supposed to render at all for the current user role. This discovery raised a red flag.</p><p>Visiting a single route — like <em>/home</em> — triggered about <strong>30 database requests</strong>, even though it should have only triggered <strong>3</strong> requests at most.</p><p>This inefficiency is not acceptable for a production-grade application.</p><h4>The Root Cause</h4><p>After investigating, I realized that all the **Parallel Routes** were being rendered, regardless of whether the current user role had access to them. This unnecessary rendering was causing a significant performance bottleneck, resulting in excessive database requests and slower responses.</p><h4>The Solution</h4><p>After some deep research and community help, I stumbled upon this [GitHub issue](<a href="https://github.com/vercel/next.js/issues/53292">https://github.com/vercel/next.js/issues/53292</a>), which clarified the behavior.</p><p>To fix the issue, I needed to ensure that all <em>page.tsx</em> files in the routes irrelevant to the current user role explicitly <strong>returned</strong> <em>null.</em> This prevents unnecessary rendering and ensures that only the required routes are processed.</p><p>Here’s how I implemented the solution for my existing <em>page.tsx</em> for Client Role:</p><pre><br>import { getTasksByUserId } from &quot;@/db/repositories/users/getTasksByUserId&quot;;<br>// rest of code<br>const RecentTasksPage = async () =&gt; {<br> const user = await getCurrentUser();<br>// Only render for CLIENT role<br> if (user.role.name !== Roles.CLIENT) {<br> return null;<br> }<br>const { tasks } = await getTasksByUserId(user.id, Roles.CLIENT, null, 5);<br>return (....)<br>....<br></pre><h4>Key Takeaways</h4><ol><li><strong>Parallel Routes</strong> are powerful, but you need to ensure that only the relevant routes are rendered for the current context.</li><li>Always validate your user’s role before rendering any content on <em>page.tsx.</em></li><li>Returning <em>null</em> for irrelevant routes can significantly improve performance and avoid unnecessary API/database calls.</li></ol><h4>Closing Thoughts</h4><p>Working with Next.js 14 has been a rollercoaster of challenges and learning. While some of its features come with quirks, the process of debugging and optimizing has been a valuable experience.</p><p>If you’ve faced similar issues with Next.js 14 or have insights into optimizing Parallel Routes, I’d love to hear your thoughts in the comments.</p><p>Let’s keep growing and learning together! 🚀</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=61ee39c9a312" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>