<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Poorshad Shaddel on Medium]]></title>
        <description><![CDATA[Stories by Poorshad Shaddel on Medium]]></description>
        <link>https://medium.com/@p-shaddel?source=rss-8f1fd23dbcf6------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*FB-WPFCYdfqie7PY5TXPUg.png</url>
            <title>Stories by Poorshad Shaddel on Medium</title>
            <link>https://medium.com/@p-shaddel?source=rss-8f1fd23dbcf6------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Wed, 18 Nov 2026 19:47:14 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@p-shaddel/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[How We Cut Infra Costs from $4,000 to $100 by Fixing Our Architecture?]]></title>
            <link>https://levelup.gitconnected.com/how-we-cut-infra-costs-from-4-000-to-100-by-fixing-our-architecture-885728e58b5a?source=rss-8f1fd23dbcf6------2</link>
            <guid isPermaLink="false">https://medium.com/p/885728e58b5a</guid>
            <category><![CDATA[postgresql]]></category>
            <category><![CDATA[railways]]></category>
            <category><![CDATA[elasticsearch]]></category>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[kinesis]]></category>
            <dc:creator><![CDATA[Poorshad Shaddel]]></dc:creator>
            <pubDate>Mon, 13 Jul 2026 15:30:26 GMT</pubDate>
            <atom:updated>2026-07-13T15:30:26.923Z</atom:updated>
            <content:encoded><![CDATA[<p>We cut infra costs from $4,000 to $100 by simplifying our over‑engineered AWS setup and moving to a leaner Railway‑based architecture.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wDXVtAyYqogYzNkbMAPqAQ.png" /><figcaption>Sample costs in just one environment</figcaption></figure><p><strong><em>Spoiler</em></strong><em>: We have decided to move our services to </em><strong><em>Railway</em></strong><em> but that was not the only reason we could save, by just moving the same services we could not reduce the costs 40x, main reason was many architecture changes!</em></p><h4>Main Reasons We Have Achieved It</h4><ul><li>Simplifying the system, not designing for every possible scenario</li><li>Doing some trade offs, especially for consistency, some lags(minutes) in our Full-Text search is 100% tolerable.</li></ul><h4>Why Did We Go For Some Changes?</h4><p>The Infrastructure costs were exceeding 4000$ and we got the request to review what is the cause, as first step I reduced the resources of Database and some other backend services deployed on AWS, but they did not have a large effect. it was obvious that <em>we cannot change much with the existing architecture</em>. another idea was to detach our services from AWS and make it deployable almost everywhere, including some bare metal servers and VPSs. we were using Railway for a few small services we have, and we were quite satisfied with what we have done there, also in terms of costs it was reasonable. so for doing the PoC I tried to extract pieces and deploy them on <a href="https://railway.com/"><strong>Railway</strong></a>.</p><p>The general main idea(also my intuition) was to go away from <a href="https://aws.amazon.com/rds/aurora/serverless/"><strong>Serverless Database</strong></a> and <a href="https://aws.amazon.com/fargate/">Serverless Services</a> in AWS to save some money. I tried to do a PoC on Railway to see the cost difference and the performance differences and if it makes sense to do some radical changes or not. during the process of detaching our setup from AWS I was trying out new ideas for each step that resulted in saving even more. In the article you are reading the big chunks that helped us save.</p><h4>Why Moving to Railway?</h4><ul><li>Our services are now capable of being deployed anywhere, including VPS with solutions like <a href="https://coolify.io/">Coolify</a>.</li><li>Straightforward to use, in our opinion enough abstracted enough.</li><li>Better Project Level Separation of Costs for the many small Projects we have.</li></ul><h3>What Are We Doing? (Business)</h3><p>This app is responsible for syncing an important entity to another software, then this data is going to be used for search and also making changes. let’s call it <strong>product</strong>. we have to keep an updated copy of these products in our system, we have to sync them to some other systems, the basic simplified functionality is this:</p><figure><img alt="Architecture of the System, we have Product Source of Truth and we get updates from this and we have a copy of it with some variations and we serve it to other people" src="https://cdn-images-1.medium.com/max/800/1*ETMcDafkJcofKSN5XNSmTg.png" /><figcaption>Architecture</figcaption></figure><p>We have <strong>12 Million</strong> of the same entity, on average we get maximum <strong>1 Million</strong> updates per day that we need to handle, the amount of requests that we get from the clients, which are some mobile apps and web apps are not exceeding more than 100K per day.</p><p>We get into details of our setup on each one of the topics we discuss.</p><h3><strong>Amazon Aurora PostgreSQL(Serverless)</strong></h3><p><strong>Approximate costs: 250–300$ per environment(3 environment: 750–1000$)</strong></p><p>We were not doing massive operation, as I have said, max 1 Million records getting updated, also not boosts of thousands of requests per second, relatively smooth number of requests a couple times per day. the biggest mistake was that we were using the Serverless variations, and another mistake that we could have even fixed on previous setup was that our services were trying to keep a connection to the Database, as a result, our Serverless database could not shut down.</p><p>In the new setup, we have a normal <strong>PostgreSQL on Railway</strong>, we can tune the maximum amount of Memory and CPU, what we figured out is that we do not need more than 8GB of RAM or 2 vCPUs for primary.</p><p>Another good feature that helped us a lot in Dev environment is the <strong>Point in Time Recovery</strong> from Railway.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*flGFrWzWem717mw_CznRlg.png" /><figcaption>Point in Time Recovery in Railway</figcaption></figure><p>We still have normal backups everyday that we can deploy anytime(for now Cross Environment Backup Restore is not possible in Railway).</p><p>What I <em>recommend</em> for HA is this setup on Railway:</p><p><a href="https://railway.com/deploy/postgres-ha">Deploy Postgres HA</a></p><h4>How Could you Restore a Database on AWS?</h4><p>When you try to restore a snapshot it creates a new cluster(new URL!) and the existing one remains as it is, it requires some extra steps until your services use the restored database, something that you do not want, we are doing some pilot migrations on a service last year and the most annoying thing that was happening to us was restoring the system state to a specific timeframe.</p><p>for more info check out this link(if you are a AWS expert and what I am saying is non-sense feel free to comment, I have the courage to admit and fix the article)</p><h4>Recovery Point Objective(RPO)</h4><p>On Railway our <strong>RPO</strong> is 60 seconds, with the PITR feature.(AWS Aurora offers even better than that it is couple of seconds)</p><h4>Recovery Time Objective(RTO)</h4><p>For Railway it is minutes to hours based on size of the Data, our Data is a couple of Gigabytes so probably it takes minutes. AWS Recovery time is also minutes.</p><p><a href="https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/aurora-pitr.html">Restoring a DB cluster to a specified time</a></p><h3>AWS Database Migration Service</h3><p><strong>Almost 150$ per environment for DMS + Kinesis + Lambda functions (all together maybe around 500–600$)</strong></p><p>We are using Elastic Search for searching our entities, so we had to</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/574/1*6Ka-ZdN8GqEkXs7uNpp6Bw.png" /><figcaption>Syncing Postgres to Elastic Setup</figcaption></figure><p>In the new structure, we are not reading the WAL stream, but whenever an incoming change is coming we write it first to our Postgres Database and if it succeeds then we write it to Elastic Search(the chances that writing to Elastic fails, does not differ much from the process we have in that Lambda function, in here at least we have a re-try process + we have a cron job to sync our Database to Elastic anyway). but anyway if we cannot write to Elastic successfully we log the specific event and the reason it failed.(we rely on our cron job to fix it)</p><p>Our cron job is doing an incremental sync, based on the field “updated_at” in the database, for the current load, it takes 15 minutes mainly because it is not finding a lot of mismatches between Postgres and Elastic.</p><p><strong>If we change the Database directly, we have to execute our cron job to sync DB to elastic.</strong></p><p>We are aware of the <strong>consistency issues</strong>, but we can tolerate the inconsistencies in the duration of each cron job(if there are any, so far we did not have the case) for making sure that DB is in Sync.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/724/1*F_JUqdUZAWld_Fg51ZYZ-Q.png" /><figcaption>Manual Syncing of Elastic Search</figcaption></figure><p><strong>It’s okay if search results lag up to 5 minutes behind the source system and it is an intentional trade off we have made.</strong></p><p><em>We could do the same thing on AWS as well and remove the DMS, Kinesis.</em></p><h3>AWS ECS Fargate</h3><p><strong>~300$ per environment for this, around 1000$ just for ECS + Load Balancer + NAT costs.</strong></p><h4>Why ECS Fargate is expensive?</h4><p>because it is serverless and usually 50–60% more expensive if we have the same resource on EC2.</p><h4>Did it make sense for our traffic to use Fargate?</h4><p>No, since we do not have a huge difference in load, it did not save us money, if we would have designed for the max number of requests we would have still save money.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4oJ0KA7EAZeiChaVnltLJw.png" /><figcaption>Number Requests</figcaption></figure><p>We did not use saving plans; if we had, our AWS costs would have been lower, but still significantly above our new setup</p><h4>Did we need 4 active instances at the same time?</h4><p>Definitely not, we are not consuming more than 2GB of memory on the application, and we have never exceeded the 0.4 vCPU, this could have <strong>been improved on our AWS Setup as well</strong>.</p><h3>Async Pattern in our Apps</h3><p>This one was not that expensive, but still waste of resources. Lambda functions, specially our Lambda functions which were Javscript were just doing some network calls, or a Database call. we could do that in our main app as well.</p><figure><img alt="Async Pattern in our old Architecture" src="https://cdn-images-1.medium.com/max/423/1*dIvnWAqzuTVAZeGacTe7ZQ.png" /><figcaption>Async Pattern in our old Architecture</figcaption></figure><p>In our new Setup we use <strong>Redis(But Persistent to not lose any job)</strong> and <strong>BullMQ</strong> to execute these jobs, no back pressure at all, we handle everything much quicker than the Lambda functions and we can handle our current load without back pressure or <a href="https://nodejs.org/learn/asynchronous-work/event-loop-timers-and-nexttick">Event Loop</a> Delay. For handling the jobs we did not see any abnormal spikes in our CPU and Memory of the main app, but the workers are decoupled and they can be deployed separately as another Nodejs app or a serverless deployment if we need to scale them separately.</p><h3>Costs at the moment</h3><p>We still have to shut down some services and remove the NAT and VPC on AWS, once we are done with moving everything over, but most expensive services are now running on Railway with the new architecture and the new setup is just 100$ a month, something that we could not believe at the beginning.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*xfS4gBytjs7ip7DjZe2mkg.png" /><figcaption>Railway costs</figcaption></figure><h3>Was it all AWS’s Fault?</h3><p>No in my opinion, we could have saved a ton by just doing these simplifications on AWS, but for sure we could not reach to such a number(100$). just having a VPC on AWS costs 50$ per environment.</p><h3>Are We Satisfied with Railway?</h3><p>It is super easy to deploy and use Railway, our whole team was able to <strong>adopt very fast</strong> to deploy everything on Railway, we have everything in just one place(logs, metrics, admin operations).</p><p>We can now distinguish different projects costs and charge business more accurately.</p><p>Railway experienced a few Incidents that mainly made the dashboard unavailable, or in another case the logs were loading slow but we were fortunately not affected by the recent ones, but something is clear, they are growing fast and I hope that reliability becomes their number 1 priority.</p><p>One major recent event:</p><p><a href="https://www.reddit.com/r/webdev/comments/1ti37ys/railway_is_down/">https://www.reddit.com/r/webdev/comments/1ti37ys/railway_is_down/</a></p><p><a href="https://status.railway.com/">Railway Status</a></p><h3>Conclusion</h3><p>For our steady traffic, designing for peak load was cheaper and simpler than building fully elastic, auto‑scaling infrastructure. If we can make some trade‑offs, we can go for a simpler design. Everyone should be explicit about the inconsistencies they can tolerate</p><h3>How LLMs Were Used to Write this Article?</h3><p>No part of the text is AI Generated, I have used AI to find typos and some mistakes, I used Google AI Mode Search to gather some information and do some double checks.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=885728e58b5a" width="1" height="1" alt=""><hr><p><a href="https://levelup.gitconnected.com/how-we-cut-infra-costs-from-4-000-to-100-by-fixing-our-architecture-885728e58b5a">How We Cut Infra Costs from $4,000 to $100 by Fixing Our Architecture?</a> was originally published in <a href="https://levelup.gitconnected.com/">Level Up Coding</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to Protect against Supply Chain Attacks like the one that happened to TanStack]]></title>
            <link>https://levelup.gitconnected.com/how-to-protect-against-supply-chain-attacks-like-the-one-that-happened-to-tanstack-3361a8b9f0de?source=rss-8f1fd23dbcf6------2</link>
            <guid isPermaLink="false">https://medium.com/p/3361a8b9f0de</guid>
            <category><![CDATA[supply-chain-attack]]></category>
            <category><![CDATA[buns]]></category>
            <category><![CDATA[security]]></category>
            <category><![CDATA[pnpm]]></category>
            <category><![CDATA[npm]]></category>
            <dc:creator><![CDATA[Poorshad Shaddel]]></dc:creator>
            <pubDate>Fri, 15 May 2026 15:32:27 GMT</pubDate>
            <atom:updated>2026-05-15T15:32:27.039Z</atom:updated>
            <content:encoded><![CDATA[<p>Takes 3 Minutes to read this article and act if your repository could be affected. no AI slop hype.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/732/1*2mquCDANps8100i-ySA7yQ.png" /><figcaption>Options in just one view</figcaption></figure><p>If you want to read the story, take a look at this postmortem article from Tanstack.</p><p><a href="https://tanstack.com/blog/npm-supply-chain-compromise-postmortem">Postmortem: TanStack npm supply-chain compromise | TanStack Blog</a></p><h3>What is a Supply Chain Attack?</h3><p>In Javascript world, this is happening more often recently. The Attackers take control of the repository, they release new verified versions and once you install the package, usually a malicious code gets injected to the host, and does the attackers actions(usually looking for other valid Github Token to spread itself, looks for cryptographic keys to wallets and similar things).</p><p>Usually the maintainers and others find it out really soon, but if you have installed the package automatically, it might be too late for you.</p><h3>What Can We Do?</h3><p>A simple action that can reduce the chance of getting caught by these kind of attacks is to:</p><blockquote>Not installing the latest packages automatically and blindly.</blockquote><p>One very effective trick would be to define a threshold, <em>for us this threshold is 3 days to install the packages, in other words</em>, <strong>we only install packages that are more than 3 days old!</strong></p><p>Let’s see how we do it in Bun, pnpm and npm.</p><h4>Bun</h4><p>In bun we can define that in the config or use it when installing something. the minimumReleaseAge is in <strong>seconds</strong>.</p><p><a href="https://bun.com/docs/runtime/bunfig#install-minimumreleaseage">bunfig.toml - Bun</a></p><pre># bunfig.toml<br><br>[install]<br>minimumReleaseAge = 259200<br># These packages will bypass the 3-day minimum age requirement<br>minimumReleaseAgeExcludes = [&quot;@types/bun&quot;, &quot;typescript&quot;]</pre><h4>pnpm</h4><p>In pnpm you define the same thing but in <strong>Minutes</strong>. before pnpm v11 the default value was 0, but since v11, default value is 1440 which is a day!</p><pre>minimumReleaseAge: 1440<br>minimumReleaseAgeExclude:<br>- webpack<br>- react</pre><p><a href="https://pnpm.io/settings#minimumreleaseage">Settings (pnpm-workspace.yaml) | pnpm</a></p><h4>npm</h4><p>this config is min-release-age you can add it to the npmrc. this config is in <strong>days</strong>. from npm version 11.10 you have this option.</p><h4>Yarn</h4><p>you have the option: npmMinimalAgeGate, in Minutes.</p><p><strong>In most cases 24h is enough in my opinion, but I prefer 3 days, and if I need to specifically install something, I exclude it from this check.</strong></p><p>Something else that I really like to do is to develop on a Github Codespace or a separate container, this could save me from these kind of attacks, but obviously I cannot do that for all the cases.</p><p><em>That was it, keep your packages safe!</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3361a8b9f0de" width="1" height="1" alt=""><hr><p><a href="https://levelup.gitconnected.com/how-to-protect-against-supply-chain-attacks-like-the-one-that-happened-to-tanstack-3361a8b9f0de">How to Protect against Supply Chain Attacks like the one that happened to TanStack</a> was originally published in <a href="https://levelup.gitconnected.com/">Level Up Coding</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to Correlate Slow Queries in Postgres and other Databases to our Backend Application?]]></title>
            <link>https://levelup.gitconnected.com/how-to-correlate-slow-queries-in-postgres-and-other-databases-to-our-backend-application-fb1d6483026a?source=rss-8f1fd23dbcf6------2</link>
            <guid isPermaLink="false">https://medium.com/p/fb1d6483026a</guid>
            <category><![CDATA[postgres]]></category>
            <category><![CDATA[distributed-tracing]]></category>
            <category><![CDATA[nodejs]]></category>
            <category><![CDATA[observability]]></category>
            <category><![CDATA[database]]></category>
            <dc:creator><![CDATA[Poorshad Shaddel]]></dc:creator>
            <pubDate>Mon, 22 Dec 2025 02:12:42 GMT</pubDate>
            <atom:updated>2025-12-22T02:12:42.694Z</atom:updated>
            <content:encoded><![CDATA[<p>You spot a slow query in Postgres. Great. But which microservice ran it? Which route? Which controller? If you’re using an ORM, that trail is usually hidden. Let’s fix that. Via SQL comments (aka query tags) that tie queries back to your application code.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*YXA1P54kL8mSFE-z6MNdHA.png" /><figcaption>Tracing Slow Queries</figcaption></figure><h3>The Problem We Want to Solve</h3><p>In a case that we were using AWS RDS, we had some really slow queries and we were able to monitor them in AWS Console like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_1wvpYL6VWjFjQQ7wqcv9g.png" /><figcaption>AWS RDS Performance</figcaption></figure><p>We had the problematic queries but we did not have the exact Micro-Service, controller or service. What we were doing was to guess were it could be based on our experience on the project and then trying it locally and checking the logs of the local database to see if it is the right place. This process is usually time consuming and error prone when you want to narrow it down to the actual problem. In this article we want to discuss how we can use a mechanism to remove this manual process and directly get to the code.</p><h3>What We Need to Correlate Queries and our Backend Application?</h3><p>Like other cases we need a <strong>Trace</strong>! This trace needs to be added to the query and include information about the execution path of the query from the Backend Application.</p><p>For tracing and observability there are some well-know tools like <a href="https://opentelemetry.io/">Open-Telemetry</a> that worth taking a look at, if this is the first time you are hearing about them, but here we do not want to dive into that world.</p><p><strong>For a simple case, we need a unique ID(which is most of the time called TraceID) somehow attached to our Query!</strong></p><h3>How to Attach the Trace ID or Metadata to our Query?</h3><p><strong><em>Super simple: Using SQL Comments!</em></strong></p><p>If this query gets executed in the database:</p><pre>select * from users /*traceId:&#39;61e8960c349ca2a3a51289e050fd3b82&#39; controller:UsersController username:jsmith */</pre><p>This is a sample that has <strong>traceId</strong> and some other extra information that might be useful.</p><p>The concept is pretty easy as you have seen.</p><p>Let’s run a Postgres Docker Container with enabled logging:</p><pre># docker-compose.yml<br>version: &#39;3.8&#39;<br><br>services:<br>  postgres:<br>    image: postgres:18<br>    container_name: query-tracing-postgres<br>    environment:<br>      POSTGRES_DB: querytracing<br>      POSTGRES_USER: postgres<br>      POSTGRES_PASSWORD: password<br>      POSTGRES_INITDB_ARGS: &quot;-c log_statement=all -c log_destination=stderr -c log_line_prefix=&#39;%t [%p]: [%l-1] user=%u,db=%d,app=%a,client=%h &#39;&quot;<br>    ports:<br>      - &quot;5432:5432&quot;<br>    volumes:<br>      - ./db_data:/var/lib/postgresql<br>    restart: unless-stopped</pre><p>Create the Database querytracing and a simple users table:</p><pre>      CREATE TABLE IF NOT EXISTS users (<br>        id SERIAL PRIMARY KEY,<br>        name VARCHAR(100),<br>        active BOOLEAN<br>      );</pre><p>Now if we run a simple code using <strong>bun</strong> and its SQL driver:</p><pre>import { SQL } from &quot;bun&quot;;<br>const sql = new SQL(&quot;postgresql://postgres:password@localhost:5435/querytracing&quot;);<br>await sql`<br>  INSERT INTO users (name, active)<br>  VALUES (${ &quot;Bob&quot; }, ${ false });/*trace-id:12345,user-id:67890*/<br>`;</pre><p>As you can see we added the Trace ID as a comment and in the logs of Postgres Container we can see this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/862/1*p0_9bl_l2Kd61lbv4iK3AQ.png" /><figcaption>Logs of Postgres</figcaption></figure><h3>What is SQL Commenter?</h3><p>You have seen that I have used a comma separated style for commenting and this was just my decision, everybody can have their own way of commenting trace information. The idea behind <a href="https://google.github.io/sqlcommenter/">SQL Commenter</a> is to use a format like this: /* field1=&#39;value1&#39;,field2=&#39;value2 */ and they are providing tool sets that you can use with your ORM to add desired tracing info to the queries generated by the Database.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9OqEnD0qqz8zcG4qoMq3DQ.png" /><figcaption>Sample of How the Tools Add Comments</figcaption></figure><p>If you are using your own custom trace_id, make sure you are taking a look at the <a href="https://www.w3.org/TR/trace-context/#considerations-for-trace-id-field-generation">W3C Tracing Recommendation</a>.</p><p>Now we are going to take a look at some Nodejs ORMs for Postgres.</p><h3>Nodejs ORMs and Support for Query Tracing</h3><h4>Prisma</h4><p>We can add comments to Prisma queries using these two packages: @prisma/sqlcommenter-query-tags and @prisma/sqlcommenter-trace-context</p><p>Add them to your client:</p><pre>const prisma = new PrismaClient({<br>  adapter,<br>  comments: [queryTags(), traceContext()],<br>})</pre><p>Now you can run queries with desired comment(or as Prisma says: metadata):</p><pre>import { queryTags, withQueryTags } from &#39;@prisma/sqlcommenter-query-tags&#39;<br>import { PrismaClient } from &#39;../prisma/generated/client&#39;<br><br>const prisma = new PrismaClient({<br>  adapter,<br>  comments: [queryTags()],<br>})<br><br>const users = await withQueryTags(<br>  { route: &#39;/api/users&#39;, requestId: &#39;abc-123&#39; },<br>  () =&gt; prisma.user.findMany()<br>)</pre><h4>Prisma Integration for Sentry</h4><p>Sentry is a error tracking and performance monitoring platform and they have their own integration for tracing. For enabling it on Prisma it is as easy as adding this piece of code:</p><pre>Sentry.init({<br>  tracesSampleRate: 1.0,<br>  integrations: [Sentry.prismaIntegration()],<br>});</pre><p>Prisma has a detailed documentation on how you can add comments and create your own custom plugins for doing that:</p><p><a href="https://www.prisma.io/docs/orm/prisma-client/observability-and-logging/sql-comments">SQL comments | Prisma Documentation</a></p><h4>Sequelize.JS</h4><p>The SQL Commenter has a tool to use with Sequelize, here is how to use it:</p><p>You can do it via using this package: @google-cloud/sqlcommenter-sequelize</p><pre>const {wrapSequelize} = require(&#39;@google-cloud/sqlcommenter-sequelize&#39;);<br>const Sequelize = require(&#39;sequelize&#39;);<br><br>// Create the sequelize client.<br>const sequelize = new Sequelize(options);<br><br>// Finally wrap the sequelize client.<br>wrapSequelize(sequelize);</pre><p>There are some other tricks to inject your comments as well, like using Sequelize.literal</p><pre>Model.findById(id, {<br>   attributes: {<br>     include: [<br>       [Sequelize.literal(&#39;/* your comment */ 1&#39;), &#39;an_alias&#39;],<br>     ],<br>   },<br> });</pre><p><a href="https://google.github.io/sqlcommenter/node/sequelize/">sqlcommenter</a></p><h4>TypeORM</h4><p>For TypeORM there is a method called comment that I have found from a PR, you can add your tracing info using this method:</p><pre>     const sql = connection.manager.createQueryBuilder(Test, &quot;inserting&quot;)<br>            .insert()<br>            .values({ id: 1 })<br>            .comment(&quot;field1=value1&quot;)<br>            .getSql();</pre><ul><li><a href="https://github.com/typeorm/typeorm/issues/3643">Query Comments · Issue #3643 · typeorm/typeorm</a></li><li><a href="https://github.com/google/sqlcommenter/issues/43">feat(support) TypeORM · Issue #43 · google/sqlcommenter</a></li><li><a href="https://github.com/typeorm/typeorm/pull/6892">feat: support query comments in the query builder by imnotjames · Pull Request #6892 · typeorm/typeorm</a></li></ul><h3>Do you Really Need Tracing in your DB Queries?</h3><p>If you have a Database that you are the only one executing raw sql queries from CLI or a DBMS Software probably you do not need to track your queries because if there is something wrong with your query, you see it immediately. <strong><em>But as an example if you have a web application with many controllers, or you have a Distributed System with different Microservices, it might be a good idea to Track your SQL Queries</em></strong>.</p><ul><li><a href="https://postgresmonitor.com/docs/query-tags/">Postgres Monitor</a></li><li><a href="https://cloud.google.com/blog/topics/developers-practitioners/enable-query-tagging-sqlcommenter-understand-application-impact-database-performance">Enable query tagging with Sqlcommenter to understand application impact on database performance | Google Cloud Blog</a></li><li><a href="https://google.github.io/sqlcommenter/">sqlcommenter</a></li><li><a href="https://www.w3.org/TR/trace-context/">Trace Context</a></li></ul><p><a href="https://docs.sentry.io/platforms/javascript/guides/express/configuration/integrations/prisma/">https://docs.sentry.io/platforms/javascript/guides/express/configuration/integrations/prisma</a></p><p><a href="https://www.prisma.io/docs/orm/prisma-client/observability-and-logging/sql-comments">https://www.prisma.io/docs/orm/prisma-client/observability-and-logging/sql-comment</a>s</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=fb1d6483026a" width="1" height="1" alt=""><hr><p><a href="https://levelup.gitconnected.com/how-to-correlate-slow-queries-in-postgres-and-other-databases-to-our-backend-application-fb1d6483026a">How to Correlate Slow Queries in Postgres and other Databases to our Backend Application?</a> was originally published in <a href="https://levelup.gitconnected.com/">Level Up Coding</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Node.js Unhandled Rejections: throw, warn, strict — Explained in 5 Minutes]]></title>
            <link>https://levelup.gitconnected.com/node-js-unhandled-rejections-throw-warn-strict-explained-in-5-minutes-654b123c91ec?source=rss-8f1fd23dbcf6------2</link>
            <guid isPermaLink="false">https://medium.com/p/654b123c91ec</guid>
            <category><![CDATA[error-handling]]></category>
            <category><![CDATA[node-options]]></category>
            <category><![CDATA[unhandledpromiserejection]]></category>
            <category><![CDATA[nodejs]]></category>
            <category><![CDATA[promises]]></category>
            <dc:creator><![CDATA[Poorshad Shaddel]]></dc:creator>
            <pubDate>Mon, 03 Nov 2025 03:09:14 GMT</pubDate>
            <atom:updated>2025-11-03T03:09:14.013Z</atom:updated>
            <content:encoded><![CDATA[<h3>Node.js Unhandled Rejections: throw, warn, strict — Explained in 5 Minutes</h3><p>Unhandled promise rejections can be noisy, confusing, or dangerous — depending on how Node.js is configured. In this quick guide, we’ll break down what “unhandled rejections” actually are, what each mode does (throw, warn, strict, warn-with-error-code, none), and when to pick which one. You’ll see simple examples, learn how the hooks work, and leave with a practical setup you can trust</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zHeQTSXBc6nTzRSJDCPYTA.jpeg" /><figcaption>Nodejs Unhandled Rejections, throw warn, strict</figcaption></figure><h3>What is an Unhandled Rejection?</h3><p>Usually refers to a Promise that fails and does not have a catch phrase.</p><p>A simple example of an unhandled rejection:</p><pre>async function dbConnection() {<br>    return Promise.reject(new Error(&quot;Failed to connect to database&quot;));<br>};<br><br>dbConnection();</pre><p>dbConnection is a function that returns a promise, but does not have a catch phrase, so if we run this we get an error and the program exits with code 1.</p><h3>Available Options</h3><p>In Node.js we have multiple options for different handlings. We can pass this option when running Node.js: --unhandled-rejections=mode . this option was introduced in Node.js v10.17. Available options are:</p><ul><li>throw(Default from v15.0.0)</li><li>warn(Default from v10.17.0 to v15.0.0)</li><li>strict</li><li>warn-with-error-code</li><li>none</li></ul><h4>throw</h4><p>This is the default option now, if you are using nodejs without setting this option, this is the option you are using. it simply throws the error and stops your program.</p><p>Let’s see that in a simple express app, that asyncronously connects to the Database, but it cannot connect to the database and it fails:</p><pre>const app = require(&#39;express&#39;)();<br>const port = process.env.PORT || 4001;<br>app.get(&#39;/&#39;, (_req, res) =&gt; {<br>  res.send(&#39;Hello World!&#39;);<br>});<br>app.listen(port, () =&gt; {<br>  console.log(`Server is running on port ${port}`);<br>});<br>async function stablishDatabaseConnection() {<br>    return new Promise((_, reject) =&gt; {<br>        setTimeout(() =&gt; {<br>            console.log(&quot;Trying to connect to database...&quot;);<br>            reject(new Error(&quot;Failed to connect to database&quot;));<br>        }, 1000);<br>    });<br>};<br>stablishDatabaseConnection();</pre><p>We can add this piece as well to see the exit code:</p><pre>process.on(&#39;exit&#39;, (code) =&gt; {<br>    console.log(`Process exiting with code: ${code}`);<br>});</pre><p>Now if we run this program, it exists after the first second.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/722/1*ds6e-2Q3AKA0iXjMi_cnvw.gif" /><figcaption>Running Nodejs with default option for Unhandled Rejections = throw</figcaption></figure><p><strong>UnhandledRejection Hook</strong></p><p>When Node captures these unhandled rejections it emits an event that we can get via a Hook, listening for it:</p><pre>process.on(&#39;unhandledRejection&#39;, (reason, promise) =&gt; {<br>    console.error(&#39;Unhandled Rejection at:&#39;, promise, &#39;reason:&#39;, reason);<br>});</pre><p><strong><em>The Most Important thing is that if you listen to this event, your program is not exiting anymore, you are responsible for handling what should happen next</em></strong>. As an example if you want to exit the program, you can use this:</p><pre>process.on(&#39;unhandledRejection&#39;, (reason, promise) =&gt; {<br>    console.log(&#39;Unhandled Rejection at:&#39;, promise, &#39;reason:&#39;, reason);<br>    process.exit(1);<br>});</pre><p>If there is an Error thrown, the reason is then the Error. in this case it would be new Error(“Failed to connect to database”) .</p><h4>warn</h4><p>This time we run it using this command:</p><pre>node --unhandled-rejections=warn ./server.js</pre><p>This time, our express app keeps working, but it shows a warning:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/722/1*Up3H-rWYHlvW50z1EGPNuA.gif" /><figcaption>Express App keep running when we use “warn”</figcaption></figure><p><strong><em>Most important thing about this option is that, does not matter if you listen to unhandled rejections or not, you get the warning!</em></strong></p><h4>warn-with-error-code</h4><p>If the unhandled rejection hook is not set, it does 2 things:</p><ul><li>Show a warning</li><li>Set the exit code to 1(without exiting the program)</li></ul><p>We can set the exit code, without actually exiting the program using this:</p><pre>process.exitCode = 1;</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/836/1*c6ZUl_cEFqhK0DZ7Ky_HrA.gif" /><figcaption>warn-with-error when hook is not set. shows a warning and set the exit code</figcaption></figure><p>As you can see in the other timeout function, I log the exit code and it is 1.</p><p><strong>If we set the hook for unhandled rejections, it does not show any warning and also it does not set the exit code.</strong></p><h4>none</h4><p>Pretty obvious, it does not do anything when an unhandled rejection happens. <strong>The unhandled rejection event is emitted no matter what, so you can handle it there.</strong></p><h4>strict</h4><p>It raises the unhandled rejection as a <strong>uncaught exception</strong>. if the exception is handled, then unhandled rejection is emitted. Let’s explain them one by one!</p><p><strong>What is a uncaught exception?<br></strong>It is a error that is not handled.</p><p>Super simple example:</p><pre>function connectDB() {<br>    throw new Error(&quot;Database connection failed&quot;);<br>};<br><br>connectDB();</pre><p>If we run this we get an exception. we can also register a hook for getting this event as well:</p><pre>function connectDB() {<br>    throw new Error(&quot;Database connection failed&quot;);<br>};<br><br>process.on(&#39;uncaughtException&#39;, (err) =&gt; {<br>    console.log(&#39;Uncaught Exception:&#39;, err.message);<br>});<br><br>connectDB();</pre><p>Now let’s back to our main application.</p><p>We have our unhandled rejection hook but still the application stops running because of the uncaught exception:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/836/1*HVq9oXr2IWNO_NToJhgcRA.gif" /><figcaption>unhandled-rejections=strict stops the program.</figcaption></figure><p>But if we handle the uncaught exception, it emits the unhandled rejection afterwards.</p><p>Now we are listening on three events:</p><pre>process.on(&#39;unhandledRejection&#39;, (reason, _promise) =&gt; {<br>    console.log(&#39;Unhandled Rejection at capture:&#39;, reason.message || reason);<br>});<br>process.on(&#39;exit&#39;, (code) =&gt; {<br>    console.log(`Process exiting with code: ${code}`);<br>});<br>process.on(&#39;uncaughtException&#39;, (err) =&gt; {<br>    console.log(&#39;Uncaught Exception:&#39;, err.message);<br>});</pre><p>Let’s see the result with strict option:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/836/1*WRk3wLjGiyN9WteDlH09AQ.gif" /><figcaption>Use Strict Option with the uncaught exception handler</figcaption></figure><p>Now first we get uncaught exception events, and after that we get the unhandled rejection.</p><p>Be careful that if you add a catch phrase to your promise:</p><pre>stablishDatabaseConnection().catch((err) =&gt; {<br>    console.log(&#39;Database connection error caught:&#39;, err.message);<br>});</pre><p><strong>You are not getting the unhandled rejection event anymore.(does not matter which option you are using)</strong></p><h3>When To Use Which Option and My Opinion</h3><p>For a general use case like a simple web app, doing some simple CRUD operation, I think, the default option which is throw makes a lot sense. it might stop your application but you can capture the possible issues.</p><p>If you have a use case that you run a lot of things in the background as async operations and time to time they fail and you do not want your app to stop, you should either add a catch function for each one of them or use a warn , if it needs some extra handling when fails, simply add the hook for handling those ones. simlarly you can use warn-with-error-code .</p><p>When you have an application with critical promises(without catch phrase) that each one of those ones are critical for running the app in a healthy state, you can go for strict so that you get uncaught exception .</p><p>In my opinion, each Promise need to have a catch phrase. It makes the code clean and you do not need to capture the specific event in a separate hook. so for a general web app use, I would go with default option + catching the rejections and preventing events like unhandled rejection or caught exception.</p><h3>How I used AI for writing this Article</h3><p>For this article I barely used AI, just using hiting Tab a few times, when I was writing the code on my IDE, as it is enabled by default. I also used AI for finding a better title for my article. The image for the article is also generated via Gemini.</p><h3>Resources</h3><p><a href="https://nodejs.org/api/cli.html#--unhandled-rejectionsmode">https://nodejs.org/api/cli.html#--unhandled-rejectionsmode</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=654b123c91ec" width="1" height="1" alt=""><hr><p><a href="https://levelup.gitconnected.com/node-js-unhandled-rejections-throw-warn-strict-explained-in-5-minutes-654b123c91ec">Node.js Unhandled Rejections: throw, warn, strict — Explained in 5 Minutes</a> was originally published in <a href="https://levelup.gitconnected.com/">Level Up Coding</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Why Databases Don’t Use Plain B-Trees for Indexes]]></title>
            <link>https://levelup.gitconnected.com/why-databases-dont-use-plain-b-trees-for-indexes-898176e74045?source=rss-8f1fd23dbcf6------2</link>
            <guid isPermaLink="false">https://medium.com/p/898176e74045</guid>
            <category><![CDATA[mongodb]]></category>
            <category><![CDATA[b-tree-index]]></category>
            <category><![CDATA[postgresql]]></category>
            <category><![CDATA[database-indexing]]></category>
            <category><![CDATA[database]]></category>
            <dc:creator><![CDATA[Poorshad Shaddel]]></dc:creator>
            <pubDate>Mon, 03 Nov 2025 03:09:05 GMT</pubDate>
            <atom:updated>2025-11-03T03:09:05.362Z</atom:updated>
            <content:encoded><![CDATA[<p>Many developers believe that databases use a standard B-Tree for indexing — but that’s a misconception. In reality, modern systems (like PostgreSQL) use specialized variations designed for performance and concurrency. Let’s explore why a plain B-Tree isn’t enough.</p><h4>What you would not see in this article</h4><ul><li>Details of implementing Binary Search Tree, B-Tree, B+-Tree or B*-Tree</li></ul><h3>What is an Index in the Database?</h3><p>Without an index in a book, you have to go through the whole pages, before finding what you want. Databases use the same concept so that they don’t go over all pages(page is indeed a concept in database as well, a logical block that fits data in it).</p><h3>Binary Search Tree</h3><p>We probably know that if we want to retrieve something from a sorted array, the most efficient way to do it is to to use Binary Search Tree.(BST)</p><p>BST is a tree structure that has 2 rules:</p><ul><li><em>All nodes on the left subtree are strictly less than the parent node.</em></li><li><em>All nodes on the right subtree are greater then the parent node.</em></li></ul><p><strong>Each node can have maximum of 2 child nodes.</strong></p><p><a href="https://en.wikipedia.org/wiki/Big_O_notation"><strong>Complexity</strong></a><strong> of operations:</strong></p><ul><li>Finding an Element: <strong>O(log n)</strong></li><li>Insertion: <strong>O(log n)</strong></li><li>Deletion: <strong>O(log n)</strong></li></ul><h4>Example of finding 14 in this BST</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/946/1*HUn51wOYo2RRzuK_eN_ENg.gif" /><figcaption>Finding 14 in a Binary Search Tree</figcaption></figure><h3>Why Can’t we use Binary Search Tree for Database Indexes?</h3><p>Main reason is that Database Index is stored on Disk(Secondary Storage). in simple words the height of our Tree becomes an important factor and it correlates with more IO requests to the Disk before reaching to actual data. Database usually caches the index to use it efficiently, but there are many situations that you cannot fit the index into memory. Even if you can fit your Tree to the memory, it does not mean that a BST outperforms a B-Tree, it depends on the fact that if you can fit your Tree in <a href="https://en.wikipedia.org/wiki/CPU_cache">Cpu’s L1, L2 Caches</a>. L1 cache and L2 cache hit are both under 5 ns, but RAM access is 50–100 ns and SSD data access is 100,000 ns.</p><h3>B-Tree Index</h3><p>B-Tree indexes are invented by <a href="https://de.wikipedia.org/wiki/Rudolf_Bayer_(Informatiker)">Rodulf Bayer</a> and <a href="https://de.wikipedia.org/w/index.php?title=Edward_M._McCreight&amp;action=edit&amp;redlink=1">Edward M. McCreight</a> B-Tree is similar to Binary Search Tree with the difference that <strong>it allows more children under one node</strong> with the intention of <strong>reducing the height of tree</strong> <strong>which results in storing data in fewer blocks</strong>. B-Tree was initially invented at Boeing company for better random access on disk.</p><p>A B-Tree has an order m(the maximum number of children per node.)</p><p>Conditions of having a B-Tree Index:</p><ul><li>Every node has at most m children</li><li>Every node should have at least m/2 node(except root and leaf nodes)<em>[This rule is there to keep the tree height low — if we have empty nodes, then the tree becomes tall and similar to Binary Search Tree]</em></li><li>All leaves appear on the same level</li><li>A non-leaf node with <em>k</em> children contains <em>k</em>−1 keys.(<em>if you have 2 children at your node, for example 10, 20 then you have 3 spaces: less than 10, between 10 and 20, greater than 20</em>)</li></ul><p>Here is a B-Tree of order 5.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/1*FRyY4EkIxSO1h53qDThtiw.png" /><figcaption>B-Tree — Order 5</figcaption></figure><p>Process of finding an Item:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/1*Y6OM_Vr1IchiwFpAnNLhNw.gif" /><figcaption>Finding 110 in B-Tree — order 5</figcaption></figure><p><a href="https://en.wikipedia.org/wiki/Big_O_notation"><strong>Complexity</strong></a><strong> of operations:</strong></p><ul><li>Finding an Element: <strong>O(log n)</strong></li><li>Insertion: <strong>O(log n)</strong></li><li>Deletion: <strong>O(log n)</strong></li></ul><p><strong>The most important thing to know is that in B-Tree as you can see, is that the internal nodes contain the keys and values. We later see why this could hurt performance when reading from the Disk.</strong></p><h3>B+Tree Index</h3><p>This is a B-Tree with the difference that each node contains <strong>only the Keys</strong>, not the values. we add <strong>another layer</strong> at the bottom, that <strong>contains values</strong>.</p><p><em>internal records are copies used only for routing to actual records(Bottom Layer).</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*fVKHCxGw4b6NA93EV1wnYw.png" /><figcaption>Same Values but in a B+Tree</figcaption></figure><h4>Advantages of B+ Tree</h4><ul><li><strong>Fitting more data</strong>: because in internal nodes, you just have the keys, you can fit more data to memory, which means fewer cache miss.</li><li><strong>Full-Scan</strong>: you have all the leaf nodes linked together, it is like a linked list that you want to navigate.(in a B-Tree you have to go up and down the height of tree to get the full-list)</li><li><strong>Range</strong>: if you are targeting a specific range, once you found the lower bound, you can sequentially navigate the elements till you get to the upper range.</li></ul><p>See the advantage when we want to navigate all elements:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*VzXugZPBscRdRCl3q6yQfw.gif" /><figcaption>B+ Tree Full Scan — Advantage</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*TCwa1fXV5KGN8J1UxH3r9w.gif" /><figcaption>B-Tree Full-Scan — Disadvantage</figcaption></figure><h4>Disadvantages of B+ Tree vs B Tree</h4><ul><li><strong>Navigating to Bottom Layer</strong>: Even if the element you are looking for is really near the root, you have to get to the bottom layer to find out the value associated with that.</li></ul><p>See the example below for the disadvantage:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-45CoYMQ2dhg06cwGTrUCg.gif" /><figcaption>Finding an item near the root in B+ Tree</figcaption></figure><h3>B* Tree</h3><p>B-Start Tree is a B+ Tree with some extra rules, <strong><em>that each node is at least two-thirds full instead of half full</em></strong>.(We have seen in B Tree definition rules that each node should have at least m/2 children.)</p><p>In short: B* Tree design makes the tree more compact(which means less Disk or RAM I/O)</p><p>In a B-Tree or B+ Tree if the node is becoming full it divides to two different nodes with half capacity, but in a B* Tree it shares data to neighbor nodes to keep the restriction of 2/3 capacity.</p><p>Complexity of operations are similar to B+ Tree, but the height could be smaller due to the fact that the tree is 2/3 full.</p><h3>Lehman Yao B-Tree(Blink Tree)</h3><p>This data structure was developed for usage in Databases. The problem there is that we don’t have just one process going over tree to find, insert or delete, we have multiple processes doing that. to ensure index data stays consistent we have to either lock a path that we are working on, or in worst case lock the whole tree. <strong>to solve this issue</strong></p><ul><li><strong>it adds one extra pointer to each node, this points to the next right node at the same level of the tree(at all levels of the tree)</strong></li><li><strong>Lehman Yao Tree also keep another value for each node: High Key, this is the maximum key the node has.</strong></li></ul><p>See the image from the paper</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/842/1*V57Sdggoko8SV14LKoXTLA.png" /><figcaption>Right Link Pointer in Lehman Yao B-Tree</figcaption></figure><h4>But how it solves the issue of concurrency by adding a pointer to the sibling at the same level plus having a High Key?</h4><p>Having a pointer to the right, helps the process to recover in case there was change like a split page, it can still navigate to the new or changed pages.<br>Also having the High Key helps you check if the value you are looking for exists in the current page, or for example the page was splitted and you have to look into the sibling pages. you don not need any read locks during a search.</p><h4>Why Pointer is Left to Right?</h4><p>Because in case of a page split, larger keys are going to be place to the right.</p><pre>Before:  [10 | 20 | 30 | 40]   (full)<br>Split -&gt; Left:  [10 | 20]<br>         Right: [30 | 40]</pre><p>Here is the paper: <a href="https://www.csd.uoc.gr/~hy460/pdf/p650-lehman.pdf">https://www.csd.uoc.gr/~hy460/pdf/p650-lehman.pdf</a></p><h3>Postgres Uses Tweaked Version of Lehman Yao B-Tree</h3><p>In this page you find detailed information about how Postgres implements Lehman Yao B-Tree,</p><p><a href="https://github.com/postgres/postgres/tree/master/src/backend/access/nbtree">postgres/src/backend/access/nbtree at master · postgres/postgres</a></p><p>There are some slight changes in the implementation, one of them is the fact that Postgres puts a lock on pages(that have the indexes) when reading them.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/819/1*BqR5CA0vznUq4_lHE0WhnA.png" /><figcaption>One of the Differences of Postgres B-Tree with the Paper from Lehman Yao</figcaption></figure><p>We can see the Links to the right page in the source code as well(nbtsearch.c):</p><pre>static Buffer _bt_moveright(Relation rel, Relation heaprel, BTScanInsert key, <br>                           Buffer buf, bool forupdate, BTStack stack, int access)<br>{<br>    // Move right if our key is &gt;= page&#39;s high key<br>    while (P_IGNORE(opaque) || _bt_compare(rel, key, page, P_HIKEY) &gt;= cmpval) {<br>        buf = _bt_relandgetbuf(rel, buf, opaque-&gt;btpo_next, access);<br>        // Continue until we find the correct page<br>    }<br>}</pre><h3>MongoDB uses another Variation of B+ Tree</h3><p>MongoDB also uses a variation of B+ Trees which means that data exists in leaf nodes not the internal nodes(internal nodes contain data for routing).</p><p>We usually describe the B+ Tree as something that the leaf nodes are linked together, but for MongoDB, that is a bit different. there was a <a href="https://www.mongodb.com/community/forums/t/does-index-use-btree-or-b-tree/248053">discussion on MongoDB Forum</a> that you can find in the references, and an email from <em>Keith Smith</em>, describing the difference.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*fK3vc1u6VmyW1t8JE4RIAQ.png" /><figcaption>Keith Smith</figcaption></figure><p>As he says, in the B+ Tree internal nodes, they are not storing anything except the keys(or pointers) for navigation. but the leaf nodes are not connected to each other, the pointer has to go back to parent node to find the next connected node!(<strong>Probably the Reason</strong>: WiredTiger uses copy-on-write semantics where updated pages are written to new locations <strong><em>A-Direct sibling links would require updating all neighboring pages when any page is modified</em></strong> B-<strong><em>This would create massive write amplification — updating one page could cascade to rewriting the entire leaf level</em></strong>)</p><p>In MongoDB’s source code there is a file: bt_walk.c and the function _tree_walk_internal that contains the logic of going back to parent, here the role of this function to change the pointer to parent node: __wt_ref_ascend</p><pre>   for (;;) {<br>        /*<br>         * If we&#39;re at the last/first slot on the internal page, return it in post-order traversal.<br>         * Otherwise move to the next/prev slot and left/right-most element in that subtree.<br>         */<br>        while ((prev &amp;&amp; slot == 0) || (!prev &amp;&amp; slot == pindex-&gt;entries - 1)) {<br>            /* Ascend to the parent. */<br>            __wt_ref_ascend(session, &amp;ref, &amp;pindex, &amp;slot);<br><br>            /*<br>             * If at the root and returning internal pages, return the root page, otherwise we&#39;re<br>             * done.<br>             */<br>            if (__wt_ref_is_root(ref)) {<br>                if (!LF_ISSET(WT_READ_SKIP_INTL)) {<br>                    *refp = ref;<br>                    WT_ASSERT(session, ref != ref_orig);<br>                }<br>                goto done;<br>            }</pre><p>It is still performant, because they have a rule for caching pages, if a page’s children are cached, the direct parent stays cached, so when trying to go back to parent, it is fast and they do not have to do any IO to get that.</p><h3>How I Used AI to Write This Article</h3><p>I have used AI to do a fact check for what I wrote after the article finished and corrected a few things. During the process of reading source codes from Postgres and MongoDB I used AI(specifically Github Copilot) to extract relevant sections of source code, as I am not a contributed to those source codes). Also I have used Claude Sonnet 4 to understand the B* Tree better and being able to simplify it.</p><h3>References and Resources Used</h3><ul><li><a href="https://stackoverflow.com/questions/15768409/why-not-b-tree-mongodb">Why not B+-Tree MongoDB</a></li><li><a href="https://www.mongodb.com/community/forums/t/does-index-use-btree-or-b-tree/248053">Does index use btree or b+tree?</a></li></ul><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FaZjYr87r1b8%3Fstart%3D2232%26feature%3Doembed%26start%3D2232&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DaZjYr87r1b8&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FaZjYr87r1b8%2Fhqdefault.jpg&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/edf8b8f169c245d580aa1fa7231d5ff8/href">https://medium.com/media/edf8b8f169c245d580aa1fa7231d5ff8/href</a></iframe><ul><li><a href="https://en.wikipedia.org/wiki/B-tree">B-tree - Wikipedia</a></li><li><a href="https://groups.google.com/g/wiredtiger-users/c/1YHbNXPw-1A">Does index use btree or b+tree?</a></li><li><a href="https://www.cs.usfca.edu/~galles/visualization/BTree.html">B-Tree Visualization</a></li></ul><p><a href="https://www.quora.com/What-are-the-differences-between-b-b+-and-b*-trees">https://www.quora.com/What-are-the-differences-between-b-b+-and-b*-trees</a></p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2Fn5-xEEQFqPY%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3Dn5-xEEQFqPY&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2Fn5-xEEQFqPY%2Fhqdefault.jpg&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/078afbf02aea9c74efe66d27fcb3d52d/href">https://medium.com/media/078afbf02aea9c74efe66d27fcb3d52d/href</a></iframe><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=898176e74045" width="1" height="1" alt=""><hr><p><a href="https://levelup.gitconnected.com/why-databases-dont-use-plain-b-trees-for-indexes-898176e74045">Why Databases Don’t Use Plain B-Trees for Indexes</a> was originally published in <a href="https://levelup.gitconnected.com/">Level Up Coding</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Is MCP the Wrong Abstraction?(Is that True or Another Clickbait — Hype?)]]></title>
            <link>https://levelup.gitconnected.com/is-mcp-the-wrong-abstraction-is-that-true-or-another-clickbait-hype-88a4376dc935?source=rss-8f1fd23dbcf6------2</link>
            <guid isPermaLink="false">https://medium.com/p/88a4376dc935</guid>
            <category><![CDATA[mcps]]></category>
            <category><![CDATA[abstraction]]></category>
            <category><![CDATA[cloudflare]]></category>
            <category><![CDATA[code-mode]]></category>
            <category><![CDATA[ai]]></category>
            <dc:creator><![CDATA[Poorshad Shaddel]]></dc:creator>
            <pubDate>Thu, 09 Oct 2025 02:35:58 GMT</pubDate>
            <atom:updated>2025-10-09T02:35:58.866Z</atom:updated>
            <content:encoded><![CDATA[<h3>Is MCP the Wrong Abstraction?(Is that True or Another Clickbait — Hype?)</h3><p>There has been a few contents created and mentioning the challenges of using MCP Tools and maybe in a wider perspective the tool calling capabilities of LLMs, here we are going to take a look at these critics and try to find out the actual issues.</p><p>First let’s see what is in the article and the video.</p><h3>Cloudflare Article: Better Way to Use MCP</h3><p><a href="https://blog.cloudflare.com/code-mode/">Code Mode: the better way to use MCP</a></p><p>This article first explains the MCP and it shows how a tool call looks like to an LLM:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/754/1*Vc16Z2SkKd64dh8C2lKdTQ.png" /><figcaption>Tool Call from LLM’s Perspective</figcaption></figure><p>And then the result would be something like this when the tool call is over:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/754/1*UMcdzDffOaqzrJZM9pCh-A.png" /><figcaption>Tool Call Result from LLM’s Perspective</figcaption></figure><p>And then they say:</p><p><strong><em>Making an LLM perform tasks with tool calling is like putting Shakespeare through a month-long class in Mandarin and then asking him to write a play in it. It’s just not going to be his best work.</em></strong></p><h4>Which is kind of true! in my opinion</h4><p>This is true, and then in the article it says that our LLM’s are not good with tool calling but they are really good with code generation, so let’s create a sandbox that the LLM can run the code and interact with the API’s via this sandbox. In their case they used some Typescript Interfaces as the context and then the LLM is capable of creating API Calls to get the result. Check <a href="https://blog.cloudflare.com/code-mode/#converting-mcp-to-typescript">here</a> for more details.</p><p><strong>Summary:</strong> <em>Instead of the using the different MCP tools for each API Call, we give the whole context and the only tool available to the LLM becomes the code_run tool or something similar that gets a piece of code and executes it.</em></p><p>This is their solution diagram:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*c7MRTNFHoc0_6i_0.png" /><figcaption>Cloudflare Code Mode</figcaption></figure><h4>LLMs are not good at Tool Calling(Really?)</h4><p>They are not talking about some specific models, but if we take a look at this data from <a href="https://gorilla.cs.berkeley.edu/leaderboard.html">the article</a>, we can see that Anthropic models are performing good on tool calling, the other models are lagging behind. <strong>so it is not Mandarin for Shakespeare as they say</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ccaqi6beNVuKcgnFmKmbKA.png" /></figure><p><em>I am not against the idea of code mode, on the other hand I find it super useful for many cases, but I am not comfortable simply moving away from function calling just because at the moment raw code generation is better than tool calling.</em></p><p>I think, what they came up is something that we needed to progress at some point, but it is not the ultimate solution, we still have to go forward with normal tool calling. we get into other points of the article below.</p><h3>Youtube Video from Theo T3: MCP is the wrong Abstraction</h3><p>This video is kind of confirming the validity of what Cloudflare has done and points out issues of MCP.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FbAYZjVAodoo%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DbAYZjVAodoo&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FbAYZjVAodoo%2Fhqdefault.jpg&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/36a120f4dcc1c59c5ca543b2a78bb3d8/href">https://medium.com/media/36a120f4dcc1c59c5ca543b2a78bb3d8/href</a></iframe><h4><strong>Key criticisms From the Video</strong></h4><p><strong>1-Too many tools degrade performance:</strong> Exposing large MCP toollists makes agents worse — more confusion, poorer choices, and context bloat from chaining calls. ​⁠ ​⁠</p><p><em>This is true! BUT! the large context and everything </em><strong><em>is not going to be resolved</em></strong><em> by using code mode or similar solutions! Actually it is not mentioned in the video that cloudflare article itself says this:</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HgzKSVf_EscmKoEn0HEICQ.png" /><figcaption><em>Search and Browse the API more dynamically</em></figcaption></figure><p><em>It is still an open issue in MCP world, but there are already some solutions, one of them is dynamically searching for relevant tools and just load those ones, which is also mentioned in the article of cloudflare. So you still have all those issues like confusion poorer choices and context bloating. The only thing that might be improved is the chaining calls.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/761/1*SCCmaUt5nrFDzCALobFaTQ.png" /><figcaption>Beh Holmes — Product Engineer at Warp</figcaption></figure><p><em>Also Ben shares his experience in Warp, using just one Agent with Sonnet 4.</em></p><p>Anthropic itself is trying to solve some similar issues in <strong>claude-code</strong></p><ul><li><a href="https://github.com/anthropics/claude-code/issues/6638">Add dynamic loading/unloading of MCP servers during active sessions · Issue #6638 · anthropics/claude-code</a></li><li><a href="https://github.com/anthropics/claude-code/issues/6702">[Feature Request] Add method to clear MCP tools context · Issue #6702 · anthropics/claude-code</a></li></ul><p><strong>2- Context/token inefficiency:</strong> Traditional tool-calling requires multiple model generations; each result is fed back into the context, inflating input tokens and cost.</p><p><em>This one is a valid one, we are burning a lot of tokens with MCP Tool calls, unfortunately the output schema is also not yet available and we get a lot of tokens from the call response that we do not need(I have attempted to solve this issue for GraphQL by implementing this library: </em><a href="https://github.com/pshaddel/graphql-mcp-bridge"><em>mpc-bridge-graphql</em></a><em><br>I think, the solution would be to first, know the output schema of each tool and then coming up with something to combine and call tools on top of each other.</em></p><p><strong>3-Weak learned prior for tool-calling:</strong> MCP depends on special tokens and synthetic training; LLMs haven’t seen much real-world tool-calling, so they misuse or hallucinate formats.</p><p><em>I have answered this in the previous section of the article, some models are much better than the other ones at tool calling, but they are getting better!</em></p><h3>Conclusion</h3><p>In my opinion there are many areas that the MCP Protocol and the tools on top of that can improve, these are for example, more efficient token usage and more models good at tool calling(not just Anthropic models), also more solutions for better usage of context when there are too many tools(agent delegation or dynamic tool loading are some solutions). MCP is still very young, I am personally very optimistic to see the improvements of the LLM’s with tool calling.</p><ul><li><a href="https://blog.cloudflare.com/code-mode/?utm_source=engineering.fyi/">Code Mode: the better way to use MCP</a></li><li><a href="https://gorilla.cs.berkeley.edu/leaderboard.html">Berkeley Function Calling Leaderboard (BFCL) V4</a></li><li><a href="https://www.youtube.com/@bholmesdev">Ben Holmes</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=88a4376dc935" width="1" height="1" alt=""><hr><p><a href="https://levelup.gitconnected.com/is-mcp-the-wrong-abstraction-is-that-true-or-another-clickbait-hype-88a4376dc935">Is MCP the Wrong Abstraction?(Is that True or Another Clickbait — Hype?)</a> was originally published in <a href="https://levelup.gitconnected.com/">Level Up Coding</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[MCP Is Transforming Applications: Here’s Why Developers and Business Owners Can’t Ignore It]]></title>
            <link>https://levelup.gitconnected.com/mcp-is-transforming-applications-heres-why-developers-and-business-owners-can-t-ignore-it-06374b8c8804?source=rss-8f1fd23dbcf6------2</link>
            <guid isPermaLink="false">https://medium.com/p/06374b8c8804</guid>
            <category><![CDATA[nodejs]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[future]]></category>
            <category><![CDATA[a2a]]></category>
            <category><![CDATA[mcp-server]]></category>
            <dc:creator><![CDATA[Poorshad Shaddel]]></dc:creator>
            <pubDate>Sat, 02 Aug 2025 23:10:46 GMT</pubDate>
            <atom:updated>2025-08-04T15:06:21.947Z</atom:updated>
            <content:encoded><![CDATA[<p>Model context protocol is becoming the backbone and the common language that LLMS are doing useful things, this is going further when everybody adopts! so good that we might not imagine.</p><p><em>I will explain what is MCP and how important is the role of MCP, and why we needed something like this, let’s go through the evolutions we have seen together.</em></p><h3>First Revolution: Emerge of Powerful LLMs</h3><p><strong>November 30, 2022</strong> was the date that <strong>GPT</strong> was announced, at the time, it was a miracle, you could ask questions about anything, and you were getting sensible answers. This black box was not 100% accurate, but it was something you could get a lot of correct answers.</p><p>LLMs are token(let’s simplify it to text) generation machines, they predict next word, or a set of words.</p><p>Compare that to Claude Sonnet 4 or Google 2.5 Flash now, much more accurate answers.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*JD6_eYDgybkEPVIm1UeLPQ.png" /><figcaption>Sample of LLM Answer</figcaption></figure><p>Reasoning models were also another important step, In September 2024, <a href="https://en.wikipedia.org/wiki/OpenAI">OpenAI</a> released <a href="https://en.wikipedia.org/wiki/OpenAI_o1#release">o1-preview</a>, which was the first one, these models can tackle more complex problems, by breaking down the issue into multiple steps, and revising and revisiting the steps till getting to the answer.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GNJjwusfpurb4-KQJXSfgg.png" /><figcaption>LLM</figcaption></figure><h3>Second Revolution: Turn Text to Action</h3><p>Just generating tokens and texts was never enough after a while. we wanted more from LLMs. we simply wanted some <strong>actions</strong>. How could we achieve this? Chat GPT did the web search as the first action(let’s also call it a tool).</p><h4>How does this work?</h4><p>At the application level, we can use some specific texts to some actions. <br>For example, give this text to chat GPT in a new chat:</p><pre>If you want to send text message to somebody, <br>use this pattern: {{SMS(phone_number, text)}}  <br><br>Example:<br>{{SMS(+4301234567, &quot;Hey Buddy&quot;)}}</pre><p>Now write this: send a message to my friend, with phone number 12345, ask if she is coming to the party</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*BLpe7RS_X3oNw-fRbj3wdw.png" /><figcaption>Patterns in LLM Response</figcaption></figure><p>Now, us as Software Developer, are going to check the response for the pattern, if there is the {{SMS}} pattern somewhere, we are going to call an API and send a message to somebody.</p><p>So you get an idea of how we can turn text to action. First action that was implemented was the ability to do a web search, which is called knowledge injection. when LLM, uses a pattern for searching, at the application level, we get the search result, then feed it back to LLM, so it can continue generating tokens based on result.</p><p>We had products who quickly added support for Email, Gmail, Calendar, Slack and many more.</p><h3>Third Revolution: MCP, Can We Integrate Infinite Number of Tools?</h3><p>We have seen that AI applications like Chat GPT, Manus and other ones started implementing more integration with tools. also each one of them did it in their own way. but is there a better solution? what if I have a very specific tool that GPT is never interested to integrate?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*E8q5zh-fluRyo9jy3dXawA.png" /><figcaption>Adding Integrations to Each AI Application</figcaption></figure><p>The solution to this mess of integrations, was <a href="https://modelcontextprotocol.io/docs/getting-started/intro">MCP, the Model Context Protocol.</a> <a href="https://www.anthropic.com/">Anthropic</a>, the company behind <a href="https://claude.ai/">Claude</a> announced a protocol<br>in November 25, 2024. This is a standard way that your LLM can access tools, instead of the diagram on the top, you can add your own tools and integrations, without the need of intervention from the providers like Open AI or Claude:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*cK6WXaPxXQOQ4OySytt4OA.png" /><figcaption>After MCP — Tools are added with the same protocol</figcaption></figure><p><strong>If you are a software engineer, you should see how beautiful, the parts got decoupled!</strong></p><p>Now, on Claude Desktop, you can add MCP Servers:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*oalj3zEZkLl7fIGCXLJoqg.png" /><figcaption>MCP Tools on Claude AI Desktop App</figcaption></figure><h4>But why I think MCP is the most important piece</h4><p>Each time I am riding on my bike, and I want to do an action, and I talk to Siri, I get really mad because of how stupid it is and more important, <strong>how limit are the actions it is capable of. </strong>I think, Google has more powerful voice assistant, but even Google is not able to a lot of actions, and the reason is that each one of the applications have their own APIs, and google cannot integrate with all of them!! <strong>Now, imagine, each Application has its own official MCP Server(SSE supported) and your voice assistant can make these tools available for you to interact via the LLM.</strong></p><h4>Business Owners, Do you want to see how is the future?</h4><p>Future is already here, this is a sample of it:</p><p>Check this out,</p><p><a href="https://www.linkedin.com/feed/update/urn:li:activity:7356939874236530688/">🚀 Commerce MCP is live | Jakub Neander</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*-m_0nPkgwjrU4lbN1zC3AA.gif" /><figcaption>MCP Server of a E-Commerce</figcaption></figure><h3>Forth Evolution: A2A Protocol and AG-UI</h3><p>Let’s keep this one short, AI Agents should be also communicate in a standard way, A2A is the standard for that one. AG-UI is also an standard for the Frontend Apps to interact with LLMs in a chat bot like environment.<br>If you want to read more, check out my other article:</p><p><a href="https://levelup.gitconnected.com/agui-agent-to-ui-protocol-what-should-backend-and-frontend-developers-know-25a7a1ddf1f8">AGUI(Agent to UI Protocol): What Should Backend and Frontend Developers Know</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*P9QJaka5nqJacJngHL8ecg.png" /><figcaption>A2A protocol</figcaption></figure><h3>Why Should I Care About MCP as a Business Owner?</h3><p>As the Business Owner you should know the MCP is the <strong>API for LLMs</strong>, also many creative people <strong>can build tools with their AI Agents</strong>, if you provide and maintain an official MCP for your client, also keep in mind that this is not a fancy thing to have, very soon each application have to have this.</p><h3>MCP Server Template to Start! Nodejs and Typescript with stdio and SSE Support</h3><p>If you want to easily start an MCP Server, this is a repository that I created. It is well documented and I usually invest time in case you open issues! feel free to come up with better ideas for improving it.</p><p><a href="https://github.com/pshaddel/mcp-template">GitHub - pshaddel/mcp-template: A comprehensive Model Context Protocol (MCP) server template for Node.js with TypeScript, supporting both stdio and SSE transports</a></p><h3>Conclusion</h3><p>MCP is the new API, for the future. the only way to evolve and go to next level with AI applications is that each application and service becomes available as a tool to LLMs. Let’s build the future!<br>I will be happy to see any kind of feedback!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=06374b8c8804" width="1" height="1" alt=""><hr><p><a href="https://levelup.gitconnected.com/mcp-is-transforming-applications-heres-why-developers-and-business-owners-can-t-ignore-it-06374b8c8804">MCP Is Transforming Applications: Here’s Why Developers and Business Owners Can’t Ignore It</a> was originally published in <a href="https://levelup.gitconnected.com/">Level Up Coding</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[AGUI(Agent to UI Protocol): What Should Backend and Frontend Developers Know]]></title>
            <link>https://levelup.gitconnected.com/agui-agent-to-ui-protocol-what-should-backend-and-frontend-developers-know-25a7a1ddf1f8?source=rss-8f1fd23dbcf6------2</link>
            <guid isPermaLink="false">https://medium.com/p/25a7a1ddf1f8</guid>
            <category><![CDATA[agentic-ai]]></category>
            <category><![CDATA[mcp-server]]></category>
            <category><![CDATA[full-stack-developer]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[ag-ui]]></category>
            <dc:creator><![CDATA[Poorshad Shaddel]]></dc:creator>
            <pubDate>Wed, 16 Jul 2025 07:59:05 GMT</pubDate>
            <atom:updated>2025-07-16T14:09:35.560Z</atom:updated>
            <content:encoded><![CDATA[<p>AG-UI is another piece that completes the puzzle of the development of AI Tools. Let’s see what it is, and what we should do as web developers and software engineers</p><p>I believe this is not another hype or something that fades away; we needed it sooner or later.</p><h3>What You Are Going to Read</h3><ul><li>What is AG-UI</li><li>Why is it important to adopt</li><li>What Should Frontend Developers Know?</li><li>What Should Backend Developers Know?</li><li>Conclusion</li></ul><h3>What is AG-UI (Agent User Interaction Protocol)?</h3><p>They have a good website and proper(kind of) documentation, so I am not going to tweak a lot of parts there:</p><p><strong>AG-UI standardizes how front-end applications connect to AI agents through an open protocol.</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*j_67yCMDh30uJT9TXAnStQ.gif" /><figcaption>AG-UI protocol for communication between Agent and applications</figcaption></figure><h4>We already have two really important AI Protocols:</h4><p><strong>MCP</strong>: Gives agents(or LLMs) access to tools in a standard way<br><strong>A2A</strong>: It let AI Agents interact with each other in the same language(protocol)</p><h4>The problem that AG-UI is trying to solve is another part:</h4><p>Agents should be able to interact with Frontend Apps, and we need a single way to communicate.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rgvE5UEvYueD72vwTQ63xg.png" /><figcaption>AG-UI for the communication</figcaption></figure><p>In the documentation it says: AG-UI is a <strong><em>event-driven</em></strong> protocol. and Agents can emit 16 different type of events. we will get to it!</p><p>Also, it says that the interaction is <strong>Bidirectional</strong>, which means that the client can do some actions about the workflow or the task, even after doing this.</p><p>For example in Chat GPT Application, we can simply cancel generation of response:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*bgQbX_vkl9oqqW8RqB5b4A.png" /><figcaption>Cancel Respnose Generation</figcaption></figure><p>If you open the inspection tab in GPT, you see that there is this route https://chatgpt.com/backend-api/f/conversation, which sends some SSE events when you ask your question. When you want to stop the response generation and you hit that button, it sends another GET request to another route: <a href="https://chatgpt.com/backend-api/stop_conversation">https://chatgpt.com/backend-api/stop_conversation</a> . So also in GPT you have to have the bidirectional to handle these kinds of <strong>events</strong>!</p><h4>Compatibility of AG-UI</h4><ul><li><strong>Flexible Event Structure</strong>: your events just have to be AG-UI compatible, it does not force anything other than this fact.</li><li><strong>Transport Agnostic</strong>: SSE, Webhooks, Websockets, simply use what fits you.</li></ul><h4>Events</h4><p>This is an important building block of the AG-UI. There are different kinds of events, and the names are quite obvious what these events are announcing:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/538/1*hyOrY-pBaKE87571e5TAbg.png" /><figcaption>Events Categories</figcaption></figure><p>For example, for the <strong>Lifecycle Events</strong>:</p><ul><li>RunStarted</li><li>RunFinished</li><li>RunError</li></ul><p>For the <strong>Text Messages Events</strong>:</p><ul><li>TextMessageStart</li><li>TextMessageEnd</li><li>TextMessageContent</li></ul><p>or for the <strong>Tool Call Events</strong>:</p><ul><li>ToolCallStart</li><li>ToolCallEnd</li><li>ToolCallArgs</li></ul><p>You might ask, why in the naming of events, they have used <strong>past</strong> tense for the lifecycle events and <strong>present</strong> tense for the other ones, or why somewhere is used finish, and in other places end . Maybe I am too picky and an annoying reviewer.</p><p>Here you can find more details about events:</p><p><a href="https://docs.ag-ui.com/concepts/events">Events - Agent User Interaction Protocol</a></p><p><em>So far, we have understood that AG-UI is a way of communication between frontend AI apps and Agents.</em></p><h3>Why is it important to adopt</h3><p>The same way that we adopted MCP. Now, almost all workflow products support MCP, which makes it a very straightforward thing to reuse the set of tools you have in all of them, without even changing a line of code. This one needs adaptation of frontend apps, so that we can plug in our applications to different services, which all provide AG-UI.</p><h3>What Should Frontend Developers Know?</h3><p>At the moment, there are not so many libraries and tools adopted with this tool, but keep in mind that if you are creating an AI chatbot, using the AG-UI for the communication with the AI Agent.</p><p>Here is a sample of a Typescript code that we can take a look at (The code itself was probably also generated with AI):</p><p>Before taking a look at the code, we can see that for sending messages, it pushes messages with id, role and content as expected in the standard.<br>The Agent is created via Mastra sdk.</p><pre>import * as readline from &quot;readline&quot;<br>import { agent } from &quot;./agent&quot;<br>import { randomUUID } from &quot;node:crypto&quot;<br><br>const rl = readline.createInterface({<br>  input: process.stdin,<br>  output: process.stdout,<br>})<br><br>async function chatLoop() {<br>  console.log(&quot;🤖 AG-UI Assistant started!&quot;)<br>  console.log(&quot;Type your messages and press Enter. Press Ctrl+D to quit.\n&quot;)<br><br>  return new Promise&lt;void&gt;((resolve) =&gt; {<br>    const promptUser = () =&gt; {<br>      rl.question(&quot;&gt; &quot;, async (input) =&gt; {<br>        if (input.trim() === &quot;&quot;) {<br>          promptUser()<br>          return<br>        }<br>        console.log(&quot;&quot;)<br><br>        // Pause input while processing<br>        rl.pause()<br><br>        // Add user message to conversation<br>        agent.messages.push({<br>          id: randomUUID(),<br>          role: &quot;user&quot;,<br>          content: input.trim(),<br>        })<br><br>        try {<br>          // Run the agent with event handlers<br>          await agent.runAgent(<br>            {}, // No additional configuration needed<br>            {<br>              onTextMessageStartEvent() {<br>                process.stdout.write(&quot;🤖 Assistant: &quot;)<br>              },<br>              onTextMessageContentEvent({ event }) {<br>                process.stdout.write(event.delta)<br>              },<br>              onTextMessageEndEvent() {<br>                console.log(&quot;\n&quot;)<br>              },<br>            }<br>          )<br>        } catch (error) {<br>          console.error(&quot;❌ Error:&quot;, error)<br>        }<br><br>        // Resume input<br>        rl.resume()<br>        promptUser()<br>      })<br>    }<br><br>    // Handle Ctrl+D to quit<br>    rl.on(&quot;close&quot;, () =&gt; {<br>      console.log(&quot;\n👋 Thanks for using AG-UI Assistant!&quot;)<br>      resolve()<br>    })<br><br>    promptUser()<br>  })<br>}<br><br>async function main() {<br>  await chatLoop()<br>}<br><br>main().catch(console.error)</pre><p>Check out the whole implementation here:</p><p><a href="https://docs.ag-ui.com/quickstart/clients">Build clients - Agent User Interaction Protocol</a></p><p>Another note for me: Try to use a solid library for this instead of using your own implementation, some implementations should be available in near future.</p><h3>What Should Backend Developers Know?</h3><p>If you are providing some APIs for the chatbot, backed by OpenAI, Claude, or anything, you can adopt it to be AG-UI compatible, and then all client-side implementations that are compatible with it will work out of the box.</p><h4>Here is an example of turning the Open AI API to AG-UI compatible versions:</h4><p><a href="https://docs.ag-ui.com/quickstart/server#implement-the-streaming-serve">https://docs.ag-ui.com/quickstart/server#implement-the-streaming-serve</a>r</p><p>Long story short, before you take a look at the code(from the documents):</p><ol><li>Setup — We create an OpenAI client and emit RUN_STARTED</li><li>Request — We send the user’s messages to chat.completions with stream=True</li><li>Streaming — We forward each chunk as either TEXT_MESSAGE_CHUNK or TOOL_CALL_CHUNK</li><li>Finish — We emit RUN_FINISHED (or RUN_ERROR if something goes wrong)</li></ol><pre>import os<br>import uuid<br>import uvicorn<br>from fastapi import FastAPI, Request<br>from fastapi.responses import StreamingResponse<br>from ag_ui.core import (<br>    RunAgentInput,<br>    EventType,<br>    RunStartedEvent,<br>    RunFinishedEvent,<br>    RunErrorEvent,<br>)<br>from ag_ui.encoder import EventEncoder<br>from openai import OpenAI<br><br>app = FastAPI(title=&quot;AG-UI OpenAI Server&quot;)<br><br># Initialize OpenAI client - uses OPENAI_API_KEY from environment<br>client = OpenAI()<br><br>@app.post(&quot;/&quot;)<br>async def agentic_chat_endpoint(input_data: RunAgentInput, request: Request):<br>    &quot;&quot;&quot;OpenAI agentic chat endpoint&quot;&quot;&quot;<br>    accept_header = request.headers.get(&quot;accept&quot;)<br>    encoder = EventEncoder(accept=accept_header)<br><br>    async def event_generator():<br>        try:<br>            yield encoder.encode(<br>                RunStartedEvent(<br>                    type=EventType.RUN_STARTED,<br>                    thread_id=input_data.thread_id,<br>                    run_id=input_data.run_id<br>                )<br>            )<br><br>            # Call OpenAI&#39;s API with streaming enabled<br>            stream = client.chat.completions.create(<br>                model=&quot;gpt-4o&quot;,<br>                stream=True,<br>                # Convert AG-UI tools format to OpenAI&#39;s expected format<br>                tools=[<br>                    {<br>                        &quot;type&quot;: &quot;function&quot;,<br>                        &quot;function&quot;: {<br>                            &quot;name&quot;: tool.name,<br>                            &quot;description&quot;: tool.description,<br>                            &quot;parameters&quot;: tool.parameters,<br>                        }<br>                    }<br>                    for tool in input_data.tools<br>                ] if input_data.tools else None,<br>                # Transform AG-UI messages to OpenAI&#39;s message format<br>                messages=[<br>                    {<br>                        &quot;role&quot;: message.role,<br>                        &quot;content&quot;: message.content or &quot;&quot;,<br>                        # Include tool calls if this is an assistant message with tools<br>                        **({&quot;tool_calls&quot;: message.tool_calls} if message.role == &quot;assistant&quot; and hasattr(message, &#39;tool_calls&#39;) and message.tool_calls else {}),<br>                        # Include tool call ID if this is a tool result message<br>                        **({&quot;tool_call_id&quot;: message.tool_call_id} if message.role == &quot;tool&quot; and hasattr(message, &#39;tool_call_id&#39;) else {}),<br>                    }<br>                    for message in input_data.messages<br>                ],<br>            )<br><br>            message_id = str(uuid.uuid4())<br><br>            # Stream each chunk from OpenAI&#39;s response<br>            for chunk in stream:<br>                # Handle text content chunks<br>                if chunk.choices[0].delta.content:<br>                    yield encoder.encode({<br>                        &quot;type&quot;: EventType.TEXT_MESSAGE_CHUNK,<br>                        &quot;message_id&quot;: message_id,<br>                        &quot;delta&quot;: chunk.choices[0].delta.content,<br>                    })<br>                # Handle tool call chunks<br>                elif chunk.choices[0].delta.tool_calls:<br>                    tool_call = chunk.choices[0].delta.tool_calls[0]<br><br>                    yield encoder.encode({<br>                        &quot;type&quot;: EventType.TOOL_CALL_CHUNK,<br>                        &quot;tool_call_id&quot;: tool_call.id,<br>                        &quot;tool_call_name&quot;: tool_call.function.name if tool_call.function else None,<br>                        &quot;parent_message_id&quot;: message_id,<br>                        &quot;delta&quot;: tool_call.function.arguments if tool_call.function else None,<br>                    })<br><br>            yield encoder.encode(<br>                RunFinishedEvent(<br>                    type=EventType.RUN_FINISHED,<br>                    thread_id=input_data.thread_id,<br>                    run_id=input_data.run_id<br>                )<br>            )<br><br>        except Exception as error:<br>            yield encoder.encode(<br>                RunErrorEvent(<br>                    type=EventType.RUN_ERROR,<br>                    message=str(error)<br>                )<br>            )<br><br>    return StreamingResponse(<br>        event_generator(),<br>        media_type=encoder.get_content_type()<br>    )<br><br>def main():<br>    &quot;&quot;&quot;Run the uvicorn server.&quot;&quot;&quot;<br>    port = int(os.getenv(&quot;PORT&quot;, &quot;8000&quot;))<br>    uvicorn.run(<br>        &quot;example_server:app&quot;,<br>        host=&quot;0.0.0.0&quot;,<br>        port=port,<br>        reload=True<br>    )<br><br>if __name__ == &quot;__main__&quot;:<br>    main()</pre><p>Now you have created an API compatible with AG-UI. So in case you have services like this, or you want to implement something in future, you can always use this way, or you can have your own way and another route for the AG-UI apps, to consume your APIs.</p><p><strong><em>My Guess: There should be plug and play packages in a few months, so you should not need to implement this, you just have to use them.</em></strong></p><h3>Conclusion</h3><p>Although this is relatively new, people are trying to adopt their libraries, so we as developers should keep an eye and try to use this if possible, so that we face fewer troubles in case we want to change some of the pieces of our software.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/722/1*tZFQmjoZ_8cFno6TzwiRXQ.png" /><figcaption>Existing Integrations of AG-UI</figcaption></figure><p><a href="https://docs.ag-ui.com/introduction">AG-UI Overview - Agent User Interaction Protocol</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=25a7a1ddf1f8" width="1" height="1" alt=""><hr><p><a href="https://levelup.gitconnected.com/agui-agent-to-ui-protocol-what-should-backend-and-frontend-developers-know-25a7a1ddf1f8">AGUI(Agent to UI Protocol): What Should Backend and Frontend Developers Know</a> was originally published in <a href="https://levelup.gitconnected.com/">Level Up Coding</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Re-implement Redux Part I: There is No Magic]]></title>
            <link>https://levelup.gitconnected.com/re-implement-redux-part-i-there-is-no-magic-80739772a1d8?source=rss-8f1fd23dbcf6------2</link>
            <guid isPermaLink="false">https://medium.com/p/80739772a1d8</guid>
            <category><![CDATA[react]]></category>
            <category><![CDATA[state-management]]></category>
            <category><![CDATA[web-development]]></category>
            <category><![CDATA[javascript]]></category>
            <category><![CDATA[redux]]></category>
            <dc:creator><![CDATA[Poorshad Shaddel]]></dc:creator>
            <pubDate>Sun, 01 Jun 2025 22:36:34 GMT</pubDate>
            <atom:updated>2025-06-01T22:36:34.878Z</atom:updated>
            <content:encoded><![CDATA[<p>Peeling back Redux’s layers to reveal its surprisingly simple core.</p><p><em>This is an article for mostly beginners and non-beginners who have never looked into Redux implementation.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*w8OoUhR2-7mdLN5r2atemg.png" /><figcaption>Rebuild Redux yourself — Image created by DreamStudio</figcaption></figure><p>Redux might seem frustrating and complicated if you are a beginner and trying to use it in a React project. But the fact is that it is not that complicated under the hood; once you understand why this store is designed that way, it just clicks, and you will never have a problem working with Redux.</p><p>We are not going for the actual implementation, but a simplified version that gives us a big picture of what is going on.</p><h3>What you will read in this article</h3><ul><li><strong>What is Redux? </strong>Simply skip the first part if you are not a beginner! (Only a few sentences, as there are hundreds of Redux tutorials.)</li><li><strong>Set up a React Project</strong></li><li><strong>Why not use a Plain JS object?</strong></li><li><strong>Let’s Implement our Store!</strong></li><li><strong>Next Step</strong></li></ul><p>Redux is a popular store management library that is used in React and Angular. It has a few abstract concepts that you need to understand if you are going to choose it for your future projects. You have no idea what Redux is? No worries, you can learn it along the way.</p><h3>Set up a React Project</h3><p>For setting up, I usually use <a href="https://stackblitz.com/">StackBlitz</a>. Simply choose the React(TypeScript or JavaScript, I will use TypeScript) and follow along.</p><p>These screenshots might help:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2mpJU8_39BMJyIWeZ62YBA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*yGi_YRT-BJIC1kzLH3GY2A.png" /></figure><p>Now, you have a running React Application.</p><h3>Why not use a plain JS object?</h3><p>In my opinion, the most important step in understanding tools is to find out what is the reason behind the design. For understanding this, I would like to show you this.</p><p>Simply open the App.tsx file, remove the unnecessary parts, and create an object called state :</p><pre>let state = {<br>    userName: &#39;Poorshad&#39;,<br>    userId: 1<br>};</pre><p>Now let’s use the state in our React component and also add a button that changes the username.</p><pre>let state = {<br>  userName: &#39;Poorshad&#39;,<br>  userId: 1,<br>};<br><br>function App() {<br>  return (<br>    &lt;&gt;<br>      &lt;h1&gt;Reimplement Redux&lt;/h1&gt;<br>      &lt;div className=&quot;card&quot;&gt;<br>        &lt;p&gt;<br>          {state.userId} {state.userName}<br>        &lt;/p&gt;<br>        &lt;button<br>          onClick={() =&gt; {<br>            state.userName = &#39;something else&#39;;<br>          }}<br>        &gt;<br>          change username<br>        &lt;/button&gt;<br>      &lt;/div&gt;<br>    &lt;/&gt;<br>  );<br>}<br><br>export default App;</pre><h4>What should happen when we click on the button?</h4><pre>onClick={() =&gt; {state.userName = &#39;something else&#39;}}</pre><p>Nothing happens! You might say, How stupid, this is clear, because you have an Object and this is not used as a state in React. That is exactly the point!!!</p><h4>How does React handle state changes in a Component?</h4><p>You probably know, by using useState hook and context .</p><p>This is the working version:</p><pre>import { useState } from &#39;react&#39;;<br>import &#39;./App.css&#39;;<br><br>function App() {<br>  const [state, setState] = useState({<br>    userName: &#39;Poorshad&#39;,<br>    userId: 1<br>  })<br>  return (<br>    &lt;&gt;<br>      &lt;h1&gt;Reimplement Redux&lt;/h1&gt;<br>      &lt;div className=&quot;card&quot;&gt;<br>        &lt;p&gt;<br>          {state.userId} {state.userName}<br>        &lt;/p&gt;<br>        &lt;button<br>          onClick={() =&gt; {<br>            setState({<br>              userId: 1,<br>              userName: &#39;something else&#39;<br>            })<br>          }}<br>        &gt;<br>          change username<br>        &lt;/button&gt;<br>      &lt;/div&gt;<br>    &lt;/&gt;<br>  );<br>}<br><br>export default App;</pre><h4>But why does React have to use the hook and the <strong>setState</strong> function?</h4><p>If you want to take away <strong>just one point from this article</strong>, this should be it:</p><p><strong><em>When we are changing the state in memory, we have to use some kind of mediator, special functions(call them hooks or whatever you want). Why? Because they have to be aware of the change, to update the DOM elements(In React Terminology: Trigger a Re-render in the components affected by this state change).</em></strong></p><p>React uses <a href="https://react.dev/reference/react/useState">useState</a> or Solid use <a href="https://docs.solidjs.com/concepts/signals">Signals</a> to provide Reactivity and update UI. Svelte works a bit differently, but even there, for using a state shared between components, you are using a <a href="https://svelte.dev/docs/svelte/svelte-store">writable</a>, which in the end does the same thing!</p><p><strong>TLDR: You have to somehow let people know that you have changed the state, and they have to update DOM elements!</strong></p><h3>Let’s Implement our Store!</h3><p>We want to implement something similar to Redux, so we have to understand the terminology. Let’s get to it!</p><h4>State</h4><p>This could be a <a href="https://developer.mozilla.org/en-US/docs/Glossary/Primitive">primitive variable</a> or an object, it is usually an object that we keep the state of our application.</p><p>We said that changing this object directly does not cause any UI updates. We can make this object immutable! (In simple words, not letting anyone change the object) So at the beginning, we are sure that if somebody attempts to update the object, they will fail!</p><p>We can use Object.freeze to cause errors when somebody is trying to change the object. As this is working shallow(you can still change nested objects), we can implement a small function for freezing the state deeply:</p><pre>type DeepReadonly&lt;T&gt; = {<br>    readonly [P in keyof T]: DeepReadonly&lt;T[P]&gt;;<br>  };<br>  <br>function deepFreeze&lt;T extends {<br>    [key: string]: any<br>}&gt;(obj: T): DeepReadonly&lt;T&gt; {<br>    Object.keys(obj).forEach(key =&gt; {<br>      if (typeof obj[key] === &#39;object&#39; &amp;&amp; obj[key] !== null) {<br>        deepFreeze(obj[key]);<br>      }<br>    });<br>    return Object.freeze(obj);<br>  }<br>  <br>  <br>const frozen = deepFreeze({<br>  name: &#39;Poorshad&#39;,<br>  nested: { age: 30 }<br>});<br><br>frozen.nested.age = 50; // Will cause a runtime error, also Typescript Error</pre><p>You can also use some immutability libraries, but here, we do not need more than this.</p><p>Let’s put the implementation in a file called deepFreeze and use it for creating our state.</p><h4>Our First State</h4><p>Here is a sample:</p><pre>const state = deepFreeze({<br>    counter: 0<br>});</pre><h4>Actions</h4><p>This is another part of the Redux store, An action is simply an action that probably causes a change in the store. For example, in the case that we have here, it might have the actions increment or decrement . By a contract, people usually use this format for creating their actions:</p><pre>const action = {<br>  type: &#39;INCREMENT&#39;,<br>  payload: {} // this could be an object or anything<br>};</pre><p>In the end, for your application state, you have a list of actions that users can do. This is a helpful thing for organizing your code, too.</p><h4>Reducers</h4><p>Reducers are probably the most important piece. This is where we are changing the state. Reducers are pure functions, they get the previous state and the action you want to do and return the new state.</p><p>Let’s create a typescript file in src/store.ts and build our store:</p><pre>import { deepFreeze } from &quot;./deep_freeze&quot;;<br><br>const state = deepFreeze({<br>    counter: 0<br>});<br><br>const incrementAction = {<br>    type: &#39;INCREMENT&#39;<br>};<br><br>const decrementAction = {<br>    type: &#39;DECREMENT&#39;<br>};</pre><h4>Why is it called Reducer? (AI helped in this part!)</h4><p>The word <strong>reducer</strong> is borrowed from this concept. It means:<br>A function that <strong>reduces many actions and the current state into a new state</strong>.<br>💡 It <strong>reduces: </strong>previousState + action ➡️ newState<br>So, the name “reducer” reflects that: It takes multiple pieces of information (the current state and an action)</p><p>For the arguments are previousState and action, we should return a new state.</p><p>We can use a switch case for handling cases of increment and decrement:</p><pre>function reducer(previousState: {<br>    readonly counter: number<br>}, action: {<br>    type: string<br>}) {<br>    if (!action) return previousState;<br>    switch (action.type) {<br>        case &#39;INCREMENT&#39;: {<br>            const newState = {<br>                counter: previousState.counter + 1<br>            };<br>            return deepFreeze(newState)<br>        }<br>        case &#39;DECREMENT&#39;: {<br>            const newState = {<br>                counter: previousState.counter - 1<br>            };<br>            return deepFreeze(newState)<br>        }<br>        default: {<br>            // no change<br>            return previousState<br>        }<br>    }<br>};<br><br>const currentState = deepFreeze({<br>    counter: 0<br>});<br><br>const afterIncrement = reducer(currentState, incrementAction);<br>console.log(afterIncrement);<br></pre><p>If the action is not passed, we return the existing state itself.</p><p>With the current implementation, we have a few issues, For example, we do not have a mechanism to get the state; we have to call the reducer with the previous state, and also, we do not have any trigger to run when something is changed.</p><h4>createStore</h4><p>This is the main function for creating a store. Redux gets a reducer when creating a store, and also the initial state of the app. Then we have to have these methods implemented:</p><ul><li>getState</li><li>dispatch</li><li>subscribe</li></ul><h4><strong>getState and dispatch</strong></h4><p>Dispatch is a function that we pass the action, and it calls the store reducer to make the change.</p><pre>type Reducer = (state: any, action: { type: string }) =&gt; any;<br>function createStore(reducer: Reducer, initialState: any) {<br>    let state = deepFreeze(initialState);<br><br>    return {<br>        getState: () =&gt; state,<br>        dispatch: (action: { type: string }) =&gt; {<br>            const nextState = reducer(state, action);<br>            if (nextState !== state) { // this is a reference comparison<br>                state = deepFreeze(nextState); // state was changed<br>            }<br>        }<br>    }<br>}</pre><p>The user has to pass the reducer function and the initial state. We freeze the object, if getState is called, we simply return the state. If a dispatch with an action is called, we call the reducer function with that action and the current state.</p><p>Sample of usage:</p><pre>const store = createStore(reducer, state);<br>store.dispatch(incrementAction);<br>console.log(store.getState()); // { counter: 1 }<br>store.dispatch(incrementAction);<br>console.log(store.getState()); // { counter: 2 }<br>store.dispatch(decrementAction);<br>console.log(store.getState()); // { counter: 1 }</pre><h4>We have a very essential part missing: subscribe</h4><p>The whole point of state management was to be aware of the changes. For this, we have to define a function called subscribe, the user is going to pass a callback function, and each time there is a change, we have to call this function. The user should also be able to unsubscribe from the changes.</p><p>We can keep all listeners in an array: let listeners: (() =&gt; void)[] = [];</p><p>And the subscribe implementation would be:</p><pre>subscribe: (listener: () =&gt; void) =&gt; {<br>  listeners.push(listener);<br>  return {<br>    unsubscribe: () =&gt; {<br>        listeners = listeners.filter(l =&gt; l !== listener);<br>    }<br>  }<br>}</pre><p>We push the function to the list of our listeners, and also we return an Object that has an unsubscribe method for removing the listener(when a component is unmounted!)</p><p>Another important piece we have to change is to call these listeners each time somebody calls the reducer!</p><pre>        dispatch: (action: { type: string }) =&gt; {<br>            const nextState = reducer(state, action);<br>            if (nextState !== state) {<br>                state = deepFreeze(nextState);<br>                listeners.forEach(listener =&gt; listener()); // call all listeners<br>            }<br>        },</pre><p>The whole implementation would look like this:</p><pre>type Reducer = (state: any, action: { type: string }) =&gt; any;<br>function createStore(reducer: Reducer, initialState: any) {<br>    let state = deepFreeze(initialState);<br><br>    let listeners: (() =&gt; void)[] = [];<br><br>    return {<br>        getState: () =&gt; state,<br>        dispatch: (action: { type: string }) =&gt; {<br>            const nextState = reducer(state, action);<br>            if (nextState !== state) {<br>                state = deepFreeze(nextState);<br>                listeners.forEach(listener =&gt; listener()); // call all listeners<br>            }<br>        },<br>        subscribe: (listener: () =&gt; void) =&gt; {<br>            // This is a simple implementation, in a real-world scenario you would want to handle subscriptions properly<br>            listeners.push(listener);<br>            return {<br>                unsubscribe: () =&gt; {<br>                    listeners = listeners.filter(l =&gt; l !== listener);<br>                }<br>            }<br>        }<br>    }<br>}</pre><p>Let’s add some listeners to test everything:</p><pre>const store = createStore(reducer, state);<br>const listener1 = store.subscribe(() =&gt; {<br>    console.log(&#39;Listener 1 = I am aware of the change:&#39;, store.getState());<br>});<br>const listener2 = store.subscribe(() =&gt; {<br>    console.log(&#39;Listener 2 = I am aware of the change:&#39;, store.getState());<br>});<br>store.dispatch(incrementAction);<br>// Listener 1 = I am aware of the change: { counter: 1 }<br>// Listener 2 = I am aware of the change: { counter: 1 }<br>listener1.unsubscribe();<br>store.dispatch(decrementAction);<br>// Listener 2 = I am aware of the change: { counter: 0 }</pre><p>Till now, we have implemented a simple store, we also have our reducer for incrementing and decrementing values, and we can subscribe to changes.</p><p><em>Now that we have a working store, let’s connect it to React</em></p><h3>Bind Our Store to DOM or React!</h3><p>We are going to see a simplified version. In part 2 of the post, we are going to implement the Redux hooks: <strong>useDispatch</strong> and <strong>useSelector</strong>.</p><p>Export your store, so we can use it:</p><pre>export const store = createStore(reducer, state);</pre><h4><strong>React Class Component</strong></h4><p><em>This is for demonstration only — modern React uses hooks, which we’ll cover in Part II</em></p><p>If we were able to use Class Components(not the function components), we could have a simple version like this.</p><p>After the component is mounted, the method componentDidMount is called. We are subscribing to the store and passing a callback function, which includes a method that exists in React Class Components, called forceUpdate. So, each time we have a store change, the component is rendered. These are changes of App.tsx</p><pre>import { store } from &#39;./store&#39;;<br>class App extends React.Component {<br>  unsubscribeStore: null | Function = null;<br>  componentDidMount() {<br>    const x = store.subscribe(() =&gt; {<br>      this.forceUpdate(); // force re-render when store updates<br>    });<br>    this.unsubscribeStore = x.unsubscribe;<br>  }<br><br>  componentWillUnmount() {<br>    if (this.unsubscribeStore) {<br>      this.unsubscribeStore(); // when unmounting component<br>    }<br>  }<br><br>  render() {<br>    const state = store.getState();<br><br>    return (<br>      &lt;&gt;<br>        &lt;h1&gt;Reimplement Redux&lt;/h1&gt;<br>        &lt;div className=&quot;card&quot;&gt;<br>          &lt;p&gt;<br>            counter: {state.counter}<br>          &lt;/p&gt;<br>          &lt;button<br>            onClick={() =&gt; {<br>              store.dispatch({<br>                type: &#39;INCREMENT&#39;,<br>              });<br>            }}<br>          &gt;<br>            +<br>          &lt;/button&gt;<br>          &lt;button<br>            onClick={() =&gt; {<br>              store.dispatch({<br>                type: &#39;DECREMENT&#39;,<br>              });<br>            }}<br>          &gt;<br>            -<br>          &lt;/button&gt;<br>        &lt;/div&gt;<br>      &lt;/&gt;<br>    );<br>  }<br>}<br><br>export default App;</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*rx7lD_PPe-cLdCHnTFrliA.gif" /><figcaption>Usign Store Subscribe Method and force Update</figcaption></figure><h4>Re-render the Root Element</h4><p><em>Warning: This brute-force method is for educational purposes only and has significant performance implications</em></p><p>Go to main.ts file of your React project:</p><p>You see this:</p><pre>import { StrictMode } from &#39;react&#39;<br>import { createRoot } from &#39;react-dom/client&#39;<br>import &#39;./index.css&#39;<br>import App from &#39;./App.tsx&#39;<br><br>createRoot(document.getElementById(&#39;root&#39;)!).render(<br>  &lt;StrictMode&gt;<br>    &lt;App /&gt;<br>  &lt;/StrictMode&gt;,<br>)</pre><p>Let’s render everything, each time there is a store change:</p><pre>import { store } from &#39;./store.ts&#39;<br>store.subscribe(() =&gt; {<br>  createRoot(document.getElementById(&#39;root&#39;)!).render(<br>    &lt;StrictMode&gt;<br>      &lt;App /&gt;<br>    &lt;/StrictMode&gt;,<br>  )  <br>});</pre><p>Now, we can use the function component version of App.tsx:</p><pre>import { store } from &#39;./store&#39;;<br>function App() {<br>  const state = store.getState();<br>  return (<br>    &lt;&gt;<br>      &lt;h1&gt;Reimplement Redux&lt;/h1&gt;<br>      &lt;div className=&quot;card&quot;&gt;<br>      &lt;p&gt;<br>            counter: {state.counter}<br>          &lt;/p&gt;<br>          &lt;button<br>            onClick={() =&gt; {<br>              store.dispatch({<br>                type: &#39;INCREMENT&#39;,<br>              });<br>            }}<br>          &gt;<br>            +<br>          &lt;/button&gt;<br>          &lt;button<br>            onClick={() =&gt; {<br>              store.dispatch({<br>                type: &#39;DECREMENT&#39;,<br>              });<br>            }}<br>          &gt;<br>            -<br>          &lt;/button&gt;<br>      &lt;/div&gt;<br>    &lt;/&gt;<br>  );<br>}<br><br>export default App;</pre><p>As you can see, we have used the state from getState method, and we also dispatch two actions to change the state. Each time we dispatch actions, the whole application is rendered.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*lsvyex7Mgs_aRcEKWYw2mg.gif" /><figcaption>Rerender React Root Element to get Reactivity</figcaption></figure><h3>Next Step</h3><p>We do not want to re-render the root element or manually call forceUpdate. We want to make it simple to use. For this, we have to implement useSelector hook from Redux. This will be in part II. Please comment on your thoughts! Also, what else do you need to be in part II?</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=80739772a1d8" width="1" height="1" alt=""><hr><p><a href="https://levelup.gitconnected.com/re-implement-redux-part-i-there-is-no-magic-80739772a1d8">Re-implement Redux Part I: There is No Magic</a> was originally published in <a href="https://levelup.gitconnected.com/">Level Up Coding</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Hidden Power of Postgres Logical Replication: A Step-by-Step Guide]]></title>
            <link>https://levelup.gitconnected.com/the-hidden-power-of-postgres-logical-replication-a-step-by-step-guide-c21b3ad60d51?source=rss-8f1fd23dbcf6------2</link>
            <guid isPermaLink="false">https://medium.com/p/c21b3ad60d51</guid>
            <category><![CDATA[replication]]></category>
            <category><![CDATA[backup]]></category>
            <category><![CDATA[logical-replication]]></category>
            <category><![CDATA[replica]]></category>
            <category><![CDATA[postgres]]></category>
            <dc:creator><![CDATA[Poorshad Shaddel]]></dc:creator>
            <pubDate>Mon, 28 Apr 2025 14:40:22 GMT</pubDate>
            <atom:updated>2025-04-28T14:40:22.892Z</atom:updated>
            <content:encoded><![CDATA[<p>Struggling with database scalability, cross-platform sync, or real-time analytics? Postgres logical replication could be your answer. In this guide, you’ll learn how to harness its publisher-subscriber model to replicate data seamlessly, ensure high availability, and offload queries to replicas — all while keeping transactions consistent.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Bqwq_hYLbTcAgAmM6k4ztg.jpeg" /><figcaption>Postgres Replica</figcaption></figure><h3>What is a Database Replica?</h3><p>It is simply a copy of your Database. What is the intention of having this copy?</p><ul><li><strong>High Availability and Fault Tolerance</strong>: If one fails, others can still serve data.(also in case of network issues.)</li><li><strong>Load Balancing</strong>: Replica can also respond to queries in order to reduce the load from Primary Database.</li></ul><h3>Logical Replication</h3><p>In Logical replication, we are replicating data objects and their changes, based on their replication identity(Could be Primary Key).</p><p>A publisher and subscriber model is used for doing this. The Replica simply subscribes to the changes and it might even cascade the changes to other replicas.</p><h4>Having a Logical Replication consist of 2 steps:</h4><p><strong>1- Take a Snapshot of the Table and copy it to the subscriber.<br>2- Send changes of publisher to subscriber in real-time</strong></p><p>These changes are coming the in same order as transactions, to the data consistency is guaranteed.</p><h3>Logical Replication Use Cases</h3><ul><li>Replication between major versions of PostgreSQL or in Different Platoforms(Windows and Linux)</li><li>Analytical Purpose: Gather data from multiple database to one</li><li>Sending incremental changes of the database or a few tables to subscribers in real-time</li><li>Firing Triggers for the transactional changes that are coming to subscriber</li></ul><h3>Hands On Logical Replication</h3><p>Before getting started there are a few concepts you should know:</p><h4>WAL</h4><p>This is Write Ahead Log, is used for data integrity. “WAL’s central concept is that changes to data files (where tables and indexes reside) must be written only after those changes have been logged” From Postgres Docs.</p><p>If you are using a postgres docker container, the WAL logs are kept here:</p><pre>/var/lib/postgresql/data/pg_wal</pre><p>For getting more infor about WAL logs you can run this query:</p><pre>SELECT * FROM pg_stat_wal;</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_ZdkE0jRQgPP716NxS0JCQ.png" /><figcaption>pg_stat_wal</figcaption></figure><h4>WAL Level</h4><p>This is the level of information we want to write to the WAL.</p><p>These are the options we can use:</p><ul><li><strong>logical</strong>: this includes enough information for doing a logical replication(those transactions that we talked about)</li><li><strong>minimal</strong>: this writes the least to WAL.(used when there are no replicas, you know that it would be much faster when you do not have replica and also you do not want to write a lot to WAL)</li><li><strong>replica</strong>: information for physical replication(streaming)</li></ul><p>Let’s use Docker to create a primary and a logical replica of the database:</p><pre>version: &#39;3.8&#39;<br><br>networks:<br>  postgres_network:<br>    driver: bridge<br><br>services:<br>  primary:<br>    image: postgres:17<br>    container_name: postgres_primary<br>    environment:<br>      POSTGRES_USER: postgres<br>      POSTGRES_PASSWORD: password<br>      POSTGRES_DB: mydb<br>    networks:<br>      - postgres_network<br>    volumes:<br>      - primary_data:/var/lib/postgresql/data<br>    command: &gt;<br>      postgres -c wal_level=logical<br>               -c max_wal_senders=10<br>               -c max_replication_slots=10<br><br>  replica:<br>    image: postgres:17<br>    container_name: postgres_replica<br>    environment:<br>      POSTGRES_USER: postgres<br>      POSTGRES_PASSWORD: password<br>      POSTGRES_DB: mydb<br>    networks:<br>      - postgres_network<br>    depends_on:<br>      - primary<br>    volumes:<br>      - replica_data:/var/lib/postgresql/data<br>    command: &gt;<br>      postgres -c wal_level=logical<br>               -c primary_conninfo=&quot;host=postgres_primary port=5432 user=postgres password=password&quot;<br>               -c hot_standby=on           <br>               -c default_transaction_read_only=on<br><br><br>volumes:<br>  primary_data:<br>  replica_data:</pre><p>Run this command to have your containers up and running:</p><pre>docker compose up</pre><p>Lets Create a Table on Primary as well(connect to container and run psql):</p><pre>docker exec -it postgres_primary bash <br>psql -U postgres -d mydb</pre><pre>CREATE TABLE users (<br>    id SERIAL PRIMARY KEY,<br>    username VARCHAR(50) NOT NULL UNIQUE,<br>    created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP<br>);</pre><p>If we look into our replica, we see that we do not have it there yet! we are getting there:</p><h4>Create a Publication on Primary</h4><p>We have to specify, which tables we are willing to publish.</p><p><strong>We have a variety of options to publish. All Tables, a few Tables, Schema, even Columns, or on specific operations.</strong></p><ul><li><strong>Tables</strong></li><li><strong>Schemas</strong></li><li><strong>Operation(insert, update, delete)</strong></li><li><strong>Condition(simply add a where clause)</strong></li></ul><pre>CREATE PUBLICATION my_publication FOR ALL TABLES; -- ALL<br>CREATE PUBLICATION users_publication FOR TABLE users; -- Just on a Table<br>CREATE PUBLICATION schema_publication FOR TABLES IN SCHEMA X;<br>CREATE PUBLICATION schema_publication FOR TABLES IN SCHEMA X WHERE (created_at &gt; &#39;2023&#39;);<br>CREATE PUBLICATION my_publication FOR ALL TABLES with (publish = &#39;insert&#39;);</pre><p>I am going to create one Publication on users table:</p><pre>mydb=# CREATE PUBLICATION users_publication FOR TABLE users;<br>CREATE PUBLICATION</pre><h4>Create a Subscription on Replica</h4><p>First connect to the replica:</p><pre>docker exec -it postgres_replica bash</pre><p>connect to the database:</p><pre> psql -U postgres -d mydb</pre><p>Some important notes before subscribing to replica:<br>- “<strong>Logical replication in PostgreSQL does not replicate schema changes”<br>- “Tables must have Primary Key”</strong></p><p>So we have to create table users before subscribing.</p><pre>CREATE TABLE users (<br>    id SERIAL PRIMARY KEY,<br>    username VARCHAR(50) NOT NULL UNIQUE,<br>    created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP<br>);</pre><p>Now, we have to subscribe to the publication:</p><pre>CREATE SUBSCRIPTION users_subscription<br>CONNECTION &#39;host=postgres_primary port=5432 dbname=mydb user=postgres password=password&#39;<br>PUBLICATION users_publication;</pre><p>If you have logs of your containers you might see this:</p><p>Synchronization finished.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*VAnrOdK_iS4sJso2HI5jKw.png" /><figcaption>Docker Container Logs</figcaption></figure><p>Now Let’s insert some data in the primary:</p><pre>INSERT INTO users (username) VALUES<br>(&#39;user1&#39;),<br>(&#39;user2&#39;),<br>(&#39;user3&#39;),<br>(&#39;user4&#39;),<br>(&#39;user5&#39;),<br>(&#39;user6&#39;),<br>(&#39;user7&#39;),<br>(&#39;user8&#39;),<br>(&#39;user9&#39;),<br>(&#39;user10&#39;);</pre><p>Now, if we select it on replica, we see the result as expected.</p><p>Also on the logs, we see that</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*M4fS3C2LHlsW-kj4-D5t2w.png" /><figcaption>replica logs when inserting something</figcaption></figure><h3>Conclusion</h3><p>With Postgres Logical Replication we have a lot of control over what we want to copy(publish and subscribe). One of the benefits is that, you can maintain data between different versions of Postgres and even on different Platforms. This is also helpful when you want to test your application on a new major version of the Database. <br>Two very important things:<br>- Schema changes are not applied automatically<br>- Tables must have primary key.</p><p><a href="https://github.com/pshaddel/postgres-logical-replica">https://github.com/pshaddel/postgres-logical-replica</a></p><ul><li><a href="https://scalegrid.io/blog/comparing-logical-streaming-replication-postgresql/">PostgreSQL Streaming Vs. Logical Replication [Comparison]</a></li><li><a href="https://www.postgresql.org/docs/current/logical-replication.html">Chapter 29. Logical Replication</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c21b3ad60d51" width="1" height="1" alt=""><hr><p><a href="https://levelup.gitconnected.com/the-hidden-power-of-postgres-logical-replication-a-step-by-step-guide-c21b3ad60d51">The Hidden Power of Postgres Logical Replication: A Step-by-Step Guide</a> was originally published in <a href="https://levelup.gitconnected.com/">Level Up Coding</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>