<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Siva Gollapalli on Medium]]></title>
        <description><![CDATA[Stories by Siva Gollapalli on Medium]]></description>
        <link>https://medium.com/@learnitnow?source=rss-9618834b293a------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*8hzggKYM3zCXpAzkihjIxg.jpeg</url>
            <title>Stories by Siva Gollapalli on Medium</title>
            <link>https://medium.com/@learnitnow?source=rss-9618834b293a------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 03:57:07 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@learnitnow/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Venturing Beyond Rails: Can Rage Supercharge Ruby Web APIs?]]></title>
            <link>https://learnitnow.medium.com/venturing-beyond-rails-can-rage-supercharge-ruby-web-apis-ba4ed94cb0da?source=rss-9618834b293a------2</link>
            <guid isPermaLink="false">https://medium.com/p/ba4ed94cb0da</guid>
            <category><![CDATA[performance]]></category>
            <category><![CDATA[ruby-on-rails]]></category>
            <category><![CDATA[fiber]]></category>
            <category><![CDATA[concurrency]]></category>
            <category><![CDATA[ruby]]></category>
            <dc:creator><![CDATA[Siva Gollapalli]]></dc:creator>
            <pubDate>Thu, 29 Oct 2026 15:05:45 GMT</pubDate>
            <atom:updated>2026-11-02T10:41:14.469Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-HsBGNt1SAqUlm7dWyGbhA.png" /><figcaption>Rage Vs Rails</figcaption></figure><p>I have spent the last fifteen years living and breathing Ruby on Rails. My journey began in the era of Rails 2.3, wrestling with early asset pipelines and ActiveRecord patterns, and today I find myself deploying production workloads on Rails 8. Throughout that decade and a half, Rails has been my automatic, undisputed default whenever a project called for a backend web API.</p><p>Yet, as much as I appreciate the ecosystem, a lingering question has followed me over the years: why is Rails the default answer to <em>every</em> API question? Alternative options exist. <a href="https://github.com/sinatra/sinatra">Sinatra</a> handles lightweight microservices gracefully, and <a href="https://github.com/hanami/hanami">Hanami</a> provides a clean, modern full-stack architecture. Still, Rails continues to dominate mindshare, often leaving Rubyists locked into its conventional multi-threaded model.</p><p>Curiosity got the better of me. When evaluating performance-sensitive API layers, I stumbled upon a promising underdog:<a href="https://github.com/rage-rb/rage"> Rage</a>.</p><h4>Meeting the Underdog: What Makes Rage Different?</h4><p><a href="https://github.com/rage-rb/rage">Rage</a> is not trying to be a sprawling full-stack framework with asset compilation and view engines. Instead, it positions itself similarly to Grape or Rails in API-only mode, providing pure routing, controllers, and serialization layers while retaining familiar Rails-like ergonomics.</p><p>The real magic, however, lies under the hood. Unlike traditional Rails apps that typically deploy on Puma using native OS threads, Rage runs on<a href="https://github.com/rage-rb/iodine"> Iodine</a> and leverages Ruby Fibers. Because fibers are cooperative user-space primitives, creating and context-switching between them requires significantly less overhead than OS threads. On paper, this non-blocking fiber runtime promises a massive leap in concurrent application throughput.</p><p>Naturally, theory means little without benchmarks. To see if Rage could genuinely challenge the reigning champion, I designed a head-to-head performance test across three realistic API workloads.</p><h3>Hardware and Benchmark Environment</h3><p>To ensure a fair and consistent baseline across both runtimes, all tests were executed on the same physical machine with matching worker allocations and database connection limits:</p><ul><li><strong>Hardware:</strong> Apple M3 with 36 GB Unified Memory</li><li><strong>Database:</strong> Local MySQL instance</li><li><strong>Rage Server:</strong> Iodine configured with 8 cluster workers and a maximum pool of 16 database connections</li><li><strong>Rails Server:</strong> Puma configured with 8 cluster workers, fixed at 16 threads per worker, and matching pool capacity</li></ul><p>With resource limits aligned on both sides, any variance comes down to framework overhead, runtime scheduling, and driver behavior.</p><h3>Scenario 1: The Heavy Database Wait (SELECT SLEEP(0.5))</h3><p>In this test, every request holds a MySQL connection idle for roughly 500 milliseconds before responding. It measures how many concurrent blocking database waits each runtime can hold open.<br>Run A: The Misleading First Attempt</p><p>My first run looked like a clear win for Rails. Then I looked closer. With an ActiveRecord pool of 16 and 8 Rage workers, Rage could open up to 128 database connections. That is close to MySQL’s default limit of 151, and it triggered 1040 Too many connections errors. Connection checkout, not fiber behavior, put an artificial ceiling on Rage’s throughput.</p><p>Run A is not a valid model comparison. I’m including it only to show the connection-pool artifact.</p><p>Run A: connection-starved baseline (Rage 8 workers, Rails 8 workers × 16 threads, AR pool 16)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*hodirFZPF0C2tcqC8mi6nQ.png" /></figure><p>Run B: The Corrected Comparison</p><p>For the second run I fixed the connection budget by raising MySQL max_connections to 256. I also right-sized the workers: Rage ran with just 2 workers, against 8 for Rails.</p><p>Run B: corrected (Rage 2 workers vs Rails 8 workers, MySQL max_connections = 256)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wDnBWy3hs9vgAz5lOZ0COg.png" /></figure><h4>The Verdict: Rage Wins, With 4x Fewer Workers</h4><p>With the connection budget fixed, the picture flipped:</p><p>Throughput: Rage was on par with Rails at 100 users (+0.6%), then pulled ahead by 26% at 150 users and 57% at 200 users.</p><p>Flat latency: Rage’s p95 stayed at about 578–579ms at every load level. The query itself takes 500ms, so Rage added almost nothing on top of the sleep.</p><p>Rails under pressure: Rails’ p95 climbed from 590ms to 1.10s, and its throughput barely moved as load increased. This is the signature of thread-pool queueing once all busy threads are occupied.</p><p>This is also the clearest proof of fiber overlap. If the fibers did not yield during the database wait, 2 workers could hold only about 2 concurrent sleeps, and latency would be in the multi-second range. It isn’t. Roughly 200 concurrent sleeps were multiplexed inside just 2 workers.</p><h3>Scenario 2: Real-World Database Read (Primary Key Lookup)</h3><p>For the second test, I removed the artificial sleep and simulated a standard API read: querying an uncached record by its primary key from a local MySQL instance and returning a small JSON payload.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*mlKj19ewdpTkKQO8UjH7DQ.png" /></figure><h4>The Verdict: Rage Surges Ahead</h4><p>When database queries execute quickly, the framework’s internal dispatch overhead becomes the defining bottleneck:</p><ul><li><strong>Throughput:</strong> Rage processed between 1.66x and 1.71x more requests per second than Rails.</li><li><strong>Latencies:</strong> Rage consistently maintained lower average, median, and 95th-percentile response times across all load levels.</li></ul><p>Because query durations were negligible, Rage’s slim middleware stack and fast dispatch pipeline allowed it to process records and return responses much faster than Rails.</p><h3>Scenario 3: The Baseline Health Check (Zero I/O)</h3><p>To isolate pure framework overhead, the final test called a minimal health check endpoint that performed zero database queries and simply returned an immediate HTTP status.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*I2D0oFKrm95yij2FQX-Usw.png" /></figure><h4>The Verdict: A Resounding Victory for Rage</h4><p>Under pure request routing, Rage demonstrated the power of lightweight fibers combined with Iodine’s high-speed socket handling:</p><ul><li><strong>Throughput:</strong> Rage achieved between 4.2x and 6.1x the throughput of Rails, sustaining over 29,000 requests per second.</li><li><strong>Latency stability:</strong> Rage kept latencies consistently in the single-digit millisecond range, while Rails suffered maximum latencies exceeding 2 seconds under heavy volume, likely due to thread queue contention and garbage collection pauses.</li></ul><h3>Benchmark Takeaways</h3><h4>Throughput Comparison (reqs/s)</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-dvznq0vLtfaGfH6gQw4lg.png" /></figure><h4>95th-Percentile Latency Comparison</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/956/1*JILjQOIs45JWkbbqlNZ2xg.png" /></figure><p><strong>What Does This Mean for Your Architecture?</strong></p><p>Across all three workloads, Rage matched or beat Rails. The Scenario 1 story is just as important as the win itself.</p><p>Fibers deliver on I/O-heavy workloads. In the corrected run, Rage held about 200 concurrent 500ms database waits on just 2 workers, with latency flat at the sleep floor. Rails needed 8 workers and still queued under load.</p><p>Provisioning can decide the winner. My first run made Rails look better. The real cause was a connection budget that starved Rage, not the fiber model. Before you trust any benchmark, check pool sizes, worker counts, and the database’s max_connections.</p><p>Rage is cheaper per request. On fast reads and trivial endpoints, its slim pipeline delivered 1.7x to 6x the throughput of Rails.</p><p>Rails will likely remain the powerhouse for full-stack monoliths and its rich ecosystem. But if you are designing dedicated, high-concurrency API services, Rage proves that the Ruby ecosystem has plenty of room to surprise us.</p><p><strong>Two notes on this version:</strong></p><ul><li><strong>Scenarios 2 and 3: </strong>I kept your original configuration (Iodine with 8 workers, Puma with 8 workers × 16 threads) for these two. If you re-ran them under different settings, send me the new numbers and I’ll update the post.</li><li><strong>Earlier explanation removed:</strong> I dropped the old claim that the Trilogy driver doesn’t yield fibers. Run B contradicts it, because the fibers clearly overlapped.</li></ul><p><em>Enjoyed this article? Please share it with your network to help others learn.</em></p><p><em>Stay connected for more insights:</em></p><ul><li><em>Follow me on </em><a href="https://learnitnow.medium.com/"><em>Medium</em></a><em> for the latest articles.</em></li><li><em>Connect with me on </em><a href="https://www.linkedin.com/in/siva-gollapalli-b8026a61/"><em>LinkedIn</em></a><em> for professional updates.</em></li><li><em>Join the conversation on </em><a href="https://x.com/sivagollapalli"><em>Twitter</em></a><em> and share your thoughts.</em></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ba4ed94cb0da" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Magic Timeouts That Slashed Our Database Connections in Rails]]></title>
            <link>https://learnitnow.medium.com/the-magic-timeouts-that-slashed-our-database-connections-in-rails-2eb8975016de?source=rss-9618834b293a------2</link>
            <guid isPermaLink="false">https://medium.com/p/2eb8975016de</guid>
            <category><![CDATA[sharding]]></category>
            <category><![CDATA[activerecord]]></category>
            <category><![CDATA[mysql]]></category>
            <category><![CDATA[ruby-on-rails]]></category>
            <category><![CDATA[connection-management]]></category>
            <dc:creator><![CDATA[Siva Gollapalli]]></dc:creator>
            <pubDate>Thu, 27 Aug 2026 08:30:43 GMT</pubDate>
            <atom:updated>2026-08-28T06:07:46.730Z</atom:updated>
            <content:encoded><![CDATA[<p>In a <a href="https://medium.com/@learnitnow/connect-multiple-databases-in-rails-without-changing-code-a-multi-tenant-sharding-story-aacd7fb4b2ce">previous post</a>, we explored how to connect to multiple databases dynamically and manage shards using native Rails building blocks. To satisfy strict compliance requirements, we isolated customer data across multiple databases. For a subset of customers, we co-located data on the same physical MySQL instance using distinct schemas.</p><p>However, multi-tenancy with separate schemas introduced an unexpected scaling bottleneck: connection bloat. Because Rails maintains an independent connection pool per schema, connection reuse plummeted, and the total number of open database connections skyrocketed.</p><p>Here is how we solved our connection explosion using two critical timeout configurations.</p><h4>Understanding the ActiveRecord Connection Pool</h4><p>Before jumping into the fixes, let’s briefly look under the hood. Connection pooling exists to avoid the high overhead of establishing fresh TCP connections for every database query. Rails provides built-in pool telemetry via ActiveRecord::Base.connection_pool.stat:</p><pre>ActiveRecord::Base.connection_pool.stat<br># =&gt; { size: 15, connections: 1, busy: 1, dead: 0, idle: 0, waiting: 0, checkout_timeout: 5 }</pre><p>Breaking down the key metrics:</p><ul><li>size: The hard ceiling on the number of connections allowed in the pool.</li><li>connections: Total open database connections currently managed by the pool.</li><li>busy: Connections actively checked out and executing queries in working threads.</li><li>idle: Open, healthy connections resting in the pool and ready for checkout.</li><li>dead: Stale connections recovered after an unexpected thread termination.</li><li>waiting: The queue of threads blocked waiting for an available connection.</li><li>checkout_timeout: The maximum seconds a thread waits before raising an ActiveRecord::ConnectionTimeoutError.</li></ul><p>In a single-database setup, idle connections are reused efficiently across web requests. But when requests are split across dozens of schema-specific pools, connections remain idle in their respective pools without getting reused, quickly overwhelming the database server.</p><h4>Fix #1: Client-Side Cleanup with idle_timeout and reaping_frequency</h4><p>Our first step was ensuring that Rails cleans up its own idle connections when traffic subsides.</p><p>Rails provides idle_timeout to define how long a connection can sit unused in the pool before becoming eligible for eviction. To enforce this, you must also configure reaping_frequency. A background reaper thread runs at the specified interval to scan the pool and terminate connections that exceed their idle_timeout.</p><pre># config/database.yml<br>production:<br>  adapter: mysql2<br>  pool: 15<br>  idle_timeout: 300       # 5 minutes<br>  reaping_frequency: 60   # Run cleanup every 60 seconds</pre><blockquote><strong><em>Pro Tip:</em></strong><em> Always set </em><em>reaping_frequency to a value significantly lower than </em><em>idle_timeout so the reaper actively flushes stale connections before they accumulate.</em></blockquote><p>While this client-side cleanup helped release idle connections during traffic dips, we still observed high idle connection counts lingering on the MySQL server itself.</p><h4>Fix #2: Server-Side Enforcement with wait_timeout</h4><p>wait_timeout determines how many seconds the MySQL server will wait before terminating an inactive, idle connection.</p><p>By default, the Rails mysql2 adapter overrides MySQL&#39;s server default by setting the session wait_timeout to an astronomical value: 2147483 seconds (roughly 24 days). Rails does this intentionally to avoid dropped connection errors under standard workloads.</p><p>However, in a multi-pool architecture, this behavior prevents MySQL from ever reclaiming idle connections. Even if you modify the global wait_timeout on the database server, Rails&#39; per-session override renders the global configuration ineffective.</p><p>To fix this, we explicitly lowered the session wait_timeout in our database configuration to 1 hour (3600 seconds):</p><pre># config/database.yml<br>production:<br>  adapter: mysql2<br>  pool: 15<br>  idle_timeout: 300<br>  reaping_frequency: 60<br>  variables:<br>    wait_timeout: 3600    # Explicitly enforce 1-hour session timeout</pre><h4>The Results</h4><p>Combining client-side pool reaping with an explicit session wait_timeout yielded immediate improvements:</p><ol><li><strong>Dramatic Connection Reduction:</strong> Open database connections dropped significantly across all instances, keeping connection counts well below MySQL’s max_connections threshold.</li><li><strong>Reduced MySQL Memory Footprint:</strong> Because each open MySQL thread consumes memory buffers (such as read_buffer_size and sort_buffer_size), cutting idle connections freed up valuable RAM on the database tier.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qygnSvRi8Y7fZF6Efk6qIg.png" /><figcaption>Connections after implementing timeouts</figcaption></figure><h4>Summary</h4><p>When scaling multi-tenant or multi-schema Rails applications, default connection settings can silently consume database resources. Tuning idle_timeout, reaping_frequency, and session wait_timeout ensures your connection lifecycle is managed cleanly from both the application and database tiers.</p><p><em>Enjoyed this article? Please share it with your network to help others learn.</em></p><p><em>Stay connected for more insights:</em></p><ul><li><em>Follow me on </em><a href="https://learnitnow.medium.com/"><em>Medium</em></a><em> for the latest articles.</em></li><li><em>Connect with me on </em><a href="https://www.linkedin.com/in/siva-gollapalli-b8026a61/"><em>LinkedIn</em></a><em> for professional updates.</em></li><li><em>Join the conversation on </em><a href="https://x.com/sivagollapalli"><em>Twitter</em></a><em> and share your thoughts.</em></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2eb8975016de" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Connect Multiple Databases in Rails Without Changing Code: A Multi-Tenant Sharding Story]]></title>
            <link>https://medium.com/hackernoon/connect-multiple-databases-in-rails-without-changing-code-a-multi-tenant-sharding-story-aacd7fb4b2ce?source=rss-9618834b293a------2</link>
            <guid isPermaLink="false">https://medium.com/p/aacd7fb4b2ce</guid>
            <category><![CDATA[database-sharding]]></category>
            <category><![CDATA[ruby-on-rails]]></category>
            <category><![CDATA[scalability]]></category>
            <category><![CDATA[activerecord]]></category>
            <category><![CDATA[multitenancy]]></category>
            <dc:creator><![CDATA[Siva Gollapalli]]></dc:creator>
            <pubDate>Wed, 29 Jul 2026 14:22:23 GMT</pubDate>
            <atom:updated>2026-07-29T14:22:23.424Z</atom:updated>
            <content:encoded><![CDATA[<p><strong>The problem that kept us up at night</strong></p><p>Picture this: your Rails app is growing fast. New customers are signing up every week, and one morning an enterprise client drops a requirement that changes everything — “We need our data in a completely separate database. Compliance won’t allow shared storage.”</p><p>Then the next customer asks for the same thing. And the next.</p><p>Suddenly you’re staring at a hard question: How do you add one more customer database to a Rails app without touching the code every single time? And an even scarier one — how do you manage configuration when every customer might demand their own database?</p><p>That was exactly our situation. We needed to split customer data across multiple databases, where each customer owns a separate schema — sometimes on the same instance, sometimes on a different one — using the customer’s organization as the shard key.</p><p>Here’s the approach we landed on, and it has scaled beautifully in production. Let’s dive in.</p><h3><strong>Rails multi-database support:</strong></h3><p>Most Rails developers know that Rails supports multiple databases out of the box, letting you separate data based on your use case. To get started, you define your database details in database.yml like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ikQYM0o6K9qmjVs8tsnpUA.png" /><figcaption>Database.yml with Multi DB details</figcaption></figure><p>When the Rails application boots, it reads database.yml and builds the necessary configurations. Once it has those configs, it attempts to connect — and if a database isn’t reachable, it throws an “unable to connect” error.</p><p>Notice the three roles here:</p><ul><li><strong>primary </strong>— the main database all customers rely on.</li><li><strong>primary_replica</strong> — a read replica that distributes the load.</li><li><strong>metadb</strong> — stores configuration and metadata related to each organization.</li></ul><h3><strong>Why the static approach breaks down?</strong></h3><p>To make this setup truly customer-centric, the naive path is: spin up a new database instance, then manually update database.yml, then redeploy.</p><p>That is not scalable. Editing a config file and shipping a code change for every new customer is slow, error-prone, and turns your onboarding into an engineering ticket every single time.</p><p>So we flipped the model. Instead of hard coding database configs, we attach customer database configs dynamically while the application is booting.</p><h3><strong>The solution: dynamic shards from a database table</strong></h3><p>The key idea is simple: store customer database details as data, not as code.</p><p>We created a DBDetails table that holds each customer’s connection details. During boot, we read those rows, build the corresponding Active Record configurations, and append them as shards.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*a5GXq8p97RnI3-AaVgdVQg.png" /><figcaption>Building DB configs dynamically</figcaption></figure><p>Each shard connects using a handle like shard_1, shard_2, shard_3, and so on. So the onboarding flow for a new customer becomes refreshingly boring (in the best way):</p><ul><li>Create a new DBDetail entry with the customer’s connection details.</li><li>Re-boot the application.</li><li>Start serving that customer — no code change required.</li></ul><h3><strong>Routing each request to the right shard:</strong></h3><p>Defining shards is only half the story. During the request lifecycle, you need to point each request at the correct customer database. You do that by switching the shard based on the organization on the incoming request:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*I7wLlm2uKVfl7Ivg5aAOdA.png" /><figcaption>Using Shard in request life cycle</figcaption></figure><h3><strong>The payoff:</strong></h3><p>What started as a compliance headache became one of our most flexible architecture decisions. By treating database configuration as data instead of code, adding a new customer went from “open a PR and redeploy” to “insert a row and reboot.”</p><p>That’s the difference between an app that fights every new customer and one that welcomes them.</p><p><em>Enjoyed this article? Please share it with your network to help others learn.</em></p><p><em>Stay connected for more insights:</em></p><ul><li><em>Follow me on </em><a href="https://learnitnow.medium.com/"><em>Medium</em></a><em> for the latest articles.</em></li><li><em>Connect with me on </em><a href="https://www.linkedin.com/in/siva-gollapalli-b8026a61/"><em>LinkedIn</em></a><em> for professional updates.</em></li><li><em>Join the conversation on </em><a href="https://x.com/sivagollapalli"><em>Twitter</em></a><em> and share your thoughts.</em></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=aacd7fb4b2ce" width="1" height="1" alt=""><hr><p><a href="https://medium.com/hackernoon/connect-multiple-databases-in-rails-without-changing-code-a-multi-tenant-sharding-story-aacd7fb4b2ce">Connect Multiple Databases in Rails Without Changing Code: A Multi-Tenant Sharding Story</a> was originally published in <a href="https://medium.com/hackernoon">HackerNoon.com</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[From a Survey Drawing to Google Earth: A Simple Way to See Lake Boundaries]]></title>
            <link>https://learnitnow.medium.com/from-a-survey-drawing-to-google-earth-a-simple-way-to-see-lake-boundaries-95d064f2202c?source=rss-9618834b293a------2</link>
            <guid isPermaLink="false">https://medium.com/p/95d064f2202c</guid>
            <category><![CDATA[hyderabad]]></category>
            <category><![CDATA[ai-agent]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[gis]]></category>
            <category><![CDATA[hydraa]]></category>
            <dc:creator><![CDATA[Siva Gollapalli]]></dc:creator>
            <pubDate>Tue, 30 Jun 2026 14:32:41 GMT</pubDate>
            <atom:updated>2026-06-30T14:32:41.185Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7gREcgSS5fDJH2v9NMuI1g.png" /><figcaption>Enhanced Google image of a Lake</figcaption></figure><p>Have you ever looked at a government survey drawing and wondered, “How can I actually see this place on a map?”</p><p>For most people, survey maps filled with latitude, longitude, and technical symbols are difficult to understand. They contain valuable information, but only for those familiar with GIS software or surveying.</p><p>Recently, I worked through a simple process that transformed a survey drawing into something anyone can explore in Google Earth. Here’s the journey.</p><p><strong>Step 1: Start with the Survey Map</strong></p><p>The process begins with a survey drawing or lake boundary plan.</p><p>These documents usually contain:</p><ul><li>Boundary point numbers (A1, A2, A3…)</li><li>Latitude and longitude coordinates</li><li>Distances between points</li><li>A sketch of the lake or land parcel</li></ul><p>While this information is accurate, it isn’t easy for the average person to visualize.</p><p><strong>Step 2: Extract the Coordinates</strong></p><p>Instead of manually typing every coordinate, AI can read the survey image and extract each latitude and longitude.</p><p><em>Note: I have extract coordinates by clicking pictures instead of uploading survey file as AI unable recognize the lat and lon values from survey file.</em></p><p>For example:</p><ul><li>A1 → 17°30&#39;31.129&quot;N, 78°19&#39;17.634&quot;E</li><li>A2 → 17°30&#39;31.093&quot;N, 78°19&#39;17.807&quot;E</li><li>…</li><li>A22 → 17°30&#39;24.023&quot;N, 78°19&#39;17.530&quot;E</li></ul><p>This saves time and greatly reduces human error.</p><p><strong>Step 3: Convert the Coordinates into a KML File</strong></p><p>Once the coordinates are extracted, AI converts them into a KML (Keyhole Markup Language) file.</p><p>A KML file is recognized by:</p><ul><li>Google Earth</li><li>Google Earth Pro</li><li>QGIS</li><li>ArcGIS</li><li>Many mapping applications</li></ul><p>Instead of looking at numbers in a table, you now have a digital map.</p><p><strong>Step 4: Draw the Lake Boundary</strong></p><p>The extracted coordinates are connected in sequence.</p><p>The result is a polygon representing the actual surveyed boundary of the lake or land parcel.</p><p>Now anyone can zoom in, rotate the map, and understand the exact location.</p><p><strong>Step 5: Add a Buffer Around the Boundary</strong></p><p>Sometimes regulations require a protected area around a lake.</p><p>In this example, a 9-meter buffer was generated around the boundary.</p><p>This creates a second polygon showing the protected zone surrounding the lake.</p><p>Such buffer zones are useful for:</p><ul><li>Environmental protection</li><li>Planning permissions</li><li>Construction setbacks</li><li>Encroachment checks</li><li>Government compliance</li></ul><p><strong>Step 6: Open the File in Google Earth</strong></p><p>After opening the generated KML file, you can:</p><ul><li>View the lake in satellite imagery.</li><li>See the surveyed boundary overlaid on the landscape.</li><li>Inspect nearby roads and buildings.</li><li>Measure distances.</li><li>Share the map with others.</li></ul><p>No specialized GIS knowledge is required.</p><p><strong>Why This Matters?</strong></p><p>Many people struggle to interpret official survey documents.</p><p>By combining AI with publicly available mapping tools, complex technical drawings become easy-to-understand visual maps.</p><p>This approach can help:</p><ul><li>Citizens understand nearby lakes and water bodies.</li><li>Property owners verify whether land lies within a lake buffer.</li><li>Environmental volunteers monitor lake boundaries.</li><li>Local communities identify encroachments.</li><li>Government departments communicate survey information more effectively.</li></ul><p><strong>The Bigger Picture</strong></p><p>AI is not replacing surveyors or official records. Instead, it acts as a bridge between technical documents and everyday understanding.</p><p>A scanned survey drawing that once required expert interpretation can now be transformed into an interactive map within minutes.</p><p>As AI continues to improve, accessing and understanding public geographic information will become easier for everyone — not just GIS professionals.</p><p>Sometimes, making information useful isn’t about collecting more data. It’s about presenting existing data in a way that everyone can understand. Converting survey coordinates into interactive maps is one small but meaningful step toward making geographic information accessible to all.</p><p><em>Enjoyed this article? Please share it with your network to help others learn.</em></p><p><em>Stay connected for more insights:</em></p><ul><li><em>Follow me on </em><a href="https://learnitnow.medium.com/"><em>Medium</em></a><em> for the latest articles.</em></li><li><em>Connect with me on </em><a href="https://www.linkedin.com/in/siva-gollapalli-b8026a61/"><em>LinkedIn</em></a><em> for professional updates.</em></li><li><em>Join the conversation on </em><a href="https://x.com/sivagollapalli"><em>Twitter</em></a><em> and share your thoughts.</em></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=95d064f2202c" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Agent Harness: The Unsung Hero Behind Every Smart AI ]]></title>
            <link>https://learnitnow.medium.com/agent-harness-the-unsung-hero-behind-every-smart-ai-6eb590fc5724?source=rss-9618834b293a------2</link>
            <guid isPermaLink="false">https://medium.com/p/6eb590fc5724</guid>
            <category><![CDATA[generative-ai-tools]]></category>
            <category><![CDATA[ruby]]></category>
            <category><![CDATA[ai-agent]]></category>
            <category><![CDATA[harness-engineering]]></category>
            <category><![CDATA[ai]]></category>
            <dc:creator><![CDATA[Siva Gollapalli]]></dc:creator>
            <pubDate>Wed, 06 May 2026 09:36:39 GMT</pubDate>
            <atom:updated>2026-05-13T06:21:00.722Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="Agent Harness and its components" src="https://cdn-images-1.medium.com/max/1024/1*AGwqwy7xIZGs-sicRZ1l9w.png" /></figure><p>Imagine this.</p><p>You’ve just built a brilliant AI agent. It answers questions, calls APIs, maybe even writes code. You’re excited. You test it a few times — it works great.</p><p>Then you put it in front of real users.</p><p>Suddenly:</p><ul><li>It forgets context 😵</li><li>Calls the wrong tool 🔧❌</li><li>Produces weird outputs 🤨</li><li>And occasionally goes… completely off track 🚀</li></ul><p>What happened?</p><p>You didn’t build a system. You built a brain without a body.</p><p>That’s where the <strong>agent harness</strong> comes in.</p><h3>So… What Is an Agent Harness? 🧠➡️🚗</h3><p>Think of an AI agent as a super-smart brain.</p><p>Now imagine putting that brain on a table and expecting it to drive a car.</p><p>Not gonna work, right? 😄</p><p>The <strong>agent harness</strong> is everything around that brain that makes it usable:</p><ul><li>The steering wheel</li><li>The brakes</li><li>The dashboard</li><li>The sensors</li></ul><p>In simple terms:</p><blockquote><em>The harness is the system that </em>runs, manages, and controls<em> the agent.</em></blockquote><p>It takes care of:</p><ul><li>How inputs come in</li><li>How decisions are executed</li><li>How memory is maintained</li><li>How outputs are delivered safely</li></ul><p>Without it, your agent is just… thinking into the void.</p><h3>Wait… Isn’t That Just “Building an Agent”? 🤔</h3><p>Not quite. This is where things get interesting.</p><p>When people say they’re “building an agent,” they usually mean:</p><ul><li>Writing prompts ✍️</li><li>Designing reasoning flows 🧩</li><li>Enabling tool usage 🔌</li></ul><p>That’s like teaching someone how to think.</p><p>But the harness?</p><p>That’s teaching them how to <em>live in the real world</em>.</p><p>Here’s the difference in plain English:</p><ul><li><strong>Agent</strong> → “What should I do?”</li><li><strong>Harness</strong> → “How do I do it safely, reliably, and repeatedly?”</li></ul><p>Or even simpler:</p><ul><li>Agent = brain 🧠</li><li>Harness = body + environment 🏃</li></ul><p>You need both. Otherwise, your “genius” agent will trip over its own shoelaces 😅</p><h3>Let’s Walk Through the Harness (Like a Guided Tour 🚶)</h3><p>Let’s pretend you’re a user interacting with your AI system. What actually happens behind the scenes?</p><h4>Step 1: You Say Something 💬</h4><p>Your message enters through the <strong>input interface</strong>.</p><p>This could be:</p><ul><li>A chat UI</li><li>An API call</li><li>A trigger from another system</li></ul><p>The harness cleans it up, validates it, and says:<br> “Alright, let’s handle this properly.”</p><h4>Step 2: The Prompt Gets Assembled 🧱</h4><p>Before the model even sees your request, the <strong>prompt manager</strong> gets to work.</p><p>It carefully builds context:</p><ul><li>System instructions</li><li>Conversation history</li><li>Relevant memory</li></ul><p>Think of it as a chef plating a dish 🍽️<br> Same ingredients, but presentation matters a lot.</p><h4>Step 3: The Brain Kicks In 🧠</h4><p>Now the <strong>LLM (model)</strong> does its thing.</p><p>It:</p><ul><li>Interprets the request</li><li>Decides what to do</li><li>Generates a response or action</li></ul><p>But here’s the twist — it doesn’t act alone.</p><h4>Step 4: “Hmm… I Need a Tool” 🔧</h4><p>If the agent needs external help, the <strong>tool router</strong> jumps in.</p><p>Maybe it needs to:</p><ul><li>Fetch data</li><li>Call an API</li><li>Do a calculation</li></ul><p>The harness ensures:</p><ul><li>The right tool is used ✅</li><li>Inputs are correct ✅</li><li>Outputs come back clean ✅</li></ul><p>No chaos. No guessing.</p><h4>Step 5: Memory Time 🧠💾</h4><p>The harness also manages memory.</p><ul><li>Short-term: “What did we just talk about?”</li><li>Long-term: “What do I know about this user?”</li></ul><p>This is what makes the agent feel:</p><ul><li>Consistent</li><li>Personalized</li><li>Less forgetful (finally 😄)</li></ul><h4>Step 6: “Is This Even Good?” 👀</h4><p>Before sending anything back, the <strong>evaluation &amp; monitoring layer</strong> watches closely.</p><p>It tracks:</p><ul><li>Quality</li><li>Errors</li><li>Latency</li><li>Cost</li></ul><p>Because in production, “it worked once” is not good enough 😅</p><h4>Step 7: Safety Check 🚨</h4><p>Now comes the bouncer at the club: <strong>guardrails</strong>.</p><p>They ensure:</p><ul><li>No harmful output</li><li>No unsafe actions</li><li>No policy violations</li></ul><p>Basically:<br> “Cool response… but let’s not get fired today.”</p><h4>Step 8: Final Touch ✨</h4><p>Finally, the <strong>output formatter</strong> cleans everything up.</p><ul><li>Structures the response</li><li>Formats it nicely</li><li>Makes it usable for UI or APIs</li></ul><p>And <em>then</em> it reaches the user.</p><p>Smooth. Clean. Professional.</p><h3>So Who’s Already Doing This? 🌍</h3><p>You’re not alone in this journey. Some popular “harness-like” systems include:</p><ul><li><a href="https://www.langchain.com/langgraph"><strong>LangChain / LangGraph</strong></a> → Great for building structured agent workflows</li><li><a href="https://microsoft.github.io/autogen/stable//index.html"><strong>AutoGen</strong></a> → Multi-agent collaboration (agents talking to agents 🤯)</li><li><a href="https://github.com/anthropics/claude-code"><strong>Claude Code</strong></a> -&gt; Coding agent built on top of claude</li><li><a href="https://github.com/HKUDS/OpenHarness"><strong>openharness</strong></a> -&gt; open source harness which can be used</li><li><a href="https://github.com/sivagollapalli/openharness-rb"><strong>openharness-rb</strong></a><strong> -&gt; openharness </strong>ported to Ruby by me</li></ul><p>Each of these is essentially trying to solve the same problem:</p><blockquote><em>“How do we make agents actually work in the real world?</em></blockquote><h3>Why This All Matters (More Than You Think) ⚠️</h3><p>Here’s the truth:</p><p>Most agent demos look amazing… until they meet reality.</p><p>Without a harness:</p><ul><li>Things break</li><li>Outputs drift</li><li>Bugs hide</li><li>Costs explode 💸</li></ul><p>With a harness:</p><ul><li>Systems become reliable ✅</li><li>Behavior becomes predictable ✅</li><li>Debugging becomes possible ✅</li><li>Scaling becomes real ✅</li></ul><p>In fact, in many production systems:</p><blockquote><em>The harness is more complex — and more important — than the agent itself.</em></blockquote><p>Yeah. Let that sink in for a second 😄</p><h3>Final Thought: You’re Not Just Building Intelligence🧠</h3><p>You’re building a <strong>system that can use intelligence effectively</strong>.</p><p>Anyone can call an LLM.</p><p>But building something that:</p><ul><li>Works every time</li><li>Handles edge cases</li><li>Stays safe</li><li>Improves over time</li></ul><p>That’s where the real engineering begins.</p><p>And that… is the job of the <strong>agent harness</strong> 🚀</p><p><em>Enjoyed this article? Please share it with your network to help others learn.</em></p><p><em>Stay connected for more insights:</em></p><ul><li><em>Follow me on </em><a href="https://learnitnow.medium.com/"><em>Medium</em></a><em> for the latest articles.</em></li><li><em>Connect with me on </em><a href="https://www.linkedin.com/in/siva-gollapalli-b8026a61/"><em>LinkedIn</em></a><em> for professional updates.</em></li><li><em>Join the conversation on </em><a href="https://x.com/sivagollapalli"><em>Twitter</em></a><em> and share your thoughts.</em></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6eb590fc5724" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Building Self-Debugging AI Agents for Ruby Microservices: Our Journey Through 7 Iterations]]></title>
            <link>https://learnitnow.medium.com/building-self-debugging-ai-agents-for-ruby-microservices-our-journey-through-7-iterations-e836526a688b?source=rss-9618834b293a------2</link>
            <guid isPermaLink="false">https://medium.com/p/e836526a688b</guid>
            <category><![CDATA[multi-agent-systems]]></category>
            <category><![CDATA[ruby]]></category>
            <category><![CDATA[agents]]></category>
            <category><![CDATA[rubyllm]]></category>
            <category><![CDATA[self-debugging]]></category>
            <dc:creator><![CDATA[Siva Gollapalli]]></dc:creator>
            <pubDate>Fri, 03 Apr 2026 13:59:47 GMT</pubDate>
            <atom:updated>2026-04-03T13:59:47.270Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*TXZTO_ruUA3o1ffNlnWggg.png" /></figure><p>Imagine if your microservices could debug themselves. Not by running tests or scanning logs, but by actually reading their own source code, tracing the logic step by step, and even asking <em>you</em> questions when they get stuck.</p><p>Sounds like science fiction? We made it real. But it took seven iterations — a journey full of trial, error, and learning.</p><p>In this post, I’ll take you through that journey. Every wrong turn, every aha moment, and every architectural pivot — so you can learn from our experience without repeating our mistakes.</p><h3>The Beginning: Two Simple Sinatra Microservices</h3><p>We started with two Ruby microservices built with Sinatra:</p><ul><li><strong>Inventory Service</strong> (port 4001): Manages products and stock. It supports CRUD operations for products, updates stock, and checks availability.</li><li><strong>Order Service</strong> (port 4002): Handles order placement. When you POST to /orders, it calls the inventory service to check availability, get the price, deduct stock, then creates the order.</li></ul><p>The catch? The order service makes HTTP calls to the inventory service during order placement. This cross-service dependency is exactly where debugging gets tricky in real-world systems.</p><h3>Iteration 1: The Naive Approach — One Agent Per Service</h3><p>Our first idea was simple: give each service its own /agent endpoint. Send it a prompt, and it uses a large language model (LLM) with some tools to debug.</p><p>We added the ruby_llm gem to both services and created four tools per agent:</p><ul><li>ReadCodebase — reads the service’s own app.rb source code</li><li>CallOrderEndpoint / CallInventoryEndpoint — makes live HTTP calls to test endpoints</li><li>AskInventoryAgent / AskOrderAgent — calls the other service’s /agent endpoint for cross-service debugging</li><li>AskUser — asks the user clarifying questions</li></ul><pre>class ReadCodebase &lt; RubyLLM::Tool<br>  description &quot;Reads the order service source code (app.rb)&quot;<br>  def execute<br>    File.read(File.join(__dir__, &#39;app.rb&#39;))<br>  end<br>end</pre><p><strong>What went wrong?</strong> The agents were making <em>live</em> HTTP calls to debug. But users don’t want the agent to actually hit endpoints — they want it to <em>trace the code logic</em> with the parameters they provide and figure out what <em>would</em> happen.</p><h3>Iteration 2: Dry-Run Instead of Live Calls</h3><p>We replaced the live call tools with a single DryRunEndpoint tool. Instead of making real HTTP requests, it reads the source code and instructs the LLM to trace through the logic step by step.</p><pre>class DryRunEndpoint &lt; RubyLLM::Tool<br>  description &quot;Dry-runs a given endpoint by reading the source code and tracing logic step by step...&quot;<br>  param :method, desc: &quot;HTTP method: GET, POST, PATCH, DELETE&quot;<br>  param :path, desc: &quot;The endpoint path&quot;<br>  param :params, desc: &quot;JSON string of request parameters&quot;, required: false<br>  param :response, desc: &quot;The actual response the user received&quot;, required: false<br><br>  def execute(method:, path:, params: nil, response: nil)<br>    source = File.read(File.join(__dir__, &#39;app.rb&#39;))<br>    { source: source, instruction: &quot;Trace through the #{method.upcase} &#39;#{path}&#39; endpoint...&quot; }<br>  end<br>end</pre><p><strong>What went wrong?</strong> We still had ReadCodebase as a separate tool, but DryRunEndpoint already reads the source code internally. Two tools doing the same thing — redundant and confusing.</p><h3>Iteration 3: Eliminating Redundancy</h3><p>We dropped ReadCodebase entirely. Now, DryRunEndpoint handles both reading the source and tracing the logic. Each agent went from four tools down to three: DryRunEndpoint, AskOtherServiceAgent, and AskUser.</p><p><strong>What went wrong?</strong> The agents were talking to each other directly via HTTP. The order agent would call the inventory agent’s /agent endpoint and vice versa. This created a tangled peer-to-peer mesh. Who’s coordinating? Who decides when to stop?</p><h3>Iteration 4: Hub-and-Spoke with an Orchestrator</h3><p>To untangle the mess, we introduced a third service: agent-service (port 4000). This orchestrator became the single entry point for users. It has two tools — AskOrderAgent and AskInventoryAgent — and routes queries to the right service agent.</p><p>The service agents lost their cross-service tools. When they encounter a cross-service HTTP call in the code, they report back what URL/method/params it would send, and the orchestrator decides whether to query the other agent.</p><pre>user<br>          ↕<br>    agent-service (orchestrator)<br>      ↙         ↘<br>order-agent    inventory-agent<br>(isolated)      (isolated)</pre><p><strong>What went wrong?</strong> The session_id wasn’t being passed reliably across services. The LLM had to remember to include it as a tool parameter — and it often forgot.</p><h3>Iteration 5: Baking Session ID into Tool Initialization</h3><p>Instead of making session_id a tool parameter that the LLM has to remember to pass, we injected it via the tool’s constructor:</p><pre>class AskOrderAgent &lt; RubyLLM::Tool<br>  def initialize(session_id)<br>    @session_id = session_id<br>  end<br><br>  def execute(prompt:)<br>    # @session_id is always included automatically<br>    req.body = { prompt: prompt, session_id: @session_id }.to_json<br>    # ...<br>  end<br>end</pre><p>The /agent endpoint creates tool instances with the session ID baked in:</p><pre>chat.with_tools(AskOrderAgent.new(session_id), AskInventoryAgent.new(session_id), AskUser)</pre><p><strong>What went wrong?</strong> Conversation continuity was broken. Each new message replayed the entire history by calling chat.ask for every past user message, which re-triggered all tool calls from scratch. A simple follow-up answer would cause the agent to re-read the codebase, re-trace the logic, and re-ask the same questions.</p><h3>Iteration 6: Shared Memory and Context Injection</h3><p>Two big improvements:</p><p><strong>Shared memory:</strong> Instead of each service maintaining its own session files, we created a single shared-memory/ directory with JSON files. Every agent reads and writes to the same session file.</p><pre>{<br>  &quot;session_id&quot;: &quot;uuid&quot;,<br>  &quot;created_at&quot;: &quot;iso8601&quot;,<br>  &quot;updated_at&quot;: &quot;iso8601&quot;,<br>  &quot;entries&quot;: [<br>    {<br>      &quot;agent&quot;: &quot;orchestrator&quot;,<br>      &quot;type&quot;: &quot;routing&quot;,<br>      &quot;content&quot;: &quot;Asking order-service: Debug POST /orders...&quot;,<br>      &quot;timestamp&quot;: &quot;iso8601&quot;<br>    }<br>  ]<br>}</pre><p><strong>Context injection instead of replay:</strong> Instead of replaying history by re-calling chat.ask for each past message (which re-executes everything), we inject the conversation history into the system prompt via with_instructions. The LLM sees the full history and gets a &quot;Continue from where you left off&quot; instruction.</p><pre>history = SharedMemory.conversation_context(session_id)<br>system_prompt = &quot;#{BASE_PROMPT}\n\n&lt;shared-memory&gt;\n#{history}\n&lt;/shared-memory&gt;\n\nResume from where the conversation left off.&quot;<br>chat = RubyLLM.chat(model: &#39;gpt-4-turbo&#39;)<br>  .with_instructions(system_prompt)<br>  .with_tools(...)<br>response = chat.ask(prompt)  # Only the new message</pre><p><strong>What went wrong?</strong> We were still using HTTP calls between agents. The orchestrator called POST /agent on the order and inventory services. This meant three separate Sinatra processes, three separate LLM configurations, and a lot of network overhead for what’s essentially an in-process conversation.</p><h3>Iteration 7: Multi-Agent Framework with chatwoot/ai-agents</h3><p>The final iteration replaced the HTTP-based agent communication with <a href="https://github.com/chatwoot/ai-agents">chatwoot/ai-agents</a> — a Ruby SDK for multi-agent workflows with native handoffs.</p><p>The key insight: agents don’t need to be separate HTTP services. They’re just different LLM configurations with different tools and instructions. The framework handles handoffs natively — no HTTP calls, no serialization, no network overhead.</p><h3>The Final Architecture</h3><p>The order-service and inventory-service became pure business logic — no agent code, no LLM dependencies:</p><pre>agent-service/ (port 4000)<br>├── app.rb                              # Sinatra + POST /agent endpoint<br>├── agents_factory.rb                   # Creates 3 agents with handoff wiring<br>└── tools/<br>    ├── read_order_codebase_tool.rb     # Reads order-service/app.rb<br>    ├── read_inventory_codebase_tool.rb # Reads inventory-service/app.rb<br>    └── ask_user_tool.rb               # Asks user for missing info<br>shared-memory/<br>└── session.rb                          # JSON session persistence</pre><p>Three agents wired in a hub-and-spoke pattern:</p><pre>orchestrator.register_handoffs(order_agent, inventory_agent)<br>order_agent.register_handoffs(orchestrator)<br>inventory_agent.register_handoffs(orchestrator)</pre><p>Each agent has a focused role:</p><pre># Orchestrator — routes and synthesizes<br>Agents::Agent.new(<br>  name: &quot;Orchestrator&quot;,<br>  instructions: &quot;Route to the right service agent, coordinate cross-service flows...&quot;,<br>  tools: [AskUserTool.new]<br>)<br><br># Order Service Agent - reads and traces order code<br>Agents::Agent.new(<br>  name: &quot;Order Service Agent&quot;,<br>  instructions: &quot;Think step by step: READ the code, TRACE the logic, REASON about the result...&quot;,<br>  tools: [ReadOrderCodebaseTool.new, AskUserTool.new]<br>)<br># Inventory Service Agent - reads and traces inventory code<br>Agents::Agent.new(<br>  name: &quot;Inventory Service Agent&quot;,<br>  instructions: &quot;Think step by step: READ the code, TRACE the logic, REASON about the result...&quot;,<br>  tools: [ReadInventoryCodebaseTool.new, AskUserTool.new]<br>)</pre><p>The tools are simple:</p><pre>class ReadOrderCodebaseTool &lt; Agents::Tool<br>  description &quot;Reads the order service source code (app.rb)&quot;<br>  def perform(_tool_context)<br>    File.read(File.expand_path(&#39;../../order-service/app.rb&#39;, __dir__))<br>  end<br>end</pre><p>The runner is created once at boot and reused:</p><pre>RUNNER = Agents::Runner.with_agents(<br>  AGENTS[:orchestrator],<br>  AGENTS[:order],<br>  AGENTS[:inventory]<br>)</pre><p>Conversation continuity is handled by serializing the framework’s context to shared memory between turns:</p><pre># Load context from previous turn<br>context = JSON.parse(JSON.generate(session_data[&#39;context&#39;]), symbolize_names: true)<br><br># Run the agent<br>result = RUNNER.run(prompt, context: context)<br># Save context for next turn<br>updated_session[&#39;context&#39;] = result.context</pre><h3>A Real Conversation</h3><p>Here’s an actual multi-turn debugging session from our system. The user asks about a failing order, and the agents collaborate to trace the issue:</p><p><strong><em>User: </em></strong><em>“Debug POST /orders endpoint as it fails to place for params {product_id: 1, quantity: 100}. Unable to get specific reason why.”</em></p><p><em>The orchestrator hands off to the Order Service Agent, which reads the codebase and traces the logic:</em></p><p><strong><em>Orchestrator: </em></strong><em>“The POST /orders endpoint first checks product availability by calling the inventory service at </em><em>/products/1/availability?quantity=100. If the product isn’t available, it halts with a 422. Then it fetches product details, deducts stock, and creates the order. We need to check what the inventory service returns. Shall I simulate the inventory service calls?&quot;</em></p><p><strong><em>User: </em></strong><em>“Yes, you can simulate the inventory service call.”</em></p><p><em>The orchestrator hands off to the Inventory Service Agent, which reads the inventory codebase:</em></p><p><strong><em>Orchestrator: </em></strong><em>“I’ve reviewed the Inventory Service code. The availability check compares requested quantity with current stock. The stock deduction endpoint rejects negative stock. I need to know the current stock for product ID 1 to simulate accurately.”</em></p><p><strong><em>User: </em></strong><em>“Available quantity is 50.”</em></p><p><strong><em>Orchestrator: </em></strong><em>“The available quantity is 50 units. The order requests 100 units. The availability check at </em><em>/products/1/availability?quantity=100 would return </em><em>available: false since 100 &gt; 50. The Order Service would halt at line </em><em>halt 422, json(error: &#39;Wont able to place order&#39;). This is why the order fails — insufficient stock. Either reduce the quantity or increase the stock.&quot;</em></p><p><em>Four turns. Three agents collaborating. The system read both codebases, traced the cross-service flow, asked for the one piece of runtime data it couldn’t determine from code, and gave a clear diagnosis.</em></p><h3>What We Learned</h3><ul><li><strong>Start simple, iterate fast.</strong> Our first version was wrong in many ways, but each iteration taught us something specific. Trying to design the perfect architecture upfront would have taken longer and probably still been wrong.</li><li><strong>Don’t make agents do what code can do.</strong> Passing session_id as a tool parameter and hoping the LLM remembers? That’s asking the LLM to do what a constructor can do. Bake deterministic behavior into code; let the LLM handle reasoning.</li><li><strong>Replay is not resume.</strong> Re-executing every past message to rebuild context re-triggers all tool calls. Inject history as context instead — the LLM is good at reading and continuing.</li><li><strong>Agents don’t need to be services.</strong> Our biggest architectural mistake was making each agent a separate HTTP endpoint. Agents are just different LLM configurations. A framework like ai-agents handles handoffs in-process, which is simpler and faster.</li><li><strong>Think-first prompting matters.</strong> When we told agents to “dry-run the endpoint,” they dumped raw data. When we told them to “THINK, READ, TRACE, REASON, RESPOND,” they had actual conversations. The structure of the prompt determines the structure of the output.</li><li><strong>Shared memory over isolated sessions.</strong> When multiple agents need to see the same conversation, a shared JSON file beats per-service session stores. Simple, debuggable, and you can read the file to see exactly what happened.</li></ul><h3>Try It Yourself</h3><p>The full source code is available in the <a href="https://github.com/sivagollapalli/repo-agents">repository</a>. To run it:</p><pre># Start the business services<br>cd inventory-service &amp;&amp; bundle exec rackup &amp;<br>cd order-service &amp;&amp; bundle exec rackup &amp;<br><br># Start the agent service<br>cd agent-service &amp;&amp; bundle exec rackup &amp;<br># Send a debugging query<br>curl -X POST http://localhost:4000/agent \<br>  -H &#39;Content-Type: application/json&#39; \<br>  -d &#39;{&quot;prompt&quot;: &quot;Debug POST /orders with {\&quot;product_id\&quot;: 1, \&quot;quantity\&quot;: 5}, got 422 error&quot;}&#39;</pre><p>The response includes a session_id you can use for follow-up messages:</p><pre>curl -X POST http://localhost:4000/agent \<br>  -H &#39;Content-Type: application/json&#39; \<br>  -d &#39;{&quot;session_id&quot;: &quot;abc-123-...&quot;, &quot;prompt&quot;: &quot;The product stock is 3&quot;}&#39;</pre><p>The agents will read the code, trace the logic, and tell you exactly where things went wrong.</p><p><em>Enjoyed this article? Please share it with your network to help others learn.</em></p><p><em>Stay connected for more insights:</em></p><ul><li><em>Follow me on </em><a href="https://learnitnow.medium.com/"><em>Medium</em></a><em> for the latest articles.</em></li><li><em>Connect with me on </em><a href="https://www.linkedin.com/in/siva-gollapalli-b8026a61/"><em>LinkedIn</em></a><em> for professional updates.</em></li><li><em>Join the conversation on </em><a href="https://x.com/sivagollapalli"><em>Twitter</em></a><em> and share your thoughts.</em></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e836526a688b" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Great AI Debate: Building a Multi-Agent System in Plain Ruby (No Frameworks Required!)]]></title>
            <link>https://learnitnow.medium.com/the-great-ai-debate-building-a-multi-agent-system-in-plain-ruby-no-frameworks-required-bd4a486b9ea9?source=rss-9618834b293a------2</link>
            <guid isPermaLink="false">https://medium.com/p/bd4a486b9ea9</guid>
            <category><![CDATA[claude]]></category>
            <category><![CDATA[gemini]]></category>
            <category><![CDATA[agentic-ai]]></category>
            <category><![CDATA[ruby]]></category>
            <category><![CDATA[multi-agent-systems]]></category>
            <dc:creator><![CDATA[Siva Gollapalli]]></dc:creator>
            <pubDate>Tue, 03 Mar 2026 10:36:38 GMT</pubDate>
            <atom:updated>2026-03-03T10:36:38.702Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Gp1sZVbIzDqN5RwcQ4uHtg.png" /></figure><p>If you’ve been following the AI space lately, you’ve probably heard the buzzword “Agentic Frameworks.” Everywhere you look, there’s a new, complex library promising to help you orchestrate multiple AI agents. They come with massive dependency trees, steep learning curves, and a whole lot of magic under the hood.</p><p>But recently, I asked myself a question: <strong>Do we really need all that overhead just to get two AIs to talk to each other?</strong></p><p>To find out, I decided to run an experiment. I wanted to pit two of the smartest models on the planet — Gemini and Claude — against each other in a debate. The topic? A spicy one: <em>Is Ruby still relevant in the age of AI?</em></p><p>The catch? I wasn’t going to use any heavy AI frameworks. No LangChain, no CrewAI. Just plain old Ruby, some simple HTTP calls, and a text file.</p><p>Here is the story of how I built a bare-bones multi-agent system, and how you can too.</p><h3>The Arena: A Humble Text File</h3><p>When you strip away the fancy terminology, an AI “conversation” is really just a growing string of text. To make two agents talk to each other, they just need a shared memory space.</p><p>Instead of setting up a complex vector database or an in-memory state manager, I used the simplest database known to developers: a Markdown file. I called it RubyDebate.md.</p><p>To make sure our agents didn’t talk over each other while reading and writing to this file asynchronously, I wrapped the file operations in a standard Ruby Mutex.</p><pre>FILE_MUTEX = Mutex.new</pre><pre>def read_conversation<br>  FILE_MUTEX.synchronize do<br>    File.exist?(CONVERSATION_FILE) ? File.read(CONVERSATION_FILE) : &quot;&quot;<br>  end<br>end</pre><p><em>Boom. Shared memory solved.</em></p><h3>The Fighters: Gemini vs. Claude</h3><p>Next, I needed to set up the debaters. Using the lightweight ruby_llm gem (which essentially just wraps standard HTTP calls to the LLM provider), I configured my two agents via the Abacus.AI RouteLLM API.</p><ul><li><strong>In the Red Corner (The Defender):</strong> Gemini 2.5 Pro. I prompted Gemini to act as an expert Ruby engineer, fiercely defending Ruby’s relevance in the modern tech landscape.</li><li><strong>In the Blue Corner (The Skeptic):</strong> Claude 3.5 Sonnet. Claude’s job was to play the opposition, arguing that in an era dominated by Python-heavy AI tools, Ruby is losing its edge.</li></ul><h3>The Loop: Passing the Mic</h3><p>With the arena built and the fighters ready, all I needed was a loop. I set the debate to run for 5 rounds using Ruby’s Async block to keep things non-blocking and snappy.</p><p>The logic was beautifully simple:</p><ol><li><strong>Gemini’s Turn:</strong> Read RubyDebate.md to see what has been said. Generate a concise, 2-3 sentence argument defending Ruby. Append the response to the file.</li><li><strong>Claude’s Turn:</strong> Read the newly updated RubyDebate.md. Generate a 3-5 sentence counter-argument. Append the response to the file.</li></ol><pre># A simplified look at the loop<br>NUM_ROUNDS.times do |round|<br>  history = read_conversation<br>  <br>  # Gemini reads history and makes a point<br>  gemini_argument = question_agent.ask(&quot;Here is the debate: #{history}. Defend Ruby.&quot;)<br>  append_to_conversation(&quot;Gemini&quot;, gemini_argument)<br>  <br>  updated_history = read_conversation<br>  <br>  # Claude reads the new history and counters<br>  claude_rebuttal = answer_agent.ask(&quot;Here is the debate: #{updated_history}. Oppose Ruby.&quot;)<br>  append_to_conversation(&quot;Claude&quot;, claude_rebuttal)<br>end</pre><h3>The Result: Magic Without the Magic Tricks</h3><p>When I ran the script, my terminal lit up.</p><p><em>❓ Gemini is thinking… done.</em> <em>💡 Claude is thinking… done.</em></p><p>Round after round, the two models read the text file, analyzed the context, and fired back at each other. Gemini argued for Ruby’s unmatched developer happiness and the power of Rails. Claude countered with the sheer gravity of Python’s machine learning ecosystem.</p><p>By the end of the script, RubyDebate.md contained a rich, nuanced, and highly entertaining multi-agent dialogue.</p><h3>The Takeaway</h3><p>We often get caught up in the hype of new tools and frameworks, assuming that complex problems require complex solutions. But at their core, LLMs are just text-in, text-out engines.</p><p>By using a simple text file for state, a Mutex for thread safety, and basic HTTP calls to pass prompts back and forth, we created a fully functional multi-agent system in less than 100 lines of Ruby code.</p><p>So, is Ruby still relevant in the age of AI? If this experiment proved anything, it’s that Ruby’s elegance and simplicity make it an incredible tool for orchestrating AI — no bloated frameworks required.</p><h4>Final Source code:</h4><pre>require &#39;ruby_llm&#39;<br>require &#39;async&#39;<br>require &#39;fileutils&#39;<br><br># ---------------------------------------------------------------------------<br># Configuration<br># ---------------------------------------------------------------------------<br>RubyLLM.configure do |config|<br>  config.openai_api_key  = ENV.fetch(&#39;ABACUS_KEY&#39;, nil)<br>  config.openai_api_base = &quot;https://routellm.abacus.ai/v1&quot;<br>end<br><br>CONVERSATION_FILE = &quot;RubyDebate.md&quot;<br>NUM_ROUNDS        = 5  # how many Q&amp;A rounds to run<br><br># ---------------------------------------------------------------------------<br># Helpers – thread-safe file read/write with a Mutex<br># ---------------------------------------------------------------------------<br>FILE_MUTEX = Mutex.new<br><br>def read_conversation<br>  FILE_MUTEX.synchronize do<br>    File.exist?(CONVERSATION_FILE) ? File.read(CONVERSATION_FILE) : &quot;&quot;<br>  end<br>end<br><br>def append_to_conversation(role, text)<br>  FILE_MUTEX.synchronize do<br>    File.open(CONVERSATION_FILE, &quot;a&quot;) do |f|<br>      f.puts &quot;\n## #{role}\n\n#{text.strip}\n&quot;<br>    end<br>  end<br>end<br><br>def init_conversation<br>  File.write(CONVERSATION_FILE, &quot;# AI Conversation\n\nA multi-agent dialogue about Artificial Intelligence.\n&quot;)<br>end<br><br># ---------------------------------------------------------------------------<br># Agents<br># ---------------------------------------------------------------------------<br>def question_agent<br>  RubyLLM.chat(<br>    model: &#39;gemini-2.5-pro&#39;,<br>    provider: :openai,<br>    assume_model_exists: true<br>  )<br>end<br><br>def answer_agent<br>  RubyLLM.chat(<br>    model: &#39;claude-sonnet-4-20250514&#39;,<br>    provider: :openai,<br>    assume_model_exists: true<br>  )<br>end<br><br># ---------------------------------------------------------------------------<br># Main loop<br># ---------------------------------------------------------------------------<br>init_conversation<br>puts &quot;🤖 Starting multi-agent AI conversation (#{NUM_ROUNDS} rounds)...\n\n&quot;<br><br>NUM_ROUNDS.times do |round|<br>  puts &quot;--- Round #{round + 1} ---&quot;<br><br>  # 1) Question Agent reads the conversation so far, then generates a question<br>  Async do |task|<br>    history = read_conversation<br><br>    question_task = task.async do<br>      prompt = if round == 0<br>        &lt;&lt;~PROMPT<br>          You are in a debate. You are a expert Ruby engineer supporting about Ruby relavance <br>          despite knowing its limitations in the age of AI. Keep it concise (2-3 sentences max).<br>        PROMPT<br>      else<br>        &lt;&lt;~PROMPT<br>          You are in a debate. You are a expert Ruby engineer supporting about Ruby relavance <br>          despite knowing its limitations in the age of AI. Here is the debate so far:<br><br>          #{history}<br><br>          Based on the previous arguments, put follow-up argument that digs deeper<br>          into the topic. Keep it concise (2-3 sentences max). Do not repeat earlier arguments.<br>        PROMPT<br>      end<br><br>      print &quot;❓ Gemini is thinking... &quot;<br>      response = question_agent.ask(prompt)<br>      question = response.content<br>      puts &quot;done.&quot;<br>      puts &quot;   Q: #{question}\n\n&quot;<br><br>      append_to_conversation(&quot;Question (Gemini Pro)&quot;, question)<br>      question<br>    end<br><br>    question = question_task.wait<br><br>    # 2) Answer Agent reads the updated conversation, then answers<br>    answer_task = task.async do<br>      updated_history = read_conversation<br><br>      prompt = &lt;&lt;~PROMPT<br>        You are in a debate. You are opposing that Ruby is relavant in the age of AI. Here are the arguments so far:<br><br>        #{updated_history}<br><br>        Respond the latest argument thoroughly but concisely (3-5 sentences).<br>        Build on any prior context in the argument.<br>      PROMPT<br><br>      print &quot;💡 Claude is thinking... &quot;<br>      response = answer_agent.ask(prompt)<br>      answer = response.content<br>      puts &quot;done.&quot;<br>      puts &quot;   A: #{answer}\n\n&quot;<br><br>      append_to_conversation(&quot;Answer (Claude)&quot;, answer)<br>      answer<br>    end<br><br>    answer_task.wait<br>  end<br>end<br><br>puts &quot;✅ Conversation complete! See #{CONVERSATION_FILE} for the full transcript.&quot;</pre><p><em>Have you tried building AI workflows from scratch? Let me know your thoughts in the comments!</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=bd4a486b9ea9" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The “It Works in Console” Mystery: A Tale of Two APMs]]></title>
            <link>https://learnitnow.medium.com/the-it-works-in-console-mystery-a-tale-of-two-apms-a1e7422189bd?source=rss-9618834b293a------2</link>
            <guid isPermaLink="false">https://medium.com/p/a1e7422189bd</guid>
            <category><![CDATA[ruby-on-rails]]></category>
            <category><![CDATA[apm]]></category>
            <category><![CDATA[elastic-apm]]></category>
            <category><![CDATA[new-relic]]></category>
            <category><![CDATA[elasticsearch]]></category>
            <dc:creator><![CDATA[Siva Gollapalli]]></dc:creator>
            <pubDate>Sat, 14 Feb 2026 16:51:44 GMT</pubDate>
            <atom:updated>2026-02-16T05:00:10.218Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*NuLK5OPCdfVRf75X9FuEqw.png" /></figure><p>It started like any standard deployment. We pushed the code, the servers restarted, and we waited for the green lights. Everything looked fine — until we realized the search bar was dead.</p><p>Our setup is pretty standard: a Rails application connecting to Elasticsearch via Searchkick. It is usually rock solid. But suddenly, every search request coming from the web app was failing.</p><p><strong>The Head-Scratcher</strong> Naturally, we jumped into debugging mode. We opened the Rails console to inspect the data. We ran the exact same Searchkick query that was failing in the browser.</p><p><em>Result:</em> It worked perfectly.</p><p>This is the worst kind of bug. The code was right. The query was right. The database was healthy. So why was the web app choking while the console was fine?</p><p><strong>The Culprit: Over-Observation</strong> We stepped back and looked at what actually changed in this deployment. We had decided to improve our monitoring, so we enabled <strong>two</strong> Application Performance Monitors (APMs) simultaneously: <strong>Elastic APM</strong> and <strong>New Relic</strong>.</p><p>That was the smoking gun.</p><p>Here is what was happening behind the scenes:</p><ol><li><strong>The Web App:</strong> When running in the web environment, both APMs were active. They both tried to insert their own trace headers into every Elasticsearch API call to track the request. The collision or malformation of these double headers caused the requests to fail.</li><li><strong>The Rails Console:</strong> By default, APMs are disabled in the console environment. No APMs meant no extra headers, so the query went through cleanly.</li></ol><p><strong>The Fix</strong> The solution was simple: pick one lane. We couldn’t have two drivers trying to steer the wheel. We disabled the conflicting APM setup, and the search functionality came back to life instantly.</p><p><strong>The Lesson</strong> Sometimes, the tools you use to find problems can actually <em>be</em> the problem. If something works in the console but fails in the app, check your middleware — specifically the ones that like to “tag along” on your API requests.</p><p><em>Enjoyed this article? Please share it with your network to help others learn.</em></p><p><em>Stay connected for more insights:</em></p><ul><li><em>Follow me on </em><a href="https://learnitnow.medium.com/"><em>Medium</em></a><em> for the latest articles.</em></li><li><em>Connect with me on </em><a href="https://www.linkedin.com/in/siva-gollapalli-b8026a61/"><em>LinkedIn</em></a><em> for professional updates.</em></li><li><em>Join the conversation on </em><a href="https://x.com/sivagollapalli"><em>Twitter</em></a><em> and share your thoughts.</em></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a1e7422189bd" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Profiling Ruby apps using Vernier]]></title>
            <link>https://learnitnow.medium.com/profiling-ruby-apps-using-vernier-141c3cf842f6?source=rss-9618834b293a------2</link>
            <guid isPermaLink="false">https://medium.com/p/141c3cf842f6</guid>
            <category><![CDATA[vernier]]></category>
            <category><![CDATA[ruby]]></category>
            <category><![CDATA[performance]]></category>
            <category><![CDATA[ruby-on-rails]]></category>
            <category><![CDATA[profiling]]></category>
            <dc:creator><![CDATA[Siva Gollapalli]]></dc:creator>
            <pubDate>Mon, 08 Dec 2025 15:25:46 GMT</pubDate>
            <atom:updated>2025-12-08T15:25:46.760Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*WZQgJlBt17SVGDVDTXYuGw.png" /></figure><p>🔥 <strong>Sick of slow Ruby code and obscure bottlenecks?</strong> It’s time to stop guessing where your application’s performance is suffering. In my latest video, I dive deep into using <strong>Vernier</strong>, the powerful next-generation sampling profiler for Ruby 3.2+! Vernier gives you unprecedented visibility into everything from <strong>threads</strong> and <strong>Ractors</strong> to the <strong>Global VM Lock (GVL)</strong> and <strong>Garbage Collection (GC)</strong> activity. I’ll walk you through how to set up Vernier, capture detailed profiles, and analyze the resulting <strong>flame graphs</strong> to pinpoint exactly what’s slowing down your application. Stop writing code that <em>seems</em> fast and start writing code you <em>know</em> is fast. Click play and let’s make your Ruby apps fly!</p><p>This video explains why Vernier is considered a next-generation profiler and highlights the detailed insights it provides into Ruby application performance.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FM0HhmpR5gRs%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DM0HhmpR5gRs&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FM0HhmpR5gRs%2Fhqdefault.jpg&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/a15a683e1a4fbb3a967dcad92b650842/href">https://medium.com/media/a15a683e1a4fbb3a967dcad92b650842/href</a></iframe><p><em>Enjoyed this Video article? Please share it with your network to help others learn.</em></p><p><em>Stay connected for more insights:</em></p><ul><li><em>Follow me on </em><a href="https://learnitnow.medium.com/"><em>Medium</em></a><em> for the latest articles.</em></li><li><em>Connect with me on </em><a href="https://www.linkedin.com/in/siva-gollapalli-b8026a61/"><em>LinkedIn</em></a><em> for professional updates.</em></li><li><em>Join the conversation on </em><a href="https://x.com/sivagollapalli"><em>Twitter</em></a><em> and share your thoughts.</em></li><li><em>Subscribe to</em><a href="https://www.youtube.com/channel/UC6_mTjourDV69wIrXTX92tw"><em> Learnitnow</em></a><em> channel</em></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=141c3cf842f6" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Engineer’s Quest: How Agentic Search Redefines Information Retrieval]]></title>
            <link>https://medium.com/hackernoon/the-engineers-quest-how-agentic-search-redefines-information-retrieval-ab77955f3bc2?source=rss-9618834b293a------2</link>
            <guid isPermaLink="false">https://medium.com/p/ab77955f3bc2</guid>
            <category><![CDATA[agentic-ai]]></category>
            <category><![CDATA[pydantic-ai]]></category>
            <category><![CDATA[semantic-search]]></category>
            <category><![CDATA[neo4j]]></category>
            <category><![CDATA[agentic-search]]></category>
            <dc:creator><![CDATA[Siva Gollapalli]]></dc:creator>
            <pubDate>Wed, 03 Sep 2025 11:57:12 GMT</pubDate>
            <atom:updated>2025-09-03T11:57:12.841Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rgLdcxwd9ZFNvYdKRyAHlQ.png" /></figure><p>As a professional in the data industry, you know the drill. You’re a data analyst who just made the jump to a data engineering role. Your first task: get up to speed. Your first instinct: hit the search bar.</p><p>For years, we’ve relied on two primary tools to help us find our way:</p><ul><li><strong>Keyword Search:</strong> You type “data engineering content” and get links to pages with those exact words. It’s a passive transaction; you’re the digital librarian, manually sorting through stacks of results.</li><li><strong>Semantic Search:</strong> A smarter tool. You ask, “What is a good learning path to become a data engineer?” The search engine understands your intent and hands you a more relevant stack of books.</li></ul><p>But what if you had a tool that went beyond just giving you a list of books? What if it could read the books for you, find the specific, actionable steps, and even create a personalized learning plan based on your background? This is the promise of <strong>agentic search</strong>. It doesn’t just find information; it actively works to answer your complex, multi-layered queries.</p><h3>The Anatomy of an Agentic Search</h3><p>Agentic search redefines the process from a simple query-and-response model to a multi-step journey. It’s a process that mirrors how a skilled human mentor would guide you.</p><h4>1. Deconstructing the Intent</h4><p>Unlike semantic search, which finds results related to your intent, an agentic search uses your query as the starting point for a complex plan. It doesn’t just understand the words; it understands the goal.</p><p>Let’s use your specific query: <strong>“Suggest content to help me transition from a data analyst to a data engineer.”</strong></p><p>The agent immediately recognizes this isn’t a simple request for information — it’s a multi-step command. It interprets the intent: “You need a functional, personalized learning roadmap that leverages your existing skills as a data analyst.”</p><h4>2. The Multi-Query Breakdown</h4><p>Once it understands the overarching goal, the agent breaks the complex request down into a series of smaller, manageable sub-queries. This is its autonomous planning stage:</p><ul><li><strong>Sub-query 1:</strong> “transitioning from data analyst to data engineer”</li><li><strong>Sub-query 2:</strong> “data engineering skills and technologies content”</li><li><strong>Sub-query 3:</strong> Search using tag called “data-engineering”</li><li><strong>Sub-query 4:</strong> Get skills required for data engineering job and then search content using skill as show below</li></ul><p>The agent doesn’t just run one search; it runs a targeted, simultaneous, and often parallel series of searches.</p><h4>3. Filtering and Re-defining the Search</h4><p>As the agent receives results from each sub-query, it doesn’t just show them to you. It processes and filters them to stay on track. If a tutorial is outdated, the agent discards it. If a code example has a syntax error, the agent may generate a new query to find a corrected version.</p><p>This is where the magic happens: the agent uses the filtered results to redefine its own search path. It learns from each step. For example, if the first query identifies a specific skill gap, the next query might automatically become more specific: “Best AWS data engineering courses for a beginner.” It’s an iterative, self-correcting loop.</p><h4>4. Synthesizing the Final Result</h4><p>Once all the pieces are in place, the agent performs the final, crucial step: synthesis. It combines all the disparate search results — the skill gap analysis, the foundational tutorials, the tool-specific guides, and the practical exercises — into one final, cohesive answer.</p><p>The final output is not a list of links. It’s a complete, actionable learning plan presented clearly: “Based on your transition from a data analyst, here is a personalized learning roadmap to get you up to speed as a data engineer. I have curated a step-by-step guide with links to beginner-friendly tutorials on the key topics of ETL, data modeling, and core tools like Spark and Airflow.”</p><p>This is how agentic search transforms a vague question into a concrete solution. It handles the entire journey, from understanding the intent to sifting through the noise, to presenting a final, refined result that you can immediately use to start your new career journey.</p><p>Below is the quick demo of how agentic search will work:</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FzOVSQrGz11s%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DzOVSQrGz11s&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FzOVSQrGz11s%2Fhqdefault.jpg&amp;type=text%2Fhtml&amp;schema=youtube" width="640" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/87b9f6fa1ebd78bd43e9793eef8f1c4c/href">https://medium.com/media/87b9f6fa1ebd78bd43e9793eef8f1c4c/href</a></iframe><p>Note: Quality of results may vary depends upon quality of the data. In the demo I have used AI to generate data as well.</p><p>Hope it provides enough idea how to implement agentic search</p><p><em>Enjoyed this article? Please share it with your network to help others learn.</em></p><p><em>Stay connected for more insights:</em></p><ul><li><em>Follow me on </em><a href="https://learnitnow.medium.com/"><em>Medium</em></a><em> for the latest articles.</em></li><li><em>Connect with me on </em><a href="https://www.linkedin.com/in/siva-gollapalli-b8026a61/"><em>LinkedIn</em></a><em> for professional updates.</em></li><li><em>Join the conversation on </em><a href="https://x.com/sivagollapalli"><em>Twitter</em></a><em> and share your thoughts.</em></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ab77955f3bc2" width="1" height="1" alt=""><hr><p><a href="https://medium.com/hackernoon/the-engineers-quest-how-agentic-search-redefines-information-retrieval-ab77955f3bc2">The Engineer’s Quest: How Agentic Search Redefines Information Retrieval</a> was originally published in <a href="https://medium.com/hackernoon">HackerNoon.com</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>