<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Hari Rajesh on Medium]]></title>
        <description><![CDATA[Stories by Hari Rajesh on Medium]]></description>
        <link>https://medium.com/@haricane8133?source=rss-e8f8cb2ed41f------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*8zeqXRh9l3f52EHTsrUYsg.png</url>
            <title>Stories by Hari Rajesh on Medium</title>
            <link>https://medium.com/@haricane8133?source=rss-e8f8cb2ed41f------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sun, 22 Nov 2026 20:30:31 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@haricane8133/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Gray Timer — The story]]></title>
            <link>https://medium.com/@haricane8133/gray-timer-the-story-4ba7ca0a75e1?source=rss-e8f8cb2ed41f------2</link>
            <guid isPermaLink="false">https://medium.com/p/4ba7ca0a75e1</guid>
            <category><![CDATA[e-ink]]></category>
            <category><![CDATA[nfc]]></category>
            <category><![CDATA[e-paper]]></category>
            <category><![CDATA[watches]]></category>
            <category><![CDATA[seeed-studio]]></category>
            <dc:creator><![CDATA[Hari Rajesh]]></dc:creator>
            <pubDate>Mon, 16 Nov 2026 17:16:51 GMT</pubDate>
            <atom:updated>2026-11-16T17:52:19.324Z</atom:updated>
            <content:encoded><![CDATA[<p>You are now reading the <em>story</em> behind the World’s first NFC controlled ePaper wrist watch . Checkout the following links for other aspects.</p><ul><li>Video Demo — <a href="https://www.youtube.com/watch?v=rnAYJC3mgms">https://www.youtube.com/watch?v=rnAYJC3mgms</a></li><li>The Build guide—<a href="https://www.hackster.io/varahan/graytimer-nfc-eink-watch-06e3d7">https://www.hackster.io/varahan/graytimer-nfc-eink-watch-06e3d7</a></li><li>Repository — <a href="https://github.com/haricane8133/graytimer">https://github.com/haricane8133/graytimer</a></li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/650/1*ok5mlWVr4Xm9THw_dMNStg.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*QaGgdG96iLos9dFbbzT3bw.jpeg" /></figure><p>I have always wanted to build something combined with two technologies I adore a lot — e-ink and NFC.</p><p>It’s a watch built at home, wire by wire, rather than fabricating a PCB. It shows the time over a 1-bit picture, shuffles to a new watchface every ten minutes, and listens to my phone wirelessly through NFC for commands.</p><blockquote>eInk is a technology that moves physical ink inside a display. It is the same used in Amazon kindles and eReaders. It looks just like regular paper and doesn’t consume power while it is showing an image. It only consumes power when changing the image or moving the inks</blockquote><blockquote>NFC is a technology that involves very short range communication of 2 devices through the frequency 13.56 Mhz. The range is very short and simple, allowing for one of the devices to be completely passive (not actively powered by a source). The radio waves emitted by one active device reaches the passive device through its antenna and the electro magnetic induction powers the passive device and also completes the communication. In our case, the phone transmits data to our watch which poses as a passive NFC tag.</blockquote><h3>The Challenge</h3><p><a href="https://watchy.sqfmi.com/">Watchy</a> is an amazing open-source ePaper watch. I wanted one, but not just wanted one,… build one, not just get a PCB and slap it in a case. So I reached out in their community and asked them, after carefully inspecting their BOM, questions on whether (how) their firmware would run if I just soldered a few parts (I listed them down) in a particular way. They said something on the lines, “You do not know what you are talking about. We recommend just buying one”. That is when I knew I had to build a watch…, AND a firmware… of my own!</p><h3>Community</h3><p>Humor aside, Watchy is cool and is hands down the better product. Those folks were right too. I didn’t yet understand hardware, electronics, software and their marriage at a level that is required to engineer a complete end to end finished consumer product. But… I didn’t have to.</p><p>With the help of the amazing open source community and companies like Arduino, Seeed Studio, Waveshare, Goodisplay, Espressif Systems and Nordic Semiconductor that build a lot of easy to use practical tools with great developer facing documentation, we can build something great of our own, easily while staying at a slightly higher abstraction of understanding.</p><p>Thanks to Rajesh Kannan (VU2OW) for sharing his knowledge, experience, tools and time while being the inspiration behind all my projects. Speaking of projects, as with all projects these days, LLMs (Claude/Codex) helped us a lot with the software and debugging. So I made the repository AI Native. The repo has a perfect little setup, that guides any agent the whole project on its first message, so “make me a watchface from this picture” just works.</p><h3>The first watch</h3><p>The first Gray Timer came together in 2025: a Seeed Xiao nRF52840, a 1.54&quot; GoodDisplay e-paper panel (200×200 pixels, black and white), a clock chip and a small LiPo, in an brown 3D-printed case. No NFC; just shuffling watchfaces. It worked. It also lasted about three and a half days on a charge, and changing anything meant plugging in a cable.</p><h3>The day the screen died</h3><p>In September 2026 I tore the watch down to resolder it, as the USB connection had an issue. The battery was still connected, something shorted, and afterwards the screen showed nothing but noise. An afternoon with a multimeter, probing the display board’s high-voltage booster cap by cap, ended with a verdict: the panel had to go.</p><p>So I started again, and this time I wanted two things the first watch never had: real deep sleep, and a way to talk to it without a cable. But in the end it actually turned out that the display had a very small crack at a corner, and that had to go.</p><h3>Sleeping properly</h3><p>The nRF52840 has a mode called System OFF. It’s almost off: no clock running, no RAM kept, just a pin that can wake it. The catch is that something has to pull that pin once a minute. Our earlier used rtc ds3231 didn’t have it and so we had to keep it on.</p><p>The answer was a new clock chip, the RV-8803. It keeps time on a fraction of a microamp and pulses a pin at every minute. The watch now spends its life like this: the clock pulses, the chip wakes from scratch, draws the new minute, and goes straight back to sleep. Only one byte survives between minutes, tucked into the clock chip’s own RAM: which face is showing, and whether it’s pinned.</p><p>The rebuild had its share of surprises. The old panel was cracked, so I swapped in a Waveshare 1.54&quot; module with the same GoodDisplay panel already on it. One pad on the new Xiao leaked to ground, so the clock moved to a different pin.</p><h3>Talking to it with NFC</h3><p>For a long time I believed NFC on this chip was out of reach from Arduino. It isn’t. My own first attempt at the NFC code could see the phone but never got picked by it. Seeed’s NFC library worked, and soon the watch was a writable NFC tag.</p><p>The best part is that the phone’s field wakes the watch from deep sleep. So I open any NFC tag app (NFC Tools works on iPhone and Android), write a short text, and hold the phone to my wrist:</p><p>Reading the tag shows what the watch is up to: the face number, the mode and the time.</p><p>dfu turned out to be the most useful of all. Once the watch is sealed in its case there&#39;s no reset button to press, so a phone tap is now how it gets new firmware.</p><h3>Faces, faces, faces</h3><p>There are 116 watchfaces now: Star Wars, Harry Potter, Club Penguin, the machines of Horizon Zero Dawn, twenty treasures from Hades, planets, kolams, guitars. Some are drawn in code, some are line art, and some are photos dithered down to pure black and white. Dithered faces look far better on the panel than on a screen, and my favourite is still the Macaw. It is really simple for anyone to create a Gray Timer watchface. They have all the instructions and required python scripts in the repository. To make it simple, they can ask their agent to create one for them.</p><p>The update screen is my favourite piece of art on the watch. It’s the title screen of the 1989 Prince of Persia, reduced to the panel’s 200×200 pixels, with the game’s own pixel font for my name.</p><h3>Build Journey Pictures</h3><p>Gray Timer V2 came together over three weeks in October and November 2026. After the first watch’s display gave up in September, this one started again from a fresh set of parts. Here’s how it went, one photo at a time.</p><h4>24 October: the parts</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/900/1*fydQ3VgjBbDFhI9eixd1Pg.jpeg" /><figcaption>Everything on one table: the e-paper panel, the old case and strap, two LiPo batteries, a coin cell, the RV-8803 clock board, a DESPI-C02 display adapter, and the tiny Seeed Xiao nRF52840 that runs it all.</figcaption></figure><h4><strong>24 October: soldering</strong></h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*m5GshMooqfQSCGFiYU5WgA.jpeg" /><figcaption>Helping hands hold the boards while the wires go on</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*exlICq3TFXMnM0dl4tHOpA.jpeg" /><figcaption>The Xiao, wired onto the display adapter</figcaption></figure><p>The old panel had a cracked corner and never drew a thing, so a new display went on order.</p><h4>30 October: a new display, and an antenna</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*bkS-NNzXOypdEg4f6KNSIA.jpeg" /><figcaption>A Waveshare 1.54&quot; module: the same panel, already on its own board</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*VdFH52NpaoRWNZTsBgZDlw.jpeg" /><figcaption>Seeed’s NFC antenna joins in, so a phone can talk to the watch</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1KAGIxhHBnfzq5edEwVGoA.jpeg" /><figcaption>Everything wired together: brain, clock, display and antenna</figcaption></figure><h4>30 October: it’s alive</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/900/1*5GNBtYvECfk7FcRV6uZo8w.jpeg" /><figcaption>One of the very first screens: ready for new firmware</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/900/1*ZaNgX6NSL5SnyFDX31zFJA.jpeg" /><figcaption>On its own battery, with its first watchface</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*TS78pWHKP70Vf45WW2fB5A.jpeg" /><figcaption>Half an hour later, a different face</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*QFTx0SOQCl3PfXVwTZzS9A.jpeg" /><figcaption>The next morning, still keeping time</figcaption></figure><h4>7 to 14 November: the case</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*csSxgQj2xPxduEXoYtANeQ.jpeg" /><figcaption>A first print of the case</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/900/1*vwQ00VhdvzDOizblkx1MFw.jpeg" /><figcaption>The 3D printer at work</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*CHfT6718RaMRxZjEhDJZFg.jpeg" /><figcaption>Printing the black parts</figcaption></figure><h4>14 November: a new update screen and a new Logo!</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/900/1*5i2h4MWigE6X-HBzBfWREw.jpeg" /><figcaption>The update screen gets a makeover, in the style of the old original DOS Prince of Persia</figcaption></figure><h4>15 November: all together</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/900/1*iYbZgRiSaFFc6j32MZN3ig.jpeg" /><figcaption>Watch case with the straps sewed in</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*r8e13-iwGYrXKnqi7blRtQ.jpeg" /><figcaption>Battery, boards and antenna, jam-packed into the case</figcaption></figure><h3>Hindsight</h3><p>These were some of the issues we faced, the debugging processes and what worked in the end</p><h4>1. A screen that only showed noise</h4><p><strong>What happened.</strong> I tore the first watch down to resolder it with the battery still connected. Something shorted, and from then on the screen showed random noise and never changed.</p><p><strong>How I found it.</strong> I worked inward from the processor. The Xiao booted, USB worked and the clock chip answered, so the brain was fine. A small sketch that toggled each display pin showed all five signal lines reaching the display board at 3.3 V, so the wiring was fine. Then a test sketch timed a full refresh: 2.18 seconds, exactly what the datasheet promises, with no busy timeout. The panel’s controller was taking commands and running its refresh. It just wasn’t moving any ink.</p><p>E-paper needs voltages far above 3.3 V to push its particles around, and a small boost circuit on the driver board makes them. Probing it during a refresh, no rail rose above 2.6 V. The inductor and sense resistor measured fine. So I tried a brand-new GoodDisplay DESPI-C02 adapter with a fresh boost circuit: same 2.18 seconds, same frozen noise. The deciding test was a bigger 5.76&quot; panel on the same adapter and the same wires. It drew images perfectly. The 1.54&quot; panel itself was dead, and a closer look found a cracked corner.</p><p><strong>The fix.</strong> A new panel: a Waveshare 1.54&quot; module carrying the same GoodDisplay glass (GDEY0154D67), driven by the same GxEPD2 class.</p><p><strong>The lesson.</strong> Swap one part at a time against one you know is good. And a single number, like how long a refresh takes, can tell you which half of a system is still alive.</p><h4>2. A ribbon cable that fits both ways</h4><p>The DESPI-C02’s connector has contacts on both sides, so a ribbon goes in either way up and still connects, just with every pin reversed. That puts 3.3 V onto signal pins, which can damage a good panel. To make it worse, the 1.54&quot; ribbon goes in the opposite way to the 5.76&quot; one.</p><p>The tell was the same timing test: a correct ribbon gives a 2.18-second refresh, a flipped one gives zero. Now I match the pin-1 marks on the ribbon and the board before I power anything.</p><h4>3. A pad that leaked to ground</h4><p>On the new Xiao, the clock chip’s I²C clock line sat at about 1 V instead of 3.3 V. A meter found no short, but the edge of the board between D4 and D5 was damaged, and D5 was quietly leaking. I moved SCL to D6 with Wire.setPins() and the clock answered straight away. (The first Xiao had its own version of this: one pin ended up buried under epoxy, so the display&#39;s BUSY line moved to D9.)</p><p>If a line idles at a strange voltage, suspect the pad before the code.</p><h4>4. NFC that saw the phone but never got picked</h4><p>I wanted the phone to wake the watch and write to it, so my first attempt programmed the nRF52840’s NFC hardware directly, as a Type 4 tag. It detected the phone’s field and activated, but the phone never selected it. I spent a long time in the chip’s registers and errata. Coupling was weak too: only 5 of 59 field detections ever got as far as the phone picking the tag.</p><p><strong>The fix.</strong> Seeed’s XIAO_nRF52840_NFC library, which emulates a simpler Type 2 tag. The phone showed a tick on the first try. Where the antenna sits matters as much as the code: behind the e-paper panel it struggles, so its place in the case was planned around that.</p><p><strong>The lesson.</strong> Try the vendor’s library before writing your own driver. And a warning if you try this: the board package ships an nfc_to_gpio example that turns the NFC pads into ordinary pins, permanently. Don&#39;t run it.</p><h4>5. Arduino IDE uploads failed with a Python error</h4><p>On macOS, every upload from the Arduino IDE failed with a complaint about Python being configured for ASCII. The uploader, adafruit-nrfutil, needs a UTF-8 locale, apps launched from the Dock don&#39;t get one, and the IDE forces LC_ALL=C on top.</p><p><strong>The fix.</strong> Run launchctl setenv LANG en_US.UTF-8 in Terminal and restart the IDE (it lasts until a reboot), or upload with arduino-cli from a terminal that has LC_ALL=en_US.UTF-8.</p><h4>6. A pinned face that seemed to change by itself</h4><p>I pinned a face from my phone, and at ten past the hour it looked like the watch had shuffled anyway. It hadn’t. Every ten minutes the watch does a full refresh, flashing black and white to clear the ghosting that partial updates leave behind, and that flash looks a lot like a new face arriving.</p><p><strong>The fix.</strong> A pinned face now does its full refresh only at :00 and :30, which is plenty to keep it clean.</p><h4>7. The screen stalled halfway, but NFC still worked</h4><p><strong>What happened.</strong> After swapping in a new battery, the screen showed a mix of two images and then stopped refreshing. Yet the phone could still read and write the NFC tag.</p><p><strong>How I found it.</strong> NFC needs very little current, and a full refresh needs a lot, so this pointed at the supply. The first culprit was a dry solder joint on the battery’s ground lead: good enough for NFC, not for a refresh. I resoldered it, and the watch was fine for a night. Then the stalls came back.</p><p>This time the battery read a healthy 4.1 V at the Xiao, but the display adapter’s 3.3 V input sometimes sagged to 3.0 or 2.9 V. The clock board, fed from the same 3.3 V rail, held steady, so the Xiao’s regulator was fine and the loss was in the wire to the adapter. Tugging the adapter’s power wire dropped it to 2 V, and once it read 1 mV.</p><p>That last reading explained everything. The “3.0 V” I had been measuring wasn’t coming through the power wire at all. With the wire barely connected, the panel was being back-powered through the protection diodes on its signal pins: enough to look alive on a meter, nowhere near enough for a refresh.</p><p><strong>The fix.</strong> Resolder the power wire at the adapter end.</p><p><strong>The lesson.</strong> When the screen fails but NFC works, check the supply joints before you blame the panel. A plausible voltage doesn’t prove a wire is connected.</p><h4>8. The dfu command that slipped back to the watchface</h4><p>Once the watch is sealed in its case, there’s no reset button, so I added a phone command, dfu, that drops the watch into its bootloader for a firmware update over USB. It kept falling back to the watchface instead.</p><p><strong>How I found it.</strong> I wrote a tiny logger on the Mac that polled USB for Seeed’s vendor ID and printed which device appeared and when: the running sketch or the bootloader. It showed two separate problems.</p><ol><li>My USB-C cable carried power but no data. Nothing ever showed up on USB at all.</li><li>With a good cable, the bootloader appeared and then vanished. When software asks for the bootloader, it gives the computer about three seconds to connect, then gives up and restarts the watch. Jumping straight from a sketch that was already on USB, my Mac (through a monitor’s USB hub) was too slow to notice the change, and two tries in three missed the window.</li></ol><p><strong>The fix.</strong> After dfu, the watch waits until USB is actually connected (showing &quot;Plug in USB within 2 min&quot;), disconnects cleanly, stays off the bus for a full second, and only then jumps to the bootloader. Three tries out of three.</p><p><strong>The lesson.</strong> Test the cable first. And when a device keeps vanishing, log what the computer sees and when, instead of reasoning about it.</p><h4>9. Uploads that died halfway</h4><p><strong>What happened.</strong> After I added the 116th watchface, every upload failed right after “Sending DFU init packet”, with the watch dropping off USB. The error said the bootloader version didn’t match or the watch wasn’t in DFU mode.</p><p><strong>How I found it.</strong> Fresh from the timing fix above, I spent a long session chasing USB timing. Then I looked at the build size: 799,176 bytes. A build of 789,904 bytes had uploaded fine that same morning. The IDE reports a limit of 811,008 bytes, but this bootloader refuses anything much over about 790 KB. The error message had nothing to do with the real problem.</p><p><strong>The fix.</strong> Many faces are drawn by a Python script, and each was stored as a full 200×200 bitmap even when most of it was white. Storing each one cropped to its ink, plus an offset, saved 93 KB with pixel-identical results. The 706 KB build uploaded on the first try.</p><p><strong>The lesson.</strong> If an upload dies right after the init packet, check the size before anything else.</p><h3>Still open: where does the battery go?</h3><p>V2 runs about a week on a 400 mAh battery. That works out to roughly 2.4 mA on average, when deep sleep should bring it under 0.1 mA. So far I’ve ruled out the clock board’s power LED (its jumper is cut), leaky pull-ups (every line sits at 3.3 V in sleep) and the firmware’s sleep path. Next comes a meter in series with the battery, to see whether it’s a constant leak or something waking the watch. That’s the next chapter.</p><h3>Contact</h3><p><em>I am available through the username @haricane8133 in X, Reddit, LinkedIn, Medium and YouTube.</em> Happy hacking!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4ba7ca0a75e1" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Chapter 2— Not a fair match]]></title>
            <link>https://medium.com/@haricane8133/chapter-2-not-a-fair-match-33e7656ad6ed?source=rss-e8f8cb2ed41f------2</link>
            <guid isPermaLink="false">https://medium.com/p/33e7656ad6ed</guid>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[software-development]]></category>
            <dc:creator><![CDATA[Hari Rajesh]]></dc:creator>
            <pubDate>Sun, 21 Jun 2026 10:29:21 GMT</pubDate>
            <atom:updated>2026-06-23T15:54:44.136Z</atom:updated>
            <content:encoded><![CDATA[<blockquote>A series on using AI wisely and sustainably in software development.</blockquote><ul><li><a href="https://medium.com/p/2a640c56b0c9">Chapter 1 — The Barista, Intuition and Learning</a></li><li><strong>Chapter 2 — Human Brain and LLM</strong></li><li><a href="https://medium.com/p/cf400328ae31">Chapter 3— AI, Speed and the Sustainability of Software</a></li><li><a href="https://medium.com/p/140209a5f7cb">Chapter 4— What counts as Innovation in the Age of AI?</a></li><li><a href="https://medium.com/p/d90341113c67">Chapter 5— The AI Maturity Ladder</a></li><li><a href="https://medium.com/p/5420267d8331">Chapter 6— The AI Productivity Trap</a></li></ul><h4>Preface</h4><p>Read comment.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/788/0*ygP-LQL5Fx0uxFyT.jpg" /></figure><p>There have been numerous scientific studies that compare the brain and LLM. As with the rest of the series, I will just present my theoretical views based just on <em>intuition</em>. <strong>Richard Feynman</strong> said <em>“It doesn’t matter how beautiful your theory is, it doesn’t matter how smart you are. If it doesn’t agree with experiment, it’s wrong.”. </em>I look forward to experiments’ results.</p><h4>Time and Pain</h4><p>LLMs and machines do not feel pain and other emotions, and certainly not the effect of time on these emotional outcomes. No matter the size of the tasks, they do not “suffer”. A barista understands how much a bad coffee would affect the day of a customer. A developer understands the impact of badly kept codebase on developers, and how that ultimately affects the users. This pain triggers a good developer to forcefully work on some necessary evil, rather than only working on problem statements given or assigned. Not all of these can be modelled as textual instructions that will trigger a machine to do trigger activities at a certain threshold. The LLM doesn’t know what time is appropriate for an activity or event.</p><h4>The Bayesian Trap</h4><p>Why cannot we have coffee machines that can handle “any scenario”. LLMs operate based on probability of next best token and is trained based on a content of what the world has seen so far. So prompting something like “imagine something that has never happened before”, won’t cut it. Watch this video by popular informational YouTube channel Veritasium — <a href="https://www.youtube.com/watch?v=R13BD8qKeTg">The Bayesian Trap</a>. I do not know for sure, but I certainly feel that the human mind can overcome this limitation.</p><h4>The “Aha” moments</h4><p><strong>Discoveries and learnings</strong> form a regular part of the human experience working on something, and these profoundly shape how they work in the future. A good developer or a Barista would definitely have had a lot of these moments etched in their minds deeply, contributing to all their future work.</p><p><strong>LLMs have aha moments too</strong>; we can see Claude exclaiming excitedly multiple times. Unline humans who <em>experience</em> these moments, for an LLM, these discoveries will only ever be a part of the context window; let’s call these as <strong>bullet points</strong> in the context window.</p><p><strong>When a developer has an aha moment</strong> in a particular project they are currently working on (that is, a discovery or a learning that absolutely changes something fundamental), they immediately remind themselves about all the other projects that the learnings can be applied on. Then thinking a little bit (flavored by time, pain), they will proactively plan with project managers and business partners to prioritize the effort and apply the learnings. If they didn’t do it, they would <em>feel guilty</em>.</p><h4>Intuition, Gut feeling</h4><p>Every human has a <strong>sub-consious mind</strong> and therefore <strong>intuition</strong> and <strong>gut feeling</strong>. You can call these two as defense or shortcut mechanisms that the human brain has in place. People always say, “Trust your gut”, or “Go with your instinct”.</p><p>But are Intuitions and Gut Feelings always right?</p><p><strong>The more experience a human has in life</strong>, or in our case software development, the better chances intuition and gut feeling are right. Experienced software developers, or in general software developers who have worked in a domain and related tech for a very long time, would have built their arsenal of past experiences to such an extent that often when posed with an idea or a task, they would be able to right away tell you what to do (mostly what not to do). But when you ask them “why” their option was a “yes” or a “no”, they won’t be able to tell you exactly why, in <strong>bullet points</strong>, at least just yet. The decisioning was sub-consious.</p><h4>The Lazy developer</h4><p><strong>Bill Gates</strong> is known to have once said “<em>I choose a lazy person to do a hard job. Because a lazy person will find an easy way to do it.</em>”. The reason is that, laziness can be a super power at the right moment. A lazy developer will try to automate difficult procedures and simplify complicated workflows so that they do not have to “suffer” the next time. This well timed laziness of pragmatic developers, which is fueled by all that we saw above, is something that is mostly missed with LLMs.</p><h4>Bullet Points</h4><p>Humans internally will have a dedicated reasoning, research and plan loop too (if you want to call the process that). Trust me, there will be thousands or even millions of these invisible <strong>bullet points</strong> that contribute to each decision. After they sit down and think for while, they may be able to give you most of these <strong>bullet points</strong> (I am sure some <strong>bullet points</strong> are inaccessible to the consious mind).</p><p>At the first glance atleast, <strong>An LLM</strong> doesn’t seem to have this <strong>short circuit</strong>. It needs a deep planning and a reasoning loop before any conclusion.</p><p><strong>All important discoveries and learnings</strong> today’s coding agent has, <strong>become bullet points</strong> or a plain text in the context window. Of course one can ask the agent to write them down in a separate file or a memory system to retrieve it later into the context window.</p><h4>Needle in a haystack</h4><p>The more <strong>bullet points</strong> behind a decision, the better the decision is, right?</p><ol><li>How many bullet points do you think an LLM can handle compared to a human?</li><li>With time, we will have a lot of bullet points. How do you instruct a system to prioritize one bullet point over another bullet point? (then expand this for the complexity of a million of these with different relationships between them)<br> a) Do you add another bullet point explicitly asking the agent to prioritize the first one?<br>b) Do you remove the first bullet point completely?<br>c) Do you add decorators over the existing bullet points, like “This point is more important than the other”?</li></ol><p><strong>No amount of engineering</strong> <em>on top of an</em> LLM can fully solve an agent to use <em>the right set of bullet points</em> over another when it takes a decision. This is called as the needle in the haystack problem and it forms the basis of context engineering — Somehow make the LLM deterministically (always) prioritize certain parts of the context window. But as the number of bullet points increase, and it will keep exponentially increasing based on each phase of each project of each organization of each company, the problem keeps getting bigger and bigger to solve.</p><p><strong>The intelligence of the LLM</strong> is the only way we can solve this problem, but what is that?</p><h4>If context window is consious, then pre-training is sub-consious?</h4><p>Consious mind is the one that we can control; sub-consious mind isn’t, atleast to the vast majority of us. So if you absolutely must do a comparison, this a good one. We can control the context window of an agent, and so we can call it the consious mind. We cannot control the model’s pretraining and so it can be called sub-consious.</p><p>Each LLM has been pretrained with bullet points and weights (priorities). The better the LLM’s sub-consious is, the more intelligence it has (obviously). The more intelligence, the more bullet points are handled internally with proper priorities / weights. In other words, the quantity and quality of bullet points and inter relationships between them used in training the model, determines the intelligence of the model.</p><ul><li>We cannot fit <em>all</em> bullet points that we want in the context window; same with humans as there are only a bunch of stuff we can hold in our consious minds at a time.</li><li>Thus we need most of the important learnings, discoveries and bullet points properly ingested in the sub-consious mind.</li><li>Each human mind has <em>an algorithm</em> (lets call it the mind algorithm) that ingests stuff into the sub-consious.</li><li>But we cannot do the same to LLMs. An LLM’s sub-consious is fixed the moment Anthropic finishes the training.</li></ul><h4>The mind algorithm and diversity</h4><p>In the last section, we saw that each human brain has an algorithm that decides how each input to the consious mind is in realtime ingested into the sub-consious mind. This ingestion process is critical (and so is the algorithm used), as this determines the weightage of every bullet point that goes in and therefore the intelligence and therefore the output capacity.</p><p>This important algorithm depends on a lot of factors that are beyond the scope of this series, but we will touch upon a few points. This algorithm can be tweaked by a person, just by changing their attitude, listening or being around different people (intelligent or dumb ones), meditation, etc.. There are some factors that can’t be changed easily, like the language we speak or the location and culture we live through.</p><p>To put short, this is why diversity is important. You get people with a wide range of algorithms, and therefore the output of a team is beautiful. Later on, we will have a look at companies trying to over-engineer a harness layer to achieve ticket to pull request SDLC automation. These activities mean that we lose the benefits of hiring a diverse set of individuals to the team.</p><p>Rather than trying to make a non-deterministic system run deterministically, augment your deterministic process with these non-deterministic pieces</p><h4>Replacing developers</h4><p><strong>To replace developers</strong>, just prompt engineering, context engineering, harness engineering or loop engineering over an agent based on yesterday’s best LLM model, trained on generic bullet points from a couple of months back, is simply not enough.</p><p><strong>What we need is</strong>, unrestricted access for everyone to a system where AI models can be continuously trained and re-trained based on all the bullet points that are continuously uncovered when each developer works on each phase of each project in each organization in each company. That is, we need a system that allows us to continuously modify the sub-consious based on learnings.</p><blockquote><strong>This is the true intelligence that will lead to AGI</strong></blockquote><p><strong>We thought</strong> that given enough small (and possibly fine tuned) open source models as a part of an overall workflow, we could solve the problem. But we still see that developers are always looking for the latest from Anthropic and OpenAI, constantly praising today’s model and dissing yesterday’s model.</p><p><strong>Is this possible?</strong> Atleast currently, no, as this requires extensive hardware, time, effort and therefore crazy money. 99% of all problem statements that we are working on do not deserve it. Note that this is not the same as fine-tuning an LLM. We need new from scratch as we need to reprogram the weights of bullet points that we have seen in the past, with new bullet points that we are uncovering now. This is just not feasible.</p><p>Human brain is so well made that all of this <strong>true intelligence is available for free</strong>. We just have to take care of the algorithm.</p><h4>AGI</h4><p>Therefore, model intelligence is the one area that would absolutely determine the future of humanity. If <a href="https://www.anthropic.com/news/fable-mythos-access">Anthropic’s model Fable / Mythos</a>, which is nowhere closed to our proposed AGI system, was still blocked by US government as because it crossed an intelligence threshold, what do you think will happen when someone (from Anthropic or Google DeepMind) really builds our proposed AGI system?</p><ol><li>It will certainly not be allowed to be used by you and me.</li><li>It need not be used by you and me.</li><li>With all that intelligence, why would it or someone write code with it?</li><li>If one company can afford and have access to such a system, they would benefit more from simply using it than SaaS-ifying it and extending access to it for others.</li></ol><p>My hope is that just like how DNA experimentation on humans are banned, the whole world would come together (at least for selfish geo political reasons) to prevent us from getting there.</p><p>So, bottomline, developers cannot be replaced. If they are replaced now, or their learnings not prioritized, companies will pay dearly in a few years.</p><h4>The alternate</h4><p>This is exactly what we would be addressing, in the rest of the chapters, but here is a gist.</p><p>Developers need to pitch in and fill in these blanks (<em>only the blanks</em>) with “engineering over LLMs”. The ability of the developer to effectively do this, determines the abstraction level that each developer must set for themselves, and abide. The only way they can climb up the abstraction level (and survive more and more automation), is to themselves becoming an expert in Software Development — Domain, Tech, LLM, etc…</p><blockquote>The human brain works. Use it</blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=33e7656ad6ed" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Chapter 6— Don’t ask your barber if you need a haircut]]></title>
            <link>https://medium.com/@haricane8133/chapter-5-dont-ask-your-barber-if-you-need-a-haircut-5420267d8331?source=rss-e8f8cb2ed41f------2</link>
            <guid isPermaLink="false">https://medium.com/p/5420267d8331</guid>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[software-engineering]]></category>
            <dc:creator><![CDATA[Hari Rajesh]]></dc:creator>
            <pubDate>Tue, 16 Jun 2026 03:56:42 GMT</pubDate>
            <atom:updated>2026-06-27T15:05:39.879Z</atom:updated>
            <content:encoded><![CDATA[<blockquote>A series on using AI wisely and sustainably in software development.</blockquote><ul><li><a href="https://medium.com/p/2a640c56b0c9">Chapter 1 — The Barista, Intuition and Learning</a></li><li><a href="https://medium.com/p/33e7656ad6ed">Chapter 2 — Human Brain and LLM</a></li><li><a href="https://medium.com/p/2a640c56b0c9">Chapter 3— AI, Speed and the Sustainability of Software</a></li><li><a href="https://medium.com/p/140209a5f7cb">Chapter 4— What counts as Innovation in the Age of AI?</a></li><li><a href="https://medium.com/p/d90341113c67">Chapter 5— The AI Maturity Ladder</a></li><li><strong>Chapter 6— The AI Productivity Trap</strong></li></ul><h4>Preface</h4><p>Read comment.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/0*QxYjo6hRDtilasDO" /></figure><h3>Ticket to PR</h3><h4>Learning</h4><p>When most harness builders are asked why they are trying to automate end to end SDLC, they would say, that the idea is <em>just</em> to automate all the boring aspects of software development, so that developers can focus on <em>more important things</em>. When zoomed in, most of the boring parts, is to write code.</p><p>Now on the other hand, think about every coding, programming or software engineering tutorial that has ever existing in the world — they all say “Getting your hands dirty is the only way you can learn. Try out these exercises in your IDE”. There are too many reason for this, and throughout the series we have already thoroughly established a few too.</p><p>We also have seen that modelling complex domains into an agent is not an easy feat. If you have infinite money, you can do it by creating a frontier LLM coding model, with all the information you need properly modelled in, and keep it active with new information uncovered in every step of the journey. But, if you have infinite money, why are you writing software?</p><h4>Accountability</h4><blockquote>“If in physics there’s something you don’t understand you can always hide behind the uncharted depths of nature. You can always blame God. You didn’t make it so complex yourself.</blockquote><blockquote>But if your program doesn’t work, there is no one to hide behind. You cannot hide behind an obstinate nature. A zero is a zero, a one is a one.</blockquote><blockquote>If it doesn’t work, you’ve messed up.”</blockquote><blockquote>— <strong>Professor Edsger W. Dijkstra</strong></blockquote><p>Accountability is key when someone is automating more and more of SDLC. As SDLC automation keeps increasing, the role of the developer keeps rising abstraction levels, and AI does more and more of the work. We will see in the next section that developers are asked to allow the agent to add the “Co Author” section in the Git commit description. This arrangement makes sense (atleast a tiny bit) when each developer configures the agent according to their workflow, lets it run, and is able to keep up with the work of the agent enough to call the work their own (after all, we do not say a Bulldozer built the building).</p><p>But if too much automation is provided at the company level — that is, if each developer is handed a very highly abstracted tool to which all they have to do is to pass in a well defined Jira ticket, who is accountable for the code that is thus shipped?</p><p><strong>A)</strong> If a company provides you with an E2E SDLC automation tool, whose output they <em>fully trust</em>,</p><ol><li>The commit author name should be the tools’</li><li>You should not be made accountable to the changes.</li></ol><p>We already saw that this is not possible technically, atleast yet, based on all we saw in the chapters before. So no sane company is going to allow you to use their tool without your manual step. But if this does happen (AGI IMO), then there is no need for a developer or at a stretch even most software.</p><p><strong>B)</strong> If a company provides you with an E2E SDLC automation tool, whose output they do NOT trust fully, and ask you to manually work on top,</p><ol><li><strong>Politely refuse</strong></li><li>Set your AI abstraction (SDLC automation) to that level which you can handle, while holding 100% accountability to the code you generate.</li><li>Explain the timeline requirements based on the set abstraction level.</li></ol><h4>Takeaways</h4><ol><li><strong>If a company</strong> gives you an E2E automation tool, ask them “Do you trust the output. Can I push to production without checking anything?”.<br>- If the answer is ‘yes’, and you DO NOT care about learning or growth, be a good employee and use it.</li><li><strong>Know yourself really really well</strong>. Know how well you can use AI. Know where you need to use AI for your workflow. Stay up to date about every model and every new feature that is announced. Everything hangs in your ability to confidently declare how much it would take for you to work on something while using AI to its fullest ability while holding full accountability to the code you ship. <strong>This is the success at your level</strong>.</li></ol><h3>Measuring Success</h3><p>If you read the entire article till now, thanks. You would understand that we have already established that measuring success using some metrics is not straight forward (how do you measure someone’s gut feeling power?). The following is based on what I hear as news from many companies. Come on, let us beat a dead horse.</p><p>Since this is an inflection point in the industry, companies have been doing things that seem a little contradictory with each other.</p><ul><li>Get everyone unlimited access to AI and encourage AI usage for everything.</li><li>Build a public dashboard showing tokens and money consumed.</li><li>Try and get AI adoption to 100% — that is get every single developer to use AI.</li><li>After understanding that this can be easily gamed, start looking at the lines of code and percentage of commits written by AI.</li><li>After understanding that this can be easily gamed, start looking at the commit code diff to categorize meaningful commits and trivial commits.</li><li><em>Amidst asking developers to author more lines of code and commits using AI, and then using AI to categorize each and every commit every developer has made (which is crazy token hungry and costly),<br></em>in parallel, start capping token usage for each developer with strict limits<em>.</em></li><li>Then build the org harness, an extremely token hungry attempt to manage E2E SDLC.</li><li>Use all the valuable data points collected, to take further decisions.</li></ul><p>Throughout the article, we have seen that just using AI for AI’s or speed’s sake does more harm to the software project, the developer, the team and the company. We need to exercise extreme caution in how we use these sharp knives. Just chasing AI Adoption %, will force developers to just this wrong pattern — diverting concentration towards checking the several dashboards to ensure that their “AI usage is counted”.</p><p><a href="https://en.wikipedia.org/wiki/Goodhart%27s_law"><strong>Goodhart’s Law</strong></a> states: “When a measure becomes a target, it ceases to be a good measure.” Coined by British economist Charles Goodhart in 1975, it warns that once a metric is used as a goal for incentives, people will manipulate the system to hit that specific number, destroying the metric’s original value.</p><p>Developers usually commit directly through cmd or an IDE shortcut (in which case there won’t be any co-author in commit description). <strong>Using agent to commit</strong> (by passing “commit this” or configuring in instructions), will spend tokens and token expenditure causes environment impact as well. Here is AI providing some numbers for <strong>1000</strong> developers doing just <strong>1</strong> commit a day for a year using <strong>Opus</strong> (<em>do not quote me</em>. This is not from a study or a scientific experiment). Now these are numbers worth looking at…</p><blockquote><em>If 1,000 developers use Claude Opus to perform one “commit this” action per workday (220 days/year):</em></blockquote><blockquote><strong><em>Well-designed workflow</em></strong><em> (only staged diff sent, ~20k tokens of context): Cost: ~$54,000/year &amp; Emissions: ~0.1–0.8 tonnes CO₂/year</em></blockquote><blockquote><strong><em>Large-context workflow</em></strong><em> (agent includes substantial repo context, ~200k tokens): Cost: ~$450,000/year &amp; Emissions: ~1–7 tonnes CO₂/year</em></blockquote><blockquote><strong><em>Worst-case 1M-token context workflow</em></strong><em> (agent repeatedly processes ~1M tokens per commit, no caching): Cost: ~$3.3 million/year &amp; Emissions: ~8–55 tonnes CO₂/year</em></blockquote><p>This type of setting goals, can probably work when the only considered output of a developer is code, documentation or something that can be git committed or generated by spending tokens. <a href="https://www.jaakkoj.com/concepts/doorman-fallacy">Doorman Fallacy</a> applies.</p><p><strong>Folks on Metrics:</strong></p><ul><li>Checkout this article — <a href="https://x.com/championswimmer/status/2051807284691612099">The AI Layoffs will continue</a>. It talks in depth about how one needs to judge AI effectiveness, which is actually not that easy as it sounds.</li><li><strong>Linus Torwalds</strong> agrees that <a href="https://www.youtube.com/watch?v=rDk_LsON3CM">lines of code is not a valid metric to judge productivity</a>.</li><li><strong>Dr. Greg Wilson</strong> calls most of the metrics that companies use currently, as Dodgy metrics, in his article <a href="https://third-bit.com/2026/05/20/twelve-ways-to-be-wrong/"><strong>Twelve Ways to Be Wrong About AI-Assisted Coding</strong></a>.</li><li><strong>Martin Fowler</strong> agrees in his <a href="https://martinfowler.com/fragments/2026-06-02.html">02 Jun 2026 Fragments post</a>.</li><li>Here is what Anthropic says in <a href="https://www.anthropic.com/institute/recursive-self-improvement">an article we linked above</a> — “<em>A caveat: Lines of code is an imperfect measure, as it measures quantity over quality. So 8× lines of code/engineer/day in the second quarter of 2026 is almost certainly an overstatement of the true productivity gain. Nonetheless, it indicates an acceleration. At Anthropic, we don’t reward people for how many lines of code they write; rather, team members are producing more code simply because they’re using AI systems to write more code.”</em></li></ul><p>What we need to do, is revisit our goals. This is where senior engineers, architects and technical leaders probably need to step up. You need to work with the top management and the junior developers on actually where AI stands and how it can positively and negatively impact software and people based on how it is used. Coming up with a number (or a bunch of numbers) might be difficult; we need human understanding to gauge projects, teams and individuals. Theoretically if every single occurrence in the world can be translated as a number, an amazingly intelligent algorithm would tell you everything, including future possibilities. T<a href="https://marvelcinematicuniverse.fandom.com/wiki/Zola%27s_Algorithm">his is what Dr. Zola tried to do</a>. Fortunately he was shut down by Captain America.</p><h3>The Series Ends</h3><p>I do not claim that I would have done better if I were there in anybody else’s shoes. This inflection point we are in means that no one knows what exactly would happen, we are all trying to do what we think is right. This write is just based on my observation of what is happening in multiple places, my understanding of the industry (the abstraction level of a junior developer and a bystander of various LinkedIn and X posts). I might be wrong. However, I highly doubt the following folks are…</p><h3>Follow</h3><p><a href="https://x.com/addyosmani">Addy Osmani</a>, <a href="https://x.com/badlogicgames">Mario Zechner</a>, <a href="https://x.com/mattpocockuk">Matt Pocock</a>, <a href="https://x.com/GeoffreyHuntley">Geoffrey Huntley</a>, <a href="https://x.com/Steve_Yegge">Steve Yegge</a>, <a href="https://x.com/simonw">Simon Wilson</a>, <a href="https://x.com/Vtrivedy10">Vivek Trivedy</a>, <a href="https://x.com/karpathy">Andrej Karpathy</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5420267d8331" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Chapter 5— Ascending to Agentic Heaven]]></title>
            <link>https://medium.com/@haricane8133/chapter-4-ascending-to-agentic-heaven-d90341113c67?source=rss-e8f8cb2ed41f------2</link>
            <guid isPermaLink="false">https://medium.com/p/d90341113c67</guid>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[software-engineering]]></category>
            <dc:creator><![CDATA[Hari Rajesh]]></dc:creator>
            <pubDate>Sun, 14 Jun 2026 15:11:19 GMT</pubDate>
            <atom:updated>2026-06-21T10:00:27.217Z</atom:updated>
            <content:encoded><![CDATA[<blockquote>A series on using AI wisely and sustainably in software development.</blockquote><ul><li><a href="https://medium.com/p/2a640c56b0c9">Chapter 1 — The Barista, Intuition and Learning</a></li><li><a href="https://medium.com/p/33e7656ad6ed">Chapter 2 — Human Brain and LLM</a></li><li><a href="https://medium.com/p/2a640c56b0c9">Chapter 3— AI, Speed and the Sustainability of Software</a></li><li><a href="https://medium.com/p/140209a5f7cb">Chapter 4— What counts as Innovation in the Age of AI?</a></li><li><strong>Chapter 5— The AI Maturity Ladder</strong></li><li><a href="https://medium.com/p/5420267d8331">Chapter 6— The AI Productivity Trap</a></li></ul><h4>Preface</h4><p>Read comment.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*w3ZZCqKI5HNSye0-.jpg" /></figure><h3>Scaling people up on AI</h3><p>Since AI based developement is an extremely fast growing transformative paradigm shift for software engineers, the natural concern for companies would be to ensure that some employees don’t adopt AI fast enough and are left behind so much that catching up is not a possibility anymore.</p><p>The concern is genuine and must be addressed, but in a sustainable way. There is an immutable truth that even AI cannot come close to shaking — “<strong>There is no free lunch</strong>”. It takes effort from each individual to reach <em>agentic heaven</em> (my wording for being an expert in AI / Agentic coding)</p><blockquote>Being an expert in using a tool doesn’t necessarily mean being an expert in the domain.</blockquote><h3>Sacred Rite of Passage</h3><p><strong>Each developer’s workflow is sacred, and personal to them</strong>. The ability of a developer to automate and model their workflow with custom chains of scripts and harness features, as instructions for the LLM + harness, is an art. It is a journey of self-discovery and joy for the developer. <em>This is one of the few remaining joys in a developer’s life</em>. Please do not kill this by providing them with a customized workflow where they just have to provide high level single sentences as prompts. This is what happened when handicraft artisans were destroyed by the industrial revolution — individual decisions on the end product is replaced with a common (often dry soul-less, like what <strong>Hayao Miyazaki</strong> said about automation of art) someone else prescribed format for everything. I want every developer to go through this beautiful journey of finding out their development workflow (go discover yourself).</p><p><strong>There is another advantage</strong> (with any automation in fact) when each developer models themselves and their workflow. They can clearly see their development workflow laid down before them, providing opportunity to improve it and grow further as a developer. How does a developer improve their development workflow?</p><ol><li>Experience facing bugs, and small technical challenges while coding.</li><li>Technical debates and knowledge transfer sessions with other developers and leads</li></ol><p>… and everything else that a developer usually faces. Putting it down provides an opportunity for a developer to see <strong>a blueprint of themselves</strong>. The right developer can improve so much from this.</p><h4>Diversity is still needed</h4><p><strong>Every codebase in human history</strong> has been subject to multiple developers all with different workflows. The growth was slow and steady with action from a lot of different types of developers, each with different background, different experience working in different projects and places, different problems solved, etc… A well performing team has a diversity of these people and experiences all working together on a common mission. Let us not change that into a single monotonous workflow for the entire company.</p><blockquote>Think about this. What purpose does DEI hiring for developers have, when every developer just needs a well defined Jira ticket (created by some lead product engineer), to use with the same template / workflow (created by some lead engineer)?</blockquote><h3>My AI Ascension Karate belt ascension ladder to Agentic Heaven</h3><p>In Chaoter 2, we would have seen <strong>Addy Osmani</strong> and <strong>Steve Yeggi</strong>’s ladders of expertise in AI. Here I present one (that works as of the day or writing this), specific to a junior developer with access to a lot of tools. This assumes that you take care of the <strong>other important pre-requisites</strong> like understanding the domain, programming language, the repository, software patterns, etc… like we discussed in Chapter 2</p><p>Go through the list from first to last; step into the next only when you have done the previous steps sincerely, and can handle the increased speed of code, which would require higher and higher proficiency in general software development.</p><ul><li>Avoid building new harnesses or using harnesses that someone else built, unless there is a strong reason (next section tells you when you need a custom harness). Use popular ones like Claude Code, Codex (later OpenCode). Using these would give you transferrable ‘skills’.</li><li>Use minimal features that solves the need — first repo CLAUDE.md setups, then claude rules.</li><li>Then experiment with skills. Hand write skills based on your workflow and the work that you are doing in the current repository.</li><li>Then try to lookup skills available on the Internet. Try to study and learn from them.</li><li>If these are not sufficient, add dedicated sub-agents. Do not try to parallelize agents yet (agent teams)</li><li>Setup a plain file based docs directory and an issue tracker based on it. Take special care to segregate which documents are sources for context and which are temporarily generated for some purpose. Do not pollute the working directory. Ask the agent to write files in the OS’s temporary directory if required.</li><li>Try to give the agent longer and longer problem statements. Keep iterating on the setup, skills and agents till the agent is able to perform the right tasks at the right time, and is on track.</li><li>Experiment with Ralph Loops and techniques that orchestrate a single coding agent.</li><li>Use Claude Code’s workflows features. It is a way to write a JS script that spawns and manages agents. You get determinism and LLMs the way you want. You can manage the loop, ralph particular agents, etc… In my opinion, this is the most amazing feature of Claude Code till now. Combining it with all the other features till now, makes your setup over powered.</li><li>So by now your agent should know your development workflow. Note that this is personal to you; don’t share it with others bit by bit just yet. Instead share and learn techniques.</li><li>Try to improve your workflow and adapt it based on popular patterns that work with Agentic coding. I will give you an example — LLMs, not unlike humans, can find faults with others better than doing it themselves. To do this though, you shouldn’t use the same context window that created the code — again not unlike humans, agents that wrote the code will defend it with prejudice. Use a fresh context window, or better a new model to check what the first model wrote; create a feedback cycle and bake it into your workflow</li><li>Study the official documentation thoroughly and experiment with all features.</li><li>Do not use any packages for context management, issue management, MCP servers, unless you have enough experience working otherwise and already hit the limitations with the plain thing.</li><li>The repo setup, system prompts and skills should give any new session on the repo enough context to start working right away. So do not continue a session indefinitely. /clear a session and start new agents. If it doesn’t work properly, then work on the setup again.</li><li>Use different popular agents for various tasks — use Claude Code with a Claude model for a task, and then ask Codex with a GPT model to verify it, etc, all in different sessions in the same folder. Your repo setup thus need to support both — example CLAUDE.md and AGENTS.md.</li><li>If you are proficient with everything this far, then experiment with open source models and OpenCode. Then try to find out which setup woks best for which tasks.</li><li>Open multiple terminals with different agents (maybe Claude Code, Codex and Open Code) all from the same project directory, use them for different tasks.</li><li>As seen before, have each agent update documentation so that other agents have context of what changed. Often clear the context of each agent and check if they have enough context (and not researching extensively again). Iterate with docs directory till we reach a state where the agents need not research extensively each time.</li><li>Learn and use a terminal multiplexer like tmux to manage each agent. Later on, move to a solution like herdr.dev.</li><li>If this is not sufficient, experiment with external packages for agent memory and issue tracking.</li><li>At this point, you need to be 100% sure you have exhausted every other technique that you can do with a plain single agent.</li><li>Enabling agent teams in claude and experiment with parallel agents.</li><li>Experiment with VibeKanban, Multi or any other very high level parallel orchestrators.</li><li>Now, build you own harness (next section)</li><li>Fine tune Open Source models for particular use cases and use them in particular portions of a multi agent system.</li><li>You are ready. Why not create the next frontier LLM for coding!?</li><li>Now try to cut down on the number of tools, abstractions, and extensions you use, while not reducing productivity.</li><li>Try to move to smaller and smaller LLMs, till you can work with locally run models without reducing productivity.</li><li>Grow to a level where you can stop using all AI, without reducing productivity.</li></ul><h3>Everybody should build a harness</h3><p>Know what this means? <strong>No one should (be forced to) use yours’</strong></p><p>Here is my suggestion on when you should think of building your own agentic harness for coding</p><ol><li>As the last step of the developer AI ascension or you are simply interested. Else,</li><li>If you have “true” innovations that aren’t supported in Claude Code, or can’t be used around Claude Code.</li><li>You have a very specific use case that cannot be solved with the features that existing popular harnesses currently offer.</li><li>You have enough man power and bandwidth to keep supporting whatever innovations are made by Anthropic and OpenAI, in your custom coding harness. Conway’s law should not be a hinderance in your team to adopt new true innovations.</li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d90341113c67" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Chapter 4— Attention is all you need]]></title>
            <link>https://medium.com/@haricane8133/chapter-3-attention-is-all-you-need-140209a5f7cb?source=rss-e8f8cb2ed41f------2</link>
            <guid isPermaLink="false">https://medium.com/p/140209a5f7cb</guid>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[software-development]]></category>
            <dc:creator><![CDATA[Hari Rajesh]]></dc:creator>
            <pubDate>Sun, 14 Jun 2026 15:11:09 GMT</pubDate>
            <atom:updated>2026-06-21T09:58:18.127Z</atom:updated>
            <content:encoded><![CDATA[<blockquote>A series on using AI wisely and sustainably in software development.</blockquote><ul><li><a href="https://medium.com/p/2a640c56b0c9">Chapter 1 — The Barista, Intuition and Learning</a></li><li><a href="https://medium.com/p/33e7656ad6ed">Chapter 2 — Human Brain and LLM</a></li><li><a href="https://medium.com/p/2a640c56b0c9">Chapter 3— AI, Speed and the Sustainability of Software</a></li><li><strong>Chapter 4— What counts as Innovation in the Age of AI?</strong></li><li><a href="https://medium.com/p/d90341113c67">Chapter 5— The AI Maturity Ladder</a></li><li><a href="https://medium.com/p/5420267d8331">Chapter 6— The AI Productivity Trap</a></li></ul><h4>Preface</h4><p>Read comment.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*42y9xUqkHsoD4gyV" /></figure><h3>AI Innovation Landscape</h3><p>AI has enabled us to write new software at a scale which we could never dream of otherwise, which doesn’t sound bad on its own. AI agents are improved tools for software development. Generally when a tool improves, the problem statement to use the tool with, also grows. Cement, concrete, iron bars and complex construction machines paved the way for complex constructions like the Burj Khalifa. But that doesn’t mean that the work done by masons on a regular building a couple of centuries back took any less effort.</p><p>Innovations can be visualized in the form of a triangle with the more foundational / platformic stuff at the bottom and smaller innovations arranging themselves on top of each other, for customized use cases.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/419/0*JFr44qwVQvWemPUA.jpg" /><figcaption>Innovation Triangle</figcaption></figure><h4>LLM and Harness</h4><p>Innovations in Agentic Coding can come in from two places.</p><ol><li>The LLM / Model itself</li><li>The Harness — handles memory, context, execution loop, failures, etc… engineering required to get the best out of an LLM.</li></ol><p>For all practical purposes, we can say that the paper “<a href="https://arxiv.org/abs/1706.03762">Attention is all you need</a>” started the progress towards LLMs the way they are today. The LLM is the actual “magic”. Critical innovations happen in the model (LLM) world by ML scientists.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*VUqzYSEJtgURl4_L.png" /><figcaption>shows anthropic models’ continuous working duration capability</figcaption></figure><p><strong>The Harness</strong> is the piece of software that decides how to use this LLM magic to do something useful. Claude Code and Codex are the harnesses maintained by billion dollar companies Anthropic and OpenAI respectively and are backed by a billion developers.</p><p>Anthropic engineers confirm in the presentation<br><a href="https://www.youtube.com/watch?v=mR-WAvEPRwE">Anthropic Workshop: Build Agents That Run for Hours</a>, that whenever Anthropic builds a new model, the coding harness Claude Code gets significant changes to make use of the new features of the LLM… Andrew Wilson, an Anthropic staff (from the video linked above says) — “<em>It is interesting to see that as LLMs evolve the harnesses do not disappear, but rather they evolve to fill in the gaps</em>”. As LLM capabilities improve, the need to fill these gaps reduce. As model improves, cost and overall performance improves. As this happens, the requirement for harnesses to be complex, comes down. While LangChain and DeepAgents have a philosophy of having Harness complicated so that it can work with a wide variety of models, Anthropic and has a model first philosophy — helps focus attention on where most innovations can happen (and also maybe sell costlier tokens, and keep the user within the ecosystem).</p><h4>Benchmarks</h4><ul><li>Benchmark for LLMs — <a href="https://artificialanalysis.ai/evaluations/artificial-analysis-intelligence-index">https://artificialanalysis.ai/evaluations/artificial-analysis-intelligence-index</a></li><li>Benchmark for LLM + Harness (agent) — <a href="https://www.tbench.ai/leaderboard/terminal-bench/2.0">https://www.tbench.ai/leaderboard/terminal-bench/2.0</a></li><li><a href="https://deepswe.datacurve.ai/">https://deepswe.datacurve.ai/</a></li></ul><p>These benchmarks tell us at the moment which harness + model combination is the king. This list keeps changing wildly. Claude Code + Anthropic Models surprisingly only rarely enter the top 10.</p><p><strong>Most of us have no control over the LLM itself</strong>, and are going to consume what Anthropic, OpenAI or Google builds. But we do have control over the harness. Naturally everyone (including me) would be tempted to build the next best harness that dissolves everything in the Terminal Bench leaderboard. Several individuals, teams and companies have invested in building their own harnesses; few have succeeded.</p><p><strong>But before you start attempting that</strong>, just watch the leaderboard for a couple of weeks to a month. The list will change so frequently that you’ll wonder what happened to the harness that was in top 5 just a week back. You will notice the constant presence few members in the top 25 — Claude Code and Codex. Yes. They aren’t going anywhere.</p><h4>Open source vs first party harnesses</h4><p><strong>The advantage that custom harness builders have</strong>, is that they can make their harness work well with a variety of different models — research and practice shows that multi LLM SDLC works better (for example, development with a model and testing with another). Harness engineering has gained so much traction that there are dedicated tools like the following that help with creation of a custom harness (easier than working with vanilla <a href="https://www.langchain.com/">langchain</a> and <a href="https://crewai.com/">crewai</a>)</p><ul><li><a href="https://www.openhands.dev/">Open Hands</a></li><li><a href="https://github.com/langchain-ai/deepagents">Deep Agents</a></li><li><a href="https://flueframework.com/">FlueFramework</a> (program first)</li><li><a href="https://github.com/ai-boost/awesome-harness-engineering">Awesome Harness Engineering</a></li><li><em>Hot Take : I wouldn’t put Claude Code SDK or Codex SDK in this list.</em></li></ul><p><strong>The advantage Anthropic and OpenAI have</strong>, is that they control the model (the magic). They know the internal of their closed source model so much that it gives them an edge while designing their harness to work with it. They also have the time advantage — They can research, build and improve their harness internally multiple times before their model is even released to public for testing.</p><p><strong>These two companies</strong> are head to head with the performance of their coding agent solutions. Codex is dangerously close to overtaking Claude Code. Just as this and Anthropic / OpenAI IPO is about to happen, Claude Code hired OpenAI co-founder and AI virtuoso <strong>Andrej Karpathy</strong> while flirting openly about Mythos. OpenAI’s GPT models are still king in the benchmarks. Though a little while later, China pulled a reverse card by reducing DeepSeek V4 prices by 75%. Anthropic countered by releasing Opus 4.8 and Claude Code <a href="https://code.claude.com/docs/en/workflows">workflows</a>. OpenAI, the token is now in your context window.</p><h4>Agility</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*vpXvDwED7zcEDm0p.png" /><figcaption>LLM updates make earlier harness changes irrelevant</figcaption></figure><p>In the same presentation linked above, Anthropic talked about a working custom Harness built out of the Claude Code SDK in Opus 4.5. When Opus 4.6 was launched, most if it was not required.</p><p>Anthropic saying can be considered a marketting gimmick, but how about this? The same was confirmed by <strong>Yichao “Peak” Ji</strong> (the founder and lead developer of <a href="https://manus.im/">Manus AI</a>) in a <a href="https://www.youtube.com/watch?v=6_BcCthVvb8">tech session on context management</a> / interview by Langchain’s <strong>Lance Martin</strong> — “we have to completely re-write manus everytime Anthropic lauches a new model. We have already done it 4–5 times”.</p><blockquote>Any custom harness builders need to have this level of agility to adopt newer frontier models.</blockquote><h4>Community</h4><p>But the key point to consider here, is <em>community</em>. Anthropic introducing <a href="https://www.anthropic.com/news/model-context-protocol">MCP</a> and <a href="https://anthropic.skilljar.com/introduction-to-agent-skills">Skills</a> (and just a couple of days back, <a href="https://code.claude.com/docs/en/workflows">workflows</a>) were true innovations. Look at how widely they are adopted today by everyone else? There were open source or company / team / individual maintaining harnesses before these were introduced. But the moment these were introduced by Anthropic and millions of community people started creating contributing to MCP and skills, the custom harnesses that weren’t updated to support these two, didn’t survive (or will die a slow death).</p><p><em>Aside: I will give you an example of what happened to me. I was working on something very similar to the newly introduced Workflows feature of Claude Code — a script that triggers agents and manages context, state and execution loop externally (but one that also works with Codex and Open Code as it run on top of agents and not inside). Good idea with potential (I may still do it and use myself though, for fun). Workflows introduced by Anthropic works inside Claude Code. Even if I had released before Anthropic, after Anthropic introduced this, it wouldn’t make sense for anyone else to spend time working with my SDK and creating my script, when they can use the community backed Claude Code workflows. If different agents are required to be used, they can use a skill to make Claude Code trigger the other agents. This ‘workflow’ technique is not truly novel (I am sure many would have experimented with it, especially the “arch linux” developers who use and customize </em><a href="https://pi.dev/"><em>Pi coding agent</em></a><em> to the end of the world). But Anthropic delivering it within Claude Code is different, and it would only be a matter of time before this feature gets adopted into Codex and others. The advantage with Pi is that you can create whatever you want, yourself, using the intelligence of the LLM that you use with Pi.</em></p><h4>Ralph</h4><p>Innovation also doesn’t need to be radical and completely displace what is already available and popular. Checkout <a href="https://ghuntley.com/loop/">Ralph Loop</a> — Devised by <strong>Geoffrey Huntley</strong>, this is a comically simple but effective way you can force an agent to run multiple times, each time inching closer towards the goal. This is a layer around a single coding agent and continuously passes the same prompt to the agent every time its context window nears completion. So it keeps trying and trying till the code base is clean, after which you can pass newer tasks. This technique has been adapted in various formats in numerous places. This is an example of an innovation that will stand the test of time (unless of course the model itself gets so advanced)</p><h4>Less is more</h4><p>So what can we do? We can absolutely build a new harnesses that solve all the problems that Anthropic and OpenAI are unable to solve yet. Just that we need to approach this with humility and acknowledging that it needs a lot more effort to sustain and maintain the tool than to develop it, especially if you want all your users to use it (just like with all AI assisted software development. You can create quickly).</p><p>A lot of folks now are trying to build a <strong>Claude Code / Codex alternate harness</strong> (sometimes with their SDKs itself), to take care of a generic developer’s end to end SDLC needs, straight from a well defined problem statement (a Jira ticket). This usually works by defining new agents (new system prompt), a context / memory engine and a preset defined workflow between these agents to solve a given problem statement.</p><p>But usually what mostly happens is that these newly developed “<strong>agent abstractions</strong>” usually work for <strong>particular usecases</strong> (often the developers’ workflow). They showcase the features through articles prompting others to get it adopted soon in their teams to get a piece of the cake. Then slowly complaints will come in from developers that it isn’t working well for other (their) use cases and workflows. Then effort gets put into making it more and more generic… … … until it gets closer and closer to Claude Code itself.</p><blockquote>Hot Take: The more and more generic a developer tries to tune their custom coding agentic harness, the closer and closer they get to Claude Code itself!</blockquote><p><strong>The reason for this is simple</strong>. Claude Code as a tool is meant to be used by all sorts of developers for all sorts of activities. So any feature that they add to the Claude Code harness would be in such a way that they do not favor a particular workflow too much to compromise the others. The context management, memory and the workflow / loops are all designed to be used for a wide variety of use cases. The way these can be customized, is by leveraging the features that they have — placing CLAUDE.md, skills and agents in the repository and the user directory. These are plenty enough for majority of developers and they need not look past for shinier tools or plugins to install. The introduction of the workflow feature in my opinion is a mic drop moment by Anthropic for custom harness builders, as now all the features provided by claude code put together can cater to almost all the power users’ needs.</p><p><strong>Therefore if any enterprise</strong> wants to build a harness for all their enterprise’s developers to use, just don’t. You already have a few — Claude Code, Codex or Open Code. If a team wants to build one for a particular use case, temporarily or for educational purposes…, sure go ahead. But you have to carry the additional burden of maintenance. You alternate is just to provide clear skills, workflows and rules that the user can use in their dev. Please rethink if really we need a cloud agent (a headless coding agent that runs in cloud)…</p><h4>Some harness techniques</h4><p><strong>Memory management tools</strong> — Any additional feature like graphs for context management and MCP servers are to be used only if required.</p><ul><li><a href="https://github.com/tobi/qmd">https://github.com/tobi/qmd</a></li><li><a href="https://github.com/safishamsi/graphify">https://github.com/safishamsi/graphify</a></li><li><a href="https://github.com/MemPalace/mempalace">https://github.com/MemPalace/mempalace</a></li><li><a href="https://github.com/thedotmack/claude-mem">https://github.com/thedotmack/claude-mem</a></li><li><a href="https://github.com/gastownhall/beads">https://github.com/gastownhall/beads</a></li><li><a href="https://github.com/zilliztech/claude-context">https://github.com/zilliztech/claude-context</a></li><li><a href="https://memory.build/">https://memory.build/</a></li><li><a href="https://github.com/VectifyAI/PageIndex">https://github.com/VectifyAI/PageIndex</a></li></ul><p>This is one of the most rapidly innovated areas in the open source community. We are trying to find the best way to provide context to an AI agent, when it needs it. Underneath, the concepts are very generic and popular.</p><p><strong>Token saving tools </strong>— Food for thought… LLMs are designed to work with human language. Anthropic will train their model for human inputs, which at times can be verbose and at times can be less verbose. Would performance be the same when we use tools that stand before claude code reducing the tokens?</p><p><strong>Meta Prompting</strong> — LLMs are always verbose. Skills and CLAUDE.md markdown files are ways Claude Code allows us to “prepare the agent” with details required to handle a repository. It is often a pattern when developers use LLMs themselves to generate these files. If they are verbose, then we risk two things — contribute to context rot and contribute to quick context window compactions. It is an art to write the correct skill to be at the correct verbosity level.</p><p><strong>Multi agent orchestration</strong> — Here are some examples of claude / agent orchestrators / abstractions that use multi agents. There are tons and tons of them, and developers love to build their own too. As seen in the recommendation that follows, try to avoid using them unless you really see a need. Parallel agent execution seems fun, but there are a lot of problems that can potentially arise. <a href="https://code.claude.com/docs/en/agent-teams">Agent Teams</a>, a feature from Claude Code is in the experimental status for a very long time, for a reason.</p><ul><li><a href="https://vibekanban.com/">https://vibekanban.com/</a></li><li><a href="https://multica.ai/">https://multica.ai/</a></li><li><a href="https://github.com/gastownhall/gastown">https://github.com/gastownhall/gastown</a></li></ul><h3>“True” Innovation</h3><blockquote>Hot Take: Mostly, a “true” innovation is what can replace Claude Code or Codex or the models within.</blockquote><p><strong>Innovations need not always be at a foundational levels</strong>. Smaller innovations can be at the company / org / team or at an individual level. For example, I can create a collection of scripts, skills, markdown files and MCP to make an agent work for my (or team, org, company, everyone) use cases. This example can be called as “customizing an agent” or “creating a workflow”, rather than “creating an agent”. The reason is that skills and markdown files are a platform feature provided by actual innovations, that enables one to customize the agent to their workflow. It is an innovation at a scope , I agree — But these are not “true” innovations that you need to create a <strong>new harness</strong> or a <strong>new LLM</strong> for. You just need to configure and customize an existing harness with its existing feature sets. You can provide others with these configurations, if you want.</p><p><strong>Automations are mostly personal victories:</strong> Developers love to automate things with scripts. They pride on it. With LLMs, you can automate a lot easier and benefit from plain spoken english and the intelligence of the magical AI model to take decisions. Today’s Harnesses allow you to execute any command and read its output. So, simply identifying a script (or an MCP server) or a chain of these and attaching it with your agent, is you just discovering a nifty way to automate a part of your workflow. These are personal victories unless the approach is breathtakingly novel and benefitting to others in a way they can’t do themselves.</p><p><strong>Abstractions and “tools”:</strong> We brushed about this in a previous chapter. AI has enabled everyone to develop a tool, which mostly is a combination of these concepts and pre-existing tools. Putting a set of pre-existing concepts together that work well for you, also comes under the umbrella of “Automations, or personal victories”. So if you build one such tool that works fantastic for you, congrats. You can give it a name and even publish it with some numbers, but be cautious about asking everyone to use it. Instead of publishing the tool itself, it might even make more sense to publish the concepts behind it. This will encourage others to learn and try out these concepts, enabling true learning. Let everyone build their own tooling.</p><p><strong>Building in Silos</strong>: Using a tool clearly doesn’t level up a person. We also saw that not using any tools, is also not right too. We need to let each individual use the lightest tools first and gradually scale them up to higher level tools. Each tool has a concept which can be implemented by each individual. We will see in the upcoming section how this can be done, but a requirement is that we should not ask folks to stop experimenting on their own and instead adopt another tool. It should be the reverse way. Instead of adopting a tool, we should encourage developers to build their own tool first, but then compare with others’ tools and learn.</p><p><strong>Workflow: </strong><em>Each person’s workflow is different</em>, and so what you build, may not work for others. What is a developer’s workflow? It is too much to define straight away — but think of this as everything ranging from the developer’s thought process to prompting behavior. Also remember that every other developer out there, is also with the same ambition of making skills and markdown files to customize their agent to match (automate) either <em>their</em> workflow, or a workflow that <em>they think</em> is <em>the best</em>. So we need to take utmost care on what we are publishing out there ; best to review what is already out there, and learn from them. Distinguising between opinionated stacks and un-opininated “platformic” releases are crucial. Creating an abstraction over an existing popular agent and forcing others to use it, should not be done at all. <strong><em>This leads us to probably the most important chapter.</em></strong></p><h4>Attention is all you need</h4><p>99% of the concepts on how to make a popular coding agent work for yourself is either universal, or supported natively. So among 100s of tools that do more or less the same minor enhancement over existing processes, the one that gets the more attention, will succeed whereas others will wither out. So leadership team and senior engineers have a very important responsibility to showcase or promote true innovations and tools that will stand the test of time, so that the whole organizations’ developers do not get too much heavily invested into a tool that isn’t really a tool, or miss out on learning experiences because the tool puts them at an extremely high abstraction level.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=140209a5f7cb" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Chapter 3— The middle ground]]></title>
            <link>https://medium.com/@haricane8133/chapter-2-the-middle-ground-cf400328ae31?source=rss-e8f8cb2ed41f------2</link>
            <guid isPermaLink="false">https://medium.com/p/cf400328ae31</guid>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[software-development]]></category>
            <dc:creator><![CDATA[Hari Rajesh]]></dc:creator>
            <pubDate>Sat, 13 Jun 2026 08:24:08 GMT</pubDate>
            <atom:updated>2026-06-21T09:56:36.682Z</atom:updated>
            <content:encoded><![CDATA[<blockquote>A series on using AI wisely and sustainably in software development.</blockquote><ul><li><a href="https://medium.com/p/2a640c56b0c9">Chapter 1 — The Barista, Intuition and Learning</a></li><li><a href="https://medium.com/p/33e7656ad6ed">Chapter 2 — Human Brain and LLM</a></li><li><strong>Chapter 3— AI, Speed and the Sustainability of Software</strong></li><li><a href="https://medium.com/p/140209a5f7cb">Chapter 4— What counts as Innovation in the Age of AI?</a></li><li><a href="https://medium.com/p/d90341113c67">Chapter 5— The AI Maturity Ladder</a></li><li><a href="https://medium.com/p/5420267d8331">Chapter 6— The AI Productivity Trap</a></li></ul><h4>Preface</h4><p>Read comment.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*LtoOsGPk3XvEbcu8.jpg" /></figure><h3>How much AI to use?</h3><p>There are two schools of thought on how we are to develop software in these times</p><ol><li>Speed up</li><li>Slow down</li></ol><h4>Speed Up</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*q9NashzCPQUQaRwI.jpg" /><figcaption>Karate belt chart of AI developer ascension all the way up to Agentic heaven</figcaption></figure><p>The above image is from <strong>Addy Osmani</strong>’s write up — <a href="https://addyosmani.com/blog/code-agent-orchestra/">https://addyosmani.com/blog/code-agent-orchestra/</a>. This describes the levels an AI assisted software developer can ascend themselves, by growing skills and experience in AI (like belts in Karate). This ascension was initially conceived by <strong>Steve Yegge</strong> when he wrote about Gas Town, a very high level over powered orchestration / AI software development tool he built, here — <a href="https://steve-yegge.medium.com/welcome-to-gas-town-4f25ee16dd04">https://steve-yegge.medium.com/welcome-to-gas-town-4f25ee16dd04</a>. I urge you to read through both before proceeding. It will be a long read, but it will be worth it. I will wait right here.</p><p>Note: I have my own developer AI ascension karate belt ladder in chapter 5 (a path for folks to scale up on AI agent usage for coding).</p><p><strong>Steve Yegge</strong> in his article (and his interviews) claimed that he has not looked at any code for a long time (also <strong>Andrej Karpathy</strong>). The idea here is that you build your tooling carefully to the extent where you no longer need to be at the code level, and instead just live at the high level where you just nudge all agents to work. (<strong>Steve</strong> and <strong>Andrej</strong> are a heavily experienced software engineers and can do this (mostly) without consequences)</p><p>Given enough context engineering, multi agent systems and tooling, we should be able to build an E2E autonomous agent that takes care of entire SDLC. The aim is not to remove or redice humans from software development, but just take over the boring parts (really? That is a shame. We love to solve problems and write code). All we need is a well defined Jira ticket. But then if AI gets to a point where it can manage end to end software quite easily, effectively and efficiently, I wonder how many software companies would exist, within that how many teams, who would write software, what software would be written, and why…</p><h3>Slow down</h3><p><strong>The argument:</strong> Production code is not just about getting something working as soon as possible. Teams have real clients who don’t seem to be Hackathon judges. The output of coding is not just making the Product Owner, the QA and the clients happy in record time. Systems are not simple; they grow and grow and much like <a href="https://en.wikipedia.org/wiki/Conway%27s_law">Conway’s law</a>, software systems will start resembling the complex team structures in organizations. We simply cannot teach everything to agents — no amount of memory tools or context management or multi agents or context window <a href="https://www.humanlayer.com/blog/long-context-isnt-the-answer">will solve</a>. Till before AI, a codebase in a company would have grown organically facing lots of conflicts and debates among the several developers, even for absolutely trivial things like using tabs or spaces, which is missing now. Do we know the impact right away?</p><p>All the articles released by companies on AI wins, are focussing only on the short time gains. None of these companies are reporting an increase in revenue, except those selling AI. Sorry that is not true. <a href="https://isaiprofitable.com/">Even OpenAI and Anthropic are losing money; only NVIDIA isn’t</a>. There isn’t enough data to show what happens to companies that use too much AI. Tell me one company that has seen tremendrous increase in revenue post the agentic coding tools boom (I know a few startups may have, but large companies?). These companies are in a pressure to show some gains from AI. Can we fully trust the articles boasting AI use?</p><p>Read <strong>Mario Zechner</strong>’s unfiltered take here — <a href="https://mariozechner.at/posts/2026-03-25-thoughts-on-slowing-the-fuck-down/">Thoughts on slowing the fuck down</a> or his talk on YouTube. This is a great article explaining in depth — <a href="https://larsfaye.com/articles/agentic-coding-is-a-trap">Agentic Coding is a Trap</a></p><p>The writeup <a href="https://www.codingwithjesse.com/blog/rockstar-developers/">Cleaning up after AI rockstar developers</a>, by <strong>Jesse Skinner</strong>, perfectly summarizes the concern. Slow down, folks!!!</p><p>There are <a href="https://www.media.mit.edu/publications/your-brain-on-chatgpt/">many scientific studies</a> being conducted on the negative impacts of excessive LLMs usage on humans. If you are a software company and this doesn’t matter to you while you keep unreasonably decreasing deliverable deadlines (and also restricting token usage), just what do you think you are you doing? In the grand scheme of things, how much do you think what you are doing is worth?</p><p>Let us keep this effect aside, and focus on the impact that “not slowing down” will lead to software. Read through these articles from <strong>Addy Osmani</strong> — <a href="https://x.com/addyosmani/status/2052124873208799378">Cognitive Surrender</a>, <a href="https://x.com/addyosmani/status/2056078124346228860">Don’t Outsource the Learning</a> and <a href="https://x.com/addyosmani/status/2059844244907696186">The Orchestration Tax</a>. They will tell you that if you want software and teams to last long, we need to have proper adequate time spent in the development phase. There are too many activities that are done by a human, that cannot be modelled into a context window as text.</p><p>If you have ever felt that creating a new project using AI is much easier than collaborating with someone on an existing project, then it indicates that AI was not used in the way it is supposed to. If you give into AI too much in the development phase, we will reach a situation where we are dependant on AI for 100% of the tasks, including during production issues with real customers. If there is too much delivery pressure, developers will start to lose focus on understanding code at a level that a generation of developers before used to. Please continue reading; we will look into some other impacts of relying on AI for everything.</p><p><strong>Far wider implications</strong></p><p><em>Fiction is becoming a reality</em>. Here is an article from a company we least expect it to release — <a href="https://www.anthropic.com/institute/recursive-self-improvement">When AI Builds itself</a>. Anthropic wants us to slow down AI usage, or risk everything. Skeptics may call it a PR stunt before the IPO. Call me dramatic, but what they are claiming here, may happen. But even if it doesn’t at that level, the impact it would have on humans is undeniable. Wait till robotics catches up. Please huddle up with leads and AI experts to re-consider your current plans. That is the most that is possible from my side.</p><h3>Middle ground (for us developers)?</h3><p>Unless <strong>Micheal Burry</strong> is right for the second time and investors pull the plug (or there is a huge sunspot), AI assisted software development isn’t going anywhere. Anyway, if a huge sunspot appears, we will have bigger things to worry about.</p><p>Anthropic, OpenAI, Google Deepmind, Microsoft AI and other big companies claim that LLMs will keep getting drastically better and better with every release. One of the best sought after use cases for LLMs and AI just so happens to be writing code. All LLM models that are released now a days, are with one advertisement at the top — something akin to “<a href="https://qwen.ai/blog?id=qwen3.7">Long Duration Autonomous Execution</a>”. The world is clearly looking to invest more and more on autonomous work.</p><p>There is no other way. Developers need to learn to use AI better, and deliver faster. But what is ‘better’? It is not running several claude code sessions in parallel, or using a heavily abstracted harness that does the same interally, and definitely not ‘not understanding’ what code is being pushed to production.</p><p>Something worse than not understanding what code goes to production, is not knowing what architecture or low level conflicts the LLM faced, what the final choice was, and the reason behind…</p><p><strong>The karate belt chart of AI developer ascension is true</strong>. But it takes more than tools to get there. Being an expert in ‘using’ AI tools, will ascend you to the Agentic heaven, not more than having a sharp knife will make you a neuro surgeon. In my opinion, the more a developer grows an expert with the following skills, the better chances that developer has to ascend to Agentic heaven, and ship software faster, without consequences.</p><ol><li>Experience building software and systems.</li><li>Programming language, patterns, intricacies,… Databases, software systems…, cloud, infra, core skills like bash, git, regex, etc…</li><li>Domain knowledge, team structure of the org, user / client patterns (and ahem… politics too, if you will excuse me).</li><li>The (legacy) repositories you will be mostly working on.</li><li>Understanding LLMs, Agents, AI tools and (surprisingly) English</li></ol><p>Being an expert in all these 5 areas (and definitely more) will give a developer good mental bandwidth required to handle the speed in which AI writes code, and get it ready for production (and survive the test of time). This is one definition of a 100X developer, and these are the ones that we should target to be.</p><blockquote><em>AI would help a 100X developer become a 1000X developer, more than it would help a 10X developer become a 100X developer, and so on…</em></blockquote><p>This is not only true for software development and AI, but with any tool and any applied domain. Here is a crude chart.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*HHrqaj434hhyR09K.png" /></figure><p>How does a developer build these 5 skills? How does a team ensure they build these developers? How does the company ensure that they have these teams? We will see some options below (if you feel you are not growing in these areas you need to reorient yourself)</p><h4>The messages</h4><p><strong>If you are a non-technical leader</strong>, please consider these thoughts. Just getting developers access to AI tools might not help sustain software for a long time. It will help rapid development of software at the moment, but only a culture change would make a development team focussed on the quality and understanding of the software and systems while developing it. We can focus on upskilling developers’ core software knowledge and use of proper AI tools (we will see in the next section what this means), as these are good ways we can have rapid delivery of software without severe consequences. Choosing the right metric to judge software development with AI is important. I understand that it is an industry wide thing (including Meta, Amazon, etc…) and that you wouldn’t focus on a single metric but a combination instead. But the fact that developers are aware, creates unwanted pressure. Estimating the correct time to deliver a software is critical, and it might decide everything including whether you have good senior engineers in the future.</p><p><strong>If you are a senior engineer or a technical leader</strong>, please consider these thoughts. You have had the luxury of spending a lot of time understanding software. It is your duty to ensure that today’s junior developers are mentored to use the <em>right</em> AI tools <em>wisely</em> and are shielded from excessive delivery pressure or peer pressure to release or showcase something. This only will pave way for good quality senior developers in the future. We need you to experiment and ascend the AI karate belt ladder yourself quickly, so that you can understand where AI works and where AI doesn’t. We need your effort and experience to set the right timeline / estimate for delivering a problem statement, when your team with all AI tools, can deliver sustainable software. Please mediate between non-technical leaders and developers.</p><p><strong>If you are a junior developer</strong>, please consider these thoughts. Think.</p><ul><li>When was the last time you built something on your own?</li><li>Are you feeling that your software development skills or command on a programming language, fading?</li><li>Has AI eased your work? Or are you working more?</li><li>Are you truly happy / proud of anything you work on?</li><li>Do you think the code you wrote with AI, would survive another 3 years?</li><li>Have you learned anything new in the last month?</li></ul><p>If your answers are not satisfactory for these questions,</p><ul><li>Speak up with your managers and leads.</li><li>Skill up on the 5 points we saw above, while delivering what the Jira ticket asks you to.</li><li>Do not use over powered tools, skills or abstractions unless required or you understand what happens inside deeply, or can build it yourself.</li><li>Explore features like <a href="https://code.claude.com/docs/en/output-styles">learn-by-doing mode in Claude Code</a>. This is underrated, and I would highly recommend you to try it. Use AI for collective benefit.</li><li>Try prompting like this “/goal making me understand xyz is y our goal. Don’t implement anything till I understand xyz. Ask me questions and scenario based questions to test me knowledge. At any point if I make some mistake, please take a note of it and make me understand”, etc… If you want, you can create a skill for this. There are some skills available that do the same like this, online.</li><li>Channel your exploratory energy in these ways to sustain everything, not in using the next tool.</li></ul><p>I cannot resist this reference from Doctor Strange 2 — <a href="https://youtu.be/UdskcNUDsV0?si=1G-M4evskNrKeN7y&amp;t=141">“You are the master of the mystic arts. They are spirits. Use them. Use them” — Doctor Palmer</a></p><h3>Appendix</h3><h4>Pre-existing problems</h4><p>Most of the problems that we see due to AI are mostly pre-existing issues, now blown out of proportion. AI is a fundamentally newer and different shift that will change the course of human history unless it is used properly. We cannot just ignore these problems just because they may be pre-existing. We have to take a stand and fix them as much as possible.</p><h4>“Next to each other”</h4><p>You can find lots of examples like the following which indicates that we are at an inflection point, and everyone does what they think is right, or follow who they think is right. All that you read are ‘my thoughts’, because we all can be wrong (including me). What will help is an open conversation without fear. So as you read through this, I implore you to put yourself in ‘planning mode’, and spend adequate time……., just…………, thinking………………</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*amhLUNZWAE9WEeu1.png" /></figure><iframe src="https://cdn.embedly.com/widgets/media.html?type=text%2Fhtml&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;schema=twitter&amp;url=https%3A//x.com/steipete/status/2063697162748260627&amp;image=https%3A//i.embed.ly/1/image%3Furl%3Dhttps%253A%252F%252Fpbs.twimg.com%252Fprofile_images%252F1131851609774985216%252FOcsssQ9J_200x200.png%26key%3Da19fcc184b9711e1b4764040d3dc5c07" width="500" height="281" frameborder="0" scrolling="no"><a href="https://medium.com/media/7ba9a6249f2161bc9033c7cb17c3b082/href">https://medium.com/media/7ba9a6249f2161bc9033c7cb17c3b082/href</a></iframe><iframe src="https://cdn.embedly.com/widgets/media.html?type=text%2Fhtml&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;schema=twitter&amp;url=https%3A//x.com/ThePrimeagen/status/2063746333023043966&amp;image=https%3A//i.embed.ly/1/image%3Furl%3Dhttps%253A%252F%252Fpbs.twimg.com%252Fprofile_images%252F1924503772094517249%252FDfKkH0ph_200x200.jpg%26key%3Da19fcc184b9711e1b4764040d3dc5c07" width="500" height="281" frameborder="0" scrolling="no"><a href="https://medium.com/media/53f6168e22b3eb143efbfeaf93abfc88/href">https://medium.com/media/53f6168e22b3eb143efbfeaf93abfc88/href</a></iframe><iframe src="https://cdn.embedly.com/widgets/media.html?type=text%2Fhtml&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;schema=twitter&amp;url=https%3A//x.com/JacksonAtkinsX/status/2058621253250961912&amp;image=https%3A//i.embed.ly/1/image%3Furl%3Dhttps%253A%252F%252Fpbs.twimg.com%252Fmedia%252FHJGwqR7XwAA5xbF.png%253Alarge%26key%3Da19fcc184b9711e1b4764040d3dc5c07" width="500" height="281" frameborder="0" scrolling="no"><a href="https://medium.com/media/04bee3156410650e7536a9fc0fd0b2a0/href">https://medium.com/media/04bee3156410650e7536a9fc0fd0b2a0/href</a></iframe><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=cf400328ae31" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Chapter 1 — Barista in the Loop]]></title>
            <link>https://medium.com/@haricane8133/chapter-1-barista-in-the-loop-2a640c56b0c9?source=rss-e8f8cb2ed41f------2</link>
            <guid isPermaLink="false">https://medium.com/p/2a640c56b0c9</guid>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[software-development]]></category>
            <dc:creator><![CDATA[Hari Rajesh]]></dc:creator>
            <pubDate>Sat, 13 Jun 2026 08:20:24 GMT</pubDate>
            <atom:updated>2026-06-21T14:17:51.120Z</atom:updated>
            <content:encoded><![CDATA[<blockquote>A series on using AI wisely and sustainably in software development.</blockquote><ul><li><strong>Chapter 1 — The Barista, Intuition and Learning</strong></li><li><a href="https://medium.com/p/33e7656ad6ed">Chapter 2 — Human Brain and LLM</a></li><li><a href="https://medium.com/p/cf400328ae31">Chapter 3 — AI, Speed and the Sustainability of Software</a></li><li><a href="https://medium.com/p/140209a5f7cb">Chapter 4 — What counts as Innovation in the Age of AI?</a></li><li><a href="https://medium.com/p/d90341113c67">Chapter 5 — The AI Maturity Ladder</a></li><li><a href="https://medium.com/p/5420267d8331">Chapter 6 — The AI Productivity Trap</a></li></ul><h4>Preface</h4><p>Read comment.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*u39KKGTpvPbq3Fze.jpeg" /></figure><h3>Barista in the Loop</h3><p>If you want someone to become an expert barista, you could get them access to the newest voice activated machine to which they just have to mention what coffee they want. Different baristas have different ways of making the same coffee, and the machine has been pre-programmed with this knowledge, and a workflow on how to use this knowledge to make any coffee the basista wants made. The junior basista is extremely happy. If a customer comes up with a complicated requirement, she doesn’t break a sweat and instead just tells the machine. The machine after an intense planning and research phase, identifies what needs to be done, makes it internally once, tastes the coffee, compares it with the initial requirement, immediately understands what is missing, chucks the coffee it just made, and starts the process again. It keeps iterating, but in the end it manages to make the coffee that the customer wants. The customer and the barista is happy. But somewhere down the line, two things started to happen…</p><ul><li>The coffee shop owner gets happy thinking that with the machine, basistas could finally skip the boring parts of their jobs — making coffee (?).</li><li>The passionate Junior barista realizes that her job is now no longer about coffee.</li></ul><p><strong>What is missing now?</strong> True learning for a barista doesn’t happen just with learning new recipies, or asking something or someone else to make it. It comes with attempting to make some new coffee, failing at it, trying some work-arounds, innovating with ingredients and the process, iterating till finally succeeding and finally continuing till all of this becomes second nature. Experience gained through these steps is immutable, and cannot be gained any other way. This rigorous process makes a junior barista, a senior barista. A junior from the run of the mill coffe shop, would have gone to the senior and asked her how to make a custom drink the customer wanted. Good seniors might not just give the answer right away all the time, but make the junior basistas understand and make it themselves.</p><p><strong>The most experienced barista</strong> would be able to simply look at a coffee bean bag, a confused customer, their order, the time of the day, the state of the shop, how hot the machines are running, and make the best coffee the customer wants (rather needs or deserves). When you ask them the thought process behind each decision they took, they may not be able to tell you everything right away. Experience gives them so much that most of the process becomes sub-consious. They start listening to <strong><em>intuitions</em></strong> and <strong><em>Gut feeling</em></strong> which today’s machines lack (read the next section dedicated to these).</p><p><strong>Running a coffee shop without machines</strong>, is not practical. Most customers will not wait a long time, or do not care about a costly traditional customized human-made A-Z amazing coffee.</p><p><strong>Some questions</strong></p><ul><li>What would happen after 10–15 years when the current set of senior baristas retire? The current basistas who have all their learnings on using the machine alone, will not be ready to be seniors.</li><li>What would happen when all the machines stop working one day? Customers still want coffee.</li></ul><p><a href="https://www.anthropic.com/news/fable-mythos-access">Statement on the directive to suspend Fable 5 access</a></p><h3>Learning, Understanding and Growth</h3><p>Using AI extensively, leads to developers pushing to production, code that they do not fully understand. Tight deadlines and rewarding completion speed, force developers to not pause and learn.</p><h4>“What I cannot understand, I cannot create”</h4><p>Creating something while not understanding it, isn’t recommended by <strong>Richard Feynman</strong>, one of the world’s most celebrated physicists, teachers and thinkers. Let us change our motto to “What I cannot understand, I <em>will not</em> create“</p><h4>“Abstraction levels”</h4><p>No one understands anything fully, and everyone’s understanding is at some abstraction level or on a frame of reference at which they are satisfied. For example, you might say that a painter doesn’t know what he is doing unless he understands how photons falling on his retina, gets converted into a painting inside the brain. I will pause you there. Watch <strong>Feynman</strong> explain this in <a href="https://www.youtube.com/watch?v=Q1lL-hXO27Q"><strong>the Fun To Imagine series</strong></a>. But for now, let us be practical and move on.</p><p><strong>Tools and Abstraction levels:</strong> As tools get more and more complex, the tool users’ knowledge rises up abstraction levels and they cease to have more foundational knowledge, unless they made an outright effort. A barista shouldn’t just be happy using over-powered machines.</p><p>So <strong>what level of tools</strong> should a coffee company get for their baristas? That depends on the company and its visions.</p><ol><li>The company can decide that in 5 years they want to just have walk in kiosks with just these machines which customers directly use.</li><li>The company can decide that they will have no machines, and only baristas, a cow and a nearby coffee plantation.</li><li><strong>The middle ground:</strong> The company can decide a healthy abstraction level for their baristas so that in 5 years, we will have a healthy blend of junior basistas, senior baristas and machines that can help them. <em>Not “Barista in the loop”, but rather “machines in the barista loop”.</em></li></ol><h4>“If you’re not having fun, you’re not learning. There’s a pleasure in finding things out.”</h4><p>If you are working in a coffee shop with machines, find out what you want</p><ul><li>is it becoming a master of coffee making?</li><li>is it becoming a master of machine maintenance?</li></ul><p>If it is <strong>the middle ground</strong>, set your abstraction level and use machines to understand both machines and coffee. Try making coffee without machines (maybe not for the customer) at regular intervals.</p><p><strong>As a barista, </strong>setting this abstraction level is the <strong>most important decision</strong> you will take. Then, ensure you have learning and fun along the path you choose for yourself.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2a640c56b0c9" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[AI — two truths and one lie]]></title>
            <link>https://medium.com/@haricane8133/ai-two-truths-and-one-lie-392326733249?source=rss-e8f8cb2ed41f------2</link>
            <guid isPermaLink="false">https://medium.com/p/392326733249</guid>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[software-engineering]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[agi]]></category>
            <category><![CDATA[claude-code]]></category>
            <dc:creator><![CDATA[Hari Rajesh]]></dc:creator>
            <pubDate>Sat, 30 May 2026 08:53:39 GMT</pubDate>
            <atom:updated>2026-06-27T15:39:48.410Z</atom:updated>
            <content:encoded><![CDATA[<h3>AI — two truths and one lie</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/900/0*AKAXi1aI2KU_V3ly" /></figure><p>No AI was used to generate any part of this series, inspired by real world happenings, contains deep and (possibly) opinionated thoughts critical of how our world is dealing with AI coding. This comes from someone who can claim to have ridden a tiny domestic house cat version of the full agentic coding (“hype”?) cycle. I went all the way from refusing to use ChatGPT till after a few months it released, to spending weeks tuning my AI agent (my?), to waking up daily grinning at my entire SDLC already complete by my bedside, to realizing that it still… just… wasn’t… enough.</p><p>Throughout this article, I would be linking others. For best results, go through them too. Together these would take up a lot of your time ; But I think that it would be useful.</p><p>Clear emphasis on ‘my thoughts’, because we all can be wrong (including me). Our industry is at an inflection point, and everyone does what they think is right, or follow who they think is right. As one of my friends put it, “NOW is NOT the time to feed into dogma!”. What will help is an open conversation without fear. So as you read through this, I implore you to put yourself in ‘planning mode’, and spend adequate time……., just…………, thinking………………</p><p>This post isn’t for ‘Karma Farming’, but to bring about an awareness that, only critical thinking will help navigate said inflection point. The article will contain links to videos and articles of some folks who might not be politically correct ; let us take what we have to learn from them, and discard the rest (¯\_(ツ)_/¯)</p><blockquote>A series on using AI wisely and sustainably in software development.</blockquote><ul><li><a href="https://medium.com/p/2a640c56b0c9"><strong>Chapter 1 — The Barista, Intuition and Learning</strong></a> — Sets the stage with an analogy of a coffee shop with a dilemma on growing future baristas or using complicated automated machines</li><li><a href="https://medium.com/p/33e7656ad6ed"><strong>Chapter 2 — Human Brain and LLM</strong></a> — Compares LLMs and Human brains and tell you why you shouldn’t try to replace one with another completely</li><li><a href="https://medium.com/p/cf400328ae31"><strong>Chapter 3 — AI, Speed and the Sustainability of Software</strong></a> — Lists down the two major streams which people recommend to use AI in software development, specifies why both won’t work out on their own, and answers how one can safely find the middle ground</li><li><a href="https://medium.com/p/140209a5f7cb"><strong>Chapter 4 — What counts as Innovation in the Age of AI?</strong></a> — Everyone is pressurized to build and ship something. Some build hackathon projects with AI and proudly ship it. Some try to build alternatives to tools built by billion dollar companies. AI allows one to write software. But where does one innovate, and where do innovations actually matter</li><li><a href="https://medium.com/p/d90341113c67"><strong>Chapter 5 — The AI Maturity Ladder</strong></a> — We all want everyone to improve their AI / agentic software development skills. But that doesn’t mean just using some X / Y / Z tool, when there are millions of these tools and skills. We need a sustained way in which one can improve themselves in AI and software development. We will explore a way how that can be done.</li><li><a href="https://medium.com/p/5420267d8331"><strong>Chapter 6 — The AI Productivity Trap</strong></a> — This section is all about companies trying to increase AI adoption with various methods. They might build you a tool that converts Jira tickets to PRs, build dashboards showing token usage, etc… But an important point missed, is Accountability of code. We will address this and the chase on wrong metrics.</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=392326733249" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Node’s Module Resolution and the Perils of multiple node_modules with WebPack]]></title>
            <link>https://medium.com/@haricane8133/nodes-module-resolution-and-the-perils-of-multiple-node-modules-with-webpack-61b507e5602e?source=rss-e8f8cb2ed41f------2</link>
            <guid isPermaLink="false">https://medium.com/p/61b507e5602e</guid>
            <dc:creator><![CDATA[Hari Rajesh]]></dc:creator>
            <pubDate>Wed, 03 Dec 2025 13:18:41 GMT</pubDate>
            <atom:updated>2026-02-04T06:55:28.751Z</atom:updated>
            <content:encoded><![CDATA[<p>Before we get into the following interesting topics:</p><ol><li>How a large monorepo survived for years through not one, but two miracles.</li><li>What mistakes were made unknowingly for years.</li><li>How a single configuration in WebPack was a primary root cause of many issues.</li><li>How we were able to identify and fix the issues in our journey to remove the need for complex regressions and make SDLC simpler.</li></ol><p>… we will first go over some theory.</p><h3>Theory</h3><h4>NPM basics</h4><p>Read my other article “<a href="https://medium.com/@hari.r.nallan/some-secrets-of-package-json-and-package-lock-json-19311fe3c947">Some secrets of package.json and package-lock.json</a>” to learn about package.json, package-lock.json, node_modules, some basic npm commands and what happens when we constantly use npm ci all the time instead of npm install.</p><h4>Direct Dependencies, Transitive Dependencies</h4><p>All packages our code directly depend on (or import), are called direct dependencies and dependencies of our dependencies, or dependencies of our dependencies of our dependencies (and so on…) are called transitive dependencies.</p><h4>The Dependency Tree</h4><p>When we map out all these dependencies, imports and requirements as a diagram, we get what is called as the dependency tree of our project (named so as it actually is a tree). Actually the analogy can be extended further - at the very beginning the tree can be shaped or modified easily. But as time progresses and the tree grows, it would become difficult to re-shape or modify it.</p><p>The way npm and node is setup, each dependency brings it’s own dependency within its own node_modules folder. Do your project will have a node_modules directory in its root. Within it, we have a folder each for each of the direct dependencies we have marked in our package.json and consuming in our code. Each of these folders is a NPM package which has its own dependencies. NPM puts these dependencies in a node_modules folder within the dependencies. This pattern continues recursively in a nested folder structure.</p><p>So there can be multiple versions of a same package being used in different parts of the dependency tree and there would be no problem because each package or JS project is setup to get the exact version of each dependency that it wants.</p><p>Here is an example. Let this represent the dependency tree of our project.</p><pre>                                   Project<br>                                      |<br>                --------------------------------------------<br>                |              |              |            |<br>             package-a      package-b      package-y    package-z<br>             (1.0.0)         (1.0.0)        (1.0.0)      (2.0.0)<br>                |               |<br>                |               ----------------------------<br> --------------------------------              |           |<br> |              |               |              |           |<br>package-x     package-y      package-z      package-x   package-c<br>(2.0.0)        (2.0.0)        (2.0.0)       (2.0.0)      (2.0.0)</pre><p>After we install this dependency tree, through NPM commands, here is how our project’s folder structure would look like (Yes. It is the same tree)</p><pre><br>.<br>├── project_folder<br>│   ├── node_modules<br>│   │   └── package-a (@1.0.0)<br>│   │       └── node_modules<br>│   │           └── package-x (@2.0.0)<br>│   │               └── node_modules<br>│   │                   ...<br>│   │           └── package-y (@2.0.0)<br>│   │               └── node_modules<br>│   │                   ...<br>│   │           └── package-z (@2.0.0)<br>│   │               └── node_modules<br>│   │                   ...<br>│   │       └── index.js<br>│   │       └── package.json<br>│   │       └── README.md<br>│   │<br>│   │   └── package-b (@1.0.0)<br>│   │       └── node_modules<br>│   │           └── package-x (@2.0.0)<br>│   │               └── node_modules<br>│   │                   ...<br>│   │           └── package-c (@2.0.0)<br>│   │               └── node_modules<br>│   │                   ...<br>│   │       └── index.js<br>│   │       └── package.json<br>│   │       └── README.md<br>│   │<br>│   │   └── package-y (@1.0.0)<br>│   │       └── node_modules<br>│   │           ...<br>│   │       └── index.js<br>│   │       └── package.json<br>│   │       └── README.md<br>│   │<br>│   │   └── package-z (@2.0.0)<br>│   │       └── node_modules<br>│   │           ...<br>│   │       └── index.js<br>│   │       └── package.json<br>│   │       └── README.md<br>│   │<br>│   │<br>│   ├── index.js<br>│   ├── package.json<br>│   └── README.md</pre><p>This type of dependency management is actually elegant because it facilitates two useful properties</p><ol><li>Each code gets the exact dependency it wants and there is no constraint like in Java or Python where we can have only one version of a said dependency in a project (JAR Hell, version conflicts)</li><li>The project folder is self-containing as it has all the code it needs within it. We only need the node interpreter to execute the main file we have. But sometimes to keep a folder self-containing, we may need some additional work. We will have a look at it somewhere below. Keep following this symbol !*!.</li></ol><h4>Node’s Module Resolution policy</h4><p>All JS projects start execution with the main file, usually called as index.js or main.js, which we specify in our project’s package.json. Node works on the imports in that file before anything else.</p><p>When a particular package is requested as an import in a file, node starts with the same directory as the file and</p><ol><li>Looks for the presence of the folder node_modules.</li><li>If present, a folder with the same name as the imported package is searched (node_modules/&lt;package_name&gt;)</li><li>In case Node is unable to find this dependency, it traverses up the file tree till it finds it. (../node_modules/&lt;package_name&gt; or ../../node_modules/&lt;package_name&gt;). If the required package is still not found, then an error is thrown.</li><li>If the package is found, the package.json of that package is read to understand that package - namely the exports and <em>its</em> dependencies. (node_modules/&lt;package_name&gt;/package.json). If there are imports in these dependencies, it starts the same process again (point 1), but this time from this current folder.</li></ol><p>Therefore Node’s module resolution policy, is based on the folder structure, which is based on the dependency tree of the project. This is how each file is processed and executed with all the dependencies they require.</p><h4>Dependency Hoisting / Flattening dependency tree</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/640/1*4NdmzK_nO7hFFREapwddZg.png" /></figure><p>We’ve all seen this before. A JS project folder being self-contained, unfortunately means that the project’s total size becomes high. Fortunately, as we saw with the article, with the help of package-lock.json, which is an exact mirror of node_modules, we can deterministically re-create the node_modules directory anywhere (unless NPM, artifactory or any artifact registry that we use is down or corrupted or hacked which we will assume won’t happen, till the end of this article). So we can use “ignore” files like .gitignore or .npmignore to atleast ensure that we have this folder created only wherever necesary/</p><p>In our small example, we can see that the following dependencies are repeated and take up space twice — package-x (2.0.0) and package-z (2.0.0). Given Node’s module resolution policy, and given that in a big project there definitely would be many such duplicate dependencies in multiple places , is there a way to save some space? Yes…</p><p>Version 3 of NPM started hoisting dependencies by flattening out the node_modules directory as much as possible. With this change in NPM, the folder structure started to look like this instead</p><pre><br>.<br>├── project_folder<br>│   ├── node_modules<br>│   │   └── package-a (@1.0.0)<br>│   │       └── node_modules<br>│   │           └── package-y (@2.0.0)<br>│   │               └── node_modules<br>│   │                   ...<br>│   │       └── index.js<br>│   │       └── package.json<br>│   │       └── README.md<br>│   │<br>│   │   └── package-b (@1.0.0)<br>│   │       └── node_modules<br>│   │           └── package-c (@2.0.0)<br>│   │               └── node_modules<br>│   │                   ...<br>│   │       └── index.js<br>│   │       └── package.json<br>│   │       └── README.md<br>│   │<br>│   │   └── package-y (@1.0.0)<br>│   │       └── node_modules<br>│   │           ...<br>│   │       └── index.js<br>│   │       └── package.json<br>│   │       └── README.md<br>│   │<br>│   │   └── package-z (@2.0.0)<br>│   │       └── node_modules<br>│   │           ...<br>│   │       └── index.js<br>│   │       └── package.json<br>│   │       └── README.md<br>│   │<br>│   │   └── package-x (@2.0.0)<br>│   │       └── node_modules<br>│   │           ...<br>│   │       └── index.js<br>│   │       └── package.json<br>│   │       └── README.md<br>│   │<br>│   │<br>│   ├── index.js<br>│   ├── package.json<br>│   └── README.md</pre><p>What happened here is that the two duplicated dependencies we talked about earlier, aren’t duplicated anymore, and they are flattened out to the root of our project’s main node_modules. Even though now some dependencies aren’t present in their respective parent’s node_modules, Node’s module resolution policy ensures that still each package gets the exact version of dependency that they want.</p><p>For instance, if package-a is looking for package-x, node first notices that there is no package-x in /node_modules/package-a/node_modules/and goes up a directory to find it at /node_modules/package-x.</p><p>But on the other hand, if package-a is looking for package-y, the resolution happens from within /node_modules/package-a/node_modules/package-y itself instead of fetching from the project root node_modules. Thus the version required by each package is still respected whereas we have saved some space within the node_modules now.</p><p>But there are side effects with flattening -</p><ol><li>The project’s node_modules no longer mirrors the dependency tree</li><li>The structure of node_modules is no longer deterministic (atleast to us). It is the list of direct and transitive dependencies of the project that npm uses to determine how to flatten, or which versions of which dependencies to flatten out to the root.</li><li>But more importantly, we now have possibilities of Phantom / Ghost Dependencies.</li></ol><h4>Phantom Dependencies</h4><p>Let us focus our attention to the list of dependencies that are now in the root of our example project’s node_modules directory.</p><p>When the node_modules wasn’t flattened, we had the following</p><ol><li>package-a</li><li>package-b</li><li>package-y</li><li>package-z</li></ol><p>After flattening, we have these</p><ol><li>package-a</li><li>package-b</li><li>package-y</li><li>package-z</li><li>package-x</li></ol><p>The difference is that now we have package-x extra in the root of our node_modules. Though our code doesn’t require or use package-x, what would happen if we added for argument’s sake an import of package-x in our code, without installing it in the first place?</p><p>In the nested node_modules world, as expected, Node would throw an error. But in the flattened node_modules world….., due to how Node’s module resolution policy works….. the import would work…. just….. fine….. But since the prescence of package-x in the root of the project node_modules is not deterministic, when some developer installs or uninstalls some package, there is a chance that package-x will no longer be in the root, and our build will break.</p><p>So phantom depependencies are imports in code that aren’t specified in the project’s node_modules and just…. work….</p><h4>Peer dependencies and phantom dependencies</h4><p>There are some packages that have peer dependencies (dependencies that it expects the host application/library/JS project consuming it) to have as a part of its direct dependencies.</p><p>Let us extend our example here. Let us say that we install a new dependency called package-f.</p><ol><li>If package-f has a peer dependency on package-a, then no problems as our application has a direct dependency on package-a already.</li><li>If package-f has a peer dependency on package-g which is a package no where found before in our project’s entire dependency tree, then no problems as the npm command used to install the new package would warn the user to install the missing peer dependency</li><li>If package-f has a peer dependency on package-x, there is an extra scenario we need to look at. If our Node project has proper nested node_modules structure, then npm command would warn us just like the previous point. But if we have a flattened node_modules structure, then the install would happen without any issues as package-x, a phantom peer dependency is available in the root of the node_modules already, even though it is not marked as a direct dependency in our project’s package.json</li></ol><h4>The flak</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/927/1*PIaf8RIJF31s3BSu873LEw.png" /><figcaption><a href="https://github.com/npm/npm/issues/9809#issuecomment-151376330">Force npm 3 to install a node module always nested · Issue #9809 · npm/npm</a></figcaption></figure><p>Did flattening node_modules really save a ton of space? Not really. The black-hole meme is still as popular as it was before. But on the other hand, phantom dependencies started to grow in large projects where developers just assume some popular dependencies to be installed already and start using it in code.</p><p>Newer package managers like PNPM strongly comdemn this move and they resorted to other clever tricks to save space, instead of tampering with the tree. Read more here — <a href="https://pnpm.io/pnpm-vs-npm#npms-flat-tree">pnpm vs npm | pnpm</a></p><p>In fact, PNPM has these two configurations to enable flattening. Note how they named these… :)</p><ul><li><a href="https://pnpm.io/settings#shamefullyhoist">— shamefully-hoist</a></li><li><a href="https://pnpm.io/9.x/npmrc#public-hoist-pattern">public-hoist-pattern[]=*XYZ*</a></li></ul><p>In a little while, we will see soon how flattening dependencies by NPM causes issues in our code.</p><h4>monorepo and node_modules</h4><p>Monorepo often gets heat from many, often being associated with SDLC bottlenecks. Most of the MonoRepo’ problems actually arise from other sources, like how modules are maintained and resolved.</p><p>In monorepos, we often have multiple node_modules directories, often at various levels in the file system. So when a code requests a package, the module resolution can happen from the nearest node_modules, or some other node_modules up the directory tree but within the project.</p><p>Here are the problems</p><ol><li>!*! The self-containing ability of each sub-project (folder) is at jeopardy, as we now have dependencies of a JS project, outside it’s folder itself.</li><li>More importantly, this encourages another form of phantom dependencies — we can have dependencies in the project’s root node_modules, but not in the sub-project’s node_modules, and the sub-project would still work fine even though we have not explicitly marked the dependency in the sub-project’s package.json. So in the future, if there is a change in the root node_modules, it can affect the sub-projects too.</li></ol><p>Further reading — <a href="https://www.jonathancreamer.com/inside-the-pain-of-monorepos-and-hoisting/">Inside the pain of monorepos and hoisting</a></p><h4>Drawing parallels with WebPack</h4><p>WebPack has an amazingly jam-packed set of features for frontend web development. When it comes to resolving dependencies, simply put, it does at build time what Node does at runtime.</p><p>When building, WebPack also starts with the initial start file, say index.js, and gradually works it’s way dependency through dependency till it forms the entire dependency tree for the project. Based on the configured settings, it would then condense this entire tree into a single or multiple JS static file(s). This static file is now the equivalent of the self-contained project folder in node.</p><p>WebPack by default also follows the same Module Resolution Policy that node follows — start with the current folder’s node_modules and recursively search the parent folders. This makes sense because we use the same package management tool for both server and frontend - npm, and npm has similar logic for how the node_modules folder is to be structured given the list of direct and transitive dependencies.</p><p>WebPack performs one additional step before it condenses the dependency tree into static files — Tree Shaking.</p><h4>WebPack’s Tree Shaking</h4><p>is the missing link in the NodeJS world<br>and a better alternative to “saving node_modules space“</p><p>We have already seen that the dependency tree has a lot of duplicated and repeated dependencies. WebPack deals with files that are sent to the user’s browser. So when the condensing/compiling of the dependency tree to a few static files happen, WebPack performs tree shaking and adds only once each version of each dependency. If the module type of the dependencies is ESM, then WebPack takes this a step further and imports and adds to the final bundle, only those functions or code is required or imported. The rest is ignored. All internal references are taken care of. After this process, the node_modules folder is not required as all requirements are already added to the bundle.</p><p>The same is not possible/needed in the NodeJs world because we do not have to ship the node_modules directory to the user. It just remains in the server or in the Docker image.</p><h4>WebPack’s Modules array</h4><p>If there ever was a feature that can be called “A wolf in sheep’s clothing”, it is this one from WebPack — <a href="https://webpack.js.org/configuration/resolve/#resolvemodules">resolve.modules</a>. It looks quite safe to use — we use it to add more node_modules from where a project gets its dependencies from. The docs make this clear; I only wish it came with more explamation marks or disclaimers. When ABSOLUTE PATHS are used in this field, then WebPack would that folder and THAT FOLDER ALONE for dependencies without following Node’s official module resolution algorithm.</p><p>(assuming that this property is filled only with absolute paths) Since Node’s module resolution is not followed, even transitive dependencies (a nested dependency wants a dependency) are searched for only in the root of the folders we have passed. So every transitive dependency ever required by our application must be present in the root of the node_modules, flattened out</p><p>THIS MEANS THAT THE WHOLE APPLICATION IS FORCED TO USE ONLY THAT VERSION OF A PARTICULAR DEPENDENCY THAT IS AVAILABLE IN THE ROOT OF THE NODE_MODULES</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/32/0*RMKvhBs4FtD-NCr4.png" /></figure><h3>An example from my past</h3><p>Let’s consider a typical monorepo structure:</p><pre>.<br>├── my-monorepo<br>│   ├── client<br>│   │   └── shared-lib<br>│   │       └── ... just shared code ...<br>│   │       └── ... no node_modules ...<br>│   │       └── ... no package.json ...<br>│   │   └── app1<br>│   │       └── node_modules<br>│   │       └── src<br>│   │       └── webpack.config.js<br>│   │       └── package.json<br>│   │       └── README.md<br>│   │   └── app2<br>│   │       └── node_modules<br>│   │       └── src<br>│   │       └── webpack.config.js<br>│   │       └── package.json<br>│   │       └── README.md<br>│   │   └── ... (other apps)<br>│   ├── server<br>│   │   └── ... shared server code ...<br>│   │   └── ... no node_modules ...<br>│   │   └── ... no package.json ...<br>│   ├── node_modules (root node_modules)<br>│   ├── index.js<br>│   ├── package.json<br>│   └── README.md</pre><ul><li>The project has a package.json and node_modules in the root.</li><li>There are multiple React applications inside the /client folder, each with their own package.json and node_modules.</li><li>A /client/shared-lib folder houses code shared by all React projects.</li><li>The /server folder hosts NodeJS server code, but has no package.json or node_modules.</li></ul><h3>Module Resolution in the Monorepo</h3><h4>Node’s Module Resolution</h4><ul><li>Any dependency of the /server folder is resolved from the root node_modules.</li><li>Any dependency required in one of the React applications in /client is resolved from their own node_modules or the root node_modules, in that order.</li><li>Any dependency of the /client/shared-lib folder is resolved from the root node_modules.</li></ul><p>This already has a few problems, right? None of the sub-projects here are self-contained. To make matters worse, the WebPack configs inside the React applications were set up with absolute paths in resolve.modules! This means dependencies are resolved only from specified folders, bypassing Node’s official module resolution algorithm.</p><h4>The Problems</h4><ol><li>No JS package within the repository is self-contained.</li><li>Multiple node_modules directories lead to phantom dependencies everywhere.</li><li>The whole project is forced, for every dependency import, to use versions available in the root node_modules.</li><li>At build time, for the same code, two different versions of the same package could be resolved, leading to subtle and hard</li></ol><h4>Adding Fuel to the Fire: When Small Issues Compound</h4><p>It’s one thing to have a single architectural quirk in your codebase. But when several of them combine, the result isn’t just additive — it’s exponential. In our monorepo, three seemingly independent decisions worked together to create a perfect storm:</p><ol><li><strong>Monorepo with Nested </strong>node_modules and Shared Code<br>Our monorepo had multiple sub-projects, each with its own node_modules, but also a shared folder with no package.json or node_modules of its own. This meant shared code could “see” dependencies from the root or sibling projects, even if they weren’t declared locally. The root node_modules, originally meant for backend/server code, started accumulating frontend dependencies as well.</li><li><strong>WebPack’s </strong>resolve.modules with Absolute Paths<br>To “simplify” module resolution, we configured WebPack to look in both the sub-project’s and the root’s node_modules—using absolute paths. This broke Node’s standard resolution algorithm, forcing all dependencies (even transitive ones) to be present at the root, and making it impossible for sub-projects to use different versions of the same dependency.</li><li><strong>npm’s Automatic Dependency Flattening</strong><br>npm, trying to be helpful, hoisted shared dependencies up to the root node_modules. This meant dependencies could appear or disappear from the root based on seemingly unrelated changes elsewhere in the dependency tree.</li></ol><h4>The Problems: What Goes Wrong</h4><p><strong>1. No Self-Contained Packages</strong></p><p>None of the sub-projects are truly self-contained. They rely on dependencies being present elsewhere in the repo, making it hard to reason about what each project actually needs.</p><p><strong>2. Phantom Dependencies Everywhere</strong></p><p>Because dependencies can be resolved from multiple places, code can import packages that aren’t declared in its own package.json. This works—until someone removes or upgrades a dependency elsewhere, causing mysterious breakages.</p><p><strong>3. Forced Version Uniformity</strong></p><p>All code is forced to use the version of a dependency that happens to be at the root, even if that’s not the version it was developed or tested against. This can lead to subtle bugs and incompatibilities.</p><p><strong>4. Simultaneous Multiple Versions</strong></p><p>In some cases, the same build can end up using two different versions of the same package, depending on which node_modules directory WebPack finds first. This is a recipe for unpredictable behavior and hard-to-debug issues.</p><h4>Real-World Example (Generified)</h4><p>Imagine you have two frontend apps, app1 and app2, and a shared library, all in the same monorepo. Both apps depend on a package called core-utils, but at different versions. The shared library doesn’t declare core-utils as a dependency, but uses it anyway.</p><ul><li>app1/node_modules/core-utils@3</li><li>app2/node_modules/core-utils@2</li><li>shared-lib (no node_modules, no package.json)</li></ul><p>WebPack is configured to resolve modules from both the app’s and the root’s node_modules. npm has hoisted some dependencies to the root.</p><p>When building app1, if core-utils@3 is missing a submodule, WebPack falls back to the root, where it finds core-utils@2. Now, app1 is using both versions in the same build, depending on which submodule is imported.</p><h4>The Miracles</h4><p><strong>Miracle 1:</strong><br>All requests for a particular dependency resolved to a single version, which happened to be compatible across the codebase. This was pure luck — if any sub-project had needed a different version, or if the root version changed incompatibly, the whole system could have broken down.</p><p><strong>Miracle 2:</strong><br>When two versions of the same dependency were present, the build could end up using both versions at once—sometimes even within the same file! Miraculously, these versions were compatible enough that the application didn’t crash or misbehave in obvious ways.</p><p><strong>Why is this dangerous?</strong></p><ul><li>The build process became non-deterministic: the same code could resolve to different versions of a dependency depending on the order of resolution or the structure of node_modules.</li><li>Developers started relying on hacks, like adding aliases or manually copying dependencies, to “fix” issues that were actually symptoms of the underlying problem.</li><li>Over time, the codebase became dependent on this broken resolution logic. When we finally tried to fix module resolution, we discovered that some code had come to rely on the quirks of the old system — fixing things broke other things.</li></ul><h3>Our Fixes</h3><h4>Wave 1</h4><p><strong>Target:</strong></p><ul><li>Move the shared folder out as a package that can be versioned and consumed by the different React applications.</li><li>Remove the root node_modules from the resolve.modules list in WebPack, so only the app’s own node_modules is used for module resolution.</li></ul><p><strong>Fixes:</strong></p><ul><li>Study which packages need to be moved from the root package.json into each app’s package.json.</li><li>Still, with absolute paths in resolve.modules, packages resolve only from the root of the flattened node_modules of the app.</li><li>Node’s module resolution is still not fully followed, but we are one node_modules short now.</li><li>Sometimes, two versions of the same package had to be added with different names using npm aliases.</li></ul><h4>Wave 2</h4><p><strong>Target:</strong></p><ul><li>Remove the resolve.modules field completely so that Node’s module resolution policy is followed by WebPack.</li><li>Ensure that no module resolution ever happens outside the app’s folder.</li></ul><p><strong>Fixes:</strong></p><ul><li>Write a WebPack plugin to log every module resolution and fail the build if any import is resolved outside the app folder.</li><li>Iterate with npm install until all dependencies are available and the build is clean.</li><li>The plugin remains to ensure that any dependency pulled in for the WebPack build is only within the same folder.</li></ul><h4>The Ramp / The Challenge</h4><ul><li>Changing the module resolution policy is risky; a gradual rollout is needed.</li><li>Build scripts and pipelines were adjusted to generate two bundles (old and new), and users were ramped up to the new system gradually.</li><li>Developers had to make changes in both configurations until the migration was complete.</li></ul><h4>The Aftermath</h4><ul><li>No major production issues occurred during the ramp.</li><li>However, some code had become dependent on the old, incorrect module resolution, and needed to be fixed once the correct resolution was enforced.</li></ul><h4>Wave 3?</h4><ul><li>The only remaining problem is that npm still flattens dependencies, causing phantom dependencies.</li><li>Moving to newer package managers like PNPM could help, as they use different strategies to avoid these issues.</li></ul><h3>The End</h3><p>This adventure contributed a great deal:</p><ul><li>We were able to eliminate unnecessary regressions.</li><li>We reduced SDLC time to the lowest possible — builds in 5 minutes.</li></ul><h4>Key Takeaways</h4><ul><li><strong>Node’s module resolution is critical.</strong> Always ensure that each sub-project or app is self-contained, with its own node_modules and package.json.</li><li>Avoid flattening or hoisting dependencies in ways that break the deterministic resolution of modules.</li><li>In monorepos, be especially careful with shared folders and root-level dependencies.</li><li>WebPack’s resolve.modules can be powerful, but using absolute paths can break expected behavior.</li></ul><p>Consider using tools and plugins to enforce correct module resolution and catch phantom dependencies early.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=61b507e5602e" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How a change in asset URL redefines SDLC]]></title>
            <link>https://medium.com/@haricane8133/how-a-change-in-asset-url-redefines-sdlc-2e41c80a5682?source=rss-e8f8cb2ed41f------2</link>
            <guid isPermaLink="false">https://medium.com/p/2e41c80a5682</guid>
            <dc:creator><![CDATA[Hari Rajesh]]></dc:creator>
            <pubDate>Mon, 01 Dec 2025 07:42:23 GMT</pubDate>
            <atom:updated>2026-02-13T04:39:33.263Z</atom:updated>
            <content:encoded><![CDATA[<h3>How to redefine SDLC using a different “unique key”</h3><p>Disclaimer: I used AI to generify this document, which I originally wrote (without AI) on September 2025 with examples from my company’s internal repositories and dev tool offerings.</p><p>We will take a look at a couple of cases - (1) Asset URLs (2) caching steps and stages in a build pipeline. We will focus primarily on (1), but then at the end explain how the same logic can be used in (2) to make build pipelines so much more efficient.</p><h3>Asset URLs</h3><p>A trivial way to build and deploy a Node.js-backed <a href="https://jamstack.org/">JAM-Stack</a> web application is to generate a single docker image with the NodeJS and the UI dependencies installed and the UI built into a static /dist folder. But this doesn’t get you the full benefits of a JAM stack application.</p><p>A better way to deploy a JAM-stack application is to build the UI into a bunch of static assets (called a bundle) and deploy them to a Content Distribution Network (CDN) and related services, while the NodeJS server gets run, well, in a server or server-like service. The server which handles the routing, routes the user’s browser to fetch the assets from the CDN (a 302 redirect).</p><p>So, if there are any UI changes, they require a new CDN bundle, which is usually generated by one pipeline. The output of each ‘CDN’ build is <em>usually</em> a random string — which is used to construct the public URL of the asset bundle where the available in the CDN. Then, in the application, configuration JSON files must be updated with this new random string so that the server can construct the new URLs for the browser to fetch the new assets.</p><p>In many modern web applications, the process of building and deploying UI assets is multi-step and often involves generating bundles, publishing them to a CDN, and updating configuration files so that the server can construct the correct URLs for browsers to fetch the latest assets. This process, while functional, introduces complexity and redundancy — especially when it comes to keeping server and UI versions in sync.</p><h3>The Problem with Traditional SDLC</h3><p>The standard SDLC for UI assets typically requires two cycles:</p><ol><li><strong>Asset Generation and Publishing:</strong> Developers generate and publish new bundles to the CDN.</li><li><strong>Configuration Update:</strong> They then update configuration files (or similar metadata) to point to the new bundles, requiring another round of code changes, reviews, and deployments.</li></ol><p>This process is complicated by manual steps, tribal knowledge, and the need to keep server and UI versions in sync. Manual config updates are error-prone, and there is no deterministic way for the server to know which asset version to serve without consulting external configuration.</p><h3>Why Is This a Problem?</h3><ul><li><strong>Manual and Error-Prone:</strong> Updating configs is a manual process, prone to mistakes and omissions.</li><li><strong>Redundant Cycles:</strong> Every asset change requires two SDLC cycles — one for publishing the asset, another for updating the reference.</li><li><strong>Sync Issues:</strong> There’s always a risk of the server and UI being out of sync, especially in monorepos or multi-team environments.</li><li><strong>Tribal Knowledge:</strong> The process relies on unwritten rules and conventions, making onboarding and troubleshooting difficult.</li></ul><h3>Types of Asset URLs</h3><h3>Dynamic Links</h3><p>Dynamic links always point to the latest version of an asset. They are simple and never change, allowing clients to fetch the newest version without updating URLs. This enables rapid releases, as clients don’t need to update code to fetch new assets. However, this makes caching difficult and can cause issues with version mismatches or rollbacks, since it’s hard to guarantee which version is being served at any given time.</p><p>Example — https://www.mycdn.com/mybucket/mybundle/main.js</p><p><strong>Pros:</strong></p><ul><li>No need to update URLs for each release.</li><li>Enables quick, single-cycle releases.</li></ul><p><strong>Cons:</strong></p><ul><li>Caching is unreliable — clients may get stale or mismatched assets.</li><li>Rollbacks are hard to control.</li><li>No easy way to reference or debug a specific version.</li></ul><h3>Permanent Links (Perma-links)</h3><p>Permanent links embed version information (such as a hash) in the URL, making each asset version uniquely addressable. This enables reliable caching and easy rollbacks, as clients can always fetch a specific version. However, if the version information is generated randomly or managed externally, the server must consult a config file or database to construct the correct URL — leading to the need for a second SDLC cycle whenever assets change.</p><p>Example — https://www.mycdn.com/mybucket/mybundle&lt;random&gt;/main.js</p><p><strong>Pros:</strong></p><ul><li>Each asset version is uniquely addressable.</li><li>Enables robust caching and rollbacks.</li></ul><p><strong>Cons:</strong></p><ul><li>If version info is random or external, the server must look up the correct value.</li><li>Requires updating configs and a second SDLC cycle.</li></ul><h3>Deterministic vs. Non-Deterministic Links</h3><ul><li><strong>Deterministic links:</strong> The server can generate the asset URL algorithmically, based solely on the current state of the codebase, without consulting any external source.</li><li><strong>Non-deterministic links:</strong> The server must look up or persist some random or externally-generated value to construct the asset URL.</li></ul><p><strong>Current Industry Practice:</strong><br>Most setups use non-deterministic permanent links, where the server must read from a config file or database to know which asset version to serve. This is the root cause of the redundant SDLC cycle.</p><h3>Permanent Deterministic Link System</h3><p>We now know that we need Perma-links. We also know that having deterministic links (where the server intrinsically knows the correct current bundle without having to refer anything), would mean that we cut the SDLC cycles needed from one to two.</p><p>Git maintains different types of hashes internally for its tracking. Though commit hashes are the primary hash we are used to, we can get some of these internal hashes out of Git, with specific commands.</p><p>Inside Git, every file is called a blob and there is a hash for each blob (Blob ID or Blob Hash). A collection of blobs based on the folder structure is called a tree. Each tree has a hash (Tree Object ID or Tree Hash). The hashes returned by these are purely internal and not visible anywhere else. This Tree Object ID / hash would change only when the content inside that tree/folder changes. So a plain merge commit created on the repository won’t change that hash.</p><p>That is exactly what we are looking for. The ID for each bundle can now be entirely based on this new hash. So when a build is triggered, or when the application is running, our algorithm can generate the exact ID fits a discipline at a particular state. When other disciplines change, the ID for our original discipline doesn’t change. So we now have a system where we have an ID mapped to a particular state of the repository (in particular a particular state of the discipline UI code).</p><ul><li>The CDN build job that creates the bundle can generate this change hash based bundle ID</li><li>The NodeJS server running can generate the same to construct the URLs for asssets to be downloaded in user’s browser.</li></ul><h3>The Git Hashes</h3><h3>Tree OID</h3><p>These commands are used to get the Tree Object Id/Hash.<br>-&gt; This hash doesn’t include any metadata and changes only when the files change internally<br>-&gt; The commands can be used to get the Tree OID of any sub folder (any (sub)tree)</p><p>git rev-parse HEAD^{tree}</p><p>git rev-parse HEAD:</p><p>git rev-parse HEAD:&quot;myfolder/&quot;</p><p>-&gt; If you are looking for a command that returns changed hash when file changes are staged, use the following</p><p>git write-tree git write-tree --prefix=&quot;myfolder/&quot;</p><h3>The folder blame commit</h3><p>These commands are used to get the commit hash<br>-&gt; includes metadata like time of commit and author<br>-&gt; The commands can help you get the latest commit that touched a particular folder</p><p>git rev-parse HEAD</p><p>git log --no-pager -1 --format=&quot;%H&quot;</p><p>git log --no-pager -1 --format=&quot;%H&quot; -- client/sar</p><h3>Which Hash to use?</h3><p>Though a using a deterministic URL based on Tree OIDs would be ideal, cleaner and straight-forward (no meta-data involved in hash computation and thus determinism within determinism), a developer has no way to determine which <em>commit</em> it is related to.</p><p>GitHub UI shows only commits, not any other internal ID. It also shows the exact latest commit ID for each folder — it is shown for every folder. A merge commit or any metadata will not affect this GitBlame commit hash.</p><p>If we have your entire UI code inside a particular folder, we can use the Folder Blame Commit to create our deterministic URL for the assets generated from that folder. Then, a developer would be able to easily navigate and find the commit from which that bundle/URL was generated, making it helpful for debugging.</p><h3>Why This Eliminates the Second SDLC Cycle</h3><p>With deterministic asset URLs based on Git hashes:</p><ul><li>The server can always generate the correct asset URL based on the current code state, without consulting any external config or database.</li><li>There is no need to update configuration files or trigger a second deployment cycle just to update asset references.</li><li>This approach ensures perfect sync between server and UI, reduces manual steps, and eliminates ambiguity about which version is in production.</li></ul><p><strong>In summary:</strong></p><ul><li>No more manual config updates.</li><li>No more redundant SDLC cycles.</li><li>No more ambiguity about which asset version is being served.</li></ul><h3>Additional Benefits</h3><ul><li><strong>Idempotency:</strong> Each code state produces exactly one asset version and URL.</li><li><strong>Debuggability:</strong> Developers can trace asset versions directly to code changes.</li><li><strong>Simplicity:</strong> The process is straightforward — just run the main build pipeline with the branch as input. No code gets built more than once, and it doesn’t matter which branch the code is.</li><li><strong>Scalability:</strong> Works well in monorepos and multi-team environments, as each folder’s hash is independent.</li></ul><h3>Caching in a code build pipeline</h3><p>A build pipeline has a lot of stages and steps that run for each code change. A code travels from many branches until it lands on the final branch from where one deploys. When code moves from one branch to another, we invariably add merge commits that change the latest commit hash of the repository, but not the code itself.</p><p>Mostly pipelines use the branch name and the repository latest commit hash as a key to cache results. But this would mean that all the steps get repeated for all the branches that the code travels to.</p><p>Instead if this Git OID is used as a caching key, re-build happens only when the code itself has changed, potentially saving huge amounts of resources (compute and developer time). This compounds positively in the org!</p><h3>Summary</h3><p>By moving from non-deterministic, config-driven asset URLs to deterministic, Git-hash-based URLs, teams can:</p><ul><li>Reduce SDLC complexity and manual steps</li><li>Eliminate redundant deployment cycles</li><li>Improve reliability and traceability of asset versions</li><li>Enable straightforward, algorithmic URL generation for assets</li></ul><p>This approach is broadly applicable to any system where static assets are versioned and served via a CDN, and can significantly streamline the build and deployment process for UI applications.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2e41c80a5682" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>