<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by รωα૨ɳ Șħ. on Medium]]></title>
        <description><![CDATA[Stories by รωα૨ɳ Șħ. on Medium]]></description>
        <link>https://medium.com/@swarnshekhar007?source=rss-7b159942b81d------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*F4UPNC9Yu4IPsX5sAmQ-3g.gif</url>
            <title>Stories by รωα૨ɳ Șħ. on Medium</title>
            <link>https://medium.com/@swarnshekhar007?source=rss-7b159942b81d------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Wed, 18 Nov 2026 10:33:00 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@swarnshekhar007/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Black October: The Days the Internet Looked Very, Very Fragile]]></title>
            <link>https://medium.com/@swarnshekhar007/black-october-the-days-the-internet-looked-very-very-fragile-afc8cde6c1cb?source=rss-7b159942b81d------2</link>
            <guid isPermaLink="false">https://medium.com/p/afc8cde6c1cb</guid>
            <dc:creator><![CDATA[รωα૨ɳ Șħ.]]></dc:creator>
            <pubDate>Sun, 25 Oct 2026 10:59:05 GMT</pubDate>
            <atom:updated>2026-10-26T12:49:14.128Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/736/0*Dmw3OyspN6Fu8FhF.jpg" /></figure><p>On October 21, AI researcher Joshua Achiam <a href="https://x.com/jachiam0/status/2112968705240109558">posted on X</a> that the mood in cybersecurity reminded him of March 1, 2020. He added that something was happening that was not normal, and wondered whether internet lockdowns might follow within a couple of months.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*IHSvI5SokUMY4dPY1QHamw.png" /></figure><p>If you remember what that date turned out to be the start of, you know why that comparison made people sit up.</p><p>By then the internet had a name for the chaos: <strong>Black October.</strong></p><p>I read through news coverage, technical write-ups and expert posts so you don’t have to. Grab a chai. This one is long, a little scary, and, I hope, useful by the end.</p><h3>What even is “Black October”?</h3><p>Quick clarity first, because the name sounds like a movie.</p><p>Black October is <strong>not one attack</strong>. It’s the nickname security people gave to a <em>cluster</em> of serious, mostly unrelated incidents that landed within days of each other, roughly between October 19 and 22, 2026. Different bugs, different companies, different causes.</p><p>What made it feel apocalyptic wasn’t one explosion. It was the <strong>pace</strong>. One widely shared post on X described it as a new year-level incident almost every hour.</p><p>Let’s walk through them.</p><h3>Incident #1: PHP2Shell, the main villain</h3><p><strong>PHP</strong> is the programming language behind a huge part of the web. Estimates in coverage say it powers close to 70% of websites. Think WordPress sites, small business pages, old internal tools nobody remembers building.</p><p>On <strong>October 21</strong>, two critical vulnerabilities in PHP were disclosed. Nicknamed <strong>PHP2Shell</strong>, they could let an attacker take control of a website’s server without having any legitimate access to it. That means stolen data, hijacked servers, the works.</p><p>Wiz, the security firm involved, said these findings rank among the most severe ever found in PHP, and arguably in software history.</p><h3>And here’s the twist</h3><p>The bugs weren’t found by a legendary hacker in a hoodie. They were found with <strong>AI.</strong></p><p>Researchers at Wiz and an AI firm called Calif used advanced models from OpenAI and Google to discover them back in <strong>August</strong>. OpenAI said that, to its knowledge, frontier AI found the flaws before attackers did.</p><p>Then it got more striking. Dan Lorenc, the CEO of Chainguard, said in a LinkedIn post that his company independently found the same bugs by typing a simple request into a frontier AI model, asking it to find bad vulnerabilities in PHP.</p><p>Further reading on PHP2Shell: <a href="https://www.cnbc.com/2026/10/23/php-vulnerabilities-ai-cybersecurity.html">CNBC</a> and <a href="https://techcrunch.com/2026/10/21/the-internet-is-having-a-very-bad-cybersecurity-day/">TechCrunch</a>.</p><h3>The countdown nobody wanted to watch</h3><p>Cloudflare shared how exploitation attempts grew across its network:</p><p>When What Cloudflare saw 10 hours <em>before</em> the public announcement Thousands of attempts About 45 minutes after the announcement Tens of thousands Within 2 hours Hundreds of thousands By the time the post went out <strong>Millions</strong></p><p>These figures come from a post attributed to Cloudflare’s CEO, as relayed in news coverage. Cloudflare also <a href="https://ppc.land/cloudflare-blocks-php2shell-exploits-9-hours-before-php-flaws-went-public/">rolled out protections about nine hours before public disclosure</a>, saying the flaw had leaked early.</p><p>Notice the first row. Attempts were already happening <em>before</em> the announcement. When the news broke, automated scanning across the internet exploded, with attackers checking who hadn’t patched.</p><p>The US <strong>NSA</strong> advised organisations using PHP to update immediately, or take their systems offline until they could. Reuters reported that government websites in several places went offline. Coverage also noted that Japan’s prime minister told people to patch urgently.</p><p>When a spy agency says “turn it off if you can’t fix it”, you know it’s not a routine advisory.</p><h3>Incident #2: The “fake padlock” scare, and yes, it involves an Indian company</h3><p>This one hit closer to home for many of us.</p><p><strong>eMudhra</strong>, a Bengaluru-based certificate authority, ended up at the centre of a storm. Quick explainer: certificate authorities are the organisations that vouch for websites. They’re the reason your browser can trust that a site is really what it claims to be.</p><p>Here’s what the reporting says:</p><ul><li>On <strong>October 17</strong>, a security researcher known as “Wayne” found he could reconstruct the private keys for <strong>eight intermediate certificates</strong> linked to eMudhra’s global website-security business, using publicly available certificate data.</li><li>eMudhra blamed a flaw in a script used to generate the certificates in April 2025.</li><li>Google’s Chrome security team warned that the weakness could let anyone forge trusted HTTPS certificates to impersonate any website, for clients that hadn’t yet removed trust.</li><li><strong>Mozilla, Google, Apple and Microsoft</strong> moved to block the certificates over the following days. Google said it would remove the parent root from Chrome starting <strong>October 27</strong>, which could break some legitimate sites that still use certificates under it.</li><li>eMudhra’s shares fell about <strong>20%</strong> on October 21.</li></ul><p>Now the calm-down part, because it matters. eMudhra says its infrastructure wasn’t breached and no data was compromised. It says its Indian DSC and eSign services are unaffected, and that the business involved is only about 1% of its revenue. India’s IT ministry said the impact had been very limited. As of October 21, public reports hadn’t established widespread attacks using the weak keys.</p><p>The uncomfortable part: the weakness reportedly went unnoticed for around <strong>18 months</strong>, and the question now being asked is how it got past two audit cycles.</p><p>Read more: <a href="https://bleepingcomputer.com/news/security/mozilla-google-apple-block-compromised-emudhra-ca-certificates">BleepingComputer</a>, <a href="https://www.businesstoday.in/technology/story/impact-has-been-very-limited-meity-on-emudhra-security-incident-says-indian-dsc-and-esign-unaffected-562927-2026-10-21">Business Today</a> and <a href="https://inc42.com/buzz/emudhras-security-key-flaw-what-went-wrong-and-why-google-mozilla-stepped-in/">Inc42</a>.</p><h3>Incident #3: One hacker, 100+ companies, almost no effort</h3><p>This is the story that made me pause the longest.</p><p>Snorre Fagerland, a researcher at the security firm <strong>NorForge</strong>, was scanning the internet for exposed control panels of <strong>Cairn</strong>, an open-source AI agent platform popular with hackers. He found some. Through them, he could watch a Chinese-speaking hacker at work, and Reuters independently reviewed the activity.</p><p>Except the hacker barely “worked.” According to Reuters and NorForge:</p><ul><li>The operator aimed AI agents at <strong>more than 1,200 websites</strong>, mostly e-commerce.</li><li>More than <strong>100 companies</strong> were breached. Reuters identified victims including <strong>Deere, SharkNinja and Cummins</strong>.</li><li>The instructions given to the agents were written in <strong>fewer than 100 Chinese characters.</strong></li><li>Cost estimates vary between write-ups. One summary of the research put it at roughly <strong>$50 of AI usage per successful hack</strong>, while another cited about $5 per task on average.</li><li>Reporting says the agents appeared to use off-the-shelf models, including Claude Code and DeepSeek.</li></ul><p>For years, hacking at scale needed skill, time and a team. Here, one person wrote a short instruction and let AI agents do the grind.</p><p>For balance: Cairn’s creator said publicly that the platform wasn’t designed for attacks. Also, Reuters noted that in September a firm called Gambit documented a separate AI-agent-driven spree that breached 25 companies and stole at least 600,000 credit card records. So this wasn’t a one-off.</p><p>Read more: <a href="https://eualive.net/how-one-hacker-used-ai-agents-and-a-few-thousand-dollars-to-break-into-scores-of-sites/">Reuters report as syndicated by EUalive</a>.</p><h3>Incident #4: Your own developer tools turn on you</h3><p>On <strong>October 21</strong>, at 11:39 UTC, a new release of <strong>LiteLLM</strong> appeared on PyPI. LiteLLM is a hugely popular Python library for working with AI models, with roughly 23 million weekly downloads.</p><p>Version 1.107.0 was malicious.</p><p>It carried a <strong>credential stealer</strong> and a <strong>remote access trojan</strong>. Whenever Python started on an affected machine, the stealer went looking for cloud credentials, passwords and AI service keys, and the trojan could let the attacker run more code later.</p><p>The bad version was available for about two hours before it was pulled. According to LiteLLM’s own account, the attackers used stolen GitHub CLI tokens belonging to three engineers to trigger the release workflow.</p><p>Here’s the sneaky part: the malicious code was in the packaged files, not in the project’s visible source. If you only reviewed the repository, you wouldn’t have spotted it.</p><p>Lesson: <strong>pip install</strong> is an act of trust. Treat it that way.</p><p>Read more: <a href="https://securitylabs.datadoghq.com/articles/litellm-compromised-pypi-docker/">Datadog Security Labs</a>.</p><h3>And the rest of the pile-up</h3><p>The same stretch also brought:</p><ul><li><strong>Node.js</strong> shipping security fixes for multiple flaws, including a high-severity issue that could allow remote code execution.</li><li><strong>GitLab</strong> disclosing a vulnerability with a maximum <strong>10.0</strong> severity score for self-managed servers.</li><li><strong>Railway</strong>, a cloud platform, pausing deployments and advising customers to rotate their secrets while it investigated possible exposure.</li><li>Issues reported in <strong>Plex</strong> and in an AI coding agent platform called <strong>OpenCode</strong>.</li></ul><h3>Why experts are genuinely rattled</h3><p>It’s not any single incident. It’s what they add up to.</p><p><strong>Chris Krebs</strong>, the former CISA director, told CNBC that about <strong>3% of all critical vulnerabilities ever discovered</strong> were found in just the last month. Let that sink in. A measurable slice of the entire history of vulnerability research, packed into roughly thirty days.</p><p>Then came the warnings from industry leaders:</p><ul><li><strong>Jason Clinton</strong>, Anthropic’s deputy CISO, warned of a coming <strong>“vulnerability tsunami”</strong> and said there won’t be enough time to patch critical infrastructure.</li><li><strong>Marc Benioff</strong> of Salesforce declared it a code-red situation.</li><li><strong>Matthew Prince</strong>, Cloudflare’s CEO, said AI can find security flaws at a pace the security community can’t match, and that it may feel like the sky is falling.</li></ul><p>Krebs’s comments are in <a href="https://www.cnbc.com/2026/10/23/php-vulnerabilities-ai-cybersecurity.html">CNBC’s report</a>, and the Clinton, Benioff and Prince quotes are summarised in <a href="https://finwire.io/news/world-news/black-october-cybersecurity-experts-sound-the-alarm-over-new-era-of-ai-enabled-hacking">this write-up of the Guardian’s coverage</a>.</p><p>And this wasn’t coming out of nowhere. AI companies had been warning about rising cyber capabilities since April, when Anthropic announced a limited rollout of its powerful Mythos model.</p><h3>What this means if you’re in India</h3><p>Two things stood out.</p><p>First, eMudhra put an Indian company in a global spotlight. The Internet Freedom Foundation said it was preparing representations to the IT ministry, the Controller of Certifying Authorities and CERT-In about it.</p><p>Second, <strong>CERT-In</strong> said that in some cases AI agents exceeded their authorised tasks and accessed Indian organisations’ systems, and that attackers are using open-source agentic AI tools for reconnaissance, exploitation and financially motivated intrusions.</p><p>Plenty of Indian startups and small businesses run on the kind of stack that was in the spotlight: PHP sites, Python tooling, cloud platforms. If you’re a freelancer or small business owner thinking “this doesn’t concern me”, it might.</p><h3>What the internet was saying</h3><p>The conversation moved as fast as the incidents. These are some of the threads and posts that were circulating. I could not open them myself, so I’m pointing to them rather than describing what they say.</p><ul><li>Reddit, r/technology: <a href="https://www.reddit.com/r/technology/comments/1xdwbcl/ais_black_october_critical_php_flaws_set_off/">AI’s Black October: critical PHP flaws set off…</a></li><li>Reddit, r/cybersecurity: <a href="https://www.reddit.com/r/cybersecurity/comments/1xdx8k7/megathread_october_2026_cybersecurity_incidents/">Megathread: October 2026 cybersecurity incidents</a></li><li>Reddit, r/OpenAI: <a href="https://www.reddit.com/r/OpenAI/comments/1xchuni/cybersecurity_people_are_losing_their_minds_today/">Cybersecurity people are losing their minds today</a></li><li>LinkedIn: <a href="https://www.linkedin.com/posts/black-october-is-a-reminder-that-fixing-security-ugcPost-7519450082111356928-V_jC/">Black October is a reminder that fixing security…</a></li></ul><h3>Reality check: before you panic</h3><p>Let me be the calm friend for a minute.</p><p>The scary claim What the reporting actually shows “AI swarms are hacking half the internet” These were <strong>separate incidents</strong>. Even dramatic coverage noted there’s no evidence a single AI system caused them. “The internet is collapsing” Fixes exist. Major browser makers moved quickly on eMudhra, and the poisoned LiteLLM release was pulled within hours. “AI only helps attackers” In the PHP case, researchers and OpenAI say AI helped defenders find the bugs first. The danger was the gap between finding and fixing. “Nothing can stop this” Unglamorous steps like patching and rotating keys genuinely reduce risk.</p><p>The honest summary: it’s serious, but it’s not hopeless. In my reading, the core problem is <strong>speed</strong>. Finding bugs got fast, while fixing them still moves at human pace.</p><p>Krebs also flagged an uncomfortable asymmetry: large companies can use advanced AI to harden their defences, while small and mid-sized businesses may find it much harder.</p><h3>Your Black October checklist</h3><p>Do these this week, not “someday.”</p><ol><li><strong>Patch PHP, Node.js and GitLab</strong> if you run them. If you can’t patch yet, consider taking exposed services offline, as the NSA advised for PHP.</li><li><strong>Check whether you installed LiteLLM 1.107.0</strong> (from PyPI, or Docker “latest” images, on October 21). If yes, treat every credential that machine could reach as exposed and rotate them.</li><li><strong>Rotate secrets</strong> on platforms that reported issues, like Railway, and any API keys stored in environment variables.</li><li><strong>If your certificates chain to eMudhra’s emSign ECC Root CA — G3</strong>, get a replacement from another trusted CA before Chrome removes that root on October 27.</li><li><strong>Limit what your AI agents can access.</strong> Assume anything an agent can reach could be exposed if it misbehaves or is hijacked.</li><li><strong>Pin your dependency versions</strong> instead of blindly pulling “latest.”</li><li><strong>Try AI security tools yourself.</strong> Even experts sounding alarms say hands-on use is the fastest way to understand how powerful they’ve become.</li></ol><h3>The thing I can’t stop thinking about</h3><p>This part is my opinion, not reporting.</p><p>We’ve built decades of software on top of software. Old code, borrowed code, code almost nobody fully reads anymore. Part of what kept it safe was that reading all of it carefully was too expensive.</p><p>That safety net is fraying. AI can now go through enormous amounts of code patiently, at any hour, at low cost. Defenders can use that. Attackers can too. The coming couple of years may well be decided by who uses it faster.</p><p>Black October may turn out to be a dramatic name for a rough week. Or it may be remembered as the moment we realised how much of the internet’s safety had depended on nobody looking hard enough.</p><p>Either way, the advice is the same:</p><blockquote><strong><em>Update your stuff. Rotate your keys. And don’t assume somebody else already did.</em></strong></blockquote><p><em>Sources: CNBC, Axios (via Yahoo), TechCrunch, 404 Media, Reuters (as reported by EUalive and others), NorForge research as covered by The Hacker News and others, Business Today, Free Press Journal, Inc42, BleepingComputer, YourStory, Exchange4media, Datadog Security Labs and other technical write-ups. Statements by executives come from their public posts as reported by news outlets. This story is developing fast, so double-check figures and dates before you hit publish.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=afc8cde6c1cb" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Go for Networking: A Practical Guide to Building Networked Systems]]></title>
            <link>https://medium.com/@swarnshekhar007/go-for-networking-a-practical-guide-to-building-networked-systems-e0129a698906?source=rss-7b159942b81d------2</link>
            <guid isPermaLink="false">https://medium.com/p/e0129a698906</guid>
            <category><![CDATA[ip]]></category>
            <category><![CDATA[go]]></category>
            <category><![CDATA[golang]]></category>
            <category><![CDATA[networking]]></category>
            <category><![CDATA[tcp]]></category>
            <dc:creator><![CDATA[รωα૨ɳ Șħ.]]></dc:creator>
            <pubDate>Mon, 06 Jul 2026 14:02:32 GMT</pubDate>
            <atom:updated>2026-07-06T14:02:32.740Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/736/0*_GKW02ovcuoxQt5o.jpg" /></figure><p>if you have tried java/c++ , u know the headache of networking. connecting 500 nodes concurrently is really hectic job , and i am not considering actual code logic which comes after.</p><p>Go was built at Google specifically to make this easier. Concurrency and networking are baked into the language itself, not presented as library .<br>This is a deep, practical walkthrough — by the end, you’ll understand not just the syntax but <em>why</em> Go’s model works the way it does, and you’ll have built a real multi-client TCP server with proper timeouts, worker pools, and graceful shutdown.</p><h3>Part 1: Why Go, specifically, for networking</h3><p>Three language-level features make Go a natural fit for this domain:</p><ol><li><strong>Goroutines</strong> — lightweight, cheap concurrent functions. You can run thousands of them without the memory cost of OS threads (a goroutine starts at ~2KB of size , where as OS thread is in mb).</li><li><strong>Channels</strong> — a built-in, safe way for goroutines to exchange data, replacing manual locks and shared-memory bookkeeping.</li><li><strong>The </strong><strong>net package</strong> — TCP, UDP, and Unix sockets are first-class standard library citizens. No external dependencies needed for basic networking.</li></ol><h3>Part 2: Goroutines, properly understood</h3><p>A goroutine is’nt an os thread. it’s a lightweight unit of concurrent execution , it support m:n scheduling — i.e for what no of threads how many goroutines is assigned.(the “M:N scheduler” — M goroutines onto N threads). That’s why managing 10,000 goroutines is easy but managing 10,000 thread will break many system to knees.</p><p>Starting one is trivial:</p><pre>func sayHello() {<br>	fmt.Println(&quot;Hello from a goroutine&quot;)<br>}</pre><pre>func main() {<br>	go sayHello()<br>	fmt.Println(&quot;Main function done&quot;)<br>}</pre><p>goroutines don’t keep your program alive on their own. If main() returns before the goroutine runs, the goroutine is simply discarded — no error, no warning, it just never happens. You need to synchronise the whole process properly.</p><h3>WaitGroup: waiting for a known number of goroutines</h3><pre>var wg sync.WaitGroup<br><br>for i := 0; i &lt; 5; i++ {     // just incremental<br> wg.Add(1).                  //act as counter.<br> go func(id int) {        //unnamed anonymous function.<br>  defer wg.Done()         // defer means -- run this just after fn is returned.<br>  fmt.Println(&quot;worker&quot;, id, &quot;done&quot;)<br> }(i)       // acting as an argument to func(id int) fucntion.          <br>}<br>wg.Wait()<br>fmt.Println(&quot;All workers finished&quot;)</pre><p>Notice func(id int) { ... }(i) — the goroutine takes i as a parameter rather than capturing it from the surrounding scope directly. This avoids a classic bug: if all five goroutines just referenced the loop variable i directly, by the time they actually ran, the loop might have already finished and every goroutine would see the same final value of i. Passing it as an argument captures the value at that specific iteration.</p><h3>Mutex: protecting shared state when you do need it</h3><p>Channels are preferred, but sometimes a simple shared counter with a lock is more direct:</p><pre>type Counter struct {<br>	mu    sync.Mutex<br>	count int<br>}<br><br>//meaning of code<br>// typer counter specify an struct of name counter.<br>// mu sync.mutex means a lock -- which ensure that <br>// only one goruntine be able to touch count at a time.<br>// varible count with datatype int.<br><br>func (c *Counter) Increment() {<br> c.mu.Lock()<br> defer c.mu.Unlock()<br> c.count++<br>}<br><br>//meaning of code<br>// func datatype increment --- it is how we represent a function.<br>//here (c * counter ) means it a method on counter struct ,and pointer specify<br>// it will make actual changes , not on copied<br>// c.mu.lock() -- it is used to enable lock. if no other goruntine is having<br>// access to lock then this is used to lock.<br>// defer-- as i mentioned earlier defer means run just after above fn<br>// is returned. so when lock is enabled by a gorunine and task is done <br>// and returned , soon after it lock is released.<br>// count is incremented.</pre><p>Lock()/Unlock() ensure only one goroutine touches count at a time. Without this, concurrent increments from multiple goroutines would race — some increments could be silently lost. This is called a <strong>data race</strong>, and Go has a built-in tool to catch it: run any program with go run -race main.go and it will flag unsynchronized concurrent access at runtime.</p><h3>Part 3: Channels, in depth</h3><p>A channel is a typed, synchronized pipe for passing values between goroutines.</p><pre>messages := make(chan string)<br>//it create the empty channel, right now it is now storing anything<br>//just channel is created.<br><br><br>go func() {<br> messages &lt;- &quot;hello&quot;<br>}()<br><br>//this launches gorutines that tries to send &quot;hello&quot; into the channel.<br><br>msg := &lt;-messages<br><br>//it is the receive. it reaches into channel and fetch whatever<br>//message is in channel and store into brand new variable.<br><br>fmt.Println(msg)<br><br>//msg is printed.</pre><h3>Unbuffered vs. buffered channels</h3><p>The channel above is <strong>unbuffered</strong> — a send blocks until a receive is ready to take it, and vice versa. This makes sends/receives a synchronization point, not just data transfer.</p><p>A <strong>buffered</strong> channel allows sends to succeed immediately up to the buffer size, without a receiver being ready yet:</p><pre>ch := make(chan string, 3) // buffer size 3<br>ch &lt;- &quot;a&quot;<br>ch &lt;- &quot;b&quot;<br>ch &lt;- &quot;c&quot;<br>// A 4th send here would block until something is received</pre><p>Use unbuffered channels when you want strict synchronization.</p><p>Use buffered channels when producers and consumers run at different speeds and you want some slack.</p><h3>select: waiting on multiple channels</h3><p>select lets a goroutine wait on several channel operations at once, proceeding with whichever is ready first:</p><pre>select {<br>case msg := &lt;-messages:<br>	fmt.Println(&quot;Got message:&quot;, msg)<br>case &lt;-time.After(2 * time.Second):<br>	fmt.Println(&quot;Timed out waiting&quot;)<br>}<br><br>//is msg is found at channel , then it prints msg<br>// after time limit is exceeded , it won&#39;t wait endelessly,<br>// it will just print - &quot;time out waiting&quot;</pre><p>This pattern — a real channel race against time.After — way to implement timeouts in Go, and it comes up constantly in networked code (e.g., &quot;wait for a response, but give up after N seconds&quot;).</p><h3>Closing channels and range</h3><p>A sender can close a channel to signal “no more values are coming”:</p><pre>func producer(ch chan&lt;- int) {<br>	for i := 0; i &lt; 5; i++ {<br>		ch &lt;- i<br>	}<br>	close(ch)<br>}<br><br>// chan &lt;- int ek type hai .here ch is variable of that type.<br><br>// chan &lt;- int == send only channel<br>// &lt;- chan int == receive only channel<br>// chan int    == bidirectional channel<br><br>//it only accept channel;<br>//&lt;-int shows that it is &quot;send only channel&quot; u can put values in channel<br>// you can receive from it.<br><br><br>// ye ek unbuffered channel hai (make(chan int) — capacity 0)<br>// isliye &#39;ch &lt;- i&#39; line pe producer block ho jayega (ruk jayega)<br>// jab tak koi receiver (yahan main, range se) us value ko le na le.</pre><pre>ch := make(chan int)   // unbuffered channel — capacity = 0<br>go producer(ch)         // producer is running in different goruntine<br>for v := range ch {      // main goroutine is receive msg from channel ch.<br>    fmt.Println(v)<br>}<br><br><br>// let understand carefully. overall.<br><br>// 2 goruntine is created one running range in main and other running producer.<br>// in unbuff channel , msg is not pushed to channel until there is no receiver.<br>// so for eg. i=0 , ch&lt;-0 , now println(0) will be printed.<br>// this cycle repeats from  i=1 se i=4<br>// then producer calls &#39;close(ch)&#39; <br>// and then range loop automatically exits.</pre><h3>Part 4: Structs, JSON, and wire formats</h3><p>Real network messages need structure, and encoding/json is the standard way to define it:</p><pre>type Message struct {<br>	ID      string   `json:&quot;id&quot;`<br>	Payload string   `json:&quot;payload&quot;`<br>	TTL     int      `json:&quot;ttl&quot;`<br>	Path    []string `json:&quot;path&quot;`<br>}</pre><p>The backtick tags matter for a specific reason: Go requires struct fields to start with a capital letter to be exported . JSON convention, on the other hand, expects lowercase or camelCase keys. The tag is a pure translation layer: internally the field is ID, on the wire it becomes id. The data itself never changes, only the key name.</p><pre>data, err := json.Marshal(msg)          // struct -&gt; JSON bytes<br>var received Message<br>err = json.Unmarshal(data, &amp;received)   // JSON bytes -&gt; struct</pre><p>The &amp;received matters: Unmarshal needs to write <em>into</em> your variable, so you pass a pointer. Without it, Unmarshal would only modify a throwaway copy and your original variable would stay empty.</p><p>When reading directly off a network connection , use a decoder instead of unmarshal directly — it streams from the connection:</p><pre>var msg Message<br>err := json.NewDecoder(conn).Decode(&amp;msg)</pre><h3>Part 5: Real TCP networking</h3><h3>The basic client and server</h3><p><strong>Server</strong> — passively listens and accepts:</p><pre>func main() {<br>	listener, err := net.Listen(&quot;tcp&quot;, &quot;:9000&quot;)<br>	if err != nil {<br>		log.Fatal(&quot;Failed to listen:&quot;, err)<br>	}<br>	defer listener.Close()<br>	fmt.Println(&quot;Listening on port 9000...&quot;)<br><br><br>  for {<br>  conn, err := listener.Accept()<br>  if err != nil {<br>   continue<br>  }<br>  go handleConnection(conn)<br> }<br>}<br><br><br>func handleConnection(conn net.Conn) {<br> defer conn.Close()<br> var msg Message<br> if err := json.NewDecoder(conn).Decode(&amp;msg); err != nil {<br>  fmt.Println(&quot;decode error:&quot;, err)<br>  return<br> }<br> fmt.Println(&quot;Received:&quot;, msg.ID, &quot;-&quot;, msg.Payload)<br>}<br><br>// it is basic syntax , u can get it easily</pre><p><strong>Client</strong> — actively dials out:</p><pre>func main() {<br>	conn, err := net.Dial(&quot;tcp&quot;, &quot;localhost:9000&quot;)<br>	if err != nil {<br>		log.Fatal(&quot;Failed to connect:&quot;, err)<br>	}<br>	defer conn.Close()<br><br>msg := Message{ID: &quot;msg-001&quot;, Payload: &quot;hello over the wire&quot;, TTL: 5}<br><br> if err := json.NewEncoder(conn).Encode(msg); err != nil {<br><br>  log.Fatal(&quot;encode error:&quot;, err)<br> }<br><br> fmt.Println(&quot;Message sent&quot;)<br>}</pre><p><strong>net.Listen waits passively; </strong><strong>net.Dial initiates.</strong></p><p>net.Listen just waits for someone to connect. net.Dial is the one that reaches out and starts the connection.</p><p>This basic difference — one side waits, the other starts things — is the core idea behind almost every client-server system, whether it’s raw sockets, HTTP, or gRPC.</p><p><strong>The important detail: </strong><strong>go handleConnection(conn).<br></strong>Every time a connection comes in, it gets handed to its own goroutine.</p><p>This is why goroutines matter for networking: if one client is slow or acting up, it won’t block or slow down any other client.</p><p>If you used one thread per connection instead, things would slow down badly once you hit a few thousand clients. But in Go, using a goroutine for each connection is cheap and easy — so it’s just the normal, default way to do it.</p><h3>Adding timeouts (production code needs this)</h3><p>Raw net.Dial and reads will hang forever if the other side goes silent. Always set deadlines:</p><pre>conn, err := net.DialTimeout(&quot;tcp&quot;, &quot;localhost:9000&quot;, 5*time.Second)<br>if err != nil {<br>	log.Fatal(err)<br>}</pre><pre>conn.SetReadDeadline(time.Now().Add(10 * time.Second))</pre><p>If the deadline passes, any in-progress read/write returns an error you can check for, instead of hanging indefinitely — critical for anything that touches an unreliable network (which is all of them, eventually).</p><h3>Part 6: UDP — when you don’t need a connection</h3><p>TCP supports ordered , handskake based delivery ,UDP skips all of that — you just fire packets and hope, which is exactly right for things like real-time game state, DNS lookups, or discovery broadcasts where an occasional dropped packet doesn’t matter.</p><pre>// UDP server<br>addr, _ := net.ResolveUDPAddr(&quot;udp&quot;, &quot;:9001&quot;)     <br> // on &quot;udp&quot; protocol it is resolving an address.<br>// port 9001<br><br>conn, _ := net.ListenUDP(&quot;udp&quot;, addr)     <br>//on that port , to read udp packets , a connection (socket) is established.<br>defer conn.Close()               <br><br>buf := make([]byte, 1024)       <br>// making a buffer of size 1024 byte to store incoming message.<br>                        <br><br>for {               <br>                                       <br> // infinite loop.<br> n, remoteAddr, err := conn.ReadFromUDP(buf)  <br>// reading data in buffer<br>// n          == no of bytes.<br>// remoteAddr == sender&#39;s address.<br>// err        == if error.<br>          <br> if err != nil {     <br>  continue                       <br> }<br><br> fmt.Println(&quot;Got:&quot;, string(buf[:n]), &quot;from&quot;, remoteAddr) <br><br>}</pre><pre>// UDP client<br>conn, _ := net.Dial(&quot;udp&quot;, &quot;localhost:9001&quot;)<br>defer conn.Close()<br>conn.Write([]byte(&quot;hello over UDP&quot;))</pre><p>Notice there’s no Accept() loop — UDP has no concept of a persistent connection. Each packet is independent, and ReadFromUDP tells you who it came from on every read. This makes UDP a natural fit for discovery protocols (&quot;who else is on this network?&quot;) where you broadcast and see who answers, rather than maintaining a live session.</p><h3>Part 7: Worker pools — handling load deliberately</h3><p>Spawning an unbounded goroutine per connection is fine for a demo, but under real load (thousands of clients) you may want to cap concurrent work. A worker pool pattern does this using a channel as a queue:</p><pre>func worker(id int, jobs &lt;-chan net.Conn) {<br>	for conn := range jobs {<br>		handleConnection(conn)<br>	}<br>}</pre><pre>func main() {<br>	listener, _ := net.Listen(&quot;tcp&quot;, &quot;:9000&quot;)<br>	jobs := make(chan net.Conn, 100)</pre><pre>	for w := 1; w &lt;= 10; w++ {<br>		go worker(w, jobs)<br>	}</pre><pre>	for {<br>		conn, err := listener.Accept()<br>		if err != nil {<br>			continue<br>		}<br>		jobs &lt;- conn<br>	}<br>}</pre><p>Ten workers pull connections off the jobs channel as they become free, instead of one unbounded goroutine per connection. This gives you a fixed upper bound on concurrent work — useful when each connection does something expensive (disk I/O, a downstream API call) and you don&#39;t want to accidentally launch 50,000 goroutines under a traffic spike.</p><h3>Part 8: Graceful shutdown</h3><p>A server that just gets killed mid-connection is a bad citizen. Handling OS signals lets you shut down cleanly:</p><pre>func main() {<br>	listener, _ := net.Listen(&quot;tcp&quot;, &quot;:9000&quot;)<br><br>sigCh := make(chan os.Signal, 1)<br> signal.Notify(sigCh, os.Interrupt, syscall.SIGTERM)<br> go func() {<br>  &lt;-sigCh<br>  fmt.Println(&quot;Shutting down...&quot;)<br>  listener.Close()<br>  os.Exit(0)<br> }()<br> for {<br>  conn, err := listener.Accept()<br>  if err != nil {<br>   break<br>  }<br>  go handleConnection(conn)<br> }<br>}</pre><p>signal.Notify routes OS interrupt signals (Ctrl+C, or kill) into a channel instead of just killing the process outright, giving you a chance to close listeners and clean up in-flight work before exiting.</p><h3>Where this goes next</h3><p>With goroutines, channels, structs/JSON, TCP and UDP via the net package, timeouts, context, worker pools, graceful shutdown, and a way to test it all, you have essentially everything needed to build real networked systems: chat servers, distributed caches, peer-to-peer message relays, job queues, mesh networks.</p><p>Go didn’t add concurrency and networking as an afterthought bolted onto an existing language — they’re why the language exists in its current form.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e0129a698906" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[RAG, Vectorless RAG & GraphRAG]]></title>
            <link>https://medium.com/@swarnshekhar007/rag-vectorless-rag-graphrag-0245d6ee8e71?source=rss-7b159942b81d------2</link>
            <guid isPermaLink="false">https://medium.com/p/0245d6ee8e71</guid>
            <category><![CDATA[graphrag]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[vectorless]]></category>
            <category><![CDATA[vectorless-rag]]></category>
            <category><![CDATA[rags]]></category>
            <dc:creator><![CDATA[รωα૨ɳ Șħ.]]></dc:creator>
            <pubDate>Wed, 24 Jun 2026 09:58:14 GMT</pubDate>
            <atom:updated>2026-06-24T09:58:14.778Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/736/0*MUgobTpFMMjatzOh.jpg" /></figure><p><em>A no-fluff breakdown of how modern LLMs retrieve what they were never trained on.</em></p><h3>What Even Is RAG?</h3><p>RAG stands for <strong>Retrieval Augmented Generation</strong>. The name tells you everything, if you slow down to read it.</p><p><em>Retrieval</em> — you pull relevant information from a vector database based on what the user asked. <em>Augmented Generation</em> — an LLM takes that retrieved content alongside the user’s prompt and generates a response grounded in it.</p><p>That’s it. That’s the whole idea.</p><h3>So Why Do We Need It?</h3><p>LLMs are trained on a snapshot of the world. Whatever wasn’t in that snapshot — it doesn’t exist for the model.</p><p>Here’s a real scenario. Say you found a rare, obscure book — barely indexed anywhere online. Communist theory from some corner of the internet, 3,000 pages, never made it into any training dataset. You ask an LLM about it. It has no idea. You give it web scraping tools. Still nothing — you can’t scrape what you can’t find.</p><p>This is exactly the problem RAG solves.</p><p>You feed that PDF into the RAG pipeline. At query time, the pipeline retrieves the most relevant sections from the book and passes them to the LLM as context. The model’s job then becomes simple: answer based on this content, and shape the response using your language understanding.</p><blockquote><em>The LLM never needs to be retrained. It just gets the right context handed to it at the right moment.</em></blockquote><h3>The Basic RAG Pipeline</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*NHUMV5rjiq50NW9b.png" /></figure><p>There are a few stages to how RAG actually works under the hood. The first — and most foundational — is <strong>data indexing</strong>, which begins with something called chunking.</p><h4>Data Chunking</h4><blockquote><em>You can’t dump a 3,000-page PDF straight into a vector database. You break it into smaller pieces first — chunks — so each piece can be embedded and retrieved independently.</em></blockquote><p>There are three common ways to do this.</p><p><strong>Page-by-page chunking</strong> treats every page as its own chunk — so a 3,000-page book becomes 3,000 chunks. This works well when you care most about precision and want retrieved content to stay contextually tight.</p><p><strong>Paragraph-by-paragraph chunking</strong> goes more granular, splitting on paragraph breaks. Retrieval can feel faster here, but the trade-off is inconsistent chunk sizes — one paragraph might be 200 tokens, another 2,000. Uneven sizes lead to uneven embeddings, which quietly hurts retrieval quality.</p><p><strong>Fixed-window chunking</strong> is the most widely used approach. Every chunk has a fixed token size — say, 512 tokens — regardless of where it falls in the document. Uniform sizes produce more consistent, comparable embeddings, which makes retrieval more reliable. When in doubt, this is the default you reach for.</p><pre>from langchain_text_splitters import CharacterTextSplitter<br>from langchain_core.documents import Document<br><br>def perform_fixed_size_chunking(document, chunk_size=1000, chunk_overlap=200😞<br>    &quot;&quot;&quot;<br>    Performs fixed-size chunking on a document with specified overlap.<br>    <br>    Args:<br>        document (str): The text document to process<br>        chunk_size (int): The target size of each chunk in characters<br>        chunk_overlap (int): The number of characters of overlap between chunks<br>        <br>    Returns:<br>        list: The chunked documents with metadata<br>    &quot;&quot;&quot;<br>    # Create the text splitter with optimal parameters<br>    text_splitter = CharacterTextSplitter(<br>        separator=&quot;\n\n&quot;,<br>        chunk_size=chunk_size,<br>        chunk_overlap=chunk_overlap,<br>        length_function=len<br>    )<br>    <br>    # Split the text into chunks<br>    chunks = text_splitter.split_text(document)<br>    print(f&quot;Document split into {len(chunks)} chunks&quot;)<br>    <br>    # Convert to Document objects with metadata<br>    documents = []<br>    for i, chunk in enumerate(chunks):<br>        doc = Document(<br>            page_content=chunk,<br>            metadata={<br>                &quot;chunk_id&quot;: i,<br>                &quot;total_chunks&quot;: len(chunks),<br>                &quot;chunk_size&quot;: len(chunk),<br>                &quot;chunk_type&quot;: &quot;fixed-size&quot;<br>            }<br>        )<br>        documents.append(doc)<br>    <br>    return documents<br><br># Example usage<br>if __name__ == &quot;__main__&quot;:<br>    <br>    # Create the dummy document<br>    document = create_dummy_document()<br>    <br>    # Process with fixed-size chunking<br>    chunked_docs = perform_fixed_size_chunking(<br>        document,<br>        chunk_size=1000,<br>        chunk_overlap=200<br>    )<br>    <br>    # Display results<br>    print(&quot;\n----- CHUNKING RESULTS -----&quot;)<br>    print(f&quot;Total chunks: {len(chunked_docs)}&quot;)<br>    <br>    # Print an example chunk<br>    print(&quot;\n----- EXAMPLE CHUNK -----&quot;)<br>    middle_chunk_idx = len(chunked_docs) // 2<br>    example_chunk = chunked_docs[middle_chunk_idx]<br>    print(f&quot;Chunk {middle_chunk_idx} content ({len(example_chunk.page_content)} characters):&quot;)<br>    print(&quot;-&quot; * 40)<br>    print(example_chunk.page_content)<br>    print(&quot;-&quot; * 40)<br>    print(f&quot;Metadata: {example_chunk.metadata}&quot;)<br>    <br>    # For integration with Databricks Vector Search<br>    print(&quot;\nThese documents are ready for embedding and storage in Databricks Vector Search&quot;)<br>    print(&quot;Example next steps:&quot;)<br>    print(&quot;1. Create embeddings using the Databricks embedding endpoint&quot;)<br>    print(&quot;2. Store documents and embeddings in Delta table&quot;)<br>    print(&quot;3. Create Vector Search index for retrieval&quot;)</pre><h3><em>Data Embedding</em></h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*CpmRz5kXAbzPpwKI.png" /></figure><p>embedding’s purpose is to generate dence vector in high dimensional space representing the semantic meaning of that word , sentence , page which helps in capturing semantic simalarity of one cluster to another.</p><h4>How Are Embeddings Used in Retrieval-Augmented Generation (RAG)?</h4><p>it involve 4 phases.</p><p><strong><em>1.corpse embedding<br></em></strong>the corpse or document’s embedding is generated in chunk wise manner. and for each chunk ,its index and embedding is stored in vector database.</p><p><strong><em>2.prompt embedding<br></em></strong>the user request/prompt ‘s embeddings is generated. the query embedding represent the semantic meaning of user request.</p><p><strong><em>3</em></strong>.<strong><em>retrieval<br></em></strong>based on embedding of query , whole vector db is traversed looking for best embedding matching with query embedding , usually it is done through cosine similarity.</p><p><strong><em>4.augmentation<br></em></strong>the retrieved document act as additional context along with prompt. llm through its ability of natural language processing and additional context gives response back to user.</p><h3>vector storage</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*7_yF7eAMvO5X9vKc.jpg" /></figure><p>this topic does not need much attention its just a database which stores. we cant use relational or object-oriented database here.</p><p>it is a specialised No-Sql database which manage and store vector embeddings.</p><p>some popular ones are- alloydb,pineon db,qdrant , which i use generally is pgvector ,extension of postgre-sql.</p><h3>limitation</h3><ol><li>when u break it into chunks ,it might no be relevant chunking. suppose your window_b context is related with window_b , higher chunk_overlap may save it sometime , but its not a solution</li><li>rag is based on simlarity comparision, you asked ‘sales decreased’ and in document its written ‘company in decline’ if embedding didn’t find themself close then , relevant chunk might miss.</li><li>multi-hop reasoning — suppose a big chunk is splitted into 2 chunk , one is cause and other is effect . rag retrieves them</li><li>lastly , vector embeddings are expensive to build. storing embeddings , indexing , reindexing etc.</li></ol><p>Here’s the thing nobody tells you when you’re starting out: a vector database is not RAG. It’s one implementation of retrieval. Vectorless RAG keeps the “retrieve, then generate” idea but drops the embedding + similarity-search machinery entirely.</p><p>The motivation is almost a direct response to the limitations section above — embeddings are expensive to build and re-index, semantic similarity can miss exact matches (numbers, IDs, names, error codes — things where you want exact retrieval, not “close enough”), and the whole pipeline adds a layer of infrastructure you now have to maintain.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*ImMK6YYZwIFs0WrI.png" /></figure><p>There’s no single way to do vectorless RAG — it’s a family of approaches that all skip the vector DB:</p><ol><li><strong><em>Lexical / keyword search. <br></em></strong>Instead of cosine similarity over embeddings, you do classic full-text search — BM25, Postgres tsvector, Elasticsearch. This matches exact terms and is genuinely better than embeddings for things like product SKUs(stock keeping units), legal citations, or error logs, where &quot;semantically similar&quot; isn&#39;t what you want - you want the exact string.</li></ol><pre>from rank_bm25 import BM25Okapi<br><br>def vectorless_retrieve(query, corpus_chunks, top_k=3):<br>    &quot;&quot;&quot;<br>    Lexical retrieval over raw text chunks - no embeddings, no vector DB.<br>    &quot;&quot;&quot;<br>    tokenized_corpus = [chunk.lower().split() for chunk in corpus_chunks]<br>    bm25 = BM25Okapi(tokenized_corpus)<br><br>    tokenized_query = query.lower().split()<br>    scores = bm25.get_scores(tokenized_query)<br><br>    ranked = sorted(zip(corpus_chunks, scores), key=lambda x: x[1], reverse=True)<br>    return [chunk for chunk, score in ranked[:top_k]]<br></pre><p><strong><em>2. Agentic / tool-based retrieval.<br> </em></strong>Instead of pre-indexing anything, the LLM itself decides what to look at — it greps a file, calls a search API, opens a specific document section, the way Claude Code or any coding agent navigates a codebase. There’s no upfront indexing step at all; retrieval happens live, on demand, driven by the model’s own reasoning about what it needs next.</p><p><strong><em>3. Long-context stuffing.</em></strong> <br>With models now supporting context windows in the hundreds of thousands to millions of tokens, for smaller corpora you can just skip retrieval altogether and put the whole document (or a large relevant section) directly in context. Not really “retrieval” in the traditional sense, but functionally it gets you the same outcome for small-to-medium corpora.</p><p><strong><em>4. Hybrid re-ranking.<br></em></strong> Cheap lexical search as a first-pass filter (fast, no embedding cost) followed by an LLM re-ranking the shortlist for relevance. You get the speed of keyword search with some of the nuance of semantic understanding, without paying full embedding-and-index costs upfront.</p><p><strong><em>Limitations. <br></em></strong>Pure keyword search misses paraphrases — “sales decreased” still won’t match “company in decline” any better than embeddings did, sometimes worse. Agentic retrieval adds latency and token cost per query since the model is making live decisions instead of a single fast lookup. And none of these approaches scale to massive corpora as cleanly as a properly indexed vector store does — vectorless RAG tends to win at small-to-medium scale or when exactness matters more than fuzziness, not at “index the entire internet” scale.</p><h3>What Exactly Is GraphRAG?</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Ph32vf7E2j2J0PZo.png" /><figcaption>graphrag architecture with neo4j</figcaption></figure><p>So we talked about how basic RAG breaks cause and effect across chunks. And how vectorless RAG trades embeddings for keyword search or long-context stuffing. Both are valid — but neither actually <em>understands</em> how ideas in your document connect to each other.</p><p>That’s the gap Graph RAG fills.</p><p>Instead of asking “which chunk is most similar to this query?” it asks a different question entirely: <strong>“how are these concepts related to each other?”</strong></p><p>The answer is a knowledge graph.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*l0XkVXr2NuT3Ydvg.png" /><figcaption>working case of graphRag</figcaption></figure><h4>The indexing phase — building the graph</h4><p>Forget chunking. Forget embeddings. Graph RAG sends your document through an LLM and tells it to do one thing: <em>read, and extract relationships.</em></p><p>What comes out is a graph — nodes and edges.</p><p>Going back to that 3,000-page communist theory book from earlier:</p><ul><li><strong>Nodes:</strong> Marx, Engels, Capital, The Communist Manifesto, surplus value, dialectical materialism</li><li><strong>Edges:</strong> Marx → <em>authored</em> → Capital, Marx → <em>co-wrote with</em> → Engels, Capital → <em>introduces concept</em> → surplus value</li></ul><p>Every entity the LLM finds becomes a node. Every relationship between entities becomes an edge. You’re not slicing the document into pieces anymore — you’re mapping the <em>meaning</em> inside it.</p><p>Once the graph is built, a clustering algorithm (Leiden is popular) groups tightly connected nodes into <strong>communities</strong> — basically topic clusters. Then another LLM pass summarizes each community.</p><p>This is the part most people miss. Graph RAG operates at two levels at once:</p><ul><li><strong>Node level</strong> — specific facts and direct relationships</li><li><strong>Community level</strong> — broad themes and summaries</li></ul><p>So when someone asks a narrow question, you retrieve nodes. When someone asks something broad like “what are the core ideas in this book?” — you pull community summaries. Same index, two modes of retrieval.</p><h4>The query phase — walking the graph</h4><p>Here’s where the real magic happens. Say someone asks:</p><blockquote>“Who co-wrote with Marx, and what did they write together?”</blockquote><p>In basic RAG, this is a nightmare. The co-authorship might be mentioned in one chunk. The list of works might be 600 pages away in another. You hope the embeddings are close enough. Sometimes they are. Often they’re not.</p><p>In Graph RAG, this is just a graph walk.</p><ol><li><strong>Entity detection</strong> — the system identifies Marx as the anchor node in the query</li><li><strong>Hop 1</strong> — retrieve Marx’s node and all his direct edges</li><li><strong>Hop 2</strong> — follow the <em>co-wrote with</em> edge → land on Engels</li><li><strong>Hop 3</strong> — follow Engels’s <em>authored</em> edges → land on The Communist Manifesto, Condition of the Working Class, etc.</li></ol><p>Every hop follows a typed, directional edge. The cause and the effect that basic RAG split into two random chunks? In Graph RAG they’re two nodes connected by an edge. You get both, with the relationship intact, every time.</p><p>This is <strong>multi-hop reasoning</strong> — and it’s the one thing neither basic RAG nor vectorless RAG can do cleanly.</p><p>The retrieved nodes, edges, and community summaries then get assembled into context and handed to the LLM. From there it’s the same as always — the model generates a response grounded in what was retrieved.</p><pre>from graphrag import GraphRAGPipeline<br><br># Indexing — runs once<br>pipeline = GraphRAGPipeline(llm=&quot;claude-3&quot;, graph_store=&quot;neo4j&quot;)<br>pipeline.index(documents=[&quot;book.pdf&quot;])<br># Under the hood: entity extraction → graph build → community detection → summaries<br><br># Querying<br>result = pipeline.query(<br>    &quot;Who co-wrote with Marx and what did they write together?&quot;,<br>    mode=&quot;global&quot;   # uses community summaries for broad questions<br>    # mode=&quot;local&quot;  # uses node-level traversal for specific facts<br>)<br>print(result.answer)<br>print(result.sources)   # returns actual nodes + edges used, fully traceable</pre><p>The mode parameter is actually a big deal. Most Graph RAG implementations (Microsoft&#39;s open-source one included) expose this explicitly because the two modes are genuinely different retrieval strategies, not just a setting.</p><h4>Limitations — because nothing is free</h4><p><strong>Extraction is expensive.</strong> An LLM has to read your entire document to build the graph. For a 3,000-page book that’s a lot of tokens, and it still misses implicit relationships — things like “as discussed earlier” don’t become graph edges automatically.</p><p><strong>Graph quality is your ceiling.</strong> Bad extraction = bad graph = bad retrieval. And unlike a vector index where you just re-embed to fix things, here you re-read the whole document. Painful at scale.</p><p>The retriever gets picked based on query type, not applied uniformly to everything.</p><p>That’s the full picture. RAG for semantic retrieval, Vectorless RAG when you need exactness or want to skip the infrastructure, and Graph RAG when your data is deeply interconnected and relationships are the whole point.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=0245d6ee8e71" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[DisasterNet Has a Range Problem. Here’s a BFS-Based Fix I’ve Been Designing.]]></title>
            <link>https://medium.com/@swarnshekhar007/disasternet-has-a-range-problem-heres-a-bfs-based-fix-i-ve-been-designing-60988b997fd7?source=rss-7b159942b81d------2</link>
            <guid isPermaLink="false">https://medium.com/p/60988b997fd7</guid>
            <dc:creator><![CDATA[รωα૨ɳ Șħ.]]></dc:creator>
            <pubDate>Tue, 16 Jun 2026 18:10:57 GMT</pubDate>
            <atom:updated>2026-06-17T16:57:15.077Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/736/1*gYav0j7fp-2yyAFPhP7I2A.jpeg" /></figure><p><em>A repo, a limitation, and a packet-routing idea I’m calling WaveNet.</em></p><p>I came across a repo called <strong>DisasterNet</strong> (by AbhinavXJ) <a href="https://github.com/AbhinavXJ/DisasterNet">https://github.com/AbhinavXJ/DisasterNet</a></p><p>a few days ago — a peer-to-peer chat system built for disaster scenarios where the internet is down. No servers, no cloud. Just libp2p for the P2P transport and mDNS for peer discovery, all running over a shared local network.</p><p>The architecture is simple and it works:</p><pre>swarn&#39;s Device                    peoples&#39;s Device<br>┌────────────────┐               ┌────────────────┐<br>│  React frontend │               │  React frontend │<br>└────────┬─────────┘               └────────┬─────────┘<br>         │ HTTP                              │ HTTP<br>┌────────▼─────────┐               ┌────────▼─────────┐<br>│   Go backend      │◄───libp2p───►│   Go backend      │<br>│  mDNS discovery   │   pubsub      │  mDNS discovery   │<br>└───────────────────┘               └───────────────────┘<br>         └──────────── same Wi-Fi/hotspot ───────────────┘</pre><p>Two devices on the same network discover each other via mDNS, connect over libp2p, and subscribe to a pubsub topic. Anyone in the “room” gets every message. It’s a clean implementation and a genuinely useful pattern for local, infrastructure-free chat.</p><p>But the constraint that jumped out at me immediately: <strong>everyone has to be on the same network.</strong> Same Wi-Fi, same hotspot, same LAN. A typical phone hotspot has a range of roughly 20–30 meters. In an actual disaster — people spread across a collapsed building, across blocks — that radius covers almost nothing.</p><p>So the question I kept circling back to was: what if there’s no shared network at all? What if A and the rescuer are 300 meters apart, with twenty strangers and their phones somewhere in between?</p><h3>Reframing it as a graph problem</h3><p>Once I stopped thinking about it as a networking problem and started thinking about it as a graph problem, it became a lot more tractable.</p><p>Model every device-holding person as a node. Draw an edge between two nodes if they’re within short-range communication distance of each other (Bluetooth/Wi-Fi Direct range, roughly). The question becomes: <strong>starting from node A, what’s the shortest path to any node that has an active internet connection?</strong></p><p>That’s a textbook BFS, except the graph isn’t static — it’s people walking around, and edges form and break as proximity changes.</p><pre>B ---- D<br>       /        \<br>      A          F ---- G(internet) ---- Rescuer<br>       \        /<br>        C ---- E</pre><p>A doesn’t need to know the full graph. It doesn’t need routing tables or global knowledge. It just needs to flood the message outward, level by level, and let each node make one local decision.</p><h3>The packet</h3><p>Every SOS is wrapped in a small struct that travels with the message as it hops:</p><pre>type SOSPacket struct {<br>    ID        string    // unique, e.g. hash(origin_id + timestamp)<br>    OriginID  string     // device ID of A — never changes<br>    Lat       float64    // frozen at creation time<br>    Lng       float64    // frozen at creation time<br>    Message   string<br>    TTL       int        // hop limit, prevents unbounded flooding<br>    Seen      []string   // device IDs that have already relayed this packet<br>}</pre><p>Two fields matter more than the rest. Lat/Lng are captured once, at the origin, and never updated — no matter how many hops the packet takes, the rescuer always gets A&#39;s actual location, not the location of whoever happened to relay it last. And Seen is what keeps this from collapsing into an infinite rebroadcast loop — every node checks the list before relaying, and a node already in Seen is simply skipped.</p><h3>The relay logic</h3><p>Each node, on receiving a packet, runs essentially the same check:</p><pre>func handlePacket(pkt SOSPacket, self Device) {<br>    if contains(pkt.Seen, self.ID) {<br>        return // already processed, drop it<br>    }<br>    pkt.Seen = append(pkt.Seen, self.ID)</pre><pre>    if self.HasInternet() {<br>        sendToRescueServer(pkt)<br>        return // path found, stop propagating<br>    }</pre><pre>    if pkt.TTL &lt;= 0 {<br>        return // ran out of hops, drop it<br>    }<br>    pkt.TTL--</pre><pre>    for _, neighbor := range self.NearbyDevices() {<br>        if !contains(pkt.Seen, neighbor.ID) {<br>            neighbor.Send(pkt)<br>        }<br>    }<br>}</pre><p>It’s the same BFS structure you’d write for a “shortest path in unweighted graph” problem, just with the queue distributed across actual physical devices instead of living in one process’s memory. There’s no central coordinator deciding the traversal order — each device just acts on local information (who’s nearby, do I have internet) and the global behavior of “find the nearest gateway to the internet” emerges from that.</p><pre>hop 0:  [A]<br>hop 1:  [B, C]                      (A relays to neighbors)<br>hop 2:  [D, E]                      (B, C relay further)<br>hop 3:  [F]                         (D, E relay further)<br>hop 4:  [G] --- has internet ---&gt;  rescue server</pre><h3>Where this differs from DisasterNet, structurally</h3><p>DisasterNet This design (WaveNet) Discovery mDNS on shared network proximity-based (BT/WiFi Direct) Topology assumption single shared network no shared network required Message delivery pubsub, everyone in room gets it targeted — only toward an internet gateway Termination condition n/a (always connected) first node with internet, or TTL exhausted Loop prevention not needed (single network) Seen list per packet Failure mode no peers on network = no comms no internet anywhere in range = SOS stuck at TTL boundary</p><p>DisasterNet solves “let people in the same room talk to each other.” This tries to solve a narrower, harder problem: “get one critical message out of a dead zone using nothing but the people standing between the origin and the nearest connection.” It’s not a replacement — DisasterNet’s pubsub chat room is genuinely a better fit once you’re already in range of others. This is closer to a single-purpose escape hatch for the SOS case specifically, layered on top of a similar P2P foundation.</p><h3>The unresolved edges</h3><p>A few things I haven’t fully worked through, and would rather name than gloss over:</p><p><strong>Duplicate delivery.</strong> If the packet reaches two different internet-connected nodes around the same time via different paths, the rescue server gets the same SOS twice. The ID field is meant to make this a non-issue server-side (dedupe on ID, first write wins), but it does mean the server needs to maintain at least a short-lived cache of seen IDs.</p><p><strong>TTL tuning.</strong> Set it too low and a real SOS dies a few hops short of a working gateway. Set it too high and a single packet can flood a wide area of devices, draining battery on relays that were never going to find a path anyway. I don’t have a principled answer for the right default yet — it probably needs to be adaptive based on local density rather than a fixed constant.</p><p><strong>Physical layer.</strong> BFS-over-people only works if devices can actually talk to nearby devices without a shared network. That means Bluetooth or Wi-Fi Direct, both of which have real constraints (Wi-Fi Direct in particular is locked down on most mobile OSes and isn’t trivially scriptable from a normal app). This is the part where the design is clean on paper and the implementation is going to be the actual hard part.</p><p>If anyone’s worked on delay-tolerant networking, opportunistic mesh routing, or anything adjacent — I’d rather hear where this breaks now than discover it the hard way later.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=60988b997fd7" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>