<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Kartik Labhshetwar on Medium]]></title>
        <description><![CDATA[Stories by Kartik Labhshetwar on Medium]]></description>
        <link>https://medium.com/@code_kartik?source=rss-c3102ac98f5b------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*kWdLyQH9X5taI8y6bmRbDA.png</url>
            <title>Stories by Kartik Labhshetwar on Medium</title>
            <link>https://medium.com/@code_kartik?source=rss-c3102ac98f5b------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Wed, 18 Nov 2026 23:17:48 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@code_kartik/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[The speech model was never the problem]]></title>
            <link>https://medium.com/@code_kartik/the-speech-model-was-never-the-problem-9664b5986880?source=rss-c3102ac98f5b------2</link>
            <guid isPermaLink="false">https://medium.com/p/9664b5986880</guid>
            <category><![CDATA[macos]]></category>
            <category><![CDATA[productivity]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[open-source]]></category>
            <category><![CDATA[software-development]]></category>
            <dc:creator><![CDATA[Kartik Labhshetwar]]></dc:creator>
            <pubDate>Wed, 18 Nov 2026 16:06:06 GMT</pubDate>
            <atom:updated>2026-11-18T16:06:06.679Z</atom:updated>
            <content:encoded><![CDATA[<blockquote><em>I shipped my Mac dictation app four times on launch day. Here’s what actually broke, what I built to fix it, and how it compares with Wispr Flow, Superwhisper and MacWhisper.</em></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*fUradgIw-5BtDyArUmxlsw.png" /></figure><p>On October 8, 2026, I released BetterWispr, a free, open source dictation app for the Mac. By that evening I had released it three more times.</p><p>The second release was the bad one. In version 0.1.1, every speech model download closed the app the moment it started. A dictation app without a speech model is a microphone icon that does nothing. Version 0.1.2 fixed it a few hours later. The fourth release, that same evening, was about AirPods.</p><p>The speech model was fine that day, and it has been fine in every release since. What broke was everything around it: a download, a microphone, a paste, a cleanup rule, a word list. That’s the most useful thing I’ve learned about dictation apps, and it’s why I built my own.</p><h3>Why I built another dictation app</h3><p>I talk faster than I type, so I’d tried the usual options. Each one was good at something and made me give up something else.</p><p>Wispr Flow writes the most polished text of the lot, but your speech is processed in the cloud and the app needs internet to start. Apple Dictation is free and built in, but it does little cleanup and doesn’t know the names I say all day. The local apps kept my audio on my Mac but were built around transcribing files, or kept the features I wanted in a paid tier.</p><p>What I wanted was boring to describe and hard to find in one app: hold a key, talk, let go, and clean text appears in whatever app I’m in. Nothing leaves the Mac. No account. Free, with source code I can read. So I wrote it.</p><h3>How it works</h3><p>BetterWispr is one Swift package with three parts: the macOS app, a core library that does the real work, and a command-line tool that runs the same code without the interface. Every dictation goes through four steps:</p><ol><li><strong>Capture.</strong> The microphone opens when you press ⌥ Space and keeps every frame until you let go.</li><li><strong>Recognize.</strong> A local speech model turns the audio into text: NVIDIA’s Parakeet, OpenAI’s Whisper, Cohere Transcribe or Apple’s own speech model.</li><li><strong>Clean up.</strong> Rules remove “um”, stutters and self-corrections, then apply punctuation, numbers and the style you picked for that app.</li><li><strong>Insert.</strong> The text is pasted into the field you were typing in, and your clipboard is put back.</li></ol><p>Step 2 is the one everyone benchmarks, and it’s the fast one. On an Apple M5, Parakeet transcribed 5 to 17 second clips in at most 0.28 seconds, 132 to 182 times faster than real time.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xj8HkX_xfNMf-otubP4njQ.png" /></figure><p>Recognition is not where the wait comes from. Steps 1, 3 and 4 are where dictation apps succeed or fail, so that’s where most of my seven releases went.</p><h3>The parts I’m proud of</h3><p>A few features came straight out of my own annoyances:</p><ul><li><strong>A tone per app.</strong> Slack gets your work tone, Mail gets your email tone, and Very casual drops the commas and capitals for texting friends.</li><li><strong>Vocabulary that learns.</strong> Fix a misheard name once and it’s spelled right next time.</li><li><strong>Voice commands</strong> such as “new line”, “comma” and “scratch that”.</li><li><strong>History with Use Original.</strong> Every dictation keeps the raw transcript next to the cleaned one, because cleanup is sometimes wrong and you deserve a way back.</li><li><strong>Nothing gets lost.</strong> If transcription fails or you quit mid-dictation, the recording is kept so you can retry it from History.</li><li><strong>Meeting notes without a bot.</strong> Your microphone is labeled Me, the call audio is labeled Them, and a local model writes the summary.</li><li><strong>Transcribe files and links</strong>, such as a podcast or a YouTube video, into timestamped text.</li><li><strong>Other apps go quiet while you talk.</strong> A video in Safari or a song in Spotify is muted while the microphone records.</li></ul><h3>Lesson 1: Test the model with your own accent</h3><p>Most speech benchmarks use clean, read-aloud audio. Real people dictate in Indian, Nigerian or Scottish English, over AirPods, between video calls.</p><p>So I built a benchmark into the command-line tool and ran it on conversational speech:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*SxsO2q92QfO395hSYu211g.png" /></figure><p>On 300 video-call clips, Parakeet Ultra got 14.45% of words wrong against 15.96% for Parakeet TDT v3, so setup now recommends Ultra. Then the surprise: on 597 more Indian English clips, Ultra won before cleanup, but after Light cleanup the two models were level at about 12.1%. The older model wrote more filler words, and cleanup removed them.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*brfbtIiR_MZusM6S9jM1mw.png" /></figure><p>If an app lets you pick a model, test it with a minute of your own speech. A leaderboard recorded in a quiet room can’t tell you how it handles your accent.</p><h3>Lesson 2: A vocabulary list can make accuracy worse</h3><p>Here’s a bug I’m still a little embarrassed about. You say “cloud”. The app types “Claude”.</p><p>Speech models misspell names, so every dictation app lets you add a custom vocabulary. Parakeet supports a “phrase booster” that nudges recognition toward those words. The problem is that the nudge works everywhere, including where you didn’t want it. An early version turned “not like” into “Notion” and “Thank” into “Thai”.</p><p>I measured it. With 20 vocabulary terms that nobody in the clips actually said, the booster changed 44 of 300 transcripts and pushed the error rate from 16.2% to 17.4%.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*eGO_EvsnUcrtJXsvBXJLfw.png" /></figure><p>A stricter match brought 44 down to 5. The final fix uses macOS’s own spell checker. If the word you said is spelled correctly and only loosely resembles a vocabulary term, it stays:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*j6y--Ps0g8-nfYuoHcA4ng.png" /></figure><h3>Lesson 3: Cleanup is where the subtle bugs live</h3><p>Removing “um” is easy. Removing a <em>correction</em> is not.</p><p>“I live in Delhi, no sorry, I live in Ahmedabad” should become “I live in Ahmedabad”. But “Should we go, no, we should stay” is not a correction. An early version deleted half of “We read it in the last book, no, the whole thing with the houses was different”, because “no” looked like one.</p><p>Repeats have the same trap. “And then and then we shipped it” is a stutter. “More and more and more” is emphasis. “Twenty twenty five” is a year, and an early version dropped one “twenty”.</p><p>That’s why the default cleanup uses rules and the heavier AI rewrite is opt-in. A rule that fails, fails the same way every time, and I can write a test for it. A language model that rewrites your sentence can drop a word you needed, and you won’t notice until someone replies to the wrong thing.</p><h3>Lesson 4: Pasting into another app is a minefield</h3><p>Paste sounds trivial: put the text on the clipboard, press ⌘V, restore the clipboard. Then real use happens.</p><ul><li><strong>macOS Screen Sharing typed the letter “v”.</strong> It forwards Command to the remote Mac only when Command arrives as its own key press, so BetterWispr now sends the keystroke exactly as a physical keyboard would: Command down, V down, V up, Command up.</li><li><strong>You clicked another field.</strong> If you click into a different text box before the transcript is ready, pasting there would be wrong. The app checks that the same field still has focus, and copies the text instead if it doesn’t.</li><li><strong>Some fields block paste entirely</strong>, so the text lands on the clipboard with a visible card instead of disappearing.</li></ul><p>None of these show up in a demo.</p><h3>Lesson 5: Audio hardware is the real boss</h3><p>AirPods taught me the most. When they’re playing for your iPhone, they can hand themselves to the Mac a moment after recording starts and then send pure digital silence: every sample exactly zero. A real microphone never does that, because even a quiet room has a noise floor. So the recorder watches for it:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*QL0lXL-jWIESArF4EWaV6Q.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*cCDszCRmp7tQJ7tmzJt34Q.png" /></figure><p>After 2 seconds of exact zeros, the microphone restarts once. The first real sound sets the counter to nil for good, so a quiet speaker never triggers it.</p><p>The other AirPods bug was the fourth release on launch day. Recording from a Bluetooth headset’s microphone switches its playback to call quality, so your music sounds awful while you dictate. The default now uses the Mac’s built-in microphone when the lid is open.</p><h3>Lesson 6: AI meeting notes need a fact check</h3><p>Early meeting summaries had the problem every AI notetaker has. They listed introductions as action items, gave me tasks I never agreed to, and wrote “No explicit decisions were made” as a decision.</p><p>I changed the instructions so notes only count what someone actually agreed to or took on, then had a separate model judge each summary blind against its transcript. On seven recorded meetings, unsupported claims fell from about 4 per set of notes to 1.3.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*M53TIct-Z2PvTnpy01Zz7g.png" /></figure><p>Better, not perfect. Read AI notes before you send them, whichever app wrote them.</p><h3>How it compares</h3><p>I’m biased, so this table only uses what each product says on its own website.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*uY5c1GCEZay-8dLlBCXYuQ.png" /></figure><p>Where the others win:</p><ul><li><strong>Wispr Flow</strong> writes more polished text and works on your phone. If you dictate on four devices, BetterWispr helps with one.</li><li><strong>Superwhisper</strong> has more modes and lets you choose local or cloud models.</li><li><strong>MacWhisper</strong> is the better tool for transcribing interviews and recordings.</li><li><strong>Apple Dictation</strong> is already on your Mac and may be enough.</li></ul><p>And where BetterWispr falls short: it’s Mac only, with no sync. Cleanup, styles and voice commands work in English only, even though recognition covers 25 European languages with Parakeet and many more with Whisper. Speaker separation in meeting notes has only been measured on synthetic voices. And it’s young, so things still change fast.</p><h3>Before you pick one</h3><p>Run three tests on any dictation app before you commit: a minute of your normal speech through your normal headphones, a sentence where you correct yourself halfway, and a paragraph with a name the app has never heard. Those three tell you more than any comparison table, including mine.</p><p>On launch day I worried the speech model wouldn’t be good enough. It turned out to be the one part I never had to fix.</p><p>BetterWispr is free and Apache 2.0 licensed. The code, releases and full changelog are on GitHub at <a href="https://github.com/opennookorg/betterwispr">opennookorg/betterwispr</a>. If you try it and something breaks, opening an issue is the most useful thing you can do.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9664b5986880" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Building a Text-to-Speech System for 25K Users (Without Melting a Server)]]></title>
            <link>https://medium.com/@code_kartik/building-a-text-to-speech-system-for-25k-users-without-melting-a-server-78f9e21518cb?source=rss-c3102ac98f5b------2</link>
            <guid isPermaLink="false">https://medium.com/p/78f9e21518cb</guid>
            <category><![CDATA[genai]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[design-systems]]></category>
            <category><![CDATA[web-performance]]></category>
            <category><![CDATA[text-to-speech]]></category>
            <dc:creator><![CDATA[Kartik Labhshetwar]]></dc:creator>
            <pubDate>Sun, 15 Mar 2026 12:30:40 GMT</pubDate>
            <atom:updated>2026-03-15T12:30:40.924Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DN5DqOsqIO9nHxut-sdYLQ.png" /></figure><blockquote>TL;DR I built a scalable text-to-speech system for a 25K-DAU article reader by chunking articles into ~1,800 character segments, caching audio at both chunk and article levels, deduplicating in-flight requests, and enforcing strict concurrency limits.</blockquote><blockquote>This reduced TTS costs by ~60%, kept memory usage stable on a 4GB server, and fixed a tricky iOS Safari bug caused by incorrect MP3 Xing metadata when concatenating audio chunks.</blockquote><p>“Just add a play button.”</p><p>That was the feature request. The product was <a href="https://smry.ai/">smry.ai</a> an article reader serving 25,000+ daily active users and 583,000+ monthly actives. Users paste a URL, get a clean, distraction-free article. Simple.</p><p>Adding text-to-speech should have been simple too. Call a TTS API, get audio, stream it back. Maybe a weekend project.</p><p>It ended up being 900+ lines of server-side audio engineering, a custom MP3 frame parser, three tiers of caching, bounded concurrency control, and a deep dive into why iOS Safari interprets Xing VBR headers differently than every other browser on the planet.</p><p>Here’s every decision I made, what broke along the way, and what I’d do differently.</p><h4><strong>The Problem Space</strong></h4><p>Before writing any code, I mapped out the constraints:</p><ul><li><strong>25K DAU</strong> not a toy project. A bad TTS implementation would either crash the server or bankrupt us on API costs.</li><li><strong>Articles are long</strong> a 5,000-word article generates ~20 minutes of audio (~15MB at 64kbps). At 100 concurrent requests, that’s 1.5GB of in-flight audio buffers on a 4GB server.</li><li><strong>Users expect instant playback </strong>TTS APIs have latency. A 5,000-word article might take 15–30 seconds to fully synthesize. Nobody waits that long.</li><li><strong>Mobile is half the traffic</strong> and mobile browsers have their own opinions about audio playback.</li><li><strong>Budget matters </strong>TTS APIs charge per character. At scale, the difference between $5/1M chars and $30/1M chars is the difference between a feature that pays for itself and one that eats your margins.</li></ul><p>The naive implementation send full text, get full audio fails on every single one of these constraints. Here’s what actually works.</p><h4><strong>The Architecture: Three-Tier Caching + Bounded Concurrency</strong></h4><p>The system splits articles into ~1,800-character chunks, generates audio for each chunk independently, then concatenates the results. This chunking is the foundation everything else builds on.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Kela9GX2dYtkEA9k0so2YA.png" /></figure><p>Let me walk through each layer.</p><h4><strong>Layer 1: Per-Chunk Caching</strong></h4><p>Each chunk gets its own cache entry, keyed by “SHA-256(version + text + voiceId)”. The cache is a bounded LRU with a 150MB cap and 1-hour TTL.</p><p><strong>Why per-chunk instead of per-article?</strong></p><p>Because articles share content patterns. More importantly, partial cache hits are powerful if 8 out of 10 chunks are cached, you only make 2 API calls instead of 10. For long articles, this regularly cuts generation time by 60–80%.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*NJGFSclrx4ZgMPkuoaJx_A.png" /></figure><p>The trick here is using JavaScript’s `Map` insertion order as an implicit LRU queue. Delete and re-insert on access moves an entry to the “most recently used” position. No separate linked list needed.</p><p><strong>Cache Key Versioning</strong></p><p>The cache key includes a version prefix (`v3` currently). When the chunking algorithm or audio processing changes, bumping the version invalidates all stale entries without a cache flush:</p><ul><li><strong>v1:</strong> Original chunking</li><li><strong>v2:</strong> Switched to MP3 frame-based duration calculation (fixed alignment drift)</li><li><strong>v3:</strong> Xing VBR header fix (invalidates old concatenated audio)</li></ul><p>This is a pattern I’d use again. It’s cheaper than a cache flush and safer than hoping old entries are “close enough.”</p><h4><strong>Layer 2: Article-Level Caching</strong></h4><p>The second cache stores the fully merged audio + word alignment data for complete articles. Key: “SHA-256(cleaned_full_text + voiceId)”. 100MB cap, 2-hour TTL.</p><p><strong>Why a second cache if chunks are already cached?</strong></p><p>Because even with cached chunks, you still pay for:</p><p>1. Looking up 10+ chunk keys</p><p>2. Concatenating buffers</p><p>3. Stripping MP3 metadata from chunks 2+</p><p>4. Generating a Xing VBR header</p><p>5. Merging word alignment timings</p><p>For a re-listen (same article, same voice), the article cache serves the complete response in one lookup zero processing. At 25K DAU, popular articles get replayed constantly. This cache turns repeat requests from ~200ms of processing into ~2ms.</p><h4><strong>Layer 3: Request Deduplication</strong></h4><p>When a trending article goes viral, 50 users might hit “play” within the same second. Without dedup, that’s 50 identical API calls generating the same audio simultaneously.</p><p>The fix is a simple in-flight promise map:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jqhNehhlF8cPuQ_x1sd3kg.png" /></figure><p>The critical detail: <strong>release the concurrency slot before awaiting</strong>. If you hold the slot while waiting for someone else’s generation to finish, you’re blocking other users for no reason. Fifty users, one API call, one slot used, fifty responses served.</p><p>This single optimization is probably the highest-ROI code in the entire TTS system.</p><p><strong>Bounded Concurrency: The Part Everyone Skips</strong></p><p>Caching handles repeated requests. Concurrency control handles the thundering herd.</p><p>The system enforces three limits:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*fG34dVbz2a1GCVAuWEVwdg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*MwyCDXdhXqyNxtOITaCa-w.png" /></figure><p><strong>The `AbortSignal` propagation is crucial.</strong> It flows from the HTTP request all the way down to the TTS API `fetch()` call. When a user navigates away mid-generation, the API call cancels immediately — no wasted compute, no wasted money, no zombie slots.</p><p>Without bounded concurrency, a traffic spike from a viral article means 200 concurrent TTS requests, rate limits from the provider, 1.5GB of buffers, and a server restart. With it, excess requests queue for up to 15 seconds and get a clean 503 if the system is genuinely overloaded. Users see “TTS service is busy, try again shortly” instead of a blank error page.</p><h4><strong>The iOS Safari Bug That Changed Everything</strong></h4><p>Everything worked perfectly in Chrome, Firefox, and Android. Then I tested on an iPhone.</p><p><strong>The symptom:</strong> Audio plays for about 30 seconds (roughly one chunk), then the player shows “ended.” But audio keeps coming out of the speaker. The scrub bar freezes. The word highlights stop advancing. The duration shows “0:30” for a 10-minute article.</p><p>I spent an embarrassing amount of time thinking this was a React state bug.</p><p><strong>The actual root cause:</strong> MP3 file format metadata.</p><p>When you concatenate MP3 chunks, each chunk carries its own headers:</p><p>1. <strong>ID3v2 tag </strong>metadata (title, artist, etc.)</p><p>2. <strong>Xing/Info frame </strong>encoder metadata including <strong>total frame count</strong> and <strong>total byte count</strong></p><p>iOS Safari reads the Xing header from the <em>first</em> MPEG frame it encounters. It uses the “total frames” field to calculate duration. When that field says “83 frames” (one chunk), Safari thinks the entire stream is 30 seconds long even though there are 9 more chunks of audio after it.</p><p>Chrome ignores Xing headers and calculates duration from the actual stream length. Safari trusts the metadata. Both behaviors are technically valid per the MP3 spec. The spec just never anticipated concatenated chunks.</p><h4><strong>The Fix: MP3 Frame Surgery</strong></h4><p>Two steps:</p><p><strong>Step 1: Strip all metadata from chunks.</strong></p><p>Every chunk gets its ID3v2 tags and Xing/Info/VBRI frames removed before concatenation:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*J-e0QySiPErpvU6fLET_9A.png" /></figure><p>Note: “buffer.subarray()” instead of “buffer.slice()”. It returns a zero-copy view into the same memory important when you’re processing dozens of audio buffers and don’t want to double your allocation.</p><p><strong>Step 2: Generate a correct Xing header for the full stream.</strong></p><p>After stripping metadata from all chunks and concatenating them, I prepend a single Xing frame with the correct total frame count and byte count:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qSy9q_eSGKcRnq2oPstMIQ.png" /></figure><p><strong>Why “Xing” and not “Info”?</strong> Both are valid markers, but iOS Safari has had bugs where “Info” frames (intended for CBR streams) aren’t processed correctly. “Xing” (intended for VBR) works reliably across all versions I tested.</p><p>The TOC (Table of Contents) is also important — it’s a 100-entry seek table that maps percentages to byte offsets. Without it, scrubbing on iOS jumps to wrong positions because the browser can’t calculate byte offsets for VBR audio.</p><h4><strong>Duration Calculation: Another Subtlety</strong></h4><p>Before this fix, I calculated audio duration from the last word boundary’s end time. This works for single chunks but <strong>drifts on concatenated audio </strong>each chunk has trailing silence/padding that the word boundaries don’t account for. After 5 chunks, the highlights are running 2–3 seconds ahead of the audio.</p><p>The fix: parse actual MPEG frames to get exact duration:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*bAHtgx7L_l084DvgRmV_og.png" /></figure><p>This gives frame-accurate duration regardless of VBR bitrate changes across chunks. The word-level highlights now stay perfectly in sync for 20+ minute articles.</p><h4><strong>Word-Level Highlight Tracking</strong></h4><p>TTS without synchronized text highlighting feels like a podcast. With it, it feels like a reading assistant. This was the most complex client-side piece.</p><h4><strong>Character-to-Word Conversion</strong></h4><p>The TTS provider returns character-level timestamps. I convert these to word boundaries for the UI:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-6UMbucYNb_7um3Hl04_kQ.png" /></figure><p>The conversion groups consecutive non-whitespace characters into words, using each word’s first character start time and last character end time.</p><p>The non-obvious part: <strong>handling repeated words</strong>. The word “the” appears 50+ times in any article. A naive “text.indexOf(“the”)” would always match the first occurrence. Instead, I use forward-search from the last matched position:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6WIrzfLrNvpRJ3TkjCot5w.png" /></figure><h4><strong>Merging Alignments Across Chunks</strong></h4><p>Each chunk generates its own alignment timeline starting from 0. When merging, I offset each chunk’s timestamps by the cumulative duration of all previous chunks:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GavcqYeX1NSFCsvm2uMbTA.png" /></figure><p>Using the frame-parsed duration (not the last word’s end time) is critical here. The trailing silence in each chunk is real time that passes — skip it, and your highlights drift forward with every chunk boundary.</p><h4><strong>The Three-Headed Sync Monster</strong></h4><p>On desktop, “requestAnimationFrame” gives smooth 60fps highlight updates. On mobile, it’s not enough.</p><p>Here’s why: at MP3 chunk boundaries, mobile Safari sometimes fires a spurious “ended” event. RAF stops because the audio appears paused. “timeupdate” events stop firing. But the pre-decoded audio buffer keeps playing through the speaker. The user hears audio with no visual feedback.</p><p>The fix required three separate sync mechanisms:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*MwNO-vzYHgEE33SMhbG3Yg.png" /></figure><p>Three sync sources is not elegant. But it works on every device and browser I’ve tested, including the iPhone SE running iOS 16 that exposed the original bug. Sometimes the right solution is a belt, suspenders, <em>*and*</em> a safety pin.</p><h4><strong>User Scroll Independence</strong></h4><p>When TTS is playing and the user manually scrolls to a different part of the article, the auto-scroll should stop — not fight the user for control. But when they tap “scroll to current,” it should smoothly animate back to the highlighted word.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5yLsrUwTLuGN5UQtrEnlWQ.png" /></figure><p>The `programmaticScrollRef` flag distinguishes user scrolls from programmatic ones — without it, every auto-scroll triggers the “user scrolled away” state, creating an infinite loop.</p><h4><strong>Provider Evolution: What $25/1M Chars Taught Me</strong></h4><p>I didn’t start with the right TTS provider. The evolution:</p><p>1. <strong>Azure Speech Service</strong> high quality, WebSocket-based, but complex setup and inconsistent latency</p><p>2. <strong>Cartesia </strong>faster iteration, but costs added up at scale</p><p>3. <strong>ElevenLabs </strong>best voice quality, $15–30/1M chars. At 25K DAU, this was $600–1,800/month for TTS alone</p><p>4. <strong>Inworld AI </strong>$5/1M chars, 100ms median latency, character-level timestamps. The winner.</p><p>The abstraction I built during the Cartesia-to-ElevenLabs migration made the ElevenLabs-to-Inworld switch almost trivial. The `generateSpeechForChunk()` interface stayed the same only the HTTP call and response parsing changed.</p><p><strong>Lesson:</strong> If you’re integrating an external API, build the abstraction after your first provider switch, not before. You’ll know what actually varies vs. what you <em>think</em> will vary.</p><h4><strong>Memory Management: The Invisible Feature</strong></h4><p>On a 4GB server, TTS is the biggest memory threat. Each request can allocate 5–50MB of audio buffers. Without tracking, you find out about leaks when the health check restarts your server at 3 AM.</p><p>I built a memory tracking system with three tiers:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5rjvVulFDC1uGhcI2-J-NQ.png" /><figcaption>\</figcaption></figure><p><strong>Periodic monitoring </strong>runs every 30 seconds and logs RSS snapshots. Alerts fire at:</p><ul><li><strong>Warning:</strong> heap grew 20MB+ since last snapshot</li><li><strong>Error: </strong>RSS spike &gt;400MB in 30 seconds</li><li><strong>Critical:</strong> RSS &gt;1.5GB (health check will restart)</li><li><strong>Explicit GC assistance</strong> after large operations:</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_GUjGub8zAGWRvnu_y4OEg.png" /></figure><p>The cache stats are registered with the memory tracker and exposed via the “/health” endpoint:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*OblMjRE3QCfCEt_e-FK3gQ.png" /></figure><p>This isn’t glamorous code. But it’s the difference between a system you can debug in production and one that fails mysteriously under load.</p><h4><strong>Text Cleaning: The Unglamorous Necessity</strong></h4><p>Articles scraped from the web aren’t clean text. They contain “SKIP ADVERTISEMENT,” “Share on Twitter,” cookie consent text, and navigation labels. If you pass this through TTS, your user hears “Skip advertisement. Share this article on Facebook. Accept all cookies” in the middle of a paragraph about quantum computing.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wF48ADWrabXSa3ajPSwsuA.png" /></figure><p>The chunk splitter then respects sentence boundaries — never breaking mid-sentence unless a single sentence exceeds the 1,800-character limit (it happens with legal documents and academic papers). Force-splitting on word boundaries handles those edge cases without producing garbled audio.</p><h4><strong>What I’d Build Differently</strong></h4><p><strong>Start with bounded concurrency, not add it after.</strong> My first production deployment had no concurrency limits. It survived exactly 4 hours before a traffic spike caused an OOM. Build the semaphore before the feature.</p><p><strong>Use frame-accurate duration from day one. </strong>The word-boundary-based duration estimate seemed “close enough” in testing. It was off by 200–500ms per chunk barely noticeable for 2 chunks, completely wrong for 15. I wasted a week debugging “highlight drift” before realizing the duration source was the problem, not the sync logic.</p><p><strong>Don’t build the provider abstraction upfront.</strong> I originally planned a clean “TTSProvider” interface before writing a single API call. Then I switched providers three times, and the actual variation points were completely different from what I predicted. Build the abstraction after your first migration, when you know what actually changes.</p><p><strong>Test on real iPhones in your CI. </strong>The Xing header bug, the RAF stopping at chunk boundaries, the spurious “ended” events none of these appeared in Chrome DevTools’ device emulation. If half your traffic is mobile, you need a real device in the loop.</p><h4><strong>The Numbers</strong></h4><p>After all this engineering, the system runs comfortably at scale:</p><ul><li><strong>Cache hit rate:</strong> ~60% at the article level (popular articles get replayed)</li><li><strong>Cost reduction:</strong> 60% lower than the naive “generate everything, cache nothing” approach</li><li><strong>Memory footprint:</strong> 150–250MB steady state on a 4GB server</li><li><strong>Zero OOMs</strong> since adding bounded concurrency (it’s been months)</li></ul><h4><strong>Wrapping Up</strong></h4><p>Building for 25K DAU, 583k MAU forced me to think about every allocation, every cache, every concurrent request. The features that seem simple “add a play button”, “add a chat sidebar” are icebergs. The UI is 10% of the work. The other 90% is making it not crash under load, work on every browser, and cost less than the revenue it generates.</p><p>If you’re building something similar, start with bounded concurrency and two-level caching. Add real device testing to your CI. And always, always check what happens when your cache is cold and 100 users hit the same endpoint simultaneously.</p><p>The code isn’t perfect. But it works at scale, it degrades gracefully, and it doesn’t melt the server. Sometimes that’s enough.</p><p><em>I’m currently open to full-time and contract opportunities in full-stack engineering, AI infrastructure, and developer tools. If you’re building something interesting, let’s talk find me on </em><a href="https://www.linkedin.com/in/kartikcode/"><em>LinkedIn</em></a><em> or </em><a href="https://github.com/KartikLabhshetwar"><em>GitHub.</em></a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=78f9e21518cb" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How I Choose Problems, Solve Them, and End Up Getting Traction]]></title>
            <link>https://medium.com/@code_kartik/how-i-choose-problems-solve-them-and-end-up-getting-traction-89f259eadd4b?source=rss-c3102ac98f5b------2</link>
            <guid isPermaLink="false">https://medium.com/p/89f259eadd4b</guid>
            <category><![CDATA[ai-developer-workflow]]></category>
            <category><![CDATA[solving-real-problem]]></category>
            <category><![CDATA[developer-tools]]></category>
            <category><![CDATA[build-products]]></category>
            <category><![CDATA[side-project]]></category>
            <dc:creator><![CDATA[Kartik Labhshetwar]]></dc:creator>
            <pubDate>Thu, 04 Dec 2025 11:09:52 GMT</pubDate>
            <atom:updated>2025-12-04T11:19:59.903Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="How I Choose Problems, Solve Them, and End Up Getting Traction" src="https://cdn-images-1.medium.com/max/1024/1*ZcEyqay5byEH-SZtaPsJvg.png" /></figure><p>People overcomplicate “idea validation” like it’s some magical process. My method is far simpler and far more effective:</p><p><strong>I solve problems I’m personally sick of dealing with.</strong></p><p>Not hypothetical problems.<br>Not “startup ideas.”<br>Actual pain. Repetitive pain. Irritating pain.<br>If something annoys me every day, every week, or whenever I’m building, I don’t ignore it i build a tool to eliminate it. That’s the entire strategy.</p><p>And ironically, that’s why my projects get traction: if something is painful for me, it’s almost always painful for thousands of other developers too.</p><h3><strong>Let me explain this to you with some examples:</strong></h3><p>Example #1 → <strong>LazyCommit</strong>: Because Writing Commit Messages Sucks.</p><figure><img alt="lazycommit" src="https://cdn-images-1.medium.com/max/1024/1*xGxwxRREHigUK2FVxUn5Kw.png" /></figure><p>I built <a href="https://lazycommit.kartikk.tech/"><strong>LazyCommit</strong></a> for one reason: I was tired of writing commit messages.<br>Not “inspired,” not “experimenting with AI” just annoyed as hell.</p><p>So I automated the entire thing with an AI-powered commit generator.<br>Today, LazyCommit has <strong>130,000+ downloads on npm</strong>.</p><p>Did I plan that?<br>No. I just solved something that was repeatedly wasting my time.</p><p>Example #2 → <strong>Doable</strong>: Bulk Issue Management Without Losing My Mind</p><figure><img alt="doable" src="https://cdn-images-1.medium.com/max/1024/1*Oye4B9wNzvVfp7VyPp_WrA.jpeg" /></figure><p>Creating issues in Linear was turning into a mini-job.</p><p>Bulk actions? Slow. Repetitive tasks? Endless.</p><p>I wanted something where I could:</p><ul><li>create multiple issues with one prompt</li><li>delete in bulk</li><li>assign things instantly</li><li>avoid the manual grind entirely</li></ul><p>So I built. <a href="https://doable.kartikk.tech/"><strong>Doable</strong></a> a Linear-style UI + an AI agent built using Vercel SDKs.<br>Bring your own API keys, and it just works.</p><p>Today it has over 200+ signups, 2000 organic visitors until now.</p><p>Again, the pattern is the same:<br>Annoyance → automation → traction.</p><p>Example #3 → <strong>Quotick</strong>: Because Adding Backticks Manually Should Be Illegal</p><figure><img alt="quotick" src="https://cdn-images-1.medium.com/max/1024/1*2kyO3Fbv0k04bqE8n70nGw.png" /></figure><p>Adding backticks manually in code was driving me insane.<br>It’s repetitive, pointless, and easy to mess up.</p><p>So I built <a href="https://quotick.kartikk.tech/"><strong>Quotick</strong></a> a tool that automatically inserts backticks when you use ${} syntax.</p><p>Lightweight, fast, and exactly what I needed.</p><p>It now has <strong>500+ downloads across VS Code and Cursor</strong>, and it exists for one reason:</p><p>Manual backticks should not be a 2025 problem.</p><p>Example #4 → <strong>Stage</strong>: Because I Needed Better Showcase Screenshots</p><figure><img alt="stage" src="https://cdn-images-1.medium.com/max/1024/1*jde1VCtQl60Wm33rmsdRnw.png" /></figure><p>When I was building my portfolio, I needed clean screenshots that looked <em>good</em> not like a last-minute crop from my desktop.</p><p>I wanted something fast, aesthetic, and export-ready.</p><p>So I built <a href="http://stagee.art/"><strong>Stage</strong></a>, a screenshot showcase generator where you upload an image and instantly get a polished, portfolio-ready export.</p><p>This wasn’t a “big idea.” It was just a missing tool in my workflow.</p><p>Now it has over 12k visitors and over 5.5k images have been exported with 300+ stars on github, yes it’s completely open source → <a href="https://github.com/KartikLabhshetwar/stage"><strong>Github</strong></a></p><h3><strong>The Pattern You Should Notice</strong></h3><p>Every one of these projects came from the same formula:</p><ol><li><strong>Notice what you’re repeating</strong></li><li><strong>Notice what irritates you</strong></li><li><strong>Automate the hell out of it</strong></li><li><strong>Ship</strong></li><li><strong>Watch others adopt it because they’re annoyed by the same thing</strong></li></ol><p>This isn’t “idea genius.”<br>This is <strong>paying attention to what annoys you</strong> and having the skills to fix it.</p><h3>“Should I Use AI While Building?” : Yes, But Only If You Know What You’re Doing</h3><p>Let’s be real: today, using AI tools isn’t optional.</p><p>They speed up development massively <em>if</em> you already know the fundamentals.</p><p>If your CS basics and coding foundation are weak, building with AI is like building a house of cards one nudge and the whole thing collapses.</p><p>But if you have strong fundamentals and you’ve built similar projects before, AI becomes a force multiplier. You iterate faster, test faster, ship faster.</p><p>Just don’t expect AI to turn inexperience into skill. It won’t.</p><h3><strong>My Tech Stack (and Why You Shouldn’t Obsess Over Yours)</strong></h3><p>I build most of my stuff with:</p><ul><li><strong>Next.js/React.js</strong> for the frontend</li><li><strong>Node + Express sometimes FastAPI </strong>for backend logic</li><li><strong>Postgres + Prisma/Drizzle or MongoDB </strong>for database</li><li><strong>Better Auth</strong> for authentication</li></ul><p>Not because it’s “the best stack.”<br>But because I know it so well that I can ship fast without slowing down to Google every error.</p><p>Familiarity &gt; trendiness.<br>If you want speed, pick a stack you actually understand.</p><p>Hosting is trivial: Vercel, Netlify, Cloudflare whatever gets the job live quickly.</p><h3>The Real Takeaway</h3><p>If you want traction, stop chasing inspiration and start paying attention to your own friction.</p><p>your repeated annoyances are roadmap items.<br>your daily irritations are product opportunities.<br>your workflow struggles are problems millions share.</p><p>Build solutions for those, and the traction shows up naturally because you built something that actually matters.</p><p>You can connect with me on:</p><ul><li><a href="https://x.com/code_kartik">x</a></li><li><a href="https://www.linkedin.com/in/kartikcode">linkedin</a></li></ul><p><em>Thank you for Reading</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=89f259eadd4b" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[What is RAG? Why you should learn it?]]></title>
            <link>https://medium.com/@code_kartik/what-is-rag-why-you-should-learn-it-c9408f233086?source=rss-c3102ac98f5b------2</link>
            <guid isPermaLink="false">https://medium.com/p/c9408f233086</guid>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[ai-agent]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[rags]]></category>
            <category><![CDATA[langchain]]></category>
            <dc:creator><![CDATA[Kartik Labhshetwar]]></dc:creator>
            <pubDate>Wed, 12 Nov 2025 12:29:33 GMT</pubDate>
            <atom:updated>2025-11-12T12:29:33.272Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="retrieval augumented generation" src="https://cdn-images-1.medium.com/max/1024/1*Bs5Z_SIERhdSgnSId6y5CA.png" /></figure><p>Imagine asking a super-smart friend a question and getting an answer that sounds perfect, but turns out completely wrong. That’s how most large language models behave today. They’re confident, fluent, and occasionally full of it.</p><p>Why? Because an LLM only knows what it was trained on. It can’t look up new facts, check the latest data, or reference your internal documents.</p><p>That’s where <strong>Retrieval-Augmented Generation (RAG)</strong> comes in. It’s how you make an AI that <em>doesn’t just guess</em> it <em>looks things up</em> before answering.</p><h3>The one-sentence definition</h3><p>RAG = retrieve relevant documents from your knowledge base, then feed those documents plus the user query into an LLM so the reply is grounded in evidence rather than the model’s static training memory.</p><p>Think of it like this: you give the model a librarian (retriever) and a stack of relevant books (retrieved docs) before asking it to write an answer.</p><h3>Why RAG matters (and where plain LLMs fail)</h3><ul><li>LLMs are trained on a snapshot of the web. They don’t automatically know events/documents created after that snapshot.</li><li>They confidently fabricate facts to fill gaps (hallucinations).</li><li>Re-training or fine-tuning to update knowledge is expensive and slow.</li></ul><p>RAG solves this: add a small retrieval step to bring in fresh, domain-specific content at query time. You get up-to-date answers without retraining the giant model.</p><p>Real payoff: accurate help-desk replies, enterprise assistants that read your internal docs, compliance-aware legal answer bots, and research assistants that cite papers.</p><h3>RAG architecture</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ybzDaRQ5IrHyqlcdDb2HSQ.png" /></figure><p>Pipeline: Query → Embed → Retrieve → (optional Re-rank) → Generate → Return.</p><p>Analogy: user = question, query encoder = translates the question into librarian-speak, vector DB = library index, retriever = librarian fetching a few passages, LLM = writer who writes the answer while quoting the pages.</p><h4>Step-by-step:</h4><ol><li><strong>Query encoding: </strong>turn the user question into an embedding (dense vector).</li><li><strong>Retrieval: </strong>run k-NN on your vector store (FAISS, Chroma, Pinecone…) to get top-k chunks.</li><li><strong>(Optional) Re-rank: </strong>rerank those hits using a second, stronger signal (cross-encoder or lexical match).</li><li><strong>Generation: </strong>call the LLM with a prompt that includes the question + retrieved contexts.</li><li><strong>Response + citations: </strong>return the answer, ideally citing the passages used.</li></ol><h3>Tiny LangChain example (practical)</h3><p>Let’s make this real.</p><p>Here’s a minimal Python example using LangChain to build your own RAG pipeline:</p><pre># minimal RAG with LangChain (conceptual)<br>from langchain.document_loaders import TextLoader<br>from langchain.text_splitter import RecursiveCharacterTextSplitter<br>from langchain.embeddings import OpenAIEmbeddings<br>from langchain.vectorstores import Chroma<br>from langchain.chat_models import ChatOpenAI<br>from langchain.chains import RetrievalQA<br>from langchain.prompts import PromptTemplate<br><br># 1. load and split<br>loader = TextLoader(&quot;knowledge.txt&quot;)<br>docs = loader.load()<br>splitter = RecursiveCharacterTextSplitter(chunk_size=800, chunk_overlap=100)<br>chunks = splitter.split_documents(docs)<br><br># 2. embed + index<br>embeddings = OpenAIEmbeddings()   # swap in a local/open model if needed<br>vector_store = Chroma.from_documents(chunks, embeddings)<br><br># 3. make retriever (tune search params: k, score_threshold)<br>retriever = vector_store.as_retriever(search_type=&quot;similarity&quot;, search_kwargs={&quot;k&quot;:5})<br><br># 4. make an LLM and prompt (ask for explicit citations)<br>llm = ChatOpenAI(model=&quot;gpt-4o&quot;, temperature=0)<br>prompt = PromptTemplate.from_template(<br>    &quot;Use the documents below to answer the question. Cite the source chunk ID or short title for every factual claim.\n\n&quot;<br>    &quot;Documents:\n{context}\n\nQuestion: {query}\nAnswer:&quot;<br>)<br><br>qa = RetrievalQA.from_chain_type(<br>    llm=llm,<br>    chain_type=&quot;stuff&quot;,            # &quot;map_reduce&quot; or &quot;refine&quot; for large contexts<br>    retriever=retriever,<br>    return_source_documents=True,<br>    chain_type_kwargs={&quot;prompt&quot;: prompt}<br>)<br><br># 5. ask<br>res = qa({&quot;query&quot;: &quot;What are the early signs of glaucoma?&quot;})<br>print(res[&quot;result&quot;])<br>for d in res[&quot;source_documents&quot;]:<br>    print(&quot;SOURCE:&quot;, d.metadata.get(&quot;source&quot;, &quot;unknown&quot;), &quot;LEN:&quot;, len(d.page_content))</pre><p>Notes:</p><ul><li>k=5 is a decent starting point, tune per domain.</li><li>Use map_reduce or refine chain types when the retrieved text exceeds context limits.</li><li>Always ask the model to cite or list the passages it usedth at makes debugging and trust much easier.</li></ul><h3>Practical examples that make the idea concrete</h3><ul><li><strong>Medical assistant</strong><br>User asks: “What are early signs of diabetes?”<br>RAG retrieves paragraphs from authoritative medical docs and returns a short, cited answer (“increased thirst, frequent urination, fatigue” citing two WHO or ADA passages). The model isn’t inventing symptoms it’s summarizing documented sources.</li><li><strong>Customer support bot</strong><br>RAG pulls the product’s current FAQ, recent release notes, and the user’s account metadata, then answers with the exact command or setting plus a link to the relevant doc.</li><li><strong>Enterprise sales report</strong><br>Ask: “Show me last quarter’s sales by region.” RAG retrieves Q4 reports and spreadsheets (or summaries), the LLM synthesizes them, and you get a narrative plus numbers that match what’s in the docs.</li><li><strong>Multimodal RAG</strong><br>Your index can include image captions or audio transcripts. A multimodal retriever can find relevant image pages and the LLM can describe or quote the image text.</li></ul><h3>Common pitfalls (and how to fix them)</h3><p>Don’t build a RAG system and hope the model will behave. It’s an engineering problem.</p><ol><li><strong>Bad retrieval → bad answers</strong><br>If the retriever misses the right passages, the model will still produce an answer (and may hallucinate).<br><em>Fix:</em> tune embeddings, chunk size, and use overlapping chunks add metadata (titles, dates).</li><li><strong>Hallucinations still happen</strong><br>RAG reduces hallucinations but doesn’t eliminate them LLMs can still invent when context is contradictory or sparse.<br><em>Fix:</em> require citations, implement post-generation verification, or run a verification LLM that checks claims against retrieved docs.</li><li><strong>Latency</strong><br>More steps = more delay. Searching large indexes takes time.<br><em>Fix:</em> cache results for frequent queries, keep hot indexes in memory, and precompute embeddings for inbound docs.</li><li><strong>Context window limits</strong><br>Pasting too much text into the prompt blows past the model’s window.<br><em>Fix:</em> summarize long docs, use an extractive step to pick the most relevant sentences, or apply map_reduce style chains.</li><li><strong>Security &amp; privacy</strong><br>Your vector DB can leak internal content if not access controlled. Also watch prompt injection via bad user inputs.<br><em>Fix:</em> restrict access, encrypt data at rest, sanitize inputs, and vet retrieved docs before sending them to the model when operating in regulated domains.</li><li><strong>Maintenance overhead</strong><br>Indexes must be re-embedded when docs change. If you ignore this, your “fresh” answers will still be stale.<br><em>Fix:</em> automate indexing (batch or streaming updates), and keep timestamps in metadata so you can prefer recent docs.</li></ol><h3>Engineering best practices (no frills)</h3><ul><li><strong>Chunking:</strong> paragraph–level chunks (500–1,200 chars) with small overlaps work well. Test.</li><li><strong>Embeddings:</strong> use a good semantic embedder (higher quality = better retrieval). For low-cost options, test open models vs managed ones.</li><li><strong>Reranker:</strong> add a cross-encoder reranker if precision matters. First pass: fast approximate search, second pass: slow but accurate re-rank top N.</li><li><strong>Prompting:</strong> ask the LLM to cite source IDs and quote short passages. Force transparency.</li><li><strong>Monitoring:</strong> log retrieved docs, query embeddings, and model outputs. Evaluate retrieval recall and answer accuracy separately.</li><li><strong>Fallbacks:</strong> classify queries first if trivial, skip retrieval, if high-risk (medical/legal), add human review or extra verification.</li><li><strong>Testing:</strong> build test sets with expected source hits. Unit test retrieval and generation stages independently.</li></ul><h3>Quick checklist to launch a useful RAG (do this first)</h3><ul><li>Pick a small, high-value dataset (FAQs, policy docs, or a subset of internal knowledge).</li><li>Chunk and add metadata (title, doc id, date).</li><li>Index with reliable embeddings and a fast vector DB.</li><li>Build a retriever that returns top-k and write a prompt that <em>requires citations</em>.</li><li>Evaluate: for a set of 50 queries, confirm the correct passage is among top-k at least 90% of the time.</li><li>Add a reranker or verification LLM if precision &lt; 90%.</li><li>Deploy with monitoring, access control, and an update pipeline.</li></ul><h3>Simple evaluation metrics you should track</h3><ul><li><strong>Retrieval recall@k:</strong> fraction of queries where the ground truth doc is in top-k.</li><li><strong>Answer accuracy:</strong> human or programmatic check that the model’s facts match sources.</li><li><strong>Hallucination rate:</strong> percentage of claims that are unsupported by retrieved docs.</li><li><strong>Latency:</strong> average time per query (encode + search + generate).</li><li><strong>Freshness:</strong> percent of answers referencing docs older than your freshness threshold.</li></ul><h3>When RAG is the wrong tool</h3><ul><li>Tiny apps with no specialized or changing knowledge the LLM’s internal knowledge is fine.</li><li>Ultra-low latency requirements where adding retrieval costs are unacceptable and you cannot cache.</li><li>Systems where you can’t guarantee the quality of source documents (garbage in, garbage out).</li></ul><h3>The Core Idea to Remember</h3><p>RAG doesn’t make your model “smarter.”<br>It makes it <em>honest</em>.<br>By forcing the model to back up what it says with retrieved evidence, you get the best of both worlds:</p><p><strong>search + generation = grounded intelligence.</strong></p><p>So next time your LLM starts confidently guessing, ask yourself:<br>“Does it actually know this or did it just make it up?”</p><p>If it’s the latter, it’s time to give it RAG.</p><h3>TL;DR</h3><blockquote>RAG turns your LLM from a storyteller into a researcher.<br>It retrieves first, then generates giving you answers that are not just fluent, but <strong>factual</strong>.</blockquote><p>You can connect with me on:</p><ul><li><a href="https://x.com/code_kartik">twitter</a></li><li><a href="https://www.linkedin.com/in/kartikcode">linkedin</a></li></ul><p><em>Thank you for Reading</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c9408f233086" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to SSH Into Your Server (The Right Way) A Beginner-Friendly Guide with Pro Tips]]></title>
            <link>https://medium.com/@code_kartik/how-to-ssh-into-your-server-the-right-way-a-beginner-friendly-guide-with-pro-tips-cbd0e8855c9a?source=rss-c3102ac98f5b------2</link>
            <guid isPermaLink="false">https://medium.com/p/cbd0e8855c9a</guid>
            <category><![CDATA[ssh]]></category>
            <category><![CDATA[linux]]></category>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[developer-productivity]]></category>
            <category><![CDATA[cloud-computing]]></category>
            <dc:creator><![CDATA[Kartik Labhshetwar]]></dc:creator>
            <pubDate>Sat, 18 Oct 2025 18:51:19 GMT</pubDate>
            <atom:updated>2025-10-18T18:59:30.353Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="ssh into your server like a pro" src="https://cdn-images-1.medium.com/max/1024/1*4qiTZ_-hsbybbxbwqzgSEQ.png" /></figure><p>If you’re setting up a VPS, deploying an app, or managing a cloud server, you’ll need to SSH into your server. It’s one of the most fundamental skills in DevOps and backend development, yet it’s also one of the most misunderstood.</p><p>This guide will walk you through what SSH is, how to use it securely, and a few time-saving tricks that will make you feel like a seasoned sysadmin.</p><h3>What Is SSH and Why You Should Care</h3><p>SSH (Secure Shell) is a protocol that lets you securely connect to remote machines over an encrypted channel. Think of it as a secure remote terminal you type commands from your computer, and they execute safely on a server somewhere else.</p><h4>Here’s what SSH does behind the scenes:</h4><ul><li>Authenticates who you are (via password or key)</li><li>Encrypts everything you send</li><li>Prevents snooping, tampering, or man-in-the-middle attacks</li></ul><p>If you’ve ever used a cloud provider like AWS EC2, DigitalOcean, then SSH is how you log in.</p><h3>The Basic SSH Command</h3><p>Here’s the simplest possible SSH command:</p><pre>ssh username@server_ip</pre><p>Example:</p><pre>ssh ubuntu@192.168.1.10</pre><p>That’s it, SSH connects you to the server as the user ubuntu.</p><p>If this is your first time connecting, you’ll see a message asking if you trust the host. Type yes, and SSH will remember it for future connections.</p><h3>Why Password Login Is a Bad Idea</h3><p>Using passwords over SSH works but it’s outdated and insecure. Anyone can brute-force or guess a weak password. The better way is SSH key authentication, which is faster, safer, and automated.</p><h3>Setting Up SSH Key Authentication (Best Practice)</h3><h4>Step 1: Generate Your SSH Key Pair</h4><p>On your local machine, run:</p><pre>ssh-keygen -t rsa -b 4096 -C &quot;your_email@example.com&quot;</pre><p>This creates two files:</p><ul><li>Private key: ~/.ssh/id_rsa: never share this.</li><li>Public key: ~/.ssh/id_rsa.pub: this goes on the server.</li></ul><h4>Step 2: Copy Your Public Key to the Server</h4><p>Instead of manually editing files, use:</p><pre>ssh-copy-id username@server_ip</pre><p>Now try logging in:</p><pre>ssh username@server_ip</pre><p>No password prompt. No friction. Just instant, secure access.</p><h3>What’s Happening Under the Hood</h3><p>Here’s what really happens when you connect via SSH:</p><ol><li>Key Exchange: SSH negotiates encryption algorithms.</li><li>Authentication: Your public/private key pair verifies your identity.</li><li>Encryption: Every command and response is securely encrypted.</li></ol><p>That’s why SSH is trusted by everyone from indie developers to enterprise sysadmins.</p><h3>SSH Power Moves (Tips &amp; Tricks You’ll Actually Use)</h3><p>Once you’ve got the basics, here’s how to make SSH work for you instead of against you.</p><ol><li>Use ~/.ssh/config to Create Shortcuts</li></ol><p>Stop typing full commands. Create an SSH config file:</p><pre>nano ~/.ssh/config</pre><p>Add entries like this:</p><pre>Host myserver<br>    HostName 192.168.1.10<br>    User ubuntu<br>    IdentityFile ~/.ssh/id_rsa</pre><p>Now connect with a simple:</p><pre>ssh myserver</pre><p>Clean, readable, and efficient.</p><p>2. Prevent SSH Timeouts</p><p>Hate when your SSH session drops mid-command? <br>Add this to your SSH config:</p><pre>Host *<br>    ServerAliveInterval 60<br>    ServerAliveCountMax 3</pre><p>This keeps your connection alive by sending periodic “heartbeat” packets.</p><p>3. Avoid Re-Typing Passphrases (Use SSH Agent)</p><p>If your SSH key has a passphrase (it should), start an agent:</p><pre>eval &quot;$(ssh-agent -s)&quot;<br><br>ssh-add ~/.ssh/id_rsa</pre><p>Now the agent keeps your decrypted key in memory, so you don’t re-enter your passphrase every single time.</p><p>4. Tunnel Ports Like a Pro (Port Forwarding)</p><p>SSH can also securely tunnel traffic for example, accessing a remote database locally:</p><pre>ssh -L 5432:localhost:5432 user@server_ip</pre><p>Now your remote PostgreSQL database is accessible at localhost:5432. Secure and invisible to the outside world.</p><h3>Troubleshooting Common Issues</h3><ol><li>Permission Denied (publickey) → Check your key permissions:</li></ol><pre>chmod 600 ~/.ssh/id_rsa<br>chmod 700 ~/.ssh</pre><p>2. Timeouts or Connection Refused → Ensure SSH is running on the server:</p><pre>sudo systemctl status ssh</pre><p>3. Server IP Changed? → Remove the old fingerprint:</p><pre>ssh-keygen -R server_ip</pre><p>Keep this handy, it’ll save you time.</p><h3>Final Thoughts</h3><p>SSH isn’t just about remote access it’s the foundation of secure system administration. Learn it well, automate your setup with keys and configs, and you’ll save hours of frustration (and avoid security headaches).</p><p>Once you get comfortable, SSH becomes second nature, your invisible, secure bridge to every machine you manage.</p><blockquote>Pro Tip: If you ever catch yourself typing the same SSH command twice, automate it. SSH is powerful not because of complexity, but because it rewards efficiency.</blockquote><p>You can connect with me on:</p><ul><li><a href="https://x.com/code_kartik">twitter</a></li><li><a href="https://www.linkedin.com/in/kartikcode/">linkedin</a></li></ul><p><em>Thank you for Reading</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=cbd0e8855c9a" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>