<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Masum Osman Khan on Medium]]></title>
        <description><![CDATA[Stories by Masum Osman Khan on Medium]]></description>
        <link>https://medium.com/@masum26?source=rss-87ffc94184ab------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*IvwxmClIK1F51ba1-oXelw.jpeg</url>
            <title>Stories by Masum Osman Khan on Medium</title>
            <link>https://medium.com/@masum26?source=rss-87ffc94184ab------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sun, 22 Nov 2026 13:01:54 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@masum26/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[What Actually Happens If You Reply to no-reply@? Does no-reply@ Actually Have an Inbox?]]></title>
            <description><![CDATA[<div class="medium-feed-item"><p class="medium-feed-image"><a href="https://masum26.medium.com/what-actually-happens-if-you-reply-to-no-reply-does-no-reply-actually-have-an-inbox-593f14bd14ab?source=rss-87ffc94184ab------2"><img src="https://cdn-images-1.medium.com/max/1765/1*vvDNC2zfsBblWHFnG_9txQ.jpeg" width="1765"></a></p><p class="medium-feed-snippet">A technical look at why transactional email comes from an address nobody is listening to.</p><p class="medium-feed-link"><a href="https://masum26.medium.com/what-actually-happens-if-you-reply-to-no-reply-does-no-reply-actually-have-an-inbox-593f14bd14ab?source=rss-87ffc94184ab------2">Continue reading on Medium »</a></p></div>]]></description>
            <link>https://masum26.medium.com/what-actually-happens-if-you-reply-to-no-reply-does-no-reply-actually-have-an-inbox-593f14bd14ab?source=rss-87ffc94184ab------2</link>
            <guid isPermaLink="false">https://medium.com/p/593f14bd14ab</guid>
            <category><![CDATA[email]]></category>
            <category><![CDATA[systems-thinking]]></category>
            <category><![CDATA[smtp]]></category>
            <category><![CDATA[software-engineering]]></category>
            <dc:creator><![CDATA[Masum Osman Khan]]></dc:creator>
            <pubDate>Sat, 22 Aug 2026 06:13:28 GMT</pubDate>
            <atom:updated>2026-08-22T06:43:42.018Z</atom:updated>
        </item>
        <item>
            <title><![CDATA[The Problem With ‘All or Nothing’ in Distributed Systems: Saga vs 2PC Explained With Real-Life…]]></title>
            <description><![CDATA[<div class="medium-feed-item"><p class="medium-feed-image"><a href="https://blog.devops.dev/the-problem-with-all-or-nothing-in-distributed-systems-saga-vs-2pc-explained-with-real-life-4b73e5490534?source=rss-87ffc94184ab------2"><img src="https://cdn-images-1.medium.com/max/1920/1*xnnKGaF_S225re5iZYoUHw.jpeg" width="1920"></a></p><p class="medium-feed-snippet">The Battle of Distributed Transactions: Saga vs Two-Phase Commit (2PC)</p><p class="medium-feed-link"><a href="https://blog.devops.dev/the-problem-with-all-or-nothing-in-distributed-systems-saga-vs-2pc-explained-with-real-life-4b73e5490534?source=rss-87ffc94184ab------2">Continue reading on DevOps.dev »</a></p></div>]]></description>
            <link>https://blog.devops.dev/the-problem-with-all-or-nothing-in-distributed-systems-saga-vs-2pc-explained-with-real-life-4b73e5490534?source=rss-87ffc94184ab------2</link>
            <guid isPermaLink="false">https://medium.com/p/4b73e5490534</guid>
            <category><![CDATA[distributed-locks]]></category>
            <category><![CDATA[saga]]></category>
            <category><![CDATA[systems-thinking]]></category>
            <category><![CDATA[distributed-systems]]></category>
            <category><![CDATA[2pc]]></category>
            <dc:creator><![CDATA[Masum Osman Khan]]></dc:creator>
            <pubDate>Sun, 21 Jun 2026 09:13:01 GMT</pubDate>
            <atom:updated>2026-06-23T08:52:02.082Z</atom:updated>
        </item>
        <item>
            <title><![CDATA[JWT Signatures in Production: How Trust Is Actually Verified Behind Stateless Auth]]></title>
            <description><![CDATA[<div class="medium-feed-item"><p class="medium-feed-image"><a href="https://blog.devops.dev/jwt-signatures-in-production-how-trust-is-actually-verified-behind-stateless-auth-19d652b9fdf0?source=rss-87ffc94184ab------2"><img src="https://cdn-images-1.medium.com/max/1024/1*Q-jZ36RKyLF58QuV1_XLLQ.png" width="1024"></a></p><p class="medium-feed-snippet">JWT&#x200A;&#x2014;&#x200A;Three Parts, One Contract</p><p class="medium-feed-link"><a href="https://blog.devops.dev/jwt-signatures-in-production-how-trust-is-actually-verified-behind-stateless-auth-19d652b9fdf0?source=rss-87ffc94184ab------2">Continue reading on DevOps.dev »</a></p></div>]]></description>
            <link>https://blog.devops.dev/jwt-signatures-in-production-how-trust-is-actually-verified-behind-stateless-auth-19d652b9fdf0?source=rss-87ffc94184ab------2</link>
            <guid isPermaLink="false">https://medium.com/p/19d652b9fdf0</guid>
            <category><![CDATA[jwt-authentication]]></category>
            <category><![CDATA[security]]></category>
            <category><![CDATA[password-management]]></category>
            <category><![CDATA[stateless]]></category>
            <category><![CDATA[authentication]]></category>
            <dc:creator><![CDATA[Masum Osman Khan]]></dc:creator>
            <pubDate>Sat, 03 Jan 2026 12:23:58 GMT</pubDate>
            <atom:updated>2026-01-07T07:52:44.899Z</atom:updated>
        </item>
        <item>
            <title><![CDATA[USB-C: Reversible Signals, Lane Switching & The Genius of CC Pins]]></title>
            <description><![CDATA[<div class="medium-feed-item"><p class="medium-feed-image"><a href="https://masum26.medium.com/usb-c-reversible-signals-lane-switching-the-genius-of-cc-pins-1c21a50f920c?source=rss-87ffc94184ab------2"><img src="https://cdn-images-1.medium.com/max/1718/1*jHrIwqcofaTMssZ_0sbhcQ.jpeg" width="1718"></a></p><p class="medium-feed-snippet">A deep-dive into how one connector handles power, data, and video all at once</p><p class="medium-feed-link"><a href="https://masum26.medium.com/usb-c-reversible-signals-lane-switching-the-genius-of-cc-pins-1c21a50f920c?source=rss-87ffc94184ab------2">Continue reading on Medium »</a></p></div>]]></description>
            <link>https://masum26.medium.com/usb-c-reversible-signals-lane-switching-the-genius-of-cc-pins-1c21a50f920c?source=rss-87ffc94184ab------2</link>
            <guid isPermaLink="false">https://medium.com/p/1c21a50f920c</guid>
            <category><![CDATA[tech-deep-dive]]></category>
            <category><![CDATA[electronics]]></category>
            <category><![CDATA[unified-communication]]></category>
            <category><![CDATA[signal-processing]]></category>
            <category><![CDATA[hardware-design]]></category>
            <dc:creator><![CDATA[Masum Osman Khan]]></dc:creator>
            <pubDate>Sun, 07 Dec 2025 17:45:18 GMT</pubDate>
            <atom:updated>2025-12-08T07:36:37.069Z</atom:updated>
        </item>
        <item>
            <title><![CDATA[ গো (Go) রুটিনে এরর হ্যান্ডলিং ও আইডিওম্যাটিক (Idiomatic) প্যাটার্ন]]></title>
            <link>https://masum26.medium.com/%E0%A6%97%E0%A7%8B-go-%E0%A6%B0%E0%A7%81%E0%A6%9F%E0%A6%BF%E0%A6%A8%E0%A7%87-%E0%A6%8F%E0%A6%B0%E0%A6%B0-%E0%A6%B9%E0%A7%8D%E0%A6%AF%E0%A6%BE%E0%A6%A8%E0%A7%8D%E0%A6%A1%E0%A6%B2%E0%A6%BF%E0%A6%82-%E0%A6%93-%E0%A6%86%E0%A6%87%E0%A6%A1%E0%A6%BF%E0%A6%93%E0%A6%AE%E0%A7%8D%E0%A6%AF%E0%A6%BE%E0%A6%9F%E0%A6%BF%E0%A6%95-idiomatic-%E0%A6%AA%E0%A7%8D%E0%A6%AF%E0%A6%BE%E0%A6%9F%E0%A6%BE%E0%A6%B0%E0%A7%8D%E0%A6%A8-67f5ed9be4c4?source=rss-87ffc94184ab------2</link>
            <guid isPermaLink="false">https://medium.com/p/67f5ed9be4c4</guid>
            <category><![CDATA[golang]]></category>
            <category><![CDATA[idiomatic-go]]></category>
            <category><![CDATA[error-handling]]></category>
            <category><![CDATA[concurrency]]></category>
            <dc:creator><![CDATA[Masum Osman Khan]]></dc:creator>
            <pubDate>Tue, 07 Oct 2025 12:11:03 GMT</pubDate>
            <atom:updated>2025-10-07T12:11:03.747Z</atom:updated>
            <content:encoded><![CDATA[<p>Go ভাষার সবচেয়ে শক্তিশালী ফিচারের মধ্যে একটা হলো <strong>goroutine</strong> — অর্থাৎ হালকা ওজনের concurrent execution.<br>কিন্তু goroutine ব্যবহারের সময় অনেকেই একটা প্রশ্নে আটকে যান 👇</p><blockquote><em>“goroutine এর ভিতর error ধরবো কীভাবে?”<br>“আর go func() ব্যবহার না করে সরাসরি go myFunc() কেন কাজ করে না?”</em></blockquote><p>এই লেখায় আমরা খুব সহজ উদাহরণ ও বাস্তব analogy দিয়ে বুঝবো:</p><ul><li>goroutine থেকে error কিভাবে handle করতে হয়,</li><li>এবং কোনটা Go-র <strong>idiomatic</strong> বা “Go-way” solution।</li></ul><h3>শুরু করি বেসিক দিয়ে</h3><p>ধরা যাক একটা ফাংশন আছে, যা দুটি সংখ্যা ভাগ করে দেয়:</p><pre>func calculate(a, b int) (int, error) {<br>    if b == 0 {<br>        return 0, fmt.Errorf(&quot;division by zero&quot;)<br>    }<br>    return a / b, nil<br>}</pre><p>একদম সহজ, তাই না?<br>এখন তুমি এটা goroutine হিসেবে চালাতে চাও — যাতে main থ্রেড ব্লক না হয়।</p><h3>প্রথম চেষ্টা — result এবং error আলাদা চ্যানেলে পাঠানো</h3><pre>func main() {<br>    resultChan := make(chan int)<br>    errChan := make(chan error)<br><br>    go func() {<br>        result, err := calculate(10, 0)<br>        if err != nil {<br>            errChan &lt;- err<br>            return<br>        }<br>        resultChan &lt;- result<br>    }()<br><br>    select {<br>    case res := &lt;-resultChan:<br>        fmt.Println(&quot;Result:&quot;, res)<br>    case err := &lt;-errChan:<br>        fmt.Println(&quot;Error:&quot;, err)<br>    }<br>}</pre><p>✅ এই কোড <strong>চমৎকারভাবে কাজ করবে।</strong></p><p>এখানে যা হচ্ছে:</p><ul><li>goroutine আলাদাভাবে calculate() চালাচ্ছে।</li><li>যদি error হয়, তাহলে সেটা errChan-এ পাঠায়।</li><li>আর যদি সব ঠিক থাকে, তাহলে resultChan-এ রেজাল্ট পাঠায়।</li><li>select ব্লক অপেক্ষা করে — কোনটা আগে আসে সেই অনুযায়ী কাজ করে।</li></ul><h3>তাহলে সমস্যা কোথায়?</h3><p>যদিও এটা কাজ করে, কিন্তু <strong>idiomatic Go নয়</strong>।<br>অর্থাৎ — এটা “Go-way” না।</p><p>কারণ:</p><ol><li>দুইটা channel মানে দুইটা responsibility (error + result)।</li><li>অনেক goroutine থাকলে এক চ্যানেল মিস হয়ে গেলে memory leak হতে পারে।</li><li>কোডটা maintain করা কষ্টকর হয়।</li></ol><h3>💡 Go-এর Idiomatic উপায় — struct ব্যবহার করে একটাই channel</h3><p>Go কমিউনিটিতে প্রচলিত, “যদি সম্ভব হয়, result এবং error একসাথে পাঠাও।”</p><p>একটা ছোট struct তৈরি করো 👇</p><pre>type calcResult struct {<br>    value int<br>    err   error<br>}</pre><p>এবার একই কোডটা একটু পরিষ্কারভাবে লিখি:</p><pre>func main() {<br>    ch := make(chan calcResult)<br><br>    go func() {<br>        result, err := calculate(10, 0)<br>        ch &lt;- calcResult{value: result, err: err}<br>    }()<br>    res := &lt;-ch<br>    if res.err != nil {<br>        fmt.Println(&quot;Error:&quot;, res.err)<br>        return<br>    }<br>    fmt.Println(&quot;Result:&quot;, res.value)<br>}</pre><h3>এইটা কেন ভালো:</h3><ul><li>error ও result একই সাথে এক channel দিয়ে আসে।</li><li>কোন confusion নেই, কোন channel leak নেই।</li><li>কোড অনেক readable ও extendable (চাইলে আরও data যোগ করা যায়)।</li></ul><h3>Real-life analogy (বাস্তব উদাহরণে বোঝো)</h3><p>ধরো তুমি অফিসে কাজ করছো, আর দুইটা courier service ব্যবহার করছো:</p><ul><li>একটা শুধু success report দেয় 📦</li><li>আরেকটা শুধু failure report দেয় 🚫</li></ul><p>তুমি প্রতিদিন দুইটা সার্ভিসের মেইল চেক করছো।<br>😩 মাঝে মাঝে একটাতে রিপোর্ট আসে, আরেকটাতে আসে না — ঝামেলা!</p><p>এখন যদি দুটো মিলে <strong>একটাই courier সার্ভিসে success বা failure দুটোই জানায়</strong>,<br>তাহলে কাজটা হবে পরিষ্কার, দ্রুত এবং সহজ ✅</p><p>এই analogy টা Go চ্যানেলগুলোর ক্ষেত্রেও ঠিক তাই।</p><h3>একটু বাস্তব উদাহরণ — context সহ</h3><p>Production কোডে তুমি সময়সীমা বা cancellation handle করতে চাও।<br>তাহলে context.Context ব্যবহার করো 👇</p><pre>ctx, cancel := context.WithTimeout(context.Background(), 2*time.Second)<br>defer cancel()<br><br>ch := make(chan calcResult)<br>go func() {<br>    result, err := calculate(10, 0)<br>    ch &lt;- calcResult{value: result, err: err}<br>}()<br>select {<br>case &lt;-ctx.Done():<br>    fmt.Println(&quot;Timeout:&quot;, ctx.Err())<br>case res := &lt;-ch:<br>    if res.err != nil {<br>        fmt.Println(&quot;Error:&quot;, res.err)<br>    } else {<br>        fmt.Println(&quot;Result:&quot;, res.value)<br>    }<br>}</pre><p>এভাবে তুমি নিশ্চিত করতে পারো — <br>যদি goroutine hang হয়ে যায়, প্রোগ্রাম পুরোপুরি আটকে থাকবে না।</p><h3>আরেকটা বিষয় — কেন go processData(10, results) কাজ করে না?</h3><p>অনেকে ভাবে “results তো channel, সরাসরি পাঠালেই হয়!”<br>কিন্তু সমস্যা হলো: go keyword দিয়ে ডাকা ফাংশনের return value তুমি ধরতে পারো না।</p><pre>go processData(10, results) // ❌ কাজ করবে না, কারণ এর return তুমি ধরতে পারবে না।</pre><p>তাই anonymous function (closure) ব্যবহার করো 👇<br>এতে goroutine main scope-এর variable access করতে পারে।</p><pre>go func() {<br>    processData(10, results)<br>}()</pre><p>এটা হলো idiomatic, clean Go approach.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=67f5ed9be4c4" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Taming the Go Goroutine Flood: Why You Need a Semaphore]]></title>
            <link>https://masum26.medium.com/taming-the-go-goroutine-flood-why-you-need-a-semaphore-da8c5bd292de?source=rss-87ffc94184ab------2</link>
            <guid isPermaLink="false">https://medium.com/p/da8c5bd292de</guid>
            <category><![CDATA[concurrency]]></category>
            <category><![CDATA[golang]]></category>
            <category><![CDATA[semaphore]]></category>
            <category><![CDATA[go-waitgroup]]></category>
            <category><![CDATA[optimization]]></category>
            <dc:creator><![CDATA[Masum Osman Khan]]></dc:creator>
            <pubDate>Mon, 06 Oct 2025 12:19:55 GMT</pubDate>
            <atom:updated>2025-10-06T12:19:55.264Z</atom:updated>
            <content:encoded><![CDATA[<p>Go is blazingly fast. It makes launching thousands of concurrent operations — <strong>goroutines</strong> — trivial. This power is fantastic for CPU-bound tasks, but it quickly becomes a liability when dealing with external resources like third-party APIs or databases.</p><p>When your job is to process 5,000 records, you can’t simply launch 5,000 simultaneous network calls. You need a governor, a traffic cop, or, in programming terms, a <strong>Counting Semaphore</strong>.</p><p>Let’s dive into why this is essential and how a simple buffered channel is the best tool for the job.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*akGKrfoUYp_mRYZhzDz8Kw.jpeg" /></figure><h3>The Chaos of the Uncontrolled Burst</h3><p>Imagine your Go program as a massive, powerful <strong>floodgate</strong>. When you hit that for i := range metrics { go func() { ... } } loop, you open the floodgate wide.</p><p>If your batch size is large (say, 5,000), you create 5,000 network requests instantly. This results in two major failures:</p><ol><li><strong>External Overload (The API Panic):</strong> The external service you’re calling is not ready for 5,000 requests in 10 milliseconds. It quickly throws its hands up and sends back <strong>HTTP 429 “Too Many Requests”</strong> errors, failing most of your data processing job. This is called <strong>Rate Limiting</strong>.</li><li><strong>Local Exhaustion (The Socket Failure):</strong> Even your local operating system can’t handle it. Every network request opens a socket. Too many sockets opened at once can crash your program with a fatal error like <strong>“too many open files.”</strong></li></ol><p>To prevent this chaos, we must introduce <strong>Controlled Flow</strong>.</p><h3>The Go Standard: The Channel Semaphore</h3><p>In Go, we implement a counting semaphore using a <strong>buffered channel</strong> that holds empty struct values (struct{}). The size of the buffer determines the maximum number of concurrent workers, often set to a value like 50 or 100.</p><p>Let’s look at the core logic:</p><pre>const MaxConcurrentPushes = 50<br>// The semaphore is a buffered channel with a capacity of 50<br>sem := make(chan struct{}, MaxConcurrentPushes)</pre><pre>// ... inside the processing loop ...<br>// 1. ACQUIRE SLOT<br>sem &lt;- struct{}{} <br>// 2. START CONCURRENT WORK<br>go func(metric *AggregatedMetric) {<br>    // 3. RELEASE SLOT (guaranteed execution)<br>    defer func() { &lt;-sem }() <br>    // The actual network work happens here<br>    j.pushSingleEvent(ctx, metric) <br>}(aggMetric)</pre><h3>Analogy: The 50-Lane Toll Plaza</h3><p>Think of the buffered channel (sem) as a <strong>Toll Plaza with 50 open lanes</strong>.</p><ol><li><strong>Acquire (</strong><strong>sem &lt;- struct{}):</strong> When a car (goroutine) reaches the plaza, it drives into an open lane. If all 50 lanes are busy, the car <strong>stops and waits</strong> in line before the plaza. This pause prevents chaos.</li><li><strong>Release (</strong><strong>&lt;-sem in defer):</strong> Once the car has passed the toll and exited the plaza (the network call is complete), it sends a signal back. This immediately <strong>frees up a lane</strong>, allowing the next waiting car to enter.</li></ol><p><strong>Why is this the best method?</strong> It offers <strong>dynamic slot filling</strong>. The slots are refilled the millisecond they are freed, ensuring you maintain exactly 50 active network calls at all times. This maximizes throughput while remaining safely under the API’s limit.</p><h3>Alternatives and Their Trade-offs</h3><p>The channel semaphore isn’t the only way to manage concurrency, but the alternatives introduce specific drawbacks:</p><h3>A. Alternative 1: Rigid Chunking with sync.WaitGroup</h3><p>Instead of blocking on a channel, you can launch tasks in sequential groups (e.g., a group of 50). You use sync.WaitGroup to block the main loop until the entire group is done.</p><ul><li><strong>Pacing Problem (Rigid Barrier):</strong> This approach creates a rigid barrier. If 49 jobs finish in 10ms, but the 50th job is slow and takes 5 seconds, the program <strong>sits idle</strong> for 5 seconds, wasting the 49 available slots. It cannot start the next batch until the slowest task in the current block is finished.</li></ul><h3>B. Alternative 2: Time-Based Limiting with golang.org/x/time/rate</h3><p>If your external API constraint is measured in <em>time</em> rather than <em>simultaneity</em> (e.g., “Max 50 Requests Per Second (RPS)”), then a dedicated rate limiter is a better fit.</p><ul><li><strong>When to Use:</strong> Use this when you need to perfectly space out your requests over time. You call limiter.Wait(ctx)before launching each goroutine, which pauses execution until the next request is permitted according to your RPS configuration.</li></ul><h3>Conclusion: Choose the Right Tool</h3><p>For most API integration tasks where you need to limit the number of <strong>simultaneous connections</strong> to prevent socket exhaustion and resource strain, the <strong>Channel Semaphore</strong> is the simplest, most efficient, and most idiomatic Go solution.</p><p>It is the robust hostess that keeps your system stable, serving the maximum number of customers without overwhelming the kitchen.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=da8c5bd292de" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[From NAT to WebRTC: How the Internet Finds a Way to Connect Us & Solve The Invisible Puzzle Behind…]]></title>
            <link>https://masum26.medium.com/from-nat-to-webrtc-how-the-internet-finds-a-way-to-connect-us-solve-the-invisible-puzzle-behind-d75c1840fc68?source=rss-87ffc94184ab------2</link>
            <guid isPermaLink="false">https://medium.com/p/d75c1840fc68</guid>
            <category><![CDATA[video-streaming-service]]></category>
            <category><![CDATA[computer-networking]]></category>
            <category><![CDATA[webrtc]]></category>
            <dc:creator><![CDATA[Masum Osman Khan]]></dc:creator>
            <pubDate>Wed, 25 Jun 2025 10:24:13 GMT</pubDate>
            <atom:updated>2025-06-25T10:24:13.977Z</atom:updated>
            <content:encoded><![CDATA[<h3>From NAT to WebRTC: How the Internet Finds a Way to Connect Us &amp; Solve The Invisible Puzzle Behind Every Video Call</h3><p>All of us use NAT every day without realising it.</p><p>It runs inside our home router, letting all your devices — phone, laptop, TV — share a single public IP address. <em>Without NAT, each device would need its own unique public IP, which isn’t feasible given the limited number of IPv4 addresses and Price.</em></p><p>While NAT makes internet access convenient and affordable, it also creates challenges for direct peer-to-peer connections. Let’s explore how NAT works and why it complicates device-to-device communication.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*F7dTccLq7-ZCcSMnrWoJ4Q.jpeg" /></figure><h3>Why Was NAT Created?</h3><p>The short answer is:</p><blockquote><em>NAT was created to solve the shortage of public IP addresses and to provide a basic layer of security.</em></blockquote><h3>1. There Weren’t Enough IPv4 Addresses</h3><p>The IPv4 system only allows <em>approximately 4.3 billion unique IP addresses.</em> But today, there are far more devices — phones, laptops, servers, IoT sensors, and even connected toasters. Giving each one a public IP isn’t feasible.</p><p><strong>NAT lets multiple devices inside a private network share a single public IP.</strong></p><h4>Analogy: Shared Parking Spot</h4><p>Imagine there’s only one parking space outside a house (the public IP), but 20 people live inside (private IPs). To go out, each person signs out with the front desk and uses that space. Only responses to their trip are allowed back in.</p><p>That’s NAT.</p><h3>Security Benefits (As a Side Effect)</h3><p>While NAT wasn’t designed as a security system, it acts like one.</p><ul><li>Inbound traffic is blocked by default.</li><li>Only responses to outbound requests are allowed.</li></ul><p>This acts like a <strong>passive firewall</strong>. External threats can’t reach your devices unless the communication was initiated from inside the network. And I wish you know how much it is needed. The Invisible Cloak…</p><h3>But It Comes at a Cost…</h3><p>Applications like online games, peer-to-peer file sharing, and video calling need devices to connect directly. NAT makes that hard or impossible.</p><p>To work around this, we use technologies like <strong>STUN</strong>, <strong>TURN</strong>, and <strong>ICE</strong> (used in WebRTC). But before we get there, let’s understand the <strong>NAT traversal problem</strong>.</p><h3>The NAT Traversal Problem</h3><p>Let’s say Alice wants to call Bob. Both are behind NATs.</p><p>Alice tries to send a packet to Bob’s IP. But NAT blocks it, because Bob hasn’t contacted Alice yet. The reverse is true too. So <strong>they can’t see each other or reach each other directly</strong> — unless they have help.</p><h3>Analogy: Gated Buildings</h3><ul><li>Alice and Bob live in separate gated communities.</li><li>Visitors can’t enter unless they were invited first.</li><li>Neither Alice nor Bob can invite the other without knowing where to send the invite.</li></ul><p>So they’re stuck — unless a <strong>matchmaker</strong> helps them coordinate.</p><h3>Types of NAT — And Why Some Are Worse Than Others</h3><p>Understanding the kind of NAT matters a lot for NAT traversal.</p><h3>1. Full Cone NAT (Best for P2P)</h3><p>Once Alice sends a packet from 10.0.0.2:5000 to any IP, NAT opens a port (203.0.113.1:40000). Anyone can send to that external port.</p><h3>2. Restricted Cone NAT</h3><p>Only the IP Alice contacted (say, 8.8.8.8) can respond.</p><h3>3. Port-Restricted Cone NAT</h3><p>Even stricter — only 8.8.8.8:1234 can respond. Any other port is blocked.</p><h3>4. Symmetric NAT (Worst for P2P)</h3><p>NAT creates a unique mapping per destination. If Alice sends packets to two different destinations, her NAT uses <strong>different external ports</strong> for each. This breaks most NAT traversal techniques.</p><p>NAT Type Allows Inbound? Good for Hole Punching? Full Cone Yes (any IP/port) Yes Restricted Cone Yes (same IP) Yes Port-Restricted Cone Yes (same IP:port) Maybe Symmetric No No (needs relay)</p><h3>How NAT Traversal Works</h3><p>The solution is called <strong>UDP hole punching</strong>, often used by WebRTC.</p><h3>Step-by-step</h3><ol><li>Both Alice and Bob send a packet to a STUN server.</li><li>The server replies: “Your public IP is X.X.X.X and port Y.”</li><li>Alice and Bob exchange their public IP/port via a <strong>signalling server</strong>.</li><li>They <strong>both try to send</strong> a UDP packet to each other.</li><li>NATs see the outgoing packets and open return paths.</li></ol><p>If NAT type is friendly (like full cone), the connection succeeds. If not, a fallback <strong>TURN server</strong> relays traffic.</p><h3>What About IPv6?</h3><p>With IPv6, every device can have a <strong>globally unique public IP</strong>. That removes the need for NAT as an address-conserving mechanism.</p><p>But in practice:</p><ul><li>Most IPv6 networks <strong>still block unsolicited inbound traffic</strong>.</li><li>Firewalls mimic NAT behavior for safety.</li><li>Many applications (and users) still rely on IPv4.</li></ul><p>So while IPv6 makes NAT less necessary, we still <strong>need traversal techniques</strong> for real-world compatibility.</p><h3>Final Analogy</h3><p>Imagine every house had only one phone line (public IP), and all the family members (private IPs) had to take turns using it. No one could call them directly unless they had made a call first. To connect two such houses, you’d need a <strong>friend in the middle</strong> to coordinate the timing, identities, and contact paths.</p><p>That’s NAT traversal in a nutshell.</p><p>NAT was a necessary invention to save the internet from running out of addresses. But it introduced complexity in direct peer communication. Technologies like STUN, TURN, and ICE — alongside coordination via signalling servers — allow us to bypass NAT’s barriers. IPv6 might eventually simplify this, but until then, NAT traversal will remain a core challenge for real-time applications.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d75c1840fc68" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[A Slice of Humility — Discovering Go Generics in the slices Library]]></title>
            <link>https://masum26.medium.com/a-slice-of-humility-discovering-go-generics-in-the-slices-library-e01d65ef9625?source=rss-87ffc94184ab------2</link>
            <guid isPermaLink="false">https://medium.com/p/e01d65ef9625</guid>
            <category><![CDATA[slice]]></category>
            <category><![CDATA[libraries]]></category>
            <category><![CDATA[generic-programming]]></category>
            <category><![CDATA[golang]]></category>
            <dc:creator><![CDATA[Masum Osman Khan]]></dc:creator>
            <pubDate>Mon, 16 Jun 2025 12:02:11 GMT</pubDate>
            <atom:updated>2025-06-16T12:02:11.078Z</atom:updated>
            <content:encoded><![CDATA[<h3><strong>A Slice of Humility — Discovering Go Generics in the </strong><strong>slices Library</strong></h3><p>A few days ago, I had to dive into Go’s slices library. While browsing through the source code, I came across this function:</p><pre>func Max[S ~[]E, E cmp.Ordered](x S) E {<br>   if len(x) &lt; 1 {<br>      panic(&quot;slices.Max: empty list&quot;)<br>   }<br>   m := x[0]<br>   for i := 1; i &lt; len(x); i++ {<br>      m = max(m, x[i])<br>   }<br>   return m<br>}</pre><p>Even though I have over 4 years of experience working with Go, this code felt completely foreign to me at first glance. I genuinely felt dumb for not understanding what was going on — especially that</p><p>[S ~[]E, Ecmp.Ordered] part.</p><p>So, I took some time, sat down with LLMs, and started breaking it all apart.</p><p>What followed was one of the most insightful learning sessions I’ve had in a while. I gained a much deeper understanding of Go’s generics, type parameters, and how powerful (yet elegant) the new syntax can be.</p><p>I’m sharing my findings with you all in case you’ve ever felt the same way. Learning never stops — even after years of experience.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gjRUkoW6Kq00WrT8asfrhQ.jpeg" /></figure><h3>1. Decoding This Syntax: func Max[S ~[]E, E cmp.Ordered](x S) E</h3><h3>Your Observation:</h3><p>You noticed a function definition:</p><pre>func Max[S ~[]E, E cmp.Ordered](x S) E</pre><p>You understood the name is Max and the return type is E. But what&#39;s going on with S ~[]E, E cmp.Ordered, and x S?</p><h4>Explanation:</h4><p>This is a <strong>generic function</strong> using <strong>type parameters</strong>.</p><ul><li>S ~[]E means: S is a type that behaves like a slice of E. The ~ allows compatibility with types that are <em>based on</em> []E (e.g., custom slice types).</li><li>E cmp.Ordered means: E is a type that supports comparison (like &lt;, &gt;, etc.). cmp.Ordered is a constraint from Go&#39;s constraints package.</li><li>(x S) means: the function takes one argument x, which must be of the type S.</li></ul><h3>Analogy:</h3><p>Think of S as saying: <em>&quot;I want something slice-like (e.g., list of numbers).&quot;</em><br> And E is: <em>&quot;The kind of item in that list, which must be comparable like numbers or strings.&quot;</em></p><h3>2. New Function Declarations in Go</h3><h4>Traditional Go Functions:</h4><pre>func greet() {}<br>func (s Student) greet() {}</pre><p>The first is a regular function. The second is a method bound to a Student type.</p><h4>What’s New:</h4><pre>func Max[S ~[]E, E cmp.Ordered](x S) E</pre><p>Now we have a new way:</p><pre>func FunctionName[TypeParams](args...) returnType</pre><p>This introduces <strong>type parameters</strong> to the function, enabling generics.</p><h4>Are There Other Ways?</h4><p>Yes. Here’s a list:</p><ul><li>Regular function: func doSomething()</li><li>Method: func (r Receiver) Method()</li><li>Generic function: func Add[T any](a, b T) T</li><li>Generic method: func (s MyStruct[T]) Compute() T</li></ul><h3>3. Why Do We Need Type Parameters If We Can Define Types in Args?</h3><p>You asked: <em>“Why not just do: </em><em>func add(x int, y int)? Why bother with type parameters?&quot;</em></p><h4>The Power of Type Parameters</h4><p>Defining specific types (like int) limits the function to that one type.</p><pre>func addInt(x int, y int) int<br>func addFloat(x float64, y float64) float64</pre><p>With generics:</p><pre>func add[T constraints.Ordered](x, y T) T</pre><p>Works with int, float64, string, etc. No duplication. Still type-safe</p><h3>4. What Are the Equivalents in Other Languages?</h3><p>Let’s explore how generics or similar functionality looks in other popular languages, not just with syntax, but practical examples too.</p><h3>Java:</h3><pre>public class Utils {<br>    public static &lt;T extends Comparable&lt;T&gt;&gt; T max(T a, T b) {<br>        return a.compareTo(b) &gt; 0 ? a : b;<br>    }<br><br>    public static void main(String[] args) {<br>        System.out.println(max(10, 20));       // prints 20<br>        System.out.println(max(&quot;apple&quot;, &quot;dog&quot;)); // prints dog<br>    }<br>}</pre><p><em>Java uses </em><em>&lt;T extends Comparable&lt;T&gt;&gt; to ensure the values can be compared.</em></p><h3>C++:</h3><pre>template&lt;typename T&gt;<br>T max(T a, T b) {<br>    return (a &gt; b) ? a : b;<br>}<br><br>int main() {<br>    std::cout &lt;&lt; max(3, 5) &lt;&lt; std::endl;         // 5<br>    std::cout &lt;&lt; max(3.2, 2.8) &lt;&lt; std::endl;     // 3.2<br>    std::cout &lt;&lt; max(std::string(&quot;foo&quot;), std::string(&quot;bar&quot;)) &lt;&lt; std::endl; // foo<br>}</pre><p><em>C++ templates offer compile-time generic programming.</em></p><h3>Rust:</h3><pre>fn max&lt;T: Ord&gt;(a: T, b: T) -&gt; T {<br>    if a &gt; b { a } else { b }<br>}<br><br>fn main() {<br>    println!(&quot;{}&quot;, max(5, 3));        // 5<br>    println!(&quot;{}&quot;, max(&quot;cat&quot;, &quot;dog&quot;)); // dog<br>}</pre><p><em>Rust uses trait bounds (</em><em>T: Ord) to ensure comparability.</em></p><h3>Python &amp; JavaScript:</h3><p>No need for generics since they’re dynamically typed. <br>But tools like MyPy (Python) or TypeScript add optional type safety.</p><h3>5. Why Use Generics At All? (Besides Duplication)</h3><p>Let’s explore this with <strong>practical motivations</strong>:</p><h4>Reason 1: Reusability</h4><p>Instead of writing StackInt, StackFloat, etc., write:</p><pre>type Stack[T any] struct {<br>    items []T<br>}</pre><p>Now use it with any type:</p><pre>s := Stack[int]{}<br>s.items = append(s.items, 10)</pre><h4>Reason 2: Type Safety With Flexibility</h4><p>Compare:</p><pre>func show(x interface{})</pre><p>vs.</p><pre>func show[T any](x T)</pre><p>The second version is safer — no need for casting or runtime type guessing.</p><h4>Reason 3: Performance</h4><p>Using interface{} can slow things down due to boxing/unboxing and runtime checks. Generics can be compiled per type, so they run faster.</p><h4>Reason 4: Expressive APIs</h4><p>Instead of:</p><pre>func max(a, b int) int<br>func max(a, b float64) float64</pre><p>You do:</p><pre>func Max[T cmp.Ordered](a, b T) T</pre><h4>Reason 5: Functional Programming Utilities</h4><p>Generics allow reusable patterns like Map, Filter, Reduce:</p><pre>func Map[T any, R any](items []T, f func(T) R) []R</pre><h4>Reason 6: Better Tooling</h4><p>Static analysis, IDE hints, and tests all benefit from well-typed generic code.</p><p><strong>Final Thoughts</strong></p><p>Diving into unfamiliar parts of a language you thought you knew can feel overwhelming — but that’s also where the most meaningful growth happens. Whether you’re just starting out with Go or have years of experience, there’s always something new to uncover. If this post helped you untangle the mystery behind Go generics, feel free to share it with others who might be in the same boat.</p><p>Stay curious. Keep building. And never stop learning.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e01d65ef9625" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[MongoDB Cursors: Experiencing with analogies]]></title>
            <link>https://masum26.medium.com/mongodb-cursors-experiencing-with-analogies-9b84703ec960?source=rss-87ffc94184ab------2</link>
            <guid isPermaLink="false">https://medium.com/p/9b84703ec960</guid>
            <category><![CDATA[cursor]]></category>
            <category><![CDATA[mongodb]]></category>
            <category><![CDATA[database]]></category>
            <dc:creator><![CDATA[Masum Osman Khan]]></dc:creator>
            <pubDate>Wed, 16 Apr 2025 17:49:54 GMT</pubDate>
            <atom:updated>2025-04-16T17:49:54.370Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*G19DUIQTpj_s2bs4MbT3NQ.jpeg" /></figure><h3>What’s a Cursor, Anyway?</h3><p>In its simplest form, a database cursor is a pointer to the result set of a query. But this abstract concept has real performance implications.</p><p>Back in my SQL Server days, I used cursors sparingly — they were considered performance anti-patterns since SQL databases are optimized for set-based operations. MongoDB forced me to get intimate with cursor behavior because they’re the primary interface for query results, not a specialized tool.</p><p>Think of a cursor like your Kindle keeping track of your place in a book. You don’t download the entire book to your brain at once — you read page by page, but the device remembers exactly where you are.</p><h3>MongoDB Cursors: Behind the Scenes</h3><p>When you run a query in MongoDB, something fascinating happens:</p><ol><li>The server finds all matching documents</li><li>It creates a cursor that points to this result set</li><li>It sends back just an initial batch (around 101 documents by default)</li><li>The cursor remains open, waiting for you to ask for more</li></ol><p>This was a revelation to me. The query returns almost instantly, but most of your data might still be sitting on the server!</p><h3>The Two Paths: cursor.All() vs cursor.Next()</h3><h3>cursor.All(): The Iftar Spread Approach</h3><pre>var results []Document<br>if err := cursor.All(ctx, &amp;results); err != nil {<br>    return err<br>}</pre><p>Using cursor.All() reminds me of an Iftar spread during Ramadan at my family home in Bangladesh. The table is filled all at once with everything—dates, piyaju, beguni, halim, fruits, and several main dishes. It&#39;s a beautiful sight and convenient for breaking fast together, but sometimes the table can barely hold everything when we host extended family.</p><p>My team used cursor.All() everywhere because it&#39;s clean and straightforward. But that approach caused our dashboard to crash—we tried to load 500k records into memory simultaneously during peak usage.</p><h3>cursor.Next(): The Course-by-Course Approach</h3><pre>for cursor.Next(ctx) {<br>    var doc Document<br>    if err := cursor.Decode(&amp;doc); err != nil {<br>        return err<br>    }<br>    // Process each document individually<br>}</pre><p>After our production incident, we refactored critical paths to use cursor.Next(). This approach reminds me of how we eat at formal Bengali weddings—one course served after another, each fresh and perfectly timed. First the appetizers, then the main dishes, and finally the sweet mishti doi at the end.</p><p>The dashboard that previously crashed now handles millions of records without breaking a sweat.</p><h3>When to Use Each Approach</h3><p>Look, I’m not dogmatic about this. There are times for each approach:</p><h3>I Choose cursor.All() When:</h3><ul><li>I know my result set will stay small (config data, dropdown options)</li><li>I need to sort or process everything together anyway</li><li>I’m prototyping and want cleaner code</li></ul><h3>I Choose cursor.Next() When:</h3><ul><li>Working with user-generated content that grows unpredictably</li><li>Running analytics or reports on large collections</li><li>Building admin tools that might need to scan entire collections</li><li>Handling long-running operations where timeouts are a concern</li></ul><h3>Real Talk: Moving Houses</h3><p>The best analogy came to me last summer when moving across town. We had two options:</p><p>First attempt: Rent one giant moving truck, pack everything, and transport it all at once. This was our cursor.All() approach. It fell apart when we realized our apartment elevators couldn&#39;t handle the timing.</p><p>Second attempt: My partner’s SUV, multiple trips over a weekend. This was our cursor.Next() approach. We moved high-priority items first, could arrange things at the new place between trips, and avoided elevator timing issues.</p><h3>The Context Timeout Trap</h3><p>My biggest headache with MongoDB cursors has been context timeouts:</p><pre>ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)</pre><p>With cursor.All(), everything must complete in those 10 seconds—query execution, data transfer, and decoding.</p><p>With cursor.Next(), each incremental fetch has the full 10 seconds, making the whole operation more resilient.</p><p>When I start a new project now, I use cursor.Next() by default for anything touching user-generated data. The extra few lines of code are worth the peace of mind. I reserve cursor.All() for stable, bounded data sets where I genuinely need everything at once.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9b84703ec960" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Understanding “Quantisation” with examples and analogies: Public notebook]]></title>
            <link>https://masum26.medium.com/understanding-quantisation-with-examples-and-analogies-public-notebook-2b282e386c53?source=rss-87ffc94184ab------2</link>
            <guid isPermaLink="false">https://medium.com/p/2b282e386c53</guid>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[model-quantization]]></category>
            <category><![CDATA[quantization]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[llmops]]></category>
            <dc:creator><![CDATA[Masum Osman Khan]]></dc:creator>
            <pubDate>Sat, 22 Mar 2025 09:07:34 GMT</pubDate>
            <atom:updated>2025-03-22T09:07:34.142Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*B5uuh8OcFMcjNkm--ghKgg.jpeg" /></figure><p>Some weeks back, we, 3 friends were discussing various topics, and one keyword that came to mind from them was “Quantisation.” At that time, I didn’t know much about it, so I made a note to explore it further.</p><p>Recently, on my way home, I started a conversation with ChatGPT about quantisation. The first response I received was dry, textbook-like, and honestly, a bit overwhelming. But then I guided the conversation in a way that made quantisation clearer and much more approachable. Now, I’m here to share my understanding of “Quantisation” with you, using examples and analogies:</p><blockquote>Quantisation is like <strong>compressing</strong> a high-quality image into a smaller file size while keeping it <strong>as clear as possible</strong>.</blockquote><p>Imagine you have a <strong>4K image (large LLM model in full precision, FP32)</strong> but you want to reduce its size so that it loads faster. You can <strong>compress it</strong> into <strong>1080p (FP16), 720p (INT8), or even 480p (INT4)</strong>.</p><p>The lower the resolution, the smaller the file size, but the more details you might lose.</p><h3>Example 1: Quantisation in Daily Life (Analogy)</h3><p>Let’s say you’re measuring weight in a <strong>grocery store</strong>:</p><ul><li><strong>Full Precision (FP32)</strong> → The scale shows <strong>2.7563 kg</strong> (very detailed).</li><li><strong>Lower Precision (FP16)</strong> → The scale rounds to <strong>2.76 kg</strong> (slightly less detail).</li><li><strong>Lower Precision (INT8)</strong> → The scale shows <strong>2.8 kg</strong> (rounded to one decimal).</li><li><strong>Very Low Precision (INT4)</strong> → The scale shows <strong>3 kg</strong> (very rough estimation).</li></ul><p>Just like reducing the precision in a weighing scale, quantisation <strong>reduces</strong> how precisely numbers are stored in an LLM to save memory and computation.</p><h3>Example 2: Quantisation in an LLM</h3><p>Let’s say an AI model stores weights like this:</p><pre>Weight_1 = 2.7563<br>Weight_2 = -1.4328<br>Weight_3 = 0.9876</pre><p>If the model is in <strong>FP32 (Full Precision, 32-bit)</strong>, these numbers are stored with full accuracy. But if we quantise them:</p><pre>Precision     | Weight_1 | Weight_2 | Weight_3<br>--------------|----------|----------|---------<br>FP32 (32-bit) |  2.7563  | -1.4328  |  0.9876  <br>FP16 (16-bit) |  2.76    | -1.43    |  0.99  <br>INT8 (8-bit)  |  3       | -1       |  1  <br>INT4 (4-bit)  |  2       | -1       |  1</pre><p>Lower precision means <strong>less accuracy</strong>, but it speeds up processing!</p><h3>Example of Running an LLM With vs. Without Quantisation</h3><h3>1. Loading a Model Without Quantisation (Full-Precision FP32)</h3><p>This is the standard way to load an LLM (e.g., LLaMA-7B) <strong>without quantisation</strong>.</p><pre>from transformers import AutoModelForCausalLM, AutoTokenizer<br>import torch<br><br>model_name = &quot;meta-llama/Llama-2-7b-hf&quot;<br><br># Load model in full precision (FP32)<br>model = AutoModelForCausalLM.from_pretrained(model_name, torch_dtype=torch.float32, device_map=&quot;auto&quot;)<br>tokenizer = AutoTokenizer.from_pretrained(model_name)<br><br>print(model)  # Check the model size</pre><h3>What happens?</h3><ul><li>Model is <strong>7 billion parameters (~30GB RAM needed for FP32)</strong></li><li>Needs <strong>A100 or 3090+ GPU with 40GB VRAM</strong></li><li>Slow inference time</li><li><strong>Not feasible on a consumer GPU</strong></li></ul><h3>2. Loading the Same Model With Quantisation (8-bit)</h3><p>Now, let’s apply <strong>8-bit quantisation</strong> using bitsandbytes.</p><pre>from transformers import AutoModelForCausalLM, AutoTokenizer<br>import torch<br><br>model_name = &quot;meta-llama/Llama-2-7b-hf&quot;<br><br># Load quantized model in 8-bit (much smaller memory)<br>model = AutoModelForCausalLM.from_pretrained(model_name, load_in_8bit=True, device_map=&quot;auto&quot;)<br>tokenizer = AutoTokenizer.from_pretrained(model_name)<br><br>print(model)  # Check reduced model size</pre><h3>What changes?</h3><ul><li>Model size <strong>reduces to ~4GB (instead of 30GB)</strong></li><li>Can run on a <strong>RTX 3090 (24GB VRAM) or even lower GPUs</strong></li><li><strong>Inference speed increases 2x-3x</strong></li><li>Minimal loss in accuracy</li></ul><h3>3. Going Even Further: 4-bit Quantisation (QLoRA)</h3><p>If 8-bit is still too big, we can use <strong>QLoRA (4-bit quantisation)</strong>.</p><pre>from transformers import AutoModelForCausalLM, AutoTokenizer<br>import torch<br><br>model_name = &quot;meta-llama/Llama-2-7b-hf&quot;<br><br># Load quantized model in 4-bit<br>model = AutoModelForCausalLM.from_pretrained(model_name, load_in_4bit=True, device_map=&quot;auto&quot;)<br>tokenizer = AutoTokenizer.from_pretrained(model_name)<br><br>print(model)  # Check reduced model size</pre><h3>What changes?</h3><ul><li>Model size <strong>shrinks to ~2GB (instead of 30GB)</strong></li><li>Can <strong>run on a laptop GPU (e.g., RTX 2060, 16GB RAM)</strong></li><li><strong>Inference is 5x faster</strong> than FP32</li><li>Slight drop in accuracy, but still useful for most tasks</li></ul><h3>Comparison of Quantisation Levels: When to Use Quantisation?</h3><pre>Precision  | Bit  | Model Size | GPU/VRAM        | Speed  | Accuracy  <br>-----------|------|------------|-----------------|--------|----------  <br>FP32       | 32   | ~30GB      | A100, 3090 40GB | Slow   | No loss  <br>FP16       | 16   | ~15GB      | 3090, 4090 24GB | Faster | Tiny loss  <br>INT8       | 8    | ~4GB       | 3090, 3060 16GB | 2x     | Minor  <br>INT4       | 4    | ~2GB       | 2060, Laptop    | 5x     | Noticeable</pre><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2b282e386c53" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>