<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by AbdulRasheed Bakare on Medium]]></title>
        <description><![CDATA[Stories by AbdulRasheed Bakare on Medium]]></description>
        <link>https://medium.com/@abdulrasheedolakiitan?source=rss-f095285de912------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*UAJ1X-QQZPm9CRAm.jpg</url>
            <title>Stories by AbdulRasheed Bakare on Medium</title>
            <link>https://medium.com/@abdulrasheedolakiitan?source=rss-f095285de912------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Wed, 18 Nov 2026 09:05:26 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@abdulrasheedolakiitan/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Multi-Latent Attention Explained: How DeepSeek Broke the KV Cache Bottleneck]]></title>
            <link>https://medium.com/@abdulrasheedolakiitan/multi-latent-attention-explained-how-deepseek-broke-the-kv-cache-bottleneck-fce64edc6786?source=rss-f095285de912------2</link>
            <guid isPermaLink="false">https://medium.com/p/fce64edc6786</guid>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[deepseek]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <dc:creator><![CDATA[AbdulRasheed Bakare]]></dc:creator>
            <pubDate>Tue, 10 Feb 2026 18:27:25 GMT</pubDate>
            <atom:updated>2026-02-10T18:27:25.088Z</atom:updated>
            <content:encoded><![CDATA[<h3>Part 1: When Attention Became Memory-Bound</h3><p>In our previous article, we looked at DeepSeek-V2 and V3 models that proved you don’t need a fortune to train frontier AI. Today, we zoom into the smart battery that makes this possible: Multi-Latent Attention (MLA).</p><p>The crisis is simple: VRAM bloat.</p><p>But before we touch equations, we need to understand the problem deepseek was trying to solve: the “Context Crisis”. As we ask models to process much longer context, the KV cache of the model’s short-term memory becomes a massive dead weight. Memory scales linearly with tokens, not intelligence. Eventually, the memory required to “remember” a conversation exceeds the VRAM of even the most expensive GPUs, therefore, inference isn’t compute-bound anymore; it’s memory-bound.</p><p>DeepSeek’s MLA is the Blade Battery of AI: same range (intelligence), dramatically less weight (memory).</p><p><strong>The Result: A Structural Breakthrough</strong></p><p>Just as the Blade Battery allows a car to be lighter, safer, and travel further on the same amount of lithium, MLA allows DeepSeek-V2 and V3 to handle massive amounts of information without needing a terabyte of VRAM.<br>You aren’t just adding more memory; you are rethinking the blueprints so that the memory you have is 100% &quot;active&quot; and 0% &quot;dead weight.&quot;</p><h3>Part 2: The Mathematical ZIP File of Attention</h3><p>MLA doesn’t just share memory; it compresses attention itself using low-rank factorization</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/889/1*uSEYt0rsVsPXpiRAeSEJsg.jpeg" /><figcaption>Complete MLA formulation from the DeepSeek-V2 paper (Equations 37–47).<br>The blue-boxed vectors indicate the only tensors cached during inference.</figcaption></figure><p>The Symmetric Bottleneck: DeepSeek forces both sides of attention through a latent space (Latent 𝐂ᵠₜ and 𝐜ᵏᵛₜ)</p><ul><li>Latent Query 𝐂ᵠₜ: The hidden state ℎ𝑡 is compressed into a small vector(downward projection). (Formula 37)</li><li>Latent KV 𝐜ᵏᵛₜ : Keys and Values are jointly compressed into a shared latent vector(downward projection), it contains the ‘semantics’ of the token (Formula 41).</li><li>kₜᴿ : positional (RoPE) key, it tells the model where the token is</li></ul><p>By squeezing both the question and the memory through the same bottleneck, the model learns a hyper-dense representation of attention-relevant information.</p><p>Think of this as zipping the attention tensors.</p><p>But compression alone is not enough</p><h3>Part 3: The Unzipping Problem</h3><p>In a naive MLA implementation, you would store the latent vector 𝐜ᵏᵛₜ in the KV cache, but then reconstruct full content keys and values during inference:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/901/1*_ZFDAM1bPbBTmupzcRsVuw.jpeg" /><figcaption>(Formula 42) and (Formula 45)</figcaption></figure><p>This ruins everything.</p><p>You saved memory, but now you’re:</p><ul><li>rebuilding massive matrices every step</li><li>increasing compute</li><li>and killing inference speed.</li></ul><p>This is where DeepSeek makes its non-obvious leap.</p><h3>Part 4: Weight Absorption as a Mathematical Breakthrough</h3><p>This is the cornerstone of MLA’s efficiency. Let’s start from the naïve attention score:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/626/1*jcu53bCtDpK03EjR8Ghbcw.jpeg" /></figure><p>This formulation implies that, at inference time, the GPU must explicitly multiply by Wᵁᴷ effectively unzipping the latent KV.</p><h3>The Insight: Associativity Saves the Day</h3><p>Matrix multiplication is associative:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/500/1*xc4T5EtSkD6q117d1WjIBA.jpeg" /></figure><p>DeepSeek reorders the computation.</p><p>Instead of expanding the latent KV and then dotting with the query, they absorb the up-projection weights into the Query path. The Query learns how to read the compressed latent space directly</p><p>Formally, Wᵁᴷ is absorbed into Wᵁᴽ, and Wᵁⱽ is absorbed into Wᴼ</p><h3>The Consequence</h3><ul><li>Content keys kⱼᶜ</li><li>Content values vⱼᶜ are never reconstructed.</li></ul><p>The model performs the entire attention dance in the compressed latent Space.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/883/1*QLbFIwpQ04ZVWWBtSPo3_A.jpeg" /><figcaption>MLA computation flow diagram.<br>Only the latent KV vector 𝐜ᵏᵛₜ and positional key kₜᴿ are cached. Content keys and values are never materialized</figcaption></figure><h3>Part 5: Decoupled RoPE and Positional Stability</h3><p>Position usually ruins compression because RoPE entangles dimensions.</p><p>DeepSeek avoids this by splitting attention into two streams:</p><ul><li>Content stream (qₜ,ᵢᶜ , kⱼ,ᵢᶜ)<br>Compressed, latent, absorbed.</li><li>Position stream (qₜ,ᵢᴿ , kₜᴿ)<br>Small, uncompressed, handles where.<br>(Formulas 40 &amp; 44)</li></ul><p>These streams only meet at the final attention score, keeping compression intact while preserving positional accuracy.</p><p>This separation is visible in the architecture diagram above.</p><h3>Part 6 : From Equations to Code</h3><p>Here’s the associative law materialized in PyTorch:</p><h3>Constants:</h3><ul><li><strong>H (Heads):</strong> 128</li><li><strong>qk_nope_head_dim (Content):</strong> 128</li><li><strong>qk_rope_head_dim (Position):</strong> 64</li><li><strong>kv_lora_rank (ZIP size):</strong> 512</li></ul><pre># --- Configuration Constants (DeepSeek-V3 Official Values) ---<br># H = 128                   # Number of Attention Heads<br># qk_nope_head_dim = 128       # Content (Non-Positional) Key dim<br># qk_rope_head_dim = 64        # Position (RoPE) Key dim<br># kv_lora_rank = 512           # Latent Compression Rank (The ZIP size)<br><br># 1. Query path: Compressing and splitting<br># q shape: [Batch, Seq, Heads, 128 + 64]<br>q = self.wq_b(self.q_norm(self.wq_a(x)))<br>q_nope, q_pe = torch.split(q, [self.qk_nope_head_dim, self.qk_rope_head_dim], dim=-1)<br>q_pe = apply_rotary_emb(q_pe, freqs_cis)<br><br># 2. KV ZIP file: Joint Compression<br># kv_latent shape: [Batch, Seq, 512] | k_pe shape: [Batch, Seq, 64]<br>kv = self.wkv_a(x)<br>kv_latent, k_pe = torch.split(kv, [self.kv_lora_rank, self.qk_rope_head_dim], dim=-1)<br><br># 3. Absorption Magic: Merging Up-Projection into the Query<br># wkv_b shape: [Heads, 256, 512] (Stacked Key and Value unzippers)<br>wkv_b = self.wkv_b.weight.view(self.n_local_heads, -1, self.kv_lora_rank)<br><br># q_nope_absorbed: [Batch, Seq, Heads, 512] <br># The Query absorbs W_UK to talk to the latent cache directly<br>q_nope_absorbed = torch.einsum(&quot;bshd,hdc-&gt;bshc&quot;, q_nope, wkv_b[:, :self.qk_nope_head_dim])<br><br># 4. Cache ONLY the Blue Boxes (VRAM Storage)<br># Only the 512-dim latent and the 64-dim RoPE key are stored<br>kv_cache[...] = kv_norm(kv_latent) <br>pe_cache[...] = k_pe.unsqueeze(2)  <br><br># 5. Scores: The final search in latent space<br>content_scores = torch.einsum(&quot;bshc,btc-&gt;bsht&quot;, q_nope_absorbed, kv_cache)<br>rope_scores    = torch.einsum(&quot;bshr,btr-&gt;bsht&quot;, q_pe, pe_cache)<br>scores = (content_scores + rope_scores) * self.scale</pre><h3>Part 7: The Architectural Lesson from DeepSeek-V2</h3><p>In The BYD Moment in AI, we argued that DeepSeek-V2 marked a shift away from brute-force scaling toward efficiency-first design. Multi-Latent Attention is where that philosophy becomes concrete.</p><p>DeepSeek-V2 does not treat MLA as an optimization layered on top of attention. It is a structural decision driven by a simple reality: at long context, memory, not compute is the bottleneck. By reordering the mathematics of attention and absorbing the key and value projections into the query and output paths, DeepSeek eliminates the need to ever materialize content keys and values during inferences.</p><p>The result is not approximate attention, fewer heads, or reduced expressiveness. It is full multi-head attention with a KV cache that stores only what is essential: the latent memory vector and the positional signal.</p><p>This is the real lesson of DeepSeek-V2 and V3. Progress didn’t come from more GPUs or heavier hardware, it came from asking which tensors actually need to exist. The BYD moment wasn’t about doing more with less compute. It was about doing more by changing the blueprint.</p><p>References</p><p><a href="https://arxiv.org/pdf/2405.04434">DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=fce64edc6786" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The BYD Moment in AI]]></title>
            <link>https://medium.com/@abdulrasheedolakiitan/the-byd-moment-in-ai-438bfb8581ee?source=rss-f095285de912------2</link>
            <guid isPermaLink="false">https://medium.com/p/438bfb8581ee</guid>
            <dc:creator><![CDATA[AbdulRasheed Bakare]]></dc:creator>
            <pubDate>Fri, 30 Jan 2026 18:37:27 GMT</pubDate>
            <atom:updated>2026-01-30T18:37:27.541Z</atom:updated>
            <content:encoded><![CDATA[<p>In the automotive world, the &quot;BYD Moment&quot; happened when the industry realized that the biggest threat to legacy carmakers wasn’t a flashier luxury brand, but a company that had reimagined the battery, the most expensive, heaviest of an electric car — making it cheaper, thinner, and more efficient.</p><p>While everyone else was focused refining gasoline engines or incremental EV improvements, BYD changed the math of electric vehicles.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*u0uAxwIjxkRmcRUhb87sgw.webp" /><figcaption>The BYD Han – sleek and sporty flagship sedan (Image: Official BYD Auto)</figcaption></figure><p>In the AI world, DeepSeek just had its BYD moment</p><h3>From Brute Force to Engineering Elegance.</h3><p>In the early years, the &quot;OpenAI-Google-Meta&quot; triumvirate defined the pace of progress. Their strategy was largely &quot;Brute Force&quot; approach: more data, more GPUs, and more electricity. Cutting-edge LLMs were assumed to be a rich man’s game, confined to a handful of labs with unlimited checkbooks.</p><p>​Then came DeepSeek-V2 and V3.</p><p>​While much of the public attention focused on the &quot;reasoning&quot; capabilities of DeepSeek-R1, the real revolution was happening under the hood. DeepSeek didn’t just build a bigger model; they out-engineered the incumbents by rethinking the &quot;battery&quot; of large language models: the KV Cache.</p><h3>​The Engineering Obsession: Shaving the Grams</h3><p>​In the 1950s, cars were massive slabs of steel. They were durable, but heavy, inefficient, and feel-hungry. Modern EVs from BYD or Tesla rely on advanced composites and tightly integrated battery systems to extract maximum efficiency from every kilogram.</p><p>​The engineers at DeepSeek looked at the &quot;heavy steel&quot; of modern LLM models like GPT-4 or Llama 3 and noticed a massive weight problem. Specifically, the KV Cache.</p><p>​ Every time an LLM generates a response to you, it has to store a massive amount of data in its &quot;short-term memory&quot; (the KV Cache) to retain conversational context. In traditional Multi-Head Attention (MHA), this memory (the KV Cache) is bloated and becomes a growing memory burden. As context length increases, inference slows, GPU memory usage explodes and infrastructure costs skyrocket.</p><p>The industry’s first serious attempt at weight reduction was Grouped Query Attention (GQA)</p><h3>&quot;Can we do better than GQA?&quot;</h3><p>GQA forces multiple query heads to share the same keys and values. It’s a sensible compromise — like switching from individual cars to a carpool lane. Memory usage drops, but at the cost of representational fidelity. The model effectively “squints” to save resources.</p><p>DeepSeek’s engineers weren’t satisfied with squinting.</p><p>Instead, they asked a more radical question:</p><p><em>What if we didn’t share memory — what if we compressed it?</em></p><h3>Enter: Multi-Head Latent Attention (MLA)</h3><p>​This was the &quot;Voilà!&quot; moment.</p><p>With Multi-Head Latent Attention (MLA), DeepSeek introduced a fundamentally different approach to managing attention memory. Rather than storing full K/V tensors to every head, ​MLA projects and then into a low-rank latent space, preserving essential information while dramatically reducing size.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*czYNPnJIdn72SGiP2dO1wQ.jpeg" /><figcaption>Multi-Head Latent Attention (MLA) Architecture — DeepSeek-V2 (Source: DeepSeek-V2 Technical Report)</figcaption></figure><p>Think of it as high-fidelity ZIP compression for attention memory. MLA allows the model to:</p><ul><li>Shrink the Memory: It stores a tiny &quot;latent vector&quot; that is even smaller than what GQA uses.</li><li>Maintain the Accuracy: Unlike GQA, it doesn’t &quot;squint.&quot; It keeps the high-definition detail of the original Multi-Head Attention.</li><li>Boost the Speed: It uses a mathematical &quot;absorption trick&quot; that allows the GPU to process data faster than almost any other model in existence.</li></ul><p>The result: lighter memory, faster execution, no loss in capability</p><h3>Smart Power Management: The Muscle</h3><p>Reducing weight was only half the battle. DeepSeek also needed a way to use compute efficiently. This is where DeepSeekMoE (Mixture of Experts) comes in.</p><p>In a traditional dense model, every token activates the entire network — all parameters firing at once, regardless of task complexity. It’s like flooring the accelerator just to reverse out of a driveway.</p><p>DeepSeek-V3 takes a different approach.</p><p>Despite having 671 billion parameters, it activates only about 37 billion parameters per token.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*J_0BR7iqJAbyxAUY3oxCbA.jpeg" /><figcaption>DeepSeekMoE Architecture with Routed and Shared Experts (Source: DeepSeek-V2 Technical Report)</figcaption></figure><h3>​The Selective Power Strategy</h3><p>Imagine an EV smart enough to disable 90% of its systems when cruising on a flat road.</p><ul><li>Dense models: Every subsystem runs at full power, all the time.</li><li>MoE models: A gating network selectively activates only the experts required for the current input.</li></ul><p>​DeepSeek refined this with two secret engineering tricks:</p><ol><li>​ Fine-Grained Experts: Instead of a few massive experts, DeepSeek deploys hundreds of small, specialized ones — each optimized for a narrow class of tasks. This fine granularity allows the model to activate exactly what it needs and nothing more.</li><li>​Shared Experts (The &quot;Always-On&quot; Systems): Some capabilities — grammar, syntax, general reasoning — are needed every time. DeepSeek assigns these to shared experts that remain active, freeing specialized experts to focus on harder, domain-specific problems.</li></ol><h3>​The Result: A New World Order</h3><p>​By combining MLA (compressed memory) and DeepSeekMoE (selective compute), DeepSeek achieved what many thought impossible:<br>state-of-the-art intelligence at a fraction of the cost.</p><p>This is the AI equivalent of building a supercar that performs like a hypercar but runs on the battery budget of a family sedan.</p><p>The BYD moment has officially arrived.</p><h3>Conclusion: The Future is Lean</h3><p>DeepSeek’s breakthrough isn’t just a better model — it’s a shift in philosophy. For years, progress meant building larger, heavier engines. DeepSeek proved that intelligence scales faster when you rethink the blueprints.</p><p>Optimize the battery. Optimize the power flow.<br>The future of AI isn’t just bigger — it’s smarter, leaner, and engineered.</p><p>References</p><ul><li><a href="https://arxiv.org/abs/2405.04434">DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=438bfb8581ee" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Same Engine, Sharper Handling: How LLaMA Refined the GPT-Style Transformer]]></title>
            <link>https://medium.com/@abdulrasheedolakiitan/same-engine-sharper-handling-how-llama-refined-the-gpt-style-transformer-e332022a8a2b?source=rss-f095285de912------2</link>
            <guid isPermaLink="false">https://medium.com/p/e332022a8a2b</guid>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <dc:creator><![CDATA[AbdulRasheed Bakare]]></dc:creator>
            <pubDate>Fri, 23 Jan 2026 17:23:06 GMT</pubDate>
            <atom:updated>2026-01-23T17:24:08.980Z</atom:updated>
            <content:encoded><![CDATA[<p>You’re standing at a busy intersection when something sleek glides into view. Low stance. Clean lines. That unmistakable premium presence. From a distance, you’re sure it’s a Mercedes-Benz, the kind of car that whispers luxury, status, and someone else is paying for the maintenance.</p><p>You think, that’s the GPT of the road.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*VHOSWO_-nArEjqDrunAOdA.jpeg" /></figure><p>But as it draws closer, the illusion cracks. The engine note is sharper. The stance more aggressive. And then you see the badge, not the silver star, but the blue-and-white propeller.</p><p>It’s a BMW.</p><p>From afar, the confusion makes sense. Both are built on elite engineering. But up close, the difference is philosophy. One optimizes for polish and abstraction. The other wants the driver involved.</p><p>That distinction maps cleanly onto modern language models.</p><p>If you’ve worked through GPT-2, LLaMA 2 and LLaMA 3 won’t feel foreign. They’re decoder-only Transformers, trained autoregressively, built on the same foundational blueprint. The differences aren’t about what they are, they’re about how they’re engineered and who they’re for.</p><p>So let’s park the analogy here and look at the machinery.</p><h4>How do GPT-2, LLaMA 2, and LLaMA 3 actually differ—architecturally?</h4><h4>Scaling and Philosophy</h4><p>While GPT-2 demonstrated that decoder-only Transformers scale, LLaMA 2 explores what it looks like to optimize that architecture for efficiency, openness, and practical self-hosted deployment.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6-y7XrYfxgN91C3QjeKkog.png" /><figcaption>Figure adapted from Umar Jamil (YouTube)</figcaption></figure><p>LLaMA 2 keeps the core Transformer design, but each refinement (RMSNorm, RoPE, and Grouped-Query Attention) is aimed at making large models usable and controllable outside of proprietary servers. It’s about giving researchers and engineers full visibility and flexibility, while keeping the model fast and memory-efficient.</p><p>Where GPT-2 was a proof-of-concept showing the power of decoder-only Transformers, LLaMA 2 and LLaMA 3 are engineered for scale, performance, and open-weight experimentation, bridging the gap between research and deployable, high-performance LLMs.</p><h3>RMSNorm: Trimming the Fat</h3><p>To pop the hood on LLaMA 2, let’s start with the first major optimization that separates it from the GPT-2 engine. If GPT-2 was about getting the architecture right, LLaMA 2 is about making it lean.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GcOx2OZvz8vMxSLKM_jiHQ.jpeg" /><figcaption>Layer Normalisation Formula</figcaption></figure><p>Both models use pre-normalization, but LLaMA 2 swaps the standard LayerNorm for something faster and more aggressive: RMSNorm (Root Mean Square Normalization).</p><p>LayerNorm stabilizes training by normalising the input of each sub-layer(i.e. subtracting the mean and scaling by the variance). RMSNorm asks a blunt question: do we really need the centering step at all? The answer, it turns out, is no.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6jjMebwcrGKgJYSSx1To_A.jpeg" /><figcaption>Root Mean Square Normalisation Formula</figcaption></figure><p>RMSNorm drops mean subtraction entirely and only normalizes based on the magnitude of the activation vector. It’s like a racing team removing air conditioning and the stereo from a car. You lose some comfort, but you get a lighter, faster machine that does the exact same job on the track. Below is the pytorch implementation of Root Mean Square Normalization</p><pre>class RMSNorm(nn.Module):<br>    def __init__(self, dim, eps):<br>        super().__init__()<br>        self.weight = nn.Parameter(torch.ones(dim))   # (D,)<br>        self.eps = eps # to prevent zero division error<br><br>    def forward(self, x):<br>        # x: (B, T, D)<br>        return self.weight * x * torch.rsqrt(<br>            x.pow(2).mean(-1, keepdim=True) + self.eps<br>        )</pre><h4>Why Does This Matter?</h4><p>RMSNorm isn’t just a theoretical tweak, it’s an economic one.</p><p>By removing mean subtraction and reducing normalization overhead, RMSNorm lowers the number of floating-point operations performed per layer, per token. On a small model, that difference is barely noticeable. On a model with tens of billions of parameters trained over trillions of tokens, it compounds rapidly.</p><p>What looks like a minor optimization at the block level turns into meaningful reductions in training time, compute cost, and energy consumption. It’s an early signal that LLaMA 2 was engineered with scale in mind, where architectural choices are judged not just by elegance, but by how well they survive real-world training budgets.</p><h4>Grouped Query Attention (GQA)</h4><p>In standard multi-head attention, like the version used in GPT-2 each query head has its own keys and values. During autoregressive inference, this design creates a large Key–Value (KV) cache that grows linearly with the number of heads.</p><p>At small scales, this isn’t a problem, but at tens of billions of parameters, it becomes a serious memory bottleneck.</p><p>LLaMA 2 addresses this with Grouped-Query Attention (GQA).</p><p>Instead of every query head maintaining its own keys and values, GQA allows multiple query heads to share a smaller set of KV heads. Query diversity is preserved where it matters, while the expensive KV cache is dramatically reduced</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/850/1*FJRmFcolwzqEt03LlAVMYA.png" /><figcaption>Figure from Baroian et al. (2025)</figcaption></figure><p>This is the code representation of Grouped Query Attention (GQA)</p><pre>class LlamaAttention(nn.Module):<br>    def __init__(self, config):<br>        super().__init__()<br>        self.n_heads = config.n_heads # 32<br>        self.n_kv_heads = config.n_kv_heads # 8<br>        self.head_dim = config.dim // config.n_heads #128<br>        self.n_rep = self.n_heads // self.n_kv_heads #4<br><br>        self.wq = nn.Linear(config.dim, self.n_heads * self.head_dim, bias=False)<br>        self.wk = nn.Linear(config.dim, self.n_kv_heads * self.head_dim, bias=False)<br>        self.wv = nn.Linear(config.dim, self.n_kv_heads * self.head_dim, bias=False)<br>        self.wo = nn.Linear(config.dim, config.dim, bias=False)<br><br><br>   def forward(self, x):<br>        # x: (B, T, D)<br>        B, T, D = x.shape<br><br>        q = self.wq(x).view(B, T, self.n_heads, self.head_dim).transpose(1, 2) # (B, T, 32, self.head_dim)<br>        k = self.wk(x).view(B, T, self.n_kv_heads, self.head_dim).transpose(1, 2) # (B, T, 8, self.head_dim)<br>        v = self.wv(x).view(B, T, self.n_kv_heads, self.head_dim).transpose(1, 2) # (# (B, T, 8, self.head_dim)<br>        if self.n_rep &gt; 1:<br>            k = k.repeat_interleave(self.n_rep, dim=1) #(the real trick, where k and v shape is made to match q shape)<br>            v = v.repeat_interleave(self.n_rep, dim=1) # (# (B, T, 32, self.head_dim)</pre><p>The causal masking logic itself doesn’t change. LLaMA still uses the familiar lower-triangular “no-peeking” mask to prevent tokens from attending to the future. What changes is how keys and values are handled.</p><p>Instead of duplicating keys and values for every query head, they are broadcast across groups of queries, aligning efficiently at compute time. The attention math stays causal and correct, only the memory layout becomes smarter. It’s efficiency without sacrificing expressiveness</p><h4>RoPE (Rotary Positional Embeddings): The Mathematical Spin</h4><p>If Grouped-Query Attention was a clever way to save memory, RoPE is the quiet genius that keeps LLaMA from losing its sense of direction.</p><p>In GPT-2, positional information is injected by adding a learned position vector to each token embedding. It’s like attaching a sticky note to every word that says, “I’m at position 5.”<br>This works, but it encodes position in an absolute way. The model knows where a token is in the sequence, but distance between tokens is not represented explicitly, only inferred indirectly through attention</p><p>RoPE takes a different approach.<br>It doesn’t add positional information at all, instead, it rotates the embeddings. Think of each token embedding as a point on a dial. As the token moves forward in the sequence, RoPE turns that dial by an angle determined by its position. When the model compares two tokens, it doesn’t just see their content, it sees the angle between them.</p><p>Mathematically, RoPE operates on pairs of embedding dimensions, treating each pair as coordinates on a 2D plane. For a token at position m, the embedding is rotated by a position-dependent angle θm . Queries and keys are expressed in this rotated (Euler) form before the dot product is computed.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*sevyYxUosHMkgHVFSmts6g.jpeg" /><figcaption>RoPE Matrix Multiplication and Complex Form (Su et al., 2021)</figcaption></figure><p>Here’s the key insight:</p><p>When computing the attention score (the dot product) between a Query at position m and a Key at position n, the result depends only on their relative distance (𝑚−𝑛) not their absolute positions. This means positional information flows through attention rather than being tacked onto embeddings. Relative distance becomes a geometric property of the representations themselves, naturally aligned with how attention operates.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ZT03_KZ4vCxeErERT7Zqbw.jpeg" /><figcaption>Generalization of Rotary Position Embeddings (RoPE) to d-dimensional Space</figcaption></figure><p>Below is the pytorch implementation of RoPE</p><pre>class RotaryEmbedding(nn.Module):<br>    def __init__(self, head_dim, base):<br>        super().__init__()<br>        inv_freq = 1.0 / (base ** (torch.arange(0, head_dim, 2).float() / head_dim))<br>        self.register_buffer(&quot;inv_freq&quot;, inv_freq, persistent=False)<br>        self.cos = None<br>        self.sin = None<br>        self.seq_len_cached = 0<br><br>    def _build_cache(self, seq_len, device, dtype):<br>        if seq_len &lt;= self.seq_len_cached:<br>            return<br>        self.seq_len_cached = seq_len<br>        t = torch.arange(seq_len, device=device).float()              # (T,)<br>        freqs = torch.outer(t, self.inv_freq)                         # (T, Hd/2)<br>        emb = torch.cat((freqs, freqs), dim=-1)                       # (T, Hd)<br>        self.cos = emb.cos()[None, None, :, :].to(dtype)              # (1,1,T,Hd)<br>        self.sin = emb.sin()[None, None, :, :].to(dtype)              # (1,1,T,Hd)<br><br>    def rotate_half(self, x):<br>        x1 = x[..., :x.shape[-1] // 2]<br>        x2 = x[..., x.shape[-1] // 2:]<br>        return torch.cat((-x2, x1), dim=-1)<br><br>    def forward(self, q, k):<br>        # q, k: (B, H, T, Hd)<br>        seq_len = q.size(-2)<br>        self._build_cache(seq_len, q.device, q.dtype)<br>        cos = self.cos[:, :, :seq_len]<br>        sin = self.sin[:, :, :seq_len]<br>        return (<br>            q * cos + self.rotate_half(q) * sin,<br>            k * cos + self.rotate_half(k) * sin<br>        )</pre><h4>Why This Matters?</h4><p>RoPE delivers several practical advantages:</p><ul><li>Length generalization<br>Because position is encoded as a rotation, the mechanism naturally extends to sequences longer than those seen during training.</li><li>Zero extra parameters<br>RoPE introduces no learned weights. Positional information comes entirely from deterministic trigonometric functions.</li><li>Natural distance bias<br>As tokens get farther apart, their rotated representations align less, causing attention to weaken with distance in a smooth, implicit way.</li></ul><h4>Refining the Decoder-Only Transformer</h4><p>LLaMA 2 doesn’t replace the GPT-style decoder, it refines it.<br>RMSNorm trims unnecessary computation, GQA reduces inference-time memory pressure, and RoPE reshapes how positional information flows through attention.</p><p>None of these changes alter the core Transformer blueprint. What they do change is how well that blueprint holds up at scale.</p><p>With these refinements in place, LLaMA 2 and later LLaMA 3 turn the familiar decoder-only architecture into something leaner, more stable, and better suited for modern large-scale deployment.More importantly, it shows that meaningful progress in Transformer design now comes less from reinvention, and more from disciplined, engineering-driven iteration. Same engine. Sharper handling. Built to run longer and faster, when the road gets demanding.</p><p>References</p><ul><li>Su, J., Lu, Y., Pan, S., Murtadha, A., Wen, B., &amp; Liu, Y. (2021). RoFormer: Enhanced Transformer with rotary position embedding</li><li>Ainslie, J., Lee-Thorp, J., de Jong, M., Zemlyanskiy, Y., Lebrón, F., &amp; Sanghai, S. (2023). GQA: Training generalized multi-query transformer models from multi-head checkpoints. Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, 4895–4901. https://doi.org/10.18653/v1/2023.emnlp-main.29</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e332022a8a2b" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Evolution of the Second Generation of Transformers: From Prediction to Intelligence]]></title>
            <link>https://medium.com/@abdulrasheedolakiitan/the-evolution-of-the-second-generation-of-transformers-from-prediction-to-intelligence-1ef248e76b30?source=rss-f095285de912------2</link>
            <guid isPermaLink="false">https://medium.com/p/1ef248e76b30</guid>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[AbdulRasheed Bakare]]></dc:creator>
            <pubDate>Thu, 22 Jan 2026 19:45:01 GMT</pubDate>
            <atom:updated>2026-01-22T19:45:01.753Z</atom:updated>
            <content:encoded><![CDATA[<p>The Evolution of the Second Generation of Transformers: From Prediction to Intelligence</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/400/1*-8weZXieL6e2yNN_MB2s3g.jpeg" /><figcaption>Image © 2007 Dreamworks Pictures</figcaption></figure><p>When GenZs hear the word &quot;Transformer &quot;, they picture Optimus Prime , a towering robot that shifts from a heavy-duty truck into a galaxy-saving hero. It’s a cool mental image, but in the realm of AI, Transformers are far more powerful (and significantly less likely to step on your car).<br>They are the foundational architecture behind the Large Language Models (LLMs) currently rewriting how we work, code, and argue on the internet. While the original 2017 Transformer gave us the &quot;spark,&quot; the Second Generation “the sleek”, decoder-only models like GPT-2 (2019) and its descendants like Llama and DeepSeek is where the real revolution took hold. These models stopped trying to &quot;translate&quot; and started trying to &quot;predict,&quot; moving from rigid supervised learning to massive, unsupervised pre-training ingredients</p><h3>The Origin Story: &quot;Attention Is All You Need&quot;</h3><p>Before 2017, AI processed text like a slow reader: one word at a time, left to right, using Recurrent Neural Networks (RNNs). It was slow, and by the time the model reached the end of a sentence, it often forgot how it started.<br>The landmark paper “Attention Is All You Need” threw that out the window. It replaced clunky recurrence with Attention, allowing a model to look at every word in a paragraph simultaneously. It’s the difference between reading through a straw and seeing the whole page at once.</p><h3>The GPT-2 Pivot: Less is More</h3><p>While the original Transformer was a &quot;bi-cameral&quot; system (an Encoder to understand and a Decoder to speak), OpenAI realized that for generating text, having two heads was overkill.<br>GPT-2 essentially performed a &quot;lobotomy&quot; on the original design. It ditched the Encoder entirely, creating a Decoder-only architecture. Because we aren’t translating from French to English, we don’t need a separate &quot;source&quot; sequence. We just need to know what word comes next.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jIJbbbUQPbiA9O8oq4VO-Q.jpeg" /><figcaption>image by CampusX</figcaption></figure><h4>What Got the Axe?</h4><p>To make GPT-2 more efficient, OpenAI removed:<br>1. The entire Encoder stack.<br>2. Cross-Attention sub-layers (the bridge that used to connect the Encoder to the Decoder).</p><p>What’s left is a streamlined, high-speed engine consisting of stacked blocks. Each block is a refined loop of self-correction and prediction</p><h3>The &quot;Fine-Tuning&quot; (The Professional Tweaks)</h3><p>OpenAI didn’t just delete parts; they polished what remained:</p><ul><li>Pre-normalization: They moved the &quot;Layer Norm&quot; to the start of the block. It’s like stretching before a workout; it makes the training much more stable.</li><li>GELU Activations: They swapped the standard ReLU for GELU (Gaussian Error Linear Units). Think of it as a smoother, more sophisticated way for the model to decide which neurons should &quot;fire.&quot;</li><li>The Final Polish: An extra layer of normalization was added after the final block to ensure the output doesn’t go off the rails.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*848h2AZsqFppSMe8kgZcGA.png" /><figcaption>Wu, H. (2024, September 15). GPT-2 Architectural Diagram</figcaption></figure><h3><strong>Under the Hood: The Layer-by-Layer Breakdown</strong></h3><p>To understand how a string of text becomes &quot;intelligence,&quot; we have to look at the assembly line inside each Transformer block.</p><p><strong>1. Token and Positional Embeddings:</strong></p><p>​Computers don’t read words; they do math. Token Embeddings are the &quot;translation&quot; layer where words are converted into high-dimensional vectors. If the word &quot;Apple&quot; is a vector, it lives in a mathematical space close to &quot;Orange&quot; and far from &quot;Spaceship.&quot;<br>​However, because Transformers look at all words at once, they have no natural sense of order. To fix this, we use Positional Embeddings. We add a specific mathematical &quot;stamp&quot; to each token that tells the model exactly where it sits in the sentence. It’s the GPS for the vocabulary.</p><pre>import tiktoken <br>enc = tiktoken.get_encoding(&quot;gpt2&quot;)   <br>tokens = enc.encode(text)<br>tokens.append(enc.eot_token)                 </pre><pre>pos = nn.Embedding(config.block_size, config.n_embed)  </pre><p><strong>2. Layer Normalization: The Great Equalizer</strong></p><p>Deep learning models are prone to &quot;internal covariate shift,&quot; where data values get unpredictable as they pass through deeper layers. Layer Normalization (LayerNorm) keeps these values in line.<br>In GPT-2, we use Pre-Normalization. Unlike the original Transformer that normalized after the sub-layers, GPT-2 normalizes before the data enters the Attention or Feed-Forward blocks. To normalize a vector x, we use:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/208/1*cGTYWC-a4TsXTw0lmDsAGA.png" /><figcaption>LayerNorm Equation</figcaption></figure><p>Breakdown of the Variables:<br>x: The input vector (the embedding of a specific token).<br>E[x]: The mean of all values in that specific vector.<br>Var[x]: The variance of the values in that vector.<br>ε : A very small constant (like 10^-5) added to the denominator to prevent division by zero.<br>γ and β: Learnable parameters that allow the model to scale and shift the normalized values if it learns that a specific distribution is better for performance</p><p><strong>3. Masked Multi-Head Self-Attention: The Brain of the Operation</strong></p><p>If embeddings are the vocabulary and LayerNorm is the stabilizer, Masked Multi-Head Self-Attention is the &quot;intelligence.&quot; This is where the model stops looking at words as isolated islands and starts understanding how they relate to one another.</p><p>In a sentence like &quot;The bank of the river,&quot; the word &quot;bank&quot; only makes sense because the model &quot;attends&quot; to the word &quot;river.&quot; Without attention, the model wouldn&#39;t know if you&#39;re talking about geography or a place to withdraw twenty bucks.</p><p>The &quot;Masked&quot; Part: No Spoilers!</p><p>GPT-2 is an autoregressive model, meaning it predicts the next word based only on the words that came before it. To train it, we have to cheat a little: we give it the whole sentence but put &quot;blinders&quot; on it so it can’t look at the future.</p><p>We use a Causal Mask; a triangular lower matrix of zeros . This ensures that when the model is calculating the representation for the 3rd word, it has a 0% chance of seeing the 4th word. It’s like taking a test where you’re not allowed to flip the page ahead.</p><p>[ 1, 0, 0, 0 ] &lt;- Word 1 can only see itself</p><p>[ 1, 1, 0, 0 ] &lt;- Word 2 can see Word 1 and itself</p><p>[ 1, 1, 1, 0 ] &lt;- Word 3 can see 1, 2, and itself</p><p>[ 1, 1, 1, 1 ] &lt;- Word 4 can see everything before it</p><p>then the zeros are converted to negative infinities to ensure the future tokens weighted zero percent after applying softmax</p><p><strong>The Math: Queries, Keys, and Values</strong></p><p>Each token is projected into three different vectors: Query (Q), Key (K), and Value (V).</p><p>Query: &quot;What am I looking for?&quot;</p><p>Key: &quot;What do I contain?&quot;</p><p>Value: &quot;If I’m relevant, here is what I contribute.&quot;</p><p>The model calculates the &quot;score&quot; (relevance) by taking the dot product of Q and K, scaling it, applying the mask, and then running a Softmax to turn those scores into percentages:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/396/1*YZMvxqslkx1zbcrSCCsbTA.png" /><figcaption>Attention Equation</figcaption></figure><p><strong>4. The Feed-Forward Network: The &quot;Knowledge Store&quot;</strong></p><p>If Attention is about &quot;gathering context,&quot; the Feed-Forward Network (FFN) is about &quot;processing knowledge.&quot; It consists of two linear transformations with a non-linear activation function. GPT-2 swapped the standard ReLU for GELU (Gaussian Error Linear Units), a smoother curve that helps gradients flow better</p><pre>FFN= GELU(xW1 +b1)W2 +b2</pre><p><strong>Why do we need it?</strong></p><p>Without the FFN, the Transformer would just be a series of weighted averages. It wouldn’t have the capacity to learn complex, non-linear relationships. The FFN is where the model’s &quot;long-term memory&quot; or &quot;world knowledge&quot; lives—like knowing that &quot;Paris&quot; is the capital of &quot;France</p><p><strong>5. Residual Connections: The Information Highways</strong></p><p>​In a deep model like GPT-2 (which can have up to 48 layers), the data has a long, perilous journey. By the time the input reaches the final layer, it has been multiplied by so many matrices and squashed by so many activations that the original signal—the actual words you typed—can get &quot;lost in translation.&quot; This is known as the vanishing gradient problem.</p><p>​Residual Connections (or &quot;skip connections&quot;) are the fix. Instead of forcing the data to go through every complex layer, we provide a &quot;side path&quot; that allows the original input to skip the layer and be added back to the output.</p><h3>The &quot;Shortcut&quot; Logic</h3><p>​Think of a Residual Connection like a manager checking a draft.</p><p>​The Sub-layer (Attention or FFN) is the &quot;editor&quot; who makes changes.</p><p>​The Residual Path is the &quot;original copy.&quot;</p><p>​The output is: Output = Original + Changes(Original)</p><p>​This ensures that even if the &quot;editor&quot; (the layer) does a terrible job or makes the numbers too small, the &quot;original copy&quot; still flows through to the next stage. It keeps the gradient &quot;healthy&quot; during training, acting like an express lane for the learning signal.</p><p>The Math: Simple Addition</p><p>​Mathematically, for any function f(x) (like Attention), the residual output is simply</p><p>y= f(x) + x</p><p>In GPT-2, specifically, we apply this after the LayerNorm and the sub-layer. So the full block looks like:</p><p>x= x+Attention(LayerNorm(x))</p><p>x= x + Feedforward(LayerNorm(x))</p><p><strong>Conclusion: The Legacy of the Decoder</strong></p><p>​The Second Gen started with a simple, almost radical question: What if we just predict the next word? By stripping the Transformer down to its bare essentials, GPT-2 proved that scale and simplicity could mimic reasoning.<br>​Today’s models are the ultimate realization of this &quot;less is more&quot; evolution. We have moved from giant, clunky machines to sleek, specialized architectures that can reason through math, code, and human emotion, all by mastering the art of the &quot;next word.&quot;</p><h3>References</h3><ul><li>Karpathy, Andrej. “nanoGPT” repo.</li><li>Vaswani, Ashish, et al. “Attention Is All You Need. ArXiv, 2017.</li></ul><p><a href="https://github.com/AbdulRasheed6/GPT2/blob/main/GPT-2%20(124M)/gpt-2.py">GPT2/GPT-2 (124M)/gpt-2.py at main · AbdulRasheed6/GPT2</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1ef248e76b30" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Defining Success in Machine Learning: A Multifaceted Approach]]></title>
            <link>https://medium.com/@abdulrasheedolakiitan/defining-success-in-machine-learning-a-multifaceted-approach-a1317413abd3?source=rss-f095285de912------2</link>
            <guid isPermaLink="false">https://medium.com/p/a1317413abd3</guid>
            <dc:creator><![CDATA[AbdulRasheed Bakare]]></dc:creator>
            <pubDate>Thu, 26 Dec 2024 15:08:53 GMT</pubDate>
            <atom:updated>2024-12-26T15:08:53.929Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*UClKcpTAni-Zc9Fp" /><figcaption>@copywright ENCORD</figcaption></figure><p>In the realm of machine learning (ML), success is inherently multifaceted. The age-old adage, <em>“success is relative,”</em> resonates deeply in this domain. For instance, economic metrics like GDP growth may be celebrated by government officials, yet citizens often prioritize inflation rates and purchasing power as indicators of well-being. This dichotomy underscores the necessity of considering diverse perspectives when defining success.</p><p>In ML projects, engineers often emphasize technical metrics such as accuracy or F1 scores, while stakeholders may prioritize business outcomes like return on investment (ROI) or customer satisfaction. Reconciling these differing views is essential to establish a shared understanding of success.</p><p>This article explores key considerations for defining success in ML projects, focusing on problem definition, stakeholder alignment, and success metrics.</p><h3>Problem Definition</h3><p>A well-defined problem lays the foundation for any successful ML project. It aligns teams, clarifies goals, and sets realistic expectations.</p><h4>Characteristics of a Well-Defined Problem</h4><ol><li><strong>Specificity</strong>: Clearly stated and easy to understand.</li><li><strong>Measurability</strong>: Quantifiable for tracking progress.</li><li><strong>Achievability</strong>: Realistic within constraints.</li><li><strong>Relevance</strong>: Aligned with organizational goals.</li></ol><h4>Best Practices for Problem Definition</h4><ul><li><strong>Involve Stakeholders</strong>: Engage stakeholders early to capture needs.</li><li><strong>Conduct Analysis</strong>: Gather and analyze relevant data.</li><li><strong>Challenge Assumptions</strong>: Validate hypotheses with data.</li><li><strong>Iterate and Refine</strong>: Adjust as new insights emerge.</li></ul><h3>Success Metrics</h3><p>Success metrics evaluate model performance and determine goal attainment.</p><h4>Characteristics of Effective Success Metrics</h4><ol><li><strong>Relevance</strong>: Aligned with organizational goals.</li><li><strong>Measurability</strong>: Quantifiable and objective.</li><li><strong>Actionability</strong>: Inform decisions and improvements.</li></ol><h4>Common Success Metrics for ML Projects</h4><ol><li><strong>Accuracy</strong>: Proportion of correct predictions.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/281/1*C-PIN3kCCsNE3R9Skz6bAg.png" /></figure><p><strong>2.Precision</strong>: True positives among all predicted positives</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/182/1*Zd0_MLIjRINHebWxok0LNw.png" /></figure><p><strong>3. Recall</strong>: True positives among all actual positives.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/157/1*46pNKev0tXLPDMTQGPTOqA.png" /></figure><p><strong>4. F1 Score</strong>: Harmonic mean of precision and recall.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/295/1*DqebEUnzrwvLst4O6_HMPA.png" /></figure><p><strong>5.Mean Squared Error (MSE)</strong>: Average squared difference between predicted and actual values.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/213/1*9sL_Gm_0b6ky7un6mQAkkQ.png" /></figure><p><strong>6. Return on Investment (ROI)</strong>: Financial return relative to investment.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/248/1*dps6UxbfuyOugvnsLaBGDQ.png" /></figure><h4>Best Practices for Success Metrics</h4><ul><li><strong>Align with Goals</strong>: Ensure metrics match organizational objectives.</li><li><strong>Establish Baselines</strong>: Provide a reference for comparison.</li><li><strong>Monitor and Adjust</strong>: Continuously optimize performance.</li><li><strong>Communicate Results</strong>: Share findings with stakeholders for transparency.</li></ul><h3>Examples and Use Cases</h3><p><strong>Healthcare Recommendation Systems: Success Metrics</strong></p><p><strong>Introduction:<br></strong> Healthcare recommendation systems aim to improve patient outcomes, optimize resource utilization, and enhance the decision-making process for healthcare providers. To evaluate their effectiveness, it is essential to define success metrics aligned with clinical accuracy, business performance, and patient satisfaction.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/685/0*CtxDrLhnM-16vEDx.png" /><figcaption>Gräßer, F., Tesch, F., Schmitt, J. <em>et al.</em> A pharmaceutical therapy recommender system enabling shared decision-making. <em>User Model User-Adap Inter</em> <strong>32</strong>, 1019–1062 (2022). <a href="https://doi.org/10.1007/s11257-021-09298-4">https://doi.org/10.1007/s11257-021-09298-4</a></figcaption></figure><h3>1. Clinical Accuracy Metrics (For Engineers and Data Scientists)</h3><p>Healthcare systems must ensure precise and reliable recommendations to avoid errors and misdiagnoses. Key metrics include:</p><ol><li><strong>Precision (Positive Predictive Value):<br></strong> Measures the proportion of correctly recommended treatments or diagnoses among all positive predictions.</li></ol><p><strong>Formula:</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*Zd0_MLIjRINHebWxok0LNw.png" /></figure><ul><li><strong>TP</strong>: True Positives — Correct recommendations.</li><li><strong>FP</strong>: False Positives — Incorrect recommendations.</li></ul><p><strong>2.Recall (Sensitivity):<br></strong>Evaluates the system’s ability to identify all actual positive cases.</p><p><strong>Formula:</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*46pNKev0tXLPDMTQGPTOqA.png" /></figure><ul><li><strong>FN</strong>: False Negatives — Missed positive cases.</li></ul><p><strong>3. F1 Score:<br></strong> Provides a balanced measure of precision and recall.</p><p><strong>Formula:</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*DqebEUnzrwvLst4O6_HMPA.png" /></figure><ol><li><strong>Specificity (True Negative Rate):<br></strong> Measures the ability to correctly identify negative cases.</li></ol><p><strong>Formula:</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/195/1*gr9CG3YpvuO3GgnRRGtCLQ.png" /></figure><p><strong>TN</strong>: True Negatives — Correctly rejected cases.</p><p><strong>2. ROC-AUC (Receiver Operating Characteristic — Area Under Curve):<br></strong> Evaluates the model’s ability to distinguish between positive and negative cases. Values close to <strong>1</strong> indicate high performance.</p><h3>2. Business Performance Metrics (For Administrators and Stakeholders)</h3><p>Business stakeholders focus on financial and operational impacts to assess the return on investment (ROI) of the system. Key metrics include:</p><ol><li><strong>Cost-Effectiveness Ratio:<br></strong> Compares implementation costs to health benefits.</li></ol><p><strong>Formula:</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/506/1*Go4-gLKOYOr44ClwRHWyFw.png" /></figure><ul><li><strong>QALY</strong>: Quality-Adjusted Life Years, representing improved patient health.</li></ul><p><strong>2 . Reduction in Readmission Rates:<br></strong> Tracks whether recommendations reduce unnecessary hospital readmissions.</p><p><strong>Formula:</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/420/1*wqsN0CxIcvJtp6DlIJY2EA.png" /></figure><p><strong>3. Improvement in Treatment Adherence:<br></strong> Measures how well patients follow prescribed treatments.</p><p><strong>Formula:</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/494/1*DvnADkEAQAl5bB29HGtfdg.png" /></figure><p><strong>4. Return on Investment (ROI):<br></strong> Quantifies the system’s financial returns relative to the cost of implementation.</p><p><strong>Formula:</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/274/1*P13aEVOMtBYEMcFMv1G7dQ.png" /></figure><h3>3. Patient Satisfaction Metrics (User Experience)</h3><p>These metrics measure the system’s effectiveness in delivering personalized care and improving patient engagement.</p><ol><li><strong>Personalization Score:<br></strong> Assesses whether recommendations align with patient preferences.</li></ol><p><strong>Formula:</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/409/1*F-EpcW94soEl2FkZKVN0rA.png" /></figure><p><strong>2 . Patient Satisfaction Rating:<br></strong> Based on feedback from surveys about system usability and recommendation relevance.</p><p><strong>Formula:</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/358/1*AdAYNLutlFV6EXH9oSMj4A.png" /></figure><p><strong>3 . Time-to-Recommendation:<br></strong> Tracks the system’s response time in generating recommendations.</p><p><strong>Formula:<br></strong> Response Time (RT)=Time Taken to Generate Recommendations</p><p><strong>4. Engagement Rate:<br></strong> Measures how actively patients interact with the recommendations provided.</p><p><strong>Formula:</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/377/1*dyYDLak3b0w2_w0UnZ6Htw.png" /></figure><p>Success metrics for healthcare recommendation systems span clinical accuracy, business performance, and patient satisfaction. By monitoring precision, recall, cost-effectiveness, and personalization, stakeholders can evaluate the system’s effectiveness and continuously refine it to meet evolving healthcare needs. A balanced focus on these metrics ensures improved outcomes, optimized costs, and enhanced patient experiences.</p><p><strong>E-commerce Recommendation System: Key Metrics for Success</strong></p><p>An effective e-commerce recommendation system is essential for enhancing user experience, driving business growth, and ensuring customer satisfaction. Various stakeholders within the e-commerce ecosystem, such as machine learning engineers, business stakeholders, and customers, each have distinct success metrics to evaluate the system’s performance. These metrics provide a comprehensive assessment of how well the recommendation system is functioning from different perspectives.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/642/0*MyBC4j7B5cvEKJb5.png" /><figcaption><a href="https://www.semanticscholar.org/author/T.-Badriyah/9738736">T. Badriyah</a> <a href="https://www.semanticscholar.org/author/Erry-Tri-Wijayanto/28305004">Erry Tri Wijayanto</a> <a href="https://www.semanticscholar.org/author/I.-Syarif/2664847">I. Syarif</a> <a href="https://www.semanticscholar.org/author/P.-Kristalina/3210510">P. Kristalina</a> A Hybrid recommendation system for E-commerce based on product description and user profile, Published in 2017</figcaption></figure><ol><li><strong>Machine Learning Engineer’s Performance Metrics:</strong> Machine learning engineers focus on the technical accuracy and completeness of the recommendation system. Their success is primarily measured using precision, recall, and the F1 score, which are critical for understanding how well the system predicts relevant products for users.</li></ol><ul><li><strong>Precision:</strong> Precision measures the proportion of recommended products that are actually relevant. It is calculated using the formula:</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*Zd0_MLIjRINHebWxok0LNw.png" /></figure><p>Where:</p><p>TP (True Positives) represents the number of relevant items recommended.</p><p>FP (False Positives) represents the number of irrelevant items recommended.</p><ul><li><strong>Recall:</strong> Recall measures the proportion of relevant products that the system successfully recommends. It is defined as:</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*46pNKev0tXLPDMTQGPTOqA.png" /></figure><p>Where:</p><p>FN (False Negatives) represents the number of relevant products not recommended by the system.</p><ul><li><strong>F1 Score:</strong> The F1 score provides a harmonic mean of precision and recall, balancing both metrics. It is calculated as:</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*DqebEUnzrwvLst4O6_HMPA.png" /></figure><p>The F1 score serves as a comprehensive metric for evaluating the system’s overall performance, ensuring that the system delivers both accuracy and completeness.</p><ol><li><strong>Business Stakeholder’s Success Metrics:</strong> For business stakeholders, the primary concern is the impact of the recommendation system on business outcomes, including revenue growth and customer retention. The key metrics for this group include Average Order Value (AOV) and customer retention rate.</li></ol><ul><li><strong>Average Order Value (AOV):</strong> AOV measures the average amount of money customers spend per transaction. This metric helps assess the effectiveness of product recommendations in increasing order values.</li></ul><p>The formula for AOV is:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/250/1*hvmYtECoXfHwSxr8lyxiTg.png" /></figure><p>A higher AOV indicates that the recommendation system is successfully encouraging customers to purchase more valuable items or add more items to their carts.</p><ul><li><strong>Customer Retention Rate:</strong> Retaining customers is crucial for long-term business growth. This metric measures the percentage of customers who make repeat purchases over a given period. It is defined as:</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/417/1*sEYKftNNT9eGnO3ZVSbhjw.png" /></figure><p>A higher retention rate indicates that the recommendation system is effective in keeping customers engaged and loyal to the brand.</p><ol><li><strong>Customer’s Success Metrics:</strong> For customers, the success of the recommendation system is centered around the relevance and personalization of the recommended products. Customers are more likely to make a purchase if the recommendations are tailored to their individual preferences and needs.</li></ol><ul><li><strong>Relevance:</strong> Relevance measures how well the recommended products align with a customer’s preferences. This metric is calculated as:</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/380/1*Jn73y06WZLUbH0iGNGmrAg.png" /></figure><p>A higher relevance score signifies that the system is doing a better job at suggesting products that customers are likely to find appealing, enhancing the likelihood of conversions and improving customer satisfaction.</p><p>The success of an e-commerce recommendation system is multi-dimensional and requires consideration of both technical performance and business outcomes. Machine learning engineers, business stakeholders, and customers each have distinct but interconnected metrics that provide valuable insights into the system’s effectiveness. By focusing on precision, recall, F1 score, AOV, customer retention, and relevance, businesses can ensure that their recommendation systems deliver meaningful value across all aspects of the e-commerce experience.</p><h3>Conclusion</h3><p>Defining success in machine learning projects/products requires careful consideration of diverse perspectives. By aligning success metrics with the goals and priorities of various stakeholders, machine learning engineers can develop more effective and impactful solutions. As a notable Machine Learning Academia notes “ Machine learning is a field that is all about metrics. if you cannot measure it, you cannot manage it “. By prioritizing well-defined success metrics, machine learning practitioners can ensure that their projects/products drive meaningful value and achieve lasting impact</p><h3>References</h3><p>[1] Ng, A. (2018). Machine Learning Yearning. Chapter 2: Define the Problem.</p><p>[2] Géron, A. (2019). Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow. Chapter 1: The Machine Learning Landscape.</p><p>[3] Moore, G. A. (2014). Crossing the Chasm: Marketing and Selling High-Tech Products to Mainstream Customers. Chapter 3: The Whole Product and the Generic Product.</p><p>[4] Cross, N. (2011). Design Thinking. Chapter 2: The Design Process.</p><p>[5] Bishop, C. M. (2006). Pattern Recognition and Machine Learning. Chapter 1: Introduction.</p><p>[6] Murphy, K. P. (2012). Machine Learning: A Probabilistic Perspective. Chapter 2: Probability and Statistics.</p><p>[7] Rose, D. (2018). Machine Learning for Business. Chapter 5: Recommendation Systems.</p><p>[8] Microsoft. (n.d.). The Machine Learning Lifecycle. Chapter 1: Plan.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a1317413abd3" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Pre-ML Considerations(PART I): Learning from Elon Musk’s Visionary Approach]]></title>
            <link>https://medium.com/@abdulrasheedolakiitan/pre-ml-considerations-part-i-learning-from-elon-musks-visionary-approach-f2228e58fbb8?source=rss-f095285de912------2</link>
            <guid isPermaLink="false">https://medium.com/p/f2228e58fbb8</guid>
            <dc:creator><![CDATA[AbdulRasheed Bakare]]></dc:creator>
            <pubDate>Sat, 07 Dec 2024 13:39:45 GMT</pubDate>
            <atom:updated>2024-12-07T13:39:45.025Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="Space X falcon 9" src="https://cdn-images-1.medium.com/max/1024/0*tyicXZ874CheO5bY.jpeg" /><figcaption>Space X Falcon 9</figcaption></figure><p>In our previous article, <em>“</em><a href="https://medium.com/@abdulrasheedolakiitan/from-lions-to-honeybees-lessons-in-strategic-project-planning-for-machine-learning-engineers-part-71a0e829f617"><em>From Lions to Honeybees: Lessons in Strategic Project Planning for Machine Learning Engineers</em></a><em>”</em> we explored the importance of deliberate forethought in machine learning (ML) projects, drawing parallels from the meticulous planning behaviors of lions and honeybees. This article builds on those principles by delving into <strong>pre-ML considerations</strong> — critical factors that can determine the success of your ML initiatives.</p><p>To provide a fresh perspective, we take inspiration from Elon Musk’s visionary approach to innovation, particularly his groundbreaking efforts in space exploration (SpaceX) and global connectivity (Starlink). Musk’s strategic thinking offers valuable lessons that can inform the way we approach ML projects.</p><h3>Visionary Insights from Elon Musk</h3><p>Musk’s journey to establish SpaceX and launch Starlink was driven by his ability to identify gaps and innovate with hybrid solutions:</p><ol><li><strong>Space Exploration (SpaceX):</strong><br> Recognizing NASA’s limitations post-Apollo, Musk envisioned a private-sector solution to advance space exploration. By combining existing technologies (e.g., liquid-fueled engines and grid fins) with innovative design, SpaceX took a crucial step towards achieving rocket reusability, a key factor in reducing the costs access to space.</li><li><strong>Global Connectivity (Starlink):</strong><br> Faced with the inadequacies of traditional telecommunications infrastructure — especially in rural areas — Musk developed a constellation of low-Earth orbit satellites to deliver fast, reliable internet worldwide, regardless of geography or weather conditions.</li></ol><p>Musk’s ability to question conventional methods and integrate existing and novel solutions underscores a critical lesson for ML practitioners: <strong>always evaluate whether ML is truly necessary</strong> and explore simpler or hybrid approaches first.</p><h3>Key Takeaway: Question the Necessity of ML</h3><p>Before committing to an ML solution, consider these foundational questions:</p><ol><li><strong>Simplicity First:</strong> Can a non-ML approach achieve the desired outcome?</li><li><strong>Problem Definition:</strong> Is the problem well-defined and aligned with business goals?</li><li><strong>Success Metrics:</strong> How will success be measured and evaluated?</li></ol><p>By stepping back and assessing the broader context, you may discover that simpler alternatives or hybrid approaches (combining traditional methods with ML) could be more efficient and effective.</p><h3>Scenarios Where ML Might Not Be the Best Solution</h3><ol><li><strong>Simple, Well-Defined Problems: </strong>Rule-based solutions often outperform ML models when the problem has clear rules and outcomes.</li><li><strong>Small Datasets: </strong>Limited data can hinder ML model performance, making traditional statistical methods more suitable.</li><li><strong>High-Stakes, Low-Tolerance Decisions: </strong>Human oversight is crucial in areas like medical diagnosis, financial fraud detection, and aerospace operations, where errors can have severe consequences.</li><li><strong>Poor Data Quality or Availability: </strong>Biased or incomplete datasets can undermine ML effectiveness, necessitating alternative methods.</li><li><strong>Real-Time, Mission-Critical Applications: </strong>Latency, predictability, and explainability challenges may require edge computing, model optimization, or hybrid solutions to ensure reliability in real-time scenarios like autonomous vehicles and industrial control systems</li></ol><h3>Scenarios Where ML Is the Optimal Solution</h3><ol><li><strong>Complex, High-Dimensional Data: </strong>ML excels at detecting patterns in images, speech, and text that humans cannot easily discern.</li><li><strong>Large, Diverse Datasets: </strong>With ample data, ML models can improve performance over time.</li><li><strong>Dynamic, Non-Linear Systems: </strong>ML can model complex relationships in areas like weather forecasting or financial markets.</li><li><strong>Natural Language Processing (NLP): </strong>Tasks like language translation and sentiment analysis are well-suited for ML models.</li><li><strong>Recommendation Systems: </strong>ML can create personalized recommendations based on user behavior and preferences</li></ol><h3>Selecting the Right Model</h3><p>When choosing between a pre-existing model and building a custom one, consider:</p><ol><li><strong>Data Similarity:</strong> Does your dataset resemble the training data of a pre-trained model?</li><li><strong>Data Size:</strong> Do you have enough data to justify a custom model?</li><li><strong>Computational Resources:</strong> Are resources available to support custom model training?</li></ol><p>Renowned ML expert Andrew Ng advocates starting with pre-trained models to save time and resources. As he famously noted: <em>“A lot of machine learning is just using somebody else’s pre-trained model… You can download a pre-trained model and fine-tune it on your data.”</em></p><h3>Conclusion</h3><p>Before diving into ML projects, reflect on the bigger picture. Elon Musk’s innovative strategies remind us of the importance of thoughtful planning, questioning assumptions, and exploring hybrid solutions. By applying these principles, and following expert guidance like Andrew Ng’s, you can maximize the potential of ML in your organization.</p><p>As the Yoruba proverb wisely states: <em>“You cannot build a house without first counting the cost.”</em></p><p>By grounding your ML efforts in thoughtful planning and leveraging the right tools, you’ll unlock transformative opportunities while avoiding unnecessary complexity.</p><h3>Recommended Resources</h3><ul><li><em>“Machine Learning: A Probabilistic Perspective”</em> by Kevin P. Murphy</li><li><em>“Human-in-the-Loop Machine Learning”</em> by Robert Munro</li><li><em>“Data Quality: Concepts, Methodologies, and Techniques”</em> by Carlo Batini and Monica Scannapieco</li><li><em>“Deep Learning”</em> by Ian Goodfellow, Yoshua Bengio, and Aaron Courville</li><li><em>“Recommender Systems: An Introduction”</em> by Dietmar Jannach et al.</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f2228e58fbb8" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[From Lions to Honeybees: Lessons in Strategic Project Planning for Machine Learning Engineers (PART…]]></title>
            <link>https://medium.com/@abdulrasheedolakiitan/from-lions-to-honeybees-lessons-in-strategic-project-planning-for-machine-learning-engineers-part-71a0e829f617?source=rss-f095285de912------2</link>
            <guid isPermaLink="false">https://medium.com/p/71a0e829f617</guid>
            <dc:creator><![CDATA[AbdulRasheed Bakare]]></dc:creator>
            <pubDate>Tue, 26 Nov 2024 14:00:08 GMT</pubDate>
            <atom:updated>2024-11-26T14:11:07.449Z</atom:updated>
            <content:encoded><![CDATA[<h3>From Lions to Honeybees: Lessons in Strategic Project Planning for Machine Learning Engineers (PART 1)</h3><p>In the animal kingdom, survival depends on strategic planning. Lions meticulously stalk prey, leveraging terrain and teamwork. Honeybees meticulously plan hive architecture, allocating resources and optimizing workflow. Similarly, successful machine learning engineers approach projects with deliberate forethought.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/612/0*iCegEYBGdm-ZXwaa" /></figure><h4>Beyond Data to Deployment: The Complexity of Machine Learning</h4><p>The notion that machine learning begins with data and concludes with model deployment oversimplifies the complexity of the process. Every machine learning journey starts with defining a clear end goal, dictating the entire lifecycle.</p><h4>Critical Considerations for Machine Learning Success</h4><p>Before diving into machine learning projects, critical questions must be asked. Ensure project readiness with this comprehensive checklist:</p><h3>Machine Learning Project Readiness Checklist</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*h-Ujno9MojcgPgaX.png" /><figcaption>Lifecycle of a Machine Learning Project. Source: <a href="https://www.deeplearning.ai/courses/machine-learning-engineering-for-production-mlops/">Machine Learning Engineering for Production (MLOps) Specialization — DeepLearning.AI</a></figcaption></figure><h4>Pre-ML Considerations</h4><p>1. Simplicity First: Can non-ML approaches achieve objectives?</p><p>2. Problem Definition: Is the problem well-defined and aligned with business goals?</p><p>3. Success Metrics: How will success be measured and evaluated?</p><h4>Data Considerations</h4><p>1. Data Availability: Is relevant data available and accessible?</p><p>2. Data Quality: Is data accurate, complete, and consistent?</p><p>3. Data Volume: Is data sufficient for training and testing?</p><p>4. Data Privacy: Are data handling and storage compliant with regulations?</p><h4>Project Planning</h4><p>1. Budget Constraints: Are resource allocations defined?</p><p>2. Timeline: Is the project timeline realistic?</p><p>3. Stakeholders: Are key stakeholders identified and aligned?</p><p>4. Risk Management: Are potential risks and mitigations identified?</p><h4>Technical Considerations</h4><p>1. Prediction Mode: Online/real-time or batch processing?</p><p>2. Inference Time: Are customer satisfaction thresholds set?</p><p>3. Model Complexity: Is model complexity aligned with data and objectives?</p><p>4. Scalability: Can the solution scale with growing data and traffic?</p><h4>Model Evaluation and Deployment</h4><p>1. Evaluation Metrics: Are metrics defined for model performance?</p><p>2. Model Interpretability: Can model decisions be explained and visualized?</p><p>3. Deployment Strategy: Is a deployment plan in place?</p><p>4. Monitoring and Maintenance: Are processes established for ongoing model evaluation and updates?</p><h4>Ethics and Governance</h4><p>1. Bias Detection: Are measures in place to detect and mitigate bias?</p><p>2. Transparency: Are model decisions and data handling transparent?</p><p>3. Regulatory Compliance: Does the solution comply with relevant regulations?</p><p>4. Intellectual Property: Are IP rights and ownership clear?</p><h4>Conclusion</h4><figure><img alt="https://images.app.goo.gl/QuoFm6J5EGBXdZkt9" src="https://cdn-images-1.medium.com/proxy/0*3UozFiXC3IV1iL5B.jpg" /><figcaption>A photograph of propolis on a bee hive.<br>Photograph by <a href="http://commons.wikimedia.org/wiki/User:Abalg">Abalg</a></figcaption></figure><p>By addressing these comprehensive questions, you’ll ensure a well-planned, executed, and responsible machine learning project. Emulate the strategic planning of lions and honeybees to achieve success in your machine learning endeavors.</p><p>Just as honeybees carefully plan and construct their hives, machine learning engineers must meticulously plan and execute their projects. By doing so, they can create robust, efficient and effective machine learning solutions that drive business value and success</p><p>Thank you for reading !</p><p>In the next article, we’ll dive deeper into <strong>Pre-ML considerations</strong> and explore practical strategies for successful ML project planning. Stay tuned!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=71a0e829f617" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>