<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Anurag Pandey on Medium]]></title>
        <description><![CDATA[Stories by Anurag Pandey on Medium]]></description>
        <link>https://medium.com/@anunitb?source=rss-7389704abd15------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*A88XQ9VzGPgb4bJdMeZUxQ.png</url>
            <title>Stories by Anurag Pandey on Medium</title>
            <link>https://medium.com/@anunitb?source=rss-7389704abd15------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sun, 22 Nov 2026 09:32:14 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@anunitb/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[The Processor Everyone Forgot Is Quietly Powering the AI We Use Every Day]]></title>
            <link>https://anunitb.medium.com/the-processor-everyone-forgot-is-quietly-powering-the-ai-we-use-every-day-913eaed30d83?source=rss-7389704abd15------2</link>
            <guid isPermaLink="false">https://medium.com/p/913eaed30d83</guid>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[hardware]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Anurag Pandey]]></dc:creator>
            <pubDate>Sat, 21 Feb 2026 07:20:53 GMT</pubDate>
            <atom:updated>2026-02-21T07:34:22.234Z</atom:updated>
            <content:encoded><![CDATA[<p><em>How the humble CPU got sidelined by the GPU gold rush — and why it’s staging a remarkable comeback.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/768/1*Ld9Y9Kf2tuJDZ9z8nDIqPg.png" /><figcaption>Image by Author</figcaption></figure><p><a href="https://anurag-code.github.io/training_inference_gpu_cpu/src/">CPU vs GPU</a></p><p>When you type a message to an AI assistant and it starts responding word by word, something is happening under the hood that most people in the AI industry spent the last five years ignoring. A processor that predates the deep learning revolution, that was declared architecturally obsolete for AI work, that got pushed to the sidelines while the world scrambled to buy GPUs — that processor is doing a lot of the work.</p><p>The CPU is back. And understanding why tells you something important about where AI is actually headed.</p><p>The CPU is back. And understanding why tells you something important about where AI is actually headed.</p><h3>The GPU Gold Rush</h3><p>To understand the comeback, you first need to understand why CPUs got side lined in the first place.</p><p>When the deep learning revolution took off in the early 2010s, researchers discovered that training neural networks — the process of teaching a model by showing it billions of examples and adjusting billions of internal parameters — was spectacularly well-suited to GPUs. Graphics processing units were originally built to render video games: take millions of pixels, apply the same mathematical transformation to all of them simultaneously, do it sixty times per second. That architectural bias toward massive parallelism turned out to be exactly what training a neural network needed.</p><p>Training is fundamentally a matrix multiplication problem. You have a batch of examples — say, 32 sentences at once — and you multiply that entire batch against a weight matrix, producing outputs. Then you calculate how wrong those outputs were, and work backwards through the network updating every weight accordingly. Every step of this process is deeply parallel: thousands of calculations that are independent of each other, begging to be done simultaneously. GPUs, with their thousands of simple cores all firing at once, were a natural fit. CPUs, with their handful of powerful but sequential cores, were not.</p><p><a href="https://anurag-code.github.io/training_inference_gpu_cpu/src/matrix_operations.html">Matrix Operations - GPU Training vs CPU Inference</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*SwwoKyYfcawraLdl6YuAHA.png" /><figcaption>For Animation click here: <a href="https://anurag-code.github.io/training_inference_gpu_cpu/src/matrix_operations.html">During Training</a></figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tpbeqLzeRYxnOm33K5noKQ.png" /></figure><p>So the industry built itself around GPUs. NVIDIA became one of the most valuable companies on earth. Cloud providers raced to accumulate GPU clusters. The benchmark for AI ambition became how many H100s you had access to. CPUs, in this framing, were the unsexy legacy hardware you ran your operating system on while the real work happened elsewhere.</p><h3>What Nobody Talked About: The Inference Problem</h3><p>Here’s what got lost in the GPU frenzy: training is only half the story. The other half is <em>inference</em> — actually using a trained model to generate responses.</p><p>And inference is a completely different kind of problem.</p><p>When you ask an AI to generate text, the model doesn’t produce your entire response in one shot. It generates one token at a time — one word, or fragment of a word. For each token, the model has to read its entire set of weights (billions of floating-point numbers, potentially tens of gigabytes of data), do a small amount of arithmetic, and output one token. Then it does the entire thing again for the next token. And again. And again.</p><p>Think about what that means mathematically. During training, you’re multiplying a large matrix (your batch of examples) by another large matrix (the model weights). Thousands of independent calculations happen simultaneously, and your GPU cores are fully occupied. During inference, your “batch” is just one token — one row of that matrix instead of hundreds. The operation shrinks from matrix × matrix to vector × matrix. Most of your GPU cores are sitting idle because there simply isn’t enough parallel work to give them.</p><p>The operation stops being compute-bound — limited by how fast you can do math — and becomes memory-bound: limited by how fast you can read data from memory. You’re not waiting for arithmetic. You’re waiting for gigabytes of weights to travel from storage to processor on every single token.</p><p>This is the architectural mismatch that went under-discussed for years. The industry built its entire hardware stack around training, deployed those same GPUs for inference because it was convenient, and quietly accepted the inefficiency.</p><h3>Why CPUs Are Actually Better Suited Here</h3><p>CPUs were built for sequential, memory-streaming workloads. They have sophisticated cache hierarchies — multiple layers of fast memory sitting close to the processor, designed to anticipate what data will be needed next and fetch it in advance. They are extraordinarily good at the “read a lot, compute a little” pattern that inference represents.</p><p><a href="https://anurag-code.github.io/training_inference_gpu_cpu/src/matrix_operations.html">Matrix Operations - GPU Training vs CPU Inference</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*mlsDCrs51rmf4pgSLn6f4Q.png" /><figcaption>For Animation click here: <a href="https://anurag-code.github.io/training_inference_gpu_cpu/src/matrix_operations.html">During Inference</a></figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xTuZGJlP6qdh2bI1Q6gEww.png" /></figure><p>They also have one overwhelming practical advantage: RAM. A server-grade CPU can access hundreds of gigabytes of standard system memory, which is cheap, abundant, and fast enough for inference workloads. Running a large language model on a GPU requires fitting it into VRAM — specialized memory soldered directly onto the GPU card — which is expensive, limited in capacity, and becomes a hard constraint when you want to run large models.</p><p>This matters enormously for what inference actually looks like at scale. When a billion people use an AI-powered product, the vast majority of those interactions are inference. Someone asking a chatbot a question. A recommendation system deciding which post to show you. An AI feature summarizing an email. Training happens once (or periodically, at great expense). Inference happens constantly, across an almost unimaginable number of requests, with tight requirements on latency, cost, and energy consumption.</p><p>The economics of inference at scale are profoundly different from the economics of training. And increasingly, the hardware needs to reflect that.</p><h3>Meta, NVIDIA, and the Signal in the Noise</h3><p>In February 2026, NVIDIA announced a sweeping multiyear partnership with Meta. The headline was the expected one: millions of NVIDIA Blackwell and Rubin GPUs to power Meta’s AI training clusters. But buried in the announcement — and far more interesting — was a quieter signal.</p><p>Meta and NVIDIA are deploying NVIDIA Grace CPUs at large scale across Meta’s data center production infrastructure. The press release called it <em>“the first large-scale NVIDIA Grace-only deployment.”</em> Grace is NVIDIA’s ARM-based CPU, purpose-built for data center workloads with a particular emphasis on energy efficiency and memory bandwidth.</p><p><em>Read that again: NVIDIA — a company whose entire identity is built on the GPU — is now making CPUs, and one of the world’s largest AI infrastructure deployments is using those CPUs specifically for production workloads.</em> <em>The same infrastructure that serves billions of daily active users across Facebook, Instagram, and WhatsApp.</em></p><p>This isn’t a footnote. It’s a strategic statement about how the industry understands the inference problem. Meta’s production workloads — the personalization systems, the feed ranking, the content recommendations that run continuously at planetary scale — are being powered by CPUs, not GPUs. The GPU clusters handle training. The CPUs handle the constant, relentless, economically sensitive work of serving AI to actual humans.</p><p><em>The announcement also mentioned NVIDIA Vera CPUs, the next generation, with “potential for large-scale deployment in 2027.” NVIDIA is treating CPU development as a long-term infrastructure investment, not a side project.</em></p><h3>What This Means If You’re Not an Engineer</h3><p><em>You might reasonably ask: why should I care which type of processor is serving my AI responses?</em></p><p>Here’s why it matters to you as an AI user.</p><p><strong>Speed:</strong> Inference efficiency directly affects how fast AI responds. A well-matched processor running a memory-bound workload doesn’t stall waiting for data. Better hardware matching means faster token generation, which means the AI types its response to you more quickly.</p><p><strong>Cost:</strong> GPU time is expensive. Inference on GPUs that are architecturally mismatched to the workload wastes money. That cost eventually flows through to pricing — whether you pay for AI subscriptions or the companies building AI products do. <em>Efficient inference infrastructure means AI products can reach more people at lower cost.</em></p><p><strong>Energy:</strong> This one is increasingly urgent. The energy consumption of AI infrastructure has become a serious concern, both environmentally and practically (data centers are straining power grids in some regions). The Meta/NVIDIA announcement explicitly highlights <em>“significant performance-per-watt improvements” as a primary motivation for the CPU deployment.</em> Running the right processor for the job is not just a performance question — it’s an energy question.</p><p><strong>Access:</strong> <em>As inference hardware becomes more efficient and more varied, AI moves closer to the edge — to your phone, your laptop, your local device. CPUs are everywhere</em>. The architecture that runs your device already. The shift toward CPU-friendly inference workloads is part of what enables smaller, faster, locally-run AI models that don’t require a cloud server to function.</p><h3>The Deeper Shift</h3><p><em>What the CPU comeback represents is a maturation of the industry’s thinking.</em></p><p>The GPU gold rush was necessary and correct for its time. You couldn’t have trained GPT, LLaMA, Gemini, or any other frontier model without massive GPU parallelism. That phase of AI development demanded that hardware, and the industry rightly prioritized it.</p><p>But we are now in a different phase. The models are trained. Billions of people are using them. The dominant workload has shifted from training to inference. And the industry is slowly, belatedly catching up to what that shift requires.</p><p>NVIDIA building CPUs. Meta deploying them at hyperscale. Entire research subfields dedicated to efficient inference, model quantization (reducing the precision of weights to shrink memory requirements), and hardware-software co-design. These are all symptoms of the same underlying recognition: the problem has changed, and the hardware stack needs to change with it.</p><p>The CPU wasn’t wrong for AI. It was wrong for training. Inference is a different game, and CPUs have been waiting patiently to play it.</p><h3>A Note on the Diagrams</h3><p><a href="https://anurag-code.github.io/training_inference_gpu_cpu/src/cpu_gpu_diagrams.html">CPU vs GPU</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*VLsr_mOfnZ8Kjsyf2QAUCw.png" /><figcaption><a href="https://anurag-code.github.io/training_inference_gpu_cpu/src/cpu_gpu_diagrams.html">CPU Vs GPU</a></figcaption></figure><p>The diagram above shows something visceral about the difference. Those 8 large CPU cores versus the grid of thousands of tiny GPU cores aren’t just a visual — they represent fundamentally different design philosophies. The GPU was built for the moment when you have thousands of identical calculations begging to be done at once. The CPU was built for the moment when you have one complex sequential task that needs to be done correctly and quickly.</p><p>The below diagrams shows some maths. During training, you’re multiplying a fat matrix against another fat matrix — all those rows representing different training examples, all processed simultaneously. During inference, that left matrix shrinks to a single row. One token. One request. The work that remains is almost entirely just reading data. That’s the CPU’s home territory.</p><p><a href="https://anurag-code.github.io/training_inference_gpu_cpu/src/matrix_operations.html">Matrix Operations - GPU Training vs CPU Inference</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rlgZDe3szGoH6dKHMY5Ftw.png" /><figcaption><a href="https://anurag-code.github.io/training_inference_gpu_cpu/src/matrix_operations.html">During Training</a></figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*woFkZwxkfY1cuvvCWTyqBg.png" /><figcaption><a href="https://anurag-code.github.io/training_inference_gpu_cpu/src/matrix_operations.html">During Inference</a></figcaption></figure><h3>Where This Goes</h3><p>The trajectory is clear. As AI moves from being a research project to being a utility — something woven into every application, every device, every interaction — the economics of inference will dominate the economics of training. Not because training stops mattering, but because inference happens so much more often.</p><p>Every person who uses an AI product generates inference requests every day. Every company that wants to embed AI into their software has to serve those requests efficiently and economically. The question of which hardware handles that load most effectively is not a boutique technical concern — it’s a central question for the entire industry.</p><p>The CPU’s resurgence is the industry’s answer. And it’s only the beginning. Specialized inference chips, ARM-based processors optimized specifically for transformer workloads, memory architectures designed around the vector × matrix operation rather than matrix × matrix — all of this is in active development, driven by the recognition that inference is now the main event.</p><p>The processor everyone forgot is quietly becoming the processor that AI at scale depends on. Jensen Huang and Mark Zuckerberg both signed their names to that fact in February 2026. It might be worth paying attention.</p><p>— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —</p><p>For interactive visualisations visit :</p><p><a href="https://anurag-code.github.io/training_inference_gpu_cpu/src/">CPU vs GPU</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HhK1THwrychA7p0LwxVgJw.png" /><figcaption>Image by Author</figcaption></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=913eaed30d83" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Explainable AI: Application of shapely values in Marketing Analytics]]></title>
            <description><![CDATA[<div class="medium-feed-item"><p class="medium-feed-image"><a href="https://medium.com/data-science/explainable-ai-application-of-shapely-values-in-marketing-analytics-57b716fc9d1f?source=rss-7389704abd15------2"><img src="https://cdn-images-1.medium.com/max/2600/0*K2jZxX05hLbk4urE" width="4000"></a></p><p class="medium-feed-snippet">Intuition, and Implementation of SHAP in Python for Marketing Analytics</p><p class="medium-feed-link"><a href="https://medium.com/data-science/explainable-ai-application-of-shapely-values-in-marketing-analytics-57b716fc9d1f?source=rss-7389704abd15------2">Continue reading on TDS Archive »</a></p></div>]]></description>
            <link>https://medium.com/data-science/explainable-ai-application-of-shapely-values-in-marketing-analytics-57b716fc9d1f?source=rss-7389704abd15------2</link>
            <guid isPermaLink="false">https://medium.com/p/57b716fc9d1f</guid>
            <category><![CDATA[marketing]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[analytics]]></category>
            <category><![CDATA[ai]]></category>
            <dc:creator><![CDATA[Anurag Pandey]]></dc:creator>
            <pubDate>Fri, 30 Oct 2020 15:39:24 GMT</pubDate>
            <atom:updated>2020-12-21T13:45:10.576Z</atom:updated>
        </item>
        <item>
            <title><![CDATA[Curious Case of Decorators in Python]]></title>
            <description><![CDATA[<div class="medium-feed-item"><p class="medium-feed-image"><a href="https://medium.com/data-science/curious-case-of-decorators-in-python-9644a99e538?source=rss-7389704abd15------2"><img src="https://cdn-images-1.medium.com/max/1352/1*nphtlrDbU-l1Tlfsp_nlvg.jpeg" width="1352"></a></p><p class="medium-feed-snippet">Learn to use Functions as an object in Python</p><p class="medium-feed-link"><a href="https://medium.com/data-science/curious-case-of-decorators-in-python-9644a99e538?source=rss-7389704abd15------2">Continue reading on TDS Archive »</a></p></div>]]></description>
            <link>https://medium.com/data-science/curious-case-of-decorators-in-python-9644a99e538?source=rss-7389704abd15------2</link>
            <guid isPermaLink="false">https://medium.com/p/9644a99e538</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[functional-programming]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[python-programming]]></category>
            <dc:creator><![CDATA[Anurag Pandey]]></dc:creator>
            <pubDate>Thu, 01 Oct 2020 07:39:35 GMT</pubDate>
            <atom:updated>2020-11-02T11:09:09.073Z</atom:updated>
        </item>
        <item>
            <title><![CDATA[Survival Analysis: Intuition & Implementation in Python]]></title>
            <description><![CDATA[<div class="medium-feed-item"><p class="medium-feed-image"><a href="https://medium.com/data-science/survival-analysis-intuition-implementation-in-python-504fde4fcf8e?source=rss-7389704abd15------2"><img src="https://cdn-images-1.medium.com/max/856/1*Ckhi9soE9Lx2lIf9tPVLMQ.png" width="856"></a></p><p class="medium-feed-snippet">How long will a particular customer remain with our business? In other words, after how much time this customer will churn?</p><p class="medium-feed-link"><a href="https://medium.com/data-science/survival-analysis-intuition-implementation-in-python-504fde4fcf8e?source=rss-7389704abd15------2">Continue reading on TDS Archive »</a></p></div>]]></description>
            <link>https://medium.com/data-science/survival-analysis-intuition-implementation-in-python-504fde4fcf8e?source=rss-7389704abd15------2</link>
            <guid isPermaLink="false">https://medium.com/p/504fde4fcf8e</guid>
            <category><![CDATA[python]]></category>
            <category><![CDATA[probability]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[analytics]]></category>
            <category><![CDATA[statistical-analysis]]></category>
            <dc:creator><![CDATA[Anurag Pandey]]></dc:creator>
            <pubDate>Sun, 06 Jan 2019 15:23:45 GMT</pubDate>
            <atom:updated>2020-04-24T23:26:18.573Z</atom:updated>
        </item>
        <item>
            <title><![CDATA[How to interpret Concordance in logistic regression?]]></title>
            <description><![CDATA[<div class="medium-feed-item"><p class="medium-feed-image"><a href="https://anunitb.medium.com/how-to-interpret-concordance-in-logistic-regression-18b0f1d59d75?source=rss-7389704abd15------2"><img src="https://cdn-images-1.medium.com/max/2600/0*RtaZzX9Td-n-YEtN" width="3472"></a></p><p class="medium-feed-snippet">I wrote this article in 2015, and it became quite popular among the data community on Quora. I joined Medium very recently, so I thought&#x2026;</p><p class="medium-feed-link"><a href="https://anunitb.medium.com/how-to-interpret-concordance-in-logistic-regression-18b0f1d59d75?source=rss-7389704abd15------2">Continue reading on Medium »</a></p></div>]]></description>
            <link>https://anunitb.medium.com/how-to-interpret-concordance-in-logistic-regression-18b0f1d59d75?source=rss-7389704abd15------2</link>
            <guid isPermaLink="false">https://medium.com/p/18b0f1d59d75</guid>
            <category><![CDATA[r-programming]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[logistic-regression]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Anurag Pandey]]></dc:creator>
            <pubDate>Fri, 26 Oct 2018 12:49:34 GMT</pubDate>
            <atom:updated>2020-12-03T01:41:26.744Z</atom:updated>
        </item>
        <item>
            <title><![CDATA[Split-Apply-Combine Strategy for Data Mining.]]></title>
            <description><![CDATA[<div class="medium-feed-item"><p class="medium-feed-image"><a href="https://medium.com/analytics-vidhya/split-apply-combine-strategy-for-data-mining-4fd6e2a0cc99?source=rss-7389704abd15------2"><img src="https://cdn-images-1.medium.com/max/600/0*C0VRpep8xusyKj8d" width="600"></a></p><p class="medium-feed-snippet">In a typical exploratory data analysis we approach the problem by dividing the data set at some granular level and then aggregating the&#x2026;</p><p class="medium-feed-link"><a href="https://medium.com/analytics-vidhya/split-apply-combine-strategy-for-data-mining-4fd6e2a0cc99?source=rss-7389704abd15------2">Continue reading on Analytics Vidhya »</a></p></div>]]></description>
            <link>https://medium.com/analytics-vidhya/split-apply-combine-strategy-for-data-mining-4fd6e2a0cc99?source=rss-7389704abd15------2</link>
            <guid isPermaLink="false">https://medium.com/p/4fd6e2a0cc99</guid>
            <category><![CDATA[pandas]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[feature-engineering]]></category>
            <category><![CDATA[python]]></category>
            <dc:creator><![CDATA[Anurag Pandey]]></dc:creator>
            <pubDate>Fri, 26 Oct 2018 01:54:39 GMT</pubDate>
            <atom:updated>2020-10-07T02:30:31.293Z</atom:updated>
        </item>
    </channel>
</rss>