<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Ashish Mishra on Medium]]></title>
        <description><![CDATA[Stories by Ashish Mishra on Medium]]></description>
        <link>https://medium.com/@meashumishra76?source=rss-52937aa784e4------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*WufP_RE7mpQcivC6l2BOgQ.png</url>
            <title>Stories by Ashish Mishra on Medium</title>
            <link>https://medium.com/@meashumishra76?source=rss-52937aa784e4------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 16:37:04 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@meashumishra76/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Types of Loss Functions in Machine Learning — MSE, MAE and Huber Explained Simply]]></title>
            <link>https://medium.com/@meashumishra76/types-of-loss-functions-in-machine-learning-mse-mae-and-huber-explained-simply-e1781af90638?source=rss-52937aa784e4------2</link>
            <guid isPermaLink="false">https://medium.com/p/e1781af90638</guid>
            <category><![CDATA[loss-function]]></category>
            <category><![CDATA[mlbeginner]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[linear-regression]]></category>
            <dc:creator><![CDATA[Ashish Mishra]]></dc:creator>
            <pubDate>Fri, 27 Mar 2026 14:32:10 GMT</pubDate>
            <atom:updated>2026-03-27T14:32:10.248Z</atom:updated>
            <content:encoded><![CDATA[<h3>A simple guide to loss functions — the secret scorekeepers behind every model that learns.</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wh0789DmxN8t__YzcYNTdQ.png" /></figure><p>Imagine teaching a child to guess your age. They say “70.” You’re 28. You don’t just say <em>“wrong”</em> — you say <em>“you’re off by 42 years!”</em></p><p>That gap, that distance between guess and reality, is exactly what a <strong>loss function</strong> measures in machine learning.</p><p>Every ML model starts out completely clueless. It makes random predictions, gets told how wrong it was, and slowly adjusts. The thing that tells it <em>how wrong</em> it was? That’s the loss function.</p><blockquote>“Loss functions are the model’s report card. Without them, there’s no learning — just guessing forever.”</blockquote><h3>First — What’s a Regression Problem?</h3><p>Quick distinction before we dive in. There are two main types of ML problems:</p><ul><li><strong>Regression</strong> — predict a number. <em>“What will this house sell for?”</em> → ₹45 lakh.</li><li><strong>Classification</strong> — predict a category. <em>“Does this person have diabetes?”</em> → Yes or No.</li></ul><p>Loss functions are different for each type. Today we’re focusing on <strong>regression only</strong>. Classification has its own loss functions (cross-entropy) — that’s a story for another day.</p><h3>The Three Regression Loss Functions</h3><p>Let’s go deep on each one.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/11ab7c682d1309e2e0bd3de7811eaf16/href">https://medium.com/media/11ab7c682d1309e2e0bd3de7811eaf16/href</a></iframe><h3>1. MSE — Mean Squared Error</h3><h3>The formula</h3><pre>MSE = (1/n) × Σ(actual − predicted)²</pre><p>In plain English: for every data point, find the gap between actual and predicted, <strong>square it</strong>, then average all of them.</p><h3>The intuition — step by step</h3><p>Let’s say your model is predicting people’s weight from their height. You have two predictions:</p><p><strong>Prediction A:</strong> Model says 67 kg. Actual is 65 kg. Error = 2 kg.</p><p><strong>Prediction B:</strong> Model says 95 kg. Actual is 65 kg. Error = 30 kg.</p><p>Now let’s calculate the MSE loss for each:</p><pre>Prediction A → error = 2  → MSE loss = 2²  = 4<br>Prediction B → error = 30 → MSE loss = 30² = 900</pre><p>Stop and look at those numbers carefully.</p><p>The error in Prediction B is <strong>15× bigger</strong> than Prediction A (30 ÷ 2 = 15).</p><p>But the MSE loss is <strong>225× bigger</strong> (900 ÷ 4 = 225).</p><p>How? Because squaring amplifies the gap:</p><pre>Error grows 15×  →  15² = 225×  loss growth</pre><p>This is the core behaviour of MSE. It does not punish mistakes proportionally. It punishes big mistakes <em>explosively</em>. A mistake that is twice as large doesn’t get penalised twice as much — it gets penalised <strong>four times</strong> as much. A mistake that is ten times larger gets penalised <strong>one hundred times</strong> as much.</p><p>Think of it like this. Imagine two employees:</p><ul><li>Employee A arrives 2 minutes late → fine of ₹4</li><li>Employee B arrives 30 minutes late → fine of ₹900</li></ul><p>Same company, same rule (error²). But Employee B’s fine is 225× larger, not 15× larger. That’s MSE in action — it creates a system where being very wrong is catastrophically expensive.</p><h3>Why is this actually useful?</h3><p>During training, the model adjusts its weights to reduce the loss. If big errors cause an explosively large loss, the model becomes <em>very motivated</em> to fix those big errors first. It naturally prioritises the worst predictions.</p><h3>When to use MSE</h3><p>Use MSE when:</p><ul><li>Your data is <strong>clean</strong> (no weird outlier values)</li><li>You want the model to pay <strong>extra attention to large mistakes</strong></li><li>Example: predicting exam scores, where a 30-point error matters far more than a 2-point error</li></ul><h3>2. MAE — Mean Absolute Error</h3><h3>The formula</h3><pre>MAE = (1/n) × Σ|actual − predicted|</pre><p>Same idea as MSE, but instead of squaring the error we just take the <strong>absolute value</strong> — the straight-line distance, always positive.</p><h3>The intuition — same two predictions</h3><pre>Prediction A → error = 2  → MAE loss = |2|  = 2<br>Prediction B → error = 30 → MAE loss = |30| = 30</pre><p>Prediction B’s error is 15× bigger. The MAE loss is also exactly 15× bigger.</p><p>That’s it. <strong>Linear growth. Proportional. No drama.</strong></p><blockquote>“MAE treats all errors fairly. A mistake twice as big gets penalised exactly twice as hard — no more, no less.”</blockquote><h3>The outlier problem — and why MAE saves you</h3><p>Here’s a real-world scenario. You’re building a model to predict house prices in Mumbai. Your dataset has 99 houses, all priced between ₹40–50 lakh. But House 99 is listed at ₹50 crore — almost certainly a data entry error (someone forgot to write “lakh”).</p><pre>Normal houses: ₹40L, ₹42L, ₹45L, ₹47L ... (98 houses)<br>House 99:      ₹50 crore  ← outlier (data entry mistake)</pre><p>An <strong>outlier</strong> is a data point that is very far from the rest of the data.</p><p>Now imagine your model predicts ₹1 crore for House 99. The error is roughly ₹49 crore.</p><p><strong>With MSE:</strong></p><pre>error = ₹49 crore<br>MSE loss = (49 crore)² = 2,401 crore²</pre><p>This number is so astronomically large that during training, the model spends <em>all its effort</em> trying to fix just that one house. It completely ignores the other 98 normal houses. Result? Its predictions for normal houses become terrible.</p><p><strong>With MAE:</strong></p><pre>error = ₹49 crore<br>MAE loss = |49 crore| = 49 crore</pre><p>Still a large number — but it doesn’t explode. The model treats it proportionally alongside the other houses and still learns well from the majority of the data.</p><h3>When to use MAE</h3><p>Use MAE when:</p><ul><li>Your data has <strong>outliers</strong> — which real-world data almost always does</li><li>Data entry errors, sensor glitches, or freak events are present</li><li>Example: house prices, salary predictions, any messy real-world dataset</li></ul><h3>3. Huber Loss — The Clever Hybrid</h3><h3>The problem with choosing</h3><p>You’ve probably noticed that MSE and MAE each have a weakness:</p><ul><li><strong>MSE</strong> → great smooth training, but destroyed by outliers</li><li><strong>MAE</strong> → handles outliers well, but harder to train with (the gradient never smoothly settles)</li></ul><p>Huber loss says: <em>“Why choose? Let’s use both.”</em></p><h3>How it works</h3><p>Huber has one setting called <strong>delta (δ)</strong>. Think of it as a threshold line:</p><pre>If |error| ≤ δ  →  behave like MSE  (smooth, punishes big errors more)<br>If |error| &gt; δ  →  behave like MAE  (linear, outliers don&#39;t explode)</pre><p>Example with δ = 1:</p><pre>error = 0.5  → below delta → use MSE → loss = 0.5² / 2 = 0.125<br>error = 5.0  → above delta → use MAE → loss = 1 × (5 - 0.5) = 4.5</pre><p>By choosing delta, you control where the switch happens. It’s a tunable dial between two worlds.</p><h3>When to use Huber</h3><p>Use Huber when:</p><ul><li>You want <strong>smooth training</strong> like MSE</li><li>But your data has <strong>some outliers</strong> that need capping</li><li>Example: robotics, self-driving cars — where you want precise control for small errors but don’t want crashes to ruin the whole model</li></ul><h3>See the Difference — A Concrete Numbers Table</h3><p>Here’s what happens to the loss as error grows, for all three functions (δ = 1 for Huber):</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/055b457906d5a5323286f9ce16e3b314/href">https://medium.com/media/055b457906d5a5323286f9ce16e3b314/href</a></iframe><p>Notice how MSE shoots up to <strong>900</strong> when MAE is still only <strong>30</strong>. That’s the squaring explosion. And Huber stays very close to MAE for large errors — it’s doing its job.</p><h3>Quick Comparison — Which One to Pick?</h3><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/e2aee0ad85846a9e480aab1b4446e7f6/href">https://medium.com/media/e2aee0ad85846a9e480aab1b4446e7f6/href</a></iframe><h3>The Big Insight</h3><p>Here’s something most beginners miss.</p><p>MSE’s biggest “weakness” — being extra sensitive to large errors — is actually its biggest <em>strength</em> in the right situation.</p><p>If you’re building a model to identify students who are struggling the most, you <em>want</em> large errors to scream loudly. MSE does that automatically.</p><p>If you’re building a model on messy real-world data full of typos and sensor glitches, you want those outliers to stop screaming. MAE quiets them down.</p><blockquote>“There is no best loss function. There is only the right loss function for your data and your goal.”</blockquote><p>That’s the real skill — not memorising formulas, but asking: <strong>what do I want my model to care about?</strong></p><h3>Key Equations to Remember</h3><pre>MSE   = (1/n) × Σ(yᵢ − ŷᵢ)²<br>MAE   = (1/n) × Σ|yᵢ − ŷᵢ|<br>Huber = 0.5 × error²          if |error| ≤ δ<br>      = δ × (|error| − δ/2)   if |error| &gt; δ</pre><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e1781af90638" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How Machines Learn: Supervised, Unsupervised, and Reinforcement Learning Explained With Real…]]></title>
            <link>https://medium.com/@meashumishra76/how-machines-learn-supervised-unsupervised-and-reinforcement-learning-explained-with-real-1775a7e1d6f9?source=rss-52937aa784e4------2</link>
            <guid isPermaLink="false">https://medium.com/p/1775a7e1d6f9</guid>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[technology]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Ashish Mishra]]></dc:creator>
            <pubDate>Thu, 26 Mar 2026 17:23:28 GMT</pubDate>
            <atom:updated>2026-03-26T17:23:28.430Z</atom:updated>
            <content:encoded><![CDATA[<h3>How Machines Learn: Supervised, Unsupervised, and Reinforcement Learning Explained With Real Examples</h3><p><em>No math. No jargon. Just honest explanations of the ideas powering the technology you use every day.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9RzrZqHjBViqPzMsjogyeQ.png" /></figure><p>Yesterday, Spotify played five songs you had never searched for. Every single one felt like it was made for you.</p><p>That is not magic. And it is not a coincidence.</p><p>But here is the part most people miss — there was no human at Spotify who looked at your history and thought “this person might like this.” Nobody curated your playlist. Some algorithm did. And understanding <em>how</em> that algorithm thinks is more interesting than most people expect.</p><p>Machines learn in three fundamentally different ways. Once you see the difference between them, you start noticing these patterns everywhere — in your email inbox, in your banking app, in the games that seem to read your mind. This is that explanation.</p><h3>What Is Supervised Learning? (And Why Your Spam Filter Depends on It)</h3><p>Here is the simplest way I know to put it.</p><p>When you were a child and learning to read, a teacher pointed at the letter A and said <em>“this is A.”</em> Then B. Then C. Enough examples, and something clicked — you could recognize letters you had never seen written in that exact font before. You were not memorizing shapes. You were learning a pattern.</p><p>Supervised learning is a machine doing that same thing, at scale.</p><p>You take thousands — sometimes millions — of examples where every input already has the correct answer attached. You feed all of it to the algorithm. It finds the pattern connecting inputs to outputs. After that, you give it inputs it has never seen, and it predicts the output.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*FfCbrQE86ZBZfp8yGYxiJA.png" /><figcaption><em>Step 1 — the model trains on labeled examples. Step 2 — it predicts on inputs it has never seen</em></figcaption></figure><p>A concrete example: imagine a hospital with 100,000 old X-ray scans. Doctors have already reviewed every one and labeled each scan as <em>“healthy”</em> or <em>“cancer.”</em> That labeled collection is the training data. The algorithm studies it, picks up on visual patterns a healthy lung has that a cancerous one does not, and builds an internal understanding of the difference.</p><p>Now give it a scan it has never seen. It does not look up the answer. It applies what it learned. In under a second.</p><p>That is what supervised learning does. Gmail uses it for spam. Google Photos uses it to recognize your face. Every voice assistant uses it to understand what you just said.</p><p>But there is an uncomfortable reality here. All of that only works because a human labeled the training data first. Someone had to sit down and mark 100,000 scans. Someone reviewed millions of emails and decided which ones were spam. That is expensive. That is slow. And for a lot of problems, that labeled data simply does not exist.</p><p>Which is where things get more interesting.</p><h3>What Is Unsupervised Learning? (The Netflix Problem Nobody Talks About)</h3><p>In 2009, Netflix had roughly 10 million subscribers and a mountain of behavioral data — every play, every pause, every show someone abandoned after 8 minutes. But none of it was labeled. There was no column in their database that said <em>“this person is a thriller addict”</em> or <em>“this user only watches on Sunday nights.”</em> Those categories had not been invented yet.</p><p>They needed to understand their users. But they could not ask them. And they could not label 10 million profiles by hand.</p><p>So they let the data speak.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gld_HozkGBkvWcZuRtDk_w.png" /><figcaption><em>Raw unlabeled data on the left — just dots. Natural groups discovered by the algorithm on the right, with no human guidance</em></figcaption></figure><p>Unsupervised learning has no correct answers to chase. You give the algorithm raw data and one instruction: <em>find structure.</em> It groups similar things together, surfaces patterns, and shows you what naturally belongs with what — without you defining the categories in advance.</p><p>When Netflix did this, the algorithm found things nobody had put there. Binge-watchers who consumed full seasons in a weekend. Casual viewers who watched one episode before bed, most weeknights. Film-only buffs who never touched a TV series. Three distinct groups, sitting inside the data the whole time. The algorithm just made them visible.</p><p>And this is not only useful for recommendations. Researchers have used unsupervised learning to discover new subtypes of cancer — by running patient genetic data through clustering algorithms and finding groupings that no oncologist had identified before. Banks use it to catch fraud, not by matching against a list of known fraud patterns, but by flagging transactions that just do not look like anything else in the data.</p><blockquote><em>The difference matters. Supervised learning finds answers you already defined. Unsupervised learning finds questions you had not thought to ask.</em></blockquote><p>But both of these approaches share a limit. They are reactive. You give them data, they produce output. Neither one can do something more fundamentally human — figure out how to behave in a world that changes based on what you do.</p><h3>What Is Reinforcement Learning? (How AlphaZero Became the Best Chess Player Alive in 24 Hours)</h3><p>Nobody taught you to ride a bicycle by handing you a manual.</p><p>You sat on the seat. You tried to pedal. You fell almost immediately. You tried again. Something about the way you shifted your weight that time kept you upright for two seconds longer. Your brain noticed. You did not, consciously — but your brain did. That tiny signal, repeated across hundreds of falls and recoveries, built the skill from scratch.</p><p>Reinforcement learning is that loop, running inside a machine.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zVmC7uUESrhfeQODBMTkhA.png" /><figcaption><em>Skill starts near zero and compounds over thousands of attempts — not because anyone programmed it, but because consequences teach</em></figcaption></figure><p>The setup is always three things: an <strong>agent</strong> (the learner), an <strong>environment</strong> (the world it operates in), and a <strong>reward signal</strong> (feedback on whether what just happened was good or bad). The agent takes actions. The environment responds. The reward signal says <em>yes, more of that</em> or <em>no, not that.</em> Over millions of attempts, the agent develops a strategy.</p><p>No dataset. No labels. Just consequences.</p><p>In 2017, Google DeepMind released AlphaZero. They gave it the rules of chess — nothing else. No human games to study, no opening books, no endgame theory. It played against itself, starting from complete ignorance.</p><p>After 4 hours, it was playing at the level of a strong amateur. After 9 hours, it was beating most dedicated chess programs. After 24 hours, it beat Stockfish — the strongest traditional chess engine in the world — winning 28 games, drawing 72, and losing zero.</p><p>It invented opening strategies that human grandmasters had never considered. Some of them are now being studied by professional players.</p><p>Nobody programmed AlphaZero to be brilliant. It found brilliance because it had the freedom to fail, over and over, until it stopped failing.</p><p>This same principle is running in Google’s data centers right now, managing cooling systems across thousands of servers, having reduced energy consumption by 40%. It is how robotics companies train machines to walk on uneven terrain — millions of virtual falls in simulation before a single real step. And it is partly how the AI assistants you use daily were trained to respond helpfully rather than just accurately.</p><blockquote><em>The central tension in reinforcement learning is one every human recognizes immediately: do you stick with what already works, or try something that might be better? An agent that never explores gets trapped with the first decent strategy it stumbles on. One that never stops exploring never gets consistently good. The best systems — and honestly, the best people — find the balance.</em></blockquote><h3>So Which One Is Spotify Using?</h3><p>All three. At the same time.</p><p>Unsupervised learning figured out what kind of listener you are — without you ever telling it. Supervised learning predicts whether you will enjoy a specific song, trained on millions of labeled skip-or-not signals from other users. And reinforcement learning optimizes your entire listening session, treating every recommendation as a decision and your continued listening as the reward.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*oSZvmePHYaqEplM99YlRPA.png" /><figcaption><em>The three approaches side by side — what each one asks, what it learns from, and where you already encounter it</em></figcaption></figure><p>They are not competitors. They are layers. Most real AI systems you interact with are built from combinations of all three, each handling the part it is best at.</p><p>Understanding this does not make you a machine learning engineer. But it does make you someone who can look at any AI product — a recommendation engine, a fraud detector, a game-playing agent, a chatbot — and have a reasonable mental model of what is actually happening inside it.</p><p>The machines around you did not arrive knowing what they know. They learned it. And now, so do you.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1775a7e1d6f9" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How Machine Learning Actually Works: 4 Core Concepts Explained Simply]]></title>
            <link>https://medium.com/@meashumishra76/how-machine-learning-actually-works-4-core-concepts-explained-simply-16849b55fd66?source=rss-52937aa784e4------2</link>
            <guid isPermaLink="false">https://medium.com/p/16849b55fd66</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[deep-learning]]></category>
            <dc:creator><![CDATA[Ashish Mishra]]></dc:creator>
            <pubDate>Mon, 23 Mar 2026 05:54:27 GMT</pubDate>
            <atom:updated>2026-03-23T05:54:27.509Z</atom:updated>
            <content:encoded><![CDATA[<p><em>Model training, cost functions, gradient descent, and the chain rule — broken down so they actually make sense.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*UldFdQ8bhWPER4UjKTvc1g.png" /></figure><p>Every machine learning model — from a spam filter to ChatGPT — learns through the same fundamental process. It makes a prediction, checks how wrong it was, figures out what to adjust, and tries again. Thousands of times.</p><p>That process relies on exactly four concepts: model creation, cost functions, gradient descent, and the chain rule. Most explanations of these ideas jump straight into calculus and lose people within the first paragraph. This article takes a different approach. We will build intuition through a real-world analogy — running a chai stall — and then connect each idea to the actual math. By the end, you will understand not just what these concepts are, but why they work and how they fit together.</p><h3>What Is a Model?</h3><p>A model is a mathematical equation with adjustable settings called parameters. It takes in data and produces a prediction.</p><p>The simplest useful model is a straight line:</p><pre>prediction = w * input + b</pre><p>Here, w (the weight) controls the steepness of the line, and b (the bias) shifts it up or down. These two parameters are the only things the model can adjust. Everything else — the equation structure, the data — stays fixed.</p><p>Consider a concrete example. Suppose we want to predict a person’s weight in kilograms using only their height in centimeters. We have data from five people:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/96e521eb30f3f7b2215cc255f042fda4/href">https://medium.com/media/96e521eb30f3f7b2215cc255f042fda4/href</a></iframe><p>The model’s job is to find values of <strong>w</strong> and <strong>b</strong> such that <strong>w * height + b</strong> produces weight estimates as close to reality as possible.</p><p><strong>Why a straight line and not something more complex?</strong> Three reasons. First, a line is the simplest relationship between two quantities, and in machine learning, simplicity is not a compromise — it is a strategy. A model with two parameters is far less likely to memorize the noise in the data than one with twenty. That failure mode — fitting the noise instead of the real pattern — is called overfitting, and it is one of the most persistent problems in the field. Second, a straight line often works surprisingly well. Many real-world relationships are approximately linear over practical ranges. Third, and most importantly, every complex model in deep learning is constructed from layers of these simple linear equations, each followed by a nonlinear transformation. The straight line is the atom of machine learning. Neural networks are the molecules.</p><p><strong>The chai stall parallel:</strong> Think of the model as your recipe. The amount of sugar and ginger are the adjustable parameters. The recipe takes in raw ingredients and produces a cup of chai. A different combination of sugar and ginger produces a different result. Training is the process of finding the right combination.</p><h3>The Cost Function: Measuring How Wrong You Are</h3><p>At the start of training, w and b are set to arbitrary values — often zero or small random numbers. The predictions will be terrible. The question is: how terrible?</p><p>The cost function answers this by collapsing all prediction errors into a single number. For regression problems, the standard choice is Mean Squared Error (MSE):</p><pre>MSE = (1/n) * sum of (actual — predicted)²</pre><p>For each data point, you compute the difference between the real value and the prediction, square it, and then average all the squared differences.</p><p>Squaring serves three purposes. It ensures errors in both directions contribute positively — without squaring, a prediction five kilograms too high and another five kilograms too low would cancel out, making the model appear accurate when it is not. It punishes large errors disproportionately — an error of ten contributes one hundred to the cost while an error of two contributes only four. And it produces a smooth, continuous curve, which is essential for the optimization technique that comes next.</p><p><strong>The chai stall parallel:</strong> A customer takes a sip and says, “I wanted a sweetness of 5 out of 10, but this is a 7.” That gap of 2 points, squared and averaged across all your customers, is your cost function. It does not tell you what to fix. It tells you how far off you are overall.</p><h3>Gradient Descent: Walking Downhill Blindfolded</h3><p>Knowing the cost is not enough. We need a method for reducing it. That method is gradient descent.</p><p>The intuition is this: imagine standing blindfolded on a hilly landscape. Your altitude represents the cost. You want to reach the lowest valley. You cannot see, but you can feel the slope of the ground beneath your feet. If the ground slopes downward to the left, you step left. If it slopes downward to the right, you step right. Repeat until you reach flat ground.</p><p>That is gradient descent. The slope you feel is called the gradient — the derivative of the cost with respect to a parameter. It tells you two things: which direction is uphill, and how steep the slope is.</p><p>The update rule is a single line:</p><pre>w_new = w_old — learning_rate * gradient</pre><p>The minus sign is doing critical work. If the gradient is positive (cost increases when w increases), the formula decreases w. If the gradient is negative (cost increases when w decreases), the formula increases w. In both cases, w moves in the direction that reduces the cost.</p><h3>The Learning Rate</h3><p>The learning rate is a number chosen before training begins. It controls the size of each step.</p><p>If the learning rate is too small, the model inches toward the optimal values and may require millions of steps to converge. If the learning rate is too large, the model overshoots the minimum, swinging from one side of the valley to the other without settling. In extreme cases, the overshooting compounds at each step and the parameters diverge to infinity. The model does not just fail to learn — it explodes.</p><p>Finding an appropriate learning rate is one of the first practical challenges every ML practitioner encounters. It is not derived from the data. It is set by the engineer through experimentation.</p><p><strong>The chai stall parallel:</strong> Gradient descent is the act of tasting the chai, deciding it is too sweet, and reducing the sugar. The gradient tells you the direction: reduce, not increase. The learning rate determines whether you reduce by a cautious pinch or a reckless spoonful. Too cautious and you spend weeks perfecting the recipe. Too aggressive and you overshoot — too sweet becomes too bitter, and tomorrow you overcorrect back to too sweet, endlessly bouncing.</p><h3>The Chain Rule: Tracing Cause Through Effect</h3><p>Here is the part that trips up most beginners, and the part that is most rewarding once it clicks.</p><p>We want to compute the gradient: how does changing w affect the cost? The difficulty is that w does not affect the cost directly. Instead, w affects the prediction, and the prediction affects the cost. There is an intermediate step, and we need to account for it.</p><h3>The Chai Stall Version</h3><p>Suppose your chai stall has two stages. First, you dissolve sugar in hot water to create a syrup. Then, you add the syrup to the milk-tea mixture to produce the final cup.</p><p>If someone asks, “how much does the final sweetness change if you add one more spoon of sugar?”, you cannot answer in one step. You need two pieces of information:</p><p>1. How much does one spoon of sugar change the syrup concentration?</p><p>2. How much does the syrup concentration change the final sweetness?</p><p>Say one spoon of sugar increases the syrup concentration by 3 units, and each unit of syrup concentration increases the final sweetness by 2 points. Then one spoon of sugar increases the final sweetness by 3 multiplied by 2, which is 6 points.</p><p>You just multiplied rates along a chain. That is the chain rule.</p><h3>The Math Version</h3><p>In notation:</p><pre>dCost/dw = (dCost/dPrediction) * (dPrediction/dw)</pre><p>The first term asks: if the prediction shifts by a small amount, how much does the cost change? For MSE with a single data point, this equals 2 * (prediction — actual).</p><p>The second term asks: if w shifts by a small amount, how much does the prediction change? Since <strong>prediction = w * input + b</strong>, changing <strong>w</strong> by a tiny amount changes the prediction by that amount multiplied by the input. So this term equals the input value — in our case, the person’s height.</p><p>Multiply the two:</p><pre>dCost/dw = 2 * (prediction — actual) * input</pre><p>Notice that the input acts as an amplifier. When the input is a large number like 170, even a tiny change in w creates a large change in the prediction, which creates a large change in the cost. This amplification effect explains why the gradient for w includes the input value but the gradient for b does not — b is added to the prediction without any multiplication, so there is nothing to amplify the change.</p><h3>Why This Matters Beyond Simple Models</h3><p>A neural network with one hundred layers is just a chain with one hundred links. To determine how a parameter in the first layer affects the final output, you multiply one hundred rates along the chain, working backward from the output to the input. This backward traversal has a specific name: backpropagation. Despite its reputation as an advanced concept, it is nothing more than the chain rule applied link by link through a longer chain.</p><h3>Putting It All Together: A Numeric Walkthrough</h3><p>Let us trace one complete training step with actual numbers to make the entire process concrete.</p><p><strong>Setup:</strong> One person in our dataset — height is 170 cm, actual weight is 65 kg. Current parameter values: w = 0.5, b = 10. Learning rate: 0.00001.</p><p><strong>Step 1 — Forward pass:</strong></p><pre>prediction = 0.5 * 170 + 10 = 95 kg</pre><p>The model predicts 95 kg. The actual weight is 65 kg. That is 30 kg too high.</p><p><strong>Step 2 — Compute the cost:</strong></p><pre>cost = (95–65)² = 900</pre><p><strong>Step 3 — Compute the gradient using the chain rule:</strong></p><pre>Link 1: dCost/dPrediction = 2 * (95–65) = 60<br>Link 2: dPrediction/dw = 170 (the input)<br>Multiply: dCost/dw = 60 * 170 = 10,200</pre><p>The gradient of 10,200 is enormous. It means that increasing w by even a tiny amount would increase the cost substantially. The model needs to decrease w.</p><p><strong>Step 4 — Update:</strong></p><pre>w_new = 0.5–0.00001 * 10,200 = 0.398</pre><p>The parameter w moved from 0.5 down to 0.398 — a small nudge in the right direction. Repeat this process a few hundred times, and w converges to approximately 0.75, which means each centimeter of height adds about 0.75 kg to the predicted weight. That is a reasonable real-world relationship.</p><h3>The Practical Trap: Why Normalization Is Not Optional</h3><p>There is an important practical lesson embedded in the walkthrough above. The gradient was 10,200 — a massive number — because the input height of 170 acted as an amplifier through the chain rule. With raw input values in the range of 150 to 190, even a conservatively chosen learning rate can cause the parameters to overshoot. In my own experiment, the first version of the model diverged to infinity within a few training steps.</p><p>The fix is normalization: transforming the data so all values fall within a small range, typically zero to one.</p><pre>normalized = (value — minimum) / (maximum — minimum)</pre><p>After normalization, the heights became 0.0, 0.25, 0.5, 0.75, and 1.0. The gradients dropped from the thousands to single digits. The learning rate could be set to a sensible value of 0.1, and the cost decreased smoothly across two hundred training cycles.</p><p>The normalized model predicted 66.0 kg for someone 170 cm tall — an error of just one kilogram, learned from five data points.</p><p>Normalization is not a theoretical nicety. It is the difference between a model that converges and one that explodes. Every production ML pipeline includes it as a standard preprocessing step.</p><h3>The Complete Picture</h3><p>These four concepts form a closed loop that drives all of machine learning:</p><p>1. <strong>The model</strong> takes data and produces a prediction using adjustable parameters.</p><p>2. <strong>The cost function</strong> measures how far off the predictions are, reduced to a single number.</p><p>3. <strong>Gradient descent</strong> uses the slope of the cost surface to determine which direction to adjust each parameter, and by how much.</p><p>4. <strong>The chain rule</strong> enables the gradient calculation by tracing the effect of each parameter through intermediate steps — multiplying rates along the chain from parameter to prediction to cost.</p><p>This loop repeats hundreds or thousands of times. Each iteration, the parameters shift slightly, the cost drops, and the predictions improve.</p><p>What makes this framework remarkable is its universality. A linear regression model with two parameters uses this loop. A convolutional neural network with millions of parameters uses the same loop. A large language model with billions of parameters uses the same loop. The scale changes. The architecture changes. The data changes. The loop does not.</p><p>Understanding these four concepts does not make you an ML expert. But it gives you something more valuable at the beginning: a mental model that is correct. Every advanced topic you encounter from this point forward — regularization, batch normalization, Adam optimizer, attention mechanisms — is either a refinement of one of these four ideas or a technique for making the loop work better at scale.</p><p>The engine is the same. It always has been.</p><p><em>If you found this useful, follow along for the next article where I break down the three families of machine learning — supervised, unsupervised, and reinforcement learning — and when to reach for each one.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=16849b55fd66" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>