<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Ramyashree Suresha on Medium]]></title>
        <description><![CDATA[Stories by Ramyashree Suresha on Medium]]></description>
        <link>https://medium.com/@ramyashreesde96?source=rss-81f205c9654d------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*--1Qrd6eirrtfupgVqWD4w.jpeg</url>
            <title>Stories by Ramyashree Suresha on Medium</title>
            <link>https://medium.com/@ramyashreesde96?source=rss-81f205c9654d------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 00:26:25 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@ramyashreesde96/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Tree-Based Algorithms: What They Are, Why They Work, and How They Make Decisions]]></title>
            <link>https://medium.com/@ramyashreesde96/tree-based-algorithms-what-they-are-why-they-work-and-how-they-make-decisions-2980a9f8f29f?source=rss-81f205c9654d------2</link>
            <guid isPermaLink="false">https://medium.com/p/2980a9f8f29f</guid>
            <category><![CDATA[gini-index]]></category>
            <category><![CDATA[cart]]></category>
            <category><![CDATA[decision-tree]]></category>
            <category><![CDATA[tree-based-algorithms]]></category>
            <category><![CDATA[decision-tree-classifier]]></category>
            <dc:creator><![CDATA[Ramyashree Suresha]]></dc:creator>
            <pubDate>Sat, 14 Mar 2026 03:46:38 GMT</pubDate>
            <atom:updated>2026-03-14T20:17:53.838Z</atom:updated>
            <content:encoded><![CDATA[<p><em>A technical walkthrough of decision trees, Gini impurity, and regression trees — built from my actual study notes, with datasets I made up myself.</em></p><p>Most introductions to machine learning start with linear regression. Draw a line through some data points, minimize the error, done. It’s a clean story — but it falls apart fast when you throw real-world data at it. Relationships between features and targets aren’t linear. They rarely are.</p><p>That’s the core motivation behind tree-based algorithms. Instead of fitting a single equation, they ask a sequence of yes/no questions about features until they arrive at a prediction. Simple in concept, surprisingly powerful in practice.</p><p>I’ve been working through these in detail recently. Below is a structured breakdown — with full manual Gini and MSE calculations, explained step by step.</p><h3>Why Tree-Based Algorithms Exist</h3><p>The fundamental problem with linear models is their assumption: that a unit change in feature <em>x</em> produces a constant change in target <em>y</em>. This almost never holds in practice.</p><p>In real-world problems, the relationship between features and target is non-linear. We need models that can capture complex, irregular boundaries — tree-based algorithms do this by learning a hierarchy of binary conditions directly from data.</p><p><strong>Why they’re so popular:</strong></p><ul><li>Handle non-linear relationships natively — no polynomial transforms needed</li><li>Require no feature scaling (unlike SVMs or gradient descent-based models)</li><li>Work for both classification and regression without architecture changes</li><li>A single tree is fully interpretable — every decision can be traced by hand</li></ul><h3>The Algorithm Family</h3><pre>Tree-Based Algorithms<br>│<br>├── Single Tree<br>│   └── CART (Classification and Regression Trees)<br>│<br>└── Ensemble Trees<br>    ├── Bagging  [sampling without replacement]<br>    │   ├── Random Forest<br>    │   └── Pasting<br>    │<br>    └── Boosting  [sequential, error-correcting]<br>        ├── AdaBoost<br>        ├── Gradient Boosting<br>        ├── XGBoost<br>        └── LightGBM</pre><p><strong>Bagging methods</strong> (Random Forest, Pasting) train trees independently on random subsets and average the outputs. This reduces variance.</p><p><strong>Boosting methods</strong> (AdaBoost, Gradient Boosting, XGBoost, LightGBM) train trees sequentially — each tree corrects the errors of the previous one. This reduces bias.</p><p>Single trees are interpretable but prone to overfitting. If you let them grow unconstrained, they’ll memorize training data instead of learning from it. Ensemble methods were built to fix exactly that.</p><h3>Part 1 — Classification Tree</h3><h3>Dataset: Coffee Shop Tipping</h3><p>To make this concrete, I needed a dataset that wasn’t pulled from a textbook. So here’s one I put together: eight coffee shop orders, with three features — <strong>order type</strong>, <strong>time of day</strong>, and <strong>bill amount</strong> — and a single binary target: did the customer leave a tip?</p><p>I’ve intentionally mixed the classes within each feature split so the nodes come out <strong>impure</strong>. That’s the realistic case — and it’s where the Gini calculation actually does something interesting.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*RYcOxPMJeQX4qZ65__ZEIw.png" /></figure><p>8 records. 5 tipped (Yes), 3 didn’t (No). Every feature split will produce impure child nodes — which is exactly what we want for a meaningful walkthrough.</p><h3>Understanding Gini Impurity — Before the Numbers</h3><p><strong>Gini impurity</strong> answers one question: if I randomly pick two samples from this node, what is the probability that they belong to <em>different</em> classes? A pure node has Gini = 0. A maximally mixed node (50/50 binary split) has Gini = 0.5.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*SbXIqxE77HrcXR8egIs_6Q.png" /></figure><p>Quick sanity check:</p><ul><li>Pure node: p(Yes) = 1, p(No) = 0 → Gini = 1 − (¹² + ⁰²) = <strong>0</strong> ✓</li><li>50/50 node: p(Yes) = 0.5, p(No) = 0.5 → Gini = 1 − (0.⁵² + 0.⁵²) = <strong>0.50</strong> ✓</li></ul><p>When evaluating a split, the tree computes Gini for each child node, then combines them as a weighted average proportional to how many samples fell into each child:</p><pre>Weighted Gini = (n_left / n_total) × Gini(left) + (n_right / n_total) × Gini(right)</pre><p>The split with the <strong>lowest weighted Gini</strong> wins and becomes the node. Let’s apply this to all three features.</p><h3>Weighted Gini — Order Type Feature</h3><p>Splitting on <strong>Order Type</strong> gives two groups:</p><ul><li><strong>Dine-in</strong> (IDs 1, 3, 5, 7): Yes = 3, No = 1 → 4 records</li><li><strong>Takeaway</strong> (IDs 2, 4, 6, 8): Yes = 2, No = 2 → 4 records</li></ul><p><strong>Dine-in node:</strong> p(Yes) = 3/4 = 0.75, p(No) = 1/4 = 0.25</p><pre>Gini(Dine-in) = 1 − (0.75)² − (0.25)²<br>              = 1 − 0.5625 − 0.0625<br>              = 0.375</pre><p><strong>Takeaway node:</strong> p(Yes) = 2/4 = 0.50, p(No) = 2/4 = 0.50</p><pre>Gini(Takeaway) = 1 − (0.50)² − (0.50)²<br>               = 1 − 0.25 − 0.25<br>               = 0.500</pre><p>The Takeaway node is maximally impure — a 50/50 split. Dine-in is better at 0.375, but still impure. Now combine:</p><pre>Weighted Gini (Order Type) = (4/8) × 0.375 + (4/8) × 0.500<br>                           = 0.1875 + 0.25<br>                           = 0.4375</pre><h3>Weighted Gini — Time of Day Feature</h3><p>Splitting on <strong>Time of Day</strong> gives three groups:</p><ul><li><strong>Morning</strong> (IDs 1, 4, 7): Yes = 3, No = 0 → pure node</li><li><strong>Afternoon</strong> (IDs 2, 5): Yes = 0, No = 2 → pure node</li><li><strong>Evening</strong> (IDs 3, 6, 8): Yes = 2, No = 1 → impure node</li></ul><pre>Gini(Morning)   = 1 − (3/3)² − (0/3)² = 0.000<br>Gini(Afternoon) = 1 − (0/2)² − (2/2)² = 0.000<br>Gini(Evening)   = 1 − (2/3)² − (1/3)²<br>                = 1 − 0.444 − 0.111<br>                = 0.444</pre><pre>Weighted Gini   = (3/8) × 0.000 + (2/8) × 0.000 + (3/8) × 0.444<br>                = 0 + 0 + 0.1665<br>                = 0.1665</pre><h3>Weighted Gini — Bill Amount (Numerical Feature)</h3><p>For a numerical feature, we sort the values, compute midpoints between consecutive values, and try each midpoint as a split threshold.</p><p>Sorted bills: 75, 90, 110, 130, 150, 180, 210, 320 Midpoints: 82.5, 100, 120, 140, 165, 195, 265</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*klgUaeEJ75QaccRDb46cRA.png" /></figure><p>Best weighted Gini for Bill Amount: <strong>0.367</strong> — still higher than Time of Day.</p><h3>Choosing the Root Node</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*L1nH-GYzy7s_9uEQS1limw.png" /></figure><p><strong>Time of Day wins</strong> with a weighted Gini of 0.1665. Morning and Afternoon nodes are already pure. The tree recurses only on the Evening node (2 Yes, 1 No, Gini = 0.444) — trying Order Type and Bill Amount within just those 3 records to resolve it further.</p><p>The algorithm isn’t looking for a perfect split. It’s looking for the split that reduces impurity the most at each step — a locally greedy search that builds the tree top-down.</p><h3>What Happens Next — Recursion on the Impure Node</h3><p>After the first split on Time of Day, Morning and Afternoon nodes are pure. But the <strong>Evening node</strong> still has a Gini of 0.444. The tree recurses: it treats the Evening records (IDs 3, 6, 8) as a sub-dataset and repeats the whole process on just those 3 records.</p><pre>[ Time of Day? ]<br>        /       |        \<br>    Morning  Afternoon  Evening<br>       ↓         ↓         ↓<br>  Gini=0.00  Gini=0.00  Gini=0.444<br>  Predict:   Predict:    (recurse)<br>    Yes        No      → try Order Type<br>                         or Bill Amount</pre><p>This recursion continues until every leaf is pure, or a stopping condition is hit — like max_depth, min_samples_split, or a minimum Gini improvement threshold.</p><h3>Part 2 — Regression Tree</h3><h3>Dataset: Street Food Vendor Daily Revenue</h3><p>Now let’s switch to regression. Instead of predicting a category, we want to predict a <strong>continuous number</strong>.</p><p>Here’s the setup: a street food vendor records their sales across 8 days. Features are <strong>location type</strong> (market vs. residential), <strong>weather</strong> (sunny vs. cloudy), and <strong>items sold</strong>. The target is <strong>daily revenue in ₹</strong>. We want to build a tree that predicts revenue for a new day given those three inputs.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*oiVYLjFEvkJlAiq0YbG3ig.png" /></figure><p>8 days of data. Revenue varies widely — from ₹560 on a quiet residential cloudy day to ₹2450 at a sunny market. Both location types have a mix of weather conditions and item counts, so no feature will give us perfectly pure nodes.</p><h3>MSE — The Regression Equivalent of Gini</h3><p>For regression, impurity is measured as <strong>Mean Squared Error</strong> — how spread out the target values are around their mean within a node. A node where all revenue values are identical has MSE = 0. Wide variance → high MSE.</p><pre>MSE(node) = (1/n) × Σ (yᵢ − ȳ)²</pre><p>Where ȳ is the mean of the target values at the node. When the tree predicts for a leaf node, it returns this mean. The whole game is: find the split that makes the left and right group means the most representative of their respective groups.</p><h3>Step 1 — Baseline: Root Node MSE (No Split)</h3><pre>ȳ = (2100 + 720 + 1540 + 1080 + 2450 + 560 + 1260 + 1320) / 8<br>  = 11030 / 8<br>  = 1378.75</pre><pre>MSE = (1/8) × [(2100−1378.75)² + (720−1378.75)² + (1540−1378.75)²<br>              + (1080−1378.75)² + (2450−1378.75)² + (560−1378.75)²<br>              + (1260−1378.75)² + (1320−1378.75)²]</pre><pre>    = (1/8) × [519,900.6 + 433,640.6 + 25,990.6 + 89,265.6<br>               + 1,147,640.6 + 670,390.6 + 14,090.6 + 3,452]</pre><pre>    = (1/8) × 2,904,371.4<br>    ≈ 363,046.4</pre><p>Very high baseline MSE. Any good split should cut this down significantly.</p><h3>Step 2 — Try Splitting on Location</h3><ul><li><strong>Market</strong> (IDs 1, 3, 5, 7): revenues ₹2100, ₹1540, ₹2450, ₹1260 → mean = 1837.5</li><li><strong>Residential</strong> (IDs 2, 4, 6, 8): revenues ₹720, ₹1080, ₹560, ₹1320 → mean = 920</li></ul><pre>MSE(Market) = [(2100−1837.5)² + (1540−1837.5)² + (2450−1837.5)² + (1260−1837.5)²] / 4<br>            = [68,906.25 + 88,506.25 + 375,156.25 + 333,306.25] / 4<br>            = 215,468.8</pre><pre>MSE(Residential) = [(720−920)² + (1080−920)² + (560−920)² + (1320−920)²] / 4<br>                 = [40,000 + 25,600 + 129,600 + 160,000] / 4<br>                 = 88,800</pre><pre>Weighted MSE (Location) = (4/8) × 215,468.8 + (4/8) × 88,800<br>                        = 107,734.4 + 44,400<br>                        = 152,134.4</pre><h3>Step 3 — Try Splitting on Weather</h3><ul><li><strong>Sunny</strong> (IDs 1, 4, 5, 8): revenues ₹2100, ₹1080, ₹2450, ₹1320 → mean = 1737.5</li><li><strong>Cloudy</strong> (IDs 2, 3, 6, 7): revenues ₹720, ₹1540, ₹560, ₹1260 → mean = 1020</li></ul><pre>MSE(Sunny)  = [(2100−1737.5)² + (1080−1737.5)² + (2450−1737.5)² + (1320−1737.5)²] / 4<br>            = [131,406.25 + 432,562.5 + 507,656.25 + 174,306.25] / 4<br>            = 311,482.8</pre><pre>MSE(Cloudy) = [(720−1020)² + (1540−1020)² + (560−1020)² + (1260−1020)²] / 4<br>            = [90,000 + 270,400 + 211,600 + 57,600] / 4<br>            = 157,400</pre><pre>Weighted MSE (Weather) = (4/8) × 311,482.8 + (4/8) × 157,400<br>                       = 155,741.4 + 78,700<br>                       = 234,441.4</pre><h3>Step 4 — Try Splitting on Items Sold (Numerical)</h3><p>Sorted items: 14, 18, 27, 29, 33, 35, 42, 50 We evaluate threshold: <strong>Items &lt; 31</strong></p><ul><li><strong>Items &lt; 31</strong> (IDs 2, 4, 6, 7 → items: 18, 27, 14, 29): revenues ₹720, ₹1080, ₹560, ₹1260 → mean = 905</li><li><strong>Items ≥ 31</strong> (IDs 1, 3, 5, 8 → items: 42, 35, 50, 33): revenues ₹2100, ₹1540, ₹2450, ₹1320 → mean = 1852.5</li></ul><pre>MSE(Items &lt; 31) = [(720−905)² + (1080−905)² + (560−905)² + (1260−905)²] / 4<br>                = [34,225 + 30,625 + 119,025 + 126,025] / 4<br>                = 77,475</pre><pre>MSE(Items ≥ 31) = [(2100−1852.5)² + (1540−1852.5)² + (2450−1852.5)² + (1320−1852.5)²] / 4<br>                = [61,256.25 + 97,656.25 + 356,406.25 + 283,256.25] / 4<br>                = 199,643.75</pre><pre>Weighted MSE (Items &lt; 31) = (4/8) × 77,475 + (4/8) × 199,643.75<br>                          = 38,737.5 + 99,821.9<br>                          = 138,559.4</pre><h3>Step 5 — Compare and Choose Root Node</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HU5UtZN52H-dFuJ3rAWNlQ.png" /></figure><p><strong>Items Sold &lt; 31 wins</strong> — a 61.8% reduction from baseline. It edges out Location (58.1%) and comfortably beats Weather (35.4%).</p><p>This is a realistic outcome worth noting: the numerical feature (Items Sold) beat the seemingly obvious categorical one (Location). The tree found that <em>how many items were sold</em> is more informative than <em>where</em> the vendor set up that day.</p><h3>Step 6 — Final Tree and Predictions</h3><pre>[ Items Sold &lt; 31? ]<br>          /                  \<br>        Yes                   No<br>   (14, 18, 27, 29)      (33, 35, 42, 50)<br>         ↓                      ↓<br>   predict ₹905          predict ₹1852.5</pre><ul><li>New day: <strong>25 items sold</strong> → 25 &lt; 31 → go left → predict <strong>₹905</strong></li><li>New day: <strong>40 items sold</strong> → 40 ≥ 31 → go right → predict <strong>₹1852.5</strong></li></ul><p>Both leaf nodes still have non-zero MSE. The tree can recurse — trying Location and Weather within each branch to reduce variance further. On the left branch (low-sales days), a split on Location would likely separate the ₹560 days from the ₹1260 days. max_depth and min_samples_split are what tell the tree when to stop.</p><h3>Wrapping Up</h3><p>Decision trees are one of those algorithms where the math is transparent enough to trace by hand. That’s both their strength and their weakness. A single tree is readable and auditable — but if left unconstrained, it will keep splitting until every leaf holds a single sample, perfectly memorizing training data.</p><p>That’s exactly why ensemble methods exist. Random Forest trains hundreds of trees on random subsets and averages their outputs. XGBoost trains trees sequentially, each one correcting the residual errors of the last. Both of these only make sense once you understand what a single tree is doing at each split — which is just one thing: greedily finding the feature and threshold that reduce impurity the most.</p><p>For classification: impurity is <strong>Gini</strong> (or entropy). For regression: it’s <strong>MSE</strong>. Same logic, different formula. The tree always asks the same question — <em>does this split make the child nodes more homogeneous than the parent?</em></p><p>Next up: <strong>Random Forests</strong> — how bagging works, why averaging trees reduces variance, and what feature importance actually measures. Follow along if you’re working through this material too.</p><p><em>Written by Ramyashree Suresha · All calculations done manually before touching code. Both datasets are entirely made up — but the math is real.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2980a9f8f29f" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>