<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Gayatri Nikam on Medium]]></title>
        <description><![CDATA[Stories by Gayatri Nikam on Medium]]></description>
        <link>https://medium.com/@gayatrinikam5103?source=rss-f175867c9dc8------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*o4zKEwTLh4Z-BkH0V-56fg.jpeg</url>
            <title>Stories by Gayatri Nikam on Medium</title>
            <link>https://medium.com/@gayatrinikam5103?source=rss-f175867c9dc8------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 17:47:38 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@gayatrinikam5103/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[ Ridge Regression — Making Your Model Stable (Without Removing Features)]]></title>
            <link>https://medium.com/@gayatrinikam5103/ridge-regression-making-your-model-stable-without-removing-features-49d1f77daf22?source=rss-f175867c9dc8------2</link>
            <guid isPermaLink="false">https://medium.com/p/49d1f77daf22</guid>
            <category><![CDATA[regularization]]></category>
            <category><![CDATA[l2-regularization]]></category>
            <category><![CDATA[ridge-regression]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Gayatri Nikam]]></dc:creator>
            <pubDate>Fri, 17 Apr 2026 12:39:43 GMT</pubDate>
            <atom:updated>2026-04-17T12:42:21.494Z</atom:updated>
            <content:encoded><![CDATA[<p>Welcome back to <strong>100 Days of Machine Learning with Gayatri!</strong> 💻📈✨</p><p>In the last blog, we understood <strong>Regularization</strong> — how we can control overfitting by adding a penalty to our model.</p><p>Now naturally, one question comes up —</p><p>👉 <em>“Okay… but how does this actually work inside a real model?”</em> 🤔</p><p>That’s exactly what we’re going to explore today with <strong>Ridge Regression</strong>.</p><p>Think of Ridge Regression like a calm and practical teacher.</p><p>It doesn’t throw students out of the class (like removing features ❌),<br> but it tells them:</p><blockquote><em>“Everyone gets to stay… but behave properly.” 😄</em></blockquote><h3>🧠 How Ridge Regression is Different from Linear Regression</h3><p>Let’s quickly revisit linear regression.</p><p>In simple terms, linear regression tries to draw the best line that fits the data by minimizing error.</p><p>Sounds perfect, right?</p><p>But here’s the problem…</p><p>When your data has:</p><ul><li>Too many features</li><li>Highly correlated features</li><li>Or a bit of noise</li></ul><p>👉 Linear regression can become unstable.</p><h3>🧩 Imagine This</h3><p>You’re predicting salary 💼 using:</p><ul><li>Years of experience</li><li>Number of projects</li></ul><p>Now these two are naturally related.</p><p>But linear regression might do something weird like:</p><ul><li>Experience → very high positive weight</li><li>Projects → large negative weight</li></ul><p>Even though both are important!</p><p>👉 This happens because the model is trying too hard to fit the data.</p><p>This is where <strong>Ridge Regression steps in</strong>.</p><p>Instead of letting coefficients grow freely, Ridge says:</p><blockquote><em>“Let’s keep things balanced.”</em></blockquote><h3>🧮 The Idea Behind Ridge Regression</h3><p>In linear regression, the model only focuses on reducing prediction error.</p><p>But Ridge changes the goal slightly.</p><p>Now the model tries to do two things at once:</p><ol><li>Fit the data well</li><li>Keep the coefficients small</li></ol><h3>📌 The Formula</h3><p>Instead of just minimizing error, we use:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/240/1*TIjb2izb8IVBQ-Rx1fz5Iw.png" /></figure><p>At first glance, this looks scary… but it’s actually simple.</p><ul><li>The first part is the usual error</li><li>The second part is the penalty</li><li>λ (lambda) decides how strict we are</li></ul><p>👉 So now the model is thinking:</p><blockquote><em>“I want to fit the data… but I shouldn’t use extreme values for weights.”</em></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/500/0*mOxaf7w2JFxHFogz.png" /></figure><h3>🧠 Mathematical Intuition</h3><p>Let’s simplify this.</p><p>Your model is something like:</p><blockquote><em>ŷ = w₁x₁ + w₂x₂ + w₃x₃+….</em></blockquote><p>Each <strong>w (weight)</strong> tells how important a feature is.</p><p>Now here’s the key idea:</p><p>👉 Ridge adds a penalty on <strong>w² (square of weights)</strong></p><h4>🧠 Why is it Called L2 Regularization?</h4><p>You might have noticed that Ridge Regression is also called <strong>L2 Regularization</strong>.</p><p>But what does “L2” even mean? 🤔</p><h4>🎯 The Simple Idea</h4><p>The “L2” comes from the type of penalty we add to the model.</p><p>In Ridge, we penalize the <strong>square of the weights</strong>:</p><p>👉 w₁² + w₂² + w₃² + …</p><p>In mathematics, this is called the <strong>L2 norm</strong>.</p><p>Don’t worry about the fancy term — just remember this:</p><blockquote><em>L2 = sum of squares of values</em></blockquote><p>Which means:</p><ul><li>Small weights → small penalty</li><li>Large weights → very large penalty</li></ul><h4>🧩 Example</h4><ul><li>Weight = 2 → penalty = 4</li><li>Weight = 10 → penalty = 100 😳</li></ul><p>👉 So naturally, the model avoids large weights.</p><p>And this makes it:</p><ul><li>More stable</li><li>Less sensitive to noise</li></ul><h3>⚙️ What Happens During Training? (Gradient Descent Intuition)</h3><p>While training, the model updates weights step by step.</p><p>In normal regression:</p><ul><li>It only tries to reduce error</li></ul><p>But in Ridge:</p><ul><li>It reduces error</li><li>AND pulls weights towards zero</li></ul><p>Think of it like this:</p><p>👉 Every weight has a small “rubber band” attached to zero</p><p>No matter how much it stretches, it gets pulled back.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/0*QoGfRVZgirn140CL.gif" /></figure><h3>🔍 Understanding Ridge Through 5 Key Ideas</h3><p>Instead of memorizing formulas, let’s understand how Ridge <em>behaves</em>.</p><h4>1. How Coefficients Are Affected</h4><p>Ridge doesn’t remove any feature.</p><p>Instead, it gently reduces all coefficients.</p><p>👉 So every feature stays, but with controlled importance.</p><h4>2. Why Larger Values Are Penalized More</h4><p>Because we square the weights, large values grow very fast.</p><p>This means:</p><p>👉 Extreme values get punished heavily</p><p>And the model avoids over-dependence on any one feature.</p><h4>3. Impact on Bias-Variance Tradeoff</h4><p>Here’s where things connect to your previous blog.</p><ul><li>Bias increases a little (Since we are restricting the model: 👉It cannot fit the data perfectly anymore. So: <em>Error due to simplification increases a little</em>)</li><li>Variance decreases a lot(Earlier, your model was reacting too much to data.Small change in data → big change in prediction 😵‍💫.Now, because weights are controlled:👉 Model becomes more stable<br>and 👉 Predictions don’t jump too much. so:Variance reduces significantly ✅ )</li></ul><p>👉 Overall, the model performs better on new data</p><h4>4. Effect on Loss Function</h4><p>Earlier, the model had only one job: reduce error.</p><p>Now it has two:</p><ul><li>Reduce error</li><li>Stay simple</li></ul><p>👉 This balance is what improves generalization</p><h4>5. Why is it Called “Ridge” Regression?</h4><p>Ridge Regression gets its name from <strong>ridge analysis</strong>, where instead of finding just one solution, we find a solution under a constraint (keeping weights small). As we change the value of λ, the solution changes smoothly rather than suddenly jumping.</p><p>You can imagine this like walking along a narrow path on a mountain ridge ⛰️ — the solution moves in a stable and controlled way. That smooth path of solutions is why it’s called <em>Ridge Regression</em>.</p><h3>💡 Practical Tip</h3><p>Always <strong>scale your features</strong> before applying Ridge.</p><p>Because:</p><p>👉 The penalty depends on the size of values</p><p>If one feature is large and another is small, the model may behave unfairly.</p><h3>🚀 Where Do We Use Ridge Regression?</h3><p>Let’s connect this to a real machine learning situation — the kind you’ll actually face while building models.</p><h4>🏠 House Price Prediction</h4><p>Imagine you’re building a model to predict house prices.</p><p>Your dataset has features like:</p><ul><li>Size of the house (sq ft)</li><li>Number of rooms</li><li>Number of bathrooms</li></ul><p>Now think about it…</p><p>👉 Bigger houses usually have more rooms and bathrooms<br> 👉 So these features are highly related (correlated)</p><h4>❗ What Happens Without Ridge?</h4><p>Your model might do something weird like:</p><ul><li>Size → very high weight</li><li>Rooms → negative weight</li></ul><p>Even though both clearly increase price!</p><p>👉 This makes the model unstable and confusing</p><h4>✅ What Ridge Does</h4><p>Ridge Regression understands that:</p><p><em>“These features are related, so instead of fighting… let’s balance them.”</em></p><p>So it:</p><ul><li>Distributes importance across features</li><li>Avoids extreme weights</li><li>Keeps predictions stable</li></ul><h3>✅ Advantages of Ridge Regression</h3><p>Ridge is powerful because it keeps things balanced.</p><p>It reduces overfitting without removing features and works well when features are related to each other.</p><p>It also improves how the model performs on new data, which is what we ultimately care about.</p><h3>⚠️ Limitations</h3><p>But it’s not perfect.</p><p>Since Ridge never makes coefficients zero, it cannot perform feature selection.</p><p>If your dataset has many useless features, they will still stay in the model.</p><p>Also, choosing the right value of λ is important — too small and it won’t help, too large and the model becomes too simple.</p><h3>🧠 One Simple Line to Remember</h3><blockquote><em>Ridge Regression = “Keep all features, but control their influence”</em></blockquote><h3>📝 Quick Summary</h3><ul><li>Ridge adds L2 penalty (squared weights)</li><li>It shrinks coefficients instead of removing them</li><li>Helps with multicollinearity</li><li>Reduces overfitting</li><li>Improves model stability</li></ul><h3>🤝 Closing</h3><p>So next time your model starts behaving unpredictably 😄</p><p>👉 Don’t rush to remove features</p><p>Try <strong>Ridge Regression</strong> and let it bring balance.</p><p>In the next blog, we’ll explore:</p><p>👉 <strong>Lasso Regression (L1)</strong> — where the model actually starts removing features ✂️</p><p>Thanks for being part of this journey 🙌<br> See you in the next chapter of <strong>100 Days of Machine Learning with Gayatri!</strong> 💛</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=49d1f77daf22" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[ Regularization — Teaching Your Model When to Stop Overthinking]]></title>
            <link>https://medium.com/@gayatrinikam5103/regularization-teaching-your-model-when-to-stop-overthinking-7466681c74b3?source=rss-f175867c9dc8------2</link>
            <guid isPermaLink="false">https://medium.com/p/7466681c74b3</guid>
            <category><![CDATA[bias-variance-tradeoff]]></category>
            <category><![CDATA[overfitting]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[regularization]]></category>
            <dc:creator><![CDATA[Gayatri Nikam]]></dc:creator>
            <pubDate>Thu, 16 Apr 2026 09:23:39 GMT</pubDate>
            <atom:updated>2026-04-16T09:23:39.968Z</atom:updated>
            <content:encoded><![CDATA[<p>Welcome back to <strong>100 Days of Machine Learning with Gayatri!</strong> 💻📈✨</p><p>In the last blog, we explored the <strong>bias-variance tradeoff</strong> — where a model can either be too simple (underfitting) or too complex (overfitting).</p><p>If you’ve ever caught yourself thinking —</p><p>👉 <em>“Okay, my model is overfitting… but how do I actually fix it?”</em> 🤔</p><p>— this blog is exactly for you.</p><p>Think of it like this:<br>Your model is like a student. Some students don’t study enough (high bias), while others try to memorize everything (high variance).</p><p>What we need is a way to tell the student:</p><p><em>“Learn properly… but don’t overdo it.”</em></p><p>That’s where <strong>Regularization</strong> comes in.</p><h3>🧠 Why Regularization Came Into the Picture</h3><p>Even after selecting the best features, models can still behave badly.</p><p>Sometimes they:</p><ul><li>Learn noise instead of patterns</li><li>Become too complex</li><li>Perform well only on training data</li></ul><p>This problem is called <strong>overfitting</strong>.</p><p>Imagine predicting house prices 🏠</p><p>Your model says:</p><p><em>“If the house has 2 windows facing east and a red gate → price = ₹52,43,120”</em></p><p>😐 That’s not learning… that’s memorizing.</p><p>So we needed a way to <strong>control the model’s behavior</strong>.</p><p>👉 Regularization was introduced to <strong>reduce overfitting by controlling complexity</strong>.</p><h3>⚙️ What is Regularization? (Simple Idea)</h3><p>Regularization is a technique that:</p><p><strong><em>Adds a penalty to the model for being too complex</em></strong></p><p>Instead of just focusing on accuracy, the model is also asked to:</p><p>👉 “Stay simple.”</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/556/0*qmURXsv8wQYt6vPn.png" /></figure><h3>🧮 How Regularization Works (Core Idea)</h3><p>In machine learning, every model tries to minimize something called <strong>loss</strong>.</p><p>Loss simply means:</p><p>👉 <em>How far your predictions are from actual values</em></p><h3>Without Regularization</h3><p>The model only cares about reducing error:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/260/0*eTJ7vZ_DNiWsqwPZ.png" /></figure><h3>With Regularization</h3><p>We modify the loss:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/416/0*lJ5COWD5sUn4J0EI.png" /></figure><p>Here’s what each part means:</p><ul><li><strong>Original Loss</strong> → How wrong predictions are</li><li><strong>Regularization Term</strong> → Penalizes large weights</li><li><strong>λ (lambda)</strong> → Controls how strict the penalty is</li></ul><p>👉 Larger λ → stronger penalty → simpler model</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/423/0*VBGGxiuqqycnlXGp.png" /></figure><h3>🧠 Mathematical Intuition (But Keep It Chill 😄)</h3><p>Let’s say your model looks like this:</p><p><em>ŷ = w₁x₁ + w₂x₂ + w₃x₃ + …</em></p><p>Now think:</p><ul><li>Large weights → model reacts strongly → complex</li><li>Small weights → smoother model → stable</li></ul><p>👉 Regularization tries to <strong>keep weights small</strong></p><p>Why?</p><p>Because:</p><ul><li>Large weights = overfitting</li><li>Controlled weights = better generalization</li></ul><h3>🚨 When Should You Use Regularization?</h3><p>Now let’s make this practical — this is where things really click.</p><h4>1. Preventing Overfitting</h4><p>If your model performs like this:</p><ul><li>Training accuracy → high</li><li>Test accuracy → low</li></ul><p>👉 That’s overfitting</p><p>🧩 Example:<br>A model scoring 95% on training but only 60% on test data</p><p>👉 Regularization helps simplify the model and improve generalization</p><h4>2. High Dimensional Data</h4><p>When you have too many features compared to data points.</p><p>🧩 Example:</p><ul><li>100 features</li><li>50 data points</li></ul><p>👉 Model tries to learn too much → becomes unstable</p><p>👉 Regularization reduces complexity</p><h4>3. Multicollinearity (Correlated Features)</h4><p>When features are strongly related to each other.</p><p>🧩 Example:</p><ul><li>“Facebook Ads Spend”</li><li>“Instagram Ads Spend”</li></ul><p>👉 Both increase together → model gets confused</p><p>👉 Regularization (especially L2) balances their influence</p><h4>4. Feature Selection</h4><p>Sometimes not all features are useful.</p><p>🧩 Example:<br>Predicting house prices:</p><ul><li>Size ✅</li><li>Location ✅</li><li>Wall color 🤨</li></ul><p>👉 Regularization (L1) can remove useless features</p><h4>5. Model Simplicity &amp; Interpretability</h4><p>Simpler models are easier to understand.</p><p>👉 Important in real-world use cases like healthcare and finance</p><h4>6. Better Real-World Performance</h4><p>Even if your model looks fine…</p><p>👉 Regularization often improves performance on unseen data</p><h3>⚙️ Step-by-Step: What Actually Happens?</h3><p>Here’s what your model does internally:</p><ol><li>Makes predictions</li><li>Calculates error</li><li>Adds penalty for large weights</li><li>Tries to minimize total loss</li></ol><p>👉 So it learns:</p><ul><li>Fit data ✅</li><li>Stay simple ✅</li></ul><h3>🧩 How Regularization Helps</h3><p>Regularization mainly does two things:</p><h4>✂️ Shrinks Weights</h4><ul><li>Reduces importance of less useful features</li><li>Makes model stable</li></ul><h4>🚫 Prevents Over-Complex Learning</h4><ul><li>Stops model from learning noise</li><li>Improves generalization</li></ul><h4>🧬 Types of Regularization</h4><p>Now comes the natural question:</p><p>👉 <em>How do we apply this penalty?</em></p><p>There are mainly two ways:</p><h4>🔹 L1 Regularization (Lasso)</h4><ul><li>Can make some weights <strong>exactly zero</strong></li><li>Performs feature selection</li></ul><p>🧩 Example:<br>Removes “irrelevant” features completely</p><h4>🔹 L2 Regularization (Ridge)</h4><ul><li>Shrinks weights but doesn’t remove them</li><li>Distributes importance across features</li></ul><p>🧩 Example:<br>Handles correlated features smoothly</p><p>👉 These methods were introduced to:</p><ul><li>Control complexity</li><li>Handle too many features</li><li>Stabilize models</li></ul><h3>🧠 One-Line Intuition</h3><ul><li>L1 → removes unnecessary features ✂️</li><li>L2 → reduces impact of all features ⚖️</li></ul><h3>📝 Quick Summary</h3><ul><li>Regularization controls overfitting</li><li>It adds a penalty to the loss function</li><li>Keeps model simple and stable</li><li><strong>Useful when:</strong></li><li>Overfitting</li><li>Too many features</li><li>Correlated features</li><li><strong>Main types:</strong></li><li>L1 → feature selection</li><li>L2 → weight shrinking</li></ul><h3>🤝 Closing</h3><p>So next time your model starts overthinking 😄</p><p>Don’t rebuild everything…</p><p>👉 Just <strong>add a little discipline</strong> using regularization.</p><p>In the next blog, we’ll go deeper into:</p><p>👉 <strong>L1 vs L2 Regularization (Lasso vs Ridge)</strong><br>with clear intuition, math, and real-world examples 🔍</p><p>Thanks for being part of this journey 🙌<br>See you in the next chapter of <strong>100 Days of Machine Learning with Gayatri!</strong> 💛</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7466681c74b3" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[ Bias-Variance Tradeoff — Why Your Model is Either Too Simple or Too Smart]]></title>
            <link>https://medium.com/@gayatrinikam5103/bias-variance-tradeoff-why-your-model-is-either-too-simple-or-too-smart-51f6ba51fc0d?source=rss-f175867c9dc8------2</link>
            <guid isPermaLink="false">https://medium.com/p/51f6ba51fc0d</guid>
            <category><![CDATA[bias-variance-tradeoff]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[regularization]]></category>
            <category><![CDATA[bias]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Gayatri Nikam]]></dc:creator>
            <pubDate>Thu, 16 Apr 2026 09:20:10 GMT</pubDate>
            <atom:updated>2026-04-16T09:20:10.381Z</atom:updated>
            <content:encoded><![CDATA[<p>Welcome back to <strong>100 Days of Machine Learning with Gayatri!</strong> 💻📈✨</p><p>In the last blog, we wrapped up feature selection and even created a cheatsheet to decide <em>which method to use when</em>. That was like organizing your wardrobe — removing unnecessary clothes and keeping only what matters 👗</p><p>But here’s a question…</p><p>👉 What if your model is still making mistakes… even after selecting the best features?</p><p>That’s where today’s concept comes in — <br><strong>Bias vs Variance Tradeoff</strong></p><p>And trust me, this is one of those topics that looks scary… but once it clicks, everything starts making sense 😄</p><h3>🧠 The Real Problem Behind Model Errors</h3><p>Let’s start simple.</p><p>Whenever a model makes a mistake, we usually say:</p><p><em>“Okay, prediction is wrong.”</em></p><p>But in reality, that error has <em>reasons</em>.</p><p>There are mainly <strong>three sources of error</strong>:</p><ul><li>Bias</li><li>Variance</li><li>Irreducible Error (noise)</li></ul><p>Think of it like this:</p><p>Even if you cook perfectly 🍝</p><ul><li>Wrong recipe → your mistake (bias)</li><li>Inconsistent cooking → your mistake (variance)</li><li>Bad ingredients → not your fault (noise)</li></ul><h3>📌 What is Bias?</h3><p>Bias is when your model is <strong>too simple to understand the data properly</strong>.</p><p>It makes strong assumptions like:</p><p><em>“Everything is linear… life is simple… why complicate?” 😌</em></p><p>Sounds peaceful, but… wrong.</p><h3>🧩 Simple intuition:</h3><p>Bias = <strong>difference between average prediction and actual truth</strong></p><p>In mathematical terms (don’t worry, we’ll keep it light):</p><ul><li>True value → f(x)</li><li>Model prediction → ŷ</li></ul><p>Bias is:</p><p>👉 How far your model’s <em>average prediction</em> is from the real value</p><h3>🍕 Example:</h3><p>Let’s say you’re predicting pizza price 🍕</p><p>But your model only uses <strong>size</strong> and ignores toppings, cheese, etc.</p><p>👉 Result:</p><ul><li>Predictions are consistently wrong</li><li>Model is <em>oversimplified</em></li></ul><p>✅ This is <strong>High Bias → Underfitting</strong></p><h3>📌 What is Variance?</h3><p>Variance is when your model is <strong>too sensitive to data</strong>.</p><p>Small change in data → completely different prediction 😵‍💫</p><h3>🧩 Simple intuition:</h3><p>Variance = <strong>how much predictions change if data changes</strong></p><p>Press enter or click to view image in full size</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*FUXEuwVLp4GJjeuo.png" /></figure><h3>🍕 Example:</h3><p>Now your model says:</p><p><em>“If pizza has exactly 3 olives and 2 mushrooms → price = ₹247.63”</em></p><p>😳 Too specific!</p><p>👉 It memorizes training data instead of learning patterns</p><p>✅ This is <strong>High Variance → Overfitting</strong></p><h3>🎯 Expected Value (Quick Intuition)</h3><p>Before bias and variance, we need one small idea:</p><p>👉 <strong>Expected Value = Average outcome over many trials</strong></p><p>If a random variable is <strong>X</strong>, then:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/168/0*10v1427ehQq4kgWT.png" /></figure><h3>🎲 Example:</h3><p>Rolling a dice:</p><ul><li>Values: 1, 2, 3, 4, 5, 6</li><li>Each has probability = 1/6</li></ul><p>So average:</p><p>👉 <strong>E[X] = 3.5</strong></p><h3>Variance (How Much Things Vary)</h3><p>Now we ask:</p><p>👉 “How far are values from the average?”</p><p>Var(X) = E[X²] — (E[X])²</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/697/0*HNX1A4ap93rJjY8O.png" /></figure><h3>Now Connect This to ML</h3><p>In machine learning:</p><ul><li>True value → <strong>f(x)</strong></li><li>Prediction → <strong>ŷ</strong></li></ul><p>But here’s the twist:</p><p>👉 Our model depends on data<br>👉 Data changes → prediction changes</p><p>So prediction is like a <strong>random variable</strong></p><p>Bias is: <em>Difference between </em><strong><em>average prediction</em></strong><em> and true value</em></p><p>Variance is: <em>How much predictions vary around their average</em></p><p>We combine everything:</p><ul><li>Expected value → gives average prediction</li><li>Variance → tells how unstable predictions are</li></ul><p>And that leads to:</p><p>👉 Bias-Variance decomposition (which we saw earlier)</p><p><strong>In ML:</strong></p><p><em>We imagine training the model on many datasets and taking the average prediction</em></p><p>This helps us define bias and variance mathematically.</p><h3>🧮 Bias-Variance Decomposition (The Heart of This Topic)</h3><p>Now comes the main idea.</p><p>The total error of a model can be broken into:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/470/0*B6WdT_K7afzg8StC.png" /></figure><h3>What does this actually mean?</h3><ul><li>Left side → Total prediction error</li><li>Right side:</li><li><strong>Bias²</strong> → error from simplicity</li><li><strong>Variance</strong> → error from sensitivity</li><li><strong>Noise</strong> → error we cannot control</li></ul><h3>🧠 Connecting to your notes (important insight)</h3><p>From your diagram:</p><ul><li>Model prediction:<br><strong>y = f(x) + error</strong></li><li>That error has two parts:</li><li>Reducible error → Bias + Variance</li><li>Irreducible error → noise</li></ul><p>👉 So:</p><p><em>Reducible Error = Bias + Variance</em></p><p>This is the part we try to minimize using better models.</p><h3>🎭 Underfitting vs Overfitting</h3><p>Now everything connects beautifully.</p><h3>🟥 Underfitting (High Bias)</h3><ul><li>Model too simple</li><li>Misses patterns</li><li>Performs badly everywhere</li></ul><p>👉 Like studying only 2 questions for exam 😅</p><h3>🟦 Overfitting (High Variance)</h3><ul><li>Model too complex</li><li>Learns noise</li><li>Great on training, bad on new data</li></ul><p>👉 Like memorizing entire textbook word-to-word 🤯</p><p>Press enter or click to view image in full size</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*pHByMXN-V3-aEHmk.jpg" /></figure><h3>⚖️ Why is There a Tradeoff?</h3><p>This is the most important question.</p><p>👉 Why can’t we reduce both bias and variance?</p><h3>🎯 Simple intuition:</h3><ul><li>Simple model → stable → low variance → high bias</li><li>Complex model → flexible → low bias → high variance</li></ul><p>So:</p><p><em>If you reduce one, the other increases</em></p><h3>📈 Visual thinking (from your graph)</h3><ul><li>Straight line → high bias</li><li>Curvy line → high variance</li><li>Best model → somewhere in between</li></ul><p>Press enter or click to view image in full size</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*urBELpYJrG4lXmzr.png" /></figure><h3>🧩 Bias-Variance Decomposition (Mathematical Thinking)</h3><p>Let’s make this slightly deeper but still simple.</p><p>We look at:</p><p>👉 Expected squared error:</p><ul><li>We average errors over many datasets</li><li>Then split it into:</li></ul><ol><li>Bias² → difference from true function</li><li>Variance → variability across datasets</li><li>Noise → randomness in data</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/267/0*R52gY5DrYZAZBu1z.png" /></figure><h3>🧠 Golden Insight</h3><p><em>A perfect model is not the most complex one…</em></p><p><em>It’s the one that balances bias and variance.</em></p><h3>🧠 Final understanding:</h3><ul><li>Bias → wrong assumptions</li><li>Variance → too much flexibility</li><li>Noise → real-world randomness</li></ul><h3>⚡ One-Line Intuition</h3><ul><li>Bias = model is <strong>too simple</strong></li><li>Variance = model is <strong>too sensitive</strong></li></ul><h3>📝 Quick Summary</h3><ul><li>Every model error has 3 parts: Bias, Variance, Noise</li><li>High Bias → Underfitting</li><li>High Variance → Overfitting</li><li>Tradeoff exists → can’t minimize both</li><li>Goal → find the right balance ⚖️</li></ul><h3>🤝 Closing</h3><p>So next time your model performs badly, don’t immediately change everything 😄</p><p>Just pause and ask:</p><p>👉 “Is my model too simple… or too sensitive?”</p><p>That one question can guide your entire approach.</p><p>In the next blog, we’ll see how <strong>Regularization</strong> helps us control this balance — <br>Basically teaching our model <em>how much to think and how much to ignore</em> 🧠✨</p><p>Thanks for being part of this journey 🙌<br>See you in the next chapter of <strong>100 Days of Machine Learning with Gayatri!</strong> 💛</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=51f6ba51fc0d" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[ Feature Selection Cheatsheet: When to Use Which Method]]></title>
            <link>https://medium.com/@gayatrinikam5103/feature-selection-cheatsheet-when-to-use-which-method-1d204982ce6e?source=rss-f175867c9dc8------2</link>
            <guid isPermaLink="false">https://medium.com/p/1d204982ce6e</guid>
            <category><![CDATA[data-analysis]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[machine-learning-ai]]></category>
            <category><![CDATA[feature-engineering]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Gayatri Nikam]]></dc:creator>
            <pubDate>Wed, 08 Oct 2025 12:14:15 GMT</pubDate>
            <atom:updated>2025-10-08T12:14:15.167Z</atom:updated>
            <content:encoded><![CDATA[<p>After exploring <strong>Filter</strong>, <strong>Wrapper</strong>, and <strong>Embedded</strong> methods in our last three blogs, let’s take a little breather and simplify things.</p><p>If you’ve ever caught yourself wondering —</p><p><em>“Okay, but when should I actually use each of these?” 🤔</em></p><p>— this blog is your quick, no-fluff <strong>cheatsheet</strong>!</p><p>Think of it like choosing the right tool from a toolbox — <br> you wouldn’t use a hammer to tighten a screw, right? Similarly, each feature selection method has its own perfect use case depending on your dataset, model, and goal.</p><p>Let’s jump in 👇</p><h3>⚙️ 1. Filter Methods — “Fast and Independent”</h3><p>These are your <strong>go-to tools</strong> when you want quick, model-independent feature selection. They rely on <strong>statistical tests and correlations</strong> instead of training models.</p><h3>🧩 Variance Threshold</h3><p>Removes all features whose variance doesn’t meet a certain threshold.<br> <strong>Use when:</strong> You have many features and want to eliminate those that barely change (constant or near-constant).<br> <strong>Example:</strong><br> If a feature column “Country” has 99% of values as <em>India</em> and 1% as <em>Others</em>, it doesn’t add much value. You can safely drop it.</p><h3>🔗 Correlation Coefficient</h3><p>Measures correlation between pairs of features. Highly correlated features contain overlapping information.<br> <strong>Use when:</strong> You suspect some features are strongly correlated.<br> <strong>Example:</strong><br> If your dataset has both “Height (cm)” and “Height (inches)”, you only need one — they’re almost perfectly correlated!</p><h3>🧮 Chi-Square Test</h3><p>Checks if there’s a significant association between two <strong>categorical variables</strong>.<br> <strong>Use when:</strong> You have <strong>categorical features</strong> and want to see their dependency on the <strong>target variable</strong>.<br> <strong>Example:</strong><br> Testing if “Gender” is associated with “Purchase Decision” (Yes/No). A high Chi-square score means the feature is informative.</p><h3>🤝 Mutual Information</h3><p>Measures how much knowing one variable reduces uncertainty about another — works for both linear and non-linear relationships.<br> <strong>Use when:</strong> You want to capture <strong>both linear and non-linear</strong> dependencies.<br> <strong>Example:</strong><br> If “Ad Clicks” and “Sales” are related in a non-linear way, mutual information will still detect their connection, unlike simple correlation.</p><h3>📊 ANOVA (Analysis of Variance)</h3><p>Compares the means of different groups to see if at least one differs significantly.<br> <strong>Use when:</strong> You have <strong>categorical independent variables</strong> and a <strong>continuous target variable</strong>.<br> <strong>Example:</strong><br> Testing if “Marketing Channel” (Google, Facebook, YouTube) significantly impacts “Sales Revenue”.</p><h3>🧩 2. Wrapper Methods — “Model in the Loop”</h3><p>Wrapper methods <strong>use machine learning models</strong> to evaluate subsets of features. They’re slower but often more accurate since they test feature combinations directly on the model.</p><h3>🔁 Recursive Feature Elimination (RFE)</h3><p>Recursively removes the least important features based on model performance until the optimal subset is found.<br> <strong>Use when:</strong> You want the model itself to identify which features matter most.<br> <strong>Example:</strong><br> Training a Logistic Regression model multiple times — each round, the weakest feature (lowest coefficient) gets kicked out until only the strong ones remain.</p><h3>➕ Sequential Feature Selection (SFS)</h3><p>Adds or removes features one by one, checking model performance each time.<br> <strong>Use when:</strong> You want to find the best feature subset but can afford some computation time.<br> <strong>Example:</strong><br> Start with no features, then keep adding one at a time until adding new ones no longer improves accuracy.</p><h3>💪 Exhaustive Feature Selection</h3><p>Tests <strong>all possible combinations</strong> of features to find the best-performing subset.<br> <strong>Use when:</strong> You have <strong>very few features</strong> (like under 10), because it’s computationally expensive.<br> <strong>Example:</strong><br> If you have 5 features, it’ll test all ²⁵ = 32 combinations to find the best-performing one.</p><h3>🧠 3. Embedded Methods — “Smart &amp; Automatic”</h3><p>Embedded methods perform <strong>feature selection during model training</strong>. They’re like models that multitask — learning and selecting features at the same time.</p><h3>🧾 Lasso Regression</h3><p>Applies <strong>L1 regularization</strong>, which can shrink some coefficients to zero — effectively performing feature selection.<br> <strong>Use when:</strong> You want a <strong>simple, interpretable model</strong> that automatically ignores weak features.<br> <strong>Example:</strong><br> In predicting house prices, if “Wall Paint Color” has almost no effect, Lasso will shrink its coefficient to zero and drop it.</p><h3>📉 Ridge Regression</h3><p>Uses <strong>L2 regularization</strong>, which penalizes large coefficients but doesn’t eliminate features entirely.<br> <strong>Use when:</strong> You face <strong>multicollinearity</strong> (features correlated with each other) and want to stabilize your model.<br> <strong>Example:</strong><br> If “Square Feet” and “Number of Rooms” are correlated, Ridge will distribute their importance smoothly instead of dropping one.</p><h3>⚖️ Elastic Net</h3><p>Combines <strong>L1 (Lasso)</strong> and <strong>L2 (Ridge)</strong> penalties — best of both worlds.<br> <strong>Use when:</strong> You have <strong>many correlated features</strong> and want some to be eliminated while others are just reduced in impact.<br> <strong>Example:</strong><br> When working with 100+ health indicators, Elastic Net can reduce noise while keeping relevant variables.</p><h3>🌲 Random Forest Importance</h3><p>Random Forests rank features based on how much they reduce impurity in decision trees (Mean Decrease Impurity — MDI).<br> <strong>Use when:</strong> You want to leverage <strong>tree-based models</strong> for quick and effective feature importance.<br> <strong>Example:</strong><br> In a customer churn dataset, Random Forest might show “Monthly Charges” and “Tenure” as top features influencing churn.</p><h3>Conclusion</h3><p>Each method has its charm —</p><ul><li><strong>Filter methods</strong> are like <em>speedy scouts</em> — fast, simple, and model-independent.</li><li><strong>Wrapper methods</strong> are <em>perfectionists</em> — slow but accurate.</li><li><strong>Embedded methods</strong> are <em>smart multitaskers</em> — efficient and model-aware.</li></ul><p>So next time you’re feature selecting, choose wisely based on:<br> ⚡ Data size<br> 🧮 Feature type<br> 🧠 Model you’re using<br> 💻 Computation power</p><p>Because in the end, <strong>the best feature selection method</strong> is the one that fits <em>your data and your goal</em>. 💡</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1d204982ce6e" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[ Feature Selection Part 3: Embedded Methods — When Models Choose Their Own Features]]></title>
            <link>https://medium.com/@gayatrinikam5103/feature-selection-part-3-embedded-methods-when-models-choose-their-own-features-53ee094a8647?source=rss-f175867c9dc8------2</link>
            <guid isPermaLink="false">https://medium.com/p/53ee094a8647</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[feature-engineering]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[embedded-method]]></category>
            <category><![CDATA[machine-learning-ai]]></category>
            <dc:creator><![CDATA[Gayatri Nikam]]></dc:creator>
            <pubDate>Wed, 08 Oct 2025 12:07:51 GMT</pubDate>
            <atom:updated>2025-10-08T12:07:51.142Z</atom:updated>
            <content:encoded><![CDATA[<h3>🌱 Feature Selection Part 3: Embedded Methods — When Models Choose Their Own Features</h3><p>Welcome back to <em>100 Days of Machine Learning with Gayatri!</em><br> In the last blog, we explored <strong>wrapper methods,</strong> where we built multiple models to find the best combination of features. That approach, while accurate, was also like testing every outfit before deciding what to wear—effective but exhausting! 😅</p><p>So today, let’s talk about a smarter and more balanced approach—<strong>Embedded Methods</strong>.<br> These are like having a smart mirror that tells you, <em>“This outfit suits you best!”</em> while you’re still trying it on.<br> In other words, embedded methods <strong>perform feature selection automatically while training the model</strong> — combining the <strong>speed of filter methods</strong> and the <strong>accuracy of wrapper methods</strong>.</p><h3>🧠 What Are Embedded Methods?</h3><p>Embedded methods are feature selection techniques that are <strong>built into the model itself</strong>.<br> Instead of running a separate selection process, these models evaluate feature importance <strong>during training</strong>.</p><p>Think of it like a teacher who grades students while teaching the class — she already knows who’s paying attention and who’s not. The poor performers (less useful features) naturally get less weight or are dropped altogether.</p><p>These methods balance:</p><ul><li><strong>Filter methods’ efficiency</strong> (quick to compute)</li><li><strong>Wrapper methods’ intelligence</strong> (consider how features work together)</li></ul><p>So, you get the best of both worlds — a model that’s both <strong>smart and efficient</strong>.</p><h3>⚙️ How Do Embedded Methods Work?</h3><p>To understand this, let’s look at how some models naturally give <strong>importance to each feature</strong>.</p><p>When a model learns, it assigns <strong>coefficients or importance scores</strong> to features.<br> The bigger the coefficient (positive or negative), the stronger the impact that feature has on predictions.<br> Embedded methods monitor these coefficients and <strong>penalize or reward</strong> them based on their usefulness.</p><p>Let’s break this down for different types of models. 👇</p><h3>📈 Linear Regression (Base Case)</h3><p>In a simple linear regression, the model finds the best-fitting line:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/357/1*8oORHhmI_XaFCisOgDvEpw.png" /></figure><p>Here:</p><ul><li>Y is the target variable</li><li>X1, X2,…Xn are features</li><li>β1, β2,…βn are coefficients (the “importance” of each feature)</li><li>ϵ is the error term</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/355/1*wsB-Y63rE8m47jSrqcbCqg.png" /></figure><p>Linear regression assumes:</p><ol><li>The relationship is linear</li><li>Observations are independent</li><li>Variance of residuals is constant (homoscedasticity)</li><li>Residuals are normally distributed</li><li>Features are not highly correlated (no multicollinearity)</li></ol><p>However, if you have <strong>too many features</strong>, or some are <strong>highly correlated</strong>, it can cause instability.<br> That’s where <strong>regularized models</strong> come in—the heart of embedded feature selection.</p><h3>🧮 Regularized Models</h3><p>Regularization means adding a <strong>penalty term</strong> to the model’s cost function to discourage complexity and overfitting.<br> It automatically <strong>shrinks the coefficients</strong> of less important features — and sometimes, even eliminates them.</p><p>Let’s explore the three most popular regularized models 👇</p><h3>1. LASSO Regression (L1 Regularization)</h3><p>LASSO stands for <strong>Least Absolute Shrinkage and Selection Operator</strong>.<br> It adds the sum of absolute values of coefficients to the cost function:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/209/1*TBeDhJkjgRs-foUHu8ahGw.png" /></figure><p>Here,</p><ul><li><strong>MSE</strong> = Mean Squared Error</li><li><strong>λ (lambda)</strong> = Regularization parameter (controls how strict the penalty is)</li></ul><p>As λ increases, LASSO starts shrinking less important coefficients to <strong>zero</strong>, effectively removing those features.</p><p>🧩 <strong>Example:</strong><br> Imagine predicting house prices with 10 features (size, bedrooms, floors, etc.).<br> If “number of chimneys” barely affects the price, LASSO will reduce its coefficient to zero, automatically dropping it.</p><p>✅ LASSO = “Feature selector by nature”</p><h3>⚖️ 2. Ridge Regression (L2 Regularization)</h3><p>Ridge regression penalizes the <strong>squares</strong> of coefficients instead of their absolute values:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/208/1*qbN8GE2JYR1aAEM-N5bZ2Q.png" /></figure><p>Unlike LASSO, Ridge <strong>never makes coefficients exactly zero</strong> — it only shrinks them closer to zero.<br> So all features stay in the model, but the less important ones have very small weights.</p><p>🧩 <strong>Example:</strong><br> If you’re analyzing student scores based on multiple study habits (hours studied, sleep, notes taken), Ridge ensures all habits get some weight — but major ones like “hours studied” dominate.</p><p>✅ Ridge = “Soft shrinker”</p><h3>🧬 3. Elastic Net (Combination of L1 + L2)</h3><p>Elastic Net combines the strengths of both LASSO and Ridge:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/317/1*ML8ZtlrKf45bkBcCbBvycg.png" /></figure><p>It’s especially helpful when features are <strong>correlated</strong> — for example, if “Facebook Ad Spend” and “Instagram Ad Spend” both rise together.</p><p>✅ Elastic Net = “Balanced player” — it shrinks some features and removes others, depending on correlation strength.</p><h3>🌳 Tree-Based Models</h3><p>Not all embedded methods are linear.<br> Tree-based models like <strong>Decision Trees</strong>, <strong>Random Forests</strong>, and <strong>Gradient Boosting Machines (GBMs)</strong> also perform feature selection internally.</p><p>These models split the data based on features that reduce impurity (like Gini index or entropy).<br> Features that contribute more to reducing impurity are considered <strong>more important</strong>.</p><p>🧩 <strong>Example:</strong><br> If you’re predicting whether a patient has diabetes:</p><ul><li>“Blood sugar level” might appear near the top of the tree (most important)</li><li>“Age” might appear deeper (less important)</li></ul><p>At the end, tree models can directly give <strong>feature importance scores</strong> — helping you rank which variables matter most.</p><p>✅ Tree-based models = “Decision-makers by design”</p><h3>♻️ Recursive Feature Elimination (RFE) — A Hybrid Touch</h3><p>RFE is both a <strong>wrapper</strong> and an <strong>embedded</strong> method.<br> It trains a model (like Linear or Logistic Regression), evaluates feature importance, and <strong>recursively removes</strong> the least important features until an optimal subset remains.</p><p>It’s like peeling layers of an onion — you keep removing weaker layers until you reach the core.</p><p>🧩 Example:<br> You train a model with 20 features → remove the weakest 5 → retrain → remove 5 more → stop when accuracy stops improving.</p><h3>⚖️ Advantages and Disadvantages of Embedded Methods</h3><h3>✅ Advantages</h3><ol><li><strong>Performance:</strong> More accurate than filter methods since they consider feature interactions.</li><li><strong>Efficiency:</strong> Faster than wrapper methods since the model is trained only once.</li><li><strong>Less Overfitting:</strong> Regularization adds discipline, preventing the model from memorizing noise.</li></ol><h3>⚠️ Disadvantages</h3><ol><li><strong>Model Specific:</strong> Works well only with models that have built-in selection ability (like regression or trees).</li><li><strong>Complexity:</strong> Harder to interpret why certain features were penalized.</li><li><strong>Tuning Needed:</strong> Regularization strength (λ) must be chosen carefully.</li><li><strong>Stability Issues:</strong> Small data changes can lead to different feature subsets.</li></ol><h3>Conclusion</h3><p>Embedded methods act like smart learners — they not only study but also decide what’s worth learning along the way.<br> They balance the <strong>speed</strong> of filters and the <strong>depth</strong> of wrappers — a perfect middle path in your feature selection journey.</p><p>For the last <em>three blogs</em>, we’ve been deep-diving into the world of <strong>feature selection methods</strong> — starting with the quick and clever <strong>Filter Methods</strong>, moving on to the determined and persistent <strong>Wrapper Methods</strong>, and wrapping up with the smart and efficient <strong>Embedded Methods</strong>.</p><p>Now that we’ve met all three, it’s time for a tiny <em>coffee-break blog</em> ☕ — where we’ll settle the ultimate question every data scientist secretly wonders:</p><p>👉 <strong>“Okay, but when should I use which method?”</strong></p><p>Because let’s be honest — sometimes we don’t need a full-blown wrapper marathon 🏃‍♀️, and sometimes filters just don’t cut it. So in the next blog, we’ll have a light, practical chat to help you decide <em>which method is your best buddy</em> for different projects.</p><p>Thanks for joining the journey so far! 🙌</p><p>See you soon in the next chapter of <strong>100 Days of Machine Learning with Gayatri</strong>! 💻📈✨</p><p>Until then, Keep learning, keep exploring and I’ll see you soon! 😊</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=53ee094a8647" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Feature Selection Part 2: Wrapper Methods]]></title>
            <link>https://medium.com/@gayatrinikam5103/feature-selection-part-2-wrapper-methods-6f664673ef1e?source=rss-f175867c9dc8------2</link>
            <guid isPermaLink="false">https://medium.com/p/6f664673ef1e</guid>
            <category><![CDATA[subset-selection]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[wrapper-methods]]></category>
            <category><![CDATA[machine-learning-ai]]></category>
            <category><![CDATA[machine-leaning]]></category>
            <dc:creator><![CDATA[Gayatri Nikam]]></dc:creator>
            <pubDate>Wed, 08 Oct 2025 11:14:57 GMT</pubDate>
            <atom:updated>2025-10-08T11:14:57.314Z</atom:updated>
            <content:encoded><![CDATA[<p>Welcome back to <strong>100 Days of Machine Learning with Gayatri</strong>!<br> In our previous blog, we explored <strong>Filter Methods</strong> for feature selection — quick, statistical techniques that don’t rely on machine learning models.</p><p>Today, we’ll dive into the next powerful approach: <strong>Wrapper Methods</strong>. These methods are more accurate because they use machine learning models directly in the selection process. But of course, with great power comes greater computation! ⚡</p><h3>🔍 What Are Wrapper Methods?</h3><p>Wrapper methods are feature selection techniques that <strong>use a predictive model to evaluate different combinations of features</strong>.</p><p>They are called <em>“wrapper”</em> methods because they <em>wrap</em> a machine learning model around the feature selection process. Instead of just checking statistics like correlation (as in filters), wrapper methods test how well subsets of features actually perform in a model.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*7hJiv_vvqHOGGnP-QfSrGQ.png" /></figure><p><strong>How it works (simple steps):</strong></p><ol><li><strong>Subset Generation</strong> — Create a subset of features (either adding, removing, or testing combinations).</li><li><strong>Subset Evaluation</strong> — Train a machine learning model on this subset and measure its performance (e.g., accuracy, RMSE, AUC).</li><li><strong>Stopping Criterion</strong> — Stop when there’s no further improvement or when a set limit (like number of features) is reached.</li></ol><p>👉 Think of it like selecting a cricket team 🏏. Instead of looking only at player statistics (filter methods), you try different combinations of players in real matches (wrapper methods) to see which team performs the best.</p><h3>⚡ Types of Wrapper Methods</h3><h3>1. Exhaustive Feature Selection (Best Subset Selection)</h3><p>This method tries <strong>all possible combinations</strong> of features and selects the one that performs the best.</p><ul><li>If you have n features, the number of combinations is:<em> 2^n</em></li></ul><p>For example: With just 10 features, you’d have 2^{10}=1024 possible subsets to test! 😱</p><p><strong>Advantages:</strong></p><ul><li>Guarantees the best possible subset (in theory).</li></ul><p><strong>Disadvantages:</strong></p><ul><li>Extremely <strong>computationally expensive</strong> for large datasets.</li><li>Risk of <strong>overfitting</strong>, because the best subset on training data might not generalize.</li></ul><p>👉 Example: Imagine you’re a chef with 10 spices. Exhaustive search means trying <strong>every possible spice combination</strong> until you find the tastiest recipe. Great for a small spice rack, but impossible for 100 spices!</p><h3>2. Sequential Forward Selection (SFS)</h3><p>Start with <strong>no features</strong>. Add one feature at a time — the one that improves model performance the most — until no further improvement is possible.</p><p>👉 Example: Building a music playlist 🎶. You start with one favorite song, then keep adding songs that make the playlist better, until adding more doesn’t help.</p><p><strong>Advantages:</strong></p><ul><li>Faster than exhaustive search.</li><li>Simple and intuitive.</li></ul><p><strong>Disadvantages:</strong></p><ul><li>Might miss the best subset because once a feature is added, it can’t be removed.</li></ul><h3>3. Sequential Backward Selection (SBS) / Elimination</h3><p>Start with <strong>all features</strong>. Remove one feature at a time — the one whose removal hurts performance the least — until you reach the desired number of features.</p><p>👉 Example: Cooking 🍲. You start with all the spices in the pot and keep removing them one by one until you find the perfect balance.</p><p><strong>Advantages:</strong></p><ul><li>Useful when you start with many irrelevant features.</li><li>Often faster than forward selection for large datasets.</li></ul><p><strong>Disadvantages:</strong></p><ul><li>Still computationally expensive if dataset is huge.</li></ul><h3>4. Recursive Feature Elimination (RFE)</h3><p>RFE is a <strong>hybrid</strong> method — often considered both wrapper and embedded.</p><p>How it works:</p><ol><li>Train a model (like linear regression or SVM).</li><li>Rank features by importance (coefficients, weights).</li><li>Eliminate the least important features.</li><li>Repeat until the desired number of features is left.</li></ol><p>👉 Example: Hiring process 👔. You interview all candidates, rank them, remove the weakest, and repeat until only the best remain.</p><p><strong>Advantages:</strong></p><ul><li>Works well with models that naturally provide feature importance.</li><li>More efficient than exhaustive search.</li></ul><h3>✅ Advantages of Wrapper Methods</h3><ol><li><strong>High Accuracy</strong> — Because they test directly on the chosen model, results are usually better.</li><li><strong>Consider Feature Interactions</strong> — Unlike filter methods, wrappers test features in groups, so they can find combinations that work well together.</li></ol><h3>❌ Disadvantages of Wrapper Methods</h3><ol><li><strong>Computationally Expensive</strong> — Training models on many subsets takes a lot of time.</li><li><strong>Risk of Overfitting</strong> — Selected features might fit training data too closely.</li><li><strong>Model-Specific</strong> — The chosen features are optimized for one model; switching models might not give the same results.</li></ol><h3>Conclusion</h3><p>Wrapper methods are like <strong>custom tailoring</strong> 👗 — they give you the perfect fit for your chosen model, but they take more time and effort compared to off-the-rack filter methods.</p><p>They are best used when:</p><ul><li>You have <strong>smaller datasets</strong>.</li><li>Accuracy is more important than computation time.</li></ul><p>In the next blog, we’ll move on to the <strong>Embedded Methods</strong> of feature selection — a smarter middle ground between filters and wrappers, where the model itself decides which features matter.</p><p>Thanks for joining the journey so far! 🙌</p><p>See you soon in the next chapter of <strong>100 Days of Machine Learning with Gayatri</strong>! 💻📈✨</p><p>Until then, Keep learning, keep exploring and I’ll see you soon! 😊</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6f664673ef1e" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Feature Selection Part 1: Choose Wisely, Predict Better]]></title>
            <link>https://medium.com/@gayatrinikam5103/feature-selection-part-1-choose-wisely-predict-better-f048c7573de0?source=rss-f175867c9dc8------2</link>
            <guid isPermaLink="false">https://medium.com/p/f048c7573de0</guid>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[feature-selection]]></category>
            <category><![CDATA[feature-engineering]]></category>
            <category><![CDATA[data-features]]></category>
            <dc:creator><![CDATA[Gayatri Nikam]]></dc:creator>
            <pubDate>Mon, 07 Apr 2025 10:46:00 GMT</pubDate>
            <atom:updated>2025-04-07T10:46:00.910Z</atom:updated>
            <content:encoded><![CDATA[<p>Hello Everyone and welcome back to <strong>100 Days of Machine Learning with Gayatri</strong>! 🌟</p><p>Imagine you’re preparing a dish. You have a kitchen full of ingredients — but using <em>all</em> of them doesn’t guarantee a tasty outcome. In fact, using too many might ruin the dish. The same is true in machine learning: more features don’t always mean better performance.</p><p>That’s where <strong>Feature Selection</strong> comes in.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/474/0*L2HaxaV-ktqVug1a.png" /></figure><h3>What is Feature Selection?</h3><p><strong>Feature Selection</strong> is the process of choosing the most relevant input variables (features) from your dataset that contribute the most to the output variable.</p><p>It’s like choosing the best players for your team — you want those who’ll actually help you win, not just crowd the field.</p><h3>Why Do Feature Selection?</h3><p><strong>Improves Model Accuracy</strong><br> Unnecessary features can introduce noise. Keeping only relevant features sharpens the model.</p><p><strong>Reduces Overfitting</strong><br> Fewer features mean fewer chances of the model learning patterns that don’t generalize.</p><p><strong>Speeds Up Training</strong><br> Less data = faster computation.</p><p><strong>Simplifies Models</strong><br> Easier to interpret and explain.</p><h3><strong>Types of Feature Selection</strong></h3><p>Feature selection methods fall into three main categories:</p><ol><li><strong>Filter Methods: </strong>Select features based on statistical tests (independent of any model).</li></ol><p><strong>2. Wrapper Methods: </strong>Use a predictive model to test combinations of features.</p><p><strong>3. Embedded Methods: </strong>Feature selection happens as part of model training (e.g., Lasso).</p><p>In this blog, we’ll focus on <strong>filter methods</strong>.</p><h3>Filter-Based Feature Selection</h3><p>These methods rely on the characteristics of the data — like correlation, variance, or statistical significance — without involving a machine learning model.</p><h3>i. Duplicate Features</h3><p>Sometimes, your dataset contains columns that are exactly the same (or nearly the same).</p><p><strong>Example:</strong> If both Height_cm and Height_in_inches are present, they carry the same info in different units.</p><p>👉 <strong>What to do?</strong> Drop one of them.</p><h3>ii. Variance Threshold</h3><p>Features with little to no variance (i.e., nearly the same value in all rows) are unlikely to be useful.</p><p><strong>Example:</strong> A column like Country with 98% of values as &quot;India&quot; doesn’t add much value in classification.</p><p><strong>Mathematical Idea:</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/346/1*dM4-me4xbGPJIj9GICU8Ow.png" /></figure><p>If variance &lt; threshold (say 0.01), drop it!</p><h3>iii. Correlation</h3><p>Highly correlated features (multicollinearity!) give redundant info.</p><p><strong>Example:</strong> If Temperature_C and Temperature_F are both present, their correlation will be nearly 1.</p><p>👉 If |correlation| &gt; 0.8, consider keeping just one.</p><h3>iv. ANOVA (Analysis of Variance)</h3><p>Used to check if the mean of the target variable is significantly different across groups of a categorical feature.</p><p><strong>Example:</strong> Does the average exam score differ by study method? (Visual vs. Reading)</p><p><strong>Mathematical Idea:</strong> The F-score is computed as</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/328/1*AlETfjteX0flda7c2PS5MQ.png" /></figure><p>A high F-value suggests the feature is important.</p><h3>v. Chi-Square Test</h3><p>Works with <strong>categorical features</strong> to check if there’s a significant relationship between the feature and the target.</p><p><strong>Example:</strong> Is Genderrelated to Purchase Decision?</p><p><strong>Mathematical Formula:</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/232/1*DGYZZ7cqfIdO5QoXz_apRQ.png" /></figure><p>Where <em>O</em> is observed frequency and <em>E</em> is expected frequency.</p><p>👉 Higher χ² → more important the feature.</p><h3>👍 Advantages of Filter Methods</h3><ul><li>Simple and fast</li><li>No model dependency</li><li>Good for pre-processing before applying complex models</li></ul><h3>👎 Disadvantages</h3><ul><li>May ignore feature interactions</li><li>Can sometimes eliminate useful features if they’re only important in combination with others</li></ul><h3>Conclusion</h3><p>Feature selection is like tuning your radio to the right frequency — it helps cut out the noise so your model hears only what matters.</p><p>In this blog, we explored <strong>filter-based methods</strong> like removing duplicates, using variance thresholds, checking correlations, and applying statistical tests like ANOVA and Chi-Square.</p><p>Up next, we’ll dive into <strong>Wrapper Methods</strong>, where we let machine learning models guide us in choosing the best features step-by-step.</p><p>Thanks for joining the journey so far! 🙌</p><p>See you soon in the next chapter of <strong>100 Days of Machine Learning with Gayatri</strong>! 💻📈✨</p><p>Until then, Keep learning, keep exploring and I’ll see you soon! 😊</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f048c7573de0" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Multicollinearity: When Your Data Talks Behind Your Back]]></title>
            <link>https://medium.com/@gayatrinikam5103/multicollinearity-when-your-data-talks-behind-your-back-97d80e07d4c1?source=rss-f175867c9dc8------2</link>
            <guid isPermaLink="false">https://medium.com/p/97d80e07d4c1</guid>
            <category><![CDATA[multicollinearity]]></category>
            <category><![CDATA[linear-regression]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Gayatri Nikam]]></dc:creator>
            <pubDate>Fri, 04 Apr 2025 09:02:07 GMT</pubDate>
            <atom:updated>2025-04-04T09:02:07.014Z</atom:updated>
            <content:encoded><![CDATA[<h3>Introduction:</h3><p>Imagine you’re working on a group project with five friends. Everyone has different strengths — someone’s good at research, another at design, one at writing, and so on. Ideally, each person brings something unique. But what if two of your friends have exactly the same skills and opinions? They keep repeating each other, and it becomes hard to tell who’s really contributing.</p><p>Let us discuss another example:<br>Imagine you’re managing a marketing team. You’ve got three employees:</p><ul><li><strong>Sam</strong>, who works on <strong>social media ads</strong>,</li><li><strong>Priya</strong>, who handles <strong>email campaigns</strong>, and</li><li><strong>Aarav</strong>, who runs <strong>content marketing</strong>.</li></ul><p>Now, you want to predict your monthly sales using how much each person spent on their campaign. You collect data and fit a linear regression model.</p><p>But surprise! The results are… weird. Sam and Priya seem to be <em>super</em> important. Then sometimes… not at all. Your model can’t make up its mind. Why?</p><p>You’ve just stepped into the world of <strong>multicollinearity</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/484/0*QfnsxVUAdLlKDJoS" /></figure><h3>What is multicollinearity?</h3><p>Multicollinearity means that two or more <strong>independent variables</strong> in your regression model are <strong>highly correlated</strong> — they move together. This makes it hard for the model to figure out the individual effect of each variable.</p><p>📌 In our story, maybe Sam and Priya run very similar campaigns targeting the same audience. So their budget spend looks the same month after month. The model gets confused—who really caused the sales bump?</p><h3>When is multicollinearity a problem?</h3><p>Multicollinearity isn’t always bad. But it can cause issues when:</p><ul><li>You want to understand which variables are actually important.</li><li>You rely on p-values and confidence intervals for decision-making.</li><li>You need stable, interpretable coefficients.</li></ul><p>Even though predictions <em>might</em> still work okay, the model’s <strong>interpretability</strong> goes out the window.</p><h3>Multicollinearity — Mathematically</h3><p>Let’s say your regression model looks like this:</p><p>y=β0 +β1x1+β2x2+ϵ</p><p>If x1​ and x2​ are highly correlated, then your matrix X′X (used in OLS) becomes nearly singular — it’s hard to invert. This makes the estimates of β1​ and β2​ <strong>unstable and unreliable</strong>.</p><h3>Perfect vs. Imperfect Multicollinearity</h3><ul><li>✅ <strong>Perfect Multicollinearity</strong>: One predictor is an exact linear combination of another.</li></ul><p>Example: You accidentally include both “total spend” and “Sam + Priya + Aarav spend” in your model.</p><ul><li>⚠️ <strong>Imperfect Multicollinearity</strong>: Variables are not perfectly related but highly correlated.</li></ul><p>Example: Sam and Priya’s campaigns just happen to rise and fall together.</p><h3>Types of Multicollinearity</h3><p>Multicollinearity can sneak into your data in two main ways:</p><h4>1. Structural Multicollinearity</h4><p>This happens because of <strong>how you’ve designed your model</strong> or collected your data.</p><p><strong>Example:</strong> If you include both age and age² as predictors, they’re mathematically related. The model will struggle to separate their individual effects.</p><p>It’s like asking someone to describe both the size and the square of the size — they’re telling you the same story twice in a different way.</p><h4>2. Incidental Multicollinearity</h4><p>This shows up naturally due to <strong>real-world relationships</strong> between variables.</p><p><strong>Example:</strong> If both social media spend and email marketing spend go up during big product launches, they’ll be highly correlated — even though you didn’t plan it that way.</p><p>It’s like two friends always attending the same events — you can’t tell who brought the vibe!</p><h3>How to Detect Multicollinearity</h3><h4>1. Correlation Matrix</h4><p>Check the pairwise correlation between your independent variables.</p><p>📌 If the correlation is <strong>above 0.8 or below -0.8</strong>, that’s a warning sign.<br> The variables might be too close for comfort.</p><h4>2. VIF (Variance Inflation Factor)</h4><p>VIF shows how much the variance of a regression coefficient is inflated due to multicollinearity.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/178/1*ugw9DlRoZnAuPGuiq-sjZQ.png" /></figure><p>Where R²_i​ is the R-squared from regressing variable iii on all other predictors.</p><p>📌 If <strong>VIF &gt; 5</strong> (some say 10), the variable may be contributing to multicollinearity.</p><h4>3. Condition Number</h4><p>This comes from linear algebra and checks how “twisted” your data matrix is.</p><p>📌 If the <strong>condition number &gt; 30</strong>, your model might have multicollinearity issues.</p><p>Think of it like trying to balance a wobbly table — too much leaning, and things fall apart.</p><h3>How to Fix Multicollinearity</h3><h4>✅ Drop one of the correlated variables</h4><p>If Sam and Priya are doing almost the same work, keep just one in the model.</p><h4>✅ Combine related variables</h4><p>Instead of tracking three campaign budgets, create one feature like “total marketing spend.”</p><h4>✅ Use PCA (Principal Component Analysis)</h4><p>PCA simplifies your features into a smaller set of uncorrelated components.</p><h4>✅ Apply Regularization (e.g., Ridge Regression)</h4><p>Ridge regression reduces the effect of multicollinearity by shrinking the coefficients — giving your model more stability.</p><h3>🧾 Final Thoughts</h3><p>Multicollinearity is like office gossip — some variables just talk too much behind each other’s backs. While it may not ruin your predictions, it can mess up your interpretations and decision-making.</p><p>That’s why it’s important to detect it, understand where it’s coming from, and fix it before trusting your model results.</p><h3>Conclusion</h3><p>In this blog, we explored the world of <strong>multicollinearity—</strong>what it is, when it becomes problematic, and how it silently affects our regression models. We looked at</p><ul><li>What causes <strong>perfect</strong> and <strong>non-perfect multicollinearity</strong></li><li>The two types: <strong>Structural</strong> and <strong>Incidental</strong></li><li>How to <strong>detect</strong> it using Correlation, VIF, and Condition Number</li><li>Practical ways to <strong>fix</strong> it, from dropping variables to using PCA and Ridge Regression</li></ul><p>The key takeaway? Multicollinearity doesn’t always break your model — but it <strong>does blur the understanding of individual predictors</strong>, which can be dangerous if you’re using the model to make decisions.</p><p>Thanks for joining the journey so far! 🙌<br><br> See you soon in the next chapter of <strong>100 Days of Machine Learning with Gayatri</strong>! 💻📈✨</p><p>Until then, Keep learning, keep exploring and I’ll see you soon! 😊</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=97d80e07d4c1" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[“Key Assumptions in Linear Regression”]]></title>
            <link>https://medium.com/@gayatrinikam5103/key-assumptions-in-linear-regression-0734088c90e9?source=rss-f175867c9dc8------2</link>
            <guid isPermaLink="false">https://medium.com/p/0734088c90e9</guid>
            <category><![CDATA[linear-regression]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[assumptions]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <dc:creator><![CDATA[Gayatri Nikam]]></dc:creator>
            <pubDate>Tue, 03 Dec 2024 12:06:21 GMT</pubDate>
            <atom:updated>2024-12-03T12:06:21.321Z</atom:updated>
            <content:encoded><![CDATA[<p>Hi everyone, I’m Gayatri, and welcome back to <strong>“100 Days of Machine Learning with Gayatri”!</strong></p><p>Today, we’re going to talk about something very important in linear regression — its <strong>assumptions</strong>. Linear regression is a simple yet powerful tool, but for it to work well and give accurate results, certain conditions need to be met.</p><p>Whether you’re just starting with your first model or already experienced in working with data, knowing these assumptions will help you make sure your model is reliable. In this blog, we’ll explore why these assumptions are important, what they are, and how to deal with problems if they don’t hold true.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*pZdL8sZnQF4Lmg3X0aOs4A.png" /></figure><p>Let’s get started and understand the basics of linear regression!</p><p>Linear regression is a simple and popular method for understanding relationships between variables. However, for it to work well, certain conditions about the data and model need to be met. These are called <strong>assumptions</strong>, and they are the base on which linear regression is built. If these assumptions are not followed, the results can be misleading and unreliable.</p><p>By knowing these assumptions, you can make sure your model gives accurate and meaningful results. It also helps you find and fix problems in your data or decide if another method might work better.</p><p>In this blog, we’ll look at the five main assumptions of linear regression:</p><ol><li><strong>Linearity</strong></li><li><strong>Normality of Residuals</strong></li><li><strong>Homoscedasticity</strong></li><li><strong>No Autocorrelation</strong></li><li><strong>No Multicollinearity</strong></li></ol><p>We’ll explain each one clearly with simple examples and show how to deal with issues if these assumptions don’t hold. Let’s dive into the basics that keep linear regression strong!</p><h3>1. Linearity</h3><p><strong>Assumption:</strong>The linearity assumption in linear regression means that there is a straight-line relationship between the independent variables (predictors) and the dependent variable (response). In other words, the model assumes that a change in the independent variable causes a proportional change in the dependent variable.</p><p>Mathematically, this relationship can be written as:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/436/1*sAdIDGBjlmMLRmYZ0ItG0g.png" /></figure><p>Where:</p><ul><li>Y is the dependent variable.</li><li>X1, X2,…, and Xk are the independent variables.</li><li>β0, β1, …, βk are the model parameters (coefficients).</li><li>ϵ is the error term.</li></ul><p>If this assumption holds, changes in the independent variables cause constant, proportional changes in the dependent variable.</p><h4>What Happens When This Assumption is Violated?</h4><ol><li><strong>Bias in Parameter Estimates</strong>: If the true relationship is not linear, the estimated values of β0, β1,…, and βk may be incorrect. This bias leads to wrong conclusions about how the independent variables affect the dependent variable.</li><li><strong>Reduced Predictive Accuracy</strong>: If the model wrongly assumes a linear relationship when the data is non-linear, it can result in poor predictions. The model might miss important patterns in the data, making it unable to capture the actual relationship.</li><li><strong>Invalid Hypothesis Tests</strong>: When linearity is violated, hypothesis tests (like testing whether a variable is significant) may give misleading results. This means that the significance of the predictors might be incorrectly assessed.</li></ol><h4>How to Check This Assumption</h4><ol><li><strong>Scatter Plots</strong>: A simple way to check for linearity is to create scatter plots. Plot each independent variable against the dependent variable. If the points form a straight-line pattern, it suggests linearity. Curved patterns indicate a violation of the assumption.</li><li><strong>Residual Plots</strong>: Plot the residuals (the difference between observed and predicted values) against predicted values or independent variables. If the linearity assumption holds, residuals should be scattered randomly around zero. If there’s a pattern, such as a curve, this suggests non-linearity.</li><li><strong>Polynomial Terms</strong>: You can add polynomial terms (like X², X³) to the model. If the model fit improves significantly, it might mean that the relationship is not linear.</li></ol><h4>What to Do When the Assumption Fails?</h4><ol><li><strong>Transformations</strong>: Sometimes, transforming the variables can help. For example:</li></ol><ul><li><strong>Logarithmic Transformation</strong>: Take the log of the dependent or independent variable.</li><li><strong>Square Root Transformation</strong>: Apply the square root to the variable.</li><li><strong>Inverse Transformation</strong>: Use the inverse (1/X) of the variable. These transformations can help make the relationship more linear.</li></ul><p><strong>2. Polynomial Regression</strong>: If the relationship between variables is non-linear, you can add polynomial terms (like X², X³) to the model. This will allow the model to capture non-linear relationships.</p><p><strong>3. Piecewise Regression</strong>: You can divide the data into sections and fit separate linear models to each section. This helps in cases where the relationship is linear in parts but not across the whole range.</p><p><strong>4. Non-Parametric Methods</strong>: If linear regression does not work well, you could try methods like <strong>Generalized Additive Models (GAMs)</strong>, <strong>splines</strong>, or <strong>kernel regression</strong>. These methods do not assume linearity and can be more flexible in modeling complex relationships.</p><h3>2. Normality of Residuals</h3><p><strong>Assumption:</strong> In linear regression, one key assumption is that the error terms, or residuals, are normally distributed. This means that the difference between the observed values and the predicted values should follow a bell-shaped curve with a mean of zero and constant variance.</p><p>Mathematically, the residuals ϵ should follow a normal distribution:</p><p>ϵ ∼ N(0,σ²)</p><p>Where:</p><ul><li>ϵ represents the residuals,</li><li>N(0, σ²) indicates a normal distribution with a mean of zero and constant variance σ².</li></ul><p>This assumption ensures that the statistical tests used in linear regression, like t-tests and F-tests, work correctly and provide reliable results.</p><h4>What Happens When This Assumption is Violated?</h4><ol><li><strong>Inaccurate Hypothesis Tests</strong>: The t-tests and F-tests used to determine the significance of the regression coefficients depend on the normality of residuals. If the residuals are not normally distributed, these tests may give misleading results, leading to incorrect conclusions about the significance of the predictors.</li><li><strong>Invalid Confidence Intervals</strong>: The confidence intervals for regression coefficients rely on the assumption that residuals are normally distributed. If this assumption is violated, the intervals might be too wide or too narrow, affecting how we interpret the precision of the estimates.</li><li><strong>Model Performance</strong>: A violation of the normality assumption may signal that the model is not well-suited to the data, leading to poor predictive accuracy. This suggests that the chosen linear regression model might not be the best fit.</li></ol><h4>How to Check This Assumption</h4><ol><li><strong>Histogram of Residuals</strong>: One way to check for normality is to plot a histogram of the residuals. If the histogram forms a bell-shaped curve, this suggests the residuals are normally distributed.</li><li><strong>Q-Q Plot</strong>: A Q-Q (Quantile-Quantile) plot is another useful tool. It compares the quantiles of the residuals with the quantiles of a standard normal distribution. If the points lie along a straight line, it indicates the residuals are normally distributed. Any deviations from the straight line suggest non-normality.</li><li><strong>Statistical Tests</strong>: You can also use statistical tests like the <strong>Omnibus Test</strong>, <strong>Jarque-Bera Test</strong>, or <strong>Shapiro-Wilk Test</strong> to check for normality. These tests specifically evaluate whether the residuals follow a normal distribution.</li></ol><h4>What to Do When the Assumption Fails?</h4><ol><li><strong>Model Selection Techniques</strong>: If residuals are not normally distributed, consider using model selection techniques like cross-validation, <strong>Akaike Information Criterion (AIC)</strong>, or <strong>Bayesian Information Criterion (BIC)</strong> to choose the best model. These techniques can help find a model that better handles non-normal residuals.</li></ol><p><strong>2. Robust Regression</strong>: You can also use robust regression techniques that are less sensitive to the distribution of residuals. Some options include:</p><ul><li><strong>M-estimation</strong>,</li><li><strong>Least Median of Squares (LMS)</strong>,</li><li><strong>Least Trimmed Squares (LTS)</strong>. These techniques can provide more reliable results when the normality assumption is violated.</li></ul><p><strong>3. Non-Parametric or Semi-Parametric Methods</strong>: Consider using non-parametric or semi-parametric methods, like <strong>Generalized Additive Models (GAMs)</strong> or <strong>kernel regression</strong>, which do not assume normality of residuals and can be more flexible in handling complex data patterns.</p><p><strong>4. Bootstrapping</strong>: Bootstrapping is a statistical method that allows for inference without relying on the normality of residuals. It can be used to generate more accurate confidence intervals and perform hypothesis tests, especially when the data does not follow a normal distribution.</p><p><strong>5. Central Limit Theorem</strong>: Keep in mind that when the sample size is large, the normality assumption becomes less critical. Thanks to the <strong>Central Limit Theorem</strong>, the sampling distribution of the regression coefficients tends to be normal even if the residuals are not, especially when you have enough data points.</p><p>By understanding and checking the normality of residuals, you can ensure that your regression model is reliable and that the results of your hypothesis tests and confidence intervals are accurate.</p><h4>Omnibus Test</h4><p>The <strong>Omnibus Test</strong> is a statistical test used to determine if the residuals from a linear regression model follow a normal distribution. It specifically evaluates two key aspects of the residuals: <strong>skewness</strong> (the asymmetry of the distribution) and <strong>kurtosis</strong> (the “tailedness” of the distribution). By combining these two metrics, the Omnibus Test provides a comprehensive check for normality in the residuals.</p><h4>Steps to Perform the Omnibus Test</h4><ol><li><strong>State the Hypotheses</strong>:</li></ol><ul><li><strong>Null Hypothesis (H0)</strong>: The residuals are normally distributed.</li><li><strong>Alternative Hypothesis (H1)</strong>: The residuals are not normally distributed.</li></ul><p><strong>2. Fit the Linear Regression Model</strong>:<br>First, fit the linear regression model to your data to obtain the predicted values for the dependent variable.</p><p><strong>3. Calculate the Residuals</strong>:<br>Calculate the residuals by subtracting the predicted values from the observed values of the dependent variable.</p><p><strong>Residuals = Observed Value − Predicted Value</strong></p><p><strong>4. Calculate the Skewness</strong>:<br>Skewness measures the asymmetry of the distribution of residuals. For a perfectly normal distribution, skewness should be close to zero. Positive skewness indicates a rightward tilt, while negative skewness suggests a leftward tilt.</p><p><strong>5. Calculate the Kurtosis</strong>:<br>Kurtosis measures how “heavy” the tails of the distribution are. For a normal distribution, kurtosis is expected to be close to zero (in excess kurtosis terms). Positive kurtosis indicates heavy tails, while negative kurtosis suggests lighter tails.</p><p><strong>6. Calculate the Omnibus Test Statistic</strong>:<br>The Omnibus Test statistic (denoted K²) combines the skewness and kurtosis values. It is computed using the formula:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/682/1*CMWJQvNKvzE-GAHuXLkfpw.png" /></figure><p><strong>7. Determine the p-value</strong>:<br>The Omnibus Test statistic follows a <strong>chi-square distribution</strong> with 2 degrees of freedom. The p-value is determined by comparing the test statistic to this distribution.</p><p><strong>8. Interpret the p-value</strong>:</p><ul><li>If the p-value is <strong>greater</strong> than the significance level (e.g., 0.05), you <strong>fail to reject</strong> the null hypothesis. This suggests that the residuals are normally distributed.</li><li>If the p-value is <strong>less</strong> than the significance level, you <strong>reject</strong> the null hypothesis. This indicates that the residuals may not follow a normal distribution.</li></ul><h3>3. Homoscedasticity</h3><p><strong>Homoscedasticity</strong> is the assumption that the spread of the residuals (the differences between the observed and predicted values) should be consistent across all levels of the independent variables. In other words, the variance of the residuals should remain constant, no matter what value the independent variables take.</p><p>If this assumption is violated, it leads to <strong>heteroscedasticity</strong>, where the variance of the residuals changes with different values of the independent variables. This can affect the efficiency of the model’s estimates and make statistical tests less reliable.</p><h4>What Happens When This Assumption is Violated?</h4><ol><li><strong>Inefficient Estimates</strong>: Even though the parameter estimates (regression coefficients) remain unbiased, they are no longer the “best” estimates. The standard errors will be larger than they should be, leading to less precise estimates and reducing the statistical power of hypothesis tests.</li><li><strong>Inaccurate Hypothesis Tests</strong>: Tests like t-tests and F-tests rely on the assumption of homoscedasticity. When the assumption is violated, these tests may produce misleading results. This means we might wrongly conclude that an independent variable is significant or not.</li><li><strong>Invalid Confidence Intervals</strong>: Confidence intervals for the regression coefficients also depend on homoscedasticity. If the assumption is violated, the confidence intervals might be inaccurate, leading to unreliable interpretations of the effect sizes and the precision of estimates.</li></ol><h4>How to Check This Assumption</h4><ol><li><strong>Residual Plot</strong>: A simple way to check for homoscedasticity is by plotting the residuals against the predicted values or each independent variable. If the plot shows random scattering around zero with no clear pattern, homoscedasticity is likely present. If the plot shows a systematic pattern, like a funnel shape or curve, this suggests <strong>heteroscedasticity</strong>.</li><li><strong>Breusch-Pagan Test</strong>: This is a formal statistical test for detecting heteroscedasticity. The <strong>null hypothesis</strong> is that the error variance is constant (homoscedastic). If the p-value from the test is smaller than your chosen significance level (e.g., 0.05), you can reject the null hypothesis and conclude that heteroscedasticity is present.</li></ol><h4>What to Do When the Assumption Fails?</h4><ol><li><strong>Transformations</strong>: One common solution is to apply transformations to either the dependent or independent variables to stabilize the variance of the residuals. Some common transformations include logarithmic, square root, and inverse transformations.</li><li><strong>Weighted Least Squares (WLS)</strong>: WLS is an approach that gives different weights to each observation based on the magnitude of its residual. It places more importance on observations with smaller residuals and less on those with larger residuals, which helps in correcting heteroscedasticity.</li><li><strong>Robust Standard Errors</strong>: When heteroscedasticity is detected, you can use <strong>robust standard errors</strong> (also known as heteroscedasticity-consistent standard errors). These standard errors are adjusted for heteroscedasticity and provide more reliable hypothesis tests and confidence intervals.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/331/1*aViejAps3-B4wfi7tdCqLg.png" /></figure><h3>Standard Error: The Assumption</h3><p>The <strong>standard error (SE)</strong> measures how much the estimated values (such as the slope and intercept) of a regression model may vary. In simpler terms, it tells us how precise or uncertain our predictions are. A smaller standard error means the estimates are more reliable, while a larger standard error indicates more uncertainty in the results.</p><p>In linear regression, the standard error is used to assess the reliability of the model’s coefficients. If the standard errors are large, the estimates of the relationship between the independent and dependent variables might not be very accurate, affecting predictions and statistical tests.</p><h4>What Happens When This Assumption is Violated?</h4><ol><li><strong>Unreliable Results</strong>: If the standard errors are too large, it means the model’s estimates are uncertain, leading to inaccurate conclusions about the significance of the predictors.</li><li><strong>Poor Predictions</strong>: Larger standard errors reduce the reliability of the model’s predictions, making them less accurate.</li><li><strong>Wrong Confidence Intervals</strong>: Confidence intervals around the estimated coefficients may be too wide, affecting the precision of our conclusions.</li></ol><h4>Mathematical Formula:</h4><p>For simple linear regression, the standard error of the slope (b1b₁b1​) is calculated as:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/357/1*vYjdlzM_hBl9dppTUULqmQ.png" /></figure><p>Where:</p><ul><li>y_i​ = observed value,</li><li>y^i = predicted value,</li><li>x_i = independent variable,</li><li>n = number of data points.</li></ul><h4>Example:</h4><p>If your model predicts sales based on advertising spend, and the standard error for the slope is 1.5, this means the true slope could vary by 1.5 units. A confidence interval can be calculated to show the range within which the true slope lies with a certain level of confidence.</p><h4>How to Ensure Accurate Standard Errors:</h4><ol><li><strong>Check Homoscedasticity</strong>: If the residuals don’t have constant variance, it can inflate standard errors.</li><li><strong>Use Larger Samples</strong>: Larger sample sizes tend to reduce the standard error.</li><li><strong>Refine the Model</strong>: Make sure the model includes all important variables to avoid large standard errors due to omitted variables.</li></ol><p>In short, the standard error helps us gauge how trustworthy the regression estimates are, and addressing its issues is key to making reliable predictions.</p><h4>Breusch-Pagan Test</h4><p>The <strong>Breusch-Pagan test</strong> is a statistical test used to check for <strong>heteroscedasticity</strong>, which occurs when the variance of the error terms (residuals) in a regression model is not constant. If this assumption is violated, it can affect the reliability of the model’s estimates. Here’s a simple explanation of how the test works:</p><p><strong>Steps to Perform the Breusch-Pagan Test:</strong></p><ol><li><strong>Estimate the Linear Regression Model</strong>: First, fit a linear regression model using ordinary least squares (OLS) to obtain the residuals (errors).</li><li><strong>Square the Residuals</strong>: Square each of the residuals you obtained from the regression model.</li><li><strong>Regress Squared Residuals</strong>: Perform a second regression where the squared residuals are the dependent variable, and the independent variables from the original model are the predictors.</li><li><strong>Calculate the Test Statistic (LM)</strong>: The test statistic is the <strong>Lagrange Multiplier (LM)</strong>, calculated as:</li></ol><p>LM = n × R²</p><p>Where:</p><ul><li>n is the number of data points (observations),</li><li>R² is the R-squared value from the regression of squared residuals.</li></ul><ol><li><strong>Determine the p-value</strong>: The LM statistic follows a chi-squared distribution with degrees of freedom equal to the number of independent variables (excluding the constant). Use the chi-squared distribution to find the p-value.</li><li><strong>Interpret the Results</strong>:</li></ol><ul><li>If the <strong>p-value</strong> is less than the significance level (e.g., 0.05), <strong>reject the null hypothesis</strong>. This means there is evidence of heteroscedasticity, and the variance of residuals is not constant.</li><li>If the <strong>p-value</strong> is greater than the significance level, <strong>do not reject the null hypothesis</strong>, implying homoscedasticity (constant variance of residuals).</li></ul><h4>Key Points to Remember:</h4><ul><li><strong>Null Hypothesis</strong>: The residuals have constant variance (homoscedasticity).</li><li><strong>Alternative Hypothesis</strong>: The residuals have non-constant variance (heteroscedasticity).</li><li>If the data does not meet the assumption of a linear relationship between independent variables and error variance, other tests may be needed.</li></ul><p>By using the Breusch-Pagan test, you can detect heteroscedasticity in your data, ensuring that your regression results are reliable.</p><p>That’s all for today’s blog! We’ve covered the key assumptions of linear regression, including <strong>linearity</strong>, <strong>normality of residuals</strong>, and <strong>homoscedasticity</strong>. Understanding these assumptions is crucial for building a reliable regression model and ensuring that your results are meaningful.</p><p>In our next blog, we’ll continue exploring the remaining important assumptions: <strong>No Autocorrelation</strong> and <strong>No Multicollinearity</strong>. Stay tuned for more insights into these topics and how to address them in your models.</p><h3>Conclusion</h3><p>In conclusion, understanding the assumptions behind linear regression is essential for building robust and reliable models. By ensuring that assumptions like <strong>linearity</strong>, <strong>normality of residuals</strong>, and <strong>homoscedasticity</strong> hold true, we can improve the accuracy of our regression analyses and avoid common pitfalls. Addressing violations of these assumptions helps ensure that our results are trustworthy and actionable.</p><p>Thank you for joining me on this deep dive into the assumptions of linear regression. Stay tuned as we continue our journey through machine learning, exploring more key concepts in the coming blogs.</p><p>Until then, Keep learning, keep exploring and I’ll see you soon! 😊</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=0734088c90e9" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[“Mastering Polynomial Regression for Non-Linear Data”]]></title>
            <link>https://medium.com/@gayatrinikam5103/mastering-polynomial-regression-for-non-linear-data-2fef8b42a397?source=rss-f175867c9dc8------2</link>
            <guid isPermaLink="false">https://medium.com/p/2fef8b42a397</guid>
            <category><![CDATA[regression-modeling]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[regression-analysis]]></category>
            <category><![CDATA[polynomial-regression]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <dc:creator><![CDATA[Gayatri Nikam]]></dc:creator>
            <pubDate>Fri, 22 Nov 2024 09:51:22 GMT</pubDate>
            <atom:updated>2024-11-22T09:51:22.366Z</atom:updated>
            <content:encoded><![CDATA[<p>Hello Everyone, I am Gayatri and Welcome back to the “100 Days of Machine Learning with Gayatri” series!</p><p>Imagine you’re trying to predict how the price of a house varies with its size. You plot the data, apply a linear regression line, and... it just doesn’t fit quite right. The relationship between size and price isn’t linear—it has ups and downs, peaks and plateaus. That’s where <strong>Polynomial Regression</strong> steps in, bending straight lines into curves to capture complex patterns that simple linear models miss.</p><p>While <strong>Linear Regression</strong> is excellent for modeling straightforward relationships, life isn’t always that simple. Many real-world problems involve curves rather than straight lines:</p><ul><li>The growth of a plant over time might accelerate at first, slow down, and eventually plateau.</li><li>The effect of temperature on crop yield might show an initial rise, reach an optimal point, and then decline.</li><li>Even something like predicting sales during a promotional campaign can involve spikes and dips.</li></ul><p>Polynomial regression helps us capture these nonlinear relationships by adding powers of the predictor variable (like x², x³ and so on). This transforms a straight line into a curve, allowing us to fit the data more closely without jumping to overly complex techniques like decision trees or neural networks.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/543/1*-7pzSuU_L_kDVOQvPpbzRw.png" /></figure><h3>Why Not Just Stick to Other Techniques?</h3><ul><li><strong>Linear Regression:</strong> Too simplistic for curvy patterns.</li><li><strong>Exponential or Logarithmic Models:</strong> Limited to specific types of relationships.</li><li><strong>Neural Networks:</strong> Powerful but often overkill for small datasets or simple patterns.</li></ul><p>Polynomial regression strikes a balance — it’s simple yet flexible. By adding just the right degree of complexity, it bridges the gap between underfitting and overfitting. And the best part? It’s easy to understand and implement!</p><p>In this blog, we’ll explore how polynomial regression works, its mathematical foundation, and how to implement it step-by-step with examples that will make it crystal clear. Let’s dive in!</p><h3><strong>Mathematical Foundations of Polynomial Regression</strong></h3><p>At its core, <strong>Polynomial Regression</strong> is an extension of linear regression where the relationship between the independent variable (x) and the dependent variable (y) is modeled as an n-degree polynomial.</p><p>The general equation for polynomial regression is:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/456/1*FDz02nbbx7IKQOKBTPhMSA.png" /></figure><p>Where:</p><ul><li>y is the dependent variable (what we want to predict).</li><li>x is the independent variable (the input feature).</li><li>b0,b1,b2,…,bn​ are the coefficients that we need to estimate.</li><li>n is the degree of the polynomial.</li><li>ϵ is the error term (the part we can’t predict).</li></ul><h3>How is it Different from Linear Regression?</h3><p>In linear regression, we assume a straight-line relationship:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/186/1*823rIQllrKU3VwoS__Y-Tg.png" /></figure><p>But in polynomial regression, we introduce additional terms like x², x³ and so on, to capture curves. This makes it more flexible for handling nonlinear patterns.</p><h3>Step-by-Step Explanation</h3><h4>1. Why Add Polynomial Terms?</h4><p>Let’s take an example:</p><ul><li>If you try to fit a straight line to data points shaped like a parabola, a simple linear regression won’t fit well.</li><li>By introducing an x² term, the model can curve to better fit the data.<br>This added flexibility comes at the cost of complexity, which is why choosing the right degree of the polynomial is crucial.</li></ul><h4>2. Degree of the Polynomial: How Much is Too Much?</h4><p>The degree of the polynomial (n) controls the model’s complexity:</p><ul><li><strong>Low Degree (e.g., n=1):</strong> Underfitting. The model is too simple to capture patterns.</li><li><strong>High Degree (e.g., n=10):</strong> Overfitting. The model is too complex and starts fitting noise in the data.</li></ul><p><em>Example:</em><br>Suppose you’re modeling sales over time.</p><ul><li>A degree-1 polynomial (straight line) might predict sales steadily rising.</li><li>A degree-2 polynomial might reveal a peak and then a decline.</li><li>A degree-10 polynomial might capture every small fluctuation, but it won’t generalize well to future data.</li></ul><h4>3. Feature Engineering: Transforming Data</h4><p>To implement polynomial regression, we transform the original feature (x) into polynomial features like x², x³, x^n. For example:</p><p>x=[1,2,3]</p><p>With a degree of 2, we generate:</p><p>x_poly=[[1,1],[2,4],[3,9]]</p><p>Here, the first column is x, and the second column is x².</p><h4>4. Fitting the Model</h4><p>Once the polynomial features are created, we use ordinary least squares (OLS) regression to estimate the coefficients (b0,b1,b2,…,bn​). The steps include:</p><ol><li>Creating a polynomial feature matrix.</li><li>Solving for coefficients by minimizing the residual sum of squares (RSS).</li></ol><h4>Examples to Illustrate Polynomial Regression</h4><p>Example 1: Predicting Sales</p><ul><li><strong>Scenario:</strong> A bakery observes that cake sales first increase, then decrease with the temperature.</li><li><strong>Linear Regression:</strong> Predicts a straight line, either increasing or decreasing.</li><li><strong>Polynomial Regression (Degree 2):</strong> Captures the peak sales at the optimal temperature.</li></ul><p>Example 2: Modeling Housing Prices</p><ul><li><strong>Scenario:</strong> The relationship between square footage and price is nonlinear—price plateaus for larger homes.</li><li><strong>Solution:</strong> Polynomial regression with a degree of 3 or 4 can model this pattern effectively.</li></ul><h3>Choosing the Degree of the Polynomial</h3><p>The degree of the polynomial directly affects the model’s flexibility:</p><ul><li>A low-degree polynomial (e.g., 1 or 2) might miss key patterns in the data (underfitting).</li><li>A high-degree polynomial (e.g., 10 or more) can capture noise and fail to generalize (overfitting).</li></ul><h4>How to Decide the Right Degree?</h4><ul><li><strong>Visualization:</strong> Start by plotting the data and testing different degrees.</li><li><strong>Cross-Validation:</strong> Use techniques like k-fold cross-validation to test multiple degrees and identify the one with the best validation performance.</li><li><strong>Domain Knowledge:</strong> Often, the degree of the polynomial reflects the underlying nature of the problem.</li></ul><p><em>Example:</em><br>For temperature vs. sales, a degree of 2 or 3 is often sufficient. But for more complex systems like stock market trends, higher degrees might be needed.</p><h3>Preventing Overfitting and Underfitting</h3><ul><li><strong>Regularization Techniques:</strong> Use Lasso or Ridge regression to penalize large coefficients in high-degree polynomials. This reduces the likelihood of overfitting.</li><li><strong>Train-Test Split:</strong> Always test your model on unseen data to check for generalization.</li><li><strong>Feature Scaling:</strong> Standardize or normalize your data for better optimization, especially for higher-degree polynomial features.</li></ul><h3>Evaluating Model Performance</h3><p>Just as with other regression techniques, evaluating polynomial regression involves:</p><h4>Metrics</h4><ul><li><strong>R-Squared (R²):</strong> Measures how well the model explains the variance in the data.</li><li><strong>Adjusted R-Squared:</strong> accounts for the number of predictors and penalizes overcomplexity.</li><li><strong>Mean Squared Error (MSE):</strong> Quantifies the average squared difference between predicted and actual values.</li><li><strong>Root Mean Squared Error (RMSE):</strong> The square root of MSE, offering an interpretable error measure in the original units.</li></ul><h4>Residual Analysis</h4><p>Plot the residuals (errors) to ensure they are randomly distributed. If patterns exist in residuals, it indicates the model may not be capturing the data’s structure accurately.</p><h3>Polynomial Regression in Action</h3><p>Let’s consider a Python implementation using <strong>scikit-learn</strong>:</p><pre>import numpy as np<br>import matplotlib.pyplot as plt<br><br>from sklearn.model_selection import train_test_split<br><br>from sklearn.linear_model import LinearRegression,SGDRegressor<br><br>from sklearn.preprocessing import PolynomialFeatures,StandardScaler<br><br>from sklearn.metrics import r2_score<br><br>from sklearn.pipeline import Pipeline</pre><pre>X = 6 * np.random.rand(200, 1) - 3<br>y = 0.8 * X**2 + 0.9 * X + 2 + np.random.randn(200, 1)</pre><pre>plt.plot(X, y,&#39;b.&#39;)<br>plt.xlabel(&quot;X&quot;)<br>plt.ylabel(&quot;y&quot;)<br>plt.show()</pre><pre># Train test split<br>X_train,X_test,y_train,y_test = train_test_split(X,y,test_size=0.2,random_state=2)<br><br># Applying linear regression<br>lr = LinearRegression()<br><br>lr.fit(X_train,y_train)</pre><pre>y_pred = lr.predict(X_test)<br>r2_score(y_test,y_pred)</pre><pre>plt.plot(X_train,lr.predict(X_train),color=&#39;r&#39;)<br>plt.plot(X, y, &quot;b.&quot;)<br>plt.xlabel(&quot;X&quot;)<br>plt.ylabel(&quot;y&quot;)<br>plt.show()</pre><h4>Applying Polynomial Linear Regression</h4><pre># degree 2<br>poly = PolynomialFeatures(degree=2,include_bias=True)<br>X_train_trans = poly.fit_transform(X_train)<br>X_test_trans = poly.transform(X_test)</pre><pre>print(X_train[0])<br>print(X_train_trans[0])<br><br># include_bias parameter<br><br>lr = LinearRegression()<br>lr.fit(X_train_trans,y_train)<br><br>y_pred = lr.predict(X_test_trans)<br>print(r2_score(y_test,y_pred))<br>print(lr.coef_)<br>print(lr.intercept_)</pre><pre>X_new=np.linspace(-3, 3, 200).reshape(200, 1)<br>X_new_poly = poly.transform(X_new)<br>y_new = lr.predict(X_new_poly)<br><br>plt.plot(X_new, y_new, &quot;r-&quot;, linewidth=2, label=&quot;Predictions&quot;)<br>plt.plot(X_train, y_train, &quot;b.&quot;,label=&#39;Training points&#39;)<br>plt.plot(X_test, y_test, &quot;g.&quot;,label=&#39;Testing points&#39;)<br>plt.xlabel(&quot;X&quot;)<br>plt.ylabel(&quot;y&quot;)<br>plt.legend()<br>plt.show()</pre><pre>def polynomial_regression(degree):<br>    X_new=np.linspace(-3, 3, 100).reshape(100, 1)<br>    X_new_poly = poly.transform(X_new)<br><br>    polybig_features = PolynomialFeatures(degree=degree, include_bias=False)<br>    std_scaler = StandardScaler()<br>    lin_reg = LinearRegression()<br>    polynomial_regression = Pipeline([<br>            (&quot;poly_features&quot;, polybig_features),<br>            (&quot;std_scaler&quot;, std_scaler),<br>            (&quot;lin_reg&quot;, lin_reg),<br>        ])<br>    polynomial_regression.fit(X, y)<br>    y_newbig = polynomial_regression.predict(X_new)<br>    plt.plot(X_new, y_newbig,&#39;r&#39;, label=&quot;Degree &quot; + str(degree), linewidth=2)<br><br>    plt.plot(X_train, y_train, &quot;b.&quot;, linewidth=3)<br>    plt.plot(X_test, y_test, &quot;g.&quot;, linewidth=3)<br>    plt.legend(loc=&quot;upper left&quot;)<br>    plt.xlabel(&quot;X&quot;)<br>    plt.ylabel(&quot;y&quot;)<br>    plt.axis([-3, 3, 0, 10])<br>    plt.show()<br><br><br>polynomial_regression(5)</pre><pre># Applying Gradient Descent<br><br>poly = PolynomialFeatures(degree=2)<br><br>X_train_trans = poly.fit_transform(X_train)<br>X_test_trans = poly.transform(X_test)<br><br>sgd = SGDRegressor(max_iter=100)<br>sgd.fit(X_train_trans,y_train)<br><br>X_new=np.linspace(-2.9, 2.8, 200).reshape(200, 1)<br>X_new_poly = poly.transform(X_new)<br>y_new = sgd.predict(X_new_poly)<br><br>y_pred = sgd.predict(X_test_trans)<br><br>plt.plot(X_new, y_new, &quot;r-&quot;, linewidth=2, label=&quot;Predictions &quot; + str(round(r2_score(y_test,y_pred),2)))<br>plt.plot(X_train, y_train, &quot;b.&quot;,label=&#39;Training points&#39;)<br>plt.plot(X_test, y_test, &quot;g.&quot;,label=&#39;Testing points&#39;)<br>plt.xlabel(&quot;X&quot;)<br>plt.ylabel(&quot;y&quot;)<br>plt.legend()<br>plt.show()</pre><h3>Advantages and Disadvantages of Polynomial Regression</h3><p>While polynomial regression is a versatile technique, it’s not without its strengths and limitations. Let’s weigh both sides:</p><h3>Advantages</h3><ol><li><strong>Captures Non-Linear Relationships:</strong><br>Unlike linear regression, it can model more complex, curved relationships between variables.</li><li><strong>Simple to Implement:</strong><br>With tools like Python’s scikit-learn, it’s easy to transform data into polynomial features and apply regression.</li><li><strong>Interpretable:</strong><br>Compared to more complex models (e.g., neural networks), polynomial regression remains relatively easy to understand and explain.</li><li><strong>Flexible:</strong><br>By adjusting the degree of the polynomial, you can control the model’s flexibility to fit data more accurately.</li></ol><h3>Disadvantages</h3><ol><li><strong>Prone to Overfitting:</strong><br>Higher-degree polynomials can overfit the data, capturing noise instead of meaningful patterns.</li><li><strong>Sensitive to Outliers:</strong><br>Polynomial regression models can be heavily influenced by outliers, distorting predictions.</li><li><strong>Extrapolation Issues:</strong><br>Predictions outside the range of the training data can be unreliable due to rapid changes in polynomial functions.</li><li><strong>Computationally Expensive for High Degrees:</strong><br>As the degree increases, computational complexity rises, and the model can become harder to interpret.</li><li><strong>Assumes Continuous Data:</strong><br>It’s less effective for categorical variables unless they are carefully encoded.</li></ol><h3>Conclusion</h3><p>Polynomial regression is an effective tool for modeling non-linear relationships, offering flexibility and interpretability. However, it comes with challenges like overfitting and sensitivity to outliers. By carefully selecting the polynomial degree, using cross-validation, and employing regularization, you can strike a balance between model complexity and generalization.</p><p>Whether it’s predicting house prices or understanding seasonal trends, polynomial regression equips you with a robust approach to tackle non-linear data. However, always remember that simpler models should be your first choice—polynomial regression shines when non-linearity is evident and necessary to address.</p><p>Thank you for joining us on this journey in “100 Days of Machine Learning with Gayatri!” See you in the next blog.</p><p>Until then, Keep learning, keep exploring and I’ll see you soon! 😊</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2fef8b42a397" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>