<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Divya Iyer on Medium]]></title>
        <description><![CDATA[Stories by Divya Iyer on Medium]]></description>
        <link>https://medium.com/@divyavikb?source=rss-e2f18fc7c50b------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*Moc7OmRSjG-WjfmNyQrTDQ.jpeg</url>
            <title>Stories by Divya Iyer on Medium</title>
            <link>https://medium.com/@divyavikb?source=rss-e2f18fc7c50b------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 03:51:27 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@divyavikb/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[PCA for Dummies]]></title>
            <link>https://medium.com/@divyavikb/pca-for-dummies-e64a5d8711cb?source=rss-e2f18fc7c50b------2</link>
            <guid isPermaLink="false">https://medium.com/p/e64a5d8711cb</guid>
            <category><![CDATA[neural-networks]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[agentic-ai]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Divya Iyer]]></dc:creator>
            <pubDate>Fri, 11 Sep 2026 11:40:22 GMT</pubDate>
            <atom:updated>2026-09-11T11:40:22.200Z</atom:updated>
            <content:encoded><![CDATA[<p>Principal Component Analysis (PCA) is probably one of the most confusing and complex topics in machine learning I’ve encountered. Every time I read a paper on it, I either feel like things finally click or I end up more lost than before. In this article, I’m going to break down PCA in the simplest way possible so it actually makes sense.</p><p>The secret to understanding PCA? Stop thinking in equations and start thinking in pictures.</p><p><strong>Finding the Best View of Your Data</strong></p><p>Imagine you’re at a cricket match. You’re seeing the game in 3D ,the bowler running, the batsman swinging and the crowd shouting in excitement. The ball goes high in the air, fielders dive across the ground and the whole moment feels alive around you.</p><p>Later, someone shows you a photo of that exact moment. The photo is flat, but it still tells the story the ball’s path, the batsman’s position, the fielders ready to move.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*exwnn32I3hbzOtA2.png" /></figure><p>That transformation from a full 3D scene to a simple 2D picture is exactly what PCA does. It takes big, complex data and gives you a simpler view while keeping the main story intact. But PCA doesn’t just flatten the data blindly it looks for the best directions, the angles where the data spreads out the most, because those directions carry the richest information. By projecting the data onto these directions, PCA ensures the simpler picture still captures the heart of the story.</p><p><strong>From Regularization to PCA</strong></p><p>In my previous article, I talked about dimensionality reduction through regularization (like Ridge and Lasso). PCA also reduces dimensions, but it does so in a different way.</p><p>Ridge and Lasso deal with high-dimensional data by trimming the influence of features,PCA on the other hand creates entirely new features called principal components that are best combinations of the original features.</p><p><strong>A Quick Math Refresher</strong></p><p>Before we dive deeper, let’s brush up on some high school math. After all, we’re learning machine learning and a little math foundation never hurts.</p><p>In the ML world, your dataset is a matrix of numbers (assuming all your columns are numeric , ML algorithms don’t understand words, only numbers) arranged in rows and columns.</p><p>For example, here’s a tiny cricket dataset:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/383/0*LUE2qjiYVrBW3h7A.png" /></figure><p>Take any row from that dataset, say Player A’s stats: [45, 8, 78]. That’s a vector , a list of numbers that can be visualized as a point in 3D space (since we have 3 numeric features). Each player is one point, and together they form a cloud of points floating in space.</p><p>A matrix isn’t just a table of numbers, it’s a transformation engine. Multiply your data points by the right kind of matrix and you can rotate, stretch or tilt the entire cloud of points. Just like watching the same cricket match from a different stand in the stadium, a matrix transformation gives you a completely new perspective.</p><p><strong>Understanding Variance</strong></p><p>Variance tells us how much the values of one feature are spread out. If “Runs” has high variance, it means some players score very low while others score very high. If variance is low, most players score within a similar range.</p><p>Variance matters for PCA because it’s hunting for directions where the data spreads out the most , bigger variance equals more information, while features that barely change don’t add much value. Think of variance as the “action” in your data: when you skim a scoreboard, some columns barely budge while others swing wildly. That swing is variance , it’s what makes the data interesting. PCA’s simple question is: From which angle does the data look most spread out?<em> </em>In our cricket analogy: Where would you place the camera so the frame captures the maximum action in a single shot?</p><p><strong>Covariance: How Features work Together</strong></p><p>Covariance takes this concept one step further. It doesn’t just examine a single feature’s spread,it checks whether two features move together in harmony. For example:</p><ul><li>If runs go up and boundaries also go up → <strong>positive covariance</strong></li><li>If runs scored by batsmen and economy rate of bowlers move in opposite directions → <strong>negative covariance</strong></li><li>A player’s height and their strike rate have no meaningful relationship → <strong>zero covariance</strong></li></ul><p>Covariance matrix captues variance of features along diagonal entries and the covariance between pairs of features in Off-diagonal entries</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*mjRFVSeEd8UEBZ8O.png" /></figure><p>Why do we need covariance? Because PCA isn’t only about spread in one feature,it’s about finding new directions (principal components) that capture the combined variation of multiple features. The covariance matrix reveals these intricate relationships. PCA then rotates the data cloud to align with the directions where the combined spread is largest.</p><p>In 2D scatterplot of features</p><ul><li>If both features are unrelated (zero covariance), the cloud of data points looks more like a circle, aligned with the X and Y axes</li><li>If both the features are positively related meaning both rise together , the cloud stretches into a tilted ellipse, pointing diagonally</li><li>If negatively related meaning one rises while the other falls, the ellipse tilts the opposite way</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*75d93--W7gfuek2t.png" /></figure><p>The tilt and stretch of the ellipse tell us about the underlying relationships. PCA then identifies the direction where this ellipse is longest and that becomes the first principal component, the direction with maximum variation.</p><p><strong>Finding the Perfect Angles</strong></p><p>Now in which direction is the variance the greatest? If we can find that direction, we capture the maximum information in the data.</p><p>This is where eigenvectors and eigenvalues enter the scene:</p><ul><li>An eigenvector is a direction, think of it as a special camera angle around the stadium, perfectly aligned with the longest stretch of the data cloud</li><li>An eigenvalue tells us how much variance (or “action”) is captured from that angle. Bigger eigenvalue = more spread in that direction = more information preserved.</li></ul><p>In 2D, if the ellipse is tilted diagonally, the first eigenvector lies along that tilt. Its eigenvalue is large because the data varies most along this axis.</p><p>The first principal component PC1 is the eigenvector with the largest eigenvalue,the direction of maximum variation and the second principal component PC2 is the next best, but it must reveal something new, not just a near-duplicate of the first</p><p>That’s why PCA forces the second component to be orthogonal (at right angles) to the first. Orthogonality guarantees that this new view is uncorrelated with the previous one. You get fresh information, not just another photo of the same moment from three seats away.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/850/0*ft3nYg15BwBE9eIY.png" /></figure><p><a href="https://www.researchgate.net/publication/335239031/figure/fig2/AS:11431281371582974@1744374766598/Schematic-of-the-PCA-transformation-Original-data-space-presented-on-the-left-with-3.tif">Image Source</a></p><p><strong>The Transformation in Action</strong></p><p>So far, we’ve talked about variance, covariance and how eigenvectors give us the best angles,but what happens to our data when we actually apply this transformation?</p><p>Think back to our cloud of points each player’s stats floating in space. Right now, the cloud is messy,tilted, uneven and not neatly aligned with any obvious axes. PCA steps in to organize this chaos and reveal the story.</p><p>Here’s the step-by-step process:</p><p><strong>1. Center the data</strong><br>First, we shift the entire cloud so it sits around the origin its called mean centering.This removes bias from where the data starts. Without this step, our new directions would be skewed by arbitrary starting points.</p><p><strong>2. Build the covariance matrix</strong><br>The covariance matrix acts like shining a spotlight on the cloud. It encodes both how spread out each feature is (variance) and how features move together (covariance). In effect, it captures the complete structure of the cloud,the stretches, tilts, and relationships.</p><p><strong>3. Find the eigenvectors (new axes)</strong><br>From the covariance matrix we extract <strong>eigenvectors</strong> and <strong>eigenvalues</strong>.</p><p>Each <strong>eigenvector</strong> is a special axis that cuts through the data cloud. It’s like the “cleanest possible angle” from which to view the data. And <strong>eigenvalue</strong> corresponding to the eigenvector tells us how much variance (spread, action) lies along that direction.</p><p><strong>4. Rotate the cloud onto these new axes</strong><br>Now comes the transformation. We rotate the data so the natural axes of the cloud aligns with the eigenvectors.</p><ul><li>Along the <strong>first eigenvector</strong>, the points spread out the most maximum variance, the main storyline of the data</li><li>Along the <strong>second eigenvector</strong>, the spread is smaller but completely independent of the first</li></ul><p>Geometrically, that messy tilted ellipse of points becomes neatly aligned with our new coordinate system.</p><p><strong>5.Drop weaker directions</strong><br>Not every direction in your data cloud matters equally. Some eigenvalues are huge they capture the big actions. Others are tiny , they only capture minor jitters or noise. Instead of chasing every tiny detail, PCA keeps the big patterns and drops the small noise. So In practice, this means we can reduce a dataset with 10 features down to just 2 or 3 principal components, while still keeping most of the important story.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/602/0*UVqgaeHcUT4m4cra.png" /></figure><p><a href="https://snowflect.com/principal-component-analysis/">Image Source</a></p><p>One of the clearest ways to see PCA in action is to start with simple 2D data. Imagine plotting players Runs vs Boundaries. PCA will find the best-fit line (PC1) through this cloud the direction of maximum variance. Then, imagine each 2D point is dropped perpendicularly onto this line. As a result every player that was once a 2D point (x1,x2) is now just a single coordinate t₁​ along PC1. The 2D story collapses into 1D without losing much of the action.</p><p>And if you project instead onto PC2 (the orthogonal axis), you get a complementary 1D view capturing variation that PC1 missed. Together, PC1 and PC2 preserve most of the spread, but if PC1 already explains enough, PCA may keep only PC1 reducing dimensionality and simplifying the dataset.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*qTKBCKNbRoZuAKja.png" /></figure><p><strong>Blends, Not Originals</strong></p><p>These new axes PC1 and PC2 (the eigenvectors) are not just copies of your original features like “Runs” or “Boundaries.” They are blends of them.</p><p>Let’s say wehave only 2 two features runs (x₁) and boundaries (x₂). The first principal component might look like:</p><p><strong>PC₁ = 0.71 × x₁ + 0.71 × x₂</strong></p><p>This diagonal direction represents the axis of maximum variance, slicing through the tilted cloud. Notice: it’s not “just runs” or “just boundaries” it’s a weighted mixture of both.</p><p>Those numbers (0.71 and 0.71) are called loadings. They tell you how much each original feature contributes to the new principal component:</p><ul><li>A high loading (close to +1 or -1) means that feature strongly influences the component.</li><li>A low loading (close to 0) means the feature barely matters for that component.</li></ul><p>So in our example:</p><ul><li>Runs contributes 0.71 — strong influence.</li><li>Boundaries contribute 0.71 -equally strong influence.</li></ul><p>That’s why PC1 ​ here is a balanced mix of both features.</p><p>When PCA transforms the data, each original data point gets expressed in this new coordinate system. Instead of saying <em>“A scored 45 runs and 8 boundaries,”</em> PCA might represent A as <em>“A= 52 units along PC₁, 10 units along PC₂.”</em></p><p>It’s like shifting from the raw scoreboard to highlights package the same story, but compressed into fewer and more meaningful dimensions.</p><p><strong>How many Principal Components can there be?</strong></p><p>If your dataset has n features (columns), then at most you can have n principal components.Each principal component is a new axis, formed as a blend of the original features.The first few usually explain most of the variance, while the later ones capture very little (often just noise).</p><p><strong>Knowing When to Stop</strong></p><p>Each principal component comes with its share of the total variance it explains, called the explained variance. Divide this by the total variance and you get the explained variance ratio. Add these ratios together cumulatively (PC₁ + PC₂ + PC₃ …), and you get a curve that shows how many components are enough to capture the bulk of the story.</p><p>Looking at the plot below, the first component alone explains about <strong>32%</strong> of the variance. By the time we add PC₂, we’ve covered <strong>57%</strong> and with PC₃ it jumps to <strong>77%</strong>. Including PC₄ takes us up to <strong>91%</strong>, and with just five components we’re already at <strong>96%</strong>. Beyond that, the gains are tiny.</p><p>So in this dataset, the first <strong>4–5 components</strong> are enough to capture almost all of the meaningful variation the rest contribute very little.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/752/0*RET-v1BUamE4Q5JI.jpeg" /></figure><p>To summarize<strong> , </strong>PCA is like a master photographer at a cricket match, it finds the few best angles that capture the most action while leaving out redundant or uninformative views. It does this by centering the data, studying its spread through covariance and then rotating it onto the optimal axes defined by eigenvectors. Along these new axes, the raw features combine into smart blends, each carrying unique information without repetition. The explained variance then tells us when we’ve captured enough of the story without needing every single angle.</p><p><strong>A Checklist</strong></p><ul><li><strong>Center and scale</strong> your features so no single unit dominates.</li><li><strong>Check loadings</strong> to interpret each principal component: which original features shape the new axes?</li><li><strong>Choose the number of components (k)</strong> using explained variance curves (or cross-validation</li><li><strong>Linearity</strong>: PCA only captures straight-line patterns. If your data bends like a googly, explore nonlinear methods later.</li></ul><p>Thanks for stopping by! I hope this made PCA slightly more clearer.Feel free to drop your thoughts or questions in the comments below.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e64a5d8711cb" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>