<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Suneth Nawoda De Silva on Medium]]></title>
        <description><![CDATA[Stories by Suneth Nawoda De Silva on Medium]]></description>
        <link>https://medium.com/@sunethsilva?source=rss-c1f157a43ba8------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*l5v0_Xme5Cgshkg-XaX7_Q.jpeg</url>
            <title>Stories by Suneth Nawoda De Silva on Medium</title>
            <link>https://medium.com/@sunethsilva?source=rss-c1f157a43ba8------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 22:01:34 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@sunethsilva/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Train-Test Split and Feature Scaling in Python: A Practical Scikit-Learn implementation Guide]]></title>
            <link>https://medium.com/@sunethsilva/train-test-split-and-feature-scaling-in-python-a-practical-scikit-learn-implementation-guide-25a085b69925?source=rss-c1f157a43ba8------2</link>
            <guid isPermaLink="false">https://medium.com/p/25a085b69925</guid>
            <category><![CDATA[python]]></category>
            <category><![CDATA[feature-scaling]]></category>
            <category><![CDATA[data-preprocessing]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Suneth Nawoda De Silva]]></dc:creator>
            <pubDate>Sun, 25 Oct 2026 13:23:59 GMT</pubDate>
            <atom:updated>2026-10-25T13:23:59.893Z</atom:updated>
            <content:encoded><![CDATA[<h3>Train-Test Split and Feature Scaling in Python: A Practical Scikit-Learn implementation Guide with Best Practices</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*QuxtZ8pxUN5lPRMp6Eng2Q.png" /></figure><p>Understanding <em>why</em> we split a dataset and <em>why</em> feature scaling is useful is important. But the next question is more practical:</p><p><strong>How do we actually implement these steps correctly in Python without accidentally introducing data leakage?</strong></p><p>In this article, I will focus mainly on the implementation using <strong>Scikit-learn</strong>, along with a few important details that are easy to miss when you are getting started.</p><p>If you want the theory first, you can read my previous articles:</p><ul><li><a href="https://medium.com/@sunethsilva/why-do-we-split-data-in-machine-learning-2268112fe655">Why Do We Split Data in Machine Learning?</a></li><li><a href="https://medium.com/@sunethsilva/why-machine-learning-needs-feature-scaling-a-beginner-friendly-guide-f913d6592184">Why Machine Learning Needs Feature Scaling: A Beginner-Friendly Guide</a></li></ul><p>Now, let’s put those ideas into practice.</p><h3>1. Start with the Features and Target</h3><p>Suppose we already loaded a dataset using Pandas.</p><pre>import pandas as pd</pre><pre>dataset = pd.read_csv(&quot;Data.csv&quot;)</pre><p>In a typical machine learning problem, we separate the dataset into:</p><pre>X = dataset.iloc[:, :-1].values<br>y = dataset.iloc[:, -1].values</pre><p><em>If you want the theory about X and y, you can read my previous article:</em> <a href="https://medium.com/@sunethsilva/from-a-csv-file-to-x-and-y-understanding-your-first-machine-learning-dataset-1ed163ee098e">From a CSV File to X and y: Understanding Your First Machine Learning Dataset</a></p><p>Here:</p><ul><li>X contains the input features</li><li>y contains the target we want to predict</li></ul><p>For example, if the dataset looks like this:</p><pre>| Country | Age | Salary | Purchased |<br>|---------|-----|--------|-----------|<br>| France  | 25  | 50000  | 1         |<br>| Spain   | 35  | 65000  | 0         |<br>| Germany | 45  | 90000  | 1         |</pre><p>then:</p><pre>X = Country, Age, Salary<br>y = Purchased</pre><p>This gives us the data we need before creating the training and test sets.</p><h3>2. Split the Dataset</h3><p>Scikit-learn provides train_test_split() for this.</p><pre>from sklearn.model_selection import train_test_split</pre><pre>X_train, X_test, y_train, y_test = train_test_split(<br>    X,<br>    y,<br>    test_size=0.2,<br>    random_state=42<br>)</pre><p>This gives us four variables:</p><pre>| Variable| Used for                         |<br>|---------|----------------------------------|<br>| X_train | Training input features          |<br>| X_test  | Test input features              |<br>| y_train | Training target values           |<br>| y_test  | Actual target values for testing |</pre><p>With:</p><pre>test_size=0.2</pre><p>approximately 20% of the observations are used for testing and the remaining 80% for training.</p><h3>3. Why Use random_state?</h3><p>You will often see something like:</p><pre>random_state=42</pre><p>The number 42 itself is not special.</p><p>Its purpose is simply to make the random split <strong>reproducible</strong>.</p><p>If we run the program again with the same dataset and settings, we can get the same training and test observations.</p><p>You could also use:</p><pre>random_state=1</pre><p>or another fixed integer.</p><p>The main thing to remember is:</p><blockquote><em>random_state helps reproduce the same experiment. It does not automatically improve model accuracy.</em></blockquote><h3>4. Split First, Scale Second</h3><p>This is probably the most important implementation rule in this article.</p><p>Do <strong>not</strong> fit a scaler using the entire dataset before splitting it.</p><p>The correct workflow is:</p><pre>Original Dataset<br>       ↓<br>Train-Test Split<br>       ↓<br> ┌───────────────┐<br> ↓               ↓<br>X_train        X_test<br> ↓               ↓<br>Fit scaler      Use same scaler<br>on X_train      to transform<br> ↓               ↓<br>Scaled         Scaled<br>X_train        X_test</pre><p>In short:</p><pre>Split → Fit on Training Data → Transform Training Data → Transform Test Data</pre><p>Why?</p><p>Because the test set is supposed to behave like unseen data.</p><p>If the scaler learns information from X_test, such as its mean, standard deviation, minimum, or maximum, the test set has already influenced preprocessing.</p><p>That is a form of <strong>data leakage</strong>.</p><h3>5. Implementing Standardisation with StandardScaler</h3><p>Scikit-learn provides StandardScaler:</p><pre>from sklearn.preprocessing import StandardScaler</pre><p>Then create the scaler:</p><pre>sc = StandardScaler()</pre><p>At this stage, nothing has been scaled yet.</p><p>The scaler has not learned anything from the data.</p><p>Now we fit it using the training features:</p><pre>X_train = sc.fit_transform(X_train)</pre><p>Then transform the test features:</p><pre>X_test = sc.transform(X_test)</pre><p>The complete code looks like this:</p><pre>from sklearn.model_selection import train_test_split<br>from sklearn.preprocessing import StandardScaler<br><br># Split the dataset<br>X_train, X_test, y_train, y_test = train_test_split(<br>    X,<br>    y,<br>    test_size=0.2,<br>    random_state=42<br>)<br><br># Create the scaler<br>sc = StandardScaler()<br><br># Learn from training data and scale it<br>X_train = sc.fit_transform(X_train)<br><br># Apply the same scaling to test data<br>X_test = sc.transform(X_test)</pre><p>This small difference between the two lines is extremely important.</p><h3>6. fit_transform() vs transform()</h3><p>These two methods can be confusing at first.</p><h3>fit_transform()</h3><pre>sc.fit_transform(X_train)</pre><p>It does two things:</p><ol><li><strong>Fit</strong> — learns the required values from the training data</li><li><strong>Transform</strong> — applies the scaling</li></ol><p>For StandardScaler, the values it learns include the mean and standard deviation of each feature.</p><h3>transform()</h3><pre>sc.transform(X_test)</pre><p>This does not learn anything new.</p><p>It simply uses the mean and standard deviation that were already learned from X_train.</p><p>A simple way to remember it is:</p><pre>fit = learn the rules<br>transform = apply the rules</pre><p>Therefore:</p><pre>X_train → fit_transform()<br>X_test  → transform()</pre><p>Avoid this:</p><pre>X_test = sc.fit_transform(X_test)</pre><p>That would make the scaler learn new values from the test dataset.</p><h3>7. Do We Scale Every Column?</h3><p>Not necessarily.</p><p>Suppose we originally had a categorical feature called Country.</p><p>After One-Hot Encoding, our features may look like:</p><pre>| France | Germany | Spain | Age | Salary |<br>|--------|---------|-------|-----|--------|<br>| 1      | 0       | 0     | 25  | 50000  |<br>| 0      | 0       | 1     | 35  | 65000  |<br>| 0      | 1       | 0     | 45  | 90000  |</pre><p><em>For more about One-Hot Encoding</em>: <a href="https://medium.com/@sunethsilva/turning-text-into-numbers-a-beginners-guide-to-categorical-data-encoding-in-machine-learning-d503b37a55a4">Turning Text into Numbers: A Beginner’s Guide to Categorical Data Encoding in Machine Learning</a></p><p>The first three columns are dummy variables:</p><pre>France<br>Germany<br>Spain</pre><p>Their values are already represented as:</p><pre>0 or 1</pre><p>Meanwhile:</p><pre>Age<br>Salary</pre><p>are numerical features with very different ranges.</p><p>For a simple workflow, we may want to scale only:</p><pre>Age<br>Salary</pre><p>and leave the one-hot encoded columns unchanged.</p><h3>8. Scaling Only Selected Numerical Features</h3><p>Suppose our columns are arranged like this:</p><pre>| Index | Feature |<br>|-------|---------|<br>| 0     | France  |<br>| 1     | Germany |<br>| 2     | Spain   |<br>| 3     | Age     |<br>| 4     | Salary  |</pre><p>We can select the numerical columns using:</p><pre>X_train[:, 3:]</pre><p>What does this mean?</p><pre>:</pre><p>before the comma means:</p><blockquote><em>select all rows</em></blockquote><p>while:</p><pre>3:</pre><p>means:</p><blockquote><em>select column 3 and every column after it</em></blockquote><p>So:</p><pre>X_train[:, 3:]</pre><p>selects:</p><pre>Age<br>Salary</pre><p>We can then scale only these columns:</p><pre>X_train[:, 3:] = sc.fit_transform(X_train[:, 3:])</pre><p>and for the test data:</p><pre>X_test[:, 3:] = sc.transform(X_test[:, 3:])</pre><p>The result is:</p><pre>Country dummy variables → unchanged<br>Age                     → standardised<br>Salary                  → standardised</pre><h3>9. One Small Python Detail That Matters</h3><p>This line:</p><pre>sc.fit_transform(X_train[:, 3:])</pre><p>returns the scaled values.</p><p>But if we want those scaled values to replace the original values inside X_train, we need to assign them back:</p><pre>X_train[:, 3:] = sc.fit_transform(X_train[:, 3:])</pre><p>The same applies to the test set:</p><pre>X_test[:, 3:] = sc.transform(X_test[:, 3:])</pre><p>Without the assignment, you may calculate the transformed values without actually updating the selected part of your dataset.</p><h3>10. Complete Example</h3><p>Here is the implementation together:</p><pre>import pandas as pd<br>from sklearn.model_selection import train_test_split<br>from sklearn.preprocessing import StandardScaler<br><br># Load dataset<br>dataset = pd.read_csv(&quot;Data.csv&quot;)<br><br># Separate features and target<br>X = dataset.iloc[:, :-1].values<br>y = dataset.iloc[:, -1].values<br><br># Split dataset<br>X_train, X_test, y_train, y_test = train_test_split(<br>    X,<br>    y,<br>    test_size=0.2,<br>    random_state=42<br>)<br><br># Create scaler<br>sc = StandardScaler()<br><br># Scale only Age and Salary<br>X_train[:, 3:] = sc.fit_transform(X_train[:, 3:])<br>X_test[:, 3:] = sc.transform(X_test[:, 3:])</pre><p>This assumes:</p><ul><li>the required categorical encoding has already been performed</li><li>Age starts at column index 3</li><li>the columns after it are numerical features we want to scale</li><li>the training and test arrays have the same column structure</li></ul><p>For another dataset, the column positions may be different.</p><p>So do not blindly copy:</p><pre>[:, 3:]</pre><p>Always check your feature order first.</p><h3>11. What Should the Output Look Like?</h3><p>After standardisation, you may see something similar to:</p><p>Before scaling:</p><pre>Age       Salary<br>25        50000<br>35        65000<br>45        90000</pre><p>After scaling:</p><pre>Age       Salary<br>-1.22     -1.11<br> 0.00     -0.20<br> 1.22      1.31</pre><p>The exact numbers depend on the training data.</p><p>Negative values are completely normal.</p><p>With standardisation:</p><pre>negative → below the training mean<br>0        → around the training mean<br>positive → above the training mean</pre><p>The important thing is that the features are now represented on more comparable numerical scales.</p><h3>12. Common Mistakes to Avoid</h3><p>Here are a few mistakes worth remembering.</p><h3>❌ Mistake 1: Scaling before splitting</h3><pre>X = sc.fit_transform(X)</pre><pre>X_train, X_test, y_train, y_test = train_test_split(...)</pre><p>The scaler has already seen information from what later becomes the test set.</p><h3>✅ Better approach</h3><pre>X_train, X_test, y_train, y_test = train_test_split(...)</pre><pre>X_train = sc.fit_transform(X_train)<br>X_test = sc.transform(X_test)</pre><h3>❌ Mistake 2: Fitting a separate scaler on X_test</h3><pre>X_train = sc.fit_transform(X_train)<br>X_test = sc.fit_transform(X_test)</pre><p>Now the training and test data are being scaled using different learned statistics.</p><h3>✅ Correct approach</h3><pre>X_train = sc.fit_transform(X_train)<br>X_test = sc.transform(X_test)</pre><p>Fit once. Reuse the same scaler.</p><h3>❌ Mistake 3: Assuming every feature must be scaled</h3><p>Feature scaling depends on the model and the type of feature.</p><p>For example, in this simple workflow, One-Hot Encoded columns can remain as 0 and 1, while numerical columns such as Age and Salary are standardised.</p><h3>❌ Mistake 4: Forgetting the assignment</h3><p>This:</p><pre>sc.fit_transform(X_train[:, 3:])</pre><p>creates transformed values but does not necessarily replace the selected values in X_train.</p><p>Use:</p><pre>X_train[:, 3:] = sc.fit_transform(X_train[:, 3:])</pre><h3>13. A Reusable Preprocessing Template</h3><p>A useful basic template is:</p><pre>import numpy as np<br>import pandas as pd<br>from sklearn.model_selection import train_test_split<br>from sklearn.preprocessing import StandardScaler<br><br># 1. Load dataset<br>dataset = pd.read_csv(&quot;YOUR_DATASET.csv&quot;)<br><br># 2. Separate features and target<br>X = dataset.iloc[:, :-1].values<br>y = dataset.iloc[:, -1].values<br><br># 3. Split the dataset<br>X_train, X_test, y_train, y_test = train_test_split(<br>    X,<br>    y,<br>    test_size=0.2,<br>    random_state=42<br>)<br><br># 4. Apply feature scaling if required<br>sc = StandardScaler()<br>X_train = sc.fit_transform(X_train)<br>X_test = sc.transform(X_test)</pre><p>You can then add other preprocessing steps depending on the dataset, such as:</p><pre>Missing value handling<br>Categorical encoding<br>Feature selection<br>Feature scaling</pre><p>Not every project requires every preprocessing technique.</p><h3>14. The Most Important Rules to Remember</h3><p>You do not need to memorise every line immediately.</p><p>Remember these rules:</p><pre>| Rule                 | What to do                                                       |<br>|----------------------|------------------------------------------------------------------|<br>| Split before scaling | Create `X_train` and `X_test` first                              |<br>| Fit on training data | Learn scaling parameters from `X_train` only                     |<br>| Transform test data  | Use the same fitted scaler                                       |<br>| `fit_transform()`    | Mainly for training data                                         |<br>| `transform()`        | For test and future unseen data                                  |<br>| Scale selectively    | Scale the numerical features that need it                        |<br>| Check your indexes   | Do not assume columns are always in the same positions           |<br>| Avoid leakage        | Never allow test information to influence training preprocessing |</pre><h3>Final Takeaway</h3><p>The code for train-test splitting and feature scaling is not difficult.</p><p>The important part is understanding <strong>where each operation belongs</strong>.</p><pre>Prepare X and y<br>      ↓<br>Split the dataset<br>      ↓<br>Fit preprocessing using X_train<br>      ↓<br>Transform X_train<br>      ↓<br>Transform X_test with the SAME preprocessing<br>      ↓<br>Train the model<br>      ↓<br>Evaluate using the test set</pre><p>The key idea can be reduced to one sentence:</p><blockquote><strong><em>Fit preprocessing tools on the training data once, then reuse them for test data and future unseen data.</em></strong></blockquote><p>That small rule prevents data leakage and keeps the machine learning workflow consistent.</p><p>With the preprocessing stage ready, the next step is where things become even more interesting: <strong>training the actual machine learning model.</strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=25a085b69925" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Turning Text into Numbers: A Beginner’s Guide to Categorical Data Encoding in Machine Learning]]></title>
            <link>https://medium.com/@sunethsilva/turning-text-into-numbers-a-beginners-guide-to-categorical-data-encoding-in-machine-learning-d503b37a55a4?source=rss-c1f157a43ba8------2</link>
            <guid isPermaLink="false">https://medium.com/p/d503b37a55a4</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[one-hot-encoding]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[data-preprocessing]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Suneth Nawoda De Silva]]></dc:creator>
            <pubDate>Sat, 10 Oct 2026 12:32:21 GMT</pubDate>
            <atom:updated>2026-10-10T12:45:18.971Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*p2IeBYCEdgtc5M59bpckAA.png" /></figure><p><em>Learn how to convert categorical data into numbers using One-Hot Encoding, ColumnTransformer, and Python — explained step by step with simple examples.</em></p><h3>Introduction</h3><p>When working with Machine Learning, preparing our data is just as important as choosing the right algorithm.</p><p>In the previous preprocessing step(<a href="https://medium.com/@sunethsilva/handling-missing-data-in-machine-learning-a-beginners-guide-with-python-11f2bbdcb122">quick Link to Previous step</a>), we learned how to handle missing data. Now, let’s explore another important concept: <strong>Encoding Categorical Data</strong>.</p><p>Imagine we have a dataset containing information about customers from different countries.</p><pre>| Country | Age | Salary | Purchased |<br>|---------|----:|-------:|-----------|<br>| France  | 44  | 72000  | No        |<br>| Spain   | 27  | 48000  | Yes       |<br>| Germany | 30  | 54000  | Yes       |<br>| France  | 35  | 58000  | No        |</pre><p>Our goal is to build a Machine Learning model that predicts whether a customer will purchase a product.</p><p>But there’s a problem!</p><p>The <strong>Country</strong> column contains text values such as France, Spain, and Germany.</p><p>Most traditional Machine Learning algorithms work with numerical data and cannot directly perform calculations using these text categories.</p><p>So, how can we convert these categories into numbers without changing their meaning?</p><p>That’s where <strong>Categorical Data Encoding</strong> comes in! 🚀</p><h3>🧠 What Is Categorical Data?</h3><p>Categorical data represents labels or groups rather than numerical measurements.</p><p>For example:</p><pre>| Feature        | Categories             |<br>|----------------|------------------------|<br>| Country        | France, Spain, Germany |<br>| Color          | Red, Blue, Green       |<br>| Payment Method | Cash, Card, Online     |<br>| Weather        | Sunny, Rainy, Cloudy   |</pre><p>Think about the Country column.</p><p><strong>France, Spain, </strong>and <strong>Germany</strong> are simply names of different countries.</p><p>They don’t represent numerical quantities, and they don’t have a natural mathematical order.</p><p>To use these values in many Machine Learning algorithms, we need to convert them into a suitable numerical format.</p><p>This process is called <strong>Categorical Data Encoding</strong>.</p><h3>Why Can’t We Simply Assign Numbers?</h3><p>Let’s say we convert the Country column like this:</p><pre>| Country | Number |<br>|---------|--------|<br>| France  | 0      |<br>| Spain   | 1      |<br>| Germany | 2      |</pre><p>At first, this seems like an easy solution.</p><p>But it introduces a hidden problem.</p><p>Some Machine Learning algorithms might interpret these numbers as having a meaningful order.</p><p>For example:</p><p><strong>Germany (2) &gt; Spain (1) &gt; France (0)</strong></p><p>The algorithm may also treat the numerical distance between countries as meaningful.</p><p>But does that make sense?</p><p>Of course not!</p><p>Germany is not mathematically greater than Spain, and Spain is not greater than France.</p><p>They are just different categories.</p><p>Assigning arbitrary numbers to unordered categories can create misleading relationships.</p><p>This is why we often use <strong>One-Hot Encoding</strong> instead.</p><h3>What Is One-Hot Encoding?</h3><p>One-Hot Encoding is a technique that converts categorical values into separate binary columns containing 0s and 1s.</p><p>Instead of assigning each country a single number, we create one column for each unique country.</p><p>Let’s understand this step by step.</p><h3>Step 1: Look at the Original Data</h3><p>Suppose our Country column contains:</p><pre>| Country |<br>|---------|<br>| France  |<br>| Spain   |<br>| Germany |<br>| France  |</pre><p>We have three unique countries:</p><ul><li>France</li><li>Spain</li><li>Germany</li></ul><p>So, One-Hot Encoding creates three new columns.</p><h3>Step 2: Create a Column for Each Country</h3><p>Our new columns are:</p><p><strong>France | Spain | Germany</strong></p><p>Notice that these are column names, not the actual data values.</p><h3>Step 3: Fill the Columns with 0 and 1</h3><p>Now, let’s convert each row.</p><p>For a customer from France:</p><p>FranceSpainGermany100</p><p>This means:</p><p>✅ France → Yes (1)</p><p>❌ Spain → No (0)</p><p>❌ Germany → No (0)</p><p>For a customer from Spain:</p><pre>| France | Spain | Germany |<br>|--------|-------|---------|<br>| 0      | 1     | 0       |</pre><p>And for a customer from Germany:</p><pre>| France | Spain | Germany |<br>|--------|-------|---------|<br>| 0      | 0     | 1       |</pre><p>Now, putting everything together:</p><p><strong>Before One-Hot Encoding</strong></p><pre>| Country |<br>|---------|<br>| France  |<br>| Spain   |<br>| Germany |<br>| France  |</pre><p><strong>After One-Hot Encoding</strong></p><pre>| France | Spain | Germany |<br>|--------|-------|---------|<br>| 1      | 0     | 0       |<br>| 0      | 1     | 0       |<br>| 0      | 0     | 1       |<br>| 1      | 0     | 0       |</pre><p>That’s it! 🎉</p><p>Each row still represents the same customer, but the country is now represented using numbers.</p><p>The value 1 means the customer belongs to that country, while 0 means they don&#39;t.</p><h3>💡 Why Is It Called One-Hot Encoding?</h3><p>Imagine three light bulbs representing three countries.</p><p><strong>France | Spain | Germany</strong></p><p>When the customer is from France:</p><p>💡 ⚫ ⚫ → 1 0 0</p><p>When the customer is from Spain:</p><p>⚫ 💡 ⚫ → 0 1 0</p><p>When the customer is from Germany:</p><p>⚫ ⚫ 💡 → 0 0 1</p><p>Only one light bulb is ON at a time.</p><p>Similarly, each encoded row has exactly one 1, while the remaining values are 0.</p><p>That’s why it’s called <strong>One-Hot Encoding</strong>.</p><h3>Implementing One-Hot Encoding Using Python</h3><p>Now that we understand the concept, let’s implement it using Python and Scikit-learn.</p><p>We will use two important classes:</p><p><strong>1. ColumnTransformer</strong></p><p>This allows us to select specific columns for transformation.</p><p><strong>2. OneHotEncoder</strong></p><p>This performs the actual One-Hot Encoding.</p><p>Think of them as working together:</p><ul><li>ColumnTransformer decides <strong>which column</strong> to transform.</li><li>OneHotEncoder decides <strong>how to transform</strong> it.</li></ul><h3>Step 1: Prepare the Dataset</h3><p>Let’s start with our customer dataset.</p><pre>import pandas as pd</pre><pre>data = pd.DataFrame({<br>    &quot;Country&quot;: [&quot;France&quot;, &quot;Spain&quot;, &quot;Germany&quot;, &quot;France&quot;],<br>    &quot;Age&quot;: [44, 27, 30, 35],<br>    &quot;Salary&quot;: [72000, 48000, 54000, 58000],<br>    &quot;Purchased&quot;: [&quot;No&quot;, &quot;Yes&quot;, &quot;Yes&quot;, &quot;No&quot;]<br>})</pre><pre>X = data[[&quot;Country&quot;, &quot;Age&quot;, &quot;Salary&quot;]]<br>y = data[&quot;Purchased&quot;]</pre><p>Here:</p><ul><li>X contains the input features: Country, Age, and Salary.</li><li>y contains the target variable: Purchased.</li></ul><p>Our goal is to encode only the Country column while keeping Age and Salary unchanged.</p><h3>Step 2: Import the Required Classes</h3><pre>from sklearn.compose import ColumnTransformer<br>from sklearn.preprocessing import OneHotEncoder</pre><h3>Step 3: Create the ColumnTransformer</h3><pre>ct = ColumnTransformer(<br>    transformers=[<br>        (&quot;encoder&quot;, OneHotEncoder(), [0])<br>    ],<br>    remainder=&quot;passthrough&quot;<br>)</pre><p>Let’s break down this code.</p><p><strong>&quot;encoder&quot;</strong></p><p>This is simply a name for our transformation. We can use almost any name here.</p><p><strong>OneHotEncoder()</strong></p><p>This tells Python to apply One-Hot Encoding.</p><p><strong>[0]</strong></p><p>This tells Python to encode the column at index 0.</p><p>Remember that Python starts counting from 0.</p><pre>| Index | Column  |<br>|-------|---------|<br>| 0     | Country |<br>| 1     | Age     |<br>| 2     | Salary  |</pre><p>So [0] represents the Country column.</p><p><strong>remainder=&quot;passthrough&quot;</strong></p><p>This tells Python to keep the remaining columns unchanged.</p><p>Without passthrough, the Age and Salary columns would disappear from the transformed output.</p><p>With passthrough, those columns remain available.</p><h3>Step 4: Apply the Transformation</h3><pre>X_encoded = ct.fit_transform(X)</pre><p>Here, fit_transform() performs two operations.</p><p><strong>fit() — Learn the Categories</strong></p><p>It identifies all the unique categories in the Country column:</p><p>France, Germany, Spain.</p><p><strong>transform() — Encode the Categories</strong></p><p>It converts those country names into separate binary columns.</p><p>Instead of writing:</p><pre>ct.fit(X)<br>X_encoded = ct.transform(X)</pre><p>We can use:</p><pre>X_encoded = ct.fit_transform(X)</pre><p>Both steps happen together.</p><h3>Step 5: Convert the Result into a NumPy Array</h3><pre>import numpy as np</pre><pre>X_encoded = np.asarray(X_encoded)</pre><p>This makes sure our transformed data is stored as a NumPy array.</p><p>Note that Scikit-learn already supports NumPy arrays and other compatible data formats. This conversion is useful when we specifically want an array, but it isn’t always necessary.</p><h3>Step 6: View the Final Result</h3><p>Our original dataset looked like this:</p><pre>| Country | Age | Salary |<br>|---------|-----|--------|<br>| France  | 44  | 72000  |<br>| Spain   | 27  | 48000  |<br>| Germany | 30  | 54000  |<br>| France  | 35  | 58000  |</pre><p>After One-Hot Encoding:</p><pre>| France | Germany | Spain | Age | Salary |<br>|--------|---------|-------|-----|--------|<br>| 1      | 0       | 0     | 44  | 72000  |<br>| 0      | 0       | 1     | 27  | 48000  |<br>| 0      | 1       | 0     | 30  | 54000  |<br>| 1      | 0       | 0     | 35  | 58000  |</pre><p>Notice what happened!</p><p>✅ Country names have been replaced by numerical columns.</p><p>✅ Age and Salary remain unchanged.</p><p>✅ No artificial numerical ranking exists between countries.</p><p>Our categorical feature has now been successfully encoded!</p><h3>Complete Python Code</h3><p>Here’s the complete example in one place:</p><pre>import pandas as pd<br>import numpy as np</pre><pre>from sklearn.compose import ColumnTransformer<br>from sklearn.preprocessing import OneHotEncoder</pre><pre># Create dataset<br>data = pd.DataFrame({<br>    &quot;Country&quot;: [&quot;France&quot;, &quot;Spain&quot;, &quot;Germany&quot;, &quot;France&quot;],<br>    &quot;Age&quot;: [44, 27, 30, 35],<br>    &quot;Salary&quot;: [72000, 48000, 54000, 58000],<br>    &quot;Purchased&quot;: [&quot;No&quot;, &quot;Yes&quot;, &quot;Yes&quot;, &quot;No&quot;]<br>})</pre><pre># Separate features and target<br>X = data[[&quot;Country&quot;, &quot;Age&quot;, &quot;Salary&quot;]]<br>y = data[&quot;Purchased&quot;]</pre><pre># Create the transformer<br>ct = ColumnTransformer(<br>    transformers=[<br>        (&quot;encoder&quot;, OneHotEncoder(), [0])<br>    ],<br>    remainder=&quot;passthrough&quot;<br>)</pre><pre># Apply One-Hot Encoding<br>X_encoded = np.asarray(ct.fit_transform(X))</pre><pre># Display the result<br>print(X_encoded)</pre><p>This code demonstrates how One-Hot Encoding works on a small example dataset.</p><p>In a real Machine Learning project, we should split our data into training and testing sets first, then fit the encoder only on the training data.</p><h3>What About the Purchased Column?</h3><p>Our dataset also contains the Purchased column.</p><pre>| Purchased |<br>|-----------|<br>| No        |<br>| Yes       |<br>| Yes       |<br>| No        |</pre><p>This is the target variable (y), not an input feature.</p><p>Since it contains only two possible categories, we can represent them as:</p><pre>| Purchased | Encoded |<br>|-----------|---------|<br>| No        | 0       |<br>| Yes       | 1       |</pre><p>We can easily do this using Python:</p><pre>from sklearn.preprocessing import LabelEncoder<br>le = LabelEncoder()<br>y = le.fit_transform(y)</pre><p>Now our target becomes:</p><pre>0<br>1<br>1<br>0</pre><p>This is perfectly suitable for binary classification.</p><p>Here, 0 and 1 represent two different outcomes, rather than a numerical ranking between countries.</p><h3>🤔 Is One-Hot Encoding Always the Best Choice?</h3><p>Not always!</p><p>We should first understand whether our categorical data has a meaningful order.</p><p>There are two common types of categorical data.</p><p><strong>1. Nominal Data</strong></p><p>These categories have no natural order.</p><p>Examples:</p><ul><li>Country: France, Spain, Germany</li><li>Color: Red, Blue, Green</li><li>Payment Method: Cash, Card, Online</li></ul><p>One-Hot Encoding is often a good choice for these features.</p><p><strong>2. Ordinal Data</strong></p><p>These categories have a meaningful order.</p><p>For example:</p><ul><li>Low</li><li>Medium</li><li>High</li></ul><p>In this case, the order matters.</p><p>We might encode them as:</p><p>Low → 0</p><p>Medium → 1</p><p>High → 2</p><p>This is called Ordinal Encoding.</p><p>Another consideration is the number of categories. If a dataset contains thousands of unique categories, One-Hot Encoding may create thousands of columns, which can make the model more complex.</p><p>So, always choose an encoding method based on the type of data you’re working with.</p><h3>⭐ Key Takeaways</h3><p>Let’s quickly review what we’ve learned.</p><p>✅ Categorical data represents groups or labels rather than numerical measurements.</p><p>✅ Most traditional Machine Learning algorithms require categorical features to be represented numerically.</p><p>✅ Assigning arbitrary numbers to unordered categories can introduce misleading relationships.</p><p>✅ One-Hot Encoding creates a separate binary column for each category.</p><p>✅ The value 1 indicates that a row belongs to a particular category, while 0 indicates that it does not.</p><p>✅ ColumnTransformer helps us select which columns to transform.</p><p>✅ OneHotEncoder() performs the actual encoding.</p><p>✅ remainder=&quot;passthrough&quot; keeps the remaining columns unchanged.</p><p>✅ fit_transform() learns the categories and applies the encoding.</p><p>✅ Binary target variables like Yes/No can be represented as 0 and 1.</p><h3>🚀 Final Thoughts</h3><p>Categorical Data Encoding is an essential part of Machine Learning preprocessing.</p><p>Real-world datasets often contain text values such as country names, colors, payment methods, and other labels.</p><p>By encoding these categories correctly, we help Machine Learning algorithms work with information that they couldn’t directly process before.</p><p>And the most important idea to remember is simple:</p><p><strong>One-Hot Encoding gives each category its own column and uses 0s and 1s to show which category is present.</strong></p><p>Once you understand this concept, you’re one step closer to preparing real-world datasets for Machine Learning.</p><p>Happy Learning! 🚀</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d503b37a55a4" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Handling Missing Data in Machine Learning: A Beginner’s Guide with Python]]></title>
            <link>https://medium.com/@sunethsilva/handling-missing-data-in-machine-learning-a-beginners-guide-with-python-11f2bbdcb122?source=rss-c1f157a43ba8------2</link>
            <guid isPermaLink="false">https://medium.com/p/11f2bbdcb122</guid>
            <category><![CDATA[scikit-learn]]></category>
            <category><![CDATA[data-preprocessing]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Suneth Nawoda De Silva]]></dc:creator>
            <pubDate>Sat, 10 Oct 2026 11:51:23 GMT</pubDate>
            <atom:updated>2026-10-10T11:51:23.794Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*o7wXutrSn0us2nEJg_GkcA.png" /></figure><p><em>Learn how to identify, remove, and replace missing values using Python and Scikit-learn, with simple examples and step-by-step explanations.</em></p><p>Imagine you’re building a Machine Learning model to predict students’ exam performance. You’ve collected information about their study hours, attendance, and previous exam scores.</p><p>Everything looks good until you notice something strange: <strong>some values are missing!</strong> 😕</p><p>Should you delete those records? Replace the missing values? Or simply ignore them?</p><p>This is one of the most common challenges you’ll face when working with real-world datasets.</p><p>In this article, we’ll explore what missing data is, why it matters, and how to handle it using Python’s <strong>Scikit-learn</strong> library.</p><p>Let’s get started!</p><h3>1. What Is Missing Data? 🤔</h3><p>In the real world, datasets are rarely perfect.</p><p>Some information might be missing because:</p><ul><li>A student forgot to enter their study hours.</li><li>A school failed to record attendance.</li><li>A system experienced an error while collecting data.</li><li>Some information was simply unavailable.</li></ul><p>Let’s look at a small dataset containing student information.</p><pre>| Student | Study Hours | Attendance (%) | Previous Score |<br>|---------|-------------|----------------|----------------|<br>| Alice   | 4           | 80             | 65             |<br>| Bob     | 6           | 90             | 75             |<br>| Charlie | Missing     | 85             | 70             |<br>| David   | 8           | Missing        | 85             |<br>| Emma    | 2           | 75             | 60             |</pre><p>Notice that:</p><ul><li>Charlie’s study hours are missing.</li><li>David’s attendance percentage is missing.</li></ul><p>These empty values are called <strong>missing values</strong>.</p><p>In Python, missing numerical values are commonly represented using NaN, which stands for <em>Not a Number</em>.</p><p>For example:</p><pre>import numpy as np</pre><pre>study_hours = [4, 6, np.nan, 8, 2]</pre><p>Here, np.nan represents Charlie&#39;s missing study hours.</p><h3>2. Why Is Missing Data a Problem? ⚠️</h3><p>Machine Learning models learn patterns from data.</p><p>For example, our model might discover that students who study more hours and attend more classes tend to achieve higher exam scores.</p><p>But what happens when some of that information is missing?</p><p>Missing data can create several problems:</p><ul><li><strong>Training errors:</strong> Many Machine Learning algorithms cannot directly handle missing values.</li><li><strong>Reduced accuracy:</strong> Missing information may affect the model’s ability to learn useful patterns.</li><li><strong>Misleading results:</strong> Incorrectly handling missing values can introduce bias into the dataset.</li></ul><p>That’s why handling missing data is an important part of <strong>data preprocessing</strong>.</p><p>Data preprocessing simply means preparing and cleaning raw data before using it to train a Machine Learning model.</p><h3>3. How Can We Handle Missing Data? 🛠️</h3><p>There are several techniques for handling missing values.</p><p>Let’s explore two common approaches.</p><h3>Method 1: Remove Rows with Missing Values</h3><p>The simplest approach is to remove records containing missing values.</p><p>Remember our original dataset?</p><pre>| Student | Study Hours | Attendance (%) |<br>|---------|-------------|----------------|<br>| Alice   | 4           | 80             |<br>| Bob     | 6           | 90             |<br>| Charlie | ❌          | 85             |<br>| David   | 8           | ❌             |<br>| Emma    | 2           | 75             |</pre><p>If we remove the rows containing missing values, we’ll get:</p><pre>| Student | Study Hours | Attendance (%) |<br>|---------|-------------|----------------|<br>| Alice   | 4           | 80             |<br>| Bob     | 6           | 90             |<br>| Emma    | 2           | 75             |</pre><p>Problem solved, right?</p><p>Well, not always.</p><p>We’ve removed the missing values, but we’ve also lost two student records.</p><p>In this example, that’s <strong>40% of our dataset!</strong></p><p>Removing this much information could negatively affect our model.</p><p><strong>When is removing rows a good idea?</strong></p><p>When only a small percentage of records are missing, the dataset is large enough, and removing those records is unlikely to introduce bias.</p><p><strong>When should we avoid it?</strong></p><p>When many records contain missing values or when the removed records could contain useful information.</p><p>Fortunately, there’s another approach.</p><h3>Method 2: Replace Missing Values (Imputation) ⭐</h3><p>Instead of deleting entire rows, what if we could fill the missing values with reasonable estimates?</p><p>This process is called <strong>Imputation</strong>.</p><p>One of the simplest imputation techniques is replacing missing numerical values with the average of the available values.</p><p>Let’s understand this with an example.</p><p>Suppose we have the following study hours:</p><pre>| Student | Study Hours |<br>|---------|-------------|<br>| Alice   | 4           |<br>| Bob     | 6           |<br>| Charlie | ❌          |<br>| David   | 8           |<br>| Emma    | 2           |</pre><p>We can calculate the average using the available values:</p><p><strong>Mean = (4 + 6 + 8 + 2) / 4 = 5</strong></p><p>Now, we replace Charlie’s missing value with 5.</p><pre>| Student | Study Hours |<br>|---------|-------------|<br>| Alice   | 4           |<br>| Bob     | 6           |<br>| Charlie | 5           |<br>| David   | 8           |<br>| Emma    | 2           |</pre><p>Great! 🎉</p><p>We’ve handled the missing value without deleting any student records.</p><p>However, there’s something important to understand.</p><p><strong>The value 5 is an estimate, not Charlie’s actual study hours.</strong></p><p>Imputation helps us retain data, but it doesn’t magically recover the original missing information.</p><h3>4. Different Ways to Replace Missing Values</h3><p>Using the mean is just one option.</p><p>Depending on the dataset, other strategies might be more appropriate.</p><pre>| Strategy      | What It Does                                  | Best Used For                           |<br>|---------------|-----------------------------------------------|-----------------------------------------|<br>| Mean          | Replaces missing values with the average      | Numerical data without extreme outliers |<br>| Median        | Replaces missing values with the middle value | Numerical data containing outliers      |<br>| Most Frequent | Uses the most common value                    | Categorical data                        |<br>| Constant      | Uses a specified value                        | When a fixed replacement makes sense    |</pre><p>Let’s understand two additional strategies.</p><p><strong>Median</strong></p><p>Imagine the following student study hours:</p><p>2, 3, 4, 5, 20</p><p>The mean is:</p><p>(2 + 3 + 4 + 5 + 20) / 5 = 6.8</p><p>But notice that one student studied for 20 hours, much longer than everyone else!</p><p>This extreme value pulls the average upward.</p><p>The median, however, is 4.</p><p>In this situation, the median may be a better choice because it’s less affected by extreme values.</p><p><strong>Most Frequent</strong></p><p>Now imagine a dataset containing students’ preferred learning methods:</p><p>Online, Classroom, Online, Missing, Online</p><p>The most common value is Online.</p><p>Using the most frequent strategy, we can replace the missing value with Online.</p><p>The right strategy depends on the type and distribution of your data.</p><h3>5. Meet Scikit-learn</h3><p>Now that we understand missing data, let’s handle it using Python.</p><p>For this, we’ll use <strong>Scikit-learn</strong>, one of the most widely used libraries for Machine Learning.</p><p>It provides tools for:</p><ul><li>Data preprocessing</li><li>Machine Learning algorithms</li><li>Model training and evaluation</li><li>Feature selection and transformation</li></ul><p>One particularly useful tool is called SimpleImputer.</p><h3>What Is SimpleImputer?</h3><p>SimpleImputer is a class in Scikit-learn that helps us automatically replace missing values using a selected strategy.</p><p>Instead of manually calculating averages and replacing missing values one by one, we can let Python handle the process.</p><p>First, import the necessary libraries:</p><pre>import numpy as np<br>from sklearn.impute import SimpleImputer</pre><p>Here:</p><ul><li>numpy helps us work with numerical data and missing values.</li><li>SimpleImputer handles missing value replacement.</li></ul><p>Let’s see it in action.</p><h3>6. Handling Missing Data with SimpleImputer: Step by Step</h3><p>We’ll use the numerical features from our student dataset.</p><h3>Step 1: Create the Dataset</h3><pre>import numpy as np<br>from sklearn.impute import SimpleImputer</pre><pre>X = np.array([<br>    [4, 80],<br>    [6, 90],<br>    [np.nan, 85],<br>    [8, np.nan],<br>    [2, 75]<br>])</pre><p>Our dataset contains two columns:</p><ul><li>Column 0: Study Hours</li><li>Column 1: Attendance (%)</li></ul><p>When we print X, the missing values appear as nan.</p><pre>print(X)</pre><p>Output:</p><pre>[[ 4. 80.]<br> [ 6. 90.]<br> [nan 85.]<br> [ 8. nan]<br> [ 2. 75.]]</pre><p>Now, let’s replace those missing values.</p><h3>Step 2: Create the Imputer</h3><pre>imputer = SimpleImputer(<br>    missing_values=np.nan,<br>    strategy=&quot;mean&quot;<br>)</pre><p>Let’s break down this code.</p><p><strong>missing_values=np.nan</strong></p><p>This tells the imputer which values should be considered missing.</p><p>In our dataset, missing values are represented by np.nan.</p><p><strong>strategy=&quot;mean&quot;</strong></p><p>This tells the imputer to replace missing values with the average of each column.</p><p>So our imputer will calculate:</p><ul><li>The average study hours.</li><li>The average attendance percentage.</li></ul><p>It will then use those averages to fill the missing values.</p><p>But there’s one important thing to remember.</p><p><strong>Creating an imputer doesn’t change our dataset yet.</strong></p><p>We’ve only configured the tool.</p><h3>Step 3: Fit the Imputer</h3><pre>imputer.fit(X)</pre><p>What does fit() do?</p><p>Think of it as the <strong>learning step</strong>.</p><p>The imputer looks at the dataset and calculates the necessary statistics.</p><p>For our example:</p><p><strong>Study Hours</strong></p><p>(4 + 6 + 8 + 2) / 4 = 5</p><p><strong>Attendance</strong></p><p>(80 + 90 + 85 + 75) / 4 = 82.5</p><p>The imputer now remembers:</p><ul><li>Average Study Hours = 5</li><li>Average Attendance = 82.5</li></ul><p>However, fit() doesn&#39;t replace anything yet.</p><p>It only learns the values needed for replacement.</p><h3>Step 4: Transform the Dataset</h3><p>Now that the imputer has calculated the averages, we can use them to fill the missing values.</p><pre>imputer.transform(X)</pre><p>The transform() method:</p><ol><li>Finds the missing values.</li><li>Uses the averages learned during fit().</li><li>Returns a transformed dataset with the missing values replaced.</li></ol><p>But be careful!</p><p><strong>transform() doesn&#39;t automatically update our original variable </strong><strong>X.</strong></p><p>We need to save the returned result.</p><h3>Step 5: Save the Updated Dataset</h3><pre>X = imputer.transform(X)</pre><p>Now, the transformed dataset is stored in X.</p><p>Let’s print it:</p><pre>print(X)</pre><p>Output:</p><pre>[[ 4.  80. ]<br> [ 6.  90. ]<br> [ 5.  85. ]<br> [ 8.  82.5]<br> [ 2.  75. ]]</pre><p>🎉 Success!</p><p>Notice what happened:</p><ul><li>Charlie’s missing study hours were replaced with 5.</li><li>David’s missing attendance was replaced with 82.5.</li></ul><p>We’ve handled both missing values without removing any rows.</p><h3>7. A Shortcut: fit_transform() 🚀</h3><p>So far, we’ve used two separate methods:</p><pre>imputer.fit(X)<br>X = imputer.transform(X)</pre><p>But Scikit-learn provides a shortcut!</p><pre>X = imputer.fit_transform(X)</pre><p>This method combines both operations.</p><p>First, it learns the required statistics using fit().</p><p>Then, it replaces the missing values using transform().</p><p>This is particularly useful when preprocessing training data.</p><h3>Complete Python Code</h3><p>Here’s the complete example in just a few lines:</p><pre>import numpy as np<br>from sklearn.impute import SimpleImputer</pre><pre># Student dataset: Study Hours, Attendance<br>X = np.array([<br>    [4, 80],<br>    [6, 90],<br>    [np.nan, 85],<br>    [8, np.nan],<br>    [2, 75]<br>])</pre><pre># Create an imputer<br>imputer = SimpleImputer(<br>    missing_values=np.nan,<br>    strategy=&quot;mean&quot;<br>)</pre><pre># Fit and replace missing values<br>X = imputer.fit_transform(X)</pre><pre># Display the cleaned dataset<br>print(X)</pre><p>That’s it!</p><p>With just a few lines of Python, we can automatically handle missing numerical values.</p><h3>8. An Important Detail: Training Data vs. Test Data</h3><p>There’s one more concept worth understanding.</p><p>When building a Machine Learning model, we usually divide our dataset into two parts:</p><ul><li><strong>Training data:</strong> Used to teach the model.</li><li><strong>Test data:</strong> Used to evaluate the model on unseen examples.</li></ul><p>When handling missing values, we should calculate replacement values using <strong>only the training data</strong>.</p><p>Why?</p><p>Because calculating averages using the test data can reveal information that wouldn’t normally be available during training.</p><p>This problem is known as <strong>data leakage</strong>.</p><p>For example:</p><pre># Learn averages from training data<br>imputer.fit(X_train)</pre><pre># Clean training data<br>X_train = imputer.transform(X_train)</pre><pre># Clean test data using the SAME averages<br>X_test = imputer.transform(X_test)</pre><p>Notice that we don’t call fit() on the test data.</p><p>We reuse the statistics learned from the training dataset.</p><p>This helps us evaluate our Machine Learning model more fairly.</p><p>In larger projects, Scikit-learn’s Pipeline can help automate this process safely.</p><h3>9. Can We Always Replace Missing Values with the Mean?</h3><p>Not necessarily.</p><p>Although mean imputation is simple and useful, it’s not always the best approach.</p><p>For example, imagine that students who rarely attend classes are also less likely to have their attendance recorded.</p><p>Replacing all those missing values with the average could hide an important pattern.</p><p>Similarly, replacing many missing values with one average reduces the natural variation in a feature.</p><p>Before choosing a strategy, it’s useful to ask:</p><ul><li>How many values are missing?</li><li>Why might those values be missing?</li><li>Is the feature numerical or categorical?</li><li>Does the dataset contain extreme values?</li><li>Which approach gives better results when properly evaluated?</li></ul><p>Sometimes, using the median, adding a missing-value indicator, or using more advanced imputation techniques can produce better results.</p><p>There is no single strategy that works perfectly for every dataset.</p><h3>10. Final Thoughts 🎯</h3><p>Missing data is a common challenge in almost every real-world Machine Learning project.</p><p>The important thing isn’t simply removing empty values. It’s choosing a method that preserves useful information and helps your model learn reliable patterns.</p><p>Here’s a quick recap of what we’ve learned:</p><ul><li>Missing values are commonly represented using NaN in Python.</li><li>We can handle missing values by removing records or replacing missing entries.</li><li>Replacing missing values with estimated values is called imputation.</li><li>Scikit-learn’s SimpleImputer makes imputation easier.</li><li>fit() learns the statistics needed for replacement.</li><li>transform() uses those statistics to replace missing values.</li><li>fit_transform() combines both operations into one step.</li></ul><p>We should learn imputation statistics from training data, not test data, to avoid data leakage.</p><p>Most importantly, remember that <strong>cleaner data doesn’t automatically mean better data</strong>. The goal is to handle missing information carefully while preserving the patterns that matter.</p><p>If you’re just getting started with Machine Learning, understanding missing data is an excellent first step toward building more reliable models.</p><p>Happy learning and happy coding!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=11f2bbdcb122" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[From a CSV File to X and y: Understanding Your First Machine Learning Dataset]]></title>
            <link>https://medium.com/@sunethsilva/from-a-csv-file-to-x-and-y-understanding-your-first-machine-learning-dataset-1ed163ee098e?source=rss-c1f157a43ba8------2</link>
            <guid isPermaLink="false">https://medium.com/p/1ed163ee098e</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[putin]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[split]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Suneth Nawoda De Silva]]></dc:creator>
            <pubDate>Mon, 21 Sep 2026 10:12:26 GMT</pubDate>
            <atom:updated>2026-09-21T10:12:26.790Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jpgJtkg-ynOTBW1YMyfHyw.png" /></figure><p>When you begin learning machine learning, you’ll often come across these two lines of code:</p><pre>X = dataset.iloc[:, :-1].values<br>y = dataset.iloc[:, -1].values</pre><p>They may look complicated at first. But they do one simple job: <strong>separate the information a model will use from the answer it will learn to predict.</strong></p><p>Let’s understand each part using a small example.</p><h3>1. What is a DataFrame?</h3><p>Suppose we have a CSV file named Data.csv:</p><pre>| Country | Age | Salary | Purchased |<br>|---------|-----|--------|-----------|<br>| France  | 44  | 72000  | No        |<br>| Spain   | 27  | 48000  | Yes       |<br>| Germany | 30  | 54000  | No        |</pre><p>We can read it with Pandas:</p><pre>import pandas as pd</pre><pre>dataset = pd.read_csv(&quot;Data.csv&quot;)</pre><p>Pandas stores the data in a <strong>DataFrame</strong>. Think of a DataFrame as a spreadsheet inside Python:</p><ul><li>Each <strong>row</strong> represents one customer.</li><li>Each <strong>column</strong> contains one type of information.</li><li>We can use Python to select and work with the data.</li></ul><p>The variable dataset now refers to the entire table.</p><h3>2. What are features?</h3><p>Imagine a company wants to predict whether a new customer will purchase its product. What information could help make that prediction?</p><p>In our dataset, the model can look at the customer’s:</p><ul><li>Country</li><li>Age</li><li>Salary</li></ul><p>These columns are called <strong>features</strong>. Features are the <strong>inputs</strong> we provide to the model.</p><p>For one customer, the input might be:</p><pre>Country = France<br>Age = 44<br>Salary = 72000</pre><p>The model uses examples like this to learn patterns in the data.</p><h3>3. What is the target?</h3><p>The <strong>target</strong> is the answer we want the model to predict.</p><p>In our example, the target is Purchased. Its value is either Yes or No.</p><pre>| Features: information we have | Target: answer we want |<br>|-------------------------------|------------------------|<br>| Country, Age, Salary          | Purchased              |</pre><p>For the first customer, the features are France, 44, and 72000. The known answer is No.</p><p>During training, we show the model both the customer information and the known answer. After training, we can give it information about a <em>new</em> customer and ask it to predict whether that customer will purchase.</p><h3>4. What are X and y?</h3><p>In machine learning, we usually store the features in a variable called X and the target in a variable called y.</p><pre>X = the input information<br>y = the answer to predict</pre><p>For our dataset, X should contain:</p><pre>| Country | Age | Salary |<br>|---------|-----|--------|<br>| France  | 44  | 72000  |<br>| Spain   | 27  | 48000  |<br>| Germany | 30  | 54000  |</pre><p>And y should contain:</p><pre>| Purchased |<br>|-----------|<br>| No        |<br>| Yes       |<br>| No        |</pre><p>Notice that their rows still line up. The first row of X belongs with the first answer in y, and so on.</p><p>The names come from the mathematical convention of using (x) for an input and (y) for an output. You <em>can</em> use other names, but X and y are widely used in machine learning examples.</p><h3>5. Why do we separate them?</h3><p>A supervised machine learning model learns from <strong>examples and their correct answers</strong>.</p><p>Here, X provides the examples: each customer’s country, age, and salary. y provides the correct answers: whether each customer purchased.</p><p>The model’s task is to learn a relationship between the two:</p><pre>Customer information (X) → Model → Predicted purchase (y)</pre><p>That is why we separate the table into inputs and answers before training.</p><h3>6. How do we create X and y?</h3><p>Because Purchased is the <strong>last column</strong> in our table, we can write:</p><pre>X = dataset.iloc[:, :-1].values<br>y = dataset.iloc[:, -1].values</pre><p>Let’s read the code slowly.</p><h3>dataset.iloc[rows, columns]</h3><p>iloc selects parts of a DataFrame using their <strong>positions</strong>. The part before the comma selects rows; the part after the comma selects columns.</p><p>Python counts positions starting from zero:</p><pre>| Column position  | Column name |<br>|------------------|-------------|<br>| 0                | Country     |<br>| 1                | Age         |<br>| 2                | Salary      |<br>| 3                | Purchased   |</pre><p>A colon (:) on its own means <strong>select everything</strong>.</p><h3>Creating X</h3><pre>X = dataset.iloc[:, :-1].values</pre><p>Read this as:</p><ol><li>dataset: start with the full table.</li><li>iloc[:, ...]: select <strong>all rows</strong>.</li><li>:-1: select every column <strong>before the last one</strong>.</li><li>.values: turn the selected data into a NumPy array.</li><li>X =: store the result in X.</li></ol><p>Why does :-1 leave out the last column? Python slices <strong>exclude their ending position</strong>. Here, -1 means the last column, so :-1 stops just before Purchased.</p><p>The result is Country, Age, and Salary.</p><h3>Creating y</h3><pre>y = dataset.iloc[:, -1].values</pre><p>This is almost the same, but the column part is now -1 instead of :-1.</p><ul><li>:-1 means <strong>everything before the last column</strong>.</li><li>-1 means <strong>the last column itself</strong>.</li></ul><p>So y contains only Purchased.</p><h3>7. Do we always need .values?</h3><p>No. .values converts the selected Pandas data into NumPy arrays, but many machine learning tools can also work directly with Pandas data.</p><p>For example, you could write:</p><pre>X = dataset.iloc[:, :-1]<br>y = dataset.iloc[:, -1]</pre><p>Here, X remains a DataFrame and y is a Pandas Series. Both versions express the same choice of features and target.</p><h3>8. A clearer option: select the target by name</h3><p>Our iloc example assumes Purchased is the last column. If someone moves it to another position, that code will select the wrong data.</p><p>When you know the target column’s name, this is often easier to read:</p><pre>X = dataset.drop(columns=&quot;Purchased&quot;)<br>y = dataset[&quot;Purchased&quot;]</pre><p>It means: put every column <em>except</em> Purchased in X, and put Purchased in y.</p><h3>Putting it all together</h3><pre>import pandas as pd</pre><pre>dataset = pd.read_csv(&quot;Data.csv&quot;)</pre><pre>X = dataset.drop(columns=&quot;Purchased&quot;)<br>y = dataset[&quot;Purchased&quot;]</pre><pre>print(X)<br>print(y)</pre><p>The most important idea is simple: <strong>X contains the information available for making a prediction, and </strong><strong>y contains the answer we want to predict.</strong></p><p>Once that distinction is clear, code such as dataset.iloc[:, :-1] becomes much less mysterious. It is simply one way to tell Python which columns belong in X and which column belongs in y.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1ed163ee098e" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Standard Deviation Made Simple: What Does It Actually Mean?]]></title>
            <link>https://medium.com/@sunethsilva/standard-deviation-made-simple-what-does-it-actually-mean-8a669d9cbae7?source=rss-c1f157a43ba8------2</link>
            <guid isPermaLink="false">https://medium.com/p/8a669d9cbae7</guid>
            <category><![CDATA[standard-deviation]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[statistics]]></category>
            <category><![CDATA[feature-scaling]]></category>
            <dc:creator><![CDATA[Suneth Nawoda De Silva]]></dc:creator>
            <pubDate>Sat, 08 Aug 2026 22:47:31 GMT</pubDate>
            <atom:updated>2026-08-08T22:47:31.386Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*sxywUVaSq6wBKauMMHtvBw.png" /></figure><p>If you are learning Machine Learning or Statistics, you have probably seen this sentence:</p><blockquote><strong><em>“Standard deviation tells us how spread out the data is from the average.”</em></strong></blockquote><p>Then you may hear:</p><p><strong>Small standard deviation → values are close to the average.</strong><br><strong>Large standard deviation → values are far from the average.</strong></p><p>But there is an important question:</p><p><strong>How do we actually get the standard deviation?</strong></p><p>We can’t just tell a computer:</p><pre>Standard Deviation = Small</pre><p>or</p><pre>Standard Deviation = Large</pre><p>Standard deviation is actually a <strong>number that we calculate from our data</strong>.</p><p>Let’s understand it step by step.</p><h3>First, What Does Standard Deviation Tell Us?</h3><p>Imagine five food deliveries took:</p><pre>8, 9, 10, 11, 12 minutes</pre><p>The average delivery time is:</p><pre>10 minutes</pre><p>Look at the values.</p><pre>8   9   10   11   12<br>        ↑<br>      Average</pre><p>They are all quite close to 10.</p><p>So we can say:</p><p>📌The data doesn’t have much spread.</p><p>📌Therefore, its standard deviation should be relatively <strong>small</strong>.</p><p>But we still need an actual number.</p><p>Let’s calculate it.</p><h3>Calculating Standard Deviation Step by Step</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/778/1*f592c5S2uzuJSDbe9WkvTw.png" /><figcaption>Source: <em>Standard Deviation Formula and Uses, vs. Variance</em>. <a href="http://www.investopedia.com./">www.investopedia.com.</a> <a href="https://www.investopedia.com/terms/s/standarddeviation.asp">https://www.investopedia.com/terms/s/standarddeviation.asp</a></figcaption></figure><p>Don’t worry about above formula, Let’s understand it simply.</p><p>We’ll use:</p><pre>8, 9, 10, 11, 12</pre><h3>Step 1️ — Find the Mean</h3><p>Mean simply means <strong>average</strong>.</p><p>Add all the values:</p><pre>8 + 9 + 10 + 11 + 12 = 50</pre><p>We have 5 values.</p><p>So:</p><pre>50 ÷ 5 = 10</pre><p>Our mean is:</p><p><strong>Mean = 10</strong></p><h3>Step 2️ — Find How Far Each Value Is From the Mean</h3><p>Now subtract the mean from every value.</p><pre>| Value | Value − Mean |<br>|-------|--------------|<br>| 8     | −2           |<br>| 9     | −1           |<br>| 10    | 0            |<br>| 11    | 1            |<br>| 12    | 2            |</pre><p>For example:</p><pre>8 - 10 = -2<br>9 - 10 = -1<br>10 - 10 = 0<br>11 - 10 = 1<br>12 - 10 = 2</pre><p>These numbers tell us where each value is compared with the average.</p><p>-2 → 2 below the average<br>0 → exactly at the average<br>+2 → 2 above the average</p><h3>🤔 But We Have a Small Problem</h3><p>If we add these differences:</p><pre>-2 + -1 + 0 + 1 + 2 = 0</pre><p>We get:</p><pre>0</pre><p>That’s not useful for measuring the spread.</p><p>The negative and positive numbers cancel each other out.</p><p>So we need another step.</p><h3>Step 3️ — Square Each Difference</h3><p>We square each difference:</p><pre>-2 × -2 = 4<br>-1 × -1 = 1<br> 0 ×  0 = 0<br> 1 ×  1 = 1<br> 2 ×  2 = 4</pre><p>Now we have:</p><pre>| Value | Difference | Squared Difference |<br>|-------|------------|--------------------|<br>| 8     | −2         | 4                  |<br>| 9     | −1         | 1                  |<br>| 10    | 0          | 0                  |<br>| 11    | 1          | 1                  |<br>| 12    | 2          | 4                  |</pre><p>Notice something?</p><p>There are no negative numbers anymore.</p><h3>Step 4️ — Find the Average of Those Squared Differences</h3><p>Add them:</p><pre>4 + 1 + 0 + 1 + 4 = 10</pre><p>Divide by the number of values:</p><pre>10 ÷ 5 = 2</pre><p>We get:</p><pre>2</pre><p>This number has a special name:</p><p><strong>Variance</strong></p><p>So:</p><pre>Variance = 2</pre><p>We’re almost finished!</p><h3>Step 5️ — Take the Square Root</h3><p>Remember that we <strong>squared</strong> our differences earlier?</p><p>Now we take the square root to bring the measurement back to the original unit.</p><pre>√2 ≈ 1.41</pre><p>And that’s our answer:</p><h3>Standard Deviation ≈ 1.41 minutes 🎉</h3><h3>🤔 What Does 1.41 Actually Mean?</h3><p>This is the most important part.</p><p>Our data was:</p><pre>8, 9, 10, 11, 12</pre><p>Mean:</p><pre>10</pre><p>Standard Deviation:</p><pre>1.41</pre><p>The standard deviation gives us an idea of the <strong>typical spread of the values around the mean</strong>.</p><p>So instead of saying:</p><p>❌ “The values are kind of close to 10.”</p><p>We now have a mathematical measurement:</p><p>✅ <strong>Standard Deviation ≈ 1.41 minutes</strong></p><h3>📈 What Would a Larger Standard Deviation Look Like?</h3><p>Now imagine another delivery company.</p><p>Their delivery times are:</p><pre>0, 5, 10, 15, 20</pre><p>The mean is still:</p><pre>10</pre><p>But look at the values:</p><pre>0        5        10        15        20<br>                  ↑<br>                Average</pre><p>They are much more spread out.</p><p>If we calculate the standard deviation, we get approximately:</p><pre>7.07</pre><p>Now compare the two:</p><pre>| Delivery Times   | Mean | Standard Deviation |<br>|------------------|------|--------------------|<br>| 8, 9, 10, 11, 12 | 10   | 1.41               |<br>| 0, 5, 10, 15, 20 | 10   | 7.07               |</pre><p>Both datasets have the <strong>same average</strong>.</p><p>But they are very different.</p><p>The first dataset stays close to 10.</p><p>The second dataset is much more spread out.</p><p>That’s exactly what standard deviation helps us see.</p><h3>So What Does “Small” or “Large” Mean?</h3><p>When someone says:</p><blockquote><em>“The standard deviation is small.”</em></blockquote><p><strong>Small is not the actual standard deviation.</strong></p><p>The actual value might be:</p><pre>Standard Deviation = 1.41</pre><p>We simply <strong>describe</strong> 1.41 as relatively small for that dataset.</p><p>Similarly:</p><pre>Standard Deviation = 7.07</pre><p>might be described as larger because the values are more spread out.</p><p>So:</p><p><strong>“Small” and “large” are descriptions.</strong></p><p><strong>Standard deviation itself is a calculated number.</strong></p><h3>🔁 The Whole Calculation in One Picture</h3><p>Starting data:</p><pre>8, 9, 10, 11, 12<br>        ↓</pre><p>Find the mean:</p><pre>Mean = 10<br>        ↓</pre><p>Find differences from the mean:</p><pre>-2, -1, 0, 1, 2<br>        ↓</pre><p>Square them:</p><pre>4, 1, 0, 1, 4<br>        ↓</pre><p>Find their average:</p><pre>2<br>        ↓</pre><p>Take the square root:</p><pre>√2<br>        ↓</pre><p>Final answer:</p><pre>Standard Deviation ≈ 1.41</pre><p>That’s all standard deviation calculation is doing.</p><h3>Why Does This Matter in Machine Learning?</h3><p>You may have seen standard deviation when learning about <strong>Feature Scaling</strong>.</p><p>One common scaling method is <strong>Standardization</strong>.</p><p>Standardization asks:</p><blockquote><em>“How far is this value from the mean compared with the normal spread of this feature?”</em></blockquote><p>Suppose:</p><pre>Mean = 10</pre><pre>Standard Deviation = 2</pre><p>And we have a value:</p><pre>12</pre><p>First:</p><pre>12 - 10 = 2</pre><p>Then:</p><pre>2 ÷ 2 = 1</pre><p>So the standardized value is:</p><pre>1</pre><p>This tells us:</p><p>👉 <strong>12 is one standard deviation above the mean.</strong></p><p>If the value were 8:</p><pre>(8 - 10) ÷ 2 = -1</pre><p>That means:</p><p>👉 <strong>8 is one standard deviation below the mean.</strong></p><p>This is why standard deviation is useful for Feature Scaling.</p><h3>⚠️ One Small Detail You’ll See Later</h3><p>You may sometimes see standard deviation calculated by dividing by:</p><pre>N</pre><p>and sometimes by:</p><pre>N - 1</pre><p>Don’t let this confuse you.</p><p>They are used in slightly different situations:</p><p><strong>Population Standard Deviation</strong> → divide by N</p><p><strong>Sample Standard Deviation</strong> → divide by N - 1</p><p>If you’re just learning the basic idea, focus first on understanding the N version.</p><p>You can learn the difference between population and sample standard deviation later.</p><h3>Easy Way to Remember</h3><p>Standard deviation calculation can be remembered in <strong>5 steps</strong>:</p><h3>1️ Find the Mean</h3><h3>2️ Find each value’s difference from the Mean</h3><h3>3️ Square those differences</h3><h3>4️ Find their Average</h3><h3>5️ Take the Square Root</h3><p>Or simply:</p><p><strong>Mean → Difference → Square → Average → Square Root</strong></p><p>And remember:</p><blockquote><strong><em>Standard deviation is not simply “small” or “large.” It is a number that measures how spread out our data is around its average.</em></strong></blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8a669d9cbae7" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Why Machine Learning Needs Feature Scaling: A Beginner-Friendly Guide]]></title>
            <link>https://medium.com/@sunethsilva/why-machine-learning-needs-feature-scaling-a-beginner-friendly-guide-f913d6592184?source=rss-c1f157a43ba8------2</link>
            <guid isPermaLink="false">https://medium.com/p/f913d6592184</guid>
            <category><![CDATA[standardization]]></category>
            <category><![CDATA[feature-scaling]]></category>
            <category><![CDATA[normalization]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[feature-engineering]]></category>
            <dc:creator><![CDATA[Suneth Nawoda De Silva]]></dc:creator>
            <pubDate>Sat, 08 Aug 2026 22:18:28 GMT</pubDate>
            <atom:updated>2026-08-08T22:50:57.023Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*siyWIUHKInFVWyCTcEaEDg.png" /></figure><p>Imagine you are building a Machine Learning model to compare different houses.</p><p>Your dataset looks like this:</p><pre>| House Size | Number of Rooms | Distance to City |<br>|------------|-----------------|------------------|<br>| 1500 sq.ft | 3               | 5 km             |<br>| 2200 sq.ft | 4               | 8 km             |<br>| 3000 sq.ft | 5               | 12 km            |</pre><p>Look at the numbers carefully.</p><p><strong>House Size</strong> → 1500, 2200, 3000<br><strong>Rooms</strong> → 3, 4, 5<br><strong>Distance</strong> → 5, 8, 12</p><p>There is a big difference between their number ranges.</p><p>House size is in the <strong>thousands</strong>, while the number of rooms is only around <strong>3–5</strong>.</p><p>For us, this is not confusing. We know that 3000 sq. ft and 5 rooms represent completely different things (Because they have units like sq.ft, km, Room Number).</p><p>But some Machine Learning algorithms mainly work with <strong>numbers (They can’t deal with units where human used to)</strong>.</p><p>And that can create a problem.</p><h3>What’s the Problem?</h3><p>Imagine the computer is comparing two houses.</p><p>The differences might be:</p><pre>House Size difference = 800<br>Room difference       = 1<br>Distance difference   = 3</pre><p>Look at those numbers:</p><pre>800<br>1<br>3</pre><p>800 is huge compared with 1 and 3.</p><p>So some Machine Learning algorithms may be influenced much more by <strong>House Size</strong> simply because its numbers are larger.</p><blockquote><strong><em>But larger numbers don’t necessarily mean a feature is more important.</em></strong></blockquote><p>That’s where <strong>Feature Scaling</strong> comes in.</p><h3>What Is Feature Scaling?</h3><p>Feature Scaling simply means:</p><blockquote><strong><em>Changing the numerical ranges of features so they can be compared more fairly(taking everything into one scale).</em></strong></blockquote><p>For example, before scaling:</p><pre>House Size → 1500, 2200, 3000<br>Rooms      → 3, 4, 5<br>Distance   → 5, 8, 12</pre><p>After scaling, we might get something like:</p><pre>House Size → 0.0, 0.47, 1.0<br>Rooms      → 0.0, 0.5, 1.0<br>Distance   → 0.0, 0.43, 1.0</pre><p>Now the features are on similar scales.</p><p>That’s the main idea behind Feature Scaling.</p><h3>⚠️ One Important Thing to Remember</h3><p>Feature Scaling happens <strong>column by column</strong>.</p><p>Suppose we have:</p><pre>| Size | Rooms | Distance |<br>|------|-------|----------|<br>| 1500 | 3     | 5        |<br>| 2200 | 4     | 8        |<br>| 3000 | 5     | 12       |</pre><p>We scale:</p><p><strong>Size column</strong> ↓</p><pre>1500<br>2200<br>3000</pre><p>Then:</p><p><strong>Rooms column</strong> ↓</p><pre>3<br>4<br>5</pre><p>Then:</p><p><strong>Distance column</strong> ↓</p><pre>5<br>8<br>12</pre><p>Each column is scaled separately.</p><p>❌ We don’t scale one house at a time.</p><p>✅ We scale one <strong>feature/column</strong> at a time.</p><h3>How Do We Scale Features?</h3><p>There are two common methods:</p><h3>1. Normalization</h3><h3>2. Standardization</h3><p>Don’t worry about the names. The ideas are quite simple.</p><h3>1️⃣ Normalization — Put Values Between 0 and 1</h3><p>Normalization <strong>usually </strong>changes the values of a feature so that they fall between:</p><pre>0 → 1</pre><p>Imagine delivery distances:</p><pre>2 km<br>6 km<br>10 km</pre><p>After normalization, they become:</p><pre>2 km  → 0.0<br>6 km  → 0.5<br>10 km → 1.0</pre><p>The smallest value becomes 0.</p><p>The largest value becomes 1.</p><p>Values between them get numbers somewhere between 0 and 1.</p><h3>The formula</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/335/1*Uxe6lxNrtjrtxUab_fLW3A.png" /><figcaption>Figure 1.1[Normalization Formula]</figcaption></figure><pre>Normalized Value = (Value - Minimum) / (Maximum - Minimum)</pre><p>For example, let’s normalize 6.</p><p>Our values are:</p><pre>2, 6, 10</pre><p>Minimum = 2<br>Maximum = 10</p><p>So:</p><pre>(6 - 2) / (10 - 2)</pre><pre>= 4 / 8</pre><pre>= 0.5</pre><p>That’s it!</p><h3>Easy way to remember</h3><p><strong>Normalization → Put everything between 0 and 1.</strong></p><h3>2️⃣ Standardization — Compare Values With the Average</h3><p>Standardization works a little differently.</p><p>Instead of using the smallest and largest values, it asks:</p><blockquote><strong><em>How far is this value from the average?</em></strong></blockquote><p>For this, we need two things:</p><p><strong>Mean</strong> and <strong>Standard Deviation</strong>.</p><p>Mean is easy.</p><p>Mean simply means <strong>average</strong>.</p><p>But what is Standard Deviation?</p><h3>What Is Standard Deviation?</h3><p>The name sounds complicated, but the idea is simple.</p><blockquote><strong><em>Standard Deviation tells us how spread out our values are around the average.</em></strong></blockquote><p>Imagine two groups of delivery times.</p><h3>🟢 Group A</h3><pre>28 min<br>29 min<br>30 min<br>31 min<br>32 min</pre><p>Average ≈ 30 minutes</p><p>Everyone is close to the average.</p><pre>28  29  30  31  32<br>        ↑<br>      Average</pre><p>The values are close together.</p><p>👉 <strong>Small Standard Deviation</strong></p><h3>🔴 Group B</h3><p>Now imagine:</p><pre>10 min<br>20 min<br>30 min<br>40 min<br>50 min</pre><p>The average is still:</p><pre>30 minutes</pre><p>But now the values are much more spread out.</p><pre>10       20       30       40       50<br>                  ↑<br>                Average</pre><p>Some values are far away from the average.</p><p>👉 <strong>Large Standard Deviation</strong></p><p>So remember:</p><p><strong>Small Standard Deviation</strong> → values are close to the average.</p><p><strong>Large Standard Deviation</strong> → values are spread far from the average.</p><p>That’s all you need to understand for now.</p><p>For further reading on Standard Deviation, you can follow this post :</p><p><a href="https://medium.com/@sunethsilva/standard-deviation-made-simple-what-does-it-actually-mean-8a669d9cbae7?sharedUserId=sunethsilva">https://medium.com/@sunethsilva/standard-deviation-made-simple-what-does-it-actually-mean-8a669d9cbae7?sharedUserId=sunethsilva</a></p><h3>📊 So, How Does Standardization Work?</h3><p>The formula is:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/211/1*ZQowvQPm9Xp4-Rv110YW9A.png" /><figcaption>figure 1.2[standardization formula]</figcaption></figure><pre>Standardized value = (Value - Mean) / Standard Deviation</pre><p>Let’s use a simple example.</p><p>Suppose:</p><pre>Average = 50<br>Standard Deviation = 10</pre><p>We want to standardize:</p><pre>60</pre><p>First, subtract the average:</p><pre>60 - 50 = 10</pre><p>Then divide by the Standard Deviation:</p><pre>10 / 10 = 1</pre><p>So:</p><pre>60 → 1</pre><p>What does 1 mean?</p><p>It means:</p><blockquote><strong><em>60 is one standard deviation above the average.</em></strong></blockquote><p>What about 40?</p><pre>(40 - 50) / 10</pre><pre>= -10 / 10</pre><pre>= -1</pre><p>So:</p><pre>40 → -1</pre><p>That means:</p><blockquote><strong><em>40 is one standard deviation below the average.</em></strong></blockquote><p>And what about 50?</p><pre>(50 - 50) / 10</pre><pre>= 0</pre><p>So the average becomes:</p><pre>50 → 0</pre><p>A simple way to think about it:</p><pre>Below Average     Average      Above Average</pre><pre>     -1  ←────────── 0 ──────────→  +1</pre><h3>Normalization vs Standardization</h3><p>Here’s the easiest comparison:</p><pre>| Normalization                       | Standardization                |<br>|-------------------------------------|--------------------------------|<br>| Usually puts values between 0 and 1 | Centers values around 0        |<br>| Uses Minimum &amp; Maximum              | Uses Mean &amp; Standard Deviation |<br>| `0` = smallest value                | `0` = around the average       |<br>| `1` = largest value                 | Positive = above average       |<br>| Very easy to understand             | Very commonly used in ML       |</pre><p>You don’t need to memorize which one is “better.”</p><p>It depends on your dataset and the Machine Learning algorithm you are using.</p><h3>Why Does Feature Scaling Actually Matter?</h3><p>Here’s another way to think about it.</p><p>Imagine two pizzas.</p><p>One pizza’s size is recorded in <strong>meters</strong>:</p><pre>0.3 m</pre><p>Another person’s system records the same size in <strong>millimeters</strong>:</p><pre>300 mm</pre><p>The numbers are:</p><pre>0.3<br>300</pre><p>300 looks much bigger.</p><p>But that doesn’t mean the second pizza is 1,000 times bigger!</p><p>They are simply using different units.</p><p>The same kind of problem can happen with Machine Learning features.</p><p>One feature might naturally contain:</p><pre>2, 3, 4, 5</pre><p>while another contains:</p><pre>20,000, 40,000, 80,000</pre><p>Feature Scaling helps put them onto comparable numerical scales.</p><h3>Which Algorithms Need Feature Scaling?</h3><p>Feature Scaling is especially important for algorithms that care about <strong>distance or the size of numbers</strong>, such as:</p><ul><li>K-Nearest Neighbors (KNN)</li><li>K-Means Clustering</li><li>Support Vector Machines (SVM)</li><li>Neural Networks</li><li>Principal Component Analysis (PCA)</li></ul><p><strong>On the other hand, tree-based algorithms such as:</strong></p><ul><li>Decision Trees</li><li>Random Forests</li></ul><p>usually <strong>don’t need</strong> Feature Scaling.</p><h3>🧠 The Easiest Way to Remember Everything</h3><p>Think of Feature Scaling like preparing people for a fair race.</p><p>Imagine one runner’s distance is recorded as:</p><pre>5 kilometers</pre><p>and another runner’s distance is recorded as:</p><pre>5,000 meters</pre><p>Looking only at the numbers:</p><pre>5<br>vs<br>5000</pre><p>They look completely different.</p><p>But they represent the <strong>same distance</strong>.</p><p>We need to put them on a comparable scale before making a fair comparison.</p><p>Machine Learning can have the same problem with features.</p><p>That’s why we use <strong>Feature Scaling</strong>.</p><h3>Quick Summary</h3><p><strong>Feature Scaling</strong> makes different numerical features have comparable scales.</p><p>Remember:</p><blockquote>Scaling is done <strong>column by column</strong>.</blockquote><blockquote>Large numbers don’t automatically mean a feature is more important.</blockquote><blockquote><strong>Normalization</strong> usually puts values between 0 and 1.</blockquote><blockquote><strong>Standardization</strong> uses the <strong>mean and standard deviation</strong>.</blockquote><blockquote><strong>Standard Deviation</strong> tells us how spread out values are around their average.</blockquote><blockquote>Algorithms like <strong>KNN, K-Means, SVM, PCA, and Neural Networks</strong> often benefit from scaling.</blockquote><blockquote><strong>Decision Trees and Random Forests</strong> usually don’t require it.</blockquote><p>And most importantly:</p><blockquote><strong><em>Feature Scaling changes the scale of your data — not what the data actually means.</em></strong></blockquote><p>Once you understand that sentence, you’ve understood the main idea behind Feature Scaling.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f913d6592184" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Python Libraries, Modules, Functions, and Classes Explained Simply]]></title>
            <link>https://medium.com/@sunethsilva/python-libraries-modules-functions-and-classes-explained-simply-4a2d5a356331?source=rss-c1f157a43ba8------2</link>
            <guid isPermaLink="false">https://medium.com/p/4a2d5a356331</guid>
            <category><![CDATA[function]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[python-libraries]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[class]]></category>
            <dc:creator><![CDATA[Suneth Nawoda De Silva]]></dc:creator>
            <pubDate>Fri, 31 Jul 2026 20:55:48 GMT</pubDate>
            <atom:updated>2026-07-31T20:55:48.354Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9gDZh1WlgYF0eAsydgo3TQ.png" /></figure><p>When learning Python, you will often hear words like <strong>library</strong>, <strong>module</strong>, <strong>function</strong>, and <strong>class</strong>.</p><p>At first, these terms may sound confusing. However, they are simply different ways of organizing and reusing code.</p><p>Let’s understand these concepts using a few simple examples. Don’t worry if you are unfamiliar with the libraries or functions shown below. they are only used to explain the main ideas.</p><h3>What Is a Python Library?</h3><p>A <strong>library</strong> is a collection of reusable Python code.</p><p>Instead of writing everything from the beginning, we can install and use a library created by other developers.</p><p>For example, Python libraries can help us:</p><ul><li>Analyze data</li><li>Create graphs</li><li>Build Machine Learning models</li><li>Develop websites</li><li>Work with images</li></ul><p>Some popular Python libraries are:</p><ul><li>pandas for data analysis</li><li>matplotlib for data visualization</li><li>scikit-learn for Machine Learning</li><li>numpy for numerical calculations</li></ul><p>You can think of a library as a large toolbox containing many useful tools.</p><h3>What Is a Module?</h3><p>A library can contain several <strong>modules</strong>.</p><p>Each module groups code that is related to a particular task.</p><p>Think of it like a real library:</p><pre>Library 📚<br>│<br>├── Module 📂<br>│   ├── Functions<br>│   └── Classes<br>│<br>├── Module 📂<br>│   ├── Functions<br>│   └── Classes<br>│<br>└── Module 📂<br>    ├── Functions<br>    └── Classes</pre><p>A library is the main collection, while modules are smaller sections inside it.</p><p>For example:</p><pre>import matplotlib.pyplot as plt</pre><p>Here:</p><pre>matplotlib → Library<br>pyplot     → Module<br>plt        → Short name given to the module(known as alise)</pre><p>The pyplot module contains tools for creating graphs and charts.</p><p>For example:</p><pre>import matplotlib.pyplot as plt<br><br>x = [1, 2, 3, 4]<br>y = [10, 20, 15, 30]<br>plt.plot(x, y)<br>plt.show()</pre><p>In this example, we use the pyplot module to create and display a line graph.</p><p>Another example is the datetime library:</p><pre>from datetime import date</pre><p>Here:</p><pre>datetime → Library or module<br>date     → Class inside datetime</pre><p>The exact structure can sometimes vary, but the main idea is that modules organize related code.</p><h3>What Is a Function?</h3><p>A <strong>function</strong> is reusable code that performs a specific task.</p><p>For example:</p><pre>print(&quot;Hello&quot;)</pre><p>The print() function displays something on the screen.</p><p>Libraries also provide useful functions.</p><p>Consider this example:</p><pre>import pandas as pd<br><br>data = pd.read_csv(&quot;students.csv&quot;)</pre><p>Here:</p><pre>pandas   → Library<br>read_csv → Function</pre><p>The read_csv() function reads data from a CSV file and loads it into Python.</p><p>A function usually receives some input, performs a task, and may return a result.</p><p>For example:</p><pre>numbers = [10, 20, 30]<br><br>largest_number = max(numbers)<br>print(largest_number)</pre><p>Output:</p><pre>30</pre><p>The max() function checks the list and returns the largest value.</p><p>Functions are normally called using parentheses:</p><pre>function_name()</pre><p>Sometimes we provide information inside the parentheses:</p><pre>function_name(input)</pre><p>For example:</p><pre>pd.read_csv(&quot;students.csv&quot;)</pre><p>The file name is passed as input to the function.</p><h3>What Is a Class?</h3><p>A <strong>class</strong> is a blueprint used to create objects.</p><p>Imagine a class as the design of a house.</p><p>The design explains what the house should contain, but it is not the actual house. Using the same design, we can create multiple houses.</p><p>Similarly, a Python class describes what an object should contain and what it should be able to do.</p><p>Many Machine Learning algorithms are provided as classes.</p><p>For example:</p><pre>from sklearn.linear_model import LinearRegression<br><br>model = LinearRegression()</pre><p>Here:</p><pre>sklearn             → Library<br>linear_model        → Module<br>LinearRegression    → Class<br>model               → Object created from the class</pre><p>LinearRegression is a blueprint for creating a Linear Regression Machine Learning model.</p><p>When we write:</p><pre>model = LinearRegression()</pre><p>we create an object called model.</p><p>We can then train the model:</p><pre>model.fit(X, y)</pre><p>And use it to make predictions:</p><pre>predictions = model.predict(X_test)</pre><p>The class provides the structure and behaviour of the model.</p><h3>Function vs Class</h3><p>A function performs a task directly.</p><pre>data = pd.read_csv(&quot;students.csv&quot;)</pre><p>The read_csv() function immediately reads the file.</p><p>A class is normally used to create an object first.</p><pre>model = LinearRegression()</pre><p>After creating the object, we can use its methods:</p><pre>model.fit(X, y)<br>model.predict(X_test)</pre><p>A <strong>method</strong> is simply a function that belongs to an object or class.</p><p>For example:</p><pre>model.fit()</pre><p>Here, fit() is a method of the model object.</p><h3>A Complete Example</h3><p>Consider the following code:</p><pre>import pandas as pd<br>import matplotlib.pyplot as plt<br>from sklearn.linear_model import LinearRegression</pre><p>Let’s break it down.</p><h3>First import</h3><pre>import pandas as pd</pre><ul><li>pandas is the library.</li><li>pd is the short name.</li><li>It contains functions such as read_csv().</li></ul><p>Example:</p><pre>data = pd.read_csv(&quot;students.csv&quot;)</pre><h3>Second import</h3><pre>import matplotlib.pyplot as plt</pre><ul><li>matplotlib is the library.</li><li>pyplot is a module.</li><li>plt is the short name.</li><li>It contains functions such as plot() and show().</li></ul><p>Example:</p><pre>plt.plot([1, 2, 3], [10, 20, 30])<br>plt.show()</pre><h3>Third import</h3><pre>from sklearn.linear_model import LinearRegression</pre><ul><li>sklearn is the library.</li><li>linear_model is the module.</li><li>LinearRegression is the class.</li></ul><p>Example:</p><pre>model = LinearRegression()</pre><h3>Simple Summary</h3><pre>Library<br>   └── Contains modules</pre><pre>Module<br>   └── Groups related code</pre><pre>Function<br>   └── Performs a specific task</pre><pre>Class<br>   └── Acts as a blueprint for creating objects</pre><pre>Object<br>   └── An actual item created from a class</pre><pre>Method<br>   └── A function that belongs to an object or class</pre><p>For example:</p><pre>from sklearn.linear_model import LinearRegression</pre><pre>model = LinearRegression()<br>model.fit(X, y)</pre><p>Here:</p><pre>sklearn          → Library<br>linear_model     → Module<br>LinearRegression → Class<br>model            → Object<br>fit()            → Method</pre><blockquote><em>Libraries contain reusable code, modules organize that code, functions perform tasks, and classes help us create objects such as Machine Learning models.</em></blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4a2d5a356331" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Why Do We Split Data in Machine Learning?]]></title>
            <link>https://medium.com/@sunethsilva/why-do-we-split-data-in-machine-learning-2268112fe655?source=rss-c1f157a43ba8------2</link>
            <guid isPermaLink="false">https://medium.com/p/2268112fe655</guid>
            <category><![CDATA[dataset]]></category>
            <category><![CDATA[split-data]]></category>
            <category><![CDATA[training-data]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Suneth Nawoda De Silva]]></dc:creator>
            <pubDate>Wed, 29 Jul 2026 22:26:29 GMT</pubDate>
            <atom:updated>2026-07-29T22:27:25.880Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*IwRty6GdsjoSmazW4eg8BA.png" /></figure><blockquote>Imagine you are building a machine learning model for a food delivery app. The goal is to predict how long an order will take to arrive.</blockquote><p>The dataset may contain information such as:</p><pre>| Distance | Traffic Level | Delivery Time |<br>|----------|---------------|---------------|<br>| 2 km     | Low           | 12 minutes    |<br>| 5 km     | Medium        | 25 minutes    |<br>| 6 km     | High          | 37 minutes    |<br>| 8 km     | High          | 45 minutes    |</pre><p>Here:</p><ul><li><strong>Distance</strong> and <strong>Traffic Level</strong> are inputs.</li><li><strong>Delivery Time</strong> is the output we want to predict.</li></ul><p>The model learns a relationship like this:</p><pre>Distance + Traffic<br>        ↓<br>Machine Learning Model<br>        ↓<br>Predicted Delivery Time</pre><h3>Why Not Use All the Data for Training?</h3><p>Suppose we have data from 100 previous deliveries.</p><p>If we train the model using all 100 deliveries and then test it using the same deliveries, the result may look very accurate.</p><p>But this is not a fair test.</p><p>It is similar to giving a student the exact exam questions before the exam. The student may remember the answers without truly understanding the subject.</p><p>A machine learning model may also remember patterns from the training data without performing well on new data.</p><h3>The Train-Test Split</h3><p>To avoid this problem, we split the dataset before training.</p><pre>100 Deliveries</pre><pre>80 Deliveries → Training Set<br>20 Deliveries → Test Set</pre><p>The <strong>training set</strong> is used to teach the model.</p><p>The <strong>test set</strong> is kept hidden and used only to evaluate the trained model.</p><h3>How Testing Works</h3><p>Suppose one delivery in the test set has:</p><pre>Distance: 6 km<br>Traffic: High</pre><p>The model predicts:</p><pre>Predicted Time: 34 minutes</pre><p>The actual delivery time was:</p><pre>Actual Time: 37 minutes</pre><p>The difference is only 3 minutes, so the prediction is reasonably close.</p><p>We repeat this process for all deliveries in the test set.</p><p>If the predictions are close to the actual values, the model is performing well. If the differences are large, the model may need improvement.</p><h3>Why Do We Commonly Use an 80/20 Split?</h3><p>An 80/20 split means:</p><ul><li>80% of the data is used for training.</li><li>20% of the data is used for testing.</li></ul><p>The model needs enough data to learn meaningful patterns. At the same time, we need enough unseen data to evaluate it fairly.</p><p>The exact percentages are not fixed. Other common splits include:</p><ul><li>70% training and 30% testing</li><li>75% training and 25% testing</li><li>90% training and 10% testing</li></ul><p>The best split depends on the size and type of the dataset.</p><h3>The Complete Process</h3><pre>Original Dataset<br>                  20 Diliveries<br>                      │<br>                      ▼<br>            Split Before Training<br>              80%          20%<br>        16 Diliveries    4 Diliveries<br>              │              │<br>              ▼              │<br>        Train the Model      │<br>              │              │<br>              ▼              │<br>      Learn Relationships    │<br>              │              │<br>              └───────► Predict Delivery Time<br>                              │<br>                              ▼<br>                 Compare Predictions<br>                  With Actual Times<br>                              │<br>                              ▼<br>                    Evaluate Model</pre><h3>A Real-Life Analogy: Learning to Drive</h3><p>Think about learning to drive.</p><p>Your driving lessons are the <strong>training set</strong>. During those lessons, you learn how to control the vehicle and respond to different situations.</p><p>Your actual driving test is the <strong>test set</strong>.</p><p>If you practise only one route repeatedly, you may perform well on that route but struggle on a road you have never seen.</p><p>A driving test checks whether you can apply what you learned to new roads and situations.</p><p>Similarly, the test set checks whether a machine learning model can make accurate predictions on new, unseen data.</p><h3>Train-Test Split in Python</h3><p>Using Scikit-learn, we can split a dataset with a few lines of code:</p><pre>from sklearn.model_selection import train_test_split</pre><pre>X_train, X_test, y_train, y_test = train_test_split(<br>    X,<br>    y,<br>    test_size=0.2,<br>    random_state=42<br>)</pre><p>Here:</p><ul><li>X contains the input features, such as mileage and age.</li><li>y contains the selling prices.</li><li>test_size=0.2 keeps 20% of the data for testing.</li><li>random_state=42 makes the split reproducible.</li></ul><p>The model is trained using X_train and y_train.</p><p>After training, it makes predictions using X_test. Those predictions are then compared with y_test, which contains the actual answers.</p><h3>One Important Rule</h3><p>The test set must remain unseen during training.</p><p>Using information from the test set while training the model is called <strong>data leakage</strong>. It can make the model appear more accurate than it really is.</p><p>The test set should be opened only when the final trained model is ready to be evaluated.</p><h3>Final Thoughts</h3><p>The purpose of splitting data is not to check whether a model can remember its training examples.</p><p>It is to answer a more important question:</p><blockquote><em>“Can this model make accurate predictions for data it has never seen before?”</em></blockquote><p>Remember the main idea:</p><pre>Train on one part of the data.</pre><pre>Test on a different, unseen part.</pre><p>Once you understand this idea, you understand one of the most important foundations of machine learning.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2268112fe655" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Deploying a React App Free on WSO2 Choreo]]></title>
            <link>https://medium.com/@sunethsilva/deploying-a-react-app-free-on-wso2-choreo-f7c081511c6c?source=rss-c1f157a43ba8------2</link>
            <guid isPermaLink="false">https://medium.com/p/f7c081511c6c</guid>
            <dc:creator><![CDATA[Suneth Nawoda De Silva]]></dc:creator>
            <pubDate>Sun, 14 Dec 2025 16:51:32 GMT</pubDate>
            <atom:updated>2025-12-14T16:51:32.248Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*MV9wWeqrQzMF3mXMI9rRWg.png" /></figure><blockquote>WSO2 <strong>Choreo</strong> is a <strong>cloud-native integration and application deployment platform</strong> that allows developers to deploy web applications, APIs, and microservices easily.</blockquote><h3><strong>Introduction</strong></h3><p>Deploying a React application to the cloud can sometimes feel complex due to infrastructure setup, server configuration, and deployment pipelines. <strong>WSO2 Choreo</strong> simplifies this process by providing a <strong>cloud-native platform</strong> where developers can deploy web applications without managing servers or Kubernetes manually.</p><p>This guide explains <strong>how to deploy a React application built with Vite</strong> on <strong>WSO2 Choreo (Free Tier)</strong>. It is suitable for <strong>students, beginners, and developers</strong> who want a quick and reliable deployment solution.</p><h3>Prerequisites</h3><p>Before starting, ensure the following:</p><ul><li>A React application (Vite or CRA)</li><li>A GitHub account</li><li>A WSO2 Choreo account (free)</li></ul><h3><strong>Step 1: </strong>Prepare the Project for GitHub</h3><p>Before uploading the project into GitHub, ensure unnecessary files are ignored.</p><p>ex: .gitignore</p><pre>node_modules<br>dist<br>.env</pre><p>This keeps the repository lightweight and avoids build issues.</p><h3>Step 2: Upload the Project to GitHub</h3><p>Run the following commands inside the project folder:</p><pre>git init<br>git add .<br>git commit -m &quot;Initial React app for Choreo deployment&quot;</pre><p>Create a <strong>public GitHub repository</strong> and push the project:</p><pre>git branch -M main<br>git remote add origin https://github.com/YOUR_USERNAME/your-repo.git<br>git push -u origin main</pre><h3>Step 3: Create a Choreo Account</h3><ul><li>Go to <a href="https://console.choreo.dev/">https://console.choreo.dev</a></li><li>Sign up using <strong>GitHub</strong></li><li>Select the <strong>Free / Developer Plan</strong></li><li>Choose <strong>US</strong> as the cloud region (default)</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*TRPdai334Ayc7f3s-Yxhhw.png" /></figure><h3>Step 4: Create a Web Application Component</h3><ol><li>Click <strong>Create Component</strong></li><li>Select <strong>Web Application</strong></li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*K3WOlsa-pFyeFEljmJoIXg.png" /></figure><p>3. Click on Continue with GitHub</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*mKPq7L0HLIY0rF4RTIuOlw.png" /></figure><p>4. Add your github repository</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*bjivqlM4-1ri1NvID6wJHw.png" /></figure><ul><li>Repository: Your repo</li><li>Branch: main</li><li>Component Directory: /</li></ul><h3>Component Details:</h3><ul><li><strong>Display Name:</strong> Name you want</li><li><strong>Name:</strong> name-you-want</li><li><strong>Description:</strong> your description</li></ul><h3>Step 5: Configure Build Settings</h3><p>For <strong>Vite + React</strong>, use the following settings:</p><blockquote>Build Command: npm install &amp;&amp; npm run build</blockquote><blockquote>Build Path: dist</blockquote><blockquote>Node Version: 18</blockquote><p>Click <strong>Save &amp; Deploy</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*VYRvCAj9Anu4LWsLlCmcTA.png" /></figure><h3>Step 6: Deployment Process</h3><p>Choreo automatically:</p><ul><li>Installs dependencies</li><li>Builds the React application</li><li>Deploys the static files</li></ul><p>The build usually completes within <strong>2–5 minutes</strong>.<br> Once successful, the build status changes to <strong>green (success)</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*znfoXQVqGxE25lGsrSINyg.png" /></figure><h3>Step 7: Access the Live Application</h3><ul><li>Open the component</li><li>Go to Overview</li><li>Copy the <strong>public URL</strong></li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*KT6iy1O0Wu5ht9Td-ehQ4A.png" /></figure><h3>Conclusion</h3><p>WSO2 Choreo provides a <strong>simple and efficient way</strong> to deploy React applications. With minimal configuration and no infrastructure management, developers can focus entirely on building applications.</p><p>This deployment method is ideal for:</p><ul><li>Student projects</li><li>Portfolio websites</li><li>Learning cloud-native deployment</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f7c081511c6c" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>