<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Tesly Sebastian on Medium]]></title>
        <description><![CDATA[Stories by Tesly Sebastian on Medium]]></description>
        <link>https://medium.com/@teslys88?source=rss-59820976bf5------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*SifT-CoJiKay_iDY</url>
            <title>Stories by Tesly Sebastian on Medium</title>
            <link>https://medium.com/@teslys88?source=rss-59820976bf5------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 05:26:54 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@teslys88/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[How to Build a Robust Random Forest Regression Model in Python]]></title>
            <link>https://medium.com/@teslys88/how-to-build-a-robust-random-forest-regression-model-in-python-4c0ee2a1a287?source=rss-59820976bf5------2</link>
            <guid isPermaLink="false">https://medium.com/p/4c0ee2a1a287</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[random-forest-regressor]]></category>
            <category><![CDATA[regression-modeling]]></category>
            <dc:creator><![CDATA[Tesly Sebastian]]></dc:creator>
            <pubDate>Tue, 18 Apr 2023 23:56:36 GMT</pubDate>
            <atom:updated>2024-08-20T05:54:32.310Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/975/1*CAgcMmtpsvFf2qBR_W5gtw.png" /><figcaption>Source: <a href="https://www.entrepreneur.com/en-za/growth-strategies/to-survive-and-thrive-you-need-a-growth-mindset/326828">https://www.entrepreneur.com/en-za/growth-strategies/to-survive-and-thrive-you-need-a-growth-mindset/326828</a></figcaption></figure><p>Random Forest is a supervised machine learning algorithm using decision trees for regression and classification tasks. It is used for regression problems with continuous variables rather than categorical variables. This article will focus on the implementation of the random forest regression algorithm in Python.</p><p>The basic idea behind Random Forest regression is to combine multiple trees to determine the final result rather than relying on a single decision tree. The dataset is split into multiple trees and trained for observations. The process of dividing the dataset for decision trees is called Bootstrap. Every decision tree has got a high variance. So, the final output is taken by aggregating the results of all the trees using a majority voting classifier. This method is called Aggregation (Refer to Figure 1 for a better understanding). Bagging is the next stage where all the aggregated results are combined. This helps in reducing the risk of overfitting the dataset and improves accuracy. Bagging is widely used in Decision trees, but this can also be used in other models.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/967/1*LXIgbrYQOS2bh2o_CQaSIw.png" /><figcaption>Figure 1: Random Forest Regression</figcaption></figure><p>The main advantages of Random Forest regression are its high accuracy rate, ability to handle missing values and outliers, high dimensional data can be handled easily, better feature importance handling and it is less likely to be overfitted. Even though it has many advantages, the model does not explain how the features are used to make predictions, which makes it difficult to interpret. It is computationally expensive if the dataset is huge. In addition, this model cannot predict values outside of the trained dataset. They can still overfit if there are many trees in the model, even though it is less likely.</p><p>As we have covered the basics of Random forest regression, we will now see how to implement it in Python. We will be using Jupyter Notebook to create the model and Python libraries like Scikit Learn, Numpy, Pandas, Matplotlib, Seaborn</p><h3><strong>Random Forest Regression in Python:</strong></h3><h4><strong>1.</strong> <strong>Importing Libraries</strong></h4><p>Importing libraries such as Pandas, Numpy, Matplotlib, and Seaborn.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/555/1*kq2_mVl5vznkgVMmVJB91w.png" /><figcaption>Figure 2: Importing libraries</figcaption></figure><p><strong>2.</strong> <strong>Import the dataset</strong></p><p>To demonstrate, I will be importing the Salary Data.csv file which has data from 373 individuals with 6 columns as shown in the image below. Path to the file should be mentioned the path in place of <em>“{..path}”.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/774/1*Vl0Dopdr-jMj65cV6dME3g.png" /><figcaption>Figure 3: Reading data from csv</figcaption></figure><p>Note: Here<em> “ r’ “</em> is a string modifier given to consider the path as a path to the file. Meaning <em>r’My\Path\Without\Escaping’</em> will evaluate to <em>‘My\Path\Without\Escaping’</em> — without causing the backslash to escape characters. This is equivalent to<em> ‘My\\Path\\Without\\Escaping’</em> string, but without the string modifier. Also, the most common delimiter in a .csv is a comma “ , “. In case, you have a dataset with some other delimiter like “ | “, you can mention <em>sep = “|”.</em></p><h4>3. Data preprocessing</h4><h4>3.1. Handling Missing Data</h4><p>Finding the shape of dataframe will help to find the size of your data and <em>isnull().sum()</em> will help us find the count of missing values. Here you can see, there are 2 missing values each in Age and Gender.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/895/1*oy7a06WFWiJOx3X9qptpeQ.png" /><figcaption>Figure 4: Print Dataframe column names and null values</figcaption></figure><p>Normally, missing values can be either removed from the dataframe or filled with any values like the mean of the corresponding variable. Here we are going to fill the missing values in the Age column with mean.</p><p>Note: since the remaining values in the Age column is an integer, mean value is converted to an integer with <em>int()</em> function.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/563/1*zUNxXvzVT8rPi1fje5jS2g.png" /><figcaption>Figure 5: Fill null values in Dataframe with mean</figcaption></figure><p>As you have seen the column Gender, has got categorical values. We will have to fill in the missing values which have the maximum count as mean won’t work in this case.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/467/1*h6jhwXFXTerMiSfXuljcWg.png" /><figcaption>Figure 6: Fill null values in Dataframe with string</figcaption></figure><p>Once the missing values are filled, cross-check once the count of missing values in the dataframe is 0.</p><h4>3.2. Label encoding</h4><p>Since we have many categorical features in the dataset, we will have to convert them into numerical values, so that machine learning algorithms can decide how to interpret these labels. <em>Gender, Education level, and Job Title </em>are the columns that need label encoding. So the values in these columns are processed in a for loop with <em>LabelEncoder</em> from sklearn. Each unique value in columns is converted to number. For example, column Gender has 2 distinct values <em>Male</em> and <em>Female</em>. Here <em>Male </em>is replaced with 1 and <em>Female </em>is replaced with 1.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/587/1*WluS5X5MTY-YMDHTye2vgw.png" /><figcaption>Figure 7: Label encoding</figcaption></figure><h4>3.3. Correlation matrix</h4><p>The correlation matrix is a table that shows the relation between the variables in dataframe.</p><p>Mostly, the correlation value ranges from -1 to 1. A positive value implies a good correlation, a negative value implies a weak correlation and 0 means no correlation between variables.</p><p>A heatmap is used to plot the correlation matrix here. vmin is set to -1, vmax is set to 1 and center is set to 0 here. This is to clearly plot the positive and negative correlation in the heatmap.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/657/1*LoM8968xUbPPgjmx5LBnWg.png" /><figcaption>Figure 8: Correlation matrix</figcaption></figure><p>As you see in the code above, we have defined a <em>mask</em> for the top right of the correlation matrix. This is to avoid duplication in the correlation matrix plot and avoid confusion. Mask is basically a Boolean array that is used to hide the duplicating row of corresponding variables along with the diagonal row.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/571/1*uLMM20VaD1tYopbPkDjE0Q.png" /><figcaption>Figure 9: Correlation mask matrix</figcaption></figure><p>You can also unstack this mask matrix to print the correlation as a list. We can see that correlation between Age, Salary, and Years of Experience is very high. This code displays a list of features that have a correlation coefficient greater than 0.8 (matrix &lt; 1 is to eliminate self-correlated lists).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/561/1*83G-GRy1Hp4vlCe5G1A2Yg.png" /><figcaption>Figure 10: Correlation mask matrix unstacking</figcaption></figure><h4>4. Training the Random Forest Regression model</h4><h4>4.1. Standard scaling</h4><p>Standardscaler will rescale the data in such a way that the mean is 0 and the standard deviation is 1. By applying this technique, the data is transformed to have a normal distribution. Many machine learning algorithms require the data to be normally distributed, making this step an essential preprocessing requirement.</p><h4>4.2. Separating the features and the target variable</h4><p>Once it is standardized, the dataframe needs to be separated for the target variable and the features. Here, features are the columns except for salary (last column) and the target is salary.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/558/1*zPAKWkDG0iDfLTBTNxN6BA.png" /><figcaption>Figure 11: Standard scaler</figcaption></figure><p>Usually, the target variable in the dataframe is the last column. If not, we need to move it to last in order to run this code. While splitting the features, we use “<em>iloc[:,:-1]”</em> to select all columns except the target variable. Similarly for the target variable, <em>“iloc[:,-1]” </em>is used. <em>iloc </em>is a method used in Pandas library to select columns in Dataframe using integer-based indexing. <em>iloc</em> stands for integer location.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/462/1*dcGOqMsDsns7WY7Tb2NSKw.png" /><figcaption>Figure 12: Feature and outcome separation</figcaption></figure><p><em>“iloc[:,:-1]” </em>selects all the rows and all the columns except for the last column of the DataFrame. The <em>“:”</em> before the comma specifies that we want to select all the rows, and <em>“:-1”</em> after the comma specifies that we want to select all the columns except for the last one.<em> “iloc[:,-1]”</em> selects all the rows of the last column of the DataFrame. The “: “ before the comma specifies that we want to select all the rows, and the <em>“-1”</em> after the comma specifies that we want to select the last column.</p><h4>4.3. Box plot</h4><p>The box plot, also known as a box-and-whisker plot is the best way to display the data distribution. Since our dataset contained values in different ranges, box plot was plotted after scaling the data using standard scaler, so that a clear view of the data distribution is visible.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/688/1*_n62N6m5BAtetA0BLhrq1w.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/612/1*aZIF_-Xh4mFcS0yUCG_igA.png" /><figcaption>Figure 13: Box plot</figcaption></figure><h4>4.4. Feature selection</h4><p>The process of feature engineering involves transforming the gathered data into features that will improve the model’s performance and accuracy by better representing the problem. Using LassoCV (available in sklearn library), which is a linear regression algorithm that is used for feature selection, we can see that, all the features contribute to the model. Out of which,<em> “Years of experience”</em> is the most important feature in our dataset.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/711/1*mGVkBP1LVnhSXfTZ5QYUcQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/958/1*zk6JxcyE0qcqJXPSYisPdw.png" /><figcaption>Figure 14: Feature importance</figcaption></figure><h4>4.5. Splitting the feature and target variables into train test split</h4><p>We use train_test_split from sklearn to split the data for train and test. Test size is given as 0.2 as we are splitting 80% for train and 20% for test. It is always best to use a random value for random_state, else the train and test set across different execution will be out of control. Train and test sets will be fixed if we fix a random state with a number. Usually, a random number is chosen for random_state. But here we are iterating over a range of 0 to the length of dataframe, that is 373 (this step is done just because the dataset is small). The optimal random_state is found when the accuracy score is maximum.</p><p>The same is the case with <em>n_estimator</em>. This is the number of decision trees you want in the model. A higher number of <em>n_estimator</em> will result in a better-performing model but makes your code slower. As we did for the <em>random_state</em>, we are iterating the <em>n_estimator</em> from 10 to 50 with 5 as the interval. The ideal <em>n_estimator</em> for a small dataset is between 10 to 50.</p><p>The customization limit is not over. We can define <em>criterion </em>(measures the quality of split), <em>max_depth </em>(maximum possible depth of trees), <em>max_features </em>(maximum no. of features the model will consider while splitting), <em>min_samples_split </em>(minimum number of data points in a node) <em>min_samples_leaf</em> (minimum number of data points in a leaf node) and so on.</p><p>With default values and <em>n_estimator </em>as 50, the accuracy was 91.57%.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/906/1*oUmhRq2i8runJ5i4jJeVMw.png" /><figcaption>Figure 15: Random forest regression model (without optimal random_state value)</figcaption></figure><p>with the below approach, we have got an accuracy of 94.68%. As the dataset is small, hyperparameter tuning has not been performed to avoid overfitting of the model.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/975/1*E-RYlvOT2FmmC-Ue2-fvsg.png" /><figcaption>Figure 16: Random forest regression model (without optimal random_state value)</figcaption></figure><p>If the data is huge, this approach might not work as it is very time-consuming.</p><h4>5. Visualizing the Random Forest Regression results</h4><p>Plotting the test and predicted values shows that the predicted values are within the range of real values. The green <strong><em>star </em></strong>represents real values from the dataset and red <strong><em>circle </em></strong>shows predicted values</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/906/1*Dzqab6fi9GKM9ohkWEvI5Q.png" /><figcaption>Figure 17: Real vs predicted values</figcaption></figure><h4>6. Accuracy</h4><p>The accuracy of a regression model can be calculated using metrics such as r2_score, mean square error (MSE), and root mean square error (RMSE) from sklearn library. R2_score is specifically designed for regression models to see how well the data fits the regression model. In more detail, the R2 score measures how much of the variance in the dependent variable can be explained by the independent variables. MSE is the average summation of the square difference between actual output and predicted output. As MSE reduces, model accuracy increases. RMSE is the square root of MSE. We have got MSE and RMSE values close to 0 which means the model fits the given dataset.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/583/1*j78rn99AmXd_QM9hofJ-zA.png" /><figcaption>Figure 18: MSE and RMSE values</figcaption></figure><h4>7. Conclusion</h4><p>As a final note, Random Forest regression is an easy-to-use, powerful machine-learning method. It can handle both regression and classification tasks. The main advantage of Random Forest is that it can automatically select uncorrelated decision trees and carry out feature selection. This model is ideal for datasets with many features as each decision tree is a set of a random set of features. Training this model might take a long time since it generates a lot of decision trees and makes decisions on average predictions. This model works best in applications like predictive maintenance of useful life machinery and equipment, finance sector like stock prediction, credit risk, and so on.</p><blockquote>“A forest of trees may seem chaotic, but a random forest is perfectly organized for regression.”</blockquote><p>Keep learning and happy coding!!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4c0ee2a1a287" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>