<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Changsung Moon on Medium]]></title>
        <description><![CDATA[Stories by Changsung Moon on Medium]]></description>
        <link>https://medium.com/@csmoon?source=rss-e3e0d9600473------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/2*reqRiR9yo6JK5nSYZMPOaQ.png</url>
            <title>Stories by Changsung Moon on Medium</title>
            <link>https://medium.com/@csmoon?source=rss-e3e0d9600473------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 16:56:42 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@csmoon/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[6 Approaches to Deal with Imbalanced Classes]]></title>
            <link>https://medium.com/@csmoon/6-approaches-to-deal-with-imbalanced-classes-9133f2906e86?source=rss-e3e0d9600473------2</link>
            <guid isPermaLink="false">https://medium.com/p/9133f2906e86</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[data]]></category>
            <dc:creator><![CDATA[Changsung Moon]]></dc:creator>
            <pubDate>Sun, 23 Dec 2018 07:02:09 GMT</pubDate>
            <atom:updated>2018-12-23T07:05:41.627Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jCKfQ0R2nm60Q7litsyLZw.jpeg" /></figure><p>Have you seen this problem in your dataset? You might see the imbalanced classes problem when you deal with classification problems. This is very common. Imagine that you create a classification model and get 98% accuracy. Can you say your model is good because it shows the pretty high accuracy? What if around 98% of the data belongs to one class? We can make 98% accuracy without building a model by just predicting all of the test cases as belonging to one class.</p><p>For example, in a cancer dataset, 98% of tumors might be benign, and only 2% of them might be malignant. We can say that all of the patients don’t have a cancer if we just want a high accuracy. In a house rental dataset, only 2% of tenants might have been evicted. We can make good predictions easily by saying that all of tenants will not be evicted.</p><p>You can have an imbalanced classes problem on multi-class classification as well as binary classification. Most of techniques can be used for both classifications.</p><h3>Approaches to handling imbalanced classes</h3><p><strong>1) Collecting more data</strong></p><p>In many cases, the data collection might be pretty expensive. However, this doesn’t mean that collecting more data should be ignored. If you are able to get more examples of minor classes, this approach would be pretty helpful to solve the class imbalance problem.</p><p><strong>2) Choosing the right performance metrics</strong></p><p>Selecting appropriate performance measures for you model is important to get more insight into the model. In the case of imbalanced classes, the following metrics can be applied:</p><p><strong>Precision</strong>: accuracy of the positive predictions</p><ul><li>TP / (TP + FP)</li></ul><p><strong>Recall</strong>: ratio of positive instances that are correctly detected by the classifier</p><ul><li>TP / (TP + FN)</li></ul><p><strong>F1 score</strong>: harmonic mean of precision and recall</p><ul><li>2 x (precision x recall) / (precision + recall)</li></ul><p><strong>AUC</strong>: relation between true positive rate and false positive rate</p><ul><li>Cohen’s kappa: classification accuracy normalized by the imbalance of the classes</li></ul><p><strong>3) Resampling the training set</strong></p><p>You can make more balanced data by sampling your dataset. There are two main approaches: under-sampling and over-sampling.</p><p><strong>Under-sampling</strong></p><ul><li>Under-sampling decreases the number of instances of the majority class by deleting them.</li><li>Consider this approach when you don’t have a lot of data.</li></ul><p><em>Advantages</em></p><ul><li>Improved run-time of the model</li><li>Reduced storage for the training set</li></ul><p><em>Disadvantages</em></p><ul><li>Information loss</li></ul><p><strong>Over-sampling</strong></p><ul><li>Over-sampling increases the number of instances of the minority class by adding copies of them.</li><li>Consider this approach when you have a lot of data.</li></ul><p><em>Advantages</em></p><ul><li>No information loss</li><li>Often outperforms under-sampling</li></ul><p><em>Disadvantages</em></p><ul><li>Over-sampling can lead to model overfitting since it duplicates the instances of the minority class.</li></ul><p><strong>4) Synthetic minority over-sampling</strong></p><p>Generating synthetic samples technique can be used to avoid the overfitting problem, which occurs when copies of minority instances are added. The popular algorithm is SMOTE (Synthetic Minority Over-sampling Technique). It selects a subset of minority instances, and then creates new synthetic similar instances.</p><p><em>Avantages</em></p><ul><li>helpful to avoid the overfitting problem occurred by the over-sampling with duplicates</li><li>No information loss</li></ul><p><em>Disadvantages</em></p><ul><li>SMOTE works great for low dimensional data, but it is not very effective for high dimensional data</li></ul><p><strong>5) Ensemble techniques</strong></p><p>So far, we have looked at approaches resampling the original data. There is an alternative approach to deal with imbalanced data. Ensemble methods such as bagging, boosting and gradient boosting can be used to handle imbalanced classes. For example, you train N models that use M instances of minority class and N different sets of instances of majority class. Each set would have the M number of instances. Each model can be trained with balanced classes.</p><p><strong>6) Penalizing models</strong></p><p>By modifying a cost function, you can mitigate the imbalanced classes problem. You can penalize a model by adding an additional cost for wrong classifications of the minority class. The penalties can make the model to pay more attention to the instances of the minority class. For example, penalized SVM can be used for this problem.</p><p><strong>Conclusion</strong></p><p>We have looked at many useful techniques to deal with imbalanced classes. Of course, there are more useful approaches that are not covered in this post. The listed approaches in this post can be considered as a good starting-point to solve this problem. For many cases, you might have to be more creative more than just applying an approach. Why don’t you try different approaches and find a best one for your model and data? How about trying different algorithms? For example, decision trees often work well on imbalanced classes. Why don’t you combine different approaches?</p><p>Good luck!</p><p>My blog link: <a href="http://csmoon-ml.com/index.php/2018/12/15/6-techniques-to-deal-with-imbalanced-classes-in-machine-learning/">http://csmoon-ml.com/index.php/2018/12/15/6-techniques-to-deal-with-imbalanced-classes-in-machine-learning/</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9133f2906e86" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Simple Linear Regression]]></title>
            <link>https://medium.com/@csmoon/simple-linear-regression-fed2a7affe16?source=rss-e3e0d9600473------2</link>
            <guid isPermaLink="false">https://medium.com/p/fed2a7affe16</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[linear-regression]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Changsung Moon]]></dc:creator>
            <pubDate>Fri, 11 Aug 2017 17:05:59 GMT</pubDate>
            <atom:updated>2017-08-14T20:30:52.188Z</atom:updated>
            <content:encoded><![CDATA[<p>The difference between simple linear regression and multiple linear regression is that simple linear regression has only one independent variable whereas multiple linear regression has more than one independent variable. In this post, I will discuss simple linear regression.</p><p>In linear regression, the dependent variable is continuous, and independent variables can be continuous or discrete. The simple linear regression model is represented by the following equations:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/192/1*5_GDEMhWa6I78u8asoW78g.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/164/1*So-3ekR7I7Oc5Xhm0t4uhA.png" /></figure><p>where<em> theta 0 </em>is the intercept, <em>theta 1</em> is the slope of the line, and <em>epsilon</em> is the error term. The regression function <em>f(x)</em> is used to predict the value of target variable <em>y</em> based on the predictor variable <em>x</em>. <em>epsilon</em> indicates a difference between the estimated value <em>f(x)</em> and the actual value <em>y</em>. The regression coefficients, <em>theta 0</em> and <em>theta 1</em>, are parameters in the regression model. We find the best fit line by adjusting these parameters.</p><p>First of all, let’s generate random data that shows a linear relationship between an input <em>x</em> and an output <em>y</em> as follows:</p><pre>%matplotlib inline<br>import numpy as np<br>from pylab import *</pre><pre>x = np.random.normal(3.0, 1.0, 20)<br>y = x + np.random.normal(2.0, 0.4, 20)</pre><pre>scatter(x, y)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/362/1*bFfu6Zy02q1N7vf_NbZ7fg.png" /></figure><p>The following code shows the results of linear regression by using <em>scipy.stats.linregress </em>python library:</p><pre>import matplotlib.pyplot as plt<br>from scipy import stats</pre><pre>def predict(x):<br>    return slope * x + intercept</pre><pre>slope, intercept, r_value, p_value, std_err = stats.linregress(x, y)</pre><pre>fitLine = predict(x)</pre><pre>plt.scatter(x, y)<br>plt.plot(x, fitLine, c = &#39;r&#39;)<br>plt.show()</pre><p>In this case,</p><p><em>theta 0</em> is the intercept that is 1.9813276901908634.</p><p><em>theta 1</em> is the slope that is 0.95614427225069154.</p><p>Not surprisingly, the R-squared value shows a really good fit:</p><pre>r_value ** 2</pre><pre>0.89459544947919656</pre><p>The following plot shows the regression line with the intercept and the slope :</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/362/1*Jm-X6O_m8veqUxAyygt7Eg.png" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=fed2a7affe16" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[What is Regression?]]></title>
            <link>https://medium.com/@csmoon/what-is-regression-e09d6f6a9b6d?source=rss-e3e0d9600473------2</link>
            <guid isPermaLink="false">https://medium.com/p/e09d6f6a9b6d</guid>
            <category><![CDATA[regression]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Changsung Moon]]></dc:creator>
            <pubDate>Wed, 09 Aug 2017 04:44:58 GMT</pubDate>
            <atom:updated>2017-08-14T20:31:11.533Z</atom:updated>
            <content:encoded><![CDATA[<p>Regression is a statistical process for estimating the relationships between a dependent variable (‘target’) and one or more independent variables (‘feature’ or ‘predictor’). The dependent variable (usually denoted by <strong><em>y</em></strong>) is the output, which is the quantity of interest. Regression assumes that <strong><em>y</em></strong> can be predicted from the independent variable (usually denoted by <strong><em>x</em></strong>), which is the input. The estimation target is a function of the independent variables called the regression function.</p><p><em>[Example of training set: Housing prices]</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/456/1*DhjwxR9WIMJQiEexBX-SuQ.png" /></figure><ul><li><strong><em>x</em></strong> = independent variable / feature</li><li><strong><em>y</em></strong> = dependent variable / target</li></ul><p>Regression analysis is also used to understand which among the independent variables are related to the dependent variable. It helps data analysts to find the best set of variables to be used for building predictive models.</p><p>There are various kinds of regression techniques, and these are mostly driven by three metrics: 1) the number of independent variables, 2) type of dependent variables and 3) shape of the regression line. The followings are the most commonly used regressions:</p><ul><li>Linear Regression</li><li>Logistic Regression</li><li>Polynomial Regression</li><li>Stepwise Regression</li><li>Ridge Regression</li><li>Lasso Regression</li><li>ElasticNet Regression</li></ul><p>We will discuss details of each regression in future posts.</p><p><em>References</em></p><ul><li><a href="https://en.wikipedia.org/wiki/Regression_analysis">Regression analysis - Wikipedia</a></li><li><a href="https://www.analyticsvidhya.com/blog/2015/08/comprehensive-guide-regression/">Regression Techniques in Machine Learning</a></li><li><a href="http://www.investopedia.com/terms/r/regression.asp">Regression</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e09d6f6a9b6d" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>