<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Yi-xin on Medium]]></title>
        <description><![CDATA[Stories by Yi-xin on Medium]]></description>
        <link>https://medium.com/@yixinsun_56102?source=rss-462bc2ddb9aa------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*Kfc93nFnbqn-16HW.</url>
            <title>Stories by Yi-xin on Medium</title>
            <link>https://medium.com/@yixinsun_56102?source=rss-462bc2ddb9aa------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Tue, 17 Nov 2026 00:37:42 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@yixinsun_56102/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Understanding Generalization Error in Machine Learning]]></title>
            <link>https://medium.com/@yixinsun_56102/understanding-generalization-error-in-machine-learning-e6c03b203036?source=rss-462bc2ddb9aa------2</link>
            <guid isPermaLink="false">https://medium.com/p/e6c03b203036</guid>
            <category><![CDATA[concept]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Yi-xin]]></dc:creator>
            <pubDate>Sun, 28 Oct 2018 23:58:46 GMT</pubDate>
            <atom:updated>2018-10-29T01:12:48.542Z</atom:updated>
            <content:encoded><![CDATA[<p>What determines the model’s ability to react to new unseen data?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/300/1*jbhs_j9ty-Isr_-ScB7tJw.png" /></figure><h3>Definition</h3><p>Firstly, let’s define “generalization error”.</p><blockquote><em>In supervised learning applications in machine learning and statistical learning theory, generalization error (also known as the out-of-sample error) is a measure of how accurately an algorithm is able to predict outcome values for previously unseen data. </em><a href="https://en.wikipedia.org/wiki/Generalization_error#cite_note-1"><em>wikipedia</em></a></blockquote><p>Notice that the gap between predictions and observed data is induced by <strong>model inaccuracy</strong>, <strong>sampling error</strong>, and <strong>noise</strong>. Some of the errors are reducible but some are not. Choosing the right algorithm and tuning parameters could improve model accuracy, but we will never be able to make our predictions 100% accurate.</p><h3>Bias-variance decomposition</h3><p>An important way to understand generalization error is <strong>bias-variance decomposition</strong>.</p><p>Intuitively speaking, <strong>bias</strong> is the <strong>error rate</strong> in the world of big data. A model has a high bias when, for example, it fails to capture meaningful patterns in the data. Bias is measured by the differences between the <em>expected predicted values</em> and the <em>observed values</em>, in the dataset D when the prediction variables are at the level of x (X=x).</p><p>In contrast with bias, <strong>variance</strong> is an algorithm’s <strong>flexibility</strong> to learn patterns in the observed data. <strong>Variance</strong> is the amount that an algorithm will change if a <strong>different dataset</strong> is used. A model is of high variance when, for instance, it tries too hard that it not only captures the pattern of meaningful features but also that the meaningless error (<strong>overfitting</strong>).</p><h3>Mathematical Notations</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/940/1*Uw5dwZl5KOzqjlCZ0Gi3xQ.png" /></figure><h4>Using regression as an example, we have</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/940/1*OGlaIHZdIC6At6LwXqsDtg.png" /></figure><h4>Now we could decompose generalization error</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/940/0*hckGSCSQPkwPsStw.png" /></figure><h3>Interpretation</h3><p>Bias measures the deviation between the expected output of our model and the real values, so it indicates the <strong>fit of our model</strong>.</p><p>Variance measures the amount that the outputs of our model will change if a different dataset is used. It is the impacts of using different datasets.</p><p>Noise is the irreducible error, the <strong>lowest bound of generalization error</strong> for the current task that any model will not be able to get rid of, indicating the difficulty of this task.</p><p>These 3 components above determine the model’s ability to react to new unseen data rather than just the data that it was trained on.</p><h3>Bias-Variance Tradeoff</h3><p><em>Bias-Variance Tradeoff as a Function of Model Capacity</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/438/0*ShOHvbd8oIQJxnUu.png" /></figure><p><a href="https://djsaunde.wordpress.com/2017/07/17/the-bias-variance-tradeoff/">source of the figure</a></p><p>Generalization error could be measured by MSE. As the model capacity increases, the bias decreases as the model fits the training datasets better. However, the variance increases, as your model become sophisticated to fit more patterns of the current dataset, changing datasets (even if they come from the same distribution) would be impactful. As a data scientist, our challenge lies in finding the optimal capacity — where both bias and variance are low.</p><h3>Reference</h3><p>Z. Zhou. (2016). Machine Learning (Chinese Edition). Tsinghua University Press; 1st edition (January 1, 2016).</p><p>I am a master student in Data Science at University of San Francisco. I write blogs to explain data science concepts or issues that I think important or interesting.</p><p>Welcome to connect with me on <a href="https://www.linkedin.com/in/yixin-sun/">LinkedIn</a>!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e6c03b203036" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>