<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Prateek Panda on Medium]]></title>
        <description><![CDATA[Stories by Prateek Panda on Medium]]></description>
        <link>https://medium.com/@timbaktoo?source=rss-582284e7f9ed------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*RqCZDU4EXXj8qFTS</url>
            <title>Stories by Prateek Panda on Medium</title>
            <link>https://medium.com/@timbaktoo?source=rss-582284e7f9ed------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 17:02:45 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@timbaktoo/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Evaluation Metrics for Classification Problems]]></title>
            <link>https://timbaktoo.medium.com/evaluation-metrics-for-classification-problems-9babe6193001?source=rss-582284e7f9ed------2</link>
            <guid isPermaLink="false">https://medium.com/p/9babe6193001</guid>
            <category><![CDATA[binary-classification]]></category>
            <category><![CDATA[confusion-matrix]]></category>
            <category><![CDATA[binary-classifier]]></category>
            <category><![CDATA[classification-models]]></category>
            <category><![CDATA[classification-metrics]]></category>
            <dc:creator><![CDATA[Prateek Panda]]></dc:creator>
            <pubDate>Mon, 26 Apr 2021 08:48:21 GMT</pubDate>
            <atom:updated>2021-04-26T08:48:21.556Z</atom:updated>
            <content:encoded><![CDATA[<p>If you are reading this, perhaps you already have built a classification model(logistic regression, decision tree, neural network… you name it). For the sake of discussion let’s say it’s a binary classifier. Before I comment on which metrics to use, I should talk a bit about <em>confusion matrix.</em></p><p><strong>Confusion Matrix:</strong> A table that is used to describe data where the TRUE values are known. This matrix is relatively easy to understand, but certain terminologies can be a bit confusing.</p><p>Let’s consider an example. Since I’m not the domain expert, I will refrain from real life use cases. However, the agenda is breaking down a ML (classification) model and understand all its attributes.</p><p>Hypothetically, we have the IMDB database and we have selected features for the ML model to classify movies as <strong><em>good / bad</em></strong>. Performing this operation, will lead to <em>four </em>types of outcomes:</p><p><strong>Outcome 1: TRUE POSITIVE</strong></p><p>A movie is classified as <em>good</em> by the model and is actually <em>good</em>.</p><p>Model predicted an observation belongs to a class, and it actually does.</p><p><strong>Outcome 2: TRUE NEGATIVE</strong></p><p>A movie is classified as <em>bad</em> by the model and is actually <em>bad</em>.</p><p>Model predicted that an observation doesn’t belong to a class (good), and it actually doesn’t.</p><p><strong>Outcome 3: FALSE POSITIVE</strong></p><p>A movie is classified as <em>good</em> by the model, but is actually <em>bad</em>.</p><p>Model predicts that the observation belongs to a class, but it actually doesn’t belong to that class.</p><p><strong>Outcome 4: FALSE NEGATIVE</strong></p><p>A movie is classified as <em>bad</em> by the model, but is actually <em>good</em>.</p><p>Model predicts that the observation doesn’t belong to a class, but it actually does.</p><p>Let’s put this to test. Our model for IMDB gave this result:</p><p>N = total observations/ in our case list of movies to be classified/total count</p><p>N = 165</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/542/1*wf_6Au9eD3njjh1kMmor7w.jpeg" /><figcaption>Confusion Matrix</figcaption></figure><p><strong>Inference from the model:</strong></p><p>** Out of 165 predictions, the classifier predicted:</p><p><strong>GOOD : 110 times &lt;TP+FP&gt;</strong></p><p>But, in reality:</p><p><strong>105 are GOOD and 60 are BAD &lt;TP+FN&gt;</strong></p><p><strong>** </strong>So, <strong><em>accuracy</em></strong> is a metric which focuses on TP and TN: It basically states how often the classifier is <em>correct</em>?</p><p><strong>Accuracy = (TP+TN)/ (TP+FP+FN+TN)</strong></p><p>Accuracy = 100 + 50/ 165 = 0.91</p><p>** Now, misclassification rate: how often is the classification <em>wrong</em>? Also known as the <strong><em>error rate</em></strong><em>.</em></p><p><strong>Error Rate = (FP + FN) / (TP + FP + FN + TN)</strong></p><p>Error Rate = 10 + 5 / 165 = 0.09</p><p>Let’s look at some other metrics.</p><p>** <strong>RECALL | SENSITIVITY METRIC | TRUE POSITIVE RATE :</strong></p><p>When it’s actually <strong>GOOD, </strong>how often does the model predict <strong>GOOD.</strong></p><p><strong>Recall = TP / (TP + FN)</strong></p><p>TP: Correctly predicted by the model | (TP + FN): Total number of <strong>GOOD</strong></p><p>Recall = 100 / 105 = 0.95</p><p>It basically gives the fraction the model correctly identified as <strong>GOOD</strong> out of total <strong>GOOD</strong>. So, <em>recall </em>is a measure which we should maximize for the performance of our model.</p><p>** PRECISION</p><p>When the model predicted <strong>GOOD, </strong>how often is it correct.</p><p><strong>Precision =</strong> <strong>TP / (TP + FP)</strong></p><p>Precision = 100 + (100 + 10) = 0.90</p><p>TP: Correctly predicted by the model | (TP + FP): Total number of <strong>GOOD </strong>predicted by the model</p><p>In some cases we are pretty sure we want to maximize <em>recall</em> or <em>precision</em>, one at the cost of other. Unfortunately we can’t have it both ways. Increasing precision will lead to reduction of recall and vice-versa. So, there is a <em>trade-off</em> between recall and precision based on a probability threshold(this is domain specific).Therefore we need to compare the different models with different precision-recall values.</p><p>It’s often convenient to combine precision-recall into a single metric called <strong>F1 Score.</strong></p><p><strong>F1 Score: </strong>It’s defined as the harmonic mean of the model’s precision and recall.</p><p><strong>F1 Score = 2 * (Precision * Recall) / (Precision + Recall)</strong></p><p>** <strong>ROC CURVE(</strong>Receiver Operating Characteristic<strong>)</strong></p><p><strong>x-axis: </strong>False Positive Rate&lt;specificity = 1-FPR&gt;</p><p><em>** specificity : True Negative Rate</em></p><p><strong>y-axis: </strong>True positive rate&lt;sensitivity&gt;</p><p>We can conclude that models that give curves closed to the top-left corner, indicate a better performance. The closer the curve comes to 45 degree line, the less accurate the test is.</p><p>Similarly, <strong>AUC </strong>(Area under the ROC curve) is a metric to understand the model performance. More the area (say 0.9 and above) better the model. However score above 0.8 is considered acceptable.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/632/1*b89bYKOEaVOlDYpTEHfK2A.jpeg" /></figure><p>Hope this makes sense to all who are dealing with classification problems. If you have any thoughts please drop down a comment! Happy learning:)</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9babe6193001" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>