AUC-ROC Curve in Machine Learning
Based on: StatQuest with Josh Starmer — ROC and AUC, Clearly Explained!(Special thanks)
ROC Curve (Receiver Operating Characteristic Curve) in Machine Learning
The ROC Curve is a graphical tool used to evaluate the performance of a binary classification model (e.g., Spam/Not Spam, Fraud/Not Fraud, Disease/No Disease).
NOTE : Know the Logistic regression then continue. And read it from both image and texts
Along the X-axis we have weights .
The blue dots represent obese mice …and the red dots represent mice that are not obese.
Same for probability with low .
One way to classify mice is to set a threshold at 0.5…and classify all mice with a probability of being obese > 0.5 as “obese”… , where as classify all mice with a probability of being obese <= 0.5 as “not obese”.
Now introduce weights of four new mice that we know are obese.
Now let’s talk about what happens when we use a different threshold for deciding if a sample is obese or not.
So instead of being overwhelmed with confusion matrices,
Receiver Operator Characteristic (ROC) graphs provide a simple way to summarize all of the information.
The y-axis shows the True Positive Rate, which is the same thing as Sensitivity.
What It Measures : Sensitivity measures the proportion of actual positive cases that are correctly identified by a model or test. The True Positive Rate tells you what proportion of obese samples were correctly classified.
Get AYUSHMAN SHARMA’s stories in your inbox
Join Medium for free to get updates from this writer.
The x-axis shows the False Positive Rate, which is the same thing as 1 Specificity.
Note for this Confusion Matrix every single sample that was not obese -was incorrectly classified as obese.
A point at 1, 1 means that even though we correctly classified all of the obese samples, we incorrectly classified all of the samples that were not obese.
Going back to the Logistic Regression, let’s increase the threshold so that all but the lightest sample are called obese.
Since the new point (0.75, 1) is to the left of the dotted green line, we know that the proportion of correctly classified samples that were obese (true positives) is greater than the proportion of the samples that were incorrectly classified as obese (false positives).
In other words, the new threshold for deciding if a sample is obese or not is better than the first one.
By plotting the True Positive Rate (TPR) against the False Positive Rate (FPR) for different classification thresholds, we obtain the ROC (Receiver Operating Characteristic) curve.
Then how can I choose the optimal threshold from the graph?
Method 1 (Most common): Choose the point closest to the top-left corner.
Method 2: Maximize Youden’s Index (Very popular)
Compute
Calculate it for every threshold.
The threshold with the largest value is often chosen.
— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —
What is AUC?
AUC stands for Area Under the ROC Curve.
Instead of looking at the entire curve visually, we calculate the area underneath it.
Continuing with the above example :
So if the red ROC curve represented Logistic Regression and the blue ROC curve represented a Random Forest, you would use the Logistic Regression.
Although ROC graphs are drawn using True Positive Rates and False Positive Rates to summarize confusion matrices, there are other metrics that attempt to do the same thing.
For example, people often replace the False Positive Rate with Precision.
This is because Precision does not include the number of True Negatives in its calculation, and is not effected by the imbalance.
