ML: Performance Metrics

In this article, I will explain you what the metrics are available in machine learning to measure the performance of the trained models.

Metrics in machine learning are measures used to evaluate the performance and effectiveness of machine learning models. They help quantify how well a model is performing and provide insights into its behavior.

Get Leela Jagadeeswar P’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

Here are some commonly used metrics in machine learning:

Performance Metrics for Classification:

Confusion Matrix: It is a table that gives the performance of a classification model by showing the counts of true positive (TP), true negative (TN), false positive (FP), and false negative (FN) predictions. It calculates various evaluation metrics such as accuracy, precision, recall, and F1 score.

Confusion Matrix [Image 2] (Image courtesy: My Photoshopped Collection)

• Accuracy: It measures the proportion of correctly classified instances. We should prefer this metric when the data is approximately balanced.
• Precision: It measures the proportion of true positives out of all predicted positives.

• Recall or Sensitivity: It can be calculated as True Positive or predictions that are actually true to the total number of positives, either correctly predicted as positive or incorrectly predicted as negative (true Positive and false negative).

• F1-score: It can be calculated as the harmonic mean of both precision and Recall, assigning equal weight to each of them.

• ROC curve (Receiver Operating Characteristic): Plots the true positive rate against the false positive rate at various classification thresholds.
• AUC-ROC (Area Under the ROC Curve): Represents the overall performance of a classification model across all possible classification thresholds.

from sklearn.metrics import confusion_matrix
from sklearn.metrics import accuracy_score, precision_score, recall_score, f1_score

confusion_matrix = confusion_matrix(y_test, predictions)
accuracy_score   = accuracy_score(y_test, predictions)
precision_score  = precision_score(y_test, predictions)
recall_score = recall_score(y_test, predictions)
f1_score = f1_score(y_test, predictions)

Performance Metrics for Regression:

• Mean Absolute Error (MAE): The average of the absolute differences between predicted and true values.
• Mean Absolute Percentage Error (MAPE): The average of the absolute percentage differences between predicted and true values.
• Mean Squared Error (MSE): The average of the squared differences between predicted and true values.
• Root Mean Squared Error (RMSE): The square root of the mean squared error, providing the same unit as the target variable.
• R-squared Score: The proportion of the variance in the target variable explained by the model.

from sklearn.metrics import mean_absolute_error, mean_squared_error, r2_score

mae  = mean_absolute_error(y_true, y_pred)
mse  = mean_squared_error(y_true, y_pred)
rmse = mean_squared_error(y_true, y_pred, squared=False)
r2_score = r2_score(y_true, y_pred)