Logistic Regression
Classification
Python

Classification

Asif Ahmed Khan

6 min read3 days ago

Classification is also supervised learning algorithm that works withlabel data. Unlikely linear regression which predicts conitnuous numerical value, classification predicts the categorical value based on the data.

For example we have a label data on students based on the results of their exams. The results are either pass or fail, so with the help of classification we can predict whether the value will be pass or fail.

Classification is divided into following algorithms such as logistic regression, random forest, k nearest neighbour and decision tree.

1. Logistic Regression

Logistic regression is basically supervised learning algorithm that is used to classify data based on their category. Unlike linear regression which predicts continuous numerical value, it predicts categorical outcome like spam/not spam, pass/fail, yes/no.

It uses sigmoid function that converts the numerical output into probability like value between 0 and 1. Formula of sigmoid function is:

z = Input value calculated by the model.

e = Mathemcatical constant (approx. 2.718)

σ(z) = Output of the sigmoid function.

Get Asif Ahmed Khan’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

Following are the examples that uses logistic regression to solve problems.

Example 1: Student Pass / Fail Prediction

Study Hours          Attendence %        Result
1                    40                  Fail(0)
2                    45                  Fail(0)
3                    50                  Fail(0)
4                    55                  Fail(0)
5                    70                  Pass(1)
6                    75                  Pass(1)
7                    80                  Pass(1)
8                    90                  Pass(1)

import pandas as pd
import matplotlib.pyplot as plt
import numpy as np

from sklearn.linear_model import LogisticRegression

# 1. Create Dataset using Pandas
data = {
    "Study Hours": [1, 2, 3, 4, 5, 6, 7, 8],
    "Attendance": [40, 45, 50, 55, 70, 75, 80, 90],
    "Result": [0, 0, 0, 0, 1, 1, 1, 1]
}

df = pd.DataFrame(data)

print("Dataset:")
print(df)

# 2. Separate Features and Target
X = df[["Study Hours", "Attendance"]]
y = df["Result"]

# 3. Create Logistic Regression Model
model = LogisticRegression()

# 4. Train Model
model.fit(X, y)

# 5. Predict New Student
new_student = pd.DataFrame({
    "Study Hours": [6],
    "Attendance": [75]
})

prediction = model.predict(new_student)
probability = model.predict_proba(new_student)

print("\nPrediction:", prediction[0])
print("Probabilities:", probability)

if prediction[0] == 1:
    print("Student will Pass")
else:
    print("Student will Fail")

# 6. Visualization using Matplotlib

plt.figure(figsize=(9, 6))

# Fail students
fail_students = df[df["Result"] == 0]

# Pass students
pass_students = df[df["Result"] == 1]

plt.scatter(
    fail_students["Study Hours"],
    fail_students["Attendance"],
    color="red",
    label="Fail",
    s=100
)

plt.scatter(
    pass_students["Study Hours"],
    pass_students["Attendance"],
    color="green",
    label="Pass",
    s=100
)

# New student
plt.scatter(
    new_student["Study Hours"],
    new_student["Attendance"],
    color="blue",
    marker="*",
    s=250,
    label="New Student"
)

# Graph labels
plt.xlabel("Study Hours")
plt.ylabel("Attendance (%)")
plt.title("Student Pass/Fail Classification")

plt.legend()
plt.grid(True)

plt.show()

Example 2: Loan Approval Prediction

Income (Thousands) Credit      Score Loan
20                 400         Reject (0)
25                 420         Reject (0)
30                 450         Reject (0)
35                 480         Reject (0)
50                 650         Approve (1)
60                 700         Approve (1)
70                 750         Approve (1)
80                 800         Approve (1)

import pandas as pd
import matplotlib.pyplot as plt

from sklearn.linear_model import LogisticRegression

# 1. Create Dataset using Pandas
data = {
    "Income (Thousands)": [20, 25, 30, 35, 50, 60, 70, 80],
    "Credit Score": [400, 420, 450, 480, 650, 700, 750, 800],
    "Loan Status": [0, 0, 0, 0, 1, 1, 1, 1]
}

df = pd.DataFrame(data)

print("Loan Dataset:")
print(df)

# 2. Separate Features and Target
X = df[["Income (Thousands)", "Credit Score"]]
y = df["Loan Status"]

# 3. Create Logistic Regression Model
model = LogisticRegression()

# 4. Train Model
model.fit(X, y)

# 5. Predict New Applicant
applicant = pd.DataFrame({
    "Income (Thousands)": [65],
    "Credit Score": [720]
})

prediction = model.predict(applicant)
probability = model.predict_proba(applicant)

print("\nLoan Status:",
      "Approved" if prediction[0] == 1 else "Rejected")

print("Probabilities:", probability)

# 6. Visualization using Matplotlib

plt.figure(figsize=(9, 6))

# Rejected applicants
rejected = df[df["Loan Status"] == 0]

# Approved applicants
approved = df[df["Loan Status"] == 1]

# Plot rejected applicants
plt.scatter(
    rejected["Income (Thousands)"],
    rejected["Credit Score"],
    color="red",
    label="Rejected",
    s=100
)

# Plot approved applicants
plt.scatter(
    approved["Income (Thousands)"],
    approved["Credit Score"],
    color="green",
    label="Approved",
    s=100
)

# Plot new applicant
plt.scatter(
    applicant["Income (Thousands)"],
    applicant["Credit Score"],
    color="blue",
    marker="*",
    s=250,
    label="New Applicant"
)

plt.xlabel("Income (Thousands)")
plt.ylabel("Credit Score")
plt.title("Loan Approval Classification")

plt.legend()
plt.grid(True)

plt.show()

# 7. Probability Visualization

plt.figure(figsize=(7, 5))

plt.bar(
    ["Rejected", "Approved"],
    probability[0],
    color=["red", "green"]
)

plt.ylabel("Probability")
plt.title("Loan Approval Prediction Probability")
plt.ylim(0, 1)

plt.show()

Example 3: Email Spam Detection

Spam Words     Suspicious Links      Result
8              5                     Spam (1)
7              4                     Spam (1)
9              6                     Spam (1)
6              5                     Spam (1)
1              0                     Not Spam (0)
2              1                     Not Spam (0)
1              1                     Not Spam (0)
3              0                     Not Spam (0)

import pandas as pd
import matplotlib.pyplot as plt

from sklearn.linear_model import LogisticRegression

# 1. Create Dataset using Pandas
data = {
    "Spam Words": [8, 7, 9, 6, 1, 2, 1, 3],
    "Suspicious Links": [5, 4, 6, 5, 0, 1, 1, 0],
    "Email Status": [1, 1, 1, 1, 0, 0, 0, 0]
}

df = pd.DataFrame(data)

print("Email Dataset:")
print(df)

# 2. Separate Features and Target
X = df[["Spam Words", "Suspicious Links"]]
y = df["Email Status"]

# 3. Create Logistic Regression Model
model = LogisticRegression()

# 4. Train Model
model.fit(X, y)

# 5. Predict New Email
new_email = pd.DataFrame({
    "Spam Words": [7],
    "Suspicious Links": [5]
})

prediction = model.predict(new_email)
probability = model.predict_proba(new_email)

print("\nEmail Status:",
      "Spam" if prediction[0] == 1 else "Not Spam")

print("Probabilities:", probability)

# 6. Visualization using Matplotlib

plt.figure(figsize=(9, 6))

# Not Spam Emails
not_spam = df[df["Email Status"] == 0]

# Spam Emails
spam = df[df["Email Status"] == 1]

# Plot Not Spam Emails
plt.scatter(
    not_spam["Spam Words"],
    not_spam["Suspicious Links"],
    color="green",
    label="Not Spam",
    s=100
)

# Plot Spam Emails
plt.scatter(
    spam["Spam Words"],
    spam["Suspicious Links"],
    color="red",
    label="Spam",
    s=100
)

# Plot New Email
plt.scatter(
    new_email["Spam Words"],
    new_email["Suspicious Links"],
    color="blue",
    marker="*",
    s=250,
    label="New Email"
)

plt.xlabel("Number of Spam Words")
plt.ylabel("Number of Suspicious Links")
plt.title("Email Spam Classification")

plt.legend()
plt.grid(True)

plt.show()

# 7. Probability Visualization

plt.figure(figsize=(7, 5))

plt.bar(
    ["Not Spam", "Spam"],
    probability[0],
    color=["green", "red"]
)

plt.ylabel("Probability")
plt.title("Email Classification Probability")
plt.ylim(0, 1)

plt.show()

Example 4: Realistic Spam Detector


import pandas as pd
import matplotlib.pyplot as plt
import re

from sklearn.linear_model import LogisticRegression

# 1. Create Dataset using actual email text

data = {
    "Email": [
        "Congratulations! You won a free prize. Claim now!",
        "Win money now! Click here to get your reward.",
        "Free offer! You are selected for a cash prize.",
        "Claim your lottery reward immediately.",
        "Hello, how are you doing today?",
        "Please find the meeting schedule attached.",
        "Can we discuss the project tomorrow?",
        "Your assignment submission is due Friday."
    ],

    "Links": [
        "http://free-prize.com",
        "http://win-money.com",
        "http://cash-reward.com",
        "http://lottery-win.com",
        "https://google.com",
        "",
        "https://example.com",
        ""
    ],

    "Email Status": [1, 1, 1, 1, 0, 0, 0, 0]
}

df = pd.DataFrame(data)

print("Original Email Dataset:")
print(df)


# 2. Define Spam Words

spam_words = [
    "free", "win", "won", "prize",
    "money", "claim", "reward",
    "lottery", "offer", "cash"
]


# 3. Extract Numerical Features from Text

def count_spam_words(email):

    email = email.lower()

    words = re.findall(r'\b\w+\b', email)

    count = sum(
        1 for word in words
        if word in spam_words
    )

    return count


def count_links(links):

    urls = re.findall(
        r'https?://\S+|www\.\S+',
        links
    )

    return len(urls)


def count_suspicious_links(links):

    suspicious_domains = [
        "free-prize",
        "win-money",
        "cash-reward",
        "lottery-win"
    ]

    links = links.lower()

    return sum(
        1 for domain in suspicious_domains
        if domain in links
    )


# 4. Convert Text into Numerical Data

df["Spam Words"] = df["Email"].apply(
    count_spam_words
)

df["Total Links"] = df["Links"].apply(
    count_links
)

df["Suspicious Links"] = df["Links"].apply(
    count_suspicious_links
)

print("\nExtracted Features:")
print(df)


# 5. Separate Features and Target

X = df[
    ["Spam Words", "Total Links", "Suspicious Links"]
]

y = df["Email Status"]


# 6. Create and Train Logistic Regression Model

model = LogisticRegression()

model.fit(X, y)


# 7. Enter a New Email

new_email = "Congratulations! You won free money. Claim your prize now!"

new_links = "http://free-prize.com"

# Extract features from new email

new_features = pd.DataFrame({
    "Spam Words": [count_spam_words(new_email)],
    "Total Links": [count_links(new_links)],
    "Suspicious Links": [
        count_suspicious_links(new_links)
    ]
})

print("\nNew Email Features:")
print(new_features)


# 8. Predict Email Status

prediction = model.predict(new_features)

probability = model.predict_proba(new_features)

print("\nEmail Status:",
      "Spam" if prediction[0] == 1 else "Not Spam")

print("Not Spam Probability:", probability[0][0])

print("Spam Probability:", probability[0][1])


# 9. Visualization

plt.figure(figsize=(9, 6))

not_spam = df[df["Email Status"] == 0]
spam = df[df["Email Status"] == 1]

plt.scatter(
    not_spam["Spam Words"],
    not_spam["Suspicious Links"],
    color="green",
    label="Not Spam",
    s=100
)

plt.scatter(
    spam["Spam Words"],
    spam["Suspicious Links"],
    color="red",
    label="Spam",
    s=100
)

plt.scatter(
    new_features["Spam Words"],
    new_features["Suspicious Links"],
    color="blue",
    marker="*",
    s=250,
    label="New Email"
)

plt.xlabel("Number of Spam Words")
plt.ylabel("Number of Suspicious Links")

plt.title("Email Spam Classification")

plt.legend()
plt.grid(True)
plt.show()


# 10. Probability Visualization

plt.figure(figsize=(7, 5))

plt.bar(
    ["Not Spam", "Spam"],
    probability[0],
    color=["green", "red"]
)

plt.ylabel("Probability")
plt.title("Email Classification Probability")

plt.ylim(0, 1)

plt.show()

Classification

1. Logistic Regression

Get Asif Ahmed Khan’s stories in your inbox

Example 1: Student Pass / Fail Prediction

Example 2: Loan Approval Prediction

Example 3: Email Spam Detection

Example 4: Realistic Spam Detector

Written by Asif Ahmed Khan

No responses yet

More from Asif Ahmed Khan

Data Exploration using Pandas and Matplotlib

In this article we will learn how to utilize pandas and matplotlib to do basic data exploration from the csv dataset.

Linear Regression

Linear Regression is based on supervised learning algorithm that is basically used to predict the trend based the label data. It finds the…

Recommended from Medium

Generative Adversarial Networks

First introduced by Ian Goodfellow and his colleagues in 2014, GANs revolutionized the field of generative AI. Two deep neural networks…

Evaluation Metrics for Classification and Regression in Machine Learning: A Complete Guide with…

Understanding Accuracy, Precision, Recall, F1-Score, Confusion Matrix, ROC-AUC, MAE, MSE, RMSE, and R² with Real-World Examples.

Building a Spam Message Classifier Using Machine Learning:

Introduction

Beyond Predictions: Understanding Classification and Regression Metrics

Classification and regression metrics are essential concepts in machine learning that help us evaluate how well a model performs…

Machine Learning Made Simple: A Complete Guide to the Machine Learning Pipeline with Python

From Raw Data to Intelligent Predictions — A Beginner-Friendly Guide

Understanding Classification and Regression Metrics with Real-Life Examples

Introduction

No responses yet

More from Asif Ahmed Khan

Data Exploration using Pandas and Matplotlib

In this article we will learn how to utilize pandas and matplotlib to do basic data exploration from the csv dataset.

Linear Regression

Linear Regression is based on supervised learning algorithm that is basically used to predict the trend based the label data. It finds the…

Recommended from Medium

Generative Adversarial Networks

First introduced by Ian Goodfellow and his colleagues in 2014, GANs revolutionized the field of generative AI. Two deep neural networks…

Evaluation Metrics for Classification and Regression in Machine Learning: A Complete Guide with…

Understanding Accuracy, Precision, Recall, F1-Score, Confusion Matrix, ROC-AUC, MAE, MSE, RMSE, and R² with Real-World Examples.

Building a Spam Message Classifier Using Machine Learning:

Introduction

Beyond Predictions: Understanding Classification and Regression Metrics

Classification and regression metrics are essential concepts in machine learning that help us evaluate how well a model performs…

Machine Learning Made Simple: A Complete Guide to the Machine Learning Pipeline with Python

From Raw Data to Intelligent Predictions — A Beginner-Friendly Guide

Understanding Classification and Regression Metrics with Real-Life Examples

Introduction