Logistic Regression — How AI Says Yes or No

Series 3, Part 2: ML Algorithms, Simply Explained

In the last part, we taught AI to predict a number — a house price, a salary, a revenue figure.

But what if you don’t need a number?

What if you just need a yes or no?

• Will this customer buy or not?
• Is this email spam or not?
• Will this loan applicant default or not?
• Does this scan show cancer or not?

That’s where Logistic Regression comes in.

And despite the word “regression” in the name — it’s actually a classification algorithm. One of the most widely used in all of machine learning.

What Is Logistic Regression?

Logistic Regression is a supervised learning algorithm that predicts the probability of a binary outcome — yes or no, 1 or 0 — based on one or more input features.

It doesn’t predict a number. It predicts a probability between 0 and 1 — and then converts that probability into a decision.

• Probability > 0.5 → Yes (positive class)
• Probability ≤ 0.5 → No (negative class)

Examples:

• P(customer buys) = 0.82 → Yes, likely to buy
• P(email is spam) = 0.91 → Yes, spam
• P(loan defaults) = 0.23 → No, likely safe
• P(cancer present) = 0.67 → Yes, refer for further testing

Why Not Just Use Linear Regression?

Great question. If Linear Regression predicts numbers — why not use it and say anything above 0.5 is “yes”?

Let’s see why that breaks.

Imagine predicting whether a customer buys (1) or not (0) based on time spent on the website.

Linear Regression draws a straight line through the data:

Problems:

• The line can predict values below 0 or above 1 — which make no sense as probabilities
• A customer spending 10 minutes might get a predicted value of 1.4 — meaningless
• A customer spending 0 minutes might get -0.2 — impossible

What we need is a function that:

• Takes any input value
• Always outputs something between 0 and 1
• Creates an S-shaped curve that fits binary data naturally

That function is the Sigmoid Function-which we covered in Series 2, Part 4

The Sigmoid Function — The Heart of Logistic Regression

The Sigmoid function transforms any number into a value between 0 and 1:

σ(z) = 1 / (1 + e^(-z))

Where z is the linear combination of inputs and weights — exactly like Linear Regression.

What it looks like:

This S-shaped curve is called the sigmoid curve or logistic curve — which is where the algorithm gets its name.

Key properties:

• As z → very large → output approaches 1
• As z → very small (negative) → output approaches 0
• At z = 0 → output = exactly 0.5 (the decision boundary)

How Logistic Regression Works — Step by Step

Step 1 — Calculate the linear combination

Just like Linear Regression:

z = W1X1 + W2X2 + … + WnXn + b

Step 2 — Apply the Sigmoid function

Probability = 1 / (1 + e^(-z))

Step 3 — Apply a threshold

If Probability > 0.5 → predict Yes (1) If Probability ≤ 0.5 → predict No (0)

Step 4 — Learn the weights Using Binary Cross-Entropy loss (Series 2, Part 9) and gradient descent (Series 2, Part 6) — the model adjusts weights to minimize prediction error.

A Real Example — Email Spam Detection

You’re building a spam filter. Your features are:

• X1 = contains the word “prize” (1 = yes, 0 = no)
• X2 = number of exclamation marks
• X3 = sender in contact list (1 = yes, 0 = no)

After training, the model learns these weights:

z = (2.5 × X1) + (0.8 × X2) + (−3.1 × X3) + 0.5

Now an email arrives:

• Contains “prize” → X1 = 1
• Has 3 exclamation marks → X2 = 3
• Sender NOT in contacts → X3 = 0

z = (2.5 × 1) + (0.8 × 3) + (−3.1 × 0) + 0.5 z = 2.5 + 2.4 + 0 + 0.5 = 5.4

Probability = 1 / (1 + e^(-5.4)) = 0.996

99.6% probability of spam → Mark as spam ✅

Get Pragathi Velu’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

Now a legitimate email:

• No “prize” → X1 = 0
• No exclamation marks → X2 = 0
• Sender IS in contacts → X3 = 1

z = (2.5 × 0) + (0.8 × 0) + (−3.1 × 1) + 0.5 z = 0 + 0 − 3.1 + 0.5 = −2.6

Probability = 1 / (1 + e^(-(-2.6))) = 0.069

6.9% probability of spam → Not spam ✅

The Decision Boundary

The decision boundary is the line (or curve) that separates the two classes.

For Logistic Regression — this boundary is always a straight line (in 2D) or a flat plane (in higher dimensions).

This is both its strength and limitation:

✅ Simple, clean separation — easy to interpret ❌ Can’t handle cases where the boundary is curved or complex

If the data isn’t linearly separable — Logistic Regression struggles. That’s when you need Decision Trees, SVM, or neural networks.

Adjusting the Threshold

The default threshold is 0.5 — but you can change it based on business needs.

Remember from Series 2, Part 8 — the Precision vs Recall tradeoff:

Lower the threshold (e.g. 0.3):

• More positives predicted → Higher Recall
• More false alarms → Lower Precision
• Use when missing a positive is costly (cancer detection)

Raise the threshold (e.g. 0.7):

• Fewer positives predicted → Lower Recall
• Fewer false alarms → Higher Precision
• Use when false alarms are costly (legal decisions)

This flexibility is one of Logistic Regression’s most practical advantages.

Multi-Class Logistic Regression

What if you have more than two classes?

Say you’re classifying customer sentiment: Positive, Neutral, or Negative.

Softmax Regression (also called Multinomial Logistic Regression) extends Logistic Regression to handle multiple classes.

Instead of one sigmoid output — it produces a probability for each class:

• P(Positive) = 0.70
• P(Neutral) = 0.20
• P(Negative) = 0.10

All probabilities sum to 1. The highest wins → Positive sentiment.

This uses the Softmax activation function — which we covered in Series 2, Part 4.

Real World Business Use Cases

Credit Risk Assessment Input: income, credit history, debt level, employment status Output: probability of loan default → approve or reject

Medical Diagnosis Input: patient symptoms, test results, age, BMI Output: probability of disease → refer for treatment or not

Email Spam Filtering Input: email content features, sender reputation Output: probability of spam → inbox or spam folder

Customer Churn Prediction Input: usage frequency, support tickets, subscription age Output: probability of cancellation → trigger retention campaign

HR — Candidate Screening Input: resume features, experience, skills match Output: probability of successful hire → shortlist or reject

Linear Regression vs Logistic Regression

When to Use Logistic Regression

✅ Use it when:

• Output is binary (yes/no, 1/0)
• You need probability estimates — not just a hard decision
• You need an interpretable model — weights show feature importance
• Dataset is small to medium
• Classes are roughly linearly separable
• Speed matters — Logistic Regression trains very fast

❌ Don’t use it when:

• Output has more than 2 classes (use Softmax or other algorithms)
• Decision boundary is non-linear and complex
• Features have complex interactions
• You have very high-dimensional data (images, text)

Pros and Cons

Quick Recap

Next up → Series 3, Part 3: Decision Trees — how AI learns to make decisions by asking a series of yes/no questions — and why this algorithm is one of the most intuitive and interpretable in all of machine learning.