SVM (Support Vector Machine)

10 min readDec 4, 2025

What is SVM?

Support Vector Machine (SVM) is a Supervised Machine Learning algorithm used for both Classification and Regression tasks. However, it is most commonly used for classification problems.

The primary objective of SVM is to find the best decision boundary (called a Hyperplane) that separates different classes with the maximum possible margin.

Unlike many algorithms that simply separate classes, SVM chooses the boundary that leaves the largest distance between the classes, making the model more robust and accurate.

Real-Life Example

Basic Concepts of SVM

SVM is built around three important concepts:

1. Hyperplane
2. Margin
3. Support Vectors

1. Hyperplane

A Hyperplane is the decision boundary that separates different classes.

In 2D, it is simply a line.
In 3D, it becomes a plane.
In higher dimensions, it is called a Hyperplane.

2. Margin

The Margin is the distance between the hyperplane and the nearest data points from each class.

3. Support Vectors

Support Vectors are the closest data points to the hyperplane.

Here, X and Y are the support vectors.

These points determine the position of the hyperplane.

If the support vectors change, the decision boundary also changes.

This is why the algorithm is called Support Vector Machine.

Goal of SVM

Why Maximum Margin?

Consider new data arriving later.

A model with a larger margin is more likely to classify future data correctly.

This improves generalization and reduces overfitting.

Mathematical Representation

The equation of a hyperplane is

Margin Formula

The optimization objective of SVM is to maximize the margin.

Hard Margin SVM

Hard Margin SVM is used when the data is perfectly separable.

Characteristics

No classification errors allowed
Works only on perfectly separated data

Advantages

High accuracy on clean datasets
Simple optimization

Disadvantages

Cannot handle noisy data
Very sensitive to outliers

Soft Margin SVM

Real-world datasets usually contain noise.

One point lies on the wrong side.

Soft Margin SVM allows a few misclassifications to improve overall performance.

It introduces a parameter called C.

C Parameter

The C parameter controls the trade-off between maximizing the margin and minimizing classification errors.

Large C

Fewer classification errors allowed
More complex decision boundary
Higher risk of overfitting

Small C

Allows some errors
Simpler decision boundary
Better generalization

Linear SVM

Used when the data can be separated using a straight line.

Examples

Spam detection
Text classification

Non-Linear SVM

Sometimes data cannot be separated using a straight line.

In such cases, SVM uses the Kernel Trick.

Types of SVM

Kernel Trick

The Kernel Trick maps data into a higher-dimensional feature space where it becomes linearly separable.

The kernel computes similarity without explicitly performing the transformation, making the process computationally efficient.

Types of Kernels

1. Linear Kernel

Applications

Text classification
High-dimensional datasets

2. Polynomial Kernel

Applications

Medium-complexity datasets
Non-linear relationships

3. Radial Basis Function (RBF) Kernel

Most commonly used kernel.

Get Codes With Pankaj’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

Applications

Image classification
Medical diagnosis
Complex non-linear problems

4. Sigmoid Kernel

Produces behavior similar to a neural network.

Gamma Parameter

Gamma is mainly used with the RBF Kernel.

High Gamma

Focuses on nearby points
Produces complex decision boundaries
Can cause overfitting

Low Gamma

Considers a larger region
Produces smoother boundaries
May lead to underfitting

Support Vector Machine (SVM) Example with a Real Dataset: Spam Email Detection

Spam detection is one of the most common real-world applications of SVM.

Problem Statement

Build a machine learning model that predicts whether an email is Spam or Not Spam (Ham).

Input: Email Text

Output:

Spam
Ham (Not Spam)

Download Dataset

# ============================================
# Spam Email Detection using SVM
# Complete Program
# ============================================

# Import Libraries
import pandas as pd

from sklearn.model_selection import train_test_split
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.svm import SVC
from sklearn.metrics import (
    accuracy_score,
    confusion_matrix,
    classification_report
)

# ============================================
# Load Dataset
# ============================================

df = pd.read_csv("spam_dataset.csv")
# https://raw.githubusercontent.com/Pankaj-Str/Complete-Python-Mastery/refs/heads/main/53%20DataSet/spam_dataset.csv

print("=" * 50)
print("First 5 Records")
print("=" * 50)
print(df.head())

print("\nDataset Shape")
print(df.shape)

print("\nClass Distribution")
print(df["label"].value_counts())

# ============================================
# Convert Labels
# ham = 0
# spam = 1
# ============================================

df["label"] = df["label"].map({
    "ham": 0,
    "spam": 1
})

# ============================================
# Input and Output
# ============================================

X = df["message"]
y = df["label"]

# ============================================
# Convert Text into Numbers
# ============================================

vectorizer = TfidfVectorizer(stop_words="english")

X = vectorizer.fit_transform(X)

# ============================================
# Split Dataset
# ============================================

X_train, X_test, y_train, y_test = train_test_split(
    X,
    y,
    test_size=0.20,
    random_state=42
)

print("\nTraining Samples :", X_train.shape[0])
print("Testing Samples  :", X_test.shape[0])

# ============================================
# Create SVM Model
# ============================================

model = SVC(kernel="linear")

# ============================================
# Train Model
# ============================================

model.fit(X_train, y_train)

print("\nModel Training Completed.")

# ============================================
# Prediction
# ============================================

prediction = model.predict(X_test)

# ============================================
# Accuracy
# ============================================

accuracy = accuracy_score(y_test, prediction)

print("\nAccuracy")

print(f"{accuracy*100:.2f}%")

# ============================================
# Confusion Matrix
# ============================================

print("\nConfusion Matrix")

print(confusion_matrix(y_test, prediction))

# ============================================
# Classification Report
# ============================================

print("\nClassification Report")

print(classification_report(y_test, prediction))

# ============================================
# Predict New Emails
# ============================================

emails = [

    "Congratulations! You won a FREE iPhone. Click here now.",

    "Hey, let's meet tomorrow at 5 PM.",

    "Claim your cash prize now. Limited offer.",

    "Can you send me today's notes?",

    "URGENT! Your bank account has been suspended. Verify now."

]

print("\nPrediction on New Emails")
print("=" * 50)

email_features = vectorizer.transform(emails)

results = model.predict(email_features)

for email, result in zip(emails, results):

    print("\nMessage :", email)

    if result == 1:
        print("Prediction : SPAM")
    else:
        print("Prediction : HAM")

# ============================================
# Predict Single Email
# ============================================

while True:

    print("\n")

    text = input("Enter Email (type exit to stop): ")

    if text.lower() == "exit":
        break

    sample = vectorizer.transform([text])

    result = model.predict(sample)

    if result[0] == 1:
        print("Prediction : SPAM")
    else:
        print("Prediction : HAM")

print("\nProgram Finished.")

Output

==================================================
First 5 Records
==================================================

  label                  message
0   ham                 Hey, are we meeting at 2 PM?
1   ham                 I'll call you after work.
2   ham                 Thanks for your help!
3  spam                Earn money fast from home.
4  spam                FREE iPhone waiting for you.

Dataset Shape
(6000, 2)

Class Distribution

ham     4682
spam    1318

Training Samples : 4800
Testing Samples  : 1200

Model Training Completed.

Accuracy

98.75%

Confusion Matrix

[[930   8]
 [  7 255]]

Classification Report

              precision    recall  f1-score   support

         ham       0.99      0.99      0.99
        spam       0.97      0.97      0.97

Prediction on New Emails

Message : Congratulations! You won a FREE iPhone. Click here now.
Prediction : SPAM

Message : Hey, let's meet tomorrow at 5 PM.
Prediction : HAM

Linear SVM

Used when data is linearly separable.

import numpy as np
import matplotlib.pyplot as plt
from sklearn import svm
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split

# Load iris dataset
iris = load_iris()
X = iris.data[:, :2]  # Using first two features for 2D visualization
y = iris.target

# Train-test split
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2)

# Linear SVM
model = svm.SVC(kernel='linear')
model.fit(X_train, y_train)

accuracy = model.score(X_test, y_test)

# Plot decision boundary
x_min, x_max = X[:, 0].min() - 1, X[:, 0].max() + 1
y_min, y_max = X[:, 1].min() - 1, X[:, 1].max() + 1

xx, yy = np.meshgrid(
    np.linspace(x_min, x_max, 300),
    np.linspace(y_min, y_max, 300)
)

Z = model.predict(np.c_[xx.ravel(), yy.ravel()])
Z = Z.reshape(xx.shape)

plt.figure(figsize=(8, 6))

# Decision boundary
plt.contourf(xx, yy, Z, alpha=0.3)

# Data points
plt.scatter(X[:, 0], X[:, 1], c=y, edgecolors='black')

plt.xlabel("Feature 1")
plt.ylabel("Feature 2")
plt.title(f"Linear SVM (Linearly Separable Case)\nAccuracy: {accuracy:.2f}")

plt.show()

Non-Linear SVM

Uses kernels to separate complex data.

Popular Kernels

RBF (Gaussian) → Most powerful

The RBF (Radial Basis Function) kernel is a function used by SVMs to measure similarity between points.
It creates a circular / radial decision boundary around important points.

import numpy as np
import matplotlib.pyplot as plt
from sklearn import svm
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split

# Load iris dataset
iris = load_iris()
X = iris.data[:, :2]  # Use first two features for 2D visualization
y = iris.target

# Train-test split
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2)

# RBF Kernel SVM
model = svm.SVC(kernel='rbf', gamma=0.5)
model.fit(X_train, y_train)

accuracy = model.score(X_test, y_test)

# Plot decision boundary
x_min, x_max = X[:, 0].min() - 1, X[:, 0].max() + 1
y_min, y_max = X[:, 1].min() - 1, X[:, 1].max() + 1

xx, yy = np.meshgrid(
    np.linspace(x_min, x_max, 300),
    np.linspace(y_min, y_max, 300)
)

Z = model.predict(np.c_[xx.ravel(), yy.ravel()])
Z = Z.reshape(xx.shape)

plt.figure(figsize=(8, 6))

# Decision boundary
plt.contourf(xx, yy, Z, alpha=0.3)

# Scatter plot of points
plt.scatter(X[:, 0], X[:, 1], c=y)

plt.xlabel("Feature 1")
plt.ylabel("Feature 2")
plt.title(f"SVM with RBF Kernel (Iris Dataset)\nAccuracy: {accuracy:.2f}")

plt.show()

Polynomial → Curve-based separation

Polynomial Kernel SVM is used when data cannot be separated using a straight line, but it can be separated using a curved boundary.

import numpy as np
import matplotlib.pyplot as plt
from sklearn import svm
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split

# Load iris dataset
iris = load_iris()
X = iris.data[:, :2]  # Use first two features for 2D visualization
y = iris.target

# Train-test split
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2)

# Polynomial Kernel SVM
model = svm.SVC(kernel='poly', degree=3, gamma='scale')
model.fit(X_train, y_train)

accuracy = model.score(X_test, y_test)

# Plot decision boundary
x_min, x_max = X[:, 0].min() - 1, X[:, 0].max() + 1
y_min, y_max = X[:, 1].min() - 1, X[:, 1].max() + 1

xx, yy = np.meshgrid(
    np.linspace(x_min, x_max, 300),
    np.linspace(y_min, y_max, 300)
)

Z = model.predict(np.c_[xx.ravel(), yy.ravel()])
Z = Z.reshape(xx.shape)

plt.figure(figsize=(8, 6))

# Decision boundary
plt.contourf(xx, yy, Z, alpha=0.3)

# Data points
plt.scatter(X[:, 0], X[:, 1], c=y, edgecolors='black')

plt.xlabel("Feature 1")
plt.ylabel("Feature 2")
plt.title(f"SVM with Polynomial Kernel (Degree=3)\nAccuracy: {accuracy:.2f}")

plt.show()

Sigmoid

The Sigmoid Kernel is a kernel function used in SVM that creates a non-linear decision boundary.
It is inspired by the activation function used in neural networks → the sigmoid function.

import numpy as np
import matplotlib.pyplot as plt
from sklearn import svm
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split

# Load iris dataset
iris = load_iris()
X = iris.data[:, :2]  # Using first two features for 2D visualization
y = iris.target

# Train-test split
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2)

# Sigmoid Kernel SVM
model = svm.SVC(kernel='sigmoid', gamma='scale')
model.fit(X_train, y_train)

accuracy = model.score(X_test, y_test)

# Plot decision boundary
x_min, x_max = X[:, 0].min() - 1, X[:, 0].max() + 1
y_min, y_max = X[:, 1].min() - 1, X[:, 1].max() + 1

xx, yy = np.meshgrid(
    np.linspace(x_min, x_max, 300),
    np.linspace(y_min, y_max, 300)
)

Z = model.predict(np.c_[xx.ravel(), yy.ravel()])
Z = Z.reshape(xx.shape)

plt.figure(figsize=(8, 6))

# Decision boundary
plt.contourf(xx, yy, Z, alpha=0.3)

# Data points
plt.scatter(X[:, 0], X[:, 1], c=y, edgecolors='black')

plt.xlabel("Feature 1")
plt.ylabel("Feature 2")
plt.title(f"SVM with Sigmoid Kernel\nAccuracy: {accuracy:.2f}")

plt.show()

Simple SVM Classification Example (Custom Data)

Example: Classify students as Pass/Fail.

import numpy as np
from sklearn import svm

# marks, attendance
X = np.array([
    [90, 95],
    [85, 80],
    [70, 60],
    [30, 40],
    [25, 35]
])

y = ['Pass', 'Pass', 'Pass', 'Fail', 'Fail']

# Train model
model = svm.SVC(kernel='linear')
model.fit(X, y)

print(model.predict([[60, 70]]))   # Predict new student

Advantages of SVM

High classification accuracy
Effective on small and medium-sized datasets
Works well with high-dimensional data
Reduces overfitting through margin maximization
Excellent for text classification and image recognition

Disadvantages of SVM

Training becomes slow for very large datasets
Choosing the correct kernel and parameters can be difficult
Less interpretable than Decision Trees
Computationally expensive for large-scale problems

Applications of SVM

Spam Email Detection
Face Recognition
Handwriting Recognition
Image Classification
Medical Diagnosis
Fraud Detection
Credit Risk Analysis
Sentiment Analysis
Bioinformatics
Customer Churn Prediction

Interview Questions

1. What is SVM?

SVM is a supervised machine learning algorithm that separates different classes by finding the optimal hyperplane with the maximum margin.

2. What is a Hyperplane?

A hyperplane is the decision boundary that separates different classes.

3. What are Support Vectors?

Support vectors are the data points closest to the hyperplane. They determine the position of the decision boundary.

4. What is Margin?

Margin is the distance between the hyperplane and the nearest support vectors.

5. What is the difference between Hard Margin and Soft Margin?

Hard Margin: No misclassification allowed; suitable for perfectly separable data.
Soft Margin: Allows some classification errors; better for noisy real-world datasets.

6. What is the Kernel Trick?

The Kernel Trick transforms non-linear data into a higher-dimensional feature space where it becomes linearly separable without explicitly performing the transformation.

Codes With Pankaj

SVM (Support Vector Machine)

What is SVM?

Real-Life Example

Basic Concepts of SVM

1. Hyperplane

2. Margin

3. Support Vectors

Goal of SVM

Why Maximum Margin?

Mathematical Representation

Margin Formula

Hard Margin SVM

Soft Margin SVM

C Parameter

Large C

Small C

Linear SVM

Non-Linear SVM

Types of SVM

Kernel Trick

Types of Kernels

1. Linear Kernel

2. Polynomial Kernel

3. Radial Basis Function (RBF) Kernel

Get Codes With Pankaj’s stories in your inbox

4. Sigmoid Kernel

Gamma Parameter

High Gamma

Low Gamma

Support Vector Machine (SVM) Example with a Real Dataset: Spam Email Detection

Problem Statement

Linear SVM

Non-Linear SVM

Popular Kernels

RBF (Gaussian) → Most powerful

Polynomial → Curve-based separation

Sigmoid

Advantages of SVM

Disadvantages of SVM

Applications of SVM

Interview Questions

1. What is SVM?

2. What is a Hyperplane?

3. What are Support Vectors?

4. What is Margin?

5. What is the difference between Hard Margin and Soft Margin?

6. What is the Kernel Trick?

Written by Codes With Pankaj

No responses yet

More from Codes With Pankaj

DBSCAN Clustering Tutorial for Beginners

What is DBSCAN?

A Deep Dive into Java Multithreading

In the world of Java programming, multithreading is a powerful concept that allows developers to create highly efficient and responsive…

Node JS CRUD With MySQL

Node.js is an open-source, cross-platform runtime environment for developing server-side and networking applications. You should have a…

Understanding Skewness in Machine Learning: A Beginner’s Guide with Python Example

Welcome to another exciting tutorial from Codes With Pankaj Chouhan! If you’re just starting your journey into machine learning, you’ve…

Recommended from Medium

10 Important Statistical Terms Everyone Should Know

Understanding Confidence Intervals, Z-Scores, T-Scores, P-Values, and other essential concepts in statistical analysis.

How SVD Compresses an Image — One Layer at a Time

Every photo can be rebuilt from simple layers, and the first few carry most of the picture.

K-Nearest-Neighbors (KNN)

K-Nearest Neighbors (KNN) is one of the easiest and most intuitive Machine Learning algorithms. The basic idea behind KNN is quite similar…

Support Vector Machines Explained: The Model That Tries to Draw the Best Possible Boundary

Why some classifiers do not just separate classes but try to separate them with the widest possible margin

Probability vs. Likelihood: The Subtle Difference That Is Easy to Overlook

To understand this, let’s start with a basic experiment we’re already familiar with.

P-2. Activation Functions Explained in Detail : The Secret Behind Every Neural Network

Why neural networks need them, and how to pick the right one.

No responses yet

More from Codes With Pankaj

DBSCAN Clustering Tutorial for Beginners

What is DBSCAN?

A Deep Dive into Java Multithreading

In the world of Java programming, multithreading is a powerful concept that allows developers to create highly efficient and responsive…

Node JS CRUD With MySQL

Node.js is an open-source, cross-platform runtime environment for developing server-side and networking applications. You should have a…