Sitemap
Open in app

Sign in

Medium Logo
Write
Search

Sign in

DATA PREPROCESSING

Encoding and Feature scaling

What is Encoding ?

Encoding is the process of converting categorical data into a numerical format so that Machine Learning algorithms can understand and process it.

For example:

Gender
Male
Female
Male
Female

can be converted into:

1
0
1
0

However, we should not use the same encoding technique for every categorical variable. The appropriate technique depends on the type of categorical data.

Types of Categorical Data

Before choosing an encoding technique, it is important to understand the two major types of categorical data.

1. Nominal Data

Nominal data consists of categories that do not have a natural order or ranking.

Examples:

Color:
Red
Blue
Green

There is no meaningful relationship such as:

Red < Blue < Green

Other examples include:

  • Gender
  • City
  • Country
  • Blood group
  • Product category
  • Department

For nominal data, One-Hot Encoding is commonly used.

2. Ordinal Data

Ordinal data consists of categories that have a meaningful order or ranking.

For example:

Education:
School
Graduate
Postgraduate
PhD

There is a natural progression between the categories.

Another example:

Customer Satisfaction:
Low
Medium
High

Here, the order is meaningful:

Low < Medium < High

For ordinal data, Ordinal Encoding is generally suitable.

Major Encoding Techniques

There are several techniques used to convert categorical variables into numerical values.

The commonly used techniques are:

  1. Label Encoding
  2. Ordinal Encoding
  3. One-Hot Encoding
  4. Binary Encoding

1. Label Encoding

1. Standardization

Standardization transforms numerical data so that the feature has approximately:

  • Mean = 0
  • Standard Deviation = 1

The formula is : [ z = \frac{x-\mu}{\sigma} ]

Where:

  • x = original value
  • μ = mean of the feature
  • σ = standard deviation
  • z = standardized value

After standardization, the values are centered around zero.

Python Implementation

from sklearn.preprocessing import StandardScaler
scaler = StandardScaler()X_scaled = scaler.fit_transform(X)

Example

Suppose:

Age
20
25
30
35
40

After Standard Scaler, the values will be transformed into values centered around zero.

They may look approximately like:

-1.41
-0.71
 0.00
 0.71
 1.41

The exact values depend on the mean and standard deviation of the data.

When Should You Use Standard Scaler?

Standardization is a good general-purpose choice for many algorithms, including:

  • Logistic Regression
  • Linear Regression
  • SVM
  • KNN
  • PCA
  • Neural Networks

2. Min-Max Scaling

Min-Max Scaling transforms the values into a specified range.

The most common range is:

0 to 1

The formula is : [ x’ = \frac{x-x_{min}}{x_{max}-x_{min}} ]

Where:

  • x = original value
  • xmin = minimum value
  • xmax = maximum value
  • x' = scaled value

Example

Suppose we have:

Age
20
30
40
50
60

After Min-Max Scaling:

20 → 0.00
30 → 0.25
40 → 0.50
50 → 0.75
60 → 1.00

Python Implementation

from sklearn.preprocessing import MinMaxScaler
scaler = MinMaxScaler()X_scaled = scaler.fit_transform(X)

When Should You Use MinMax Scaler?

Min-Max Scaling can be useful when:

  • You want features within a fixed range.
  • Your algorithm benefits from bounded values.
  • You are working with neural networks or distance-based algorithms.

Important Limitation

Min-Max Scaling is sensitive to outliers.

Suppose:

Salary
30,000
40,000
50,000
60,000
10,000,000

The extremely large value can significantly affect the minimum-to-maximum range.

3. Robust Scaling

Robust Scaling is designed to reduce the influence of outliers.

Instead of using:

  • Mean
  • Standard deviation

Robust Scaling uses:

  • Median
  • Interquartile Range (IQR)

The formula is : [ x’ = \frac{x-\text{Median}}{IQR} ]

where : [IQR = Q3-Q1]

Python Implementation

from sklearn.preprocessing import RobustScaler
scaler = RobustScaler()X_scaled = scaler.fit_transform(X)

Example

Consider:

Salary
30,000
40,000
45,000
50,000
55,000
5,000,000

Here, 5,000,000 is an extreme value.

Standard Scaler can be strongly influenced by this outlier.

Robust Scaler uses the median and IQR, making it more resistant to extreme observations.

When Should You Use Robust Scaler?

Use Robust Scaler when:

  • Your numerical features contain significant outliers.
  • You do not want extreme values to strongly influence the scaling parameters.

4. Max Absolute Scaling

Max Absolute Scaling divides each feature by its maximum absolute value.

It generally scales the values into:

-1 to 1

while preserving the sign of the original values.

Python Implementation

from sklearn.preprocessing import MaxAbsScaler
scaler = MaxAbsScaler()X_scaled = scaler.fit_transform(X)

When Should You Use MaxAbs Scaler?

It can be useful when:

  • Data contains positive and negative values.
  • You want to preserve sparsity.
  • You are working with sparse datasets.

5. Normalization

Normalization is different from the feature-wise scaling methods discussed above.

Feature scaling methods such as Standard Scaler generally transform each feature/column.

Normalization typically transforms each sample/row so that its norm equals a specified value.

For example:

from sklearn.preprocessing import Normalizer
normalizer = Normalizer()X_normalized = normalizer.fit_transform(X)

Common normalization methods include:

  • L1 normalization
  • L2 normalization
  • Max normalization

Normalization is particularly useful in areas such as:

  • Text classification
  • Document similarity
  • Cosine similarity
  • Sparse dataconverts each category into a unique integer.

For example:

Gender
Male
Female
Male
Female

After Label Encoding:

Male   → 1
Female → 0

Another example:

City
Hyderabad → 0
Mumbai    → 1
Delhi     → 2

Python Example

from sklearn.preprocessing import LabelEncoder
encoder = LabelEncoder()df["Gender"] = encoder.fit_transform(df["Gender"])

When to use?

Label Encoding can be useful when:

  • The variable has two categories.
  • The numerical representation will not create a problematic artificial relationship.
  • You specifically need integer labels.

2. Ordinal Encoding

Ordinal Encoding is used when categories have a natural order.

Get Nea’s stories in your inbox

Join Medium for free to get updates from this writer.

For example:

Size
Small
Medium
Large

We can encode them as:

Small  → 0
Medium → 1
Large  → 2

The numerical values preserve the order.

Python Example

from sklearn.preprocessing import OrdinalEncoder
encoder = OrdinalEncoder(
    categories=[["Small", "Medium", "Large"]]
)df["Size"] = encoder.fit_transform(df[["Size"]])

Another example:

Low    → 0
Medium → 1
High   → 2

When to use?

Use Ordinal Encoding when:

  • Categories have a meaningful order.
  • The distance/order represented by the encoding makes sense for the model.

3. One-Hot Encoding

One-Hot Encoding is one of the most commonly used techniques for nominal categorical variables.

Suppose we have:

Color
Red
Blue
Green

One-Hot Encoding creates a separate column for each category.

ColorRedBlueGreenRed100Blue010Green001

Each row contains 1 for the category it belongs to and 0 for the others.

Python Example

from sklearn.preprocessing import OneHotEncoder
encoder = OneHotEncoder(
    handle_unknown="ignore"
)encoded_data = encoder.fit_transform(
    df[["Color"]]
)

When to use?

One-Hot Encoding is commonly used for nominal variables such as:

  • City
  • Gender
  • Country
  • Product type
  • Department
  • Browser type

4. Binary Encoding

Binary Encoding converts categories into binary numbers.

For example, instead of creating one column for every category, categories are first assigned integer values and then converted into binary representation.

Suppose:

Category
A → 1
B → 2
C → 3
D → 4

Their binary representations can be used as encoded features.

Binary Encoding can produce fewer columns than One-Hot Encoding when there are many categories.

When to use?

Binary Encoding can be useful for:

  • High-cardinality categorical variables
  • Large datasets
  • Situations where One-Hot Encoding would create too many columns
  • It is less commonly used than One-Hot Encoding in basic Machine Learning workflows.

Feature Scaling

Feature Scaling in Machine Learning

After encoding categorical variables, the next important preprocessing technique is Feature Scaling.

What is Feature Scaling?

Feature scaling is the process of transforming numerical features so that they are represented on a similar scale or range.

Types of Feature Scaling

The major feature scaling techniques are:

  1. Standardization
  2. Min-Max Scaling
  3. Robust Scaling
  4. Max Absolute Scaling
  5. Normalization

1. Standardization

Standardization transforms numerical data so that the feature has approximately:

  • Mean = 0
  • Standard Deviation = 1

The formula is:

[
z = \frac{x-\mu}{\sigma}
]

Where:

  • x = original value
  • μ = mean of the feature
  • σ = standard deviation
  • z = standardized value

After standardization, the values are centered around zero.

Python Implementation

from sklearn.preprocessing import StandardScaler
scaler = StandardScaler()X_scaled = scaler.fit_transform(X)

Example

Suppose:

Age
20
25
30
35
40

After Standard Scaler, the values will be transformed into values centered around zero.

They may look approximately like:

-1.41
-0.71
 0.00
 0.71
 1.41

The exact values depend on the mean and standard deviation of the data.

When Should You Use StandardScaler?

Standardization is a good general-purpose choice for many algorithms, including:

  • Logistic Regression
  • Linear Regression
  • SVM
  • KNN
  • PCA
  • Neural Networks

2. Min-Max Scaling

Min-Max Scaling transforms the values into a specified range.

The most common range is:

0 to 1

The formula is:

[
x’ = \frac{x-x_{min}}{x_{max}-x_{min}}
]

Where:

  • x = original value
  • xmin = minimum value
  • xmax = maximum value
  • x' = scaled value

Example

Suppose we have:

Age
20
30
40
50
60

After Min-Max Scaling:

20 → 0.00
30 → 0.25
40 → 0.50
50 → 0.75
60 → 1.00

Python Implementation

from sklearn.preprocessing import MinMaxScaler
scaler = MinMaxScaler()X_scaled = scaler.fit_transform(X)

When Should You Use MinMax Scaler?

Min-Max Scaling can be useful when:

  • You want features within a fixed range.
  • Your algorithm benefits from bounded values.
  • You are working with neural networks or distance-based algorithms.

Important Limitation

Min-Max Scaling is sensitive to outliers.

Suppose:

Salary
30,000
40,000
50,000
60,000
10,000,000

The extremely large value can significantly affect the minimum-to-maximum range.

3. Robust Scaling

Robust Scaling is designed to reduce the influence of outliers. Instead of using:

  • Mean
  • Standard deviation

Robust Scaling uses:

  • Median
  • Interquartile Range (IQR)

The formula is : [ x’ = \frac{x-\text{Median}}{IQR} ]

where : [ IQR = Q3-Q1 ]

Python Implementation

from sklearn.preprocessing import RobustScaler
scaler = RobustScaler()X_scaled = scaler.fit_transform(X)

Example

Consider:

Salary
30,000
40,000
45,000
50,000
55,000
5,000,000

Here, 5,000,000 is an extreme value.

Standard Scaler can be strongly influenced by this outlier.

Robust Scaler uses the median and IQR, making it more resistant to extreme observations.

When Should You Use Robust Scaler?

Use Robust Scaler when:

  • Your numerical features contain significant outliers.
  • You do not want extreme values to strongly influence the scaling parameters.

4. Max Absolute Scaling

Max Absolute Scaling divides each feature by its maximum absolute value.

It generally scales the values into:

-1 to 1

while preserving the sign of the original values.

Python Implementation

from sklearn.preprocessing import MaxAbsScaler
scaler = MaxAbsScaler()X_scaled = scaler.fit_transform(X)

When Should You Use MaxAbsScaler?

It can be useful when:

  • Data contains positive and negative values.
  • You want to preserve sparsity.
  • You are working with sparse datasets.

5. Normalization

Normalization is different from the feature-wise scaling methods discussed above.

Feature scaling methods such as StandardScaler generally transform each feature/column.

Normalization typically transforms each sample/row so that its norm equals a specified value.

For example:

from sklearn.preprocessing import Normalizer
normalizer = Normalizer()X_normalized = normalizer.fit_transform(X)

Common normalization methods include:

  • L1 normalization
  • L2 normalization
  • Max normalization

Normalization is particularly useful in areas such as:

  • Text classification
  • Document similarity
  • Cosine similarity
  • Sparse data
Machine Learning
Data Preprocessing
Encoding
Feature Scaling

Written by Nea

1 follower
1 following

Help

Status

About

Careers

Press

Blog

Store

Privacy

Rules

Terms

Text to speech