Data Preprocessing and Feature Scaling: Preparing Data for ML
Data preprocessing has been an important step in my machine learning journey. A model cannot simply take messy real-world data and immediately produce meaningful predictions. Before training, the data needs to be inspected, cleaned, transformed, and represented in a form the algorithm can work with.
Feature scaling is one of the most important preprocessing techniques when numerical features have very different ranges.
In this article, I walk through the concepts, Python syntax, and workflow I learned while working with data preprocessing and feature scaling.
1. What is Data Preprocessing?
Data preprocessing is the process of transforming raw data into a clean and suitable format for machine learning.
Real-world datasets commonly contain missing values, duplicate records, incorrect data types, categorical variables, numerical variables with different scales, outliers, irrelevant features, and inconsistent data.
A useful preprocessing flow is:
Raw Data → Data Inspection → Data Cleaning → Missing Values → Duplicates → Outliers → Encoding → Feature Scaling → Train-Test Split → Model Training.
2. Understanding Features and Target
Before preprocessing, it is important to identify the features and the target. Suppose a dataset contains Age, Salary, Gender, and Experience. If Salary is what we want to predict, Salary becomes the target y and the remaining input columns become features X.
X = df.drop(“Salary”, axis=1)
y = df[“Salary”]3. Data Inspection
Before changing the dataset, I first inspect its structure and distributions. Pandas provides simple commands for this first stage of understanding.
df.head()
df.shapedf.columns
df.dtypes
df.info()
df.describe()
df.isnull().sum()
df.duplicated().sum()4. Handling Missing Values
Missing values are common in practical datasets. We can remove incomplete rows with dropna(), but that may discard useful information. A better choice in many cases is imputation.
For numerical data, mean imputation can work when the distribution is reasonably symmetric. Median imputation is often useful for skewed data or data containing outliers. For categorical data, mode imputation is common.
df[“Age”] = df[“Age”].fillna(df[“Age”].mean())
df[“Salary”] = df[“Salary”].fillna(df[“Salary”].median())
df[“Gender”] = df[“Gender”].fillna(df[“Gender”].mode()[0])5. Handling Duplicate Data
Duplicate records can distort analysis and model training. I check duplicates before deciding whether they should be removed.
df.duplicated().sum()
df = df.drop_duplicates()6. Handling Incorrect Data Types
A column may contain numbers stored as strings or dates stored as text. Converting them to the correct type makes later analysis and modeling more reliable.
df[“Salary”] = df[“Salary”].astype(int)
df[“Salary”] = df[“Salary”].astype(float)
df[“Date”] = pd.to_datetime(df[“Date”])7. Handling Categorical Data
Machine learning algorithms generally require numerical input. Categorical variables therefore need to be encoded.
Label encoding assigns numbers to categories, but those numbers can imply an artificial order. One-hot encoding creates a separate column for each category. Ordinal encoding is useful when the categories have a meaningful order.
from sklearn.preprocessing import LabelEncoder, OneHotEncoder
encoder = LabelEncoder()
df[“Gender”] = encoder.fit_transform(df[“Gender”])
pd.get_dummies(df, columns=[“City”])
encoder = OneHotEncoder(handle_unknown=”ignore”)
encoded = encoder.fit_transform(df[[“City”]])8. Outlier Detection
An outlier is an observation that is unusually far from the rest of the data. One common method for identifying potential outliers is the Interquartile Range (IQR).
Q1 = df[“Salary”].quantile(0.25)
Q3 = df[“Salary”].quantile(0.75)
IQR = Q3 — Q1
lower_limit = Q1–1.5 * IQR
upper_limit = Q3 + 1.5 * IQR
outliers = df[
(df[“Salary”] < lower_limit) |
(df[“Salary”] > upper_limit)
]9. Why is Feature Scaling Required?
Consider Age values from 20–30 and Salary values from 30,000–80,000. Their numerical ranges are very different.
Some algorithms calculate distances, depend on feature magnitude, or use gradient-based optimization. Without scaling, a feature with much larger numerical values can have a disproportionate influence on those calculations.
Feature scaling brings numerical features to a comparable scale.
10. Train-Test Split Before Scaling
One of the most important preprocessing rules is avoiding data leakage. The dataset should be split before fitting a scaler so that information from the test set does not influence the parameters learned from the training data.
from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(
X, y
test_size=0.2
random_state=42
)11. Standardization with StandardScaler
Standardization transforms a feature so that it has approximately mean 0 and standard deviation 1. The formula is z = (x — µ) / σ.
Get Rohithnag’s stories in your inbox
Join Medium for free to get updates from this writer.
The scaler learns its parameters from the training data and then uses those same parameters for both training and test data.
from sklearn.preprocessing import StandardScaler
scaler = StandardScaler()
X_train_scaled=scaler.fit_transform(X_train)
X_test_scaled=scaler.transform(X_test)12. Min-Max Scaling
Min-Max Scaling commonly transforms values into a range from 0 to 1. The formula is x’ = (x — xmin) / (xmax — xmin).
For example, if the minimum Age is 20 and the maximum is 30, then 20 maps to 0, 25 maps to 0.5, and 30 maps to 1.
from sklearn.preprocessing import MinMaxScaler
scaler = MinMaxScaler()
X_train_scaled = scaler.fit_transform(X_train)
X_test_scaled = scaler.transform(X_test)13. StandardScaler vs MinMaxScaler
StandardScaler uses the mean and standard deviation and produces a scale centered around zero. MinMaxScaler uses the minimum and maximum and commonly produces values between zero and one.
There is no universal scaler that is always correct. The choice depends on the dataset and the algorithm.
14. Robust Scaling
When data contains significant outliers, RobustScaler can be useful. It uses the median and IQR instead of the mean and standard deviation, making it generally less affected by extreme values.
from sklearn.preprocessing import RobustScaler
scaler = RobustScaler()
X_train_scaled = scaler.fit_transform(X_train)
X_test_scaled = scaler.transform(X_test)15. When is Feature Scaling Important?
Scaling is particularly important for algorithms that depend on distance, magnitude, or gradient optimization. Examples include K-Nearest Neighbors, K-Means, Support Vector Machines, Logistic Regression, many linear-model optimization settings, Neural Networks, and PCA.
Tree-based algorithms such as Decision Trees and Random Forests generally do not require scaling in the same way because their split decisions are not based on feature distance or magnitude.
16. The Difference Between fit(), transform(), and fit_transform()
fit() learns parameters from the data. transform() applies the learned parameters. fit_transform() performs both operations.
The important rule is: training data uses fit_transform(), while test data uses transform().
scaler.fit(X_train)
X_train_scaled = scaler.transform(X_train)
X_test_scaled = scaler.transform(X_test)
# Equivalent for training data:
X_train_scaled = scaler.fit_transform(X_train)17. Using a Scikit-learn Pipeline
A Pipeline connects preprocessing steps and the final estimator into one workflow. This reduces the risk of applying preprocessing inconsistently and helps prevent leakage during model fitting.
from sklearn.pipeline import Pipeline
from sklearn.impute import SimpleImputer
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LogisticRegression
pipeline = Pipeline([
(“imputer”, SimpleImputer(strategy=”median”)),
(“scaler”, StandardScaler()),
(“model”, LogisticRegression())
])
pipeline.fit(X_train, y_train)
y_pred = pipeline.predict(X_test)18. ColumnTransformer for Mixed Data
Real datasets often contain both numerical and categorical columns. ColumnTransformer allows each group to receive its own preprocessing steps.
from sklearn.compose import ColumnTransformer
from sklearn.preprocessing import OneHotEncoder
numeric_features = [“Age”, “Salary”, “Experience”]
categorical_features = [“Gender”, “City”]
preprocessor = ColumnTransformer([
(“num”, numeric_pipeline, numeric_features),
(“cat”, categorical_pipeline, categorical_features)
])19. Complete Machine Learning Preprocessing Workflow
The complete workflow can be summarized as: understand the dataset, inspect it, clean it, separate X and y, split into training and testing sets, fit preprocessing on training data, transform the data, train the model, make predictions, and evaluate the result.
20. Complete Python Workflow
A simplified end-to-end example looks like this:
import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
from sklearn.impute import SimpleImputer
from sklearn.pipeline import Pipeline
from sklearn.linear_model import LogisticRegression
df = pd.read_csv(“data.csv”)
X = df.drop(“target”, axis=1)
y = df[“target”]
X_train, X_test, y_train, y_test = train_test_split(
X, y, test_size=0.2, random_state=42
)
pipeline = Pipeline([
(“imputer”, SimpleImputer(strategy=”median”)\
(“scaler”, StandardScaler()),
(“model”, LogisticRegression())
])
pipeline.fit(X_train, y_train)
y_pred = pipeline.predict(X_test)21. Data Leakage
Data leakage occurs when information from outside the training data is used in a way that gives the model information it should not have.
For example, fitting a scaler on the complete dataset before the train-test split allows test-set information to influence the scaling parameters. Fitting only on X_train avoids this problem.
22. Preprocessing vs Feature Scaling
Data preprocessing is the broad process and can include missing-value handling, duplicate removal, type conversion, outlier treatment, encoding, transformation, feature selection, and feature scaling.
Feature scaling is therefore one specific technique within the larger data-preprocessing workflow.
23. Key Concepts I Learned
Through this work, I learned how raw data is prepared before it reaches a machine learning model: inspection, missing-value handling, duplicate handling, type conversion, encoding, outlier analysis, train-test splitting, leakage prevention, scaling, and pipeline construction.
24. Final Machine Learning Data Preparation Flow
RAW DATA → UNDERSTAND → INSPECT → CLEAN → HANDLE MISSING VALUES → HANDLE DUPLICATES → HANDLE OUTLIERS → SEPARATE X AND y → TRAIN-TEST SPLIT → IMPUTE / ENCODE / SCALE → MODEL TRAINING → PREDICTION → EVALUATION
25. Final Thoughts
Data preprocessing is not simply about cleaning a dataset. It is about transforming raw information into a representation that a machine learning algorithm can effectively learn from.
The most important practical lesson is to make preprocessing reproducible and to learn transformation parameters only from the training data. With Pandas and Scikit-learn, techniques such as imputation, encoding, scaling, Pipeline, and ColumnTransformer make that workflow much easier to manage.
With preprocessing and feature scaling understood, the next step is to connect these concepts with machine learning algorithms, model training, prediction, and evaluation.
END OF ARTICLE
