Data Normalization in Data Mining

Data normalization is a technique used in data mining to transform the values of a dataset into a common scale. This is important because many machine learning algorithms (KNN, neural networks, clustering algorithms (like K-Means), and gradient-based models) are sensitive to the scale of the input features and can produce better results when the data is normalized. Normalization is used to scale the data of an attribute so that it falls in a smaller range, such as -1.0 to 1.0 or 0.0 to 1.0. It is generally useful for classification algorithms.

Need of Normalization

When features in a dataset exist on different scales, models may assign more weight to attributes with larger numeric values. Consider the example image below:

• Salary values (e.g., 120000, 92000) are significantly larger than years_of_experience (e.g., 5, 7, 10), creating a scale imbalance.
• Because of this large numerical difference, salary can dominate or overpower experience during model training.
• Algorithms that rely on distance, weights, or gradients (e.g., KNN, SVM, Linear Regression) may bias decisions toward high-scale features.
• Without normalization, the model may incorrectly treat salary as the most influential predictor, even if experience is equally important.
• Scaling both attributes ensures fair contribution from each feature, improving model accuracy and stability.

Methods of Data Normalization

1. Decimal Scaling Method For Normalization

This method normalizes values by shifting the decimal point. Each value is divided by a power of 10 based on the maximum absolute value in the attribute.

v_i' = \frac{v_i}{10^j}

Where:

• v_i​ = original value
• v_i'​ = normalized value
• j = smallest integer such that \max(|v_i'|) < 1

Example: Let the input data is: -10, 201, 301, -401, 501, 601, 701. To normalize the above data,

• Maximum absolute value in given data(m) = 701
• Divide the given data by 1000 (i.e j=3)
• The normalized data is: -0.01, 0.201, 0.301, -0.401, 0.501, 0.601, 0.701

2. Min-Max Normalization

This technique rescales values to a fixed range, typically 0–1. It preserves the relationships between values proportionally. Minimum and maximum value from data is fetched and each value is replaced according to the following formula.

v' = \frac{v - \min(A)}{\max(A) - \min(A)} \left( \text{new\_max}(A) - \text{new\_min}(A) \right) + \text{new\_min}(A)

Where:

• v = original value
• v' = normalized value
• \min(A), \max(A) = minimum and maximum values of attribute AAA
• new\_min(A), new\_max(A) = required output range

Commonly used when a fixed output scale is required.

3. Z-Score Normalization (Standardization)

Also known as zero-mean normalization, this method transforms values based on mean and standard deviation. It is useful when the data must follow a normal distribution.

v' = \frac{v - \bar{A}}{\sigma_A}

Where:

• 𝑣 = original value
• 𝑣′ = normalized value
• \bar{A} = mean of attribute A
• \sigma_A​ = standard deviation of attribute A

This method centers the data around 0 with a standard deviation of 1.

Advantages of Data Normalization

1. Improved algorithm performance: Scaling inputs helps models learn efficiently, especially distance-based and gradient-based algorithms.
2. Reduced impact of outliers: Normalization brings extreme values closer to the rest, reducing their influence.
3. Better interpretability: Attributes on a common scale make comparisons more meaningful.
4. Improved model generalization: With balanced input ranges, models become less sensitive to variations in magnitude.

Disadvantages of Data Normalization

1. Possible loss of original scale information: If the raw value scale is meaningful, normalization may obscure it.
2. Outlier handling can become harder: Since normalization compresses ranges, outliers may not be easily identified.
3. Reduced interpretability in some cases: Standardized or scaled numbers may not align with real-world understanding.
4. Additional computation: Normalization adds preprocessing steps, especially for large datasets.