“𝐁𝐚𝐥𝐚𝐧𝐜𝐢𝐧𝐠 𝐈𝐦𝐛𝐚𝐥𝐚𝐧𝐜𝐞𝐝 𝐃𝐚𝐭𝐚𝐬𝐞𝐭𝐬: 𝐀 𝐃𝐞𝐞𝐩 𝐃𝐢𝐯𝐞 𝐢𝐧𝐭𝐨 𝐌𝐚𝐜𝐡𝐢𝐧𝐞 𝐋𝐞𝐚𝐫𝐧𝐢𝐧𝐠 𝐒𝐚𝐦𝐩𝐥𝐢𝐧𝐠 𝐓𝐞𝐜𝐡𝐧𝐢𝐪𝐮𝐞𝐬”

𝐒𝐚𝐦𝐩𝐥𝐢𝐧𝐠:

Sampling plays a crucial role in machine learning, especially when dealing with imbalanced datasets. The choice of a sampling technique can significantly impact model performance by ensuring balanced data distribution and preventing bias. In this blog, we’ll explore different sampling methods, including oversampling, undersampling and hybrid techniques like SMOTE, Borderline-SMOTE, and ADASYN.

Image Describing Sampling

𝟏. 𝐖𝐡𝐲 𝐢𝐬 𝐒𝐚𝐦𝐩𝐥𝐢𝐧𝐠 𝐈𝐦𝐩𝐨𝐫𝐭𝐚𝐧𝐭?

Machine learning models learn from data, and if the data is imbalanced, models may become biased toward the majority class, leading to poor generalization. Sampling techniques help by either balancing the dataset or selecting a representative subset, improving model robustness and accuracy.

𝟐. 𝐓𝐲𝐩𝐞𝐬 𝐨𝐟 𝐒𝐚𝐦𝐩𝐥𝐢𝐧𝐠 𝐓𝐞𝐜𝐡𝐧𝐢𝐪𝐮𝐞𝐬:

Sampling methods can be broadly classified into:

Types of Sampling

1. Random Sampling
2. Oversampling (Increasing the Minority Class)
3. Undersampling (Reducing the Majority Class)
4. Hybrid Techniques (Combining Over and Under Sampling)

𝟐.𝟏 𝐑𝐚𝐧𝐝𝐨𝐦 𝐒𝐚𝐦𝐩𝐥𝐢𝐧𝐠:

📌 𝐖𝐡𝐚𝐭 𝐈𝐭 𝐈𝐬?

• Random Oversampling: Duplicates existing minority class instances to balance the dataset.
• Random Undersampling : Removes majority class samples to match the minority class size.
• Risk: Can lead to overfitting (oversampling) or loss of valuable data (undersampling).

✅ 𝐖𝐡𝐞𝐧 𝐭𝐨 𝐔𝐬𝐞:

• Use random oversampling when you have a small dataset and want to quickly balance classes.
• Use random undersampling when you have a large dataset and removing data won’t impact performance significantly.

❌ 𝐖𝐡𝐞𝐧 𝐍𝐨𝐭 𝐭𝐨 𝐔𝐬𝐞:

• Avoid random oversampling when overfitting is a concern.
• Avoid random undersampling when you risk losing important patterns in the data.

𝟐.𝟐 𝐒𝐲𝐧𝐭𝐡𝐞𝐭𝐢𝐜 𝐌𝐢𝐧𝐨𝐫𝐢𝐭𝐲 𝐎𝐯𝐞𝐫-𝐬𝐚𝐦𝐩𝐥𝐢𝐧𝐠 𝐓𝐞𝐜𝐡𝐧𝐢𝐪𝐮𝐞 (𝐒𝐌𝐎𝐓𝐄):

📌 𝐖𝐡𝐚𝐭 𝐈𝐭 𝐈𝐬?

SMOTE is a widely used technique that generates synthetic minority class samples instead of duplicating them. It works by selecting a sample from the minority class, finding its k-nearest neighbors, and interpolating new points along the line connecting them.

𝐈𝐦𝐩𝐥𝐞𝐦𝐞𝐧𝐭𝐚𝐭𝐢𝐨𝐧 𝐢𝐧 𝐏𝐲𝐭𝐡𝐨𝐧 🐍:

from imblearn.over_sampling import SMOTE
from collections import Counter

# Applying SMOTE
smote = SMOTE(sampling_strategy='auto', random_state=42)
X_resampled, y_resampled = smote.fit_resample(X, y)
print("After SMOTE:", Counter(y_resampled))

✅ 𝐖𝐡𝐞𝐧 𝐭𝐨 𝐔𝐬𝐞:

• Use SMOTE when you need synthetic data without simply duplicating existing samples.
• Works well when your dataset is moderately imbalanced.

❌ 𝐖𝐡𝐞𝐧 𝐍𝐨𝐭 𝐭𝐨 𝐔𝐬𝐞:

• Avoid SMOTE when dealing with highly overlapping class distributions, as it may introduce noise.
• Not ideal when the dataset is too small, as synthetic samples might not generalize well.

𝟐.𝟑 𝐁𝐨𝐫𝐝𝐞𝐫𝐥𝐢𝐧𝐞-𝐒𝐌𝐎𝐓𝐄 (𝐁-𝐒𝐌𝐎𝐓𝐄):

📌 𝐖𝐡𝐚𝐭 𝐈𝐭 𝐈𝐬?

Unlike SMOTE, Borderline-SMOTE focuses only on hard-to-classify samples that lie close to the decision boundary.

• B-SMOTE1: Generates synthetic data only for borderline minority samples.
• B-SMOTE2: Adds both synthetic minority samples and some majority samples

𝐈𝐦𝐩𝐥𝐞𝐦𝐞𝐧𝐭𝐚𝐭𝐢𝐨𝐧 𝐢𝐧 𝐏𝐲𝐭𝐡𝐨𝐧🐍:

from imblearn.over_sampling import BorderlineSMOTE
bsmote = BorderlineSMOTE(kind='borderline-1', random_state=42)
X_resampled, y_resampled = bsmote.fit_resample(X, y)

✅ 𝐖𝐡𝐞𝐧 𝐭𝐨 𝐔𝐬𝐞:

• Ideal when minority class samples are close to decision boundaries and need reinforcement.
• Works well for datasets where the separation between classes is unclear.

❌ 𝐖𝐡𝐞𝐧 𝐍𝐨𝐭 𝐭𝐨 𝐔𝐬𝐞:

• Avoid when minority class samples are well-separated from the majority class.
• May not be suitable if your dataset already has strong decision boundaries.

𝟐.𝟒 𝐀𝐝𝐚𝐩𝐭𝐢𝐯𝐞 𝐒𝐲𝐧𝐭𝐡𝐞𝐭𝐢𝐜 𝐒𝐚𝐦𝐩𝐥𝐢𝐧𝐠 (𝐀𝐃𝐀𝐒𝐘𝐍):

📌 𝐖𝐡𝐚𝐭 𝐈𝐭 𝐈𝐬?

ADASYN is an advanced version of SMOTE that focuses on harder-to-learn instances in the minority class. It assigns more weight to these instances and generates synthetic samples accordingly.

𝐈𝐦𝐩𝐥𝐞𝐦𝐞𝐧𝐭𝐚𝐭𝐢𝐨𝐧 𝐢𝐧 𝐏𝐲𝐭𝐡𝐨𝐧 🐍:

from imblearn.over_sampling import ADASYN
adasyn = ADASYN(sampling_strategy='auto', random_state=42)
X_resampled, y_resampled = adasyn.fit_resample(X, y)

✅ 𝐖𝐡𝐞𝐧 𝐭𝐨 𝐔𝐬𝐞:

• Best for datasets where some minority samples are harder to classify than others.
• Helps in improving classifier performance by focusing on difficult instances.

❌ 𝐖𝐡𝐞𝐧 𝐍𝐨𝐭 𝐭𝐨 𝐔𝐬𝐞:

• Avoid ADASYN if your dataset is small, as it may generate too much noise.
• May not be the best choice when class distributions are already well-defined.

𝟑. 𝐂𝐨𝐦𝐩𝐚𝐫𝐢𝐬𝐨𝐧 𝐨𝐟 𝐒𝐚𝐦𝐩𝐥𝐢𝐧𝐠 𝐓𝐞𝐜𝐡𝐧𝐢𝐪𝐮𝐞𝐬:

Comparison Table for Sampling Techniques

𝟒. 𝐂𝐡𝐨𝐨𝐬𝐢𝐧𝐠 𝐭𝐡𝐞 𝐑𝐢𝐠𝐡𝐭 𝐒𝐚𝐦𝐩𝐥𝐢𝐧𝐠 𝐓𝐞𝐜𝐡𝐧𝐢𝐪𝐮𝐞:

• Use SMOTE if you need a general oversampling approach.
• Use Borderline-SMOTE if your dataset has a complex decision boundary.
• Use ADASYN if you want to focus on difficult samples.
• Use Random Sampling if you need a quick, simple solution.

𝐂𝐨𝐧𝐜𝐥𝐮𝐬𝐢𝐨𝐧:

Handling imbalanced data is one of the most crucial steps in building a reliable machine learning model. The right sampling technique can make all the difference between a biased model and one that generalizes well to unseen data.

Get Ayush_Argonda’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

✅ Key Takeaways:

• SMOTE is useful for generating synthetic minority samples without duplication.
• Borderline-SMOTE improves classification near decision boundaries.
• ADASYN focuses on harder-to-learn minority instances.
• Random Sampling is quick but has risks like overfitting (oversampling) or information loss (undersampling).

📌 Next Steps:

• Try implementing these techniques in your own ML projects.
• Experiment with different sampling methods and compare results.
• Read more on ensemble learning — another powerful way to handle class imbalance.

💬 Have you used any of these techniques in your machine learning projects? What worked best for you? Share your thoughts in the comments! If you found this guide helpful, share it with your network and follow me for more machine learning insights. 🚀