Part 3 — Validation, Selection, and Visualization: Evaluating metrics and extracting actionable business insights

by Mariangela Bonghi

In Part 2, we put our four dataset configurations to find optimal parameters for K-Means, Agglomerative Clustering, and GMM.

As a reminder, our journey is divided into three specialized stages to ensure a complete data science pipeline:

• Part 1: EDA, outliers, and feature selection(Completed) — We’ll start by exploring the data, handling outliers, and analyzing features.
• Part 2: Identifying optimal parameters for K-Means, GMM, and Linkage methods for different configurations of the dataset (Completed);
• Part 3: Validation, Selection, and Visualization — Evaluating metrics and extracting actionable business insights(You are here!)

Clustering Validation: Assessing the quality and consistency of the identified models

After identifying the potential optimal parameters for K-Means, Agglomerative Clustering, and GMM algorithms, we define a dictionary to store our candidate configurations. We will now apply specific metrics to evaluate the effectiveness of each approach. Among the available indices, we will use:

• Silhouette Score[1]: Evaluates how compact and well-separated the clusters are. It ranges from -1 to 1, where a value of 1 represents the ideal clustering (points are very close to their own cluster and far from others).
• Calinski-Harabasz Index[2]: It is the ratio of the sum of between-cluster dispersion to within-cluster dispersion. A higher value indicates better clustering, as it signifies that the clusters are dense (low internal dispersion) and well-separated (high external dispersion).
• Davies-Bouldin Index[3]: This index signifies the average similarity between each cluster and its most similar one. Unlike the previous metrics, a lower value (closer to 0) indicates better clustering, as it means the clusters are less similar to each other and more distinct.

By balancing these scores, we will determine which algorithm and feature set provide the most meaningful and stable customer segments.

# Define optimal_k_map dictionary based on the conclusions of the previous sections:
optimal_k_map = {
    'Income-Spending (All Data)': 
    {'K-Means': [5], 'Ward': [5], 'Complete': [7], 'Average': [7], 
         'GMM': [{'n_components': 5, 'covariance_type': 'spherical'},
                {'n_components': 5, 'covariance_type': 'diag'},
                {'n_components': 6, 'covariance_type': 'spherical'},
                {'n_components': 6, 'covariance_type': 'diag'}]},
    'Income-Spending (No Outliers)': 
    {'K-Means': [6], 'Ward': [5], 'Complete': [7], 'Average': [5],
         'GMM': [{'n_components': 5, 'covariance_type': 'spherical'},
                {'n_components': 5, 'covariance_type': 'diag'},
                {'n_components': 6, 'covariance_type': 'spherical'},
                {'n_components': 6, 'covariance_type': 'diag'}]},
    'Age-Income-Spending (All Data)': 
    {'K-Means': [6], 'Ward': [6], 'Complete': [5], 'Average': [5], 
     'GMM': [{'n_components': 5, 'covariance_type': 'diag'},
            {'n_components': 6, 'covariance_type': 'diag'},
            {'n_components': 7, 'covariance_type': 'diag'}]},
    'Age-Income-Spending (No Outliers)': 
    {'K-Means': [6], 'Ward': [6], 'Complete': [6], 'Average': [5],
          'GMM': [{'n_components': 5, 'covariance_type': 'diag'},
            {'n_components': 6, 'covariance_type': 'diag'},
            {'n_components': 7, 'covariance_type': 'diag'}]}
}
# Our dataset configurations
dataset_configurations = [
    {'dataset': X_IncomeSpending_all, 'label': 'Income-Spending (All Data)'},
    {'dataset': X_IncomeSpending_no_outliers, 'label': 'Income-Spending (No Outliers)'},
    {'dataset': X_AgeIncomeSpending_all, 'label': 'Age-Income-Spending (All Data)'},
    {'dataset': X_AgeIncomeSpending_no_outliers, 'label': 'Age-Income-Spending (No Outliers)'}
]
def evaluate_clustering(X, labels):
    """
    Calculate clustering metrics (Silhouette, Calinski-Harabasz, Davies-Bouldin)
    INPUTS:
    - X: The feature matrix (data points).
    - labels: The cluster labels assigned to each data point by the clustering algorithm.
    OUTPUTS (Tuple):
    - n_clusters (int): The number of clusters 
    - n_samples (int): The number of data points
    - silhouette (float): The Silhouette Score.
    - calinski (float): The Calinski-Harabasz Index.
    - davies_bouldin (float): The Davies-Bouldin Index.
    """
    n_samples = len(X)
    silhouette = silhouette_score(X, labels)
    calinski = calinski_harabasz_score(X, labels)
    davies_bouldin = davies_bouldin_score(X, labels)
    return n_samples, silhouette, calinski, davies_bouldin
def highlight_best(s):
    """
    Apply a light green background to the best value for each clustering metric.
    INPUT:
    s (pd.Series): A Pandas Series representing a column of metric scores 
    OUTPUT:
    - Returns 'background-color: lightgreen' for the best value(s) and '' otherwise.
    """
    if s.name not in ['Silhouette (Max is 1)', 'Calinski-Harabasz (Max is Inf)', 'Davies-Bouldin (Min is 0)']:
        return ['' for _ in s]
    if s.name in ['Silhouette (Max is 1)', 'Calinski-Harabasz (Max is Inf)']:
        # Max is the best. We use max() to find the max not-NaN and np.isclose to manage float
        max_val = s.max()
        is_best = np.isclose(s, max_val, equal_nan=False) if not pd.isna(max_val) else [False] * len(s)
    else: 
        # Davies-Bouldin Min is the best
        min_val = s.min()
        is_best = np.isclose(s, min_val, equal_nan=False) if not pd.isna(min_val) else [False] * len(s) 
    return ['background-color: lightgreen' if v else '' for v in is_best]
# Execution
results_dfs = {}
for config in dataset_configurations:
    X = config['dataset']
    label = config['label']
    # Retrieve optimized K values for the current dataset
    optimal_ks = optimal_k_map[label]
    all_results = []
    total_points = len(X)
    # K-Means execution
    for K in optimal_ks['K-Means']:
        kmeans = KMeans(n_clusters=K, init='k-means++', n_init='auto', random_state=42)
        labels_kmeans = kmeans.fit_predict(X)
        all_results.append(['K-Means', K, *evaluate_clustering(X, labels_kmeans)])
    # GMM execution
    for gmm_cfg in optimal_ks['GMM']:
        K = gmm_cfg['n_components']
        cov_type = gmm_cfg['covariance_type']
        gmm = GaussianMixture(n_components=K, covariance_type=cov_type, random_state=42,n_init=5)
        labels_gmm = gmm.fit_predict(X)
        all_results.append([f'GMM ({cov_type})', K, *evaluate_clustering(X, labels_gmm)])
    # AgglomerativeClustering execution
    for method in ['ward', 'complete', 'average']:
        K_list = optimal_ks[method.capitalize()]
        for K in K_list: 
            agglo = AgglomerativeClustering(n_clusters=K, linkage=method)
            labels_agglo = agglo.fit_predict(X)
            all_results.append([f'Linkage {method.capitalize()}', K, *evaluate_clustering(X, labels_agglo)])  
    # Report Creation and Display
    print(f"\n Comparative Results: {label}")
    df_result = pd.DataFrame(all_results, columns=[
        'Method', 'Found K', 'Evaluated Points', 
        'Silhouette (Max is 1)', 'Calinski-Harabasz (Max is Inf)', 'Davies-Bouldin (Min is 0)'
    ])
    #Cleaning and Sorting
    sort_order = ['K-Means', 'Linkage Ward', 'Linkage Complete', 'Linkage Average', 'GMM']
    df_result['Method_Sort'] = pd.Categorical(df_result['Method'], categories=sort_order, ordered=True)
    df_result = df_result.sort_values(by=['Method_Sort', 'Found K']).drop(columns='Method_Sort')
    styled_df = df_result.style.apply(highlight_best, axis=0).format({
            'Silhouette (Max is 1)': "{:.3f}", 'Calinski-Harabasz (Max is Inf)': "{:.0f}", 'Davies-Bouldin (Min is 0)': "{:.3f}",
            'Found K': "{:.0f}",
        }, na_rep='N/A') \
        .hide(axis='index')
    display(styled_df)
    results_dfs[label] = df_result

Looking at the results objectively, the GMM with spherical covariance and K=5 would be the theoretically optimal choice. Its Silhouette Score is the absolute maximum (0.560), the Calinski-Harabasz index (263) is essentially at its peak (second only to the maximum of 264), and the Davies-Bouldin index is among the lowest recorded.

However, when examining the metrics for the dataset containing outliers, K-Means with K=5 clearly emerges as the winner. Comparing the performance of this model against the GMM (spherical, K=5), we do not observe significant discrepancies. Although the indices for the full dataset are slightly lower, they are so close to the “No Outliers” version that they do not justify a strong preference for one model over the other.

This observation, combined with the fact that the removed outliers do not appear to be data entry errors based on their values, leads us to conclude that removing them is unnecessary. Therefore, we choose to proceed with the full dataset using K-Means with K=5.

In this case, for both the dataset with and without outliers, K-Means with K=6 emerges as the superior model. The conclusions align with those from the 2D analysis.

While the metrics for the “No Outliers” dataset show slight improvements, the differences are so marginal that there is no significant statistical evidence to justify using the cleaned dataset over the original one. This fact, combined with the observation that the two removed outliers do not appear to be data entry errors, leads us to conclude that K-Means with K=6 using the full dataset is the most appropriate choice.

Clustering Validation: The Final Verdict

• 2D Top Performer: K-Means (K=5) for the full dataset;
• 3D Top Performer: K-Means (K=6) for the full dataset.

Clustering and Interpretation of Results: “Annual Income” and “Spending Score” features — KMeans with K=5

Let’s evaluate and visualize our winner for the 2D configuration:

kmeans = KMeans(n_clusters=5, init='k-means++', n_init='auto', random_state=42)
labels_kmeans = kmeans.fit_predict(X_IncomeSpending_all)
df['Cluster_KMeans_2feat'] = labels_kmeans
total_population = len(df)
# View the Statistical Table with Percentage Data
stats_kmeans_2D = df.groupby('Cluster_KMeans_2feat').agg(
    Cluster_Size=('Cluster_KMeans_2feat', 'size'),
    Average_Income=('Annual Income (k$)', 'mean'),
    Average_Spending_Score=('Spending Score (1-100)', 'mean'),
    Average_Age=('Age', 'mean'),
    # Male and Female count
    Male_Count=('Gender', lambda x: (x == 'Male').sum()),
    Female_Count=('Gender', lambda x: (x == 'Female').sum())
).reset_index()
# Percentage evaluation
stats_kmeans_2D['Population_%'] = (stats_kmeans_2D['Cluster_Size'] / total_population) * 100
stats_kmeans_2D['Male_%'] = (stats_kmeans_2D['Male_Count'] / stats_kmeans_2D['Cluster_Size']) * 100
stats_kmeans_2D['Female_%'] = (stats_kmeans_2D['Female_Count'] / stats_kmeans_2D['Cluster_Size']) * 100
# Select the columns
stats_final = stats_kmeans_2D[['Cluster_KMeans_2feat', 'Population_%','Average_Income', 'Average_Spending_Score','Average_Age', 'Male_%', 'Female_%']]
# View final table
display(stats_final.style.hide(axis='index').set_caption("K-Means (K=5) Cluster Statistics (Percentages)").format
        ({'Population_%': "{:.1f}%", 'Average_Income': "{:.1f}", 'Average_Spending_Score': "{:.1f}", 'Average_Age': "{:.1f}",
           'Male_%': "{:.1f}%", 'Female_%': "{:.1f}%"}))
# Scatter Plot 
plt.figure(figsize=(12, 8))
sns.scatterplot(x='Annual Income (k$)', y='Spending Score (1-100)', hue='Cluster_KMeans_2feat', data=df, palette='viridis', style='Gender', s=100)
plt.title('K-Means Clustering (K=5) "Spendig Score vs Annual Income" ')
plt.xlabel('Annual Income (k$)')
plt.ylabel('Spending Score (1-100)')
plt.grid(True, linestyle='--', alpha=0.6)
# legend
plt.legend(title='Legenda', loc='lower left', bbox_to_anchor=(1.01, 0), borderaxespad=0)
plt.tight_layout(rect=[0, 0, 1, 1])
plt.show()

Business Insights:

“Low-Income Groups” (Clusters 2 & 4)
These segments represent customers with limited annual income but opposite spending habits:

• Cluster 2 — “Young High Spenders”: High spending score and the youngest average age. These are customers who prioritize trends and immediate gratification despite budget constraints. Strategy: Social media marketing, trendy/viral product drops, and “Buy Now, Pay Later” options.
• Cluster 4 — “Mature Low Spenders”: Lowest spending score and mature age. These are cost-conscious customers who focus on essentials. Strategy: Focus instead on utility-based marketing, durability, and “essential-only” promotions to build trust.

“High-Income Groups” (Clusters 1 & 3)
These are customers with the highest annual income:

• Cluster 1 — “Luxury Enthusiasts”: Extremely high spending (82.1) and relatively young (32.7). Strategy: Exclusive VIP events, personalized concierge services, and premium loyalty rewards.
• Cluster 3 — “High Income Savers”: High income, but the lowest spending score (17.6). More mature (40.7) and selective. They likely prioritize investments or high-end durable goods over frequent shopping. Strategy: “Quiet luxury” marketing. Focus on quality, craftsmanship, and long-term value to justify the price point.

“Balanced Cluster” (Cluster 0)
Represent the majority of the population (40.5%). Income (55.3k), spending (49.5), and age (42.7 years) values are close to the dataset’s overall average. No extreme behaviors. Spending is proportional to income. Strategy: As the largest group, they need mass-market marketing. The goal is to increase loyalty and purchase volume via standard promotions, value bundles, and direct communication on everyday products.

Clustering and Interpretation of Results: “Age”, “Annual Income” and “Spending Score” features — KMeans with K=6

Let’s evaluate and visualize our winner for the 3D configuration:

K = 6
kmeans = KMeans(n_clusters=K, init='k-means++', n_init='auto', random_state=42)
labels_kmeans = kmeans.fit_predict(X_AgeIncomeSpending_all)
df['Cluster_KMeans_3feat'] = labels_kmeans
total_population = len(df) 
# View the Statistical Table with Percentage Data
stats_kmeans_3D = df.groupby('Cluster_KMeans_3feat').agg(
    Cluster_Size=('Cluster_KMeans_3feat', 'size'),
    Average_Income=('Annual Income (k$)', 'mean'),
    Average_Spending_Score=('Spending Score (1-100)', 'mean'),
    Average_Age=('Age', 'mean'),
    # Male and Female count
    Male_Count=('Gender', lambda x: (x == 'Male').sum()),
    Female_Count=('Gender', lambda x: (x == 'Female').sum())
).reset_index()
# Percentage evaluation
stats_kmeans_3D['Population_%'] = (stats_kmeans_3D['Cluster_Size'] / total_population) * 100
stats_kmeans_3D['Male_%'] = (stats_kmeans_3D['Male_Count'] / stats_kmeans_3D['Cluster_Size']) * 100
stats_kmeans_3D['Female_%'] = (stats_kmeans_3D['Female_Count'] / stats_kmeans_3D['Cluster_Size']) * 100
# Select the columns
stats_final = stats_kmeans_3D[['Cluster_KMeans_3feat','Population_%', 'Average_Income','Average_Spending_Score', 'Average_Age', 'Male_%', 'Female_%']]
# View final table
display(stats_final.style.hide(axis='index').set_caption(f"K-Means (K={K}) Cluster Statistics (Percentages)").format({
            'Population_%': "{:.1f}%", 'Average_Income': "{:.1f}", 'Average_Spending_Score': "{:.1f}", 'Average_Age': "{:.1f}",
            'Male_%': "{:.1f}%", 'Female_%': "{:.1f}%"}))
#Plot 3D
fig = plt.figure(figsize=(12, 8)) 
ax = fig.add_subplot(111, projection='3d')
cmap = plt.colormaps.get_cmap('viridis') 
markers = {'Male': 'o', 'Female': 's'} # Different markers for gender
scatters = []
for gender_val, marker in markers.items():
    # Filter data (using numeric 0/1)
    df_gender = df[df['Gender'] == gender_val]
    if not df_gender.empty:
        scatter = ax.scatter(
            df_gender['Age'], df_gender['Annual Income (k$)'], df_gender['Spending Score (1-100)'], 
            c=df_gender['Cluster_KMeans_3feat'], cmap=cmap, marker=marker, s=60, label=gender_val, vmin=0, vmax=K-1) 
        scatters.append(scatter)
ax.set_title(f'K-Means Clustering (K={K}) in 3D: Cluster vs Gender')
ax.set_xlabel('Age')
ax.set_ylabel('Annual Income (k$)')
ax.set_zlabel('Spending Score (1-100)')
if scatters:
    legend1 = ax.legend(*scatters[0].legend_elements(), title="Clusters", loc='upper left', bbox_to_anchor=(1.05, 1.0))
    ax.add_artist(legend1)
# Gender Marker Legend
handles = [plt.Line2D([], [], marker='o', color='gray', linestyle='None', label='Male'),
           plt.Line2D([], [], marker='s', color='gray', linestyle='None', label='Female')]
ax.legend(handles=handles, title="Gender", loc='upper left', bbox_to_anchor=(1.05, 0.6))
plt.tight_layout()
plt.show()

3D Cluster Analysis: The Impact of Age
While the core clusters remain consistent with the 2D model, adding Age allows us to significantly refine the previously vague “Balanced Cluster”. In the 2-feature model, Cluster 0 appeared as a monolithic “average” group; however, the 3D analysis reveals it is actually composed of two distinct generational segments:
- Cluster 0 — “Senior Traditionalists”:
Mature customers (avg. age 56.3) with a majority female (57.8%) profile and Average income/spending; likely the most loyal but least impulsive.
Strategy: Traditional loyalty programs, comfort-focused products, and high-touch in-store assistance.
- Cluster 3 — “Junior Pragmatists”:
Very young (avg. age 26.1), predominantly female (60%)
Strategy: Aesthetic-driven social campaigns (Instagram/TikTok) and mobile-app gamification.

Conclusion
The 3-feature model proves that Age is a useful differentiator. While Income and Spending tell us what they can buy, Age tells us how to talk to them and which channels to use.

Conclusion & Final Thoughts

After a rigorous comparison across three algorithmic families and four dataset configurations, a clear pattern has emerged. While advanced probabilistic models like GMM offer theoretical elegance, K-Means remains the most robust and interpretable choice for this business case.

The transition from a 2D to a 3D model proved that Age is more than just a variable; it is a lens that splits ‘average’ consumers into generational segments with vastly different digital habits.

This journey demonstrates that clustering is as much an art as it is a science. Moving between the precision of coefficients and the subjectivity of analytical judgment, we learn that there is no universal recipe. Instead, we must find our own way through a labyrinth of trade-offs, where the best model is not just the most statistically sound, but the one that makes the most sense for our goals.

Thanks for following this series! The complete code for this project is available on GitHub

References

[1] GeeksforGeeks, What is Silhouette Score (2026), Technical Tutorial
[2] Misha Sv, Calinski-Harabasz Index for K-Means Clustering Evaluation using Python (2022), Towards Data Science, Article Link
[3] Misha Sv, Davies-Bouldin Index for K-Means Clustering Evaluation in Python (2021), Towards Data Science, Article Link

From Math to Marketing: Validating Clusters and Extracting Actionable Insights was originally published in Towards AI on Medium, where people are continuing the conversation by highlighting and responding to this story.

Part 2 — Identifying optimal parameters for K-Means, GMM, and Linkage methods

by Mariangela Bonghi

In Part 1, we laid the analytical foundation by exploring our data, handling outliers, and using Mutual Information to identify our most informative features. We moved from raw data to a strategic framework of four different dataset configurations.

As a reminder, our journey is divided into three specialized stages to ensure a complete data science pipeline:

• Part 1: EDA, outliers, and feature selection(Completed) — We’ll start by exploring the data, handling outliers, and analyzing features.
• Part 2: Identifying optimal parameters for K-Means, GMM, and Linkage methods for different configurations of the dataset (You are here!).
• Part 3: Validation, Selection, and Visualization — Evaluating metrics and extracting actionable business insights.

Moving from Preparation to Pattern Discovery

Now that our data is primed, it’s time to move into the modeling phase. In this part, we will identify the optimal parameters for four dataset configurations across three core algorithms: K-Means, Agglomerative Hierarchical Clustering, and Gaussian Mixture Models (GMM).”

K-Means Clustering

K-Means [1,2] is a clustering algorithm that groups n observations into k clusters based on the similarity between data points.
The algorithm begins with an initialization phase: given the desired number of clusters (the value of k), the centroids of each cluster are typically initialized (e.g., randomly or via K-Means++).

From this point, the following steps are iterated:

• Each data point is evaluated and assigned to its nearest centroid.
• The centroids are recalculated based on the new assignments.

The process stops when the centroids don’t move significantly.

Finding the Optimal K

To apply this algorithm effectively, the optimal value for k must be determined. This is achieved using the Elbow Method: a graph is plotted showing Inertia (also known as the Within-Cluster Sum of Squares, or WCSS) as a function of the number of clusters (k).
Inertia quantifies the internal cohesion of the clusters; a lower value indicates that points are closer to their respective centroids. The optimal k is found at the “elbow” of the curve. Beyond this point, adding more clusters does not significantly improve the result (i.e., the decrease in inertia becomes marginal).

# Define the range of K (number of clusters) to test
K_range = range(1, 12) 
# Define variables that we use in the algorithms 
dataset_configurations = [
    {'dataset': X_IncomeSpending_all,'label': 'Income-Spending (All Data)'},
    {'dataset': X_AgeIncomeSpending_all,'label': 'Age-Income-Spending (All Data)'},
    {'dataset': X_IncomeSpending_no_outliers,'label': 'Income-Spending (No Outliers)'},
    {'dataset': X_AgeIncomeSpending_no_outliers,'label': 'Age-Income-Spending (No Outliers)'}
]
# Evaluation: Run the clustering process for each dataset configuration
results = {}
for dc in dataset_configurations:
    data_label = dc['label']
    data_set = dc['dataset']
    # Calculate inertia for each k in the range
    inertias = [KMeans(n_clusters=k, init='k-means++', n_init='auto', random_state=42).fit(data_set).inertia_ for k in K_range]
    k_mapping = dict(zip(K_range, inertias))   
    results[data_label] = {'inertias': inertias, 'k_mapping': k_mapping}
# Visualize the Elbow Method results
fig, axes = plt.subplots(nrows=2, ncols=2, figsize=(18, 10)) 
axes = axes.flatten() 
labels = list(results.keys())
for i, ax in enumerate(axes):
    label = labels[i]
    inertias = results[label]['inertias']    
    # Plotting the elbow curve
    ax.plot(K_range, inertias, marker='o', linestyle='-', color='b') 
    # Adding text annotations for each inertia value
    for k, val in zip(K_range, inertias):
        ax.annotate(f'{val:.1f}', xy=(k, val), xytext=(0, 7), textcoords='offset points', ha='center', fontsize=8)
    ax.set_title(f"Elbow Method: {label}", fontsize=12)
    ax.set_xlabel("Number of Clusters (K)", fontsize=10)
    # Set Y-axis label for the leftmost plots in each row
    ax.set_ylabel("Inertia", fontsize=10)        
    ax.set_xticks(K_range)
    ax.grid(True, linestyle='--', alpha=0.6)
fig.tight_layout()
plt.show()

Elbow Method Analysis Findings

• Income & Spending (2D) — Graphs on the left: In the complete dataset, a clear elbow appears at K=5. However, when outliers are removed, we observe a significant secondary drop in inertia between K=5 and K=6. This suggests that removing extreme values unmasks a sixth distinct customer segment that was previously obscured by noise.
• Age, Income & Spending (3D) — Graphs on the right: The introduction of the ‘Age’ dimension smoothens the curve, indicating a more complex data distribution. A slight bend is visible at K=6, suggesting that this is the optimal number of clusters.

Impact of Outliers: Across both feature sets, the “No Outliers” plots exhibit lower overall inertia. This suggests that removing outliers in “Annual Income” could lead to more compact clusters from a purely mathematical standpoint. However, whether this actually results in a more meaningful or stable segmentation remains to be seen; in the validation section, we will employ statistical metrics to quantify if this impact is truly significant or merely marginal, as previously suggested by the Mutual Information analysis.

Candidate Selection for Validation

We will carry forward the following settings for the validation phase:

• 2D Features: K=5 (Original dataset) and K=6 (No outliers dataset);
• 3D Features: K= 6 (both Original & No outliers dataset).

Agglomerative Clustering

Agglomerative Clustering is a hierarchical algorithm used to group data points, representing their relationships via a tree-like structure called a dendrogram.

It follows a bottom-up approach:

• It begins by treating every single data point as an individual cluster.
• It then proceeds by iteratively merging the two closest clusters based on a specific distance metric.

This process continues until all points are merged into a single large cluster.

The way the “distance” between clusters is calculated is defined by the linkage method. The four most common methods are:

• Ward’s Method: Merges clusters that result in the minimum increase in total within-cluster variance;
• Single Linkage (Minimum Distance): Defines the distance between two clusters as the minimum distance between any single point in the first cluster and any single point in the second;
• Complete Linkage (Maximum Distance): Defines the distance as the maximum distance between any single point in the first cluster and any single point in the second;
• Average Linkage (Mean Distance): Calculates the distance as the average distance between all possible pairs of points between two clusters.

# Define a list for the linkage methods
LINKAGE_METHODS = ['ward', 'complete', 'average', 'single']
# Define variables that we use in the algorithms 
dataset_configurations = [
    {'dataset': X_IncomeSpending_all, 'label': 'Income-Spending (All Data)', 'heights': [8, 2, 1.5]},
    {'dataset': X_IncomeSpending_no_outliers, 'label': 'Income-Spending (No Outliers)', 'heights': [8, 2, 1.5]},
    {'dataset': X_AgeIncomeSpending_all, 'label': 'Age-Income-Spending (All Data)', 'heights': [7.5, 3.4, 2.1]},
    {'dataset': X_AgeIncomeSpending_no_outliers, 'label': 'Age-Income-Spending (No Outliers)', 'heights': [7.6, 3.4, 2.1]}
]
fig, axes = plt.subplots(nrows=4, ncols=4, figsize=(28, 20))
# Outer Loop: Iterates over Dataset Configurations
for row_idx, dc in enumerate(dataset_configurations):
    data_set = dc['dataset']
    data_label = dc['label']
    # Inner Loop: Iterates over Linkage Methods
    for col_idx, method in enumerate(LINKAGE_METHODS):
        # Linkage matrix
        Z = linkage(data_set, method=method)
        # Dendrogram generation
        ax_dendro = axes[row_idx, col_idx]
        dendrogram(Z, truncate_mode='lastp', ax=ax_dendro)
        # Add a horizontal threshold line 
        if 'heights' in dc and col_idx < len(dc['heights']):
            h = dc['heights'][col_idx]
            ax_dendro.axhline(y=h, color='r', linestyle='--', linewidth=1.5)
        title = f'{data_label} - {method.capitalize()}'
        ax_dendro.set_title(title, fontsize=12)
        ax_dendro.set_xlabel('Formed Clusters', fontsize=10)
        ax_dendro.set_ylabel('Distance', fontsize=10) 
        ax_dendro.tick_params(axis='x', which='both', bottom=True, labelbottom=False)

fig.tight_layout()
plt.show()

The output of the previous code is a grid of 16 dendrograms. Below, we present them using a different layout, organized configuration by configuration, along with their respective results.

Dataset with Incoming & Spending Score Features:

• Ward’s Method: At h=8, this method produces distinct and compact clusters. Comparing “All Data” to “No Outliers,” it is evident that outliers do not influence cluster stability;
• Complete Method: A cut at approximately h=2 yields 7 clusters. In this case as well, the presence of outliers does not alter the overall structure.

Dataset with Incoming & Spending Score Features:

• Average Method: The cut at approximately 1,5 highlights the impact of data cleaning; removing outliers reveals 5 clusters instead of 7, suggesting that this method is more sensitive to extreme values.
• Single Method: Exhibits a clear “chaining” effect. Most points merge at very low distance levels, making it unsuitable for capturing underlying spherical cluster shapes.

Dataset with Age, Incoming & Spending Score Features:

• Ward Method: Shows high stability at h=7.5 across both datasets. It remains the most robust method, maintaining a consistent cluster count regardless of the added dimension or noise.
• Complete Method: A cut at approximately h=3.3 reveals the impact of outlier removal, which unmasks 6 clusters instead of 5.

Dataset with Age, Incoming & Spending Score Features:

• Average Method: At approximately h=2, this method produces 5 clusters. In this case, the outliers do not change the result.
• Single Method: The chaining effect is evident, confirming this method is not recommended for this dataset.

Candidate Selection for Validation

We will carry forward the following settings for the validation phase:

• 2D Features: Ward K=5; Complete K=7 for both Original and No outliers dataset; Average K=7 for original and 5 for No outliers dataset;
• 3D Features: Ward K=6; Average K=5 for both Original and No outliers dataset; Complete K=5 for original and 6 for No outliers dataset.

Gaussian Mixture Model (GMM)

A Gaussian Mixture Model (GMM) [4, 2] is a probabilistic clustering method that assumes the data is generated from a mixture of several Gaussian distributions, each representing a distinct cluster.
The algorithm follows the Expectation-Maximization (EM) framework, which iterates through two main steps until convergence:

• E-step (Expectation): Each data point is assigned a probability of belonging to each Gaussian cluster (soft assignment);
• M-step (Maximization): The parameters of the Gaussians — specifically the mean, covariance, and mixing weight — are updated based on the probabilities calculated in the E-step.

To determine the ideal number of clusters and model structure, two primary metrics are used:

• Bayesian Information Criterion (BIC);
• Akaike Information Criterion (AIC)

Both metrics estimate the quality of the model by balancing goodness-of-fit with model complexity; the lower the score, the better the model.

In the analysis below, a grid search is performed to systematically test various GMM configurations by varying the number of components and the covariance type (spherical, tied, diag, and full). The results are plotted to visualize how complexity affects the BIC and AIC scores, identifying the optimal cluster structure for each dataset.

dataset_configurations = [
    {'dataset': X_IncomeSpending_all,'label': 'Income-Spending (All Data)'},
    {'dataset': X_AgeIncomeSpending_all,'label': 'Age-Income-Spending (All Data)'},
    {'dataset': X_IncomeSpending_no_outliers,'label': 'Income-Spending (No Outliers)'},
    {'dataset': X_AgeIncomeSpending_no_outliers,'label': 'Age-Income-Spending (No Outliers)'}
]
# Parameter grid for GMM
param_grid = {
    "n_components": range(3, 10),
    "covariance_type": ["spherical", "tied", "diag", "full"]
}
# GridSearchCV is designed to MAXIMIZE a score. Since the best BIC/AIC is the 
# lowest value, we return the negative value so the algorithm can maximize it.
def gmm_bic_score(estimator, X):
    """Returns the NEGATIVE BIC: allows GridSearchCV to find the minimum by maximizing."""
    return -estimator.bic(X)
def gmm_aic_score(estimator, X):
    """Returns the NEGATIVE AIC: allows GridSearchCV to find the minimum by maximizing."""
    return -estimator.aic(X)
# Set up a grid to accommodate all onfigurations
fig, axes = plt.subplots(2, 2, figsize=(22, 12))
axes = axes.flatten()
for i, dc in enumerate(dataset_configurations):
    ax = axes[i]
    data_set = dc['dataset']
    # The 'scoring' parameter uses our negative function to correctly rank the models.
    grid_search = GridSearchCV(GaussianMixture(random_state=42, n_init=5), param_grid=param_grid, scoring=gmm_bic_score,
        #Determines the cross-validation splitting strategy: we use identical indices to force calculation on the full dataset                      
        cv=[(np.arange(len(data_set)), np.arange(len(data_set)))]
    )
    grid_search.fit(data_set)
    # Process results
    results = []
    # Loop through the grid results (which are currently stored as negative scores)
    for params, score in zip(grid_search.cv_results_['params'], grid_search.cv_results_['mean_test_score']):
        # To display real (positive) AIC and BIC values, we fit the model briefly.
        # This is necessary because GridSearchCV only stores the primary 'scoring' metric.
        model = GaussianMixture(**params, random_state=42, n_init=5).fit(data_set)
        results.append({
            **params,'bic': model.bic(data_set),'aic': model.aic(data_set)
        })
    df_result = pd.DataFrame(results)
    cov_types = df_result["covariance_type"].unique()
    n_comps = sorted(df_result["n_components"].unique())
    # Plotting
    x = np.arange(len(n_comps))
    width = 0.2 
    for j, cov in enumerate(cov_types):
        subset = df_result[df_result["covariance_type"] == cov].sort_values("n_components")
        # Plot BIC (Bar)
        ax.bar(x + (j * width), subset["bic"], width, label=f"{cov} (BIC)", alpha=0.8)
        # Plot AIC (Points)
        ax.plot(x + (j * width), subset["aic"], marker='o', linestyle='', markersize=4, color='black', alpha=0.5)
    ax.set_title(f"BIC & AIC Score: {dc['label']}\n(Black Points = AIC)", fontsize=14)
    ax.set_xticks(x + width * 1.5)
    ax.set_xticklabels(n_comps)
    ax.set_xlabel("Number of Clusters")
    ax.set_ylabel("Score (Lower is Better)")
    ax.legend(title="Covariance", fontsize='x-small', ncol=2)
    ax.grid(axis='y', linestyle='--', alpha=0.3)
plt.tight_layout()
plt.show()

The BIC analysis reveals a relatively flat profile across the tested range of clusters (K=3 to K=9), and the removal of outliers does not significantly alter this trend. This lack of a clear minimum suggests that a Gaussian Mixture Model might not be the most effective algorithm for capturing the underlying structure of this specific dataset.
Nevertheless, I have selected candidate configurations that balance the lowest AIC/BIC scores with model parsimony to avoid unnecessary complexity.

Candidate Selection for Validation

We will carry forward the following settings for the validation phase:

• 2D Features (Income-Spending): K=5 and K=6 using Spherical and Diag covariance types for both versions of the dataset;
• 3D Features (Age-Income-Spending): K=5, K=6, and K=7 using Diag covariance for both versions of the dataset.

Conclusion & Next Steps

We have put our four dataset configurations to the test, uncovering how different algorithms react to noise and dimensionality. While K-Means and Ward’s linkage showed remarkable stability, the GMM analysis suggested a more nuanced underlying structure.

But which of these candidates truly represents our customer base?

In Part 3, we will move from parameter tuning to final validation. We will use statistical metrics to select the winning model and, most importantly, translate these clusters into actionable marketing strategies.

Code Availability: The full GitHub repository will be shared in Part 3.

Read Part 3: Validation, Selection, and Visualization — Evaluating metrics and extracting actionable business insights →

References

[1] Imad Dabbura, K-means Clustering: Algorithm, Applications, Evaluation Methods, and Drawbacks (2018), Towards Data Science

[2] Scikit-Learn Documentation (2024). Kmeans, Gaussian Mixture Model Selection, GridSearchCV

[3] Mounica Kommajosyula, Clustering 101: Mastering Agglomerative Hierarchical Clustering (2024), Python in Plain English

[4] Vivienne DiFrancesco, Gaussian Mixture Models for Clustering (2021), Towards Data Science

3 Algorithms and 4 Configurations for Customer Segmentation was originally published in Towards AI on Medium, where people are continuing the conversation by highlighting and responding to this story.

Part 1: Exploratory Data Analysis, handling outliers, and feature selection using Mutual Information

by Mariangela Bonghi

A short time ago, I attended a course on Unsupervised Learning. The concepts were fascinating, and I couldn’t wait to put all that new theory into practice. I decided to focus my attention on clustering, using the classic Kaggle Mall Customer Dataset as my playground.

The clustering process can be summarized into four steps (often iterative):

• Feature extraction and selection;
• Algorithm selection and tuning;
• Cluster validation;
• Interpretation of results.

When I started, I was looking for a “universal recipe” to apply to any dataset. I quickly discovered that such a recipe doesn’t exist. Instead, clustering is a field where every step requires a choice — some obvious,
some less so — guided by data knowledge, business logic, and analytical intuition.
While I didn’t find a magic formula, this experience helps me answer some questions: Do I need to consider all the features? Are there outliers, and how significantly will they warp my results? Should I try other algorithms instead of just K-means?

This guide is built to answer the very questions I had to solve for myself.

The Roadmap

Since the journey is extensive, I have divided this project into three specialized parts to ensure we cover every detail of the pipeline:

• Part 1: EDA, outliers, and feature selection(You are here!) — We’ll start by exploring the data, handling outliers, and analyzing features.
• Part 2: Identifying optimal parameters for K-Means, GMM, and Linkage methods for different configurations of the dataset.
• Part 3: Validation, Selection, and Visualization — Evaluating metrics and extracting actionable business insights

Let’s start from the basics: What is Clustering?

In short, clustering is the branch of Unsupervised Learning that helps us find hidden patterns in our data, grouping observations into homogeneous clusters with similar characteristics.
Consider how powerful this is in marketing: if you truly understand your customers’ behaviors and traits, you can create hyper-targeted campaigns. You’ll know exactly how to communicate with each group, which channel to use, and which products or offers will resonate most with them.

Exploratory Data Analysis (EDA)

First, let’s import the libraries and take a look at our raw data.

# Import Libraries
import pandas as pd
import seaborn as sns
import matplotlib.pyplot as plt
import numpy as np
from IPython.display import display, HTML

from sklearn.preprocessing import StandardScaler
from sklearn.feature_selection import mutual_info_regression, mutual_info_classif
from sklearn.cluster import KMeans, AgglomerativeClustering
from scipy.cluster.hierarchy import linkage, dendrogram
from sklearn.model_selection import GridSearchCV
from sklearn.mixture import GaussianMixture
from sklearn.neighbors import NearestNeighbors
from sklearn.metrics import silhouette_score, calinski_harabasz_score, davies_bouldin_score
# Load Data
df = pd.read_csv("Mall_Customers.csv")
df.head()

There are four features: “Gender”, “Age”, “Annual Income”, and “Spending Score”.

“Customer ID": Serves only as an identifier and will be dropped from the dataset that we’ll use for the clustering.

df.describe()

• “Annual Income” ranges up to 137k, while “Age” only goes up to 70. This suggests that scaling is needed.
• The jump from the 75th percentile (78k) to the maximum (137k) in “Annual Income” suggests outliers.

Before analyzing individual features, we must ensure our dataset is structurally sound.

# Check for missing values
print("Missing Values\n")
print(df.isna().sum())

# Check for duplicated rows
print("\nDuplicated Rows\n")
print(df.duplicated().sum())

As shown above, a quick check reveals a perfectly “clean” dataset with no missing values or duplicates.

To visualize the relationships between features, we use a pair plot.

# Remove unnecessary CustomerID column
df_features= df.copy()
df_features.drop(["CustomerID"],axis=1,inplace=True)
# Plot pairwise relationships in the dataset
custom_palette = {'Male': '#4C72B0', 'Female': '#DD8452'}
sns.pairplot(df_features,hue='Gender',palette=custom_palette)
plt.show()

Key insights

• Spending Score vs Annual Income (Row 3, Column 2):
  This plot is the most informative for clustering purposes. It reveals five distinct clusters forming a clear pattern. This visual evidence strongly suggests that a K-Means algorithm with K=5 would likely be optimal for this dataset when focusing on these two variables.
• Spending Score vs Age (Row 3, Column 1):
  High spending scores are notably concentrated among younger customers. Age appears to be a significant factor for isolating high-value segments.
• The role of Gender (Color: Orange = Female, Blue = Male):
  Observing the various plots, Gender doesn’t appear to be a variable that makes a clear, stark difference in the overall cluster formation. However, the Age vs. Spending Score plot suggests a slight prevalence of male customers (blue) in the low-spending segments.

Detecting and handling outliers

Next, it’s time to detect and handle outliers: we’ll use the IQR (Interquartile Range) method. [1,2,3]

The IQR method involves the following steps:

• Calculate the first quartile (Q1) and the third quartile (Q3): these represent the thresholds below which 25% and 75% of the data fall, respectively.
• Calculate the difference Q3 — Q1: this is what we call the Interquartile Range (IQR) and identifies the central part of the dataset.
• Establish boundaries beyond this range — typically Q1–1.5 times IQR and Q3 + 1.5 times IQR: data points falling outside these boundaries are defined as outliers.

We can visualize them, if any, with the boxplot:

# Visualizing data distribution and potential outliers using boxplots
fig, axes = plt.subplots(nrows=1,ncols=3,figsize=(15,5))
sns.boxplot(y=df_features['Age'], ax=axes[0],color="#789fb3")
sns.boxplot(y=df_features['Annual Income (k$)'], ax=axes[1],color="#789fb3")
sns.boxplot(y=df_features['Spending Score (1-100)'], ax=axes[2],color="#789fb3")
plt.show()

As shown in the box plots, there is only one visible outlier in the ‘Annual Income (k$)’ column. We will now determine the exact number of outliers, as the visual representation may hide multiple overlapping data points. To do this, we will find how many points there are above the Q3 + 1.5 times the IQR.

# Evaluation of the first and third quartiles
Q1 = np.percentile(df_features['Annual Income (k$)'], 25, method='midpoint')
Q3 = np.percentile(df_features['Annual Income (k$)'], 75, method='midpoint')
# IQR is calculated as the difference between Q3 and Q1
IQR = Q3 - Q1
# Upper bound: Q3+1.5∗IQR
upper_bound = Q3 + 1.5 * IQR
# Outlier Count
outlier = df_features[(df_features['Annual Income (k$)'] >= upper_bound)]
print(f"Number of Outliers for the column Annual Income (k$): {len(outlier)}")   

Number of Outliers for the column Annual Income (k$): 2

# Outlier removal: filter the dataframe to keep only records within the upper bound
upper_array = np.where(df_features['Annual Income (k$)'] >= upper_bound)[0]
df_features_no_outliers=df_features.drop(index=upper_array)
# Result verification
print("Old Shape: ", df_features.shape)
print("New Shape: ", df_features_no_outliers.shape)

Old Shape:  (200, 4)
New Shape:  (198, 4)

Now we have:

- df_features: 
our original dataset without “Customer ID” (200 rows)
- df_features_no_outliers: 
our original dataset without “Customer ID” and without outliers (198 rows)

Feature Selection via Mutual Information

Before proceeding with the clustering phase, it is essential to determine which features provide the most value to our model. In our dataset, there are four features: Spending Score, Annual Income, Age, and Gender. We need to identify which ones contribute to meaningful patterns and which ones act like noise.

To answer this, I employed Mutual Information (MI) [4,5,6]. Like correlation, MI measures the relationship between two variables. However, while standard correlation coefficients (such as Pearson’s) are limited to detecting linear dependencies, MI captures any kind of statistical dependency, whether linear or non-linear.

How it works
Mathematically, given two variables X and Y, MI quantifies how much knowing the value of X reduces our uncertainty about Y.

• Low MI Value: Indicates that the variables are independent. Knowing the value of X provides no information about Y.
• High MI Value: Indicates a significant degree of dependency. Knowing X allows us to infer information about Y with greater confidence.

In the context of clustering, we prioritize features with high MI scores for two practical reasons:

Noise Reduction: A feature with a low MI score is statistically independent of the others. In clustering, such a variable behaves like “noise” — it scatters data points randomly, making it harder for algorithms like K-Means to find cohesive groups.

Pattern Recognition: A high MI score means the variable “participates” in the dataset’s internal logic. For example, if Annual Income has a high MI with Spending Score, they work together to define distinct customer segments (e.g., “high earners who spend little”).

Practical Application

In our analysis, the function make_mi_scores treats each feature as a target and calculates its relationship with all other variables. By summing these scores, we measure the “centrality” of a feature. A high cumulative MI indicates a variable that shares significant information with the rest of the system, making it essential for defining stable clusters. We distinguish between data types to ensure accuracy:

• For the Gender feature (categorical/discrete), we utilized mutual_info_classif;
• For continuous features (Age, Income, Score), we employed mutual_info_regression.

This distinction is crucial because the underlying entropy estimation — the mathematical foundation of MI — differs when dealing with discrete labels versus continuous distributions.

Comparison: Raw Data vs. No Outliers
Finally, by comparing mi_scores with mi_scores_no_outliers, we can observe if the perceived importance of a feature is driven by anomalous data points or if it represents a robust statistical trend. This dual-check ensures that our feature selection is not biased by extreme values in “Annual Income”.

# Label encoding for Gender
df_features['Gender'], _ = df_features['Gender'].factorize()
df_features_no_outliers['Gender'], _ = df_features_no_outliers['Gender'].factorize()
def make_mi_scores(df_input):
    # Initialize dictionary to store the total MI (centrality) for each feature
    result = {}
    for col in df_input:
        # Define 'col' as target (y) and all other columns as predictors (X)
        X = df_input.copy()
        y = X.pop(col)
        # Create a boolean mask for discrete predictors remaining in X
        is_discrete = (X.columns == 'Gender')
        # Use classification MI for categorical target (Gender), 
        # regression MI for continuous targets (Age, Income, Spending Score)
        if col=='Gender':
            mi_scores = mutual_info_classif(X, y, discrete_features=is_discrete, random_state=42)
        else:        
            mi_scores = mutual_info_regression(X, y, discrete_features=is_discrete, random_state=42)
        result[col]=sum(mi_scores)
    return result

mi_scores = make_mi_scores(df_features)
mi_scores_no_outliers = make_mi_scores(df_features_no_outliers)

# Convert dictionaries to DataFrames for easier manipulation and sorting
scores_df = pd.DataFrame.from_dict(mi_scores, orient='index', columns=['MI Score'])
scores_df_no_outliers = pd.DataFrame.from_dict(mi_scores_no_outliers, orient='index', columns=['MI Score'])
# Sort scores in descending order to highlight the most influential features
scores_df = scores_df.sort_values(by="MI Score", ascending=False)
scores_df_no_outliers = scores_df_no_outliers.sort_values(by="MI Score", ascending=False)
# Merge the two results into a single table
score_union = pd.concat([scores_df, scores_df_no_outliers], axis=1)
# Rename columns
score_union.columns = ['With Outliers', 'Without Outliers']
# Apply a color gradient
score_union.style.background_gradient(cmap='YlGnBu')

MI Analysis Results:

• The marginal difference in MI scores between the two datasets suggests that the underlying structure of the data is stable, and the identified outliers do not significantly distort the global relationship between features.
• Dominant Features: The Spending Score remains the most informative feature in both scenarios. While its score slightly decreased (from 0.794 to 0.785), it maintains the highest MI, confirming it will be the primary driver for our clusters.
• Income Stability: Annual Income shows remarkable stability, with its MI score virtually unchanged (from 0.627 to 0.625). This confirms that the relationship between income and the other features is robust and not dependent on anomalous data points.
• Age Dynamics: Interestingly, Age is the feature that benefited most from the cleaning process, with its MI score increasing from 0.513 to 0.560. However, this increment is still relatively minor and does not fundamentally alter the feature’s importance ranking.
• The Gender Decision: In both datasets, Gender shows an extremely low MI score (below 0.10). This statistically supports our earlier hypothesis from the pairplot: gender does not share significant information with the other features. Including it in the clustering process might add more noise than value.

Dataset Configurations for Model Evaluation

Based on the previous Mutual Information and Pairplot analyses, the most promising configuration appears to be the subset including Age, Annual Income, and Spending Score with Gender removed.

For educational and comparative purposes, we will not limit our study to a single scenario. Instead, we will evaluate our clustering algorithms across four different dataset configurations:

• 2D Scenario (Annual Income, Spending Score): tested both with and without outliers;
• 3D Scenario (Age, Annual Income, Spending Score): tested both with and without outliers.

This multi-scenario approach allows us to empirically observe how the inclusion of specific features or the presence of outliers affects the performance, shape, and stability of the resulting clusters.

Feature Scaling: StandardScaler

Before clustering, it is essential to normalize the data to ensure that all features contribute equally to the Euclidean distance calculation. This prevents variables with larger scales from disproportionately dominating the distance metric.

We have opted for the StandardScaler method [7,8]; given that only two outliers were identified, their impact is not significant enough to negatively bias the scaling process.
The scaling is applied to both versions of the dataset (with and without outliers) as we intend to analyze two distinct hypotheses: one retaining the original data distribution and the other refined by outlier removal, allowing for a comparative evaluation of the clustering results.

std_scaler = StandardScaler()
# 2D Scenario: Annual Income and Spending Score 
# Original dataset
X_IncomeSpending_all = std_scaler.fit_transform(df_features.loc[:, [ 'Annual Income (k$)', 'Spending Score (1-100)']].values)
# Dataset without outliers
X_IncomeSpending_no_outliers = std_scaler.fit_transform(df_features_no_outliers.loc[:, [ 'Annual Income (k$)', 'Spending Score (1-100)']].values)
# 3D Scenario: Age, Annual Income, and Spending Score 
# Original dataset
X_AgeIncomeSpending_all = std_scaler.fit_transform(df_features.loc[:, ['Age', 'Annual Income (k$)', 'Spending Score (1-100)']].values)
# Dataset without outliers
X_AgeIncomeSpending_no_outliers = std_scaler.fit_transform(df_features_no_outliers.loc[:, ['Age', 'Annual Income (k$)', 'Spending Score (1-100)']].values)

Conclusion & Next Steps

We have successfully laid the groundwork. By combining visual EDA with statistical Mutual Information analysis, we have moved from raw data to a refined set of four dataset configurations.

Rather than simply looking for the easiest solution, we are prepared to test multiple “experimental paths” — comparing how algorithms behave across different feature sets and evaluating the true impact of outliers on the final segments.

In Part 2, we move from analysis to action. We will identify optimal parameters for K-Means, GMM, and Linkage methods for our four dataset configurations.

Code Availability: The full GitHub repository will be shared in Part 3.

Read Part 2: Identifying optimal parameters for K-Means, GMM, and Linkage methods →

References

[1] Penn State University, Identifying Outliers (2023), STAT 200: Elementary Statistics. Online Resource

[2] GeeksforGeeks, Detect and Remove the Outliers using Python(2023). Technical Tutorial

[3] A. McDonald, Creating Boxplots with the Seaborn Python Library (2022), Towards Data Science. Article Link

[4] Wikipedia, Mutual Information (2024). General Reference

[5] R. Holbrook, Mutual Information (2021). Kaggle Course

[6] Scikit-Learn Documentation (2024), mutual_info_regression & mutual_info_classif

[7] S. Sharma, Scaling Numerical Data Explained: A Visual Guide (2020), Towards Data Science. Article Link

[8] Scikit-Learn. Comparing different clustering algorithms on toy datasets (2024). Documentation Example

The Outlier and Feature Selection Dilemma: Preparing Data for Clustering was originally published in Towards AI on Medium, where people are continuing the conversation by highlighting and responding to this story.