Predicting Customer Churn with Machine Learning

Using clustering and predictive modeling to identify at-risk customers and reduce churn.

Introduction

Customer churn, or the rate at which customers stop using a service, is a critical challenge for subscription-based businesses. Identifying and retaining valuable customers is key to ensuring sustainable growth and profitability.

In this project, I analyzed customer data to uncover patterns in churn, performed clustering to segment customers based on their account profiles, and developed predictive models to estimate churn risk. By understanding the drivers of churn, this analysis provides actionable insights to help businesses proactively retain at-risk customers and improve overall customer loyalty.

Source: CXTODAY

Skills Used

• Python: For data cleaning, manipulation, and model implementation.
• Exploratory Data Analysis (EDA): Conducted initial analysis, handled missing values, and identified key patterns in the data.
• Clustering: Used K-Means to segment customers with similar characteristics.
• Dimensionality Reduction: Employed Principal Component Analysis (PCA) to reduce the number of features.
• Predictive Modeling: Built and compared multiple models (Logistic Regression and Random Forrest) to predict churn risk and identify the most influential factors.
• Data Normalization and Transformation: Prepared data for modeling by scaling and encoding features.
• Visualization: Created visual insights to illustrate and compare patterns.

Quick 0–1 Project Overview

Step 1: Exploratory Data Analysis (EDA): I began with an exploration of the dataset, calculating summary statistics and handling any missing values or outliers. Categorical features were converted to binary or dummy variables, and numeric features were normalized to prepare for modeling.

Step 2: Customer Segmentation with Clustering: Using clustering techniques, I segmented customers into groups with similar attributes and churn behavior. The choice of clusters was determined based on metrics like WCSS (Within-Cluster Sum of Squares) and the elbow method.

Step 3: Predictive Modeling: I developed two supervised learning models (Logistic Regression and Random Forrest) to predict the likelihood of customer churn. Each model was evaluated for accuracy, and I identified the top five predictive features most associated with churn, helping highlight which factors have the most impact on customer retention.

Step 4: Insights and Recommendations: Based on the model results, I identified the customer segment at the highest risk of churning and recommended actions to address the specific needs of this segment.

Step 1: Exploratory Data Analysis

I started by examining the dataset to understand its structure and ensure data quality, correlation analysis, calculating key summary statistics and imputing any missing values or outliers.

Sample of a part of the dataset

Here are some initial insights:

• The dataset summary showed a customer base with an average age of ~47 years and a tenure of 34 months (~3 years), with a large range in tenure (standard deviation of 24 months).
• The majority of clients have no family members, and referrals are few (avg. of 2 per customer).
• The average monthly fee is around 65 dollars, although charges can range from 10 dollars to 118.75 dollars, with occasional negative or zero figures suggesting billing issues or refunds.
• Revenue creation varies greatly, with some consumers contributing as little as 21 dollars while others giving more over 11,000 dollars.
• Standard deviations across key indicators show a very diverse client base in terms of overall engagement and financial effect.

Imputing Missing Values

There are some logical assumptions made here.

• The offer and internet_type columns include "None" as an option, indicating that missing records suggest these clients did not subscribe to any offer or internet service.
• Missing values in avg_monthly_gb_download are set to 0, indicating that customers without internet service do not download any data.
• The service-related columns, such as online_security and streaming_tv, are left blank to imply that the absence of data indicates these clients do not subscribe to these additional services.

# Imputing missing values for important columns

raw['offer'].fillna('None', inplace=True)
raw['internet_type'].fillna('None', inplace=True)
raw['avg_monthly_gb_download'].fillna(0, inplace=True)

# Impute missing service-related columns with 'No' since missing values indicate no subscription to these services (assumption)
missing_service_columns = ['online_security', 'online_backup', 'device_protection_plan',
                           'premium_tech_support', 'streaming_tv', 'streaming_movies',
                           'streaming_music', 'unlimited_hi-res_streaming']
raw[missing_service_columns] = raw[missing_service_columns].fillna('No')

I think this imputation technique preserves as much information as possible without removing potentially essential rows, ensuring that the dataset is good for further analysis and churn modeling.

Correlation Analysis

Next, I plotted a correlation heatmap for the numerical columns in the dataset to identify redundant or highly correlated features.

Output: Correaltion Analysis for numeric columns

Correlation Analysis Insights:

• total_revenue and total_charges: These two columns are highly correlated (close to 1), suggesting that they contain nearly identical information. Since total_charges is a more intuitive and direct measure, I will be dropping total_revenue.
• There is a strong correlation between tenure_in_months and total_charges, indicating that higher-paying or longer-tenured customers contribute more to overall revenue.
• Weak correlations, such as those between age and other variables, or between total_refunds and total_extra_data_charges, suggest there may be unexpected drivers of churn that warrant further analysis.
• tenure_in_months, monthly_charge, and avg_monthly_gb_download: Although these features show weak to moderate correlations with some variables, they are crucial for predicting churn, so I will be retaining them.

Check for Outliers

In this phase, I’ll focus on managing outliers in numerical columns where extreme values are most likely to affect the results.

These include financial and usage-related columns such as monthly_charge, total_charges, total_refunds, total_extra_data_charges, total_long_distance_charges, and avg_monthly_gb_download, where large or unusual values may indicate data entry errors or customers with significantly different behaviors.

In addition, I will verify tenure_in_months to ensure there are no unexpected extreme values in the customer tenure. Columns such as age, number_of_family_dependents, and number_of_referrals, on the other hand, are less likely to include harmful outliers since they naturally fall within tighter ranges, and extreme values (like higher ages or larger families) are feasible.

# Looking at specific columns for outliers
columns_to_check = ['monthly_charge', 'total_charges', 'avg_monthly_gb_download',
                    'tenure_in_months', 'total_refunds', 'total_extra_data_charges',
                    'total_long_distance_charges']

plt.figure(figsize=(12, 12))
for i, col in enumerate(columns_to_check, 1):
    plt.subplot(len(columns_to_check)//2 + 1, 2, i)
    sns.boxplot(x=raw[col])
    plt.title(f'Boxplot of {col}')
plt.tight_layout()
plt.show()

Output: Outlier

Outlier Analysis:

• Outliers Detected: The columns avg_monthly_gb_download, total_refunds, total_extra_data_charges, and total_long_distance_charges have clear outliers.
• No Outliers: The columns monthly_charge, total_charges, and tenure_in_months are well-distributed and do not require further adjustment for outliers.

Next, I decided to cap outliers at the 95th percentile to specific cases like retain high amount users as they could potentially be useful for the segmentation process. I also, re-plot the box plots are verified the spread of data after the outliers were removed.

# Capping outliers at the 95th percentile (I want to retain high amount users, maybe for segmentation)
for col in ['total_extra_data_charges', 'avg_monthly_gb_download', 'total_refunds', 'total_long_distance_charges']:
    # Calc. the 95th percentile
    cap_value = raw[col].quantile(0.95)
    raw[col] = raw[col].apply(lambda x: cap_value if x > cap_value else x)

raw[columns_to_check].describe()

Using the 95th percentile to cap outliers seems like a practical approach that balances retaining useful data (such as heavy users) while ensuring that extreme values don’t distort the analysis or model.

Binary and Dummy Variables

To prepare the data for modeling, I transformed categorical variables into binary and dummy variables, allowing the model to interpret them effectively.

1. Binary Variables: Several binary features, such as online_security, online_backup, and married, were converted into 0s and 1s, where "Yes" was mapped to 1 and "No" to 0. Additionally, gender was encoded as 0 for Male and 1 for Female to standardize the format.

binary_columns = ['online_security', 'online_backup', 'device_protection_plan',
                  'premium_tech_support', 'streaming_tv', 'streaming_movies', 
                  'paperless_billing', 'streaming_music', 'unlimited_hi-res_streaming', 
                  'internet_service', 'married']
raw[binary_columns] = raw[binary_columns].replace({'Yes': 1, 'No': 0})
raw['gender'] = raw['gender'].replace({'Male': 0, 'Female': 1})

2. Dummy Variables: For features with multiple categories, like internet_type and payment_method, I created dummy variables (one-hot encoding) and dropped the first category in each case to avoid multicollinearity. This conversion allowed the model to handle categorical variables without introducing redundancy.

categorical_columns = ['internet_type', 'offer', 'subscription_type', 'payment_method']
raw = pd.get_dummies(raw, columns=categorical_columns, drop_first=True)

Normalizing Data

To ensure the numeric features were on a consistent scale, I applied standard scaling (mean 0, standard deviation 1) to continuous variables, including age, tenure_in_months, and monthly_charge. This step was crucial for enhancing model performance, especially for algorithms sensitive to feature scaling.

from sklearn.preprocessing import StandardScaler

numeric_columns = ['age', 'tenure_in_months', 'avg_monthly_gb_download',
                   'monthly_charge', 'total_charges', 'total_refunds',
                   'total_extra_data_charges', 'total_long_distance_charges']
scaler = StandardScaler()
raw[numeric_columns] = scaler.fit_transform(raw[numeric_columns])

EDA Conclusion

The EDA findings show key patterns such as missing values, outliers in usage data, and correlations between features like tenure and churn. These insights guided the decision to impute or remove missing values, cap outliers, and scale numeric features for consistency. And, the strong non-linear relationships identified, such as those involving internet service and subscription type, influenced me to use both Logistic Regression for interpretability and Random Forest for capturing complex interactions in the data.

Step 2: Customer Segmentation with Clustering

Dimensionality Reduction (PCA)

To reduce dimensionality and simplify the feature space, I applied Principal Component Analysis (PCA), which is a technique that transforms the original features into a smaller set of uncorrelated components while retaining most of the data’s variability.

Get Sayli Bhavsar’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

First, I removed the viewer_id column, which serves only as an identifier, and isolated the target variable viewer_status to ensure it does not influence the PCA transformation. Then, I initialized PCA on the scaled feature set to capture the maximum variance in the data through fewer components.

df = df.drop(columns=['viewer_id'])

target_variable = df['viewer_status']  # Store the target variable
df = df.drop(columns=['viewer_status'])  # Drop the target variable from the feature set

# Initialize PCA and fit it on the scaled features
pca = PCA()
pca.fit(df)

# Check the explained variance ratio for each component
explained_variance_ratio = pca.explained_variance_ratio_
explained_variance_ratio

The explained variance ratio for each component was analyzed to understand how much information each principal component contributed to the dataset. By examining these ratios, I determined the optimal number of components to retain, ensuring we reduce feature complexity without losing significant information.

# Plot the cumulative explained variance to see how much variance is captured by each component
plt.figure()
plt.plot(pca.explained_variance_ratio_.cumsum(), marker='o', linestyle='--')
plt.xlabel('Number of Components')
plt.ylabel('Cumulative Explained Variance Ratio')
plt.title('Explained Variance vs. Number of Components')
plt.show()

Ouptut: Principal Component Analysis — Explained Variance vs Components

PCA Analysis:

• 12 components seem to capture around 80% of the variance, meaning they seem to explain most of the information.
• Beyond this point, adding more components doesn’t add much value, as the curve starts to flatten.

Clustering

To segment the customers based on their profiles, I used K-means clustering on the PCA-transformed data. First, I determined the optimal number of clusters by applying the Elbow Method, which evaluates the Within-Cluster Sum of Squares (WCSS) for up to 20 clusters.

# Checking for up to 20 clusters using the Elbow Method
wcss = []  # Within-Cluster Sum of Squares

for i in range(1, 20):
    kmeans_pca = KMeans(n_clusters=i, init='k-means++', random_state=42)
    kmeans_pca.fit(scores_pca)
    wcss.append(kmeans_pca.inertia_)

# Plotting WCSS to decide the optimal number of clusters
plt.figure(figsize=(12, 6))
plt.plot(range(1, 20), wcss, marker='o')
plt.xlabel('Number of Clusters')
plt.ylabel('WCSS (Within-Cluster Sum of Squares)')
plt.title('Elbow Method to Determine Optimal Clusters')
plt.show()

Output: Elbow Method (Suggestion for Optimal Clusters)

I experimented with both 4 and 5 clusters and checked the cluster distrubiton for each. The elbow in the plot suggested that 5 clusters was an appropriate choice, capturing essential distinctions without excessive complexity. After that, I decided to go ahead with K-means clustering with 5 clusters.

# Applying K-means clustering with 5 clusters
optimal_clusters = 5
kmeans_pca = KMeans(n_clusters=optimal_clusters, init='k-means++', random_state=42)
kmeans_pca.fit(scores_pca)

# Adding cluster labels to the DataFrame
df['clusters'] = kmeans_pca.labels_

# Checking the cluster distribution
print(df['clusters'].value_counts())

Output: Cluster Distribution

To gain insights, I analyzed how churn distribution varied across clusters:

Output: Viewer Status — Stayed or Churned

Key Clustering Insights:

• Clusters 0, 1, and 3: These clusters had a higher concentration of retained customers, likely indicating more satisfied or loyal groups.
• Cluster 2: This group showed a larger proportion of churned customers, highlighting the need to investigate specific characteristics like monthly charges or tenure.
• Cluster 4: Although smaller, this cluster had a mix of churned and retained customers, representing a nuanced group that requires further analysis.

Step 3: Predictive Modeling:

Given that churn prediction is a classification problem, I selected Logistic Regression and Random Forest as the two supervised learning models to predict customer churn.

Before modeling, I removed collinear features to avoid redundancy.

# Defining a function to remove collinear features
def remove_collinear_features(x, threshold):
    corr_matrix = x.corr()
    drop_cols = []

    for i in range(len(corr_matrix.columns) - 1):
        for j in range(i + 1):
            item = corr_matrix.iloc[j:(j+1), (i+1):(i+2)]
            val = abs(item.values)
            if val >= threshold:
                drop_cols.append(item.columns[0])

    x = x.drop(columns=set(drop_cols))
    print('Removed Columns:', set(drop_cols))
    return x

# Applying function to remove features with correlation > 0.6
use_features = remove_collinear_features(df[features_to_use], 0.6)
use_features = use_features.drop(columns=['clusters'])  # Dropping 'clusters' from features

Removed Columns: total_long_distance_charges, internet_type_Fiber Optic, number_of_referrals, internet_type_None, streaming_music, total_charges, unlimited_hi-res_streaming, monthly_charge

Model 1 — Logistic Regression: Logistic Regression was used as a baseline model to assess direct relationships between features and churn.

# Creating a Logistic Regression model
log_reg_model = LogisticRegression(max_iter=1000)

# Training and cross-validating the model
accuracy_scores = cross_val_score(log_reg_model, X_train, y_train, cv=5, scoring='accuracy')
f1_scores = cross_val_score(log_reg_model, X_train, y_train, cv=5, scoring='f1')

# Evaluating on the test set
log_reg_model.fit(X_train, y_train)
y_pred = log_reg_model.predict(X_test)

test_accuracy = accuracy_score(y_test, y_pred)
test_f1 = f1_score(y_test, y_pred)
test_precision = precision_score(y_test, y_pred)
test_recall = recall_score(y_test, y_pred)
print(f'Test set - Accuracy: {test_accuracy:.2f}, F1-Score: {test_f1:.2f}, Precision: {test_precision:.2f}, Recall: {test_recall:.2f}')

# Displaying top 5 features
coefficients_log = pd.DataFrame({
    'Feature': X_train.columns,
    'Coefficient': log_reg_model.coef_[0]
}).sort_values(by='Coefficient', ascending=False)
print("Top 5 Predictive Factors in Logistic Regression:")
print(coefficients_log.head(5))

Output: Top 5 Predictive Factors for Churn (Logistic Regression)

Model 2 — Random Forest: Random Forest was then applied to capture more complex relationships among features.

# Creating and evaluating a Random Forest model
rf_model = RandomForestClassifier(n_estimators=100, random_state=42)

# Cross-validating the model
accuracy_scores = cross_val_score(rf_model, X_train, y_train, cv=5, scoring='accuracy')
f1_scores = cross_val_score(rf_model, X_train, y_train, cv=5, scoring='f1')

# Evaluating on the test set
rf_model.fit(X_train, y_train)
y_pred = rf_model.predict(X_test)

test_accuracy = accuracy_score(y_test, y_pred)
test_f1 = f1_score(y_test, y_pred)
test_precision = precision_score(y_test, y_pred)
test_recall = recall_score(y_test, y_pred)
print(f'Test set - Accuracy: {test_accuracy:.2f}, F1-Score: {test_f1:.2f}, Precision: {test_precision:.2f}, Recall: {test_recall:.2f}')

# Displaying top 5 features by importance
feature_importances = pd.DataFrame({
    'Feature': X_train.columns,
    'Importance': rf_model.feature_importances_
}).sort_values(by='Importance', ascending=False)
print("Top 5 Important Features in Random Forest:")
print(feature_importances.head(5))

Output: Top 5 Predictive Factors for Churn (Random Forest)

Comparison and Insights:

• Accuracy: Both models performed well, with Random Forest achieving 86% accuracy and Logistic Regression 85%.
• Precision: Random Forest showed higher precision (79%), indicating it made fewer false-positive predictions compared to Logistic Regression (75%).
• Recall: Logistic Regression had a slightly higher recall (69%), meaning it captured more actual churners, albeit with more false positives.

Top 5 features for both models:

• Logistic Regression: Highlights features that have a direct, linear relationship with churn, such as specific offers or payment methods.
• Random Forest: Focuses more on complex interactions and features like tenure, age, and usage, which are harder to capture with a linear model.
• Tenure and usage patterns may have more complex effects that I think Random Forest is better at capturing since they might influence churn differently depending on customer age or other factors. Both models provide pretty good insights, but Random Forest might give a more complete picture by capturing the nuanced interactions between features, while Logistic Regression is useful for understanding the direct impact of specific features.

Step 4: Insights and Recommendations:

Based on the model results, I identified the customer segment at the highest risk of churning and recommended actions to address the specific needs of this segment, with the goal of reducing churn and improving customer loyalty.

Final Insights:

Cluster 2 seems to have the highest proportion of churners. It showed a larger number of customers who churned compared to other clusters.

Potential Characteristics:

• Short Tenure: Customers in this cluster likely have shorter tenure indicating that they might be new customers who haven’t fully settled into the service.
• Lower Engagement: Their average monthly GB download could be lower, means that they aren’t using the service as much, which is often a key indicator of dissatisfaction.
• Age and Subscription Type: These customers may be in certain age groups or might have signed up for longer contracts making them more likely to churn once their subscription period ends.

Recommendation based on models:

• Enhance Internet Service Quality: Since internet service emerged as the top predictor in the Logistic Regression model, focus on service improvements, especially for lower-tier customers. Consider offering upgrades or enhanced features to reduce dissatisfaction and prevent churn. Reliable streaming services, particularly for customers using features like TV streaming, are crucial — any performance issues in these areas could drive customers to leave.
• Revamp Offers A and E: Offers A and E are closely associated with higher churn rates. To improve retention, consider redesigning these promotions or providing follow-up incentives to customers who initially signed up under these deals. This could help sustain engagement beyond the promotion period.
• Encourage Long-Term Contracts: Random Forest analysis shows that long-term contracts are effective in reducing churn. Encourage customers on month-to-month or short-term plans to switch to two-year subscriptions by offering targeted incentives. This strategy can help lock in customer loyalty and reduce churn rates.
• Engage Low-Usage Customers: Lower usage, as indicated by Average Monthly GB Download, is linked to a higher risk of churn. Tailor engagement campaigns to encourage low-usage customers to explore more features and services. By fostering higher engagement, you can enhance their connection to the platform and decrease the likelihood of churn.
• Prioritize Early Retention Efforts: With tenure identified as a significant churn predictor, direct early retention efforts towards new customers. Implement loyalty rewards, an effective onboarding process, and special offers within the first six months to strengthen their commitment and improve long-term retention.

These targeted strategies, based on model-driven insights, can help retain at-risk customers and enhance customer satisfaction across the platform.

I write about what I research and find interesting, hoping to spark curiosity in others! Follow me for more:

Sayli Bhavsar - Medium

Connect with me on LinkedIn!