Customer Curn

In an era where market competition is fierce, keeping customers is a critical element to the success of a business. One of the main problems that companies face is the phenomenon known as “customer churn” or customer loss. The term customer churn refers to a situation when a customer stops using a product or service offered by a company.

Story Dataset:

This dataset consists of a total of 10,000 rows (entries) with row index range from 0 to 9999, and there are 7 columns.
The independent variables involve Gender, Age, CreditScore, EstimatedSalary, and HasCrCard, which contain information about the customer.
The dependent variable is Exited, which indicates whether the customer has left the service.

Features:

CustomerId: A unique ID for each customer.
Gender: The gender of the customer (Female/Male).
Age: The age of the customer.
CreditScore: Customer’s credit score.
EstimatedSalary: The estimated salary of the customer.
HasCrCard: Indicates whether the customer has a credit card (1 for yes, 0 for no).
Exited: A target variable that indicates whether the customer has exited the service (1 for yes, 0 for no).

Exploratory Data Analysis (EDA)

Exploratory Data Analysis (EDA) is the first step in the data analysis process that aims to understand the characteristics, patterns, and important information of the dataset. In this article, the EDA performed is
Analysis of Missing Values and Number of Unique Values in Each Column
In this dataset, there are no missing values, while the unique values are as follows:

Analysis of Duplicate Values

There is no duplicate data in this dataset

Overview of the Data Type of Each Column

Summary of Descriptive Statistics

Distribution Plot of Dependent Variable with Multiple Independent Variables

Distribusi variabel dependen ‘Exited’ terhadap variabel independen ‘Gender’:

• Female has a higher number in category ‘1’ (Exited), which corresponds to a low KDE value in that particular category.
• Male has a higher number in category ‘0’ (Not Exited), which corresponds to a high KDE value in that particular category.

Distribution of the dependent variable ‘Exited’ against the independent variable ‘Age’:

The majority of customers are in the age group of 30–40 and 40–50.
The 70–80 and 80–90 age groups have fewer customers.

Distribution of the dependent variable ‘Exited’ against the independent variable ‘CreditScore’:
The normal distribution shape shows that most customers have a CreditScore centered around the mean.
However, the spike above the x-axis at 800 indicates a small but significant group of customers with very high CreditScore.

Distribution of the dependent variable ‘Exited’ against the independent variable ‘EstimatedSalary’:

The uniform distribution of EstimatedSalary indicates that the estimated income of customers tends to be stable and does not vary significantly.
However, no particular pattern or trend is observed in relation to the churn rate, indicating that factors other than estimated income may play a greater role in a customer’s decision to stay or leave.

Distribution of the dependent variable ‘Exited’ against the independent variable ‘HasCrCard’:

There is a significant difference between credit card holders (HasCrCard=1) and non-credit card holders (HasCrCard=0) in terms of the number of Exited.
Credit card holders (HasCrCard=1) have a higher number of Exited compared to non-credit card holders.

Distribution Plot of Numerical Variables

“CustomerId” Variable Distribution:

The variable “CustomerId” still appears in the numeric variable plot because the data type is still numeric, in the next step it will be converted to object data type.

“Age” Variable Distribution:

The distribution of the variable “Age” shows a peak on the left and a skewed right. The age range of ‘30–50’ has a higher number of “Exited” frequencies compared to ‘20–30’, but decreases for ‘50++’.

“CreditScore” Variable Distribution:

The distribution of the variable “CreditScore” has a shape similar to a normal distribution, but there is a sudden increase jumping above the value of 800.

“EstimatedSalary” Variable Distribution:

The distribution of the “EstimatedSalary” variable tends to be uniform, with a relatively stable number of frequencies in each bin edge.

“HasCrCard” Variable Distribution:

The distribution of the variable “HasCrCard” shows that credit card holders (HasCrCard=1) have a higher number of “Exited” frequencies than non-credit card holders (HasCrCard=0).

Distribution of “Exited” Variable:

The distribution of the variable “Exited” shows that the majority of the data has a high frequency of label 0 (Not Exited), while the frequency of label 1 (Exited) only appears at the end (value 1.00).

Distribution Plot of Numerical Variables (Specific Customer Churn)

Variable “CustomerId”:
The variable “CustomerId” still appears in the numeric variable plot because the data type is still numeric, in the next step it will be converted to object data type.

Variable “Age”:
The distribution of the variable “Age” shows a normal plot shape with a slight right-skewed.
The majority of the data is within the age range of ‘36–50’.

Variable “CreditScore”:
The distribution of the variable “CreditScore” has a normal plot shape, but is slightly left-skewed.

“EstimatedSalary” variable:
The distribution of the variable “EstimatedSalary” shows a uniform plot shape, with relatively stable frequencies at each bin edge.

“HasCrCard” variable:
The distribution of the variable “HasCrCard” shows that the majority of the data is in the credit card holder category (HasCrCard=1).
Credit card holders (HasCrCard=1) have a higher number of “Exited” frequencies than non-credit card holders (HasCrCard=0).

Get Ni Putu Juliyant Ananda Rika Pangastuti’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

“Exited” variable:
In the distribution plot of the variable “Exited”, no frequencies relating to customers who did not exit the service (“Exited”=0) are shown, and the focus is on customers who have exited (“Exited”=1).

Data Pre-Processing

Preparing the Data
In preparing the data, we change the data type of the CustomerId column to object and display the distribution of the dependent variable against several independent variables, namely gender, age, credit score, estimated salary and hasCrCard.

Handling Outliers
Detecting outliers using the Local Outlier Factor (LOF) method
The Local Outlier Factor (LOF) method is an algorithm used for outlier detection in data. Outliers, or anomalies, are data points that are significantly different from the majority of data in a data set. LOF focuses on density-based outlier detection, which means it tries to find data that lies far away from its nearest neighbors in the feature space.
EstimatedSalary:
After using the LOF method the number of rows of EstimatedSalary decreases. This indicates that there are some values that are considered as outliers in the EstimatedSalary variable.
Consistency in Other Variables:
Other variables such as Age, CreditScore, HasCrCard, Gender_Male, and Exited did not experience significant changes in the number of unique values after handling outliers. This shows that LOF is more focused on identifying outliers in a particular variable (in this case, EstimatedSalary).

Handling Imbalance data

The plot shows that after using the SMOTE technique, the data distribution becomes more balanced. Before using the SMOTE technique, the amount of data for value 1 is less than the amount of data for value 0. After using the SMOTE technique, the amount of data for both values becomes more even.
Value 0
Before using the SMOTE technique, the amount of data for value 0 was 7,963. After using the SMOTE technique, the amount of data for value 0 is about 7,963.
Value 1
Before using the SMOTE technique, the amount of data for value 1 is about 2,037. After using the SMOTE technique, the amount of data for value 1 is 7,963.

SMOTE successfully balances the number of samples between the Exited (1) and Not Exited (0) classes, reducing class imbalance that can affect the performance of the classification model.

Correlation Matrix Between Numerical Variables

The results of the correlation matrix above can be concluded that the most correlated independent variable is Age with Exited, while the correlation value is 0.36.
Correlation between Age and Exited:
A positive correlation indicates that there is a moderate relationship between the age of the customer and the propensity to exit the service.
This could imply that the older a person is, the more likely they are to stay in the service.

Correlation between Gender and Exited:
The negative correlation indicates that there is a moderately negative relationship between gender (male) and the propensity to exit the service.
This could imply that female customers may tend to be more loyal to the service compared to male customers.

Correlation between Credit Card Ownership (HasCrCard) and Exited:
The negative correlation indicates that credit card ownership has a moderately negative influence on the propensity to exit the service.
This means that customers who have a credit card tend to be more loyal to the service.

Correlation between CreditScore and Exited:
The negative correlation indicates that there is a less strong relationship between credit score and exit propensity.
This may indicate that customers with higher credit scores have a slightly lower propensity to exit the service.

Correlation between Estimated Salary and Exited:

A very weak positive correlation indicates that there is no significant correlation between estimated income and the propensity to exit the service.
In other words, estimated income is not a major factor influencing customers’ decision to exit the service.

Correlation between Credit Card Ownership (HasCrCard) and Gender:
A positive correlation indicates that there is a less strong positive relationship between credit card ownership and male gender.
This means that men may be slightly more likely to own a credit card.

Model Training

• Splitting & Scaling Data

Data Splitting:
Data splitting is the process of dividing a dataset into two or more different parts. Usually, the dataset is divided into two main parts: training data and testing data.
Training Data: The training data is used to train the machine learning model. This means that the model will use the training data to adjust parameters and learn patterns from the data.
Testing Data: Testing data is used to test the performance of the model on data that it has never seen before. Using the testing data, we can evaluate how well the model can generalize on new data not seen during the training process. This data sharing is important to objectively measure the model’s performance and ensure that the model not only “memorizes” the training data but is also able to make good predictions on new data.

Data Scaling

Data scaling is the process of adjusting the range or scale of features in a dataset so that the features have a similar range or scale. This is important because most machine learning algorithms assume that all features in the dataset have similar scales. When the scales of features are not balanced, some features may dominate others in the learning and prediction process, which may lead to unreliable results. For example, distance-based algorithms, such as K-Nearest Neighbors, are very sensitive to differences in feature scales.

There are several methods for scaling data, including:
Standard Scaling (Z-score normalization): Scales features so that they have a mean of zero and a standard deviation of one.
Min-Max Scaling: Scales features into a specific range of values, for example 0 to 1.

Scaling the data helps ensure that all features have a balanced influence on the model and the machine learning algorithm can better fit the parameters. This project uses the standard scaler. The syntax for splitting and scaling data is as follows:

def model_prepare(df_model):

y = df_model[dependent_variable_name]

X = df_model.loc[:, df_model.columns != dependent_variable_name]

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.2, random_state = 42)

sc = StandardScaler()

X_train = sc.fit_transform(X_train)

X_test = sc.transform (X_test)

return X_train, X_test, y_train, y_test

Both, data splitting and scaling, are essential in data pre-processing to prepare the dataset before it is used to train machine learning models.

• Model Training

1. Logistic Regression
   Logistic Regression is one of the machine learning algorithms used for classification problems. Although the name contains the term “regression,” it is actually a classification algorithm used to predict the probability that an instance will have a certain label.
2. KNN
   K-Nearest Neighbors (KNN) is a simple machine learning algorithm used for classification and regression problems. In classification, it finds the k-nearest neighbors of an instance based on Euclidean distance or other metrics to determine its label.
3. Decision Tree
   Decision Tree is a machine learning algorithm that uses a tree structure to make rule-based decisions. Each node in the tree represents a decision based on data features, and the branches represent the results of that decision.
4. Random Forest
   Random Forest is an ensemble algorithm that consists of many decision trees. It works by combining the outputs of multiple decision trees to improve prediction performance and reduce overfitting.
5. Support Vector Classifier
   Support Vector Classifier (SVC) is a classification algorithm that uses a hyperplane to separate classes of data. The goal is to find the best hyperplane that separates the data with the maximum margin between the classes.
6. AdaBoost Classifier
   AdaBoost (Adaptive Boosting) is an ensemble algorithm that focuses on improving model performance by giving more weight to instances that are difficult to predict. It works by combining the output of multiple weak learners to create a stronger model.
7. Gradient Boosting Classifier
   Gradient Boosting is an ensemble algorithm that also focuses on improving model performance by combining the outputs of multiple weak learners. However, it works by iterating to gradually reduce the prediction error of the previous model.
8. XGBoost Classifier
   XGBoost (Extreme Gradient Boosting) is a highly efficient and fast implementation of the gradient boosting algorithm. It uses advanced optimization and regularization techniques to improve performance and reduce overfitting.
9. LightGBM Classifier
   LightGBM is a gradient boosting algorithm developed by Microsoft. It is designed to have very fast and efficient performance, and is able to handle large datasets with categorical features well.

The following is a table of the results of each machine learning model.

From the following table based on train accuracy, validation accuracy, precision, recall and F1-score, the best model is LightGBM.

Model Deployement

The following is the deployment model performed