TELCO CUSTOMER CHURN PREDICTION USING CLASSIFICATION MODELS
INTRODUCTION
Customer Churn is the rate at which commercial customers leaves a business or platform and move to another business. Understanding why these customers are leaving a business can help mitigate the move and retain the customers to continue using the products or services.
OBJECTIVE
To build a model that accurately predicts customer churn.
DATA STRUCTURE
The data for this project is in a csv format. The following describes the columns present in the data. Below are the columns description.
Gender — Whether the customer is a male or a female.
SeniorCitizen — Whether a customer is a senior citizen or not.
Partner — Whether the customer has a partner or not (Yes, No)
Dependents — Whether the customer has dependents or not (Yes, No)
Tenure — Number of months the customer has stayed with the company.
Phone Service — Whether the customer has a phone service or not (Yes, No)
MultipleLines — Whether the customer has multiple lines or not.
InternetService — Customer’s internet service provider (DSL, Fiber Optic, No)
OnlineSecurity — Whether the customer has online security or not (Yes, No, No Internet)
OnlineBackup — Whether the customer has online backup or not (Yes, No, No Internet)
DeviceProtection — Whether the customer has device protection or not (Yes, No, No internet service)
TechSupport — Whether the customer has tech support or not (Yes, No, No internet)
StreamingTV — Whether the customer has streaming TV or not (Yes, No, No internet service)
StreamingMovies — Whether the customer has streaming movies or not (Yes, No, No Internet service)
Contract — The contract term of the customer (Month-to-Month, One year, Two year)
PaperlessBilling — Whether the customer has paperless billing or not (Yes, No)
Payment Method — The customer’s payment method (Electronic check, mailed check, Bank transfer(automatic), Credit card(automatic))
MonthlyCharges — The amount charged to the customer monthly.
TotalCharges — The total amount charged to the customer.
Churn — Whether the customer churned or not (Yes or No)
HYPOTHESIS
Null Hypothesis: Customers churn based on Online Security, Tech Support and Total Charges.
Alternate Hypothesis: Customers do not churn based on Online Security, Tech Support and Total Charges
RESEARCH QUESTIONS
- What is the correlation of all variables in relation to churn?
- What is the Distribution of Tenure over the period, the maximum and minimum.
- What are the Average Total Charges for Senior Citizens and Non-Senior Citizens?
- What is the number of Monthly, Yearly and Two-Year contract customers who made payment using a particular method?
- What is the correlation between the Churn column and the rest of the features?
- Does the payment method contribute to why customers churn?
- How does the length of a customer’s contract affect their likelihood to churn?
- On an average what total amount did customers pay in order to enjoy a particular Internet Service and how did it affect customer Churn.
- What is the average Tenure of Senior Citizens and Non-Senior under a particular contract and how does it affect customer Churn.
DATA PREPARATION
The data consist of 20 columns, we have one target variable, 3 numerical columns and 16 categorical columns.
The senior citizen column was recorded as an integer and was changed to an object variable. The total charges column was recorded as an object variable and was change to a float variable. This created some missing variables on the total charges column to be handled in the feature engineering stage.
RESEARCH QUESTIONS ANSWERED
- What is the correlation of all variables in relation to churn?
There is a correlation between tenure and monthly charges and total charges and tenure.
2. What is the Distribution of Tenure over the period, the maximum and minimum.
The distribution of tenure for customers is mainly between 5 months and 70 months. We had customer who were new and had used the service for just one month. Then we had people who had used the service for as long 72 months, I believe these were the unconditional loyal customer.
3. What are the Average Total Charges for Senior Citizens and Non-Senior Citizens?
The average total charges for senior citizens were more than non-senior citizens. Senior citizens were giving the company more revenue than non-senior citizens. These might be the same people who have used the telco’s service for 72 months.
4. What is the number of Monthly, Yearly and Two-Year contract customers who made payment using a particular method?
Most of the customers were monthly subscribers. Majority of the customers using these payments methods: Bank Transfer, Electronic check and Mailed checks payments were on month-to-month contracts. 1850 out of 2365 customers who paid with electronic checks were on monthly contract and these customers were out of 3875 month-to-month customers.
5. What is the correlation between the Churn column and the rest of the features?
The Multiple Lines, Phone service, Gender, Streaming TV, Streaming Movies and the Internet Service were the features that had the least correlation with Churn. Tec Support, Online Security, Tenure and Contract were among the features that had a stronger negative correlation with Churn. Monthly Charges, Paperless Billing, Senior Citizen and Payment Method had a stronger positive correlation with Churn.
6. Does the payment method contribute to why customers churn?
Looking at the payment methods, customer who were churning most were customer that used electronic checks followed by customer that used mail. Bank transfers and credit cards payment had the lowest number of customers churning and also very fast in term off payment.
Get Anne Njeri’s stories in your inbox
Join Medium for free to get updates from this writer.
7. How does the length of a customer’s contract affect their likelihood to churn?
Most of the customers opted for month-to-month contract. The number of monthly customers that churned is even more than the number of people who signed up for the year contract. The month-month option gave customer the flexibility to try the service and then leave when they weren’t satisfied with the service the telco company was producing.
8. On an average what total amount did customers pay in order to enjoy a particular Internet Service and how did it affect customer Churn.
Under customers who churned graph:
Both male and female customers paid approximately 2500 for DSL, female customers paid a bit more than male customers for fiberoptic, they both paid slightly above 500 for not having internet service.
Under customers who did not churn:
Surprisingly under this graph, male customers paid a bit more than female customers for the fiberoptic service.
9. What is the average Tenure of Senior Citizens and Non-Senior under a particular contract and how does it affect customer Churn.
Non senior citizens under the two-year contract type who churned had a higher average tenure than senior citizen under either month-to-month or one year contract who churned. This was same for the customers who were senior citizens.
HYPOTHESIS TESTING
After using the chi-square test, we found p = 0. Hence, we reject the hypothesis that say Customers churn based on Online Security, Tech Support and Total Charges.
FEATURE ENGINEERING
1. Data Splitting
The data was split into Train and Test using an 8:2 ratio respectively. This is after dropping the Customer ID column which was not needed.
2. New Feature Creation
There were to new columns create:
· Monthly Variations which were created from using the monthly charges, tenure and the total charge.
· Tenure group that was created by creating bins from transforming the tenure into a range of 3 months.
3. Missing Variables
To compute the missing variables, the simple imputer was used with the `constant’ strategy for the numerical variables while the `most frequent’ was used for the categorical variables.
4. Encoding
The One Hot Encoder was used to compute the categorical features.
5. Scaling
The standard scaler was used to transform the numerical variables.
6. Data Balancing
The target variable `Churn’ had 73.5% No and 26.5% Yes. This shows a class imbalance. The SMOTE method was used to balance the train data set features and target variables.
MODEL CREATION AND EVALUATION
5 models were created.
- Decision Tree
The Decision Tree Model confusion matrix did not perform very well as the false positive is high as compared to true positives. The f1 score is also low.
2. Logistic Regression
The Logistic Regression Model confusion matrix performed very well with the false positive being very low. The f1 score was also good.
3. KNNeighbour
The KNN model confusion matrix false negative is too high as compared to the Logistic Regression model. The f1 score is better than the one for Decision Tree but not as good as the Logistic Regression model.
4. Support Vector Machine
The support vector machine model performed well on both the confusion matrix and the f1 score.
5. Gradient Boosting
The Gradient Boosting model performed very well in both the confusion matrix and f1 score as compared to the other models.
Below are the 5 models score based on the highest f1 score.
HYPERPARAMETER TUNING
The Logistic Regression was further tuned using the GridSearchCV method.
- Logistic Regression
There has been a slight improvement in the confusion matrix after tuning although the f1 score remained the same.
Kindly find the complete NoteBook in my GitHub Rep.
