Credit Card Applications and Logistic Regression
Having a credit card in your wallet is important. But we know, sometimes it gets tricky to get one. As you might be aware, banks get a lot of credit card issuance application, and they most certainly use a method of weeding out potential bad creditors based on their income level, existing loan balances, credit payment history and other factors. It is as easy as entering a person’s information, and with a click of a button, they automatically know if they are approved or not.
However, in this article, I will show you the behind the scenes of predicting the approval or rejection of a credit card application based on a given set of personal information. We will use the high-level programming language python and the statistical model called logistic regression to predict the outcome of an application.
So why logistic regression?
Logistic regression is a classification algorithm that can predict the probability of a particular outcome given a set of determining factors. Say for example, a certain university can predict graduate student admission based solely on the GRE score. The outcome is either pass or fail. This is usually called binomial logistic regression wherein you can only have two outcomes: 1 (true) or 0 (false) à pass and fail. And this outcome is based on one or more features, in this case, the GRE score.
For the purpose of discussing how to use logistic regression in predicting a credit card application approval or rejection, we will use Chiharu Sano’s Japanese credit card screening data set (https://archive.ics.uci.edu/ml/datasets/Japanese+Credit+Screening)
Using python’s pandas library, we first load the dataset.
At first the letters and numbers make no sense, but after some internet sleuthing, we find the meaning of our dataset thanks to Ryan Kuhn. Our data features are: Gender, Age, Debt, Married, Bank Customer, Education Level, Ethnicity, Year Employed, Prior Default, Current Employment, Credit Score, Drivers License, Citizenship, Zip Code, Income and Approval Outcome.
We then check for zero values and found them in “Debt,” “Years Employed,” “Credit Score,” and “Income.” Because these four can really have zero values in real life, i.e., no debt history, never been employed, no credit score, and no income, we keep them as they are. (Others will replace those zeroes with the mean values)
The descriptive statistics for (2) Debt, (7) Years Employed, (10) Credit Score, and (14) Income are as follows:
Upon further inspection, there are some values with “?”
We then replace all ‘?’ with NaN and display all rows with any replacement.
We can either remove all these rows from our dataset or replace NaN with the most frequent value in the column (mode). However, in this case, we choose to replace NaN with the mode value because our machine learning model might be affected if too many rows are removed.
Now, when we display any NaN values, we get nothing.
Now we do preprocessing of data.
Get MO’s stories in your inbox
Join Medium for free to get updates from this writer.
We will now convert all the non-numeric values into numeric ones. This is because machine learning models (like those developed using scikit-learn) need the data to be in a numeric format. This technique is called label encoding.
We separate the features (X) from the outcome (y).
Now we will split our data into two: training and testing set.
But before we can apply the logistic regression model to our training set, we need to standardise our dataset.
We can now finally apply the logistic regression model to our dataset.
Basically, what we have done is split our data into two: the training set and the test set. We apply the logistic regression model to our training set and use our test set to evaluate our model.
We can check the accuracy, precision and recall of the logistic regression model as follows:
It looks like our model performed very well and has achieved 86% accuracy.
Finally, we can use a given set of features (applicant’s information) to predict if it is going to be approved or not.
And as shown below, the logistic regression model is correctly predicting the outcome of a credit card application.
