Statistics in Python — Using Chi-Square for Feature Selection

Photo by Sam Moqadam on Unsplash

In my previous two articles, I talked about how to measure correlations between the various columns in your dataset and how to detect multicollinearity between them:

Statistics in Python — Understanding Variance, Covariance, and Correlation

Statistics in Python — Collinearity and Multicollinearity

However, these techniques are useful when the variables you are trying to compare with are continuous. How do you compare them if your variables are categorical? In this article, I will explain to you how you can test two categorical columns in your dataset to determine if they are dependent on each other (i.e. correlated). We will use a statistics test known as chi-square (commonly written as χ2).

Before we start our discussion on chi-square, here is a quick summary of the test methods that can be used for testing the various types of variables: