DataPrep for Exploratory Data Analysis

DataPrep is a Python library as the name suggests it is useful in data preparation using fewer lines of code.

“dataprep.connector” allows data collection from common data sources. It can also be used to load data in SQL queries.

“dataprep.eda” is used for analyzing missing values, duplicates, univariate analysis, and bivariate analysis between numerical columns.

“dataprep.clean” provides many functions to clean and standardize the data.

I am planning to dedicate each post for each component of DataPrep. In this post, we will be diving deeper into the EDA component.

Installing the library package

Library installation in Jupyter Notebook

Importing the library

Importing necessary libraries

Importing DataPrep library for exploratory data analysis using create_report function.

Reading employee data using Pandas

Reading employee_data csv using Pandas

create_report function usage

create_report function

Let’s explore the report that was generated.

Overview section

Overview

The statistics provided by these libraries include details such as the number of variables, rows, missing values (along with their percentage), duplicate rows (along with their percentage), memory size, average row size in memory, and variable types.

Variables

Drop-down option for variables

Variables can be accessed using the drop-down menu option

Different sorting options for variables

Variables can also be accessed based on feature order, alphabetical order, amount of missing rows, and approximate unique values.

Statistics of a numerical variable

The stats option of a numerical variable will show the minimum and maximum values, 1st and 3rd quartile values, median, mean, range, and IQR. It also shows information related to the dispersion of the data and the skewness.

KDE plot

KDE plot of the numerical column

KDE plot shows how the data is distributed.

Get Akshaya’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

Normal Q-Q plot

Q-Q plot

It also displays a Q-Q plot to compare the distribution of the dataset with the theoretical quantiles.

Boxplot

Boxplot

Boxplot also shows the distribution of the data.

The cursor pointed at one place in the boxplot

Anywhere the cursor is placed on the plots, data values corresponding to the place will be displayed.

Similarly, categorical variable analysis results are given below.

Stats

Stats of categorical column

Pie chart

Piechart

WordCloud

WordCloud

WordFrequency

Word Frequency

Word Length

Word Length

Interactions

The interactions section also consists of the x-axis and y-axis drop-down options which will be useful to view interactions between any 2 selective variables.

Correlations

The correlation section shows the correlation in a heatmap form and is also navigatable. We have methods like Pearson, Spearman, and KendallTau.

Missing values

Missing values in each column are displayed in the report using a bar chart, spectrum, heat map, and dendrogram.

If you have any further questions or need assistance, feel free to ask!