Descriptive Statistics using Python

The ART of capturing the PULSE of dataset and make data talk sense

Descriptive Statistics

Descriptive statistics is a branch of statistics that deals with summarizing, organizing, and presenting data in a meaningful and informative way.

Descriptive statistics is primarily broken down into two components: — — Measures of Central Tendency & Measures of Variability (or Dispersion).

1. Measures of Central Tendency

These statistics describe the center point or typical value of a distribution. They answer the question: “What is the most typical score?”

Mean : The arithmetic average; the sum of all values divided by the number of values. Best for normally distributed data; sensitive to outliers.

Median : The middle value when the data is ordered from least to greatest. Best for skewed data or data with extreme outliers.

Mode : The value that appears most frequently in a dataset. Best for categorical or nominal data.

2. Measures of Variability (or Dispersion)

These statistics describe the spread or how far apart the data points are from the center. They answer the question: “How varied or dispersed are the scores?”

Range : The difference between the highest and lowest values in a dataset. Range provides a quick, rough estimate of spread.

Variance : The average of the squared differences from the Mean. Variance is used as an intermediate step to calculate standard deviation.

Standard Deviation : The square root of the variance. It measures the average amount of variability in the dataset. This is most common measure of spread; expressed in the original data units.

Coefficient of Variation : Coefficient of Variation (CV) is equal to the standard deviation divided by the mean. CV provides a unit-less measure of risk or uncertainty relative to the expected value.

Interquartile Range (IQR) : The difference between the 75th percentile (Q3) and the 25th percentile (Q1). “Excellent for describing spread when data is skewed or contains outliers as it focuses on the middle 50% of the data.”

3. The Measures of Shape

The measure of shape are descriptive statistics that quantify how the data is distributed around its mean. They describe the qualitative appearance of a frequency distribution or a histogram.

Skewness : Skewness measures the asymmetry of a probability distribution. It tells you the direction and extent to which a dataset deviates from the symmetrical “bell-shaped” normal distribution.

a. Zero Skew / Symmetric : The left and right sides of the distribution are mirror images hence Mean, Median, and Mode are approximately equal, data is likely normally distributed.

b. Positive Skew (Right-Skewed) : The tail of the distribution is longer on the right side. The Mean is greater than the Median. Many small values, with a few extremely large outliers pull the mean to the right. (e.g., income data).”

c. Negative Skew (Left-Skewed) : The tail of the distribution is longer on the left side. The Mean is less than the Median. Many large values, with a few extremely small outliers pulling the mean to the left. (e.g., exam scores).”

Kurtosis

Kurtosis measures the “tailedness” of the distribution — specifically, it describes the extremity of the tails (the frequency of outliers) and the peakedness of the curve.

a. Mesokurtic : A distribution with tails and peakedness similar to a normal distribution. Standard variance is explained by the distribution.

b. Leptokurtic : A distribution with high peak and heavy, fat tails. More data falls into the extreme values (outliers) than a normal distribution. High probability of extreme outcomes or rare events (high risk).

c. Platykurtic : A distribution with flat peak and light, thin tails. Less data falls into the extreme values than a normal distribution. Low probability of extreme outcomes.

Descriptive Analysis using Python

We shall be computing descriptive analysis on The Auto MPG dataset — the dataset (from UCI / seaborn) contains fuel efficiency and engine characteristics for various car models manufactured between 1970 and 1982.

Dataset Dimension:

# Installing libraries
import numpy as np
import seaborn as sns
import sklearn
import pandas as pd
from matplotlib import pyplot as plt
from sklearn.linear_model import LinearRegression

# Load Data and Read data
df=pd.read_csv('mtcars.csv')
# Understand Data Shape
print(df.shape)
df.head()

#Evaluate Numerical and Categorical data in dataset
cats = list(df.select_dtypes(include=['object']).columns)
nums = list(df.select_dtypes(exclude=['object']).columns)
print(f'categorical variables:  {cats}')
print(f'numerical variables:  {nums}')

# cylinders and model_year also seems to be categorical so lets update the lists
# Add 'cylinders' and 'model_year' to the categorical list
cats.extend(['cylinders', 'model year'])

# Remove 'cylinders' and 'model_year' from the numerical list
try:
    nums.remove('cylinders')
    nums.remove('model year')
except ValueError as e:
    print(f"Warning: Could not remove a key from 'nums'. Error: {e}")

# Print the updated lists
print(f'Categorical variables: {cats}')
print(f'Numerical variables: {nums}')

# Handle missing data
# check for `nans` in each column
df.isna().sum()

# for now remove all nan rows as they are just 1.5%
df = df[~df.isnull().any(axis=1)]
df.reset_index(inplace=True)
df.drop('index', inplace=True, axis=1)
df.shape

# Handle missing data
# find total duplicate entries and drop them if any
print(f'total duplicate rows: {df.duplicated().sum()}')

# drop duplicate rows if any
df = df[~df.duplicated()]
df.shape

# before we move ahead it's a good practice to group all variables together having same type.
df = pd.concat((df[cats], df[nums]), axis=1)
df.head()

# Extract Descriptive Statistics for numerical variables
descriptive_stats = df.describe()

# Display the descriptive statistics
print(descriptive_stats)

# You can also derive standalone descriptive statistics for individual numerical varaible.
# Let us take variable mpg
df.mpg.describe()

Measure of Central Tendency

We will compute Measure of Central Tendency for MPG data

# --- Calculation of Central Tendency Measures for 'mpg' ---

# Mean (Average)
mean = df['mpg'].mean()
print("Mean of mpg variable is:", mean)

# Mode (Most Frequent Value - returns a Series as there can be multiple modes)
mode = df['mpg'].mode()
print("Mode of mpg variable is:", mode.tolist()) # Converting to list for clean display

# Median (Middle Value)
median = df['mpg'].median()
print("Median of mpg variable is:", median)

# --- Explanation of Results ---
# The mean is pulled by all values, the median is the true center,
# and the mode is the most common value.

Measure of Dispersion

We will compute Measure of Dispersion for MPG data

print("--- Measures of Dispersion for 'mpg' ---")

# Minimum Value
min_value = df['mpg'].min()
print("Minimum value of mpg variable is:", min_value)

# Maximum Value
max_value = df['mpg'].max()
print("Maximum value of mpg variable is:", max_value)

# Range
range_value = df['mpg'].max() - df['mpg'].min()
print("Range of mpg variable is:", range_value)

# Variance
variance = df['mpg'].var()
print("Variance of mpg variable is:", variance)

# Standard Deviation
std_dev = df['mpg'].std()
print("Standard Deviation of mpg variable is:", std_dev)

# --- Corrected Quartile Calculations using .quantile() ---
# 2nd Quartile - Median
Q2 = df['mpg'].quantile(0.5)
print("Median / Q2 of mpg variable is:", Q2)

# 1st Quartile
Q1 = df['mpg'].quantile(0.25)
print("First Quartile (Q1) of mpg variable is:", Q1)

# 3rd Quartile
Q3 = df['mpg'].quantile(0.75)
print("Third Quartile (Q3) of mpg variable is:", Q3)

# Interquartile Range (IQR)
IQR = Q3 - Q1
print("Inter-Quartile Range (IQR) of mpg variable is:", IQR)

Measure of Shape

We will compute measure of shape of distribution on MPG data

# --- Computation of measures of shape of distribution ---
# Skewness
skewness = df['mpg'].skew()
print("Skewness of mpg variable is:", skewness)

# Kurtosis
kurtosis = df['mpg'].kurtosis()
print("Kurtosis of mpg variable is:", kurtosis)

USE CASE : Healthcare: Patient Vitals & Clinical Trials

Get Keya Dobriyal’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

Medical researchers use Python to summarize the baseline characteristics of a study group.

The Goal: Ensure a trial group is representative of the general population.

Python in Action:

• Distribution (Skewness): Checking if the age of participants is “normally distributed” or skewed toward the elderly.
• Range & IQR: Identifying the “Normal Range” for blood pressure or heart rate within a specific demographic.
• Outlier Detection: Using Boxplots to spot extreme reactions to a new medication that might indicate a safety risk.

Dataset:

The dataset can be downloaded from Kaggle, https://www.kaggle.com/datasets/sulianova/cardiovascular-disease-dataset?resource=download&select=cardio_train.csv

import pandas as pd
import numpy as np
from statistics import mean

# load the data
data = pd.read_csv('/kaggle/input/cardiovascular-disease-dataset/cardio_train.csv', delimiter=';')
data = data.dropna()

# create feature vectors
bp_x = data.drop(labels=['ap_lo', 'ap_hi', 'cardio', 'bp', 'bmi'], axis=1)
bmi_x = data.drop(labels=['bmi', 'height', 'weight', 'bp'], axis=1)

# create label vectors
ap_lo = data['ap_lo']
ap_hi = data['ap_hi']
cardio = data['cardio']
bp = data['bp']
bmi = data['bmi']

# Descriptive Analysis of Blood Pressure
bp_summary = df[['ap_hi', 'ap_lo']].describe()

print("--- Blood Pressure Descriptive Statistics ---")
print(bp_summary)

# Finding the 'Mode' for Cholesterol
print(f"\nMost common cholesterol level: {df['cholesterol'].mode()[0]}")

# Descriptive Analysis of Blood Pressure (Table-1)
bp_x.describe()

# Descriptive Analysis of Body Mass Index (Table-2)
bmi_x.describe()

Descriptive Analysis of BP

Descriptive Analysis of BMI

# 1. Age distribution
plt.figure(figsize=(10, 6))
sns.histplot(df['age'] / 365, kde=True)  # Convert age to years
plt.title('Age Distribution')
plt.xlabel('Age (years)')
plt.show()

Age Distribution

# 2. BMI distribution
df['bmi'] = df['weight'] / ((df['height'] / 100) ** 2)
plt.figure(figsize=(10, 6))
sns.histplot(df['bmi'], kde=True)
plt.title('BMI Distribution')
plt.xlabel('BMI')
plt.show()

BMI Distribution

Let me continue descriptive analysis with autoMPG dataset, we analyse the distribution for each numerical attribute and make some insights from the plots.

In case of numerical variables an ideal (or at least loved) distribution is gaussian, we can plot data using histogram, violin plot and boxplot

# ===============================================================
# Plotting Setup and Iteration
# ===============================================================

COLUMNS_TO_PLOT = df.columns
N_FEATURES = len(COLUMNS_TO_PLOT)
N_COLS = 3 # Three plots per feature (Distribution, Violin/Swarm, Box)
N_ROWS = N_FEATURES

# Set up the figure size: 18 inches wide, and 3.5 inches tall per feature row
fig = plt.figure(figsize=(18, N_ROWS * 3.5))

i = 0
for col in COLUMNS_TO_PLOT:

    # --- PLOT 1: Distribution Plot (Histogram + KDE) ---
    i += 1
    ax1 = plt.subplot(N_ROWS, N_COLS, i)
    # Using histplot (modern replacement for distplot)
    sns.histplot(data=df, x=col, kde=True, ax=ax1, color='skyblue', alpha=0.7)
    ax1.set_xlabel(None)
    ax1.set_title(f'{col} Distribution (Histogram + KDE)')

    # --- PLOT 2: Swarm-Violin Plot ---
    i += 1
    ax2 = plt.subplot(N_ROWS, N_COLS, i)
    # Plotting vertically for cleaner visualization of distribution shape
    sns.violinplot(y=df[col], ax=ax2, color='lightcoral', inner='box')
    # Use stripplot to show individual data points
    sns.stripplot(y=df[col], ax=ax2, color='k', alpha=0.3, size=3)
    ax2.set_ylabel(None)
    ax2.set_title(f'{col} Swarm-Violin Plot')

    # --- PLOT 3: Box Plot (for Outliers) ---
    i += 1
    ax3 = plt.subplot(N_ROWS, N_COLS, i)
    # Using orient='h' for horizontal box plot
    sns.boxplot(x=df[col], orient='h', ax=ax3, linewidth=2.5, color='lightgreen')
    ax3.set_xlabel(None)
    ax3.set_title(f'{col} Box Plot (Outlier View)')

# Run tight_layout once outside the loop for best results
plt.tight_layout(pad=3.0)
plt.show()

Insights

• Acceleration is the only distribution which is gaussian. There are few values in acceleration which lie outside the whiskers(the bars extending outwards from the box), these are fliers/outliers.
• Distributions of mpg & weight seems to be right-skewed gaussian.
• Distributions of displacement & horsepower seems to be far from gaussian.

Since we find there are outliers, we can compute using Tukeys Formulae

# Since there are outliers let us compute them with tukeys formulae
def tukey_outliers(x):
    q1 = np.percentile(x,25)
    q3 = np.percentile(x,75)

    iqr = q3-q1

    min_range = q1 - iqr*1.5
    max_range = q3 + iqr*1.5

    outliers = x[(x<min_range) | (x>max_range)]
    return outliers

# Call function to get outliers
for col in nums:
    outliers = tukey_outliers(df_num[col])
    if len(outliers):
        print(f"* {col} has these tukey outliers,\n{outliers}\n")
    else:
        print(f"* {col} doesn't have any tukey outliers.\n")

# Outliers in acceleration
df.iloc[list(tukey_outliers(df.acceleration).index)]

# Outliers in Horsepower
df.iloc[list(tukey_outliers(df.horsepower).index)]

Insight

Acceleration and Horsepower are the only attributes with tukey outliers and we can also notice this from the above boxplots.

Insights

• Outliers in acceleration seems to be random and only from USA and Europe.
• Outliers in horsepower are old cars from USA, with 8 cylinders, weight above 4000, large displacement hence low mpg

Let usplot box plot and visualize the distribution

# Let us plot boxplot to see distribution
# see data is not scaled properly, we need to scale it for modelling but works fine for analysis.
# Create a figure and subplot for the boxplot
fig = plt.figure(1, (12, 4))
ax = plt.subplot(1, 1, 1)

# pd.melt() transforms the wide DataFrame (multiple columns) into a long format
# (one column for variable name, one for value), which is required for a single boxplot
# comparing all features on the y-axis. We only melt the numerical columns (`nums`).
melted_df = pd.melt(df[nums])

sns.boxplot(x="variable", y="value", data=melted_df)
plt.title("Distribution and Scale Comparison of Key Numerical Variables")
plt.xlabel("Variable")
plt.ylabel("Value")
plt.show()


# Pair Plot for Relationship and Multivariate Distribution ---

# Annotation: The pair plot creates a grid of plots:
#   - Diagonal (diag_kind='kde'): Shows the distribution (KDE) of each variable.
#   - Off-Diagonal: Shows scatter plots between every pair of variables.
#   - hue='origin': Colors the points based on the 'origin' to check if relationships differ by region.

sns.pairplot(
    data=df,
    vars=nums,
    diag_kind='kde',
    hue='origin',
    plot_kws=dict(s=50, edgecolor="k", linewidth=0.1, alpha=0.6), # Adjusted size (s) and alpha for clarity

)
plt.suptitle("Pairwise Relationships Colored by Car Origin", y=1.02)
plt.show()

Insights

• MPG is indirectly proportional to displacement, horsepower and weight (Mpg increases displacement, horsepower & weight decreases)
• MPG is directly proportional to acceleration (MPG increases with acceleration.
• As horsepower increases displacement & weight increases but acceleration decreases.
• As weight increases displacement increases but acceleration decreases.
• As acceleration increases displacement decreases.

We can conclude that all numerical attributes are related with each other.

Let us now evaluate how categorical attributes are related to numerical attributes

# --- Multivariate Scatter Plot Analysis ---

# Goal: Analyze relationship between Horsepower (x) and Acceleration (y),
# split by Origin, colored by MPG level, and sized by Cylinder count.
print("Generating Multivariate Relationship Plot...")

# sns.relplot is used to create scatter plots (kind='scatter') or line plots
sns.relplot(
    x='horsepower',
    y='acceleration',
    hue='mpg',        # Color of the points (Categorical: Low/Medium/High MPG)
    size='cylinders',       # Size of the points (Categorical: 4, 6, 8 cylinders)
    col='origin',           # Column split (FacetGrid: Separate plots for USA, Europe, Japan)
    data=df,
    kind='scatter',
    sizes=(30, 300),        # Adjust sizes range for better visual distinction
    alpha=0.7,
    palette='viridis'       # Color scheme
)

plt.subplots_adjust(top=0.9)
plt.suptitle("Horsepower vs. Acceleration, Stratified by MPG Level, Cylinders, and Origin", fontsize=14)
plt.show()

print("\nPlot successfully generated.")
print("The analysis reveals how multiple factors influence the Horsepower/Acceleration trade-off.")

Insights

• There is a negative relation b/w horsepower & acceleration for each region
• Vehicles with low mpg has low acceleration and high horsepower
• Vehicles with more cylinders has low acceleration and high horsepower.

Let us analyze relationship between numerical and categorical attributes

print('variation of numerical features with origin')
fig = plt.figure(1, (18, 8))
for idx,col in enumerate(nums): ax = plt.subplot(2, 3, idx+1) sns.boxenplot(x='origin', y=col, data=df) ax.set_xlabel(None) plt.tight_layout()

This brings to end of the descriptive analsysis using python, in my upcoming article, we will dive into Inferential Statistics, that is transforming data from a collection of numbers into a tool for decision-making.

“If Descriptive Statistics is the art of looking at a photograph to see what happened, Inferential Statistics is the superpower of looking at a single pixel and predicting the rest of the image.”

About the Author:

Keya Dobriyal is a Computer Science Engineering student at Amity University, Noida, specializing in Data Science. She excels at data wrangling, machine learning, and the art of big data storytelling. By leveraging Python and R, Keya focuses on converting complex datasets into actionable insights. Beyond the world of logic and code, she is an avid creative writer and artist.