Normal Distributions & Long-tailed Distributions #DS09

Normal Distributions

One of the most fundamental distributions in statistics is the Normal Distribution, often called the bell-shaped distribution. Its characteristic shape signifies that most data points cluster around the mean, with fewer and fewer points appearing as we move further away from it.

The Normal Distribution has significantly influenced mathematical formulas, particularly in inferential statistics. For example, the Central Limit Theorem (CLT) states that the distribution of sample means approximates a normal distribution as the sample size increases.

In a normal distribution:

• 68% of the data lies within one standard deviation from the mean.
• 95% lies within two standard deviations.
• 99.7% lies within three standard deviations.

This is known as the Empirical Rule.

(PICTURE)

The term normal distribution does not imply that it is the “typical” distribution found in real-world data. Many real-world datasets deviate from normality and instead exhibit long-tailed distributions, which are skewed toward one side.

(SKEWED DISTRIBUTIONS)

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
import scipy.stats as stats

# 1. Normal Distribution & Empirical Rule
np.random.seed(42)
data = np.random.normal(loc=50, scale=10, size=1000)  # Mean=50, Std=10

plt.figure(figsize=(8, 5))
sns.histplot(data, bins=30, kde=True, color='blue', alpha=0.7)
plt.axvline(np.mean(data), color='red', linestyle='dashed', linewidth=2, label='Mean')
plt.axvline(np.mean(data) + np.std(data), color='green', linestyle='dashed', linewidth=2, label='1 Std Dev')
plt.axvline(np.mean(data) - np.std(data), color='green', linestyle='dashed', linewidth=2)
plt.axvline(np.mean(data) + 2*np.std(data), color='orange', linestyle='dashed', linewidth=2, label='2 Std Dev')
plt.axvline(np.mean(data) - 2*np.std(data), color='orange', linestyle='dashed', linewidth=2)
plt.legend()
plt.title("Normal Distribution & Empirical Rule")
plt.show()

Origins of Normal Distribution

The Normal Distribution is also called the Gaussian Distribution, named after German mathematician Carl Friedrich Gauss, who observed that errors in measurements tend to follow this distribution. Since errors represent the difference between the actual value and an estimate (such as the mean), standard deviations are used to measure these differences. This explains why we measure distances from the mean in terms of standard deviations when analyzing a normally distributed curve.

Standard Normal Distribution and QQ plots

In a standard normal distribution, units are expressed in terms of standard deviations away from the mean. To convert data to a standard normal distribution, we:

1. Subtract the mean.
2. Divide by the standard deviation.

This process is called Normalization or Standardization, and the transformed values are called z-scores. However, standardization does not make data normally distributed — it only ensures that the data is on the same scale, making comparisons easier.

(Z SCORE FORMULA)

# 2. Standard Normal Distribution & Z-Scores
z_scores = (data - np.mean(data)) / np.std(data)
plt.figure(figsize=(8, 5))
sns.histplot(z_scores, bins=30, kde=True, color='purple', alpha=0.7)
plt.title("Standard Normal Distribution (Z-Scores)")
plt.axvline(0, color='red', linestyle='dashed', linewidth=2, label='Mean (0)')
plt.legend()
plt.show()

A QQ plot (Quantile-Quantile plot) visually assesses how closely a sample follows a specific distribution. When testing for normality:

• The y-axis shows the ordered z-scores of the data.
• The x-axis represents the quantiles of a perfectly normal distribution.
• If the data points closely align with the diagonal, the sample approximates normality.
• If there are significant deviations, the sample deviates from a normal distribution.

# 3. QQ Plot for Normality Check
plt.figure(figsize=(6, 6))
stats.probplot(data, dist="norm", plot=plt)
plt.title("QQ Plot for Normal Distribution")
plt.show()

Long-Tailed Distributions

Most real-world data is not normally distributed. While assuming normality simplifies statistical analysis, real-world distributions often follow complex, asymmetric patterns.

Get Vishal Gimhan’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

Examples of Skewed Data:

• Income Distribution: More people earn lower wages, with fewer individuals earning extremely high incomes. This results in a right-skewed distribution with a long tail to the right.
• Discrete Data (Counts): If we analyze the number of times an event occurs (e.g., customer complaints per day), the data may follow a Binomial distribution instead of a normal distribution.

Real-world distributions, whether symmetric or asymmetric, often have long tails, meaning they contain extreme values on either or both ends. These extreme values, though infrequent, can significantly impact decision-making.

# 4. Long-Tailed Distribution Example (Right-Skewed)
skewed_data = np.random.exponential(scale=10, size=1000)  # Exponential (right-skewed)
plt.figure(figsize=(8, 5))
sns.histplot(skewed_data, bins=30, kde=True, color='brown', alpha=0.7)
plt.title("Long-Tailed (Right-Skewed) Distribution")
plt.show()

The Impact of Long-Tailed Distributions

• Black Swan Theory (Nassim Taleb): Rare, unpredictable events (“black swans”) can have profound consequences, often far beyond what a normal distribution would predict. This is particularly relevant in finance, where market crashes occur more frequently than a normal distribution would suggest.
• John Tukey’s Observation: Data may be “normal in the middle” but exhibit much longer tails, meaning that while most values are clustered around the mean, extreme values occur more often than expected under normality assumptions.

Using QQ Plots to Detect Long Tails

QQ-Plot reveals deviations from normality where Long-tails can be identified

(QQ PLOT with long tail data)

Points far below the diagonal for low values and far above the diagonal for high values indicate more extreme values than expected under normality.

This suggests the presence of heavy tails, which require special statistical techniques to model properly.

# 5. QQ Plot for Long-Tailed Data
plt.figure(figsize=(6, 6))
stats.probplot(skewed_data, dist="norm", plot=plt)
plt.title("QQ Plot for Long-Tailed Distribution")
plt.show()

Final Thoughts

Fitting statistical distributions to observed data is a nuanced process that requires both domain expertise and statistical knowledge. It is both an art and a science — blindly applying a normal distribution without considering the context may lead to misleading conclusions.

Because real-world data takes on various shapes and distribution types, it is essential to use domain-specific knowledge to determine the best-fit model.

For example,

If modeling the number of events per time period, such as website traffic or call center requests, the Poisson Distribution is often more appropriate than a normal distribution.

Understanding normal and long-tailed distributions is critical to making accurate inferences and selecting the appropriate statistical models for analysis. In future articles, we will explore more distribution types and modeling techniques to enhance our ability to analyze real-world data effectively.

Reference:

- Bruce, P., Bruce, A., & Gedeck, P. (2020). Practical Statistics for Data Scientists: 50+ Essential Concepts Using R and Python. O’Reilly Media.