Data Distribution and It’s Types

Data is the raw fact that which is collected from the different source to get meaningful information. With the daily activites huge data is storing in the database with in a milisecond. Collected data are stored and distrubted to get valuable insights.

Data distribution is the organizing, clustering or categorizing the data which will be meaningful to take decision, finding patterns and do predictions. There are two main type of data distribution.

1. Descrete Data Distribution

• > Descrete Data Distribution is a set of value and ranges that is countable, such as number of student, population of a country. For example, the number of student can be 20, 30 but cannot be 20.5, 30.5.

2. Continious Data Distribution

• > Continous data is a value between a range of possible value. For example the marks of the student can be 20.5 or 20.6 but the age of the student cannot be expressed as a 26.5.

Common types of Distribution in ML

A. Normal (Gaussian) Distrubution

• > Normal distribution is the most commonly used data distributions of a random variables. It is continous probability distribution that is symmetric about the mean. Data near the mean are more frequent and the outliers are fewer.

The area under the curve of it is always 1. The curve traced but the upper values of the Normal Distribution is in the shape of a Bell, it is also called as a “Bell Curve”. Normal Distribution can be used to predict future outcomes based on past trends.

Formula of Normal Distribution

B. Bernoulli and Binomial Distributions

• > Bernoulli Trials are experiments in probability is the certain number of success or failures, True or False in a given number of trials or events. There is only two possible outcomes for each trial. Such as, Head or Tail outcomes in coin flipping. Doing the same task over and over to get the specific outcome, but the chance of outcome is same each time you experiment no matter how many time you repeat.

Probability of r success in n Trials = nCr qn — r pr

• p is the Probability of Success,
• q = 1 — p is the Probability of Failure,
• n is the Number of Independent trials, and
• r is the number of times an event occurred.
• > Binomial Distribution is a probability distribution that describes the number of success in a fixed number of independent binomial trials.

For example, you toss a coin 10 times and wants to know the probability of getting exactly 4 heads. n = 10 number of trials and p = 0.5 (probability of success).

P(X = r) = nCr qn — r pr

where,

• P(X = r) is the probability of getting exactly r successes,
• n is the number of total trials,
• r is the number of successes in n Trials,
• p is the probability of success,
• q is the probability of failure, and
• p + q = 1 and r = 0, 1, 2, …, n

C. Uniform Distribution

• > A Uniform Distribution where every outcome in a given range is equally likely to occur. That mean there will be no bias. It is often used in Weight Initialization for neural network to ensure all nurons start with an equal change of being active.

D. Poisson and Exponential Distribution

• > Poisson Distribution gives the probability of a certain number of events happening within a fixed interval of time or space, if the events occur randomly and independently. For eg: The number of phone calls a call center receives in 1 hr, The number of typing erros per page.

P(X=k)=k!e−λλk​

where

• kkk = number of events,
• λλλ = average rate,
• eee = 2.718…
• > Exponential Distribution describes the time between consecutive events in a poisson process. For eg: Time between two customer arrivals at a shop, Time until the next phone call comes in.

Formula:

f(x;λ)= λe−λx, x≥0

E. Skewed Distributions

• > Skewed Distribution is a asymetrical statistical distributions where data clusterss at an end, creating a long “ tail” on the other side, unlike a symmetrical normal distribution. Two type of Skewed Distributions:

1. Positive skewed (Right-Skewed Distribution)— When the scores pile up on the lower end of the values with fewer scores at the high end. The side with fewer scores is called the tail and is considered the direction of the skew.

2. Negative Skewed (Left-Skewed Distribution) — This is when the scores pile up on the higher end of the values with fewer scores at the low end.

Normalization and Standardization

Data Normalization and Standardization is one of the pre-processing procedures in machine learning that rescale features to similar ranges.

Get shila’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

Normalization — It is the method use to arrange the data in a database. It is a scaling method that reduces duplication in which the numbers are scaled and moved between 0 and 1. The main work of normalization is to remove the undesirable characteristics from the dataset.

The formula is:

X_normalized = (X — X_min) / (X_max — X_min)

This preserves the shape of the original distribution and is useful when you need bounded values. It’s sensitive to outliers since it uses min and max values. It works well for algorithms.

We use normalization for:

• Equal Feature Contribution: Prevents features with wider ranges (income) from overpowering features with smaller ranges (age) in distance calculations, ensuring all features are equal.
• Faster Convergence: For models using gradient descent normalization creates a more spherical cost function, allowing the algorithm to take more direct steps to the minimum, rather than oscillating.
• Improved Model Performance: Makes distance algorithms (eg. K-means,KNN) more reliable.

Common Normalization Methods

• Min–Max Scaling
• Z-score Standardization
• MaxAbs Scaling

Standardization — It is a technique to standardize the independent features present in the data in a fixed range. It is also called as Z-score Normalization. It involves transforming feature values to have a mean of 0 and a standard deviation of 1.

The formula for standardization is given by:

Where:
-z is the standardized value.
- X is the original feature value.
- mu is the mean of the feature values.
- sigma is the standard deviation of the feature values.

Standardization is valuable when you want features to have a mean close to zero, which can help speed up the convergence of optimization algorithms, such as gradient descent.

When to Use Standardization:

1. To Give Equal Importance to All Features

Different features often have different scales — for example:

• Age: 18–80
• Salary: 20,000–200,000

If we feed these directly into a model, “salary” might dominate just because it has bigger numbers.
➡ Standardization removes that bias by bringing them to the same scale.

2. To Handle Normally Distributed Data

If your data roughly follows a normal (bell-shaped) distribution, standardization centers it around 0 — this aligns with many algorithms’ mathematical assumptions (e.g., regression assumes normally distributed errors)

Real-Life Example

Scenario: Predicting Student Performance

Features:

• Study hours per week: 0–50
• Attendance rate: 0–100
• Test scores: 0–200

If you feed this raw data to a regression or neural network model:

• Test scores will have much larger numbers → the model might “think” it’s the most important.

After standardization, all three features will have:

• Mean ≈ 0
• Standard deviation ≈ 1
  Now the model focuses on patterns, not numeric scale differences.

Difference between Standardization and Normalization:-