Statistics — 102

Here is yet another story on Statistics and this time it is on Probability and several distributions which are helpful in the data analysis.

If you’d like to refresh your memory on Measures of Central Tendency and Variation, then give a read to my previous blog.

Photo by Naser Tamimi on Unsplash

Random variable

It is a numerical description of the outcome of a statistical experiment. It is of two types:

1. Discrete random variable — It assumes finite number of values. Ex: gender info, month of transaction, Production year of car, roll of the dice, etc. It can also be of multiple types — Nominal, Ordinal, etc. Ordinal is such that there is some order for the values, example, month of transaction, etc. Nominal is that there is no order for the values, example, gender info, etc.
2. Continuous random variable — It can assume infinite number of values. Ex: height, weight, salary, etc.

When we plot values of a random variable, we get a distribution plot for the random variable. Now, these distributions can be of different types, but I’ll discuss here following:

1. Bernoulli’s distribution
2. Binomial distribution
3. Gaussian or Normal distribution
4. Standard Normal distribution

Bernoulli’s distribution

A random variable is said to follow Bernoulli’s distribution, if there are only two possible outcomes, and a single trial.

Ex: In a fight between 2 players, we can have only 2 possible outcomes, win or lose. In this, the player 1 will have a probability of winning as ‘p’ and losing as ‘1-p’. Consider, a random variable, X, will take value 1 for ‘win’ and 0 for ‘loss’.

In this case, our probability mass function, will be:

Probability mass function for Bernoulli’s distribution

or, it can be written as follows:

Probability mass function for Bernoulli’s distribution

Average or expected value for X will be as follows:

Expected value for Bernoulli’s distribution

Variance of X will be as following:

Variance for Bernoulli’s distribution

Binomial distribution

An experiment where there are only two possible outcomes and the experiment is repeated ‘n’ number of times, where each trial is independent of each other, is called a Binomial distribution.

Probability mass function for Binomial distribution

Ex: A fight between 2 persons, where person 1 winning probability is ‘p’ and losing is ‘1-p’. The fight is repeated a number of times, say, ’n’ and the probabilities of person 1 does not change.

In this case, our expected value is:

Expected value for Binomial distribution

Variance will be

Variance for Binomial distribution

Gaussian or Normal distribution

Most of the random variables follows “Normal distribution”. A random variable is said to follow Normal distribution when,

1. Mean = Median = Mode
2. The curve of the distribution is bell-shaped and is symmetrical about the line x = mean
3. The area under the curve is equal to 1

The probability density function of a random variable, X, following a normal distribution is:

Probability Density Function(P.D.F.) for Normal distribution

where,

Mean = E(X) = μ, and

Get Peeush Agarwal’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

Variance = σ²

Normal distribution curve

There are empirical values from the curve (also shown in the above figure):

1. 68.2% of all values lies within 1 standard deviation of the mean,
2. 95.4% of all values lies within 2 standard deviations of the mean, and
3. 99.7% of all values lies within 3 standard deviations of the mean.

A standard normal distribution is the normal distribution with mean(μ) = 0 and standard deviation(σ) = 1.

We have yet so many other distributions, but in the sight of Data Science, we mostly consider “Normal distribution”.

Photo by Brett Jordan on Unsplash

Now, give a pat on your back as we’ve accomplished knowledge for major distributions. Next, we’ll go through some theorems that will help us in our journey to understand our data better and gain insights out of them.

Central Limit Theorem

Central Limit Theorem states that the sampling distribution of a sample mean is approximately normal if the sample size is large enough, even if the population distribution is not normal.

What is the sampling distribution of a sample mean?

If we have a population of size N, and, if we take a sample of size n (n < N), calculate the sample mean, note it and then replace the sample back in the population. If we repeat the process for a finite count and then plot the sample means, we’ll get the sampling distribution of sample mean.

Suppose, we consider height of all people in the state of Karnataka as our population and the mean height is, say, 5ft and 4 inches. Now, if we consider a sample of 1000 people and calculate the mean height, and it might come out to be 5ft and 7 inches. We take another sample of 1000 people and repeat the process for say 200 times, then if we plot the mean height of each sample, we’ll find the plot to follow a normal distribution with mean height to be 5ft and 4 inches.

The Central Limit Theorem states that the sampling distribution will follow these properties:

• The mean of the sampling distribution will be equal to the mean of the population

Sample mean = population mean

• The variance of the sampling distribution will be equal to the variance of the population distribution divided by the sample size.

Sample variance = Population variance / sample size

Chebyshev’s Inequality

This one can come handy when we know that the distribution is not Gaussian or Normal distribution. It states that at least ‘1–1/k²’ of data from a sample must fall within ‘k’ standard deviations from the mean (here ‘k’ is any positive real number greater than one). It may also be referred to as “Markov’s inequality” in the context of analysis.

Chebyshev’s inequality

Pearson correlation coefficient

It is a measure of linear correlation between two sets of data. It is the ratio between the covariance of two variables and the product of their standard deviations.

Covariance is the measure of joint variability of two random variables.

Covariance and Pearson correlation coefficient formulae

Covariance gives us the direction of linear relationship, and, Pearson correlation coefficient gives us the strength and direction of linear relationship.

Knowledge of correlation between 2 variables is helpful in the data science project as it can help us in feature selection/elimination process.

Pearson’s correlation coefficient assumes that the data is linearly related, otherwise, it doesn’t provide correct strength between 2 variables.

Spearman’s correlation coefficient

Spearman’s correlation coefficient is the Pearson’s correlation coefficient between the rank variables. It can be calculated as follows:

Spearman’s correlation coefficient

where,

• ρ is the Pearson’s correlation coefficient applied on rank variables
• cov(rgₓ, rgᵧ) is the covariance of the rank variables
• σ is the standard deviation for the rank variables.

Only if all ‘n’ ranks are distinct integers, it can be computed using following formula:

Spearman’s correlation coefficient when all ranks are unique

Where, ‘dᵢ = rg(Xᵢ) — rg(Yᵢ)’ is the difference between the ranks of each observation and ‘n’ is the number of observations.

That’s it for this blog, Part-2. I’ll come up with another blog in continuation to above related to statistics.

For your reference,

• Part — 1: Statistics — 101