Probability Distribution Function
I’m quite sure there was a time when you came across this term called ‘Probability’ and ended up making it a nightmare. I understand. But today it’ll be different because you’re gonna learn from someone who has never been an outstanding math student. So, it’ll be a little difficult for me to make it mathematically complicated for you.
What is the Probability Distribution Function?
Probability Distribution Function is a mathematical function that describes the probabilistic outcomes of a Random variable.
To put it in simpler terms, let’s assume we have performed an experiment with a coin. Now, the sample space of the experiments contains — {H, T}. Here, the chances of getting an H or T are 1/2. So, if I want to define a Probability Distribution Function over a random variable X, it’ll look something like this —
F(X) = P(X=x) = { 1/2, 1/2 } [ x = H or T ]
I know, it might seem too much for a few people. Let me take a step backward, and that will make things clearer.
Algebraic Variable vs. Random Variable
An algebraic variable is a normal variable that we’ve been using since our childhood. For example, X = 2.
On the other hand, a Random variable contains outcomes of experiments. It can be the outcomes of an experiment made with a dice or a coin. In a dice experiment, for a random variable X, P(X=1)…P(X=6) will contain 1/6 as a value because all the outcomes of a dice experiment have equal probability.
Let’s understand the example with a graph —
Sample space of a dice — {1,2,3,4,5,6}
Now, if we plot this with Probability Distribution Function —
Y = P(X), we’ll get a graph something like this —
In the above image, since all outcomes have equal probability, we observe a uniform distribution.
We can easily understand that the probability distribution function provides us with a better insight into a random variable and its outcomes. Now, there are different types of probability distribution functions. Let’s understand them one by one.
Probability Mass Function (PMF)
In the domain of Probability and Statistics, Probability Mass Function (PMF) is a type of function that takes a value of a discrete random variable and maps it to a probability value. And, if you sum up all the probability values, you’ll end up getting 1.
For instance, —
PMF: P(X=x) = { 1/6, 1/6, 1/6, 1/6, 1/6, 1/6 } ( For a dice experiment )
Here, x = {1,2,3,4,5,6} and P(X=1) = 1/6. If you add all the probability values together, you’ll get 1.
Let’s look at another example—
When two dice are rolled simultaneously, what is the probability distribution of the sum of the numbers on their faces?
Some example outcomes include: [2 →(1+1), 3 →(2+1, 1+2), 4 →(2+2), 5 →(2+3, 3+2, 4+1, 1+4), 6 →(3+3), 7 →(3+4, 4+3, 5+2, 2+5 …), 8 →(4+4 …), 9 →(4+5 …), 10 →(5+5 …), 11 →(5+6 …), 12 →(6+6)]
If you take the first example from the above, 2 →(1+1), it means the face value of both dice is 1.
To generalize it more, if you draw a table T ( 6x6 ), and for a single value of Tₘₙ put (m+n), you’ll get a table containing all the sums of the face values. The value for T₆₆ will be ( 6 + 6 ) → 12. And the probability of this is 1/36.
Get Jeet Mukherjee’s stories in your inbox
Join Medium for free to get updates from this writer.
Now, if we plot the distribution graph, it’ll look something like this —
PMF: P(X=x) = {1/36, 2/36, 3/36, 4/36, 5/36, 6/36, 5/36, 4/36, 3/36, 2/36, 1/36}
If you add all the values for P(X=x), you’ll get 1.
Cumulative Distribution Function (CDF)
Mathematically if we want to describe CDF — F(x)=P(X≤x)
This represents the probability that the random variable X takes on a value less than or equal to a specific value x. If we use the above example —
P(X≤5) = 10/36 → { 1/36 + 2/36 + 3/36 + 4/36 }
P(X≤12) = 1 → Can easily infer from the above example.
If we try to plot this example with a graph. It’ll look something like this —
We have just seen how the CDF can be used for a discrete random variable; we’ll revisit the CDF once we understand the PDF. And we’ll see how we can use CDF for Continuous Random Variable.
Probability Density Function (PDF)
Probability Density Function (PDF) is the study of the distribution of Continuous Random Variables in the field of Statistics & Probability. You can imagine this exactly like PMF, but for continuous variables rather than discrete.
Let’s try to understand PDF with an example —
Suppose a person takes 20 to 40 minutes to reach his office, depending on the traffic. The commute time is equally likely to occur, which means that the probability of the commute taking exactly 21 minutes is the same as it taking 32 or 38 minutes.
If certain commute times occurred more frequently than others, for example, if 30 minutes happened much more often than the curve would be taller at that point, indicating a higher probability density.
Now, we already know that Commute Time is a continuous random variable, and if you try to calculate the probability of the occurrence of 25.5, you’ll get 0. And this will be applicable for any value in the range. So, to calculate the probability, we have to calculate the area of the range. This is because any particular value doesn’t hold any area under the curve in a continuous distribution.
In the above example, we are calculating the probability of commute time in the range of 25 to 30. So, we perform integration.
By calculating the area over the interval of 25–30, we obtain the value of 0.25, which represents the probability. Since we are using a uniform distribution, we can also calculate this value by using the formula area = width * height. (This approach does not apply to any other distribution; you need to use integration.)
area = 0.05 * 5 = 0.25
If you want to use CDF for this example, you can do that as well. If you try to calculate P(X<20), the answer will be 0. In the same way, P(X>40) will be 1. And for P(X≤x), the formula will be — [(x-20)/20]. Here’s the visual representation.
Even though the above graph can give you an intuitive understanding, it needs more explanation. Hopefully, in another blog, we’ll see the types of distribution and dive into a more detailed explanation of CDF.
I know, this was a lot. Though I tried my best to keep it simple. Let me know what’s your thoughts about it.
Thanks :)
