Chebyshev’s Inequality

Know only the expected value and the variance? You can still limit the probability of values at least k away from the expected value

Chebyshev’s Inequality

Often we do not know the probability distribution of a random variable 𝑋. But we know two numbers about it: its expected value 𝐸[𝑋] and its variance Var(𝑋), and we know that both are finite (that is, −∞ < 𝐸[𝑋] < +∞ and 0 ≤ Var(𝑋) < +∞). In these cases, we can use Chebyshev’s inequality to find an upper bound on the probability that 𝑋 takes a value whose distance from its expected value is greater than or equal to a given number 𝑘 > 0.

Chebyshev’s inequality states the following. Let 𝑋 be a random variable with a finite expected value 𝐸[𝑋] and a finite variance Var(𝑋). Then, for every 𝑘 > 0, the probability that the distance |𝑋 − 𝐸[𝑋]| between the value taken by 𝑋 and the expected value 𝐸[𝑋] is greater than or equal to 𝑘 is at most Var(𝑋) / 𝑘². In symbols:

𝑃(|𝑋 − 𝐸[𝑋]| ≥ 𝑘) ≤ Var(𝑋) / 𝑘²

Unlike Markov’s inequality, Chebyshev’s inequality does not require 𝑋 to take only values greater than or equal to zero, but it requires one more piece of information: the variance of 𝑋 must also be known and finite.

Example

Imagine that, in a mountain town, the daily temperature in January has, according to meteorologists, an expected value of 1 °C and a variance of 9 (°C)². Let 𝑋 be the random variable that models the temperature on a January day in that town, in °C.

We can note that the following conditions are true:

1. 𝑋 can take both positive and negative values;
2. the expected value of 𝑋 is known and finite: 𝐸[𝑋] = 1 °C;
3. the variance of 𝑋 is known and finite: Var(𝑋) = 9 (°C)².

Point 1 tells us that we cannot use Markov’s inequality, even if we know that the expected value of 𝑋 is finite. Points 2 and 3 tell us that we can use Chebyshev’s inequality.

Chebyshev’s inequality gives an upper bound on the probability that the temperature is at least 𝑘 degrees away from 1 °C (the expected value), above or below, for any 𝑘 > 0. Since the variance is 9 (°C)², we have:

𝑃(|𝑋 − 1| ≥ 𝑘) ≤ 9 / 𝑘²

For example, with 𝑘 = 6 °C, that is, considering the event that the temperature on a January day differs from the expected temperature by at least 6 °C (above or below), we get:

𝑃(|𝑋 − 1| ≥ 6) ≤ 9 /6² = 9 / 36 = 0.25

The event |𝑋 − 1| ≥ 6 includes two cases: 𝑋 ≥ 1 + 6 = 7 or 𝑋 ≤ 1 − 6 = −5. So the probability that the temperature on a January day is 7 °C or more, or −5 °C or less, is at most 0.25.

Proof

The proof of Chebyshev’s inequality is based on the analysis of the following random variable:

Get Ottavio Calzone’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

𝑌 = (𝑋 − 𝐸[𝑋])²

The random variable 𝑌 has the following two properties:

1. 𝑌 takes only values greater than or equal to zero, because a square cannot be negative;
2. 𝑌 has a finite expected value:

• by the definition of 𝑌, 𝐸[𝑌] = 𝐸[(𝑋 − 𝐸[𝑋])²];
• by the definition of variance, 𝐸[(𝑋 − 𝐸[𝑋])²] = Var(𝑋);
• and Var(𝑋) is finite by hypothesis.

These two properties are the conditions of Markov’s inequality, so we can apply it to 𝑌 and state that, for every 𝑎 > 0:

𝑃(𝑌 ≥ 𝑎) ≤ 𝐸[𝑌] / 𝑎

We can replace 𝑎 with the value 𝑘², where 𝑘 > 0 (so 𝑘² > 0 too). This gives:

𝑃(𝑌 ≥ 𝑘²) ≤ 𝐸[𝑌] / 𝑘²

Since we know that 𝐸[𝑌] = Var(𝑋), we can write:

𝑃(𝑌 ≥ 𝑘²) ≤ Var(𝑋) / 𝑘²

We can note that every outcome in the event |𝑋 − 𝐸[𝑋]| ≥ 𝑘 is also in the event 𝑌 ≥ 𝑘². Indeed, if |𝑋 − 𝐸[𝑋]| ≥ 𝑘, then:

1. we can square both sides of |𝑋 − 𝐸[𝑋]| ≥ 𝑘 and get |𝑋 − 𝐸[𝑋]|² ≥ 𝑘² (since both |𝑋 − 𝐸[𝑋]| and 𝑘 are greater than or equal to zero, the direction of the inequality does not change);
2. 𝑌 ≥ 𝑘², since, by the definition of 𝑌, |𝑋 − 𝐸[𝑋]|² = (𝑋 − 𝐸[𝑋])² = 𝑌.

The two points above tell us that every outcome in the event |𝑋 − 𝐸[𝑋]| ≥ 𝑘 is also in the event 𝑌 ≥ 𝑘². This means that the probability of the event |𝑋 − 𝐸[𝑋]| ≥ 𝑘 cannot be greater than the probability of the event 𝑌 ≥ 𝑘² (by the monotonicity of probability). So we can state that:

𝑃(|𝑋 − 𝐸[𝑋]| ≥ 𝑘) ≤ 𝑃(𝑌 ≥ 𝑘²)

Since we know that 𝑃(𝑌 ≥ 𝑘²) ≤ Var(𝑋) / 𝑘², we get what Chebyshev’s inequality states:

𝑃(|𝑋 − 𝐸[𝑋]| ≥ 𝑘) ≤ Var(𝑋) / 𝑘²