Holm-Bonferroni Method

In data analysis, it's common to run many hypothesis tests at once for example, checking thousands of genes in a medical study to see which are linked to a disease. But the more tests we do, the higher the chance of getting results that look important just by accident. To reduce these false positives, statisticians use correction methods. The Holm-Bonferroni method is one of them and is popular because it controls these errors without being too strict, making it more effective than the older Bonferroni method.

How Does It Work?

Let’s say you run m hypothesis tests and get m p-values.

1. Sort the p-values in ascending order:

p_{(1)} \leq p_{(2)} \leq \cdots \leq p_{(m)}

2. Assign ranks to the sorted p-values. Let p_{(i)} be the i-th smallest p-value.

3. Compare each p-value to its adjusted threshold:

p_{(i)} \leq \frac{\alpha}{m - i + 1}

where α\alphaα is your desired significance level (e.g., 0.05).

4. Reject all hypotheses from the first p_{(1)} up to the last one that satisfies the condition.

Once a p-value fails the condition, stop rejecting further hypotheses, even if the later p-values meet the threshold.

Example- Use the Holm-Bonferroni method to test the following four hypotheses and their associated p-values at an alpha level of 0.05:

Step-by-Step Solution Using the Holm-Bonferroni Method

Step 1: Sort the p-values in ascending order and match the corresponding hypotheses

Step 2: Compute adjusted significance levels

Step 3: Compare p-values to corresponding α,

Holm-Bonferroni vs Bonferroni

• Bonferroni rejects only if p-value < α / m
• Holm-Bonferroni adapts the threshold as you go through each p-value

So, Holm-Bonferroni is always at least as powerful as Bonferroni, and usually better.

When to Use Holm-Bonferroni

• When controlling the family-wise error rate (FWER) is important
• When you have a moderate number of hypothesis tests
• When you want a method that's stronger than Bonferroni but still simple to understand and apply

Advantages of Holm-Bonferroni

• More powerful than Bonferroni (less likely to miss true positives)
• Easy to implement by sorting and comparing p-values
• Controls FWER in a reliable way
• Works for independent and some dependent tests

Limitations

• Still conservative, especially when many tests are performed
• Does not control the False Discovery Rate (FDR) use Benjamini-Hochberg for that
• Cannot recover power loss in very high-dimensional testing