Bayes's Theorem is a probability rule that helps us calculate the chance of an event by using both prior knowledge and new information. In simple terms, it allows us to update our understanding or prediction when new evidence becomes available. It is widely used in statistics, machine learning, and decision-making.
The figure below shows the formula of Bayes's Theorem, along with an explanation of each probability term used in the formula:
Read: Conditional Probability
Mathematical Formulation
To derive Bayes's Theorem, we start with the definition of conditional probability:
Rearranging the second equation to express P(A∩B):
P(A∩B)=P(B∣A)⋅P(A)
Substitute this into the first equation:
This is the formula for Bayes's Theorem.
Applications
- Signal Processing: Bayes's Theorem is used to improve signal filtering and estimation by updating system predictions using new measurements.
- Machine Learning: It forms the basis of algorithms such as Naive Bayes, which are widely used for classification tasks like spam detection, text analysis, and sentiment analysis.
- Reliability Engineering: Bayes's Theorem helps estimate the likelihood of system failures based on observed data, enabling better maintenance planning and failure prediction.
- Medical Diagnosis: It is used to update the probability of a disease based on test results, helping healthcare professionals make more accurate diagnoses.
- Robotics: Bayes's Theorem assists robots in tasks such as localization and mapping by continuously updating their position estimates using sensor data.
Solved Examples
Simple Disease Test
Problem 1: A disease affects 1% of the population. A test is 95% accurate for both positive and negative results. If a person tests positive, what's the probability they have the disease?
Solution:
P(D) = 0.01, P(T|D) = 0.95, P(T|not D) = 0.05
P(T) = 0.95 * 0.01 + 0.05 * 0.99 = 0.0585
P(D|T) = (0.95 * 0.01) / 0.0585 ≈ 0.1624 or 16.24%
Factory Quality Control
Problem 2: A factory has two production lines. Line A produces 60% of the items and has a 3% defect rate. Line B produces 40% of the items and has a 2% defect rate. If an item is defective, what's the probability it came from Line A?
Solution:
P(A) = 0.6, P(D|A) = 0.03, P(D|B) = 0.02
P(D) = 0.03 * 0.6 + 0.02 * 0.4 = 0.026
P(A|D) = (0.03 * 0.6) / 0.026 ≈ 0.6923 or 69.23%
Weather Forecast
Problem 3: The weather is sunny 70% of the time. A weather app is 80% accurate when predicting sunny days and 60% accurate for non-sunny days. If the app predicts a sunny day, what's the probability it will actually be sunny?
Solution:
P(S) = 0.7, P(P|S) = 0.8, P(P|not S) = 0.4
P(P) = 0.8 * 0.7 + 0.4 * 0.3 = 0.68
P(S|P) = (0.8 * 0.7) / 0.68 ≈ 0.8235 or 82.35%
Email Classification
Problem 4: 20% of emails are important. A filter correctly identifies 90% of important emails and 85% of unimportant emails. If an email is marked as important, what's the probability it's actually important?
Solution:
P(I) = 0.2, P(M|I) = 0.9, P(M|not I) = 0.15
P(M) = 0.9 * 0.2 + 0.15 * 0.8 = 0.3
P(I|M) = (0.9 * 0.2) / 0.3 = 0.6 or 60%
Drug Test
Problem 5: 5% of athletes use performance-enhancing drugs. A drug test is 99% accurate for users and 95% accurate for non-users. If an athlete tests positive, what's the probability they're actually using drugs?
Solution:
P(D) = 0.05, P(T|D) = 0.99, P(T|not D) = 0.05
P(T) = 0.99 * 0.05 + 0.05 * 0.95 = 0.0495 + 0.0475 = 0.0970
P(D|T) = (0.99 * 0.05) / 0.0970 = 0.0495 / 0.0970 ≈ 0.5103 or 51.03%
Customer Loyalty
Problem 6: 30% of customers are loyal. Loyal customers have a 80% chance of making a purchase, while non-loyal customers have a 20% chance. If a customer makes a purchase, what's the probability they're loyal?
Solution:
P(L) = 0.3, P(P|L) = 0.8, P(P|not L) = 0.2
P(P) = 0.8 * 0.3 + 0.2 * 0.7 = 0.38
P(L|P) = (0.8 * 0.3) / 0.38 ≈ 0.6316 or 63.16%
Fire Alarm
Problem 7: The probability of a fire in a building is 0.1%. The fire alarm has a 99% chance of detecting a fire and a 0.5% false alarm rate. If the alarm goes off, what's the probability there's actually a fire?
Solution:
P(F) = 0.001, P(A|F) = 0.99, P(A|not F) = 0.005
P(A) = 0.99 * 0.001 + 0.005 * 0.999 = 0.005994
P(F|A) = (0.99 * 0.001) / 0.005994 ≈ 0.1652 or 16.52%
College Admissions
Problem 8: 40% of applicants have high test scores. Among those with high scores, 80% are admitted. Among those without high scores, 30% are admitted. If a student is admitted, what's the probability that they had high test scores?
Solution:
P(H) = 0.4, P(A|H) = 0.8, P(A|not H) = 0.3
P(A) = 0.8 * 0.4 + 0.3 * 0.6 = 0.5
P(H|A) = (0.8 * 0.4) / 0.5 = 0.64 or 64%
Car Insurance
Problem 9: 15% of drivers are high-risk. High-risk drivers have a 20% chance of an accident in a year, while others have a 5% chance. If a driver has an accident, what's the probability they're high-risk?
Solution:
P(H) = 0.15, P(A|H) = 0.2, P(A|not H) = 0.05
P(A) = 0.2 * 0.15 + 0.05 * 0.85 = 0.0725
P(H|A) = (0.2 * 0.15) / 0.0725 ≈ 0.4138 or 41.38%
Software Bug Detection
Problem 10 : A software has a 10% chance of containing a critical bug. A testing tool detects 95% of critical bugs and has a 8% false positive rate. If the tool reports a critical bug, what's the probability the software actually has one?
Solution:
P(B) = 0.1, P(D|B) = 0.95, P(D|not B) = 0.08
P(D) = 0.95 * 0.1 + 0.08 * 0.9 = 0.167
P(B|D) = (0.95 * 0.1) / 0.167 ≈ 0.5689 or 56.89%
Practice Problems
Problem 1: In a city, 30% of residents own dogs. Among dog owners, 80% walk their dogs daily, while among non-dog owners, 10% walk daily for exercise. If a person is seen walking, what is the probability that they are a dog owner?
Problem 2: A rare genetic disorder affects 1 in 10,000 people. A diagnostic test is 99% accurate for both positive and negative results. If a person tests positive, what is the probability that they actually have the disorder?
Problem 3: A company manufactures widgets using two machines. Machine A produces 70% of the widgets and has a defect rate of 3%, while Machine B produces 30% of the widgets and has a defect rate of 1%. If a widget is found to be defective, what is the probability that it was produced by Machine A?
Problem 4: In a school, 60% of students participate in sports. Among students who play sports, 80% pass their exams, while among non-athletes, 60% pass. If a student passes an exam, what is the probability that they play sports?
Problem 5: A spam filter correctly identifies 98% of spam emails and 95% of non-spam emails. If 40% of all emails are spam and an email is flagged as spam, what is the probability that it is actually spam?