Bayes’ Rule

Bayes’ Rule

Review

8.1

Lecture 8Bayes’ RuleText: A Course in Probability by Weiss 4.4

STAT 225 Introduction to Probability ModelsFebruary 2, 2014

Whitney HuangPurdue University

Bayes’ Rule

Bayes’ Rule

Review

8.2

Agenda

1 Bayes’ Rule

2 Review

Bayes’ Rule

Bayes’ Rule

Review

8.3

Motivating example

Example (The Monty Hall Problem)

There was an old television show called Let’s Make a Deal,whose original host was named Monty Hall. The set–up is asfollows. You are on a game show and you are given the choiceof three doors. Behind one door is a car, behind the others aregoats. You pick a door, and the host, who knows what isbehind the doors, opens another door (not your pick) which hasa goat behind it. Then he asks you if you want to change youroriginal pick. The question we ask you is, “Is it to youradvantage to switch your choice?"

Bayes’ Rule

Bayes’ Rule

Review

8.4

The Monty Hall Problem

Bayes’ Rule

Bayes’ Rule

Review

8.5

The Monty Hall Problem

Solution.

Bayes’ Rule

Bayes’ Rule

Review

8.5

The Monty Hall Problem

Solution.

Bayes’ Rule

Bayes’ Rule

Review

8.6

Bayes’ Rule

Let A1,A2, · · · ,Ak form a partition of the sample space. Thenfor every event B in the sample space,

P(Aj |B) =P(B|Aj )× P(Aj )∑ki=1 P(B|Ai )× P(Ai )

, j = 1,2, · · · , k

Bayes’ Rule

Bayes’ Rule

Review

8.7

Example 24

Let us assume that a specific disease is only present in 5 out ofevery 1,000 people. Suppose that the test for the disease isaccurate 99% of the time a person has the disease and 95% ofthe time that a person lacks the disease. What is the probabilitythat the person has the disease given that they tested positive?

Solution.

P(D|+) = P(D∩+)P(+) = .005×.99

.005×.99+.995×.05 = .00495.0547 = .0905

The reason we get such a surprising result is because thedisease is so rare that the number of false positives greatlyoutnumbers the people who truly have the disease.

Bayes’ Rule

Bayes’ Rule

Review

8.7

Example 24

Let us assume that a specific disease is only present in 5 out ofevery 1,000 people. Suppose that the test for the disease isaccurate 99% of the time a person has the disease and 95% ofthe time that a person lacks the disease. What is the probabilitythat the person has the disease given that they tested positive?

Solution.

P(D|+) = P(D∩+)P(+) = .005×.99

.005×.99+.995×.05 = .00495.0547 = .0905

The reason we get such a surprising result is because thedisease is so rare that the number of false positives greatlyoutnumbers the people who truly have the disease.

Bayes’ Rule

Bayes’ Rule

Review

8.8

Basic Concepts

Random experiment: sample space, element, eventSet operations: union, intersectionSet relations: mutually exclusive, exhaust, partition

Bayes’ Rule

Bayes’ Rule

Review

8.8

Basic Concepts

Random experiment: sample space, element, event

Set operations: union, intersectionSet relations: mutually exclusive, exhaust, partition

Bayes’ Rule

Bayes’ Rule

Review

8.8

Basic Concepts

Random experiment: sample space, element, eventSet operations: union, intersection

Set relations: mutually exclusive, exhaust, partition

Bayes’ Rule

Bayes’ Rule

Review

8.8

Basic Concepts

Random experiment: sample space, element, eventSet operations: union, intersectionSet relations: mutually exclusive, exhaust, partition

Bayes’ Rule

Bayes’ Rule

Review

8.9

Probability Rules

0 ≤ P(E) ≤ 1 for any event E , P(∅) = 0, P(Ω) = 1Complement rule: P(A) = 1− P(Ac)

General addition rule: P(A ∪ B) = P(A) + P(B)− P(A ∩ B)

General multiplication rule:P(A ∩ B) = P(A|B)× P(B) = P(B|A)× P(A)

Conditional probability: P(A|B) = P(A∩B)P(B)

Law of total probability:P(B) =

∑ki=1 P(B ∩ Ai ) =

∑ki=1 P(B|Ai )× P(Ai )

Independence: if A and B are independent, thenP(A|B) = P(A), P(B|A) = P(B), and P(A ∩ B) = P(A)P(B)

Bayes’ Rule

Bayes’ Rule

Review

8.9

Probability Rules

0 ≤ P(E) ≤ 1 for any event E , P(∅) = 0, P(Ω) = 1

Complement rule: P(A) = 1− P(Ac)

General addition rule: P(A ∪ B) = P(A) + P(B)− P(A ∩ B)

General multiplication rule:P(A ∩ B) = P(A|B)× P(B) = P(B|A)× P(A)

Conditional probability: P(A|B) = P(A∩B)P(B)

Law of total probability:P(B) =

∑ki=1 P(B ∩ Ai ) =

∑ki=1 P(B|Ai )× P(Ai )

Independence: if A and B are independent, thenP(A|B) = P(A), P(B|A) = P(B), and P(A ∩ B) = P(A)P(B)

Bayes’ Rule

Bayes’ Rule

Review

8.9

Probability Rules

0 ≤ P(E) ≤ 1 for any event E , P(∅) = 0, P(Ω) = 1Complement rule: P(A) = 1− P(Ac)

General addition rule: P(A ∪ B) = P(A) + P(B)− P(A ∩ B)

General multiplication rule:P(A ∩ B) = P(A|B)× P(B) = P(B|A)× P(A)

Conditional probability: P(A|B) = P(A∩B)P(B)

Law of total probability:P(B) =

∑ki=1 P(B ∩ Ai ) =

∑ki=1 P(B|Ai )× P(Ai )

Independence: if A and B are independent, thenP(A|B) = P(A), P(B|A) = P(B), and P(A ∩ B) = P(A)P(B)

Bayes’ Rule

Bayes’ Rule

Review

8.9

Probability Rules

0 ≤ P(E) ≤ 1 for any event E , P(∅) = 0, P(Ω) = 1Complement rule: P(A) = 1− P(Ac)

General addition rule: P(A ∪ B) = P(A) + P(B)− P(A ∩ B)

General multiplication rule:P(A ∩ B) = P(A|B)× P(B) = P(B|A)× P(A)

Conditional probability: P(A|B) = P(A∩B)P(B)

Law of total probability:P(B) =

∑ki=1 P(B ∩ Ai ) =

∑ki=1 P(B|Ai )× P(Ai )

Independence: if A and B are independent, thenP(A|B) = P(A), P(B|A) = P(B), and P(A ∩ B) = P(A)P(B)

Bayes’ Rule

Bayes’ Rule

Review

8.9

Probability Rules

0 ≤ P(E) ≤ 1 for any event E , P(∅) = 0, P(Ω) = 1Complement rule: P(A) = 1− P(Ac)

General addition rule: P(A ∪ B) = P(A) + P(B)− P(A ∩ B)

General multiplication rule:P(A ∩ B) = P(A|B)× P(B) = P(B|A)× P(A)

Conditional probability: P(A|B) = P(A∩B)P(B)

Law of total probability:P(B) =

∑ki=1 P(B ∩ Ai ) =

∑ki=1 P(B|Ai )× P(Ai )

Independence: if A and B are independent, thenP(A|B) = P(A), P(B|A) = P(B), and P(A ∩ B) = P(A)P(B)

Bayes’ Rule

Bayes’ Rule

Review

8.9

Probability Rules

0 ≤ P(E) ≤ 1 for any event E , P(∅) = 0, P(Ω) = 1Complement rule: P(A) = 1− P(Ac)

General addition rule: P(A ∪ B) = P(A) + P(B)− P(A ∩ B)

General multiplication rule:P(A ∩ B) = P(A|B)× P(B) = P(B|A)× P(A)

Conditional probability: P(A|B) = P(A∩B)P(B)

Law of total probability:P(B) =

∑ki=1 P(B ∩ Ai ) =

∑ki=1 P(B|Ai )× P(Ai )

Independence: if A and B are independent, thenP(A|B) = P(A), P(B|A) = P(B), and P(A ∩ B) = P(A)P(B)

Bayes’ Rule

Bayes’ Rule

Review

8.9

Probability Rules

0 ≤ P(E) ≤ 1 for any event E , P(∅) = 0, P(Ω) = 1Complement rule: P(A) = 1− P(Ac)

General addition rule: P(A ∪ B) = P(A) + P(B)− P(A ∩ B)

General multiplication rule:P(A ∩ B) = P(A|B)× P(B) = P(B|A)× P(A)

Conditional probability: P(A|B) = P(A∩B)P(B)

Law of total probability:P(B) =

∑ki=1 P(B ∩ Ai ) =

∑ki=1 P(B|Ai )× P(Ai )

Independence: if A and B are independent, thenP(A|B) = P(A), P(B|A) = P(B), and P(A ∩ B) = P(A)P(B)

Bayes’ Rule

Bayes’ Rule

Review

8.9

Probability Rules

0 ≤ P(E) ≤ 1 for any event E , P(∅) = 0, P(Ω) = 1Complement rule: P(A) = 1− P(Ac)

General addition rule: P(A ∪ B) = P(A) + P(B)− P(A ∩ B)

General multiplication rule:P(A ∩ B) = P(A|B)× P(B) = P(B|A)× P(A)

Conditional probability: P(A|B) = P(A∩B)P(B)

Law of total probability:P(B) =

∑ki=1 P(B ∩ Ai ) =

∑ki=1 P(B|Ai )× P(Ai )

Independence: if A and B are independent, thenP(A|B) = P(A), P(B|A) = P(B), and P(A ∩ B) = P(A)P(B)

Bayes’ Rule

Bayes’ Rule

Review

8.10

Counting Problem

Basic counting rulePermutationCombinationOrdered partition