foundations of computer science lecture 15...
TRANSCRIPT
Foundations of Computer Science
Lecture 15
ProbabilityComputing Probabilities
Probability and Sets: Probability Space
Uniform Probability Spaces
Infinite Probability Spaces
The probable is what usually happens – Aristotle
Last Time
To count complex objects, construct a sequence of “instructions” that can beused to construct the object uniquely. The number of possible sequences ofinstructions equals the number of possible complex objects.
1 Counting Sequences with and without repetition. Subsets with and without repetition. Sequences with specified numbers of each type of object: anagrams.
2 Inclusion-Exclusion (advanced technique).
3 Pigeonhole principle (simple but IMPORTANT technique).
Creator: Malik Magdon-Ismail Probability: 2 / 14 Today →
Today: Probability
1 Computing probabilities.Outcome tree.
Event of interest.
Examples with dice.
2 Probability and sets.The probability space.
3 Uniform probability spaces.
4 Infinite probability spaces.
Creator: Malik Magdon-Ismail Probability: 3 / 14 Chances of Rain →
The Chance of Rain Tomorrow is 40%
What does the title mean? Either it will rain tomorrow or it won’t.
Creator: Malik Magdon-Ismail Probability: 4 / 14 Toss Two Coins →
The Chance of Rain Tomorrow is 40%
What does the title mean? Either it will rain tomorrow or it won’t.
The chances are 50% that a fair coin-flip will be H.
Creator: Malik Magdon-Ismail Probability: 4 / 14 Toss Two Coins →
The Chance of Rain Tomorrow is 40%
What does the title mean? Either it will rain tomorrow or it won’t.
The chances are 50% that a fair coin-flip will be H.
Flip 100 times. Approximately 50 will be H ← frequentist view.
Creator: Malik Magdon-Ismail Probability: 4 / 14 Toss Two Coins →
The Chance of Rain Tomorrow is 40%
What does the title mean? Either it will rain tomorrow or it won’t.
The chances are 50% that a fair coin-flip will be H.
Flip 100 times. Approximately 50 will be H ← frequentist view.
1 You toss a fair coin 3 times. How many heads will you get?
Creator: Malik Magdon-Ismail Probability: 4 / 14 Toss Two Coins →
The Chance of Rain Tomorrow is 40%
What does the title mean? Either it will rain tomorrow or it won’t.
The chances are 50% that a fair coin-flip will be H.
Flip 100 times. Approximately 50 will be H ← frequentist view.
1 You toss a fair coin 3 times. How many heads will you get?
2 You keep tossing a fair coin until you get a head. How many tosses will you make?
Creator: Malik Magdon-Ismail Probability: 4 / 14 Toss Two Coins →
The Chance of Rain Tomorrow is 40%
What does the title mean? Either it will rain tomorrow or it won’t.
The chances are 50% that a fair coin-flip will be H.
Flip 100 times. Approximately 50 will be H ← frequentist view.
1 You toss a fair coin 3 times. How many heads will you get?
2 You keep tossing a fair coin until you get a head. How many tosses will you make?
There’s no answer. The outcome is uncertain. Probability is appropriate for such settings.
Creator: Malik Magdon-Ismail Probability: 4 / 14 Toss Two Coins →
Toss Two Coins: You Win if the Coins Match (HH or TT)
1 You are analyzing an “experiment” whoseoutcome is uncertain.
Creator: Malik Magdon-Ismail Probability: 5 / 14 Event of Interest →
Toss Two Coins: You Win if the Coins Match (HH or TT)
1 You are analyzing an “experiment” whoseoutcome is uncertain.
2 Outcomes. Identify all possible outcomes using
a tree of outcome sequences. H T Coin 1
Creator: Malik Magdon-Ismail Probability: 5 / 14 Event of Interest →
Toss Two Coins: You Win if the Coins Match (HH or TT)
1 You are analyzing an “experiment” whoseoutcome is uncertain.
2 Outcomes. Identify all possible outcomes using
a tree of outcome sequences. H T
H T H T Coin 2
Coin 1
Creator: Malik Magdon-Ismail Probability: 5 / 14 Event of Interest →
Toss Two Coins: You Win if the Coins Match (HH or TT)
1 You are analyzing an “experiment” whoseoutcome is uncertain.
2 Outcomes. Identify all possible outcomes using
a tree of outcome sequences. H T
H T H T Coin 2
Coin 1
OutcomeHH HT TH TT
Creator: Malik Magdon-Ismail Probability: 5 / 14 Event of Interest →
Toss Two Coins: You Win if the Coins Match (HH or TT)
1 You are analyzing an “experiment” whoseoutcome is uncertain.
2 Outcomes. Identify all possible outcomes using
a tree of outcome sequences.
3 Edge probabilities. If one of k edges
(options) from a vertex is chosen randomly then
each edge has edge-probability 1k.
H T
H T H T Coin 2
Coin 1
OutcomeHH HT TH TT
1
2
1
2
1
2
1
2
1
2
1
2
Creator: Malik Magdon-Ismail Probability: 5 / 14 Event of Interest →
Toss Two Coins: You Win if the Coins Match (HH or TT)
1 You are analyzing an “experiment” whoseoutcome is uncertain.
2 Outcomes. Identify all possible outcomes using
a tree of outcome sequences.
3 Edge probabilities. If one of k edges
(options) from a vertex is chosen randomly then
each edge has edge-probability 1k.
4 Outcome-probability. Multiply
edge-probabilities to get outcome-probabilities.
H T
H T H T Coin 2
Coin 1
OutcomeHH HT TH TT
1
2
1
2
1
2
1
2
1
2
1
2
Probability14
14
14
14
Creator: Malik Magdon-Ismail Probability: 5 / 14 Event of Interest →
Event of Interest
Toss two coins: you win if the coins match (HH or TT)
Question: When do you win? Event: Subset of outcomes where you win.
H T
H T H T Coin 2
Coin 1
OutcomeHH HT TH TT
1
2
1
2
1
2
1
2
1
2
1
2
Probability14
14
14
14
Creator: Malik Magdon-Ismail Probability: 6 / 14 The Outcome-Tree Method →
Event of Interest
Toss two coins: you win if the coins match (HH or TT)
Question: When do you win? Event: Subset of outcomes where you win.
5 Event of interest. Subset of the outcomes
where you win.H T
H T H T Coin 2
Coin 1
OutcomeHH HT TH TT
1
2
1
2
1
2
1
2
1
2
1
2
Probability14
14
14
14
HH
14
TT
14
Creator: Malik Magdon-Ismail Probability: 6 / 14 The Outcome-Tree Method →
Event of Interest
Toss two coins: you win if the coins match (HH or TT)
Question: When do you win? Event: Subset of outcomes where you win.
5 Event of interest. Subset of the outcomes
where you win.
6 Event-probability. Sum of its
outcome-probabilities.
event-probability =1
4+
1
4=
1
2.
H T
H T H T Coin 2
Coin 1
OutcomeHH HT TH TT
1
2
1
2
1
2
1
2
1
2
1
2
Probability14
14
14
14
HH
14
TT
14
Probability that you win is 1
2, written as P[“YouWin”] = 1
2.
Go and do this experiment at home. Toss two coins 1000 times and see how often you win.
Creator: Malik Magdon-Ismail Probability: 6 / 14 The Outcome-Tree Method →
The Outcome-Tree Method
Become familiar with this 6-step process for analyzing a probabilistic experiment.
1 You are analyzing an experiment whose outcome is uncertain.
2 Outcomes. Identify all possible outcomes, the tree of outcome sequences.
3 Edge-Probability. Each edge in the outcome-tree gets a probability.
4 Outcome-Probability. Multiply edge-probabilities to get outcome-probabilities.
5 Event of Interest E. Determine the subset of the outcomes you care about.
6 Event-Probability. The sum of outcome-probabilities in the subset you care about.
P[E ] =∑
outcomes ω ∈ EP (ω).
P[E ] ∼ frequency an outcome you want occurs over many repeated experiments.
Pop Quiz. Roll two dice. Compute P[first roll is less than the second].
Creator: Malik Magdon-Ismail Probability: 7 / 14 Let’s Make a Deal →
Let’s Make a Deal: The Monty Hall Problem
1: Contestant at door 1.2: Prize placed behind random door.
1 2 3
Creator: Malik Magdon-Ismail Probability: 8 / 14 Non-Transitive Dice →
Let’s Make a Deal: The Monty Hall Problem
1: Contestant at door 1.2: Prize placed behind random door.3: Monty opens empty door (randomly if there’s an option).
1 2 3
1∅
3
Creator: Malik Magdon-Ismail Probability: 8 / 14 Non-Transitive Dice →
Let’s Make a Deal: The Monty Hall Problem
1: Contestant at door 1.2: Prize placed behind random door.3: Monty opens empty door (randomly if there’s an option).
1 2 3
1∅
3
Outcome-tree and edge-probabilities.
1
2
3
1
3
1
3
1
3
Prize
Creator: Malik Magdon-Ismail Probability: 8 / 14 Non-Transitive Dice →
Let’s Make a Deal: The Monty Hall Problem
1: Contestant at door 1.2: Prize placed behind random door.3: Monty opens empty door (randomly if there’s an option).
1 2 3
1∅
3
Outcome-tree and edge-probabilities.
1
2
3
2
3
3
2
1
3
1
3
1
3
1
2
1
2
1
1
Prize Host
Creator: Malik Magdon-Ismail Probability: 8 / 14 Non-Transitive Dice →
Let’s Make a Deal: The Monty Hall Problem
1: Contestant at door 1.2: Prize placed behind random door.3: Monty opens empty door (randomly if there’s an option).
1 2 3
1∅
3
Outcome-tree and edge-probabilities.
1
2
3
2
3
3
2
1
3
1
3
1
3
1
2
1
2
1
1
(1, 2)
(1, 3)
(2, 3)
(3, 2)
Prize Host Outcome
Creator: Malik Magdon-Ismail Probability: 8 / 14 Non-Transitive Dice →
Let’s Make a Deal: The Monty Hall Problem
1: Contestant at door 1.2: Prize placed behind random door.3: Monty opens empty door (randomly if there’s an option).
1 2 3
1∅
3
Outcome-tree and edge-probabilities.
Outcome-probabilities.
1
2
3
2
3
3
2
1
3
1
3
1
3
1
2
1
2
1
1
(1, 2)
(1, 3)
(2, 3)
(3, 2)
Prize Host Outcome
P (1, 2) = 1
6
P (1, 3) = 1
6
P (2, 3) = 1
3
P (3, 2) = 1
3
Probability
Creator: Malik Magdon-Ismail Probability: 8 / 14 Non-Transitive Dice →
Let’s Make a Deal: The Monty Hall Problem
1: Contestant at door 1.2: Prize placed behind random door.3: Monty opens empty door (randomly if there’s an option).
1 2 3
1∅
3
Outcome-tree and edge-probabilities.
Outcome-probabilities.
Event of interest: “WinBySwitching”.
1
2
3
2
3
3
2
1
3
1
3
1
3
1
2
1
2
1
1
(1, 2)
(1, 3)
(2, 3)
(3, 2)
Prize Host Outcome
P (1, 2) = 1
6
P (1, 3) = 1
6
P (2, 3) = 1
3
P (3, 2) = 1
3
Probability
Creator: Malik Magdon-Ismail Probability: 8 / 14 Non-Transitive Dice →
Let’s Make a Deal: The Monty Hall Problem
1: Contestant at door 1.2: Prize placed behind random door.3: Monty opens empty door (randomly if there’s an option).
1 2 3
1∅
3
Outcome-tree and edge-probabilities.
Outcome-probabilities.
Event of interest: “WinBySwitching”.
Event probability.
1
2
3
2
3
3
2
1
3
1
3
1
3
1
2
1
2
1
1
(1, 2)
(1, 3)
(2, 3)
(3, 2)
Prize Host Outcome
P (1, 2) = 1
6
P (1, 3) = 1
6
P (2, 3) = 1
3
P (3, 2) = 1
3
Probability
1
3+ 1
3= 2
3= P[“WinBySwitching”]
Creator: Malik Magdon-Ismail Probability: 8 / 14 Non-Transitive Dice →
Non-Transitive Dice
A:
B:
C:
Your friend picks a die; you pick a die.e.g. friend picks die B; you pick A.
Creator: Malik Magdon-Ismail Probability: 9 / 14 Probability and Sets →
Non-Transitive Dice
A:
B:
C:
Your friend picks a die; you pick a die.e.g. friend picks die B; you pick A.
What is the probability that A beats B?
Creator: Malik Magdon-Ismail Probability: 9 / 14 Probability and Sets →
Non-Transitive Dice
A:
B:
C:
Your friend picks a die; you pick a die.e.g. friend picks die B; you pick A.
What is the probability that A beats B?
Outcome-tree and outcome-probabilities.
Die A
1
3
1
3
1
3
Creator: Malik Magdon-Ismail Probability: 9 / 14 Probability and Sets →
Non-Transitive Dice
A:
B:
C:
Your friend picks a die; you pick a die.e.g. friend picks die B; you pick A.
What is the probability that A beats B?
Outcome-tree and outcome-probabilities.
Die A Die B
1
3
1
3
1
3
1
3
1
3
1
3
1
3
1
3
1
3
1
3
1
3
1
3
Creator: Malik Magdon-Ismail Probability: 9 / 14 Probability and Sets →
Non-Transitive Dice
A:
B:
C:
Your friend picks a die; you pick a die.e.g. friend picks die B; you pick A.
What is the probability that A beats B?
Outcome-tree and outcome-probabilities.
Uniform probabilities.
P ( ) = 19
P ( ) = 19
P ( ) = 19
P ( ) = 19
P ( ) = 19
P ( ) = 19
P ( ) = 19
P ( ) = 19
P ( ) = 19
Die A Die B Probability
1
3
1
3
1
3
1
3
1
3
1
3
1
3
1
3
1
3
1
3
1
3
1
3
P ( ) = 19
P ( ) = 19
P ( ) = 19
P ( ) = 19
P ( ) = 19
Creator: Malik Magdon-Ismail Probability: 9 / 14 Probability and Sets →
Non-Transitive Dice
A:
B:
C:
Your friend picks a die; you pick a die.e.g. friend picks die B; you pick A.
What is the probability that A beats B?
Outcome-tree and outcome-probabilities.
Uniform probabilities.
Even of interest: outcomes where you win.
P ( ) = 19
P ( ) = 19
P ( ) = 19
P ( ) = 19
P ( ) = 19
P ( ) = 19
P ( ) = 19
P ( ) = 19
P ( ) = 19
Die A Die B Probability
1
3
1
3
1
3
1
3
1
3
1
3
1
3
1
3
1
3
1
3
1
3
1
3
P ( ) = 19
P ( ) = 19
P ( ) = 19
P ( ) = 19
P ( ) = 19
Creator: Malik Magdon-Ismail Probability: 9 / 14 Probability and Sets →
Non-Transitive Dice
A:
B:
C:
Your friend picks a die; you pick a die.e.g. friend picks die B; you pick A.
What is the probability that A beats B?
Outcome-tree and outcome-probabilities.
Uniform probabilities.
Even of interest: outcomes where you win.
Number of outcomes where you win: 5.
Probability you win, P[A beats B] = 5
9.
P ( ) = 19
P ( ) = 19
P ( ) = 19
P ( ) = 19
P ( ) = 19
P ( ) = 19
P ( ) = 19
P ( ) = 19
P ( ) = 19
Die A Die B Probability
1
3
1
3
1
3
1
3
1
3
1
3
1
3
1
3
1
3
1
3
1
3
1
3
P ( ) = 19
P ( ) = 19
P ( ) = 19
P ( ) = 19
P ( ) = 19
Creator: Malik Magdon-Ismail Probability: 9 / 14 Probability and Sets →
Non-Transitive Dice
A:
B:
C:
Your friend picks a die; you pick a die.e.g. friend picks die B; you pick A.
What is the probability that A beats B?
Outcome-tree and outcome-probabilities.
Uniform probabilities.
Even of interest: outcomes where you win.
Number of outcomes where you win: 5.
Probability you win, P[A beats B] = 5
9.
Conclusion: Die A beats Die B.
Pop Quiz. Compute P[B beats C] and P[C beats A].Hence show A beats B, B beats C and C beats A.
P ( ) = 19
P ( ) = 19
P ( ) = 19
P ( ) = 19
P ( ) = 19
P ( ) = 19
P ( ) = 19
P ( ) = 19
P ( ) = 19
Die A Die B Probability
1
3
1
3
1
3
1
3
1
3
1
3
1
3
1
3
1
3
1
3
1
3
1
3
P ( ) = 19
P ( ) = 19
P ( ) = 19
P ( ) = 19
P ( ) = 19
Creator: Malik Magdon-Ismail Probability: 9 / 14 Probability and Sets →
Probability and Sets: The Probability Space
1 Sample Space Ω = ω1, ω2, . . ., set of possible outcomes.2 Probability Function P (·). Non-negative function P (ω), normalized to 1:
0 ≤ P (ω) ≤ 1 and∑
ω∈ΩP (ω) = 1.
Die A
versus B
Ω , , , , , , , ,
P (ω) 1
9
1
9
1
9
1
9
1
9
1
9
1
9
1
9
1
9
Creator: Malik Magdon-Ismail Probability: 10 / 14 Uniform Probability Space →
Probability and Sets: The Probability Space
1 Sample Space Ω = ω1, ω2, . . ., set of possible outcomes.2 Probability Function P (·). Non-negative function P (ω), normalized to 1:
0 ≤ P (ω) ≤ 1 and∑
ω∈ΩP (ω) = 1.
Die A
versus B
Ω , , , , , , , ,
P (ω) 1
9
1
9
1
9
1
9
1
9
1
9
1
9
1
9
1
9
Events E ⊆ Ω are subsets. Event probability P[E ] is the sum of outcome-probabilities.
“A > B” E1 = , , , ,
“Sum > 8” E2 = , , , ,
“B < 9” E3 = , , , , ,
Creator: Malik Magdon-Ismail Probability: 10 / 14 Uniform Probability Space →
Probability and Sets: The Probability Space
1 Sample Space Ω = ω1, ω2, . . ., set of possible outcomes.2 Probability Function P (·). Non-negative function P (ω), normalized to 1:
0 ≤ P (ω) ≤ 1 and∑
ω∈ΩP (ω) = 1.
Die A
versus B
Ω , , , , , , , ,
P (ω) 1
9
1
9
1
9
1
9
1
9
1
9
1
9
1
9
1
9
Events E ⊆ Ω are subsets. Event probability P[E ] is the sum of outcome-probabilities.
“A > B” E1 = , , , ,
“Sum > 8” E2 = , , , ,
“B < 9” E3 = , , , , ,
Combining events using logical connectors corresponds to set operations:
Creator: Malik Magdon-Ismail Probability: 10 / 14 Uniform Probability Space →
Probability and Sets: The Probability Space
1 Sample Space Ω = ω1, ω2, . . ., set of possible outcomes.2 Probability Function P (·). Non-negative function P (ω), normalized to 1:
0 ≤ P (ω) ≤ 1 and∑
ω∈ΩP (ω) = 1.
Die A
versus B
Ω , , , , , , , ,
P (ω) 1
9
1
9
1
9
1
9
1
9
1
9
1
9
1
9
1
9
Events E ⊆ Ω are subsets. Event probability P[E ] is the sum of outcome-probabilities.
“A > B” E1 = , , , ,
“Sum > 8” E2 = , , , ,
“B < 9” E3 = , , , , ,
Combining events using logical connectors corresponds to set operations:“A > B” ∨ “Sum > 8” E1 ∪ E2 = , , , , , , ,
“A > B” ∧ “Sum > 8” E1 ∩ E2 = ,
¬(“A > B”) E1 = , , ,
“A > B”→ “B < 9” E1 ⊆ E3
Important: Exercise 15.9. Sum rule, complement, inclusion-exclusion, union, implication and intersection bounds.
Creator: Malik Magdon-Ismail Probability: 10 / 14 Uniform Probability Space →
Uniform Probability Space : Probability ∼ Size
P (ω) =1
|Ω|
Creator: Malik Magdon-Ismail Probability: 11 / 14 Poker Probabilities →
Uniform Probability Space : Probability ∼ Size
P (ω) =1
|Ω|P [E ] =
|E|
|Ω|=
number of outcomes in E
number of possible outcomes in Ω.
Creator: Malik Magdon-Ismail Probability: 11 / 14 Poker Probabilities →
Uniform Probability Space : Probability ∼ Size
P (ω) =1
|Ω|P [E ] =
|E|
|Ω|=
number of outcomes in E
number of possible outcomes in Ω.
Toss a coin 3 times:
H T H T H T H T
H T H T
H T
Toss 3
Toss 2
Toss 1
Outcome
Probability
HHH
18
HHT
18
HTH
18
HTT
18
THH
18
THT
18
TTH
18
TTT
18
1
2
1
2
1
2
1
2
1
2
1
2
1
2
1
2
1
2
1
2
1
2
1
2
1
2
1
2
Creator: Malik Magdon-Ismail Probability: 11 / 14 Poker Probabilities →
Uniform Probability Space : Probability ∼ Size
P (ω) =1
|Ω|P [E ] =
|E|
|Ω|=
number of outcomes in E
number of possible outcomes in Ω.
Toss a coin 3 times:
H T H T H T H T
H T H T
H T
Toss 3
Toss 2
Toss 1
Outcome
Probability
HHH
18
HHT
18
HTH
18
HTT
18
THH
18
THT
18
TTH
18
TTT
18
1
2
1
2
1
2
1
2
1
2
1
2
1
2
1
2
1
2
1
2
1
2
1
2
1
2
1
2
P[“2 heads”] =number of sequences with 2 heads
number of possible sequences in Ω=
3
2
×1
8=
3
8.
Creator: Malik Magdon-Ismail Probability: 11 / 14 Poker Probabilities →
Uniform Probability Space : Probability ∼ Size
P (ω) =1
|Ω|P [E ] =
|E|
|Ω|=
number of outcomes in E
number of possible outcomes in Ω.
Toss a coin 3 times:
H T H T H T H T
H T H T
H T
Toss 3
Toss 2
Toss 1
Outcome
Probability
HHH
18
HHT
18
HTH
18
HTT
18
THH
18
THT
18
TTH
18
TTT
18
1
2
1
2
1
2
1
2
1
2
1
2
1
2
1
2
1
2
1
2
1
2
1
2
1
2
1
2
P[“2 heads”] =number of sequences with 2 heads
number of possible sequences in Ω=
3
2
×1
8=
3
8.
Practice: Exercise 15.10.
1 You roll a pair of regular dice. What is the probability that the sum is 9?
2 You toss a fair coin ten times. What is the probability that you obtain 4 heads?
3 You roll die A ten times. Compute probabilities for: 4 sevens? 4 sevens and 3 sixes? 4 sevens or 3 sixes?
Creator: Malik Magdon-Ismail Probability: 11 / 14 Poker Probabilities →
Poker: Probabilities of Full House and Flush
52 card deck has 4 suits (♠, ♥, ♦, ♣) and 13 ranks in a suit (A,K,Q,J,T,9,8,7,6,5,4,3,2).
Randomly deal 5-cards: each set of 5 cards is equally likely → uniform probability space.
number of possible outcomes =
52
5
possible hands.
Creator: Malik Magdon-Ismail Probability: 12 / 14 Infinite Probability Space →
Poker: Probabilities of Full House and Flush
52 card deck has 4 suits (♠, ♥, ♦, ♣) and 13 ranks in a suit (A,K,Q,J,T,9,8,7,6,5,4,3,2).
Randomly deal 5-cards: each set of 5 cards is equally likely → uniform probability space.
number of possible outcomes =
52
5
possible hands.
Full house: 3 cards of one rank and 2 of another. How many full-houses?
Creator: Malik Magdon-Ismail Probability: 12 / 14 Infinite Probability Space →
Poker: Probabilities of Full House and Flush
52 card deck has 4 suits (♠, ♥, ♦, ♣) and 13 ranks in a suit (A,K,Q,J,T,9,8,7,6,5,4,3,2).
Randomly deal 5-cards: each set of 5 cards is equally likely → uniform probability space.
number of possible outcomes =
52
5
possible hands.
Full house: 3 cards of one rank and 2 of another. How many full-houses?To construct a full house, specify (rank3, suits3, rank2, suits2). Product rule:
Creator: Malik Magdon-Ismail Probability: 12 / 14 Infinite Probability Space →
Poker: Probabilities of Full House and Flush
52 card deck has 4 suits (♠, ♥, ♦, ♣) and 13 ranks in a suit (A,K,Q,J,T,9,8,7,6,5,4,3,2).
Randomly deal 5-cards: each set of 5 cards is equally likely → uniform probability space.
number of possible outcomes =
52
5
possible hands.
Full house: 3 cards of one rank and 2 of another. How many full-houses?To construct a full house, specify (rank3, suits3, rank2, suits2). Product rule:
# full houses = 13×
4
3
×12×
4
2
Creator: Malik Magdon-Ismail Probability: 12 / 14 Infinite Probability Space →
Poker: Probabilities of Full House and Flush
52 card deck has 4 suits (♠, ♥, ♦, ♣) and 13 ranks in a suit (A,K,Q,J,T,9,8,7,6,5,4,3,2).
Randomly deal 5-cards: each set of 5 cards is equally likely → uniform probability space.
number of possible outcomes =
52
5
possible hands.
Full house: 3 cards of one rank and 2 of another. How many full-houses?To construct a full house, specify (rank3, suits3, rank2, suits2). Product rule:
# full houses = 13×
4
3
×12×
4
2
→ P[“FullHouse”] =13×
(
4
3
)
× 12×(
4
2
)
(
52
5
) ≈ 0.00144;
Creator: Malik Magdon-Ismail Probability: 12 / 14 Infinite Probability Space →
Poker: Probabilities of Full House and Flush
52 card deck has 4 suits (♠, ♥, ♦, ♣) and 13 ranks in a suit (A,K,Q,J,T,9,8,7,6,5,4,3,2).
Randomly deal 5-cards: each set of 5 cards is equally likely → uniform probability space.
number of possible outcomes =
52
5
possible hands.
Full house: 3 cards of one rank and 2 of another. How many full-houses?To construct a full house, specify (rank3, suits3, rank2, suits2). Product rule:
# full houses = 13×
4
3
×12×
4
2
→ P[“FullHouse”] =13×
(
4
3
)
× 12×(
4
2
)
(
52
5
) ≈ 0.00144;
Flush: 5 cards of same suit. How many flushes?
Creator: Malik Magdon-Ismail Probability: 12 / 14 Infinite Probability Space →
Poker: Probabilities of Full House and Flush
52 card deck has 4 suits (♠, ♥, ♦, ♣) and 13 ranks in a suit (A,K,Q,J,T,9,8,7,6,5,4,3,2).
Randomly deal 5-cards: each set of 5 cards is equally likely → uniform probability space.
number of possible outcomes =
52
5
possible hands.
Full house: 3 cards of one rank and 2 of another. How many full-houses?To construct a full house, specify (rank3, suits3, rank2, suits2). Product rule:
# full houses = 13×
4
3
×12×
4
2
→ P[“FullHouse”] =13×
(
4
3
)
× 12×(
4
2
)
(
52
5
) ≈ 0.00144;
Flush: 5 cards of same suit. How many flushes?To construct a flush, specify (suit, ranks). Product rule:
Creator: Malik Magdon-Ismail Probability: 12 / 14 Infinite Probability Space →
Poker: Probabilities of Full House and Flush
52 card deck has 4 suits (♠, ♥, ♦, ♣) and 13 ranks in a suit (A,K,Q,J,T,9,8,7,6,5,4,3,2).
Randomly deal 5-cards: each set of 5 cards is equally likely → uniform probability space.
number of possible outcomes =
52
5
possible hands.
Full house: 3 cards of one rank and 2 of another. How many full-houses?To construct a full house, specify (rank3, suits3, rank2, suits2). Product rule:
# full houses = 13×
4
3
×12×
4
2
→ P[“FullHouse”] =13×
(
4
3
)
× 12×(
4
2
)
(
52
5
) ≈ 0.00144;
Flush: 5 cards of same suit. How many flushes?To construct a flush, specify (suit, ranks). Product rule:
# flushes = 4×
13
5
Creator: Malik Magdon-Ismail Probability: 12 / 14 Infinite Probability Space →
Poker: Probabilities of Full House and Flush
52 card deck has 4 suits (♠, ♥, ♦, ♣) and 13 ranks in a suit (A,K,Q,J,T,9,8,7,6,5,4,3,2).
Randomly deal 5-cards: each set of 5 cards is equally likely → uniform probability space.
number of possible outcomes =
52
5
possible hands.
Full house: 3 cards of one rank and 2 of another. How many full-houses?To construct a full house, specify (rank3, suits3, rank2, suits2). Product rule:
# full houses = 13×
4
3
×12×
4
2
→ P[“FullHouse”] =13×
(
4
3
)
× 12×(
4
2
)
(
52
5
) ≈ 0.00144;
Flush: 5 cards of same suit. How many flushes?To construct a flush, specify (suit, ranks). Product rule:
# flushes = 4×
13
5
→ P[“Flush”] =4×
(
13
5
)
(
52
5
) ≈ 0.00198;
Full house is rarer. That’s why full house beats flush.
Creator: Malik Magdon-Ismail Probability: 12 / 14 Infinite Probability Space →
Toss a Coin Until Heads: Infinite Probability Space
T
H
1
2
1
2
Toss 1
H
12
Creator: Malik Magdon-Ismail Probability: 13 / 14 Game: First Person To Toss H Wins →
Toss a Coin Until Heads: Infinite Probability Space
T
H
T
H
1
2
1
2
Toss 1
1
2
1
2
Toss 2
H
12
TH
14
Creator: Malik Magdon-Ismail Probability: 13 / 14 Game: First Person To Toss H Wins →
Toss a Coin Until Heads: Infinite Probability Space
T
H
T
H
T
H
1
2
1
2
Toss 1
1
2
1
2
Toss 2
1
2
1
2
Toss 3
H
12
TH
14
TTH
18
Creator: Malik Magdon-Ismail Probability: 13 / 14 Game: First Person To Toss H Wins →
Toss a Coin Until Heads: Infinite Probability Space
T
H
T
H
T
H
T
H
1
2
1
2
Toss 1
1
2
1
2
Toss 2
1
2
1
2
Toss 3
1
2
1
2
Toss 4
H
12
TH
14
TTH
18
TTTH
116
Creator: Malik Magdon-Ismail Probability: 13 / 14 Game: First Person To Toss H Wins →
Toss a Coin Until Heads: Infinite Probability Space
T
H
T
H
T
H
T
H
T
H
1
2
1
2
Toss 1
1
2
1
2
Toss 2
1
2
1
2
Toss 3
1
2
1
2
Toss 4
1
2
1
2
Toss 5
H
12
TH
14
TTH
18
TTTH
116
TTTTH
132
Creator: Malik Magdon-Ismail Probability: 13 / 14 Game: First Person To Toss H Wins →
Toss a Coin Until Heads: Infinite Probability Space
T
H
T
H
T
H
T
H
T
H
T
H
1
2
1
2
Toss 1
1
2
1
2
Toss 2
1
2
1
2
Toss 3
1
2
1
2
Toss 4
1
2
1
2
Toss 5
1
2
1
2
Toss 6
H
12
TH
14
TTH
18
TTTH
116
TTTTH
132
TTTTTH
164
· · ·
· · ·
· · ·· · ·
Outcome
Probability
Creator: Malik Magdon-Ismail Probability: 13 / 14 Game: First Person To Toss H Wins →
Toss a Coin Until Heads: Infinite Probability Space
T
H
T
H
T
H
T
H
T
H
T
H
1
2
1
2
Toss 1
1
2
1
2
Toss 2
1
2
1
2
Toss 3
1
2
1
2
Toss 4
1
2
1
2
Toss 5
1
2
1
2
Toss 6
H
12
TH
14
TTH
18
TTTH
116
TTTTH
132
TTTTTH
164
· · ·
· · ·
· · ·· · ·
Outcome
Probability
Ω H TH T•2H T•3H T•4H T•5H · · · T•iH · · ·
P (ω) 1
2(1
2)2 (1
2)3 (1
2)4 (1
2)5 (1
2)6 · · · (1
2)i+1 · · ·
# Tosses 1 2 3 4 5 6 · · · i + 1 · · ·
Creator: Malik Magdon-Ismail Probability: 13 / 14 Game: First Person To Toss H Wins →
Toss a Coin Until Heads: Infinite Probability Space
T
H
T
H
T
H
T
H
T
H
T
H
1
2
1
2
Toss 1
1
2
1
2
Toss 2
1
2
1
2
Toss 3
1
2
1
2
Toss 4
1
2
1
2
Toss 5
1
2
1
2
Toss 6
H
12
TH
14
TTH
18
TTTH
116
TTTTH
132
TTTTTH
164
· · ·
· · ·
· · ·· · ·
Outcome
Probability
Ω H TH T•2H T•3H T•4H T•5H · · · T•iH · · ·
P (ω) 1
2(1
2)2 (1
2)3 (1
2)4 (1
2)5 (1
2)6 · · · (1
2)i+1 · · ·
# Tosses 1 2 3 4 5 6 · · · i + 1 · · ·
Sum of outcome probabilities:
1
2+ (1
2)2 + (1
2)3 + (1
2)4 + · · · =
∞∑
i=1(1
2)i =
1
2
1− 1
2
= 1.
Creator: Malik Magdon-Ismail Probability: 13 / 14 Game: First Person To Toss H Wins →
Game: First Person To Toss H Wins. Always Go First
T
H
T
H
T
H
T
H
T
H
T
H
1
2
1
2
Toss 1
1
2
1
2
Toss 2
1
2
1
2
Toss 3
1
2
1
2
Toss 4
1
2
1
2
Toss 5
1
2
1
2
Toss 6
H
12
TH
14
TTH
18
TTTH
116
TTTTH
132
TTTTTH
164
· · ·
· · ·
· · ·· · ·
Outcome
Probability
Ω H TH T•2H T•3H T•4H T•5H · · · T•iH · · ·
P (ω) 1
2(1
2)2 (1
2)3 (1
2)4 (1
2)5 (1
2)6 · · · (1
2)i+1 · · ·
Creator: Malik Magdon-Ismail Probability: 14 / 14
Game: First Person To Toss H Wins. Always Go First
T
H
T
H
T
H
T
H
T
H
T
H
1
2
1
2
Toss 1
1
2
1
2
Toss 2
1
2
1
2
Toss 3
1
2
1
2
Toss 4
1
2
1
2
Toss 5
1
2
1
2
Toss 6
H
12
TH
14
TTH
18
TTTH
116
TTTTH
132
TTTTTH
164
· · ·
· · ·
· · ·· · ·
Outcome
Probability
Ω H TH T•2H T•3H T•4H T•5H · · · T•iH · · ·
P (ω) 1
2(1
2)2 (1
2)3 (1
2)4 (1
2)5 (1
2)6 · · · (1
2)i+1 · · ·
The event “YouWin” is E = H, T•2H, T•4H, T•6H, . . ..
Creator: Malik Magdon-Ismail Probability: 14 / 14
Game: First Person To Toss H Wins. Always Go First
T
H
T
H
T
H
T
H
T
H
T
H
1
2
1
2
Toss 1
1
2
1
2
Toss 2
1
2
1
2
Toss 3
1
2
1
2
Toss 4
1
2
1
2
Toss 5
1
2
1
2
Toss 6
H
12
TH
14
TTH
18
TTTH
116
TTTTH
132
TTTTTH
164
· · ·
· · ·
· · ·· · ·
Outcome
Probability
Ω H TH T•2H T•3H T•4H T•5H · · · T•iH · · ·
P (ω) 1
2(1
2)2 (1
2)3 (1
2)4 (1
2)5 (1
2)6 · · · (1
2)i+1 · · ·
The event “YouWin” is E = H, T•2H, T•4H, T•6H, . . ..
P[“YouWin”] = 1
2+ (1
2)3 + (1
2)5 + (1
2)7 + · · · = 1
2
∞∑
i=0(1
4)i =
1
2
1− 1
4
= 2
3.
Your odds improve by a factor of 2 if you go first (vs. second).
Creator: Malik Magdon-Ismail Probability: 14 / 14
Today: Conditional Probability
1 New information changes a probability.
2 Definition of conditional probability from regular probability.
3 Conditional probability trapsSampling bias.
Transposed conditional.
4 Law of total probability.Probabilistic case-by-case analysis.
Creator: Malik Magdon-Ismail Conditional Probability: 3 / 16 Flu Season →
Flu Season
1 Chances a random person has the flu is about 0.01 (or 1%) (prior probability).
Probability of flu : P[flu] ≈ 0.01.
Creator: Malik Magdon-Ismail Conditional Probability: 4 / 16 CS, MATH and Dual CS-MATH Majors →
Flu Season
1 Chances a random person has the flu is about 0.01 (or 1%) (prior probability).
Probability of flu : P[flu] ≈ 0.01.
2 You have a slight fever – new information. Chances of flu “increase”.
Probability of flu given fever : P[flu | fever] ≈ 0.4.
New information changes the prior probability to the posterior probability. Translate posterior as “After you get the new information.”
P[A | B] is the (updated) conditional probability of A, given the new information B.
Creator: Malik Magdon-Ismail Conditional Probability: 4 / 16 CS, MATH and Dual CS-MATH Majors →
Flu Season
1 Chances a random person has the flu is about 0.01 (or 1%) (prior probability).
Probability of flu : P[flu] ≈ 0.01.
2 You have a slight fever – new information. Chances of flu “increase”.
Probability of flu given fever : P[flu | fever] ≈ 0.4.
New information changes the prior probability to the posterior probability. Translate posterior as “After you get the new information.”
P[A | B] is the (updated) conditional probability of A, given the new information B.
3 Roommie has flu (more new information). Flu for sure, take counter-measures.
Probability of flu given fever and roommie flu : P[flu | fever and roommie flu] ≈ 1.
Creator: Malik Magdon-Ismail Conditional Probability: 4 / 16 CS, MATH and Dual CS-MATH Majors →
CS, MATH and Dual CS-MATH Majors
5,000 students: 1,000 CS; 100 MATH; 80 dual MATH-CS.
ALL
CS
MATH
Creator: Malik Magdon-Ismail Conditional Probability: 5 / 16 Conditional Probability P[A | B] →
CS, MATH and Dual CS-MATH Majors
5,000 students: 1,000 CS; 100 MATH; 80 dual MATH-CS.
Pick a random student:
P[CS] = 10005000
= 0.2;
P[MATH] = 1005000
= 0.02;
P[CS and MATH] = 805000
= 0.016.
ALL
CS
MATH
Creator: Malik Magdon-Ismail Conditional Probability: 5 / 16 Conditional Probability P[A | B] →
CS, MATH and Dual CS-MATH Majors
5,000 students: 1,000 CS; 100 MATH; 80 dual MATH-CS.
Pick a random student:
P[CS] = 10005000
= 0.2;
P[MATH] = 1005000
= 0.02;
P[CS and MATH] = 805000
= 0.016.
New information: student is MATH. What is P[CS |MATH]?Effectively picking a random student from MATH.
ALL
CS
MATH
Creator: Malik Magdon-Ismail Conditional Probability: 5 / 16 Conditional Probability P[A | B] →
CS, MATH and Dual CS-MATH Majors
5,000 students: 1,000 CS; 100 MATH; 80 dual MATH-CS.
Pick a random student:
P[CS] = 10005000
= 0.2;
P[MATH] = 1005000
= 0.02;
P[CS and MATH] = 805000
= 0.016.
New information: student is MATH. What is P[CS |MATH]?Effectively picking a random student from MATH.
New probability of CS ∼ striped area |CS ∩ MATH|.
ALL
CS
MATH
Creator: Malik Magdon-Ismail Conditional Probability: 5 / 16 Conditional Probability P[A | B] →
CS, MATH and Dual CS-MATH Majors
5,000 students: 1,000 CS; 100 MATH; 80 dual MATH-CS.
Pick a random student:
P[CS] = 10005000
= 0.2;
P[MATH] = 1005000
= 0.02;
P[CS and MATH] = 805000
= 0.016.
New information: student is MATH. What is P[CS |MATH]?Effectively picking a random student from MATH.
New probability of CS ∼ striped area |CS ∩ MATH|.
ALL
CS
MATH
P[CS | MATH] =|CS ∩ MATH|
|MATH|=
80
100= 0.8.
MATH students are 4 times more likely to be CS majors than a random student.
Pop Quiz. What is P[MATH | CS]? What is P[CS | CS or MATH]?
Creator: Malik Magdon-Ismail Conditional Probability: 5 / 16 Conditional Probability P[A | B] →
Conditional Probability P[A | B]
P[A | B] = frequency of outcomes known to be in B that are also in A.
Creator: Malik Magdon-Ismail Conditional Probability: 6 / 16 Chances of Rain →
Conditional Probability P[A | B]
P[A | B] = frequency of outcomes known to be in B that are also in A.
nB ooutcomes in event B when you repeat an experiment n times.
P[B] =nB
n.
Creator: Malik Magdon-Ismail Conditional Probability: 6 / 16 Chances of Rain →
Conditional Probability P[A | B]
P[A | B] = frequency of outcomes known to be in B that are also in A.
nB ooutcomes in event B when you repeat an experiment n times.
P[B] =nB
n.
Of the nB outcomes in B, the number also in A is nA∩B,
P[A ∩ B] =nA∩B
n.
Creator: Malik Magdon-Ismail Conditional Probability: 6 / 16 Chances of Rain →
Conditional Probability P[A | B]
P[A | B] = frequency of outcomes known to be in B that are also in A.
nB ooutcomes in event B when you repeat an experiment n times.
P[B] =nB
n.
Of the nB outcomes in B, the number also in A is nA∩B,
P[A ∩ B] =nA∩B
n.
The frequency of outcomes in A among those outcomes in B is nA∩B/nB,
P[A | B] =nA∩B
nB
=nA∩B
n×
n
nB
=P[A ∩ B]
P[B].
Creator: Malik Magdon-Ismail Conditional Probability: 6 / 16 Chances of Rain →
Conditional Probability P[A | B]
P[A | B] = frequency of outcomes known to be in B that are also in A.
nB ooutcomes in event B when you repeat an experiment n times.
P[B] =nB
n.
Of the nB outcomes in B, the number also in A is nA∩B,
P[A ∩ B] =nA∩B
n.
The frequency of outcomes in A among those outcomes in B is nA∩B/nB,
P[A | B] =nA∩B
nB
=nA∩B
n×
n
nB
=P[A ∩ B]
P[B].
P[A | B] =nA∩B
nB
=P[A ∩ B]
P[B]=
P[A and B]
P[B]
Creator: Malik Magdon-Ismail Conditional Probability: 6 / 16 Chances of Rain →
Chances of Rain Given Clouds
It is cloudy one in five days, P[Clouds] = 15. It rains one in seven days, P[Rain] = 1
7.
Creator: Malik Magdon-Ismail Conditional Probability: 7 / 16 Conditioning with Dice →
Chances of Rain Given Clouds
It is cloudy one in five days, P[Clouds] = 15. It rains one in seven days, P[Rain] = 1
7.
What are the chances of rain on a cloudy day?
P[Rain | Clouds] =P[Rain ∩ Clouds]
P[Clouds].
Creator: Malik Magdon-Ismail Conditional Probability: 7 / 16 Conditioning with Dice →
Chances of Rain Given Clouds
It is cloudy one in five days, P[Clouds] = 15. It rains one in seven days, P[Rain] = 1
7.
What are the chances of rain on a cloudy day?
P[Rain | Clouds] =P[Rain ∩ Clouds]
P[Clouds].
Rainy Days ⊆ Cloudy Days → P[Rain ∩ Clouds] = P[Rain].
All Days
Cloudy
Rainy
Creator: Malik Magdon-Ismail Conditional Probability: 7 / 16 Conditioning with Dice →
Chances of Rain Given Clouds
It is cloudy one in five days, P[Clouds] = 15. It rains one in seven days, P[Rain] = 1
7.
What are the chances of rain on a cloudy day?
P[Rain | Clouds] =P[Rain ∩ Clouds]
P[Clouds].
Rainy Days ⊆ Cloudy Days → P[Rain ∩ Clouds] = P[Rain].
P[Rain | Clouds] =P[Rain]
P[Clouds]=
1715
=5
7.
All Days
Cloudy
Rainy
5-times more likely to rain on a cloudy day than on a random day.
Crucial first step: identify the conditional probability. What is the “new information”?
Creator: Malik Magdon-Ismail Conditional Probability: 7 / 16 Conditioning with Dice →
P[Sum of 2 Dice is 10 | Both are Odd]
Two dice have both rolled odd. What are the chances the sum is 10?
P[Sum is 10 | Both are Odd] =P[(Sum is 10) and (Both are Odd)]
P[Both are Odd]
Creator: Malik Magdon-Ismail Conditional Probability: 8 / 16 Computing a Conditional Probability →
P[Sum of 2 Dice is 10 | Both are Odd]
Two dice have both rolled odd. What are the chances the sum is 10?
P[Sum is 10 | Both are Odd] =P[(Sum is 10) and (Both are Odd)]
P[Both are Odd]
Probability Space
Die 1 Value
Die
2V
alue
Creator: Malik Magdon-Ismail Conditional Probability: 8 / 16 Computing a Conditional Probability →
P[Sum of 2 Dice is 10 | Both are Odd]
Two dice have both rolled odd. What are the chances the sum is 10?
P[Sum is 10 | Both are Odd] =P[(Sum is 10) and (Both are Odd)]
P[Both are Odd]
136
136
136
136
136
136
136
136
136
136
136
136
136
136
136
136
136
136
136
136
136
136
136
136
136
136
136
136
136
136
136
136
136
136
136
136
Probability Space
Die 1 Value
Die
2V
alue
Creator: Malik Magdon-Ismail Conditional Probability: 8 / 16 Computing a Conditional Probability →
P[Sum of 2 Dice is 10 | Both are Odd]
Two dice have both rolled odd. What are the chances the sum is 10?
P[Sum is 10 | Both are Odd] =P[(Sum is 10) and (Both are Odd)]
P[Both are Odd]
136
136
136
136
136
136
136
136
136
136
136
136
136
136
136
136
136
136
136
136
136
136
136
136
136
136
136
136
136
136
136
136
136
136
136
136
Probability Space
Die 1 Value
Die
2V
alue
1 P[Sum is 10] =3
36=
1
12.
Creator: Malik Magdon-Ismail Conditional Probability: 8 / 16 Computing a Conditional Probability →
P[Sum of 2 Dice is 10 | Both are Odd]
Two dice have both rolled odd. What are the chances the sum is 10?
P[Sum is 10 | Both are Odd] =P[(Sum is 10) and (Both are Odd)]
P[Both are Odd]
136
136
136
136
136
136
136
136
136
136
136
136
136
136
136
136
136
136
136
136
136
136
136
136
136
136
136
136
136
136
136
136
136
136
136
136
Probability Space
Die 1 Value
Die
2V
alue
1 P[Sum is 10] =3
36=
1
12.
2 P[Both are Odd] =9
36=
1
4.
Creator: Malik Magdon-Ismail Conditional Probability: 8 / 16 Computing a Conditional Probability →
P[Sum of 2 Dice is 10 | Both are Odd]
Two dice have both rolled odd. What are the chances the sum is 10?
P[Sum is 10 | Both are Odd] =P[(Sum is 10) and (Both are Odd)]
P[Both are Odd]
136
136
136
136
136
136
136
136
136
136
136
136
136
136
136
136
136
136
136
136
136
136
136
136
136
136
136
136
136
136
136
136
136
136
136
136
Probability Space
Die 1 Value
Die
2V
alue
1 P[Sum is 10] =3
36=
1
12.
2 P[Both are Odd] =9
36=
1
4.
3 P[(Sum is 10) and (Both are Odd)] =1
36.
Creator: Malik Magdon-Ismail Conditional Probability: 8 / 16 Computing a Conditional Probability →
P[Sum of 2 Dice is 10 | Both are Odd]
Two dice have both rolled odd. What are the chances the sum is 10?
P[Sum is 10 | Both are Odd] =P[(Sum is 10) and (Both are Odd)]
P[Both are Odd]
136
136
136
136
136
136
136
136
136
136
136
136
136
136
136
136
136
136
136
136
136
136
136
136
136
136
136
136
136
136
136
136
136
136
136
136
Probability Space
Die 1 Value
Die
2V
alue
1 P[Sum is 10] =3
36=
1
12.
2 P[Both are Odd] =9
36=
1
4.
3 P[(Sum is 10) and (Both are Odd)] =1
36.
4 P[Sum is 10 | Both are Odd] =1
36÷
1
4=
1
9.
Pop Quiz. Compute P[Both are Odd | Sum is 10]. Compare with P[Sum is 10 | Both are Odd].
Creator: Malik Magdon-Ismail Conditional Probability: 8 / 16 Computing a Conditional Probability →
Computing a Conditional Probability
1: Identify that you need a conditional probability P[A | B].
2: Determine the probability space (Ω, P (·)) using the outcome-tree method.
3: Identify the events A and B appearing in P[A | B] as subsets of Ω.
4: Compute P[A ∩ B] and P[B].
5: Compute P[A | B] =P[A ∩ B]
P[B].
Creator: Malik Magdon-Ismail Conditional Probability: 9 / 16 Monty Prefers Door 3 →