engg2012b lecture 16 conditional probability kenneth shum kshum1engg2012b
TRANSCRIPT
ENGG2012BLecture 16
Conditional probability
Kenneth Shum
kshum 1ENGG2012B
Midterm
• 22nd Mar• One and a half hour.• Bring calculator and blank papers.• Close-book and close-note exam.• Coverage:
– Lecture 1 to 15.– Tutorial 1 to 7.– Homework 1 to 3.
kshum ENGG2012B 2
Derangements of n=4 objects
• Let be the set of all 24 permutations of 1,2,3,4.• For i = 1,2,3,4, let Ai be the set of permutations
which fix i.• A1 = {1234, 1243, 1324, 1342, 1423, 1432}.• A2 = {1234, 1243, 3214, 3241, 4213, 4231}.• A3 = {1234, 1432, 2134, 2431, 4132, 4231}.• A4 = {1234, 1324, 2134, 2314, 3124, 3214}.• A permutation of {1,2,3,4} not in A1 to A4 has no
fixed point, and is called a derangement.kshum ENGG2012B 3
Venn diagram
kshum ENGG2012B 4
2143 2341 2413 3142 3412 3421 4123 4312 4321
A2
A3
A4
1234
1243
2134
14324231
32141324
13421423
32414213
2431
4132
2314
3124
The case n=4 (cont’d)• A1 A2 = {1234, 1243}.• A1 A3 = {1234, 1432}.• A1 A4 = {1234, 1324}.• A2 A3 = {1234, 4231}.• A2 A4 = {1234, 3214}.• A3 A4 = {1234, 2134}.• A1 A2 A3 = A1 A2 A4 = A1 A3 A4 =A2 A3 A4
= A1 A2 A3 A4 ={1,2,3,4}.• By PIE, the number of derangements for n=4 is equal to
4! – 46 + 62 – 41 + 1 = 9.• Indeed, the nine derangements for n=4 are
2143, 2341, 2413, 3142, 3412, 3421, 4123, 4312, 4321.kshum ENGG2012B 5
Probability of no fixed number• For n=4, if we pick a permutation randomly,
– the probability of no fixed number is 9/24= 0.375.– the probability of at least one fixed number is 15/24= 0.625.
• For large n, it can be shown that the probability of no fixed number is approximately 1/e = 0.3679…
kshum ENGG2012B 6
n Probability that a random permutation of length n is a derangement
3 0.3333
4 0.3750
5 0.3667
6 0.3681
7 0.3679
John Venn (1834-1923)
• British logician and philosopher• http://en.wikipedia.org/wiki/John_Venn
kshum ENGG2012B 7
Classical definition of probability
• From Laplace’s Théorie analytique des probabilités (1812)– “The probability of an event is
the ratio of the number of cases favourable to it, to the number of all possible cases.”
kshum ENGG2012B 8
http://en.wikipedia.org/w
iki/Pierre-
Sim
on_Laplace
BASIC PROBABILITY
kshum ENGG2012B 9
Disjoint events
• Two events are called disjoint if the intersection is empty, i.e., if they have no overlap.
• The calculation of union of events in general is complicated when the events overlap.
• It is much easier if we want to compute the probability of a union of disjoint event. We simply add the probability of the events
kshum ENGG2012B 10
Union of disjoint events
kshum ENGG2012B 11
A
B
C
Pr(A B C) = Pr(A) + Pr(B) + Pr(C).A, B, C mutually disjoint
Partition of sample space
kshum ENGG2012B 12
A
B
C
D
Pr(A) + Pr(B) + Pr(C) + Pr(D)= Pr(A B C D) = 1
A B C D = , A, B, C, D mutually disjoint
Law of total probability
kshum ENGG2012B 13
A
B
C
D
Pr(E) = Pr(EA)+ Pr(EB)+ Pr(EC)+ Pr(ED).
A B C D = , A, B, C, D mutually disjoint
E
CONDITIONAL PROBABILITY
kshum ENGG2012B 14
Example• There are four cards, two of them are red
and two of them are black.• Shuffle the four cards and lay out the cards
on the table with the front of each card facing downwards.
• What is the probability that the first card is red?
• We reveal the last card. – If it turns out that the last card is red, what is
the probability that the first card is red?– If it turns out that the last card is black, what
is the probability that the first card is red?
kshum ENGG2012B 15
Conditional probability
• If we are given an addition information that the outcome is in event B, then we can update the likelihood to |AB|/ |B|.
• If Pr(B) is nonzero, then we define the conditional probability of A given B by
Pr(A|B) = Pr(AB)/Pr(B).
kshum ENGG2012B 16
A
B
At the beginning, the probability of event A is |A| / ||, assuming that all outcomes in are equally likely.
The law of total probability in terms of conditional probability
kshum ENGG2012B 17
A
B
C
D
Pr(E) = Pr(EA)+ Pr(EB)+ Pr(EC)+ Pr(ED)
= Pr(E|A)Pr(A)+ Pr(E|B)Pr(B)+ Pr(E|C)Pr(C)+ Pr(E|D)Pr(D).
A B C D = , A, B, C, D mutually disjoint
E
Example
• Alice rolls a fair dice two times. • Suppose Bob is told that the face value of the
first roll is 6. What is the probability the second roll is also 6?
• Suppose Carol is told that the face value of one of the roll is 6. What is the probability that the other roll is also 6?
kshum ENGG2012B 18
Independent events
• If P(A|B) = P(A), then it means that the likelihood of the event A does not change after we are told that event B has occurred. In this case, event A is said to be independent of event B.
• As Pr(A|B) = Pr(AB)/Pr(B), event A is independent of B if Pr(AB) = Pr(A) Pr(B).
• Two events A and B are said to be independent, or statistically independent, if
Pr(AB) = Pr(A) Pr(B).
kshum ENGG2012B 19
Example• Roll two fair dice. • Let A be the event that the face value of the first die is
even. • Let B be the event that the sum of the dice is odd.• Are events A and B independent?
kshum ENGG2012B 20
11 12 13 14 15 16
21 22 23 24 25 26
31 32 33 34 35 36
41 42 43 44 45 46
51 52 53 54 55 56
61 62 63 64 65 66
11 12 13 14 15 16
21 22 23 24 25 26
31 32 33 34 35 36
41 42 43 44 45 46
51 52 53 54 55 56
61 62 63 64 65 66
First numberis even
Sum is odd
Pairwise independent does not imply Pr(ABC)=Pr(A)Pr(B)Pr(C).
• Let be the sample space {0,1,2,3}. The four outcomes are equally likely.
• Let A be the event {0,1}, B be the event {0,2}, and C be the event {0,3}.
• The three events A, B and C are pairwise independent, because– Pr(A) Pr(B) = (1/2)2 = 1/4 = Pr(AB)– Pr(B) Pr(C) = (1/2)2 = 1/4 = Pr(BC)– Pr(C) Pr(A) = (1/2)2 = 1/4 = Pr(CA)
• However, – Pr(ABC) = 1/4– Pr(A) Pr(B) Pr(C) = (1/2)3 =1/8
kshum ENGG2012B 21
1
23
0
A
BC
Example• There are two boxes. • The first box contains 3 red balls and 7 blue balls.• The second box contains 16 red balls and 4 blue balls.• Perform the following random experiment
– Throw a fair die.– If the number is 1, pick a ball randomly from the first box.– If the number is 2 to 5, pick a ball randomly from the second
box.• What is the probability that a red ball is drawn?• What is the probability that a blue ball is drawn?• Given that a red ball is picked, what is the probability that
the first box is chosen?
kshum ENGG2012B 22
The Bayes’ rule
• Let A and B be two events. Suppose that the probability of B is nonzero.
• Probability of A given B can be computed from the probability of B given A by
kshum ENGG2012B 23
http://en.wikipedia.org/wiki/Bayes'_theorem
Thomas Bayes (1701-1761)
• English minister and mathematician• http://en.wikipedia.org/wiki/Thomas_Bayes
kshum ENGG2012B 24
The Monty Hall Problem
• Quote from wikipedia– http://en.wikipedia.org/wiki/Monty_Hall_problem
Suppose you're on a game show, and you're given the choice of three doors: Behind one door is a car; behind the others, goats. You pick a door, say No. 1, and the host, who knows what's behind the doors, opens another door, say No. 3, which has a goat. He then says to you, "Do you want to pick door No. 2?" Is it to your advantage to switch your choice?
• An explanation from youtube– http://www.youtube.com/watch?v=mhlc7peGlGg
kshum ENGG2012B 25
Explanation using conditional probability
• We use – “100” to represent “prize behind the first door”.– “010” to represent “prize behind the second door”.– “001” to represent “prize behind the third door”.
• The host may need to flip a coin in order to open the door. We use “H” and “T” for the outcome of the coin toss.
• Set up a sample space of six elements = {100H, 100T, 010H, 010T, 001H, 001T}.
Each outcome is equally probable.
kshum ENGG2012B 26
Before any door is opened by the host
• Suppose you choose door 1
kshum ENGG2012B 27
100
010
001
1/3
1/3
1/3
The probability thatthere is a car behind thefirst door is 1/3.
After a door is opened• Suppose you choose door 1.• A door is opened by the host of the game.
kshum ENGG2012B 28
100
010
001
1/3
1/3
1/3
H
T
H
T
H
T
100
100
010
010
001
001
Door 3 is opened
Door 2 is opened
Door 3 is opened
Door 3 is opened
Door 2 is opened
Door 2 is opened
Probability given that door 3 is opened
• Suppose you choose door 1.• Suppose door 3 is opened• Pr(Prize is behind door 1 | door 3 is opened) = 1/3• If you switching to door 2, you will win with probability 2/3
kshum ENGG2012B 29
100
010
001
1/3
1/3
1/3
H
T
H
T
H
T
100
100
010
010
001
001
Door 3 is opened
Door 3 is opened
Door 3 is opened
Probability given that door 2 is opened
• Suppose you choose door 1.• Suppose door 3 is opened• Pr(Prize is behind door 1 | door 2 is opened) = 1/3• If you switching to door 3, you will win with probability 2/3
kshum ENGG2012B 30
100
010
001
1/3
1/3
1/3
H
T
H
T
H
T
100
100
010
010
001
001
Door 2 is opened
Door 2 is opened
Door 2 is opened