elec 303, koushanfar, fall’09 elec 303 – random signals lecture 2 – conditional probability...
TRANSCRIPT
ELEC 303, Koushanfar, Fall’09
ELEC 303 – Random Signals
Lecture 2 – Conditional probabilityFarinaz Koushanfar
ECE Dept., Rice UniversityAug 27, 2009
ELEC 303, Koushanfar, Fall’09
Lecture outline
• Reading: Sections 1.3, 1.4• Review• Conditional probability
– Multiplication rule– Total probability theorem– Bayes rule
ELEC 303, Koushanfar, Fall’09
Probability theory -- review
• Mathematically characterizes random events• Defined a sample space of possible outcomes• Probability axioms:
1. (Nonnegativity) 0≤P(A)≤1 for every event A2. (Additivity) If A and B are two disjoint events, then the
probability
P(AB)=P(A)+P(B)3. (Normalization) The probability of the entire sample
space is equal to 1, i.e., P()=1
AB
ELEC 303, Koushanfar, Fall’09
Discrete/continuous models -- review
• Discrete: finite number of possible outcomes– Enumerate the possible scenarios and count
• Continuous: the sample space is continuous– The probability of a point event is zero– Probability=area in the sample space
ELEC 303, Koushanfar, Fall’09
Conditional probability example 1
• Student stage: (FR), (SO), (JU), (SI)• Student standing: probation (P), acceptance
(A), honor (H)
Example courtesy of www.ms.uky.edu/~lee/amsputma507
Fresh (FR) Soph (SO) Junior (JU) Senior (SE)Probation (P) 600 200 150 50
Acceptable (A) 1300 900 730 654Honors (H) 100 100 120 96
Fresh (FR) Soph (SO) Junior (JU) Senior (SE)Probation (P) 0.12 0.04 0.03 0.01
Acceptable (A) 0.26 0.18 0.146 0.138Honors (H) 0.02 0.02 0.024 0.0192
ELEC 303, Koushanfar, Fall’09
Example (cont’d)
• What is the probability of a JU student?• What is the probability of honors standing if
the student is a FR?• What is the probability of probation for a SU?• P(A|B) is probability of A given B• B is the new universe
Fresh (FR) Soph (SO) Junior (JU) Senior (SE)Probation (P) 0.12 0.04 0.03 0.01
Acceptable (A) 0.26 0.18 0.146 0.138Honors (H) 0.02 0.02 0.024 0.0192
ELEC 303, Koushanfar, Fall’09
Conditional probability
• Definition: Assuming P(B)0, thenP(A|B) = P(AB) / P(B)
• Consequences: If P(A)0 and P(B)0, thenP(AB) = P(B).P(A|B) = P(A).P(B|A)
A B
ELEC 303, Koushanfar, Fall’09
Conditional probability example 2
1. What is the probability of both dices showing odd numbers given that their sum is 6?
{1,5},{5,1},{2,4},{4,2},{3,3} {1,5},{5,1},{3,3}
2. Let B is the event: min (X,Y)=3, Let M = max{X,Y}. What are the probabilities for M over all of its possible values?
ELEC 303, Koushanfar, Fall’09
Conditional probability example 3
• Radar detection vs. airplane presence• What is the probability of having an airplane?• What is the probability of airplane being there if
the radar reads low?• When should we decide there is an airplane and
when should be decide there is none?
Slide courtesy of Prof. Dahleh, MIT
ELEC 303, Koushanfar, Fall’09
Sequential description
• (A): aircraft present, (Ac): aircraft absent• (L): low, (M): medium, (H): high
P(A)=0.3
P(Ac)=0.7
P(L|A)=0.066
P(M|A)=0.26P(H|A)=0.66
P(L|Ac )=0.643
P(M|Ac)=0.286P(H|A c)=0.071
Multiplication rule: Assuming that all of the conditioning events have positive probability,
)|(...
)...|()|()()(
11
2131211
inin
ini
AAP
AAAPAAPAPAP
ELEC 303, Koushanfar, Fall’09
Conditional probability example 4
• Three cards are selected from a deck of 52 cards without replacement. Find the probability that none of the drawn cards is a picture, i.e., (J,Q,K) drawn set
ELEC 303, Koushanfar, Fall’09
The Monty hall problem
• You are told that a prize is equally likely behind any of the 3 closed doors
• You randomly point to one of the doors• A friend opens one of the remaining 2 doors, after ensuring
that the prize is not behind it• Consider the following strategies:
– Stick to your initial choice– Switch to the other unopened door
Picture courtesy of http://hight3ch.com
ELEC 303, Koushanfar, Fall’09
Total probability theorem
• Divide and conquer• Partition the sample space into A1, A2, A3
• For any even B: P(B) = P(A1B) + P(A2B) + P(A3B)
= P(A1)P(B|A1)+P(A2)P(B|A2)+P(A3)P(B|A3)
A1
A2
A3
B
B
B
Bc
Bc
Bc
A1B
A2B
A3B Figure courtesy of Bertsekas&Tsitsiklis, Introduction to Probability, 2008
ELEC 303, Koushanfar, Fall’09
Radar example 3 (cont’d)
• P(Present) = 0.3• P(Medium|Present)=0.08/0.3• P(Present|low)=0.02/0.47
Example courtesy of Prof. Dahleh, MIT
ELEC 303, Koushanfar, Fall’09
Radar example 3 (cont’d)
• Given the radar reading, what is the best decision about the plane?
• Criterion for decision: minimize “probability of error”
• Decide absent or present for each reading
Example courtesy of Prof. Dahleh, MIT
ELEC 303, Koushanfar, Fall’09
Radar example 3 (cont’d)
• Error={Present and decision is absent} or {Absent and decision is present}
• Disjoint events• P(error)=0.02+0.08+0.05
Example courtesy of Prof. Dahleh, MIT
ELEC 303, Koushanfar, Fall’09
Extended radar example
• P(Threat)=Prior probability of threat = p
• Threat alert affects the outcome
Example courtesy of Prof. Dahleh, MIT
P(…|Threat)
P(…|No threat)
ELEC 303, Koushanfar, Fall’09
Extended radar example
• A=Airplane, R=Radar reading
• If we let p=P(Threat), then we get:
Example courtesy of Prof. Dahleh, MIT
ELEC 303, Koushanfar, Fall’09
Extended radar example
• Given the radar registered high and a plane was absent, what is the probability that there was a threat?
• How does the decision region behave as a function of p?
Example courtesy of Prof. Dahleh, MIT
ELEC 303, Koushanfar, Fall’09
Extended radar example
Example courtesy of Prof. Dahleh, MIT
ELEC 303, Koushanfar, Fall’09
Bayes rule
• The total probability theorem is used in conjunction with the Bayes rule:
Let A1,A2,…,An be disjoint events that form a partition of the sample space, and assume that P(Ai) > 0, for all i. Then, for any event B such that P(B)>0, we have
)|()(...)|()(
)|()(
)(
)|()()|(
11 nn
iiiii ABPAPABPAP
ABPAP
BP
ABPAPBAP