thinking with probability - trinity college dublin week... · 2013-02-15 · thinking with...
TRANSCRIPT
Thinking with Probability
Thinking – fast and slow
Thinking by decomposing events
formalising probability rules
conditional probability
Brain Teasers Tijms Problems, Chap 1
15/02/2013 ST2352 2013 Week 6 1
2 dice rolled; one is 6. What prob other is 6?
Aged 22 Prob of death
2 babies dead. Pr(mother murderer)?
Monty Hall; Choose 1; Learn goat at 2. Pr(car at 3)?
Thinking with Probability Thinking about systems by decomposing events
Formalising probability rules
conditional probability
Forward modelling A makes B more likely
Inverse thinking Have observed B; Pr(A|B)
Evaluating evidence Bayes Rule
Using evidence in prediction Statistical Inference
15/02/2013 ST2352 2013 Week 6 2
Evaluating Evidence with Probs
How does evidence lead to conclusions in situations of uncertainty? Bayes Theorem
Data fusion, use of techniques that combine data from multiple sources and gather that information in order to achieve inferences, which will be more efficient and potentially more accurate than if they were achieved by means of a single source.
Spam
Cancer Screening
Law
…..
3 ST2352 2013 Week 6 15/02/2013
All Probs are Conditional
Probability
Measure of uncertainty about event given info Pr(A wins league, given probs for each match, and indep)
Pr(A wins league, given that somebody wins, and probs, and indep)
Pr(A wins league, given no info, other than probs, and indep)
Pr info(event ) Pr(event | info)
Simple Forward Theory
Subtle Inverse Inference 15/02/2013 ST2352 2013 Week 6 4
Bayes Rule
ST2352 2013 Week 6 5
Pr( ) Pr( | ) Pr( ) Pr( ) Pr( | ) Pr( )
Pr( )Pr( | ) Pr( | )
Pr( )
Aand B A B B B and A B A A
BB A A B
A
Inverting the Conditioning
Multiple Possibilities
1 2
1 1
................
Pr( | )Pr( )Pr( | )
Pr( | )Pr( ) ....... Pr( | )Pr( )
n
i ii
n n
B B OR B OR B
A B BB A
A B B A B B
15/02/2013
Inversion
15/02/2013 ST2352 2013 Week 6 6
2 dice rolled; one is 6. What prob other is 6? 2 dice rolled; first is 6. What prob other is 6?
Serious
Sally Clarke - Sudden Infant Death SID
The case was widely criticised because of the way statistical evidence was misrepresented in the original trial, particularly by Meadow. He stated in evidence as an expert witness that "one sudden infant death in a family is a tragedy, two is suspicious and three is murder unless proven otherwise" (Meadow's law). He claimed that, for an affluent non-smoking family like the Clarks, the probability of a single cot death was 1 in 8,543, so the probability of two cot deaths in the same family was around "1 in 73 million" (8543 × 8543).
ST2352 2013 Week 6 7 15/02/2013
Light
Metro Wed 10 Nov 2010
Teens at risk from hyper-texting
Teenagers who send more than 100 text messages per day are more likely to have had sex, tried drugs, research has revealed.
4200 students at 20 schools; hyper-texting 19.2%
Such teens 43% more likely to have tried alcohol.
ST2352 2013 Week 6 8 15/02/2013
Brain Teasers
Monty Hall Game Show (Tijms, Ch 1, Q11)
One car, behind one of three doors.
Player selects one: say Door 1
Before opening this door
host opens one of two others: say Door 2 GOAT!
host offers chance to change selection.
Issue Is there any point changing?
Evidence in favour of (stay with) chosen door
ST2352 2013 Week 6 9 15/02/2013
ST2352 2013 Week 6 10
Serious Life Expectancy in Ireland
Average age at death = 75
Average age at death = 80
Average age at death, given survival to 60, = 79
http://understandinguncertainty.org/node/272
15/02/2013
ST2352 2013 Week 6 11
Cond Prob for Lifetimes Knowledge of current age impacts uncertainty on age at death
Probability Distribution Poss LiveTimes 1 2 3 4 5 6 Corresp Probs 0.1 0.2 0.3 0.3 0.05 0.05
Pr(death at end day 3, given alive at start day 3)
15/02/2013
ST2352 2013 Week 6 12
Cond Prob for Lifetimes Knowledge of current age impacts uncertainty on age at death
Expt: Choose random component
Poss Lives 1 2 3 4 5 6 Probs 0.1 0.2 0.3 0.3 0.05 0.05
Pr(death at end day 3) = 0.30
Pr(alive at start day 3) = 0.70
Pr(death at end day 3, given alive at start day 3)
Pr(death 'at' 3 AND alive 'at' 3) Pr(death 'at' 3)= =
Pr(ali
What event
ve 'at'
ident ?
3)
ity
P
0.30.43
r(alive 'at' 3) 0.7
Pr(death end day , given alive day 3)
0.3 0.3 0.05 0.05, , , for 3, 4,5,6
0.7 0.7 0.7 0.7
k
k resp
15/02/2013
Odds and Weight of Evidence
ST2352 2013 Week 6 13
Pr(in favour of )
Pr
Pr1
Twice as likely
Home Advantage
Risk Factor
AOdds A
A
Odds A
15/02/2013
Odds and Weight of Evidence
ST2352 2013 Week 6 14
Odds Rule Form for Evidence
Pr( ) Pr( )Pr( | ) Pr( | ) ; Pr( | ) Pr( | )
Pr( ) Pr( )
Pr( | ) Pr( | ) Pr( )
Pr( | ) Pr( | ) Pr( )
A AA E E A A E E A
E E
A E E A A
A E E A A
15/02/2013
Weight of Evidence
ST2352 2013 Week 6 15
12
12 1
2
13
Pr( | ) Pr( | ) Pr( )
Pr( | ) Pr( | ) Pr( )
Car behind door chosen (eg 1);
Host opens door (eg 2)
Prior Odds(A) =
Pr( | ) Pr( | )
Posterior Odds(A)1
Posterior Pr(A) =
A E E A A
A E E A A
A A
E
E A E A
15/02/2013
Weight of Evidence
ST2352 2013 Week 6 16
12
12 1
2
13
Pr( | ) Pr( | ) Pr( )
Pr( | ) Pr( | ) Pr( )
Car behind door chosen (eg 1);
Host opens door (eg 2)
Prior Odds(A) =
Pr( | ) Pr( | )
Posterior Odds(A)1
Posterior Pr(A) =
A E E A A
A E E A A
A A
E
E A E A
15/02/2013
Brain Teasers
• Who is the murderer? (Tijms, Ch 1 Q6)?
Murder committed; know either X or Y – equally likely.
Evidence: actual perp has blood group A
10% of people group A; X is group A
Seek Pr( X is perp | evidence)
ST2352 2013 Week 6 17 15/02/2013
Brain Teasers
Who is the murderer? (Tijms, Ch 1 Q6)?
Murder committed; know either X or Y – equally likely.
Evidence: actual perp has blood group A
10% of people group A; X is group A
Seek Pr( X is perp | evidence)
ST2352 2013 Week 6 18
Define murderer
Define Blood group A left at crime scene
Pr( | )Pr( )Pr( | ) Pr( | )Pr( )
Pr( )
Pr( ) ? Pr( ) ?
Pr( | ) ? Pr( | ) ?
Pr( | ) ?
H X
E
E H HH E E H H
E
H H
E H E H
H E
Events are T /F
15/02/2013
Brain Teasers
Who is the murderer? (Tijms, Ch 1 Q6)?
Murder committed; know either X or Y – equally likely.
Evidence: actual perp has blood group A
10% of people group A; X is group A
Seek Pr( X is perp | evidence)
ST2352 2013 Week 6 19
Define murderer
Define Blood group A left at crime scene
Pr( | )Pr( )Pr( | ) Pr( | )Pr( )
Pr( )
Pr( ) Pr( ) 0.5
Pr( | ) 1; Pr( | ) 0.1
1 0.5 10Pr( | )
1 0.5 0.1 0.5 11
H X
E
E H HH E E H H
E
H H
E H E H
H E
15/02/2013
Weight of Evidence
ST2352 2013 Week 6 20
Evidence Fusion
1 1
1 1
Pr( | ..... ) Pr( | )Pr( | ) Pr( )....
Pr( | ..... ) Pr( | ) Pr( | ) Pr( )
if evidence indep given ,
n n
n n
A E AND AND E E AE A A
A E AND AND E E A E A A
A A
15/02/2013
Serious
Sally Clarke - Sudden Infant Death SID
He claimed that, for an affluent non-smoking family like the Clarks, the probability of a single cot death was 1 in 8,543, so the probability of two cot deaths in the same family was around "1 in 73 million" (8543 × 8543).
ST2352 2013 Week 6 21
2
18543
Pr(2 Cot Deaths | Normal Family) = Pr(Normal Family | 2 Cot Deaths)
Pr(Normal | 2 Deaths) Pr(Normal ) Pr(2 Deaths | Normal)=
Pr(Not normal | 2 Deaths) Pr(Not normal ) Pr(2 Deaths | Not normal)
15/02/2013
Serious
Sally Clarke - Evidence Fusion
ST2352 2013 Week 6 22
Pr(Normal | 2 Deaths) Pr(Normal ) Pr(1 Death | Normal)=
Pr(Not normal | 2 Deaths) Pr(Not normal ) Pr(1 Death | Not normal)
Pr(2 Death | Normal) Pr(2 Death | Not normal)
Pr(2 Deaths | Normal) =
nd
nd
Formally
Pr(2 Death | 1 Death, Normal) Pr(1 Death | Normal)nd st
15/02/2013
Background
• Overlaps completely with ST2351
15/02/2013 ST2352 2013 Week 6 23
Decomposition via Conditional Probs
ST2352 2013 Week 6 24
13
12 2
1
1 2
3
13
13
12
16
16
1
1
3
2
3
13
13
Contestantchooses door1, for example
Car in factbehind each door with equal prob
As car behind door 1quiz master opens 2 or 3 with equal prob
If car behind door 2quiz master MUST open 3 ie prob = 1; sim’ly if behind door 3
3
2
2
31
1
1
11
6
16
13
13
Car
Car
Goat
Goat
Contestantdoes not switch; remains with door 1
Car
Car
Goat
Goat
Contestantswitches with prob 1 to Door:
Prob wins =2
31
3
15/02/2013
Inversion
15/02/2013 ST2352 2013 Week 6 25
Survey of travellers in US finds: 20% have been to Europe; 15% have Amex card and have been to Europe; 55% have neither; What %age of Amex card holders have been to Europe?
Y N Y N
Y 15% Y 15
N 15% N 55
15% 20 100
Europe
Amex
Europe
Amex
Pr | Pr |A E E A
Inversion
Inverse Theory Prob ( 1st Q, given 2nd Q)
15/02/2013 ST2352 2013 Week 6 26
2 1 2 1 1
1 2
2 2
2 2 1 1 2 1 1
Pr Pr | PrPr |
Pr Pr
RecallPr Pr | Pr Pr | Pr
Q AND Q Q Q QQ Q
Q Q
Q Q Q Q Q Q Q
Queens
Inversion
15/02/2013 ST2352 2013 Week 6 27
Y N Y N
Y 15% Y 15
N 15% N 55
15% 20 100
Europe
Amex
Europe
Amex
Pr Pr | Pr
Pr |Pr Pr | Pr Pr | Pr
A E A EAN EE A
A A
D
E E A E E
Event Decomposition
ST2352 2013 Week 6 28
( )
( )
Pr( )
OR
AND
B B Certain Event
A A B B AOR AND OR ANDB A B
A
15/02/2013
1 2
1 2
1 2
................
.......
More generally
.........
...
Pr( )
n
n
n
B OR B OR B Certain Event
A A AND A AND B OR B OR B
A AND B OR A AND B OR A AND B
A
Event Identities Probability Monte Hall
15/02/2013 ST2352 2013 Week 6 29
Choose Correct Door Choose Incorrect Door Certain Event
Win by Staying Choose Correct Door
Win by Switching Choose Incorrect Door
Pr Win by Staying
OR
Pr( ) Pr( ) Pr( ) Pr( )
Important special case
Pr( ) Pr( ) Pr( )
OR AND
O
A B A B A B
A B A B whendis oi tR j n
Addition Rule
Probability Rules Conditional Prob and Independence
1 2
1 2
Pr( ) Pr( | ) Pr( )
Important special case
Pr( ) Pr( ) Pr( )
Pr 2 2
Pr 2 2
A B A B B
A B A B when independent
Die Di
AND
AND
AND
AND
e
Die S
Multiplication Rule
30 ST2352 2013 Week 6
All computed probs: need real world knowledge assumptions about real world Sometimes useful to be explicit what - if
15/02/2013
Chain Rule: Forward Theory
Extension Chain Rule
Pr Pr( ) | )Pr( )
Pr(
Pr( )Pr( ) P
| ) Pr( | )Pr( | )
Pr Pr( | )Pr( | )Pr(
r( | ) Pr( ) Pr( | )Pr( )
Pr( , , ) Pr( | , )Pr( | )P (
)
r )
A AND B ANDC A AND B C C
But A AND B C A B AND
Aand BAand B A B B A B
B
A
C B C
Thus A AND B ANDC A B ANDC
B C
B
A B C B C C
Sp
C C
Pr( )Pr( )Pr( )ecial case if events probabilistically iA B C ndep
15/02/2013 ST2352 2013 Week 6 31
Forward Theory: Decompose
Cards Prob 1st two cards Queen?
By count
By cond prob
Inverse Theory Prob ( 1st Q, given 2nd Q)
15/02/2013 ST2352 2013 Week 6 32
1 2
1 2 2 1 1
,
Pr
Defin
Pr | P
e
r
Q Q
Q ANDQ Q Q Q
Queens
Simpler than combinatorics?
Cards Prob 2nd card is Queen?
Event Identity
15/02/2013 ST2352 2013 Week 6 33
2 2
2whence Pr
Q Q AND
Q
Forward Theory: Decompose
ST2352 2013 Week 6 34
nd2
1
2
Regular pack of cards; no replacement
= 2 card is Q
by consideri
Focus
Decompose
Event Identit
ng
Pr( )
y
Q
Q
mpute QCo
Chance Tree
Simulation equiv?
15/02/2013
Forward Theory: Decompose
Queens
Password
– Draw 8 chars, unif at random, ABC…XYZ123..89
– Reject entire password if any dups within 8 • Alt, reject char if same as any prev accepted char
– What is prob ‘reject password’ (long run prop)
15/02/2013 ST2352 2013 Week 6 35
Forward Theory: Decompose
ST2352 2013 Week 6 36
Password 8 from ABC…XYZ123..89 (unif@rand, w/repl)
– Reject entire password if any dups within 8
8
7 6Pr 8
28.29.30.31.32.33.34.350.42
35
Pr 8 0.58
N N NAccept
N N N
Reject
15/02/2013
Forward Theory: Decompose
40
325.326........365Pr(at least one common birthday) =1-
365
1 0.097 0.903
ST2352 2013 Week 6 37
Password 8 from ABC…XYZ123..89 (unif@rand, w/repl)
– Reject entire password if any dups within 8
– Prob ‘reject password’ Easier: Prob accept
8
8 8 7 1
8 8 7 1
8 7 1 7 6 1 1
8 7 7 5 1
Accept first chars from alphabet of
Seek Pr
Event Identity Password OK = ...
Pr Pr ...
Pr | ... Pr | ... ... Pr
Pr | Pr | ...
Def
7
i
Pr
nek
AND AND
A
A k N
NOT A
A A A A
A A A A
A A A A A A A
ND AN
A A A A A
D
AND AN
N
D
N
6N N
N N
15/02/2013
Birthday problem
Forward Theory: Decompose
ST2352 2013 Week 6 38
Password 8 from ABC…XYZ123..89 (unif@rand, w/repl)
– Reject entire password if any dups within 8
– Prob ‘reject password’ Easier: Prob accept
8 8 7 8 7 1
8 8 7 1
8 7 1 6 7 1 1
8
Accept first chars;
Event Identity Password OK = ...
Pr Pr ...
Pr | ... Pr | ... ... Pr
7 6
Define
Pr
k
AND AND AND
AND AND
AND AN
A k
A A A A A A
A A A A
A A A A A A A
N N N
N N N
NOT A
D
15/02/2013
Birthday problem
Forward Theory: Decompose
ST2352 2013 Week 6 39
1
1...6 1 1
1
Poss values of are 1,2, 6
Pr ; 0, 6( 1)
in terms of
k
k k
k s s
k k k
y k k
k k
s
k
s
AN
S s k
S s p p s k
S s S s Y anyval
Y y S s
D
OR A y
p p
N
p
D
15/02/2013
Forward Recursion by Decomp: Dice
16
Poss values of
1,2, 6;
Pr
k
k
Y
y
Y y
Conditional Independence
15/02/2013 ST2352 2013 Week 6 40
1 1 1 1
1 1 1 1
Formally Pr , | Pr | Pr |
Notation ,
Generalisation of Pr Pr Pr
Example Markov Chain
Pr | Pr | Pr | ,
Pr , | Pr | Pr |
t t t t t t
t t t t t t t
A B C A C B C
A B A AND B
A AND B A B
Y past Y Y Y Y Y
Y Y Y Y Y Y Y
SubsysC
using RAND()C
SubsysA
using RAND()A
SubsysB
using RAND()B
Randomness in AC,BC dependent; eg AC ‘large’ BC ‘large’ (probably) Randomness in AC, BC cond indep, if we know the value of C