1

Lecture 4

! Conditionally Independent Events

! Bayes’ Theorem

2

Review 1: Conditional Probability

! The probability of an event A changes after it has been learned that some other event B has occurred. This new probability of A is called the conditional probability of the event A given that the event B has occurred. It is denoted by

)|Pr( BA

)Pr()Pr()|Pr(

BABBA = , if Pr(B)>0.

3

Review 2: Independence of Several Events

! The k events are independent if, for every subset of j of these events (j=2,3,...,k),

! E.g. Two conditions must be satisfied in order for three events A, B and C to be independent.

kAA ,...,1

jii AA ,...,1

)2( )Pr()Pr()Pr()Pr(

)Pr()Pr()Pr()1()Pr()Pr()Pr(

)Pr()Pr()Pr(

CBAABC

CBBCCAACBAAB

=

===

)1()2()2()1( ÞÞ

4

Conditionally Independent Events

! A1,…,Ak are conditionally independent given Bif, for every subset Ai1,…,Aij (j=2,3,…,k),

Question: Can conditional independence imply independence.

5

Are A,B conditionally independent given C?

Are A,B independent?

Question: can independence imply conditional independence?

6

Theorem: Suppose that A1, A2 and B are events suchthat Pr(A1B)>0. Then A1 and A2 are conditionallyindependent given B if and only if

Pr(A2|A1B)=Pr(A2|B)

7

2.3 Bayes’ Theorem虚假阳性病例

8

9

由于治疗这种疾病有严重的副作用，所以医生请教概率专家乔·贝叶斯，乔引用了一个法则给以答复，此定理最早由他的祖先牧师托马斯·贝叶斯（1744年~1809年）证明过。

10

( ) ( | )

( )From the statement of the problem, the following information is available: ( ) .001 ( ) .999

Solution:

C

P A BP B AP A

P B P BP

Ç=

= =

( | ) .99 ( | ) .02

By the Multiplicative Probability Rule, ( ) ( ) ( | ) (.001)(.99) .00099, ( ) ( ) ( | ) (.999)(.02) .01998.We ca

C

C C C

A B P A B

P A B P B P A BP A B P B P A B

= =

Ç = = =

Ç = = =n also get

( ) ( ) ( ) .00099 .01998 .02097.Therefore,

( ) .00099 ( | ) .0472 ( ) .02097

CP A P A B P A B

P A BP B AP A

= Ç + Ç = + =

Ç= = =

11

12

13

Prior and Posterior Probabilities

! Pr(B) is called the prior probability.• Before the experiment.

! Pr(B|A) is called the posterior probability.• After the experiment’s result is known.

14

Bayes’ Theorem! Probability revision

Prior Probabilities

NewInformation

Bayes’ Theorem

Posterior Probabilities

initial probabilityestimates for specificevents of interest

additional informationabout the events

revised probabilities

Pr(Bj) event A happened Pr(Bj|A)=?

15

Probability and Partitions! A partition of the sample space S:B1,…,Bk are disjoint and

! A is any other event in S, we will have a partition of A:

( ) ( ) ( )1 2 kA B A B A B A= È È ÈL

SBiki =È =1

16

1

1

Pr( ) Pr( )

Pr( ) 0, 1, Pr( ) Pr( )Pr( | )

Pr( ) Pr( )Pr( | )

k

jj

j j j j

k

j jj

A B A

B j k B A B A B

A B A B

=

=

=

> = Þ =

Þ =

å

å

L

Theorem 2.3.1 Law of total probability. Suppose that the eventsB1,…,Bk form a partition of the sample space S and Pr(Bj)>0.Then, for any event A in S,

å=

=k

jjj BABA

1)|Pr()Pr()Pr(

Proof:

17

Bayes’ Theorem! Let the events B1, ...,Bk form a partition of the space S

such that Pr(Bi)>0 for j=1,...,k, and let A be any event such that Pr(A)>0. Then, for j=1,...,k,

Proof.

å=

= k

jjj

iii

BAB

BABAB

1)|Pr()Pr(

)|Pr()Pr()|Pr(

å=

=

=

=

k

jjj

iii

ii

BABA

BABABAABAB

1)|Pr()Pr()Pr(

)|Pr()Pr()Pr()Pr()Pr()|Pr(

18

Example: Identifying the Source of a Defective Item

! Three different machines: M1, M2, M3. 20% of the items were produced by M1, 30% by machine M2, and 50% by machine M3.

! 1% of the items produced by M1 are defective; 2% of the items produced by M2 are defective; 3% of the items produced by M3 are defective. The outcomes from each machine are independent.

! One item is selected at random from the entire batch and it is found to be defective. What is the probability that this item was produced by M2?

19

! Bi: the selected item was produced by machine Mi, i=1,2,3A: the selected item is defective

We need to calculate Pr(B2|A)Pr(B1)=0.2, Pr(B2)=0.3, Pr(B3)=0.5Pr(A|B1)=0.01, Pr(A|B2)=0.02, Pr(A|B3)=0.03

2 22 3

1

Pr( )Pr( | )Pr( | )Pr( )Pr( | )

(0.3)(0.02) 0.261(0.2)(0.01) (0.3)(0.02) (0.5)(0.03)

j jj

B A BB AB A B

=

=

= =+ +

å

1 3Similarly, Pr( | ) 0.087, Pr( | ) 0.652B A B A= =

20

Computation of Posterior Probabilities in More Than One StageA box contains one fair coin and one coin with a head on each side. One coin is selected at random and when it is tossed, a head is obtained. What is the probability that the selected coin is the fair coin?

B1: the coin is fairB2: the coin has two headsH1: a head is obtained when the coin is tossed

31

)1)(5.0()5.0)(5.0()5.0)(5.0(

)|Pr()Pr()|Pr()Pr()|Pr()Pr()|Pr(

212111

11111

=+

=

+=

BHBBHBBHBHB

21

The same coin is tossed again and another head is obtained. What is the new posterior probability that the coin is fair?

H2: a head is obtained on the second toss

51

)1)(5.0()25.0)(5.0()25.0)(5.0(

)|Pr()Pr()|Pr()Pr()|Pr()Pr()|Pr(

22121211

1211211

=+

=

+=

BHHBBHHBBHHBHHB

22

遗传风险

在人类遗传学中，某种坏的基因会引起夭折。设隐性基因 a是这样一个基因，基因型 aa不能长大成人，基因型 Aa的人为携带者。假定在一般总体中携带者的概率为 p.

已知某成人有一个哥哥或一个姐姐夭折，求该人为携带者的概率。

解：首先，双亲都必须是携带者 Aa，那么，他们的孩子是 AA，Aa，aa的概率为1/4, 1/2, 1/4.

P(Aa|成人）=P(Aa)/P(AA或 Aa)=2/3