1

Lecture 13

! Conditional Expectation and Variance

! Bivariate Normal Distribution

2

Conditional Expectation! Suppose X and Y are random variables with joint p.f.

or p.d.f. f(x,y), let f1(x) denote the marginal p.f. or p.d.f. or X, for any value of x such that f1(x)>0, let g(y|x) denote the conditional p.f. or p.d.f. of Y given X=x.

! The conditional expectation of Y given X , denoted by E(Y|X), is specified as a function of X:

case)s(continuou)|()|(

case)(discrete)|()|(

ò

å¥

¥-==

==

dyxyygxXYE

xyygxXYEy

3

The Conditional Expectation Is a Random Variable

! E(Y|X) is a function of the random variable X, so it is itself a random variable.

! We can find the mean and the variance of E(Y|X).

4

! Theorem. For any random variables X and Y, E[E(Y|X)]=E(Y).

5

Proof. Assume for convenience that X and Y have a continuous joint distribution, then

)(),(

)()|(

)()|()]|([

1

1

YEdydxyxyf

dydxxfxyyg

dxxfxYEXYEE

==

=

=

ò òò òò

¥

¥-

¥

¥-

¥

¥-

¥

¥-

¥

¥-

6

Conditional Variance

! Let Var(Y|x) denote the variance of the conditional distribution of Y given X=x, i.e.,

2

2

2 2

( | ) ( ( | )) ( | ) (discrete case)

( | ) ( ( | )) ( | ) (continuous case)

or ( | ) ( | ) ( | )

yVar Y x y E Y X x g y x

Var Y x y E Y X x g y x dy

Var Y x E Y x E Y x

¥

-¥

= - =

= - =

= -

å

ò

7

Variance and Conditional Variance

)]|([)]|([)( XYEVarXYVarEYVar +=

8

Variance and Conditional Variance

Proof.)]|([)]|([)( XYEVarXYVarEYVar +=

)]|([)]|([)(,Hence)}({})]|({[

)]}|([{})]|({[)]|([})]|({[)(})]|([)|({)]|([

)]([)()(

22

22

22

22

22

XYEVarXYVarEYVarYEXYEE

XYEEXYEEXYEVarXYEEYEXYEXYEEXYVarE

YEYEYVar

+=-=

-=

-=

-=

-=

9

Example. Customers with Coupons

! Suppose that the number of customers coming to certain McDonald store in a day follows a Poisson distribution with parameter l.

! Suppose that each customer independently presents a coupon with probability p (and no coupon with probability q=(1-p).

! What are the expectation and variance of the number of customers using coupons in a day?

10

Solution. Customers with Coupons

! Let N denote the number of customers in a day.

! Let X denote the number of customers using coupons in a day.

( )~N Poisson l

( )~ ,X N Binomial N p

11

Solution. Customers with Coupons

( ) ( ) [ ] [ ]( ) ( ) ( )

[ ] [ ][ ] [ ]2

2

E X E E X N E Np pE N p

Var X Var E X N E Var X N

Var Np E Npq

p Var N pqE N

p pq

l

l l

é ù= = = =ë ûé ù é ù= +ë û ë û

= +

= +

= +

12

Bivariate Normal--Example2

1 1 12

2 2 2

1 2

2 1 1 2 2

2

~ ( , ), e.g., Farther's Height ~ ( , ), e.g., Child's Height We expect and have a linear relationship:

, where is standard normal and

X NX N

X XX a b X b Z

Z

µ s

µ s

•

••

= + +

1

1 2

independent of . And they have correlation , i.e.,

( , )

X

Corr X Xr

r•

=

Then, X1 and X2 are said to have a bivariate normal distribution, and denote it as 2

1 1 1 1 22

2 2 1 2 2

, ~ ,

, X

NX

µ s rs sµ rs s s

æ öæ öæ ö æ öç ÷ç ÷ç ÷ ç ÷ ç ÷ç ÷è ø è ø è øè ø

13

The Bivariate Normal Distribution

! We can define X1 and X2 as follows:

( )[ ]on.distributi normal edstandardiz followtly independen and where

1

21

222/12

122

1111

ZZZZX

ZX

µrrs

µs

+-+=

+=

14

( )[ ]

( )[ ]

( )[ ]

rrsrsrss

srrs

ss

µµrrs

µµsµrrs

µs

===

=-+=

==

=+-+=

=+=+-+=

+=

),()(),(

)(1)()()()(

)(1)()(

)()(1

21

2112121

222

21

2222

211

211

2222/12

122

11111

222/12

122

1111

XXZVarXXCov

ZVarZVarXVarZVarXVar

ZEZEXE

ZEXEZZX

ZX

15

The Bivariate Normal Distribution

( )( ) ( )

( )

( ) ( )( )

1 1 1 1

2 2 2 1 1 12 1/ 22

1 2 1 2

1 2 1 2

1

1/ 221 21/ 2 1/ 22 2

1 2

/

/ /

1

{( , ) : - , - }{( , ) : - , - }

1 01| | det 1 1

1 1

Z X

X XZ

S z z z zT x x x x

J

µ s

µ s r µ s

r

sr r s s

s r s r

= -

- - -=

-

= ¥ < < ¥ ¥ < < ¥= ¥ < < ¥ ¥ < < ¥

æ öç ÷ç ÷= =ç ÷ --ç ÷ç ÷- -è ø

16

• If X1 and X2 come from a bivariate normal distribution, then they have the above joint p.d.f., and denote it as

( ) ( )( ) ( ) ( )( )

( )

( ) ( )

2 21 2 1 2

1 2 1 1 2 1 1 2

1/ 221 2

2 2

1 1 1 1 2 2 2 21 22

1 1 2 2

1 1, exp2 2

, , , , | |

1

2 1

1exp 2 , for , .2 1

g z z z z

f x x g z x x z x x J

x x x x x x T

p

p r s s

µ µ µ µrs s s sr

é ù= - +ê úë û=

=-

ì üé ùæ ö æ öæ ö æ ö- - - -ï ïê ú- - + Îí ýç ÷ ç ÷ç ÷ ç ÷- ê úè ø è øè ø è øï ïë ûî þ

21 1 1 1 2

22 2 1 2 2

, ~ ,

, X

NX

µ s rs sµ rs s s

æ öæ öæ ö æ öç ÷ç ÷ç ÷ ç ÷ ç ÷ç ÷è ø è ø è øè ø

17

18

Marginal Distributions

! If X1 and X2 come from a bivariate normaldistribution, since both X1 and X2 are linearcombinations of Z1 and Z2, the marginaldistribution of both X1 and X2 are normaldistributions. So the marginal distribution ofXi is a normal distribution with mean andvariance .

iµ2is

19

Independence and Correlation

! If , then X1 and X2 are uncorrelated. Thejoint p.d.f. can be factored into the marginalp.d.f. of X1 and the marginal distribution ofX2. Hence X1 and X2 are independent.

! Two random variables X1 and X2 that have abivariate normal distribution are independentif and only if they are uncorrelated.

0=r

20

Conditional Distributions

• If X1=x1, then . The conditional distribution of X2 given X1=x1 is the same as the conditional distribution of

( ) 1111 /sµ-= xZ

( ) ÷÷ø

öççè

æ -++-=

1

112222

2/122 1

sµrsµsr xZX

( )[ ] 222/12

122

1111

1 µrrs

µs

+-+=

+=

ZZX

ZX

21

• Because Z2 is independent of X1, the conditional distribution of Z2 given X1 =x1 is the only random variable.

• So the conditional distribution of X2 given X1 =x1 is a normal distribution with mean and variance:

( )

( )[ ] ( ) 22

22

2

22/12

112

1

1122

1

112222

2/12112

1)(1)|(

)(1)|(

srsr

sµrsµ

sµrsµsr

-=-==

÷÷ø

öççè

æ -+=

÷÷ø

öççè

æ -++-==

ZVarxXXVar

x

xZExXXE

22

• Similarly, the conditional distribution of X1 given X2 =x2 is a normal distribution with mean and variance:

( )

2 21 2 2 1 1

2

2 21 2 2 1

( | )

( | ) 1

xE X X x

Var X X x

µµ rss

r s

æ ö-= = + ç ÷

è ø

= = -

23

Example! Suppose X has a normal distribution with mean

and variance . For any x, the conditional distribution of Y given X=x is a normal distribution with mean x and variance . What is the marginal distribution of Y?• The joint distribution of X and Y can be derived as

a bivariate normal distribution.• The marginal distribution of Y must be a normal

distribution.

µ2s

2t

[ ][ ] [ ]

22

2 )()()|()|()(

)()|()(

st

t

µ

+=

+=

+====

XVarEXYEVarXYVarEYVar

XEXYEEYE

24

Linear Combinations! Suppose X1 and X2 have a bivariate normal

distribution. Consider Y=a1X1+a2X2+b, where a1, a2

and b are arbitrary given constants.• Both X1 and X2 can be represented as linear

combinations of independent and normallydistributed random variables Z1 and Z2, so Y can isalso a linear combination of Z1 and Z2.

• So Y has a normal distribution.

212122

22

21

21

21212221

21

22112211

2),(2)()()(

)()()(

srsss

µµ

aaaaXXCovaaXVaraXVaraYVar

baabXEaXEaYE

++=

++=

++=++=

25

Example! Suppose that a married couple is selected at

random from some population. The jointdistribution of the height of the wife and theheight of her husband is a bivariatedistribution with

What is the probability that the wife will be taller than her husband?

68.0,in2,in2,in70,in8.66 2121 ===== rssµµ

26

! Let X denote the height of the wife, and let Ydenote the height of her husband.

! The distribution of X-Ywill be normal with

! So has a standard normal distribution.

56.2)2)(2)(68.0(244),(2)()()(

2.3708.66)( 21

=-+=-+=--=-=-=-

YXCovYVarXVarYXVarYXE µµ

0227.0)2(12Pr0Pr =F=>=> -)(Z)(X-Y

56.2/)2.3( +-= YXZ