Lecture 28

Agenda

1. Conditional Expectation for discrete random variables

2. Joint Distribution of Continuous Random Variables

Conditional Expectation for discrete random

variables

Let X and Y be two discrete random variables. For X = x we know thatinstead of using {pY (y) : y ∈ Range(Y )} it’s better to use {pY |X=x(y) : y ∈Range(Y )}. We defined this conditional distribution as,

pY |X=x(y) =pX,Y (x, y)

pX(x)=

P (X = x, Y = y)

P (X = x)

for x ∈ Range(X) and y ∈ Range(Y ). Note that this definition is con-ceptually nothing new, because we have already learned the definition ofconditional probability. Now we define conditional expectation of Y givenX = x.

Definition 1. For two discrete random variables X and Y , for some x ∈Range(X), we define the conditional expectation of Y given X = x as,

E(Y |X = x) =∑

y∈Range(Y )

y × pY |X=x(y)

Please note that E(X|Y = y) can be defined similarly.

Example

The joint distribution of X and Y is given in the following table

X=0 X=1 X=3Y=-1 0.11 0.03 0.00Y=2.5 0.03 0.09 0.16Y=3 0.15 0.15 0.06

Y=4.7 0.04 0.16 0.02

1

Let’s find E(Y |X = 1).

P (X = 1) = 0.03 + 0.09 + 0.15 + 0.16 = 0.43

Hence,

E(Y |X = 1) = −1× 0.03

0.43+ 2.5× 0.09

0.43+ 3× 0.15

0.43+ 4.7× 0.16

0.43= 3.2488

Joint Distribution of Continuous Random Vari-

ables

We previously studied the joint probability mass function of two discreterandom variables, which described their joint behaviour. Today, we repeatthe same procedure for continuous random variables.

Let’s recollect that for a continuous random variable X, there has to bea density function fX such that,

• fX(x) ≥ 0 ∀x ∈ R

• ∀a < b, P (a < X < b) =∫ b

afX(x)dx

Suppose we have two continuous random variables X and Y . The jointprobability behaviour of X and Y is described by the “joint probabilitydensity function” fX,Y of X and Y which has the following properties

• fX,Y (x, y) ≥ 0 ∀(x, y) ∈ R2

• P (a ≤ X ≤ b, c ≤ Y ≤ d) =∫ b

a

∫ d

cfX,Y (x, y)dxdy for all a < b and

c < d.

It follows that the “joint probability distribution function” FX,Y isdefined as,

FX,Y (a, b) = P (X ≤ a, Y ≤ b) =

∫ a

−∞

∫ b

−∞fX,Y (x, y)dxdy

Example

A certain process for producing an industrial chemical yeilds a product thatcontains two main types of impurities. Let X denote the proportion of im-purities of Type I and Y denote the proportion of impurities of Type II.

2

Suppose that the joint density of X and Y can be modelled as,

fX,Y (x, y) = 2(1− x) if 0 ≤ x ≤ 1, 0 ≤ y ≤ 1

= 0 otherwise

Compute P (0 ≤ X ≤ 0.5, 0.4 ≤ Y ≤ 0.7).

P (0 ≤ X ≤ 0.5, 0.4 ≤ Y ≤ 0.7) =

∫ 0.5

0

∫ 0.7

0.4

2(1− x)dydx

=

∫ 0.5

0

(0.7− 0.4)× 2(1− x)dx

= 0.3×∫ 0.5

0

2(1− x)dx

= 0.3×[−(1− x)2

]0.50

= 0.3×[−(1− 0.5)2 + (1− 0)2

]= 0.3× [1− 0.25]

= 0.3× 0.75

= 0.225

Lemma 1. If X and Y are continuous random variables, with joint pdf fX,Y

then the individual or marginal pdf ’s fX and fY are given by,

fX(x) =

∫ ∞−∞

fX,Y (x, y)dy

fY (y) =

∫ ∞−∞

fX,Y (x, y)dx

For the industrial production example considered previously, we compute

3

fX and fY .

fX(x) =

∫ ∞−∞

fX,Y (x, y)dy

=

∫ ∞−∞

2(1− x)1{0≤x≤1,0≤y≤1}dy

=

∫ 1

0

2(1− x)1{0≤x≤1}dy

= 2(1− x)1{0≤x≤1}

= 2(1− x) if 0 ≤ x ≤ 1

= 0 otherwise

fY (y) =

∫ ∞−∞

fX,Y (x, y)dx

=

∫ ∞−∞

2(1− x)1{0≤x≤1,0≤y≤1}dx

=

∫ 1

0

2(1− x)1{0≤y≤1}dx

= 1{0≤y≤1}[−(1− x)2

]10

= 1{0≤y≤1} × 1

= 1 if 0 ≤ y ≤ 1

= 0 otherwise

Please note that by using the indicator function 1{...} we are consideringthe various cases at once and making the calculations easier.

Suppose we are interseted in the behaviour of the random variable Ygiven X = x. To develop a framework for expressing this behaviour weneed the notion of “conditional probability density function” (or “conditionalpdf” in short). But before we go on let’s note one thing. When we defined

P (A|B) = P (A∩B)P (B)

, we inherently assumed that P (B) > 0. But if we want to

calculate P (Y ≤ 0.5|X = 0.3), we can’t assume P (X = 0.3) > 0 because Xis continuous. So we need another strategy.

Definition 2. Let X and Y be continuous random variables with joint pdffX,Y and marginal pdf ’s fX and fY . Then for any x such that fX(x) > 0,

4

we define the conditional pdf of Y given X = x as,

fY |X=x(y) =fX,Y (x, y)

fX(x)

for all y ∈ R.

Note that when fX(x) = 0 we don’t define the above quantity. Now wecan calculate

P (Y ≤ 0.5|X = 0.3) =

∫ 0.5

−∞fY |X=0.3(y)dy

Homework::

1. For homework from Lecture 27, find E(Y |X = 1) for Problem

1 and E(X|Y = −1) for Problem 2.

2. 5.27,5.31,5.32,5.34

5