ece353: probability and random processes lecture...

ECE353: Probability and Random Processes

Lecture 5 - Cumulative Distribution Function andExpectation

Xiao Fu

School of Electrical Engineering and Computer ScienceOregon State University

E-mail: [email protected]

From PMF to CDF

• Recall PMF of a discrete RV is PX(x) = P [X = x].

• Definition: the Cumulative Distribution Function (CDF):

FX(x) := P [X ≤ x].

• very useful since in many cases we care about P [X ≤ x].

• it comes very handy in calculating things like P [` ≤ X ≤ u].

ECE353 Probability and Random Processes X. Fu, School of EECS, Oregon State University 1

From PMF to CDF

• Example: X with PMF

PX(x) =

0.15, x = 1

0.65, x = 2

0.2, x = 3

0, o.w.

1 2 3

0.15

0.65

0.2


From PMF to CDF

• Example: from PMF to CDF (FX(x) = PX[X ≤ x])

PX(x) =

0.15, x = 1

0.65, x = 2

0.2, x = 3

0, o.w.

1 2 3

0.15

0.65

0.2

FX(x) =

0, x < 1

1 2 3


From PMF to CDF


PX(x) =

0.15, x = 1

0.65, x = 2

0.2, x = 3

0, o.w.

1 2 3

0.15

0.65

0.2

FX(x) =

0, x < 1

0.15, x = 1

1 2 3

0.15


From PMF to CDF


PX(x) =

0.15, x = 1

0.65, x = 2

0.2, x = 3

0, o.w.

1 2 3

0.15

0.65

0.2

FX(x) =

0, x < 1

0.15, 1 ≤ x < 2

1 2 3

0.15


From PMF to CDF


PX(x) =

0.15, x = 1

0.65, x = 2

0.2, x = 3

0, o.w.

1 2 3

0.15

0.65

0.2

FX(x) =

0, x < 1

0.15, 1 ≤ x < 2

0.8 x = 2

1 2 3

0.15

0.8


From PMF to CDF


PX(x) =

0.15, x = 1

0.65, x = 2

0.2, x = 3

0, o.w.

1 2 3

0.15

0.65

0.2

FX(x) =

0, x < 1

0.15, 1 ≤ x < 2

0.8 2 ≤ x < 3

1 x ≥ 3

1 2 3

0.15

0.8

1


Properties of CDF

1 2 3

0.15

0.8

1

• Some important properties of CDF:

1) FX(−∞) = 0 and FX(+∞) = 1.2) FX(x) ≥ 0.3) ∀x′ ≥ x, we have FX(x

′) ≥ FX(x).4) FX(x) is a constant between two consecutive values x1 and x2.5) P [α < X ≤ β] = FX(β)− FX(α).


Sample Mean and Expectation• Consider a collection of random samples {X1, . . . , XN}. Compute the sample

mean:

sample mean :=1

N

N∑i=1

Xi

• Xi corresponds to the ith sample; or, the outcome of ith trial of a randomexperiment.

• The “problem” with sample mean is that itself is random.

• To fix this, a solution is to take N →∞, in which case, under certain conditions,it can be shown that the sample mean converges to ensemble mean, or, theexpectation of a random variable X.

• Definition: the expectation of a RV X is definied as

E[X] := µX =∑x∈SX

xPX(x)


Expectation

• Example: Let us draw a fair die. We have SX = {1, 2, 3, 4, 5, 6} and

PX(x) =

{16, x ∈ {1, 2, 3, 4, 5, 6}0, o.w.

E[X] =∑

x∈{1,2,...,6}

1

6× x

=1

6× (1 + 2 + 3 + 4 + 5 + 6)

= 3.5

• Why did we say the sample mean converges to E[X]?


Expectation

• Consider a collection of samples {X1, . . . , XN}. Denote

N(xj) =

N∑i=1

1(Xi = xj), where 1(X = x) =

{1, X = x

0, o.w.

• In plain words, N(xj) is the times of seeing xj in the set of samples.

• Consequently, we have

1

N

N∑i=1

Xi =1

N

∑xj∈SX

xjN(xj) =∑xj∈SX

xjN(xj)

N,

where we have limN→∞N(xj)

N = PX(xj).


Expectation of Bernoulli RV

• Example: Bernoulli 0-1 RV:

PX(x) =

{0, w.p. 1− p1, w.p. p.

• By definition,E[X] = 0× (1− p) + 1× p = p.

• What if we have

PX(x) =

{−3, w.p. 1− p5, w.p. p.


Expectation of Geometric RV

• Example: Geometric RV:

PX(x) =

{p(1− p)x−1, x ∈ {1, 2, 3, . . .}0, o.w.

• By definition, we have

E[X] =

∞∑x=1

xp(1− p)x−1

= p

∞∑x=1

xqx−1 (q = 1− p)

= p

∞∑x=1

dqx

dq


Expectation of Geometric RV


E[X] =

∞∑x=1

xp(1− p)x−1

= pd (∑∞x=1 q

x)

dq(q = 1− p)

= pd(p(1 + p+ p2 + . . .)

)dq

= pd(p 11−q

)dq

=1

p

• This is intuitive: Consider the coffee shop example: the number of visits that youneed to meet your barista is inversely proportional to the probability that you canmeet him/her there each time.


Expectation of Poisson RV

• Example: Poisson RV:

PX(x) =

{αx

x! e−α, x ∈ {0, 1, 2, 3, . . .}

0, o.w.


E[X] =

∞∑x=0

xαx

x!e−α

=

∞∑x=1

xαx

x!e−α =

∞∑x=1

xαx

(x− 1)!e−α

=∞∑y=0

αy+1

y!e−α = α

∞∑y=0

αy

y!e−α

= α


Limit of Poisson RV

• Theorem: The Poisson PMF is limit of the Binomial(n, p) PMF, i.e., n → ∞and p→ 0 =⇒ np→ α.

Proof: The Binomial(n, p) PMF is(n

k

)pk(1− p)n−k.

Taking p = α/n, we wish to show(n

k

)(αn

)k (1− α

n

)n−k→ αk

k!e−α

The left hand side (LHS) can be written as

n!

k!(n− k)!αk

nk(1− p)n−k = αk

k!

[n(n− 1)(n− 2) . . . (n− k + 1)

nk

](1− p)n−k


Limit of Poisson RV

• let’s continue...

αk

k!

[n(n− 1)(n− 2) . . . (n− k + 1)

n · n . . . n

](1− p)n−k.

We have

limn→∞

αk

k!

[n(n− 1)(n− 2) . . . (n− k + 1)

n · n . . . n

](1− p)n−k = αk

k!limn→∞

(1− α

n

)n−k=αk

k!limn→∞

(1− α

n

)n(1− α

n

)k=αk

k!limn→∞

(1− α

n

)nBy basic Calculus, we have limn→∞

(1− 1

n

)n= e−1.


ece353: probability and random processes lecture...

Documents