1

Random Variable

Random variable X is a mapping that maps each outcome s in the sample space to a unique real number x, x .

Examples

Toss a coin. Define the random variable X as follows:

1

0

if headsX

if tails

Go to the queue at a random time. Define the random variable N as the number of packets waiting in the queue.

Go to the bus stop. Define the random variable W as the time you wait until the bus arrives.

Each time we repeat the experiment, the outcome varies and the value of the random variable varies, according to some probability distribution.

With random variables, we can define more complicated things:

2 2 2

0.5

, 1

sin ,

XP e

U X Y P U

W R P W w

:s outcome

X s

Sample Space Real Line

x

2

Mapping

Range (of random variable X ) is { | ( ) }XS x x X s for some s S

In general, X is a many-to-one mapping, but never one-to-many

Inverse Operation: For any subset ,XA S 1( ) { ( ) }X A s X s A

Example- Bernoulli

Toss a biased coin. The coin falls down heads with probability p.

Define the random variable X as

1

0

for headsX

for tails

Range: {0,1}XS

[ 1] [ ]P X P Head p

[ 0.5] 0, [ 0.1] 1 , [ 0] , [ 2] 0, [ 3] 1.P X P X p P X p P X P X

X is referred to as a Bernoulli random variable.

Example - Geometric

Toss a biased coin. X is the number of times we toss the coin until we see the first head.

Range: {1,2,3, }XS .

1[ ] (1 ) for 1, 2,3, , where is the probability of headjP X j p p j p .

X is a referred to as a geometric random variable.

:s outcome

X s

Sample Space Real Line

x

3

Example – Exponential Random Variable

Consider a packet router that processes arriving packets. Measure the time between successive packets arrive, referred to as the packet inter-arrival times.

Let X denote the inter-arrival time.

Range: { | 0 }XS r r

Experiments show frequently

[ ] 1 for 0, where is the packet arrival rate.xP X x e x

X is referred to as an exponential random variable.

Types of Random Variables

Discrete RV

Continuous RV

Discrete Random Variable

The range consists of finite real numbers such as{4,6,8} , or countably infinite real numbers

such as{0,1,2, } or { , 2, 1,0,1,2, }

Notation

When we write ,

is a random variable, and is a constant or a simple variable.

P X x

X x

Continuous Random Variable

When the range is not countable, the random variable is a continuous one.

4

Cumulative Distribution Function

The cdf of a random variable is defined as

( ) [ ]X

X

F x P X x

Example – Uniform Distribution

Throw a dart at a spinning wheel. X is the phase where the dart hits the wheel.

The range of the random variable X is

{0 2 }XS x .

Within the range,

2 11 2 1 2[ ] for any phase 0 2 .

2

x xP x X x x x

Therefore the cdf is

1 2

( ) P 0 220 0

X

x

xF x X x x

x

X is a continuous random variable.

X is referred to as a uniform random variable, or X is said to have a uniform probability distribution.

In short, ,X U a b .

XF x

0x

2

1

5

Example

Buses arrive periodically with a period of T. You arrive at the bus stop at a random time. Define random variable W as the time you wait until the next bus arrives.

Find the cdf of W.

Ans.

1

( ) P 0

0 0

X

x T

xF x X x x T

Tx

(0, ).W U T

Example - Bernoulli

Toss a coin. Define as

1 for a head with prob

0 for a tail with prob 1 .

X

pX

p

X is referred to as a Bernoulli random variable.

1 1

( ) P[ ] 1 0 1

0 0X

x

F x X x p x

x

A discrete random variable has discontinuities in its cdf.

The value of the cdf is taken approaching from the right.

XF x

0x

1

1

1 p

6

Properties of cdf

① 0 ( ) 1XF x

② ( ) 1XF

③ ( ) 0XF

④ ( )XF x is a non-decreasing function of x

⑤ ( )XF x is continuous from the right: that is, 0

( ) lim ( )X Xh

F b F b h

.

⑥ [ ] ( ) ( )X XP a X b F b F a

prob

Prope

① Dif

② Inte

③ To

Be

④ Xf

Examp

is an

i) Find

ii) Find

iii) Plot

Y

Ans.

i) 1

ii) Y

P

f y

bability

rties of pd

fferentiate th

egrate the p

find the pro

careful of t

( ) 0 forX x

ple

exponentia

1 2

d the pdf of

t the cdf and

P Y

2

2

2 y

Y F

y e

density

( )Xf x

df

he cdf to ge

pdf to get th

( )x

XF x

obability

[P a X

the equality

r any , andx

al random va

2 .

.

d pdf.

Y

2

0

Y YF F

y

y functi

( )Xd

F xdx

et the pdf. T

e cdf:

( )x

Xf u du

] (Xb F b

sign for dis

d ( )Xf u du

ariable with

1

ion (pd

The pdf is th

) ( )Xb F a

screte rando

1.u

h the cdf, F

df)

he rate at wh

( )b

Xa

f u du

om variable

( ) 1YF y e

hich the cdf

u

s.

2 for ye y

f increases.

0.

7

For a

i) non

ii) are

For a

i) non

any pdf,

n - negative

ea sums to 1

any cdf,

n - decreasin

1.

ng from 0 to 1

8

9

probability mass function

The cdf of a discrete random variable has discontinuities.

The pdf consists of delta functions.

0 1 2When the range is { , , , }, we often use the notation [ ] .k kx x x p P X x

kp of a discrete random variable is referred to as the probability mass function (pmf).

For a discrete random variable, it is easier to find the pmf first and then the cdf from the following relation

( ) P[ ]k

X kx x

F x X x p

.

XF x

0x

1

1

1 p

Xf x

0x

1

p1 p

10

Notes on continuous distributions

1. For a continuous random variable, it is easier to find the cdf first and then find the pdf by differentiating the cdf.

( ) ( )X Xd

f x F xdx

2. It is wrong to say ( )XP X x f x for a continuous random variable X.

is always 0 for any with a continuous random variable .

However ( ) may not be zero.

Then how are and ( ) are related?

X

X

P X x x X

f x

P X x f x

Note ( ) for any sub-range , .

Therefore we can state that, for a small ,

( ) . (1)

Eq.(1) is often used for finding the pdf directly.

b

Xa

X

P a X b f x dx a b

P x X x f x

R

Roots of PPopular Raandom Varriables

11

12

Distributions from Tossing Coins

Bernoulli Distribution

Probability of head in tossing a coin.

p is the probability of head:

X is a Bernoulli random variable.

1

0

1 ,

0 1

p P X p

p P X p

Geometric Distribution

Probability of seeing the first head at the k-th toss of a coin.

p is the probability of head:

X is a geometric random variable.

1[ ] (1 ) , 1, 2,3,... .

Assume 0 1.

kkp P X k p p k

p

Example

0

For any discrete random variable, must equal 1.

Show 1.

kk

kk

p

p

13

Binomial Distribution

Probability of k heads in n coin tosses:

[ ] (1 ) 0,1, 2, ,...,k n kk

np P X k p p k n

k

Negative Binomial Distribution

Probability of seeing the r-th head at the k-th toss:

11(1 ) , 1,...

1r k r

kk

p p p p k r rr

Example

X is a negative binomial random variable with

11(1 ) , 1,...

1r k r

kk

p p p p k r rr

.

Show 0

1kk

p

.

0.0000001

0.000001

0.00001

0.0001

0.001

0.01

0.1

10 5 10 15 20 25

n=20, p=0.5

Distr

Unifor

models

pd

The tota

Examp

Buses aa rando

Then

W

a

ribution

rm Distri

a randomly

df: ( )Xf x

al area unde

ple

arrive at a bum time. Let

(0,10)W U

ns from

bution U

y chosen po

1

b a

er the pdf sh

us stop perit W denote t

b

m Rando

( , )U a b

int within a

a x b

hould be 1 f

iodically withe time you

om Ent

a finite inter

,b a b

for all conti

ith period ofu wait until

x

try

rval.

nuous rando

f 10 minutel the next bu

om variable

s. You arrivus arrives.

es.

ve at the bus

14

s stop at

15

Distributions from Central Limit Theorem

Let 1 2, , , nX X X be a sequence of independent and identically distributed random

variables each with mean and variance 2 .

1 2Define .

Then lim (0,1) .

nn

nn

X X X nZ

n

Z N normalized Gaussian distribution

Gaussian Distribution 2,N

models a sum of many independent random variables.

pdf:

2

2

( )

22

1( )

2

x

Xf x e x

Raleigh Distribution

models the amplitude of the sum of two orthogonal, independent gaussian RVs

pdf:

2

2( ) 0 with 0

x

bX

xf x e x b

b

0

0.1

0.2

0.3

0.4

0.5

-10 -5 0 5 10

pdf N(0,1)

16

Chi-square Distribution ( )k

models the sum of squares of k independent gaussian RVs.

pdf:

12

2

1

2 2( ) 0 with 1, 2,3,

2

k

x

X

x

f x e x kk

is a special case of Gamma: 1

,2 2

kand .

Cauchy Distribution

models the ratio of two independent Gaussian RV

pdf: 2 2

/( ) with 0Xf x x

x

17

Distributions from Poisson Arrival Process

1

0

2

0

Arrivals occur

) in a memoryless manner.

) One Arrival during ( ),

No Arrival during 1 ( ),

Two or more Arrivals during ( ),

( )where ( ) is a function such that lim 0.j

jt

i

ii P t t o t

P t t o t

P t o t

o to t

t

We call as the arrival rate, and 1

as the average inter-arrival time.

We can show

1) probability of arrivals during any time interval of length is

( )( )

!2) inter-arrival time is exponential with pdf

( ) , 0.

kt

k

xX

k t

tP t e

kX

f x e x

Poisson Distribution

Probability of k arrivals during a time interval t in a poisson arrival process with arrival rate .

( )0,1, 2,

!

kt

kt

p e kk

Example

For all discrete random variables, 1kk

p . Show 0

1kk

p

.

time tn tn+1tn-1 t0 t1 t1

Cn Cn+1Cn-1 C0 C1

18

Exponential Distribution ( )Exp

models the inter-arrival time in a Poisson process with arrival rate .

pdf: ( ) 0xXf x e x

m-Erlang Distribution ( , )Erl m

models the m-th arrival time in a Poisson process with arrival rate .

pdf: 1( )

( ) 0, 0( 1)!

mx

Xx

f x e xm

Gamma Distribution ( , )G

is a general case of m-erlang.

pdf: 1( )

( ) 0 with 0, 0( )

xX

xf x e x

1

0

( )

1Properties are: (1) 1, ( 1) ( ) !, .

2

z xz x e dx

z z z z

Laplacian Distribution

is a two-sided exponential

pdf: | |( ) with 02

xXf x e x