Functions of Random Variables

Method of Distribution Functions

• X1,…,Xn ~ f(x1,…,xn)

• U=g(X1,…,Xn) – Want to obtain fU(u)

• Find values in (x1,…,xn) space where U=u

• Find region where U≤u

• Obtain FU(u)=P(U≤u) by integrating f(x1,…,xn) over the region where U≤u

• fU(u) = dFU(u)/du

Example – Uniform X• Stores located on a linear city with density

f(x)=0.05 -10 ≤ x ≤ 10, 0 otherwise• Courier incurs a cost of U=16X2 when she delivers to a

store located at X (her office is located at 0)

1600080

)()(

160004044

05.005.0)()(

44

416

2/1

4

4

2

uu

du

udFuf

uuuu

dxuUPuF

uX

uuU

uXuXuU

UU

u

uU

Example – Sum of Exponentials

• X1, X2 independent Exponential()

• f(xi)=-1e-xi/xi>0, >0, i=1,2

• f(x1,x2)= -2e-(x1+x2)/ x1,x2>0

• U=X1+X2

),2(~01

11)(

11

111

1

11)(

,

/2

/2

////

2/)(

02/

02/)(/

0

20

//

021//

0 0 2

21221

2121

22222

21221

2

GammaUuue

eu

eeufuee

dxedxedxee

dxeedxdxeeuUP

XuXuXuXXuU

xuXuXXuU

u

uuuU

uu

xuxuxuxuxu

xuxxuxxu xu

Method of Transformations• X~fX(x)

• U=h(X) is either increasing or decreasing in X

• fU(u) = fX(x)|dx/du| where x=h-1(u)

• Can be extended to functions of more than one random variable:

• U1=h1(X1,X2), U2=h2(X1,X2), X1=h1-1(U1,U2), X2=h2

-1(U1,U2)

22112121

1

2

2

1

2

2

1

1

2

2

1

2

2

1

1

1

),()(||),(),(

||

1duuufufJxxfuuf

dU

dX

dU

dX

dU

dX

dU

dX

dU

dX

dU

dXdU

dX

dU

dX

J

U

Example

• fX(x) = 2x 0≤ x ≤ 1, 0 otherwise

• U=10+500X (increasing in x)

• x=(u-10)/500

• fX(x) = 2x = 2(u-10)/500 = (u-10)/250

• dx/du = d((u-10)/500)/du = 1/500

• fU(u) = [(u-10)/250]|1/500| = (u-10)/125000 10 ≤ u ≤ 510, 0 otherwise

Method of Conditioning

• U=h(X1,X2)

• Find f(u|x2) by transformations (Fixing X2=x2)

• Obtain the joint density of U, X2:

• f(u,x2) = f(u|x2)f(x2)

• Obtain the marginal distribution of U by integrating joint density over X2

222 )()|()( dxxfxufufU

Example (Problem 6.11)• X1~Beta( X2~Beta(Independent

• U=X1X2

• Fix X2=x2 and get f(u|x2)

10))ln(1(18

)ln(1818018)ln(181818

18)()|()(

1011831

)/1)(/(6)()|(),(

01

)/1)(/(6)|(

/1/

103)(10)1(6)(

221

22

22

1

2

21

222

22

22

222222

22

222

21

2121

22221111

uuuuu

uuuuxuuxdxx

uudxxfxufuf

xux

uux

xxuxuxfxufxuf

xux

xuxuxuf

xdU

dXxUXxXU

xxxfxxxxf

uuuU

Problem 6.11

0

1

2

3

4

5

6

7

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

u

De

ns

ity

of

U=

X1

X2

f(u)

f(u|x2=.25)

f(u|x2=.5)

f(u|x2=.75)

Method of Moment-Generating Functions

• X,Y are two random variables

• CDF’s: FX(x) and FY(y)

• MGF’s: MX(t) and MY(t) exist and equal for |t|<h,h>0

• Then the CDF’s FX(x) and FY(y) are equal

• Three Properties:– Y=aX+b MY(t)=E(etY)=E(et(aX+b))=ebtE(e(at)X)=ebtMX(at)

– X,Y independent MX+Y(t)=MX(t)MY(t)

– MX1,X2(t1,t2) = E[et1X1+t2X2] =MX1(t1)MX2(t2) if X1,X2 are indep.

Sum of Independent Gammas

,~

)1()1()1(

)()()(

,...,1)1()(

nt)(independe,...,1),(~

11

)...(

1

11

1

11

n

ii

n

ii

XXtXtXXXttY

Y

n

ii

X

ii

GammaXY

ttt

tMtMeeEeEeEtM

XY

nittM

niGammaX

n

i in

n

nn

i

i

Linear Function of Independent Normals

n

iii

n

iii

n

iii

n

i iin

i iinn

nn

nXXXtaXtaXaXattY

Y

i

n

iii

iiX

iii

aaNormalXaY

tata

tata

tata

taMtaMeeEeEeEtM

aXaY

nit

ttM

niNormalX

n

nnnn

i

1

22

11

2

1

22

1

2221

21

11

1)...(

1

22

2

,~

2exp

2exp

2exp

)()()(

constants fixed }{

,...,12

exp)(

nt)(independe,...,1),(~

1

1111

Distribution of Z2 (Z~N(0,1))

2

1

21

0

/1

21

2

2/12/1

2/1

0

21

2/

12/1

0

21

2/

2/12

21

0

2/12/12

2

21

0

2

212/

2/

)2,2/(~t independenmutually ,...,

)2/1(

)(

:Notes

)2,2/1(~

)21()21(22

1

21

2)2/1(

2

1

2

1

2

1

2

12

5.05.02

1Let

0)about (symmetric 2

12

2

1

2

1)(

2

1)()1,0(~

2

2222

2

2

n

n

iin

y

tu

tu

tz

tz

tz

ztz

Z

zZ

nGammaZZZ

dyey

GammaZ

tt

tdueudueudze

duudzuudu

dzuzzu

dzedzedzeetM

zezfNZ

Distributions of and S2 (Normal data) X

data sampled theof sdifference theoffunction a is So

)1(2

)1(2)0)(0(2)1()1(

)1(2

1

2)1(2

1

2)1(2

1

2)1(2

1

)1(2

1)(

)1(2

1

: oftion representa eAlternativ

1 :Variance Sample

0 :Note

,...,111

:Mean Sample

dDistributetly Independen and Normal ),(~,...,

2

22

22

1 11

2

1

2

1 11

2

1

2

1 1

22

1 1

2

1 1

22

2

1

2

2

1111

11

1

21

S

Snn

SnnSnnSnn

nn

XXXXXXnXXnnn

XXXXXXnXXnnn

XXXXXXXXnn

XXXXnn

XXnn

S

S

n

XXS

XXXnXXX

nin

aXaXnn

XX

NIDNIDXX

n

i

n

jji

n

jj

n

ii

n

i

n

jji

n

jj

n

ii

n

i

n

jjiji

n

i

n

jji

n

i

n

jji

n

i i

n

ii

n

ii

n

ii

n

ii

i

n

iii

n

ii

n

i i

n

Independence of and S2 (Normal Data)X

))(exp())(exp(

))(exp())(exp(

)]()(exp[

)()(exp)(),(

}exp{2

20exp)()2,0(~

}2exp{2

22exp)()2,2(~

),(~,

212211

212211

212211

12221121,

2222

212

2222

221

221

21

ttXEttXE

ttXttXE

ttXttXE

XXtXXtEeEttM

tt

tMNXXD

ttt

ttMNXXT

NIDXX

ind

DtTtDT

D

T

Independence of T=X1+X2 and D=X2-X1 for Case of n=2

X

Independence of and S2 (Normal Data) P2X

)()(2

2exp

2

22exp

2

2

2

22exp

2

)22()(exp

2

)()(exp

2

)()(exp

))(exp())(exp(

))(exp())(exp(

21

22

221

2

1

22

221

2

1

2122

2121

22

21

2

2121

221

2

21

221

2

21

212211

212211

tMtMtt

t

ttt

tttttttttttt

tttt

tttt

ttXEttXE

ttXttXE

DT

ind

Independence of T=X1+X2 and D=X2-X1 for Case of n=2

Thus T=X1+X2 and D=X2-X1 are independent Normals and & S2 are independentX

Distribution of S2 (P.1)

2/

1)()1()()1(

2

2

2

2

22

1

2

2

1

2

1

2

2

1

22

2

2

12

2

12

2

1

22

1

21

22

)21()()()(

:tindependen are and Now,

)1(0

1

21

21

11

)2,2/(~

~)1,0(~),(~

2

122

2

2

2

2

22n

XXnSnXnSn

n

ii

n

ii

n

ii

n

iii

n

ii

n

ii

n

i

i

n

n

i

i

ii

ii

tMtMtMtM

SX

XnSnXnXX

XXXXnXX

XXXXXX

XXXXX

nGammaX

ZNX

ZNIDX

n

ii

Distribution of S2 P.2

212

2

2/)1()2/1()2/(2/1

2/

)(

1

)1(

2/1

)(

212

2

2

1

22

1

2

2

2

2/

1)()1()()1(

2

2,2

1~

)1(

)21()21()21(

)21()(

)21()(~

)1,0(~1

,1

~

: :consider Now,

)21()()()(

:tindependen are and Now,

2

2

2

12

2

2

2

2

2

122

2

2

2

2

22

n

nnn

Xn

X

Sn

Xn

X

XX

n

i

n

iXX

n

XXnSnXnSn

nGamma

Sn

ttt

t

M

M

tM

ttMXn

NXn

n

XXZ

nnnNX

Xn

tMtMtMtM

SX

n

ii

n

ii

Summary of Results• X1,…Xn ≡ random sample from N(2)population

• In practice, we observe the sample mean and sample variance (not the population values: , 2)

• We use the sample values (and their distributions) to make inferences about the population values

ons)distributi-F and for t,on presentati

.ppton ngconditioni of method using derivation (See

~)1(

)1()1(

/

/

tindependen are ,

~)1(

1,~

121

2

2

2

212

1

2

2

21

2

22

1

n

n

n

n

ii

n

ii

n

ii

tn

Z

nSn

n

X

nS

Xt

SX

XXSn

n

XXS

nNX

n

XX

Order Statistics• X1,X2,...,Xn Independent Continuous RV’s

• F(x)=P(X≤x) Cumulative Distribution Function• f(x)=dF(x)/dx Probability Density Function

• Order Statistics: X(1) ≤ X(2) ≤ ...≤ X(n) (Continuous can ignore equalities)• X(1) = min(X1,...,Xn)

• X(n) = max(X1,...,Xn)

Order Statistics

)()](1[)](1[

)](1[

])](1[1[)()(

:Minimum of pdf

)](1[1)(1),...,(1

: Minimum of CDF

)()]([)(

)]([)]([)(

)(

:Maximum of pdf

)]([)(),...,(

: Maximum of CDF

11

)1(1

11)1(

)1(

11)(

11)(

)(

xfxFndx

xFdxFn

dx

xFd

dx

xXdPxg

xFxXPxXPxXxXPxXP

X

xfxFndx

xdFxFn

dx

xFd

dx

xXdPxg

xFxXPxXPxXxXPxXP

X

nn

n

nnn

nnn

nn

nnnn

n

Example

• X1,...,X5 ~ iid U(0,1)

(iid=independent and identically distributed)

o.w.0

10)1(5)1)(1(5)( :Minimum

o.w.0

105)1(5)( :Maximum

o.w.0

101)(

11

10

00

)(

44

1

44

xxxxg

xxxxg

xxf

x

xx

x

xF

n

Order Stats - U(0,1) - n=5

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

x

pdf

f(x)

gn(x)

g1(x)

Distributions of Order Statistics• Consider case with n=4

• X(1) ≤x can be one of the following cases:

• Exactly one less than x• Exactly two are less than x• Exactly three are less than x• All four are less than x

• X(3) ≤x can be one of the following cases:

• Exactly three are less than x• All four are less than x

• Modeled as Binomial, n trials, p=F(x)

Case with n=4

))(1()()(12)()(12)()(12)(

)(3)(4

)()(4)(4

)](1[)]([4

4)](1[)]([

3

4

)](1[1

)](1[)]([4

4)](1[)]([

3

4

)](1[)]([2

4)](1[)]([

1

4

2323

43

443

043)3(

4

043

2231)1(

xFxFxfxfxFxfxFxg

xFxF

xFxFxF

xFxFxFxFxXP

xF

xFxFxFxF

xFxFxFxFxXP

General Case (Sample of size n)

elsewhere0

...)()...(!),...,(

:statisticsorder all ofon distributiJoint

)()()](1[)]()([)]([)!()!1()!1(

!

),(:1

l)multinomia (uses statsorder and ofon distributiJoint

1)()](1[)]([)!()!1(

!)(

111,...,12

11

1

nnnn

jijn

jij

iji

i

jiij

thth

jnjj

xxxfxfnxxg

xfxfxFxFxFxFjniji

n

xxgnji

ji

njxfxFxFjnj

nxg

Example – n=5 – Uniform(0,1)

10)(20),(:5,1

5)1(]1[][!0!4

!5)(:5

)1(20)1(]1[][!1!3

!5)(:4

)1(30)1(]1[][!2!2

!5)(:3

)1(20)1(]1[][!3!1

!5)(:2

)1(5)1(]1[][!4!0

!5)(:1

10)(1)(

513

155115

455155

345144

2235133

325122

415111

xxxxxxgji

xxxxgj

xxxxxgj

xxxxxgj

xxxxxgj

xxxxgj

xxxFxf

Distributions of all Order Stats - n=5 - U(0,1)

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

x

pdf

f(x)

g1(x)

g2(x)

g3(x)

g4(x)

g5(x)