chapter 5 multiple random variables - siueyadwang/ece352_lec5.pdf · 2017. 2. 20. · 21 §5.9...
TRANSCRIPT
1
Chapter 5 Multiple Random Variables§ 5.1 Joint Cumulative Distribution Function
The joint CDF FX,Y(x,y)=P[X<=x, Y<=y] is a complete probability model for any pair of random variables X and Y.
Definition 5.1 Joint Cumulative Distribution Function (CDF)
𝑭𝑿,𝒀 𝒙, 𝒚 = 𝑷[𝑿 ≤ 𝒙, 𝒀 ≤ 𝒚]
2
§ 5.1 Joint Cumulative Distribution Function
Theorem 5.1
𝑎. )0 ≤ 𝐹2,3 𝑥, 𝑦 ≤ 1
Foranypairofrandomvariables,X,Y,
𝑏. )𝐹2,3 ∞,∞ = 1
𝑐. )𝐹2 𝑥 = 𝐹2,3 𝑥, ∞ 𝑑. )𝐹3 𝑦 = 𝐹2,3 ∞, 𝑦
𝑒. )𝐹2,3 𝑥, −∞ =0 𝑓. )𝐹2,3 −∞, 𝑦 =0
g.)If x<=x1 andy<=y1,then
𝐹2,3 𝑥, 𝑦 ≤ 𝐹2,3(𝑥?, 𝑦?)
3
§ 5.1 Joint Cumulative Distribution Function
Example 5.2
Xyearsistheageofchildrenenteringfirstgradeinaschool.Yistheageofchildrenenteringsecondgrade.ThejointCDFofXandYis:
𝐹2,3 𝑥, 𝑦 =
0, 𝑥 < 50, 𝑦 < 6
𝑥 − 5 𝑦 − 6 , 5 ≤ 𝑥 < 6, 6 ≤ 𝑦 < 7𝑦 − 6, 𝑥 ≥ 6, 6 ≤ 𝑦 < 7𝑥 − 5, 5 ≤ 𝑥 < 6, 𝑦 ≥ 7
1, 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒
FinFX(x)andFY(y)
4
§ 5.1 Joint Cumulative Distribution Function
Theorem 5.2
𝑷 𝒙𝟏 < 𝑿 ≤ 𝒙𝟐, 𝒚𝟏 < 𝒀 ≤ 𝒚𝟐 = 𝑭𝑿,𝒀 𝒙𝟐, 𝒚𝟐 − 𝑭𝑿,𝒀 𝒙𝟐, 𝒚𝟏 − 𝑭𝑿,𝒀 𝒙𝟏, 𝒚𝟐 + 𝑭𝑿,𝒀 𝒙𝟏, 𝒚𝟏
Quiz 5.1
ExpressthefollowingextremevaluesofthejointCDF𝐹2,3 𝑥, 𝑦 asnumbersorintermsoftheCDF’s𝐹2 𝑥 and𝐹3 𝑦
𝑎. )𝐹2,3 −∞, 2 𝑏. )𝐹2,3 −∞,∞
𝑐. )𝐹2,3 ∞, 𝑦 𝑑. )𝐹2,3 ∞, −∞
5
§ 5.2 Joint Probability Mass Function (Discrete)
Definition 5.2 Joint Probability Mass Function (PMF)
ThejointprobabilitymassfunctionofdiscreterandomvariableXandYis
𝑃2,3 𝑥, 𝑦 = 𝑃[𝑋 = 𝑥, 𝑌 = 𝑦]
NecessaryandSufficientConditionsforaFunctiontobeadiscreteJointProbabilityMassFunction
1. 𝑃2,3 𝑥, 𝑦 ≥ 0
2. S S 𝑃2,3 𝑥, 𝑦 = 1�
UVVW
�
UVVX
6
§ 5.2 Joint Probability Mass Function (Discrete)
Example
Inanautomobileplanttwotasksareperformedbyrobots.Thefirstentailsweldingtwojoints;thesecond,tighteningthreebolts.LetXdenotethenumberofdefectiveweldsandYthenumberofimproperlytightenedboltsproducedpercar.SinceXandYareeachdiscrete,(X,Y)isatwo-dimensionaldiscreterandomvariable.PastdataindicatethatthejointmassfunctionisasshowninTable.
x/y
0
1
2
1 2 3 4
0.840
0.060
0.010
0.030
0.010
0.005
0.020
0.008
0.004
0.010
0.002
0.001
𝑃2,3 𝑥, 𝑦 =
0.840, 𝑥 = 0, 𝑦 = 00.030, 𝑥 = 0, 𝑦 = 10.020, 𝑥 = 0, 𝑦 = 20.010, 𝑥 = 0, 𝑦 = 3… , … . .
0, 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒
7
§ 5.2 Joint Probability Mass Function (Discrete)
Theorem 5.3
FordiscreterandomvariableXandYandanysetBintheX,Yplane,theprobabilityoftheevent{(X,Y)∈ 𝐵}is
𝑃 𝐵 = S 𝑃2,3(𝑥, 𝑦)�
(X,W)∈_
8
§ 5.3 Marginal Probability Mass Function (Discrete)
Fordiscreterandomvariable,themarginalPMFsPX(x)andPY(y)areprobabilitymodelsfortheindividualrandomvariablesXandY
𝑃2 𝑥 = S 𝑃2,3(𝑥, 𝑦)�
W∈`a
Theorem 5.4
FordiscreterandomvariablesXandYwithjointPMFPX,Y(x,y)
𝑃3 𝑦 = S 𝑃2,3(𝑥, 𝑦)�
X∈`b
Ychooseallther.v.fromSY
X chooseallther.v.fromSX
9
§ 5.4 Joint Probability Density Function (Continuous)
Theorem 5.5
Definition 5.3 Joint Probability Density Function (PDF)
ThejointPDFofthecontinuousrandomvariablesXandYisafunctionfX,Y(x,y)withtheproperty
𝐹2,3 𝑥, 𝑦 = c c 𝑓2,3 𝑢, 𝑣 𝑑𝑢𝑑𝑣f
gf
f
gf
𝑓2,3 𝑥, 𝑦 =𝜕i𝐹2,3(𝑥, 𝑦)
𝜕𝑥𝜕𝑦
10
§ 5.6 Independent Random Variable
Example 5.12
Definition 5.4 Independent Random Variable
RandomvariablesX andY areindependent ifandonlyif
Discrete: 𝑃2,3 𝑥, 𝑦 = 𝑃2(𝑥)𝑃3(𝑦)
Continuous: 𝑓2,3 𝑥, 𝑦 = 𝑓2(𝑥)𝑓3(𝑦)
𝑓2,3 = j4𝑥𝑦, 0 ≤ 𝑥 ≤ 1, 0 ≤ 𝑦 ≤ 10, 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒
AreXandYindependent?
11
§ 5.6 Independent Random Variable
Quiz 5.6
(A)RandomvariablesX andY,andRandomvariablesQandGhavejointPMFs:
PX,Y(x,y)
0
1
2
0 1 2
0.01
0.09
0
0
0.09
0
0
0
0.81
PQ,G(q,g)
0
1
0 1 2
0.06
0.04
0.18
0.12
0.12
0.8
AreXandYindependentAreQandG independent
(B)RandomvariablesX1andX2areindependentandidenticallydistributedwithprobabilitydensityfunction
𝑓2(𝑥) = j𝑥/2, 0 ≤ 𝑥 ≤ 20, 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒
WhatisthejointPDF 𝑓2l,2m 𝑥?, 𝑥i ?
12
§ 5.7 Expected Value of a Function of Two Random Variables
Theorem 5.9
ForRandomvariablesX andY,theexpectedvalueofW =g(X,Y)is
Discrete: 𝐸 𝑊 = S S 𝑔(𝑥, 𝑦)𝑃2,3(𝑥, 𝑦)�
W∈`a
�
X∈`b
Continuous: 𝐸 𝑊 = c c 𝑔 𝑥, 𝑦 𝑓2,3 𝑥, 𝑦 𝑑𝑥𝑑𝑦f
gf
f
gf
Theorem 5.10
𝐸 𝑎?𝑔? 𝑋, 𝑌 + ⋯+ 𝑎s𝑔s 𝑋, 𝑌 = 𝑎?𝐸 𝑔? 𝑋, 𝑌 + ⋯+ 𝑎s𝐸 𝑔s 𝑋, 𝑌
13
§ 5.7 Expected Value of a Function of Two Random Variables
Theorem 5.11
ForanytwoRandomvariablesX andY
Theorem 5.12
𝑉𝑎𝑟 𝑋 + 𝑌 = 𝑉𝑎𝑟 𝑋 + 𝑉𝑎𝑟 𝑌 + 2𝐸[(𝑋 − 𝜇2)(𝑌 − 𝜇3)]
E[X+Y]=E[X]+E[Y]
14
§ 5.7 Expected Value of a Function of Two Random Variables
Example
RandomvariablesXandYhavejointPDF
𝑓2,3(𝑥, 𝑦) = j5𝑥i/2, −1 ≤ 𝑥 ≤ 1, 0 ≤ 𝑦 ≤ 𝑥i
0, 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒
(a) WhatareE[X]andVar[X]?(b) WhatareE[Y]andVar[Y]?(c) WhatisCov[X,Y]?(d) WhatisE[X+Y]?(e) WhatisVar[X+Y]?
15
§ 5.8 Covariance, Correlation and Independence
Definition 5.5 Covariance
Thecovariance oftworandomvariablesX andY is
𝐶𝑜𝑣 𝑋, 𝑌 = 𝐸[(𝑋 − 𝜇2)(𝑌 − 𝜇3)]
Definition 5.6 Correlation Coefficient
ThecorrelationcoefficientoftworandomvariablesX andY is
𝜌2,3 =𝐶𝑜𝑣[𝑋, 𝑌]
𝑉𝑎𝑟 𝑋 𝑉𝑎𝑟[𝑌]�=𝐶𝑜𝑣[𝑋, 𝑌]𝜎2𝜎3
16
§ 5.8 Covariance, Correlation and Independence
Theorem 5.13 If𝑋y = 𝑎𝑋 + 𝑏 and𝑌y = 𝑐𝑌 + 𝑑,then
Theorem 5.14
−1 ≤ 𝜌2,3≤ 1
(a)𝜌2y,3y = 𝜌2,3 (b)𝐶𝑜𝑣[𝑋y, 𝑌y] = 𝑎𝑐𝐶𝑂𝑉[𝑋, 𝑌]
Theorem 5.15
IfXandYarerandomvariablessuchthatY=aX +b
𝜌2,3 = {−1 𝑎 < 00 𝑎 = 01 𝑎 > 0
17
§ 5.8 Covariance, Correlation and Independence
Definition 5.7
ThecorrelationofX andY is𝑟2,3 = 𝐸[𝑋𝑌]
Theorem 5.16(a)𝐶𝑜𝑣 𝑋, 𝑌 = 𝑟2,3 − 𝜇2𝜇3
(b)Var 𝑋 + 𝑌 = 𝑉𝑎𝑟 𝑋 + 𝑉𝑎𝑟 𝑌 + 2𝐶𝑜𝑣[𝑋, 𝑌]
Therelationbetweencovarianceandcorrelation
Therelationbetweenvarianceandcovariance
(c)IfX=Y,Cov 𝑋, 𝑌 = 𝑉𝑎𝑟 𝑋 = 𝑉𝑎𝑟 𝑌 ,and𝑟2,3 = 𝐸 𝑋i = 𝐸[𝑌i]
18
§ 5.8 Covariance, Correlation and Independence
Definition 5.8 Orthogonal Random Variables
RandomvariablesX andY areorthogonal if𝑟2,3 = 0
Definition 5.9 Uncorrelated Random Variables
RandomvariablesX andY areuncorrelated if𝐶𝑜𝑣[𝑋, 𝑌] = 0
19
§ 5.8 Covariance, Correlation and Independence
Theorem 5.17
(a)E 𝑔 𝑋 ℎ(𝑌) = 𝐸 𝑔 𝑋 𝐸[ℎ(𝑌)]
(b)𝑟2,3 = 𝐸 𝑋𝑌 = 𝐸 𝑋 𝐸[𝑌]
(c)Cov 𝑋, 𝑌 = 𝜌2,3 = 0
Forindependent randomvariablesXandY,
(c)Var 𝑋, 𝑌 = 𝑉𝑎𝑟 𝑋 + 𝑉𝑎𝑟[𝑌]
20
§ 5.9 Bivariate Gaussian Random Variables
Definition5.10BivariateGaussianRandomVariables
RandomvariableX and YhaveabivariateGaussianPDFwithparameters,,andsatisfyingif
fX,Y (x, y) =
exp
x −µXσ X
⎛
⎝⎜
⎞
⎠⎟2
−2ρX,Y (x −µX )(y−µY )
σ XσY+
y−µYσY
⎛
⎝⎜
⎞
⎠⎟2
2 1− ρX,Y2( )
⎡
⎣
⎢⎢⎢⎢⎢
⎤
⎦
⎥⎥⎥⎥⎥
2πσ XσY 1− ρX,Y2
µX µY σ X > 0σY > 0 ρX,Y −1< ρX,Y <1
21
§ 5.9 Bivariate Gaussian Random Variables
Theorem5.18
IfX and Y arethebivariateGaussianrandomvariableindefinition5.10,XistheGaussian(,)randomvariableandYistheGaussian(,)randomvariable:
fX (x) =1
σ X 2πexp
− x −µX( )2
2σ X2
⎛
⎝
⎜⎜
⎞
⎠
⎟⎟
µX µYσ X σY
fY (y) =1
σY 2πexp
− y−µY( )2
2σY2
⎛
⎝
⎜⎜
⎞
⎠
⎟⎟
Theorem5.19
BivariateGaussianrandomvariableX and Y indefinition5.10,havecorrelationcoefficientρX,Y
Theorem5.20
BivariateGaussianrandomvariableX and Yareuncorrelatedifandonlyiftheyareindependent
22
§ 5.9 Bivariate Gaussian Random Variables
Theorem5.21
IfX and Y arethebivariateGaussianrandomvariableindefinition5.10,andW1 andW2aregivenbythelinearlyindependentequations
W1 = a1X + b1Y
ThenW1 andW2 arebivariateGaussianrandomvariablesuchthat
W2 = a2X + b2Y
E[Wi ]= aiµX + biµY
Var[Wi ]= ai2σ X
2 + bi2σY
2 + 2aibiρX,Yσ XσY
Cov[W1,W2 ]= a1a2σ X2 + b1b2σY
2 + (a1b2 + a2b1)ρX,Yσ XσY
23
§ 5.10 Multivariate Probability Models
Definition5.11MultivariateJointCDFThejointCDF ofX1,X2,… Xn is
FX1,...,Xn(x1, x2,..., xn ) = P X1 ≤ x1,X2 ≤ x2,...,XN ≤ xn[ ]
Definition5.12MultivariateJointPMF
ThejointPMF ofthediscreterandomvariableX1,X2,… Xn is
PX1,...,Xn(x1, x2,..., xn ) = P X1 = x1,X2 = x2,...,XN = xn[ ]
Definition5.12MultivariateJointPDF
fX1,...,Xn(x1, x2,..., xn ) =
∂nFX1...Xn(x1, x2,..., xn )
∂x1∂x2...∂xn
ThejointPDF ofthecontinuousrandomvariableX1,X2,… Xn is
24
§ 5.10 Multivariate Probability Models
Theorem5.22IfX1,X2,… Xn arediscreterandomvariableswithjointPMF
PX1,...,Xn(x1, x2,..., xn ) ≥ 0
PX1,...,Xn(x1, x2,..., xn )
... PX1,...,Xn(x1, x2,..., xn ) =1
xn∈SXn
∑x1∈SX1
∑
Theorem5.23
IfX1,X2,… Xn arediscreterandomvariableswithjointPMF fX1,...,Xn(x1, x2,..., xn )
fX1,...,Xn(x1, x2,..., xn ) ≥ 0
...−∞
∞
∫ fX1,...,Xn(x1, x2,..., xn )dx1...dxn =1
−∞
∞
∫
FX1,...,Xn(x1,..., xn ) = ...
−∞
∞
∫ fX1,...,Xn(u1,u2,...,un )du1...dun
−∞
∞
∫
25
§ 5.10 Multivariate Probability Models
Theorem5.24TheprobabilityofaneventAexpressedintermsoftherandomvariablesX1,X2,… Xn
P[A]= PX1,...,Xn(x1, x2,..., xn )
(x1,...,xn )∈A∑
P[A]= ...A∫∫∫ fX1,...,Xn
(x1, x2,..., xn )dx1...dxn
Discrete
Continuous