88429283 probability random variables and random signal principles ch 3 4 p peebles

CHAPTER

THREEOPERATIONS ON ONE RANDOMVARIABLE—EXPECTATION

3.0 INTRODUCTION

The random variable was introduced in Chapter 2 as a means of providing asystematic definition of events defined on a sample space. Specifically, itformed a mathematical model for describing characteristics of some real,physical world random phenomenon. In this chapter we extend our work toinclude some important operations that may be performed on a randomvariable. Most of these operations are based on a single concept—expectation.

3.1 EXPECTATION

Expectation is the name given to the process of averaging when a randomvariable is involved. For a random variable X, we use the notation E[X],which may be read "the mathematical expectation of A\" "the expectedvalue of X" " the mean value of X," or " the statistical average of X." Occasionally we also use the notation X which is read the same way as E[X]; thatis,X = E[X].i

t Up to this point in this book an overbar represented the complement of a set or event.Henceforth, unless specifically stated otherwise, the overbar will always represent a mean value.

60

OPERATIONS ON ONE RANDOM VARIABLE—EXPECTATION 61

Nearly everyone is familiar with averaging procedures. An example thatserves to tie a familiar problem to the new concept of expectation may be theeasiest way to proceed.

Example 3.1-1 Ninety people are randomly selected and the fractionaldollar value of coins in their pockets is counted. If the count goes abovea dollar, the dollar value is discarded and only the portion from 0$ to99$ is accepted. It is found that 8, 12, 28, 22, 15, and 5 people had 18$,45$, 64$, 72$, 77$, and 95$ in their pockets, respectively.

Our everyday experiences indicate that the average of these values is

average S = 0.18(1)+ 0.45g) + 0.64g) + 0.72(|

0.77(1?) + 0.95(1)1$0,632

Expected Value of a Random VariableThe everyday averaging procedure used in the above example carries overdirectly to random variables. In fact, if X is the discrete random variable" fractional dollar value of pocket coins," it has 100 discrete values xt thatoccur with probabilities P(xt), and its expected value E[X] is found in thesame way as in the example:

100E [ X ] = X > i P ( x , ) ( 3 . 1 - 1 )l= *

The values xt identify with the fractional dollar values in the example, whileP(x{) is identified with the ratio of the number of people for the given dollarvalue to the total number of people. If a large number of people had beenused in the "sample" of the example, all fractional dollar values would haveshown up and the ratios would have approached P{xt). Thus, the average inthe example would have become more like (3.1-1) for many more than 90people.

In general, the expected value of any random variable X is defined by

E [ X ) = X = f x f x ( x ) d x ( 3 . 1 - 2 )' - CO

If X happens to be discrete with N possible values xt having probabilitiesP(xj) of occurrence, then

f x ( x ) = £ P ( x t ) S ( x - x j ( 3 . 1 - 3 )i = i

.-,■•■■

62 PROBABILITY, RANDOM VARIABLES, AND RANDOM SIGNAL PRINCIPLES

from (2.3-5). Upon substitution of (3.1-3) into (3.1-2), we haveN

E[X] = Z xtP(xi) discrete random variable (3.1-4); = i

Hence, (3.1-1) is a special case of (3.1-4) when N = 100. For some discreterandom variables, N may be infinite in (3.1-3) and (3.1-4).

Example 3.1-2 We determine the mean value of the continuous, exponentially distributed random variable for which (2.5-4) applies:

I e-(x-«)lb x>a

1 0 x < aFrom (3.1-2) and an integral from Appendix C:

E[X] = C \ e-l*-flW* dx = ~ Cxe-X'b dx = a + b' a 0 b • '

If a random variable's density is symmetrical about a line x = a, thenE[X] = a; that is,

E[X] = a i f fx(x + a)=fx(-x + a) (3.1-5)

Expected Value of a Function of a Random VariableAs will be evident in the next section, many useful parameters relating to arandom variable X can be derived by finding the expected value of a realfunction g(-) of X. It can be shown (Papoulis, 1965, p. 142) that this expected value is given by

% ( * ) ] = J * g ( x ) f x ( x ) d x ( 3 . 1 - 6 )* - 00

If X is a discrete random variable, (3.1-3) applies and (3.1-6) reduces toN

E[9{X)] = X g(Xi)P(Xi) discrete random variable (3.1-7)i = l

where N may be infinite for some random variables.

Example 3.1-3 It is known that a particular random voltage can berepresented as a Rayleigh random variable V having a density functiongiven by (2.5-6) with a = 0 and 6 = 5. The voltage is applied to a device


'2that generates a voltage Y = g(V) = V2 that is equal, numerically, tothe power in V (in a 1-Q resistor). We find the average power inV by means of (3.1-6):

Power in V = E[g(V)] = E[V2] = ( -— e~v2'5 dv•o 5

By letting £ = v2/5, d£ = 2v dv/5, we obtain00

Power in V = 5 | c^(/c = 5W•o

after using (C-46).

^Conditional Expected Value

If, in (3.1-2), fx{x) is replaced by the conditional density fx(x \ B), where B isany event defined on the sample space, we have the conditional expectedvalue of X, denoted E[X \ B]\

E [ X \ B ] = f x f x { x | B ) d x ( 3 . 1 - 8 )' - 00

One way to define event B, as shown in Chapter 2, is to let it depend onthe random variable X by defining

B = { X < b } - o o < b < o o ( 3 . 1 - 9 )We showed there that

1 /*(*) x < bf x { x \ X < b ) = < S b - a o f x { x ) d x ( 3 . 1 - 1 0 )

I 0 x > bThus, by substituting (3.1-10) into (3.1-8):

which is the mean value of X when X is constrained to the set {X < b}.

3.2 MOMENTS

An immediate application of the expected value of a function g() of arandom variable X is in calculating moments. Two types of moments are ofinterest, those about the origin and those about the mean.


Moments About the OriginThe function

g ( X ) = X " n = 0 , 1 / 2 , . . . ( 3 . 2 - 1 )when used in (3.1-6) gives the moments about the origin of the randomvariable X. Denote the nth moment by mn. Then,

m n = E [ X n ] = f x " f x ( x ) d x ( 3 . 2 - 2 )' - 00

Clearly m0 = 1, the area of the function fx{x), while ra, = X, the expectedvalue of X:

Central MomentsMoments about the mean value of X are called central moments and aregiven the symbol p.„. They are defined as the expected value of the function

g {X ) = {X - X f n = 0 ,1 ,2 , . . . ( 3 .2 -3 )which is

ft„ = E[(X - Xf] = j" (x - X)"fx(x) dx (3.2-4)' - 00

The moment fi0 = 1, the area offx(x), while y.x =0. (Why?)

Variance and SkewThe second central moment p.2 is so important we shall give it the namevariance and the special notation a\. Thus, variance is given byt

a\ = n2 = E[(X - X)2] =f(x- Xffx{x) dx (3.2-5)" —oo

The positive square root ox of variance is called the standard deviation of X;it is a measure of the spread in the function fx{x) about the mean.

Variance can be found from a knowledge of first and second moments.By expanding (3.2-5), we have:}:

o\ = E[X2 - 2XX + X2] = E[X2] - 2XE[X] + X2= E [ X 2 ] - X 2 = m 2 - m \ ( 3 . 2 - 6 )

t The subscript indicates that a\ is the variance of a random variable A". For a randomvariable Y its variance would be a\.

% We use the fact that the expected value of a sum of functions of X equals the sum ofexpected values of individual functions, as the reader can readily verify as an exercise.


Example 3.2-1 Let X have the exponential density function given inExample 3.1-2. By substitution into (3.2-5), the variance of X is

o2 = C(x-Xf\e'(x-a)lbdx• a l >

By making the change of variable c = .v-Xwe obtain-(X-a)lb co

a\ = r £2e-*lb dQ = (a + b- X)2 + b20 - a - X

after using an integral from Appendix C. However, from Example 3.1-2,X = E[X] = {a + b), so

o2x = b2The reader may wish to verify this result by finding the second momentE[X2] and using (3.2-6).

The third central moment /i3 = E[(X — X)3] is a measure of the asymmetry offx{x) about x — % — mv It will be called the skew of the densityfunction. If a density is symmetric about x = X, it has zero skew. In fact, forthis case /.i„ = 0 for all odd values of n. (Why ?) The normalized third centralmoment /*3 ja\ is known as the skewness of the density function, or, alternatively, as the coefficient of skewness.

*3.3 FUNCTIONS THAT GIVE MOMENTS

Two functions can be defined that allow moments to be calculated for arandom variable X. They are the characteristic function and the momentgenerating function.

^Characteristic FunctionThe characteristic function of a random variable X is defined by

G j r f o ) - £ [ « * * ] ( 3 . 3 - 1 )where j = yf— 1. It is a function of the real number — oo < a) < co. If (3.3-1)is written in terms of the density function, Q^a)) is seen to be the Fouriertransform^ (with the sign of co reversed) of/*(x):

4 > x H = f f x ( x ) e j " x d x ( 3 . 3 - 2 )

t Readers unfamiliar with Fourier transforms should interpret <J>x(w) as simply the expectedvalue of the function g(X) = exp (j(oX). Appendix D is included as a review for others wishingto refresh their background in Fourier transform theory.


Because of this fact, if 0*(co) is known, fx(x) can be found from the inverseFourier transform (with sign of x reversed)

fx(x) = ^-C ®x(a>)e-j(OX dw2it • _ ^ (3.3-3)

By formal differentiation of (3.3-2) n times with respect to co and settingco = 0 in the derivative, we may show that the nth moment of X is given by

, .^d"®x((o)(0 = 0

(3.3-4)

A major advantage of using Ox(ro) to find moments is that Ov(co) alwaysexists (Davenport, 1970, p. 426), so the moments can always be found ifOx(w) is known, provided, of course, the derivatives of <S>x(w) exist.

It can be shown that the maximum magnitude of a characteristic function is unity and occurs at co = 0; that is,

\<t>xW\ <<D*(0)=1 (3.3-5)(See Problem 3-24.)

Example 3.3-1 Again we consider the random variable with the exponential density of Example 3.1-2 and find its characteristic function andfirst moment.

By substituting the density function into (3.3-2), we get. o o 1 p a \ b . o c

Q>x(a)) = -e-{x-a)lbej0ix dx = —\■a b b ■e-Ulb-ja»x dx

Evaluation of the integral follows the use of an integral from AppendixC:

<&x(co) =,a/b , - { l / b - j ( o ) x

- ( l / b - j c o )

1 — jcobThe derivative of Ox(co) is

dco= eJ' j a + jb

\ - j c o b [ l - j c o b ) 2so the first moment becomes

mi = [-J d®x(co)dco

= a + b,

in agreement with mt found in Example 3.1-2.


^Moment Generating FunctionAnother statistical average closely related to the characteristic function isthe moment generating function, defined by

M x ( v ) = E [ e v X ] ( 3 . 3 - 6 )where v is a real number — oo < v < oo. Thus, Mx(v) is given by

M * ( v ) = f ° M x y d x ( 3 . 3 - 7 )

The main advantage of the moment generating function derives from itsability to give the moments. Moments are related to Mx(v) by theexpression:

d"Mx(v)w = dv" (3.3-8)

The main disadvantage of the moment generating function, as opposedto the characteristic function, is that it may not exist for all random variables. In fact, Mx(v) exists only if all of the moments exist (Davenport andRoot, 1958, p. 52).

3.4 TRANSFORMATIONS OF A RANDOM VARIABLE

Quite often one may wish to transform (change) one random variable X intoa new random variable Y by means of a transformation

Y = T { X ) ( 3 . 4 - 1 )Typically, the density function fx{x) or distribution function Fx(x) of X isknown, and the problem is to determine either the density function fY(y) ordistribution function FY(y) of Y. The problem can be viewed as a " blackbox" with input X, output Y, and "transfer characteristic" Y= T(X), asillustrated in Figure 3.4-1.

In general, X can be a discrete, continuous, or a mixed random variable.In turn, the transformation Tcan be linear, nonlinear, segmented, staircase,etc. Clearly, there are many cases to consider in a general study, dependingon the form of X and T. In this section we shall consider only three cases: (1)X continuous and T continuous and either monotonically increasing ordecreasing with X; (2) X continuous and T continuous but nonmonotonic;(3) X discrete and T continuous. Note that the transformation in all three

fxMY = T(X) "▶" Y Figure 3.4-1 Transformation of a random vari-

fY(y) able A" to a new random variable Y.


cases is assumed continuous. The concepts introduced in these three situations are broad enough that the reader should have no difficulty in extendingthem to other cases (see Problem 3-32).

Monotonic Transformations of a Continuous Random Variable

A transformation T is called monotonically increasing if T(xt) < T(x2) forany Xj < x2. It is monotonically decreasing if r(xt) > T(x2) for any xt < x2 .Such transformations are illustrated in Figure 3.4-2.

Consider first the increasing transformation. We assume that T is continuous and differentiate at all values of x for which fx(x) =£ 0. Let Y have aparticular value y0 corresponding to the particular value x0 of X as shown inFigure 3.4-2a. The two numbers are related by

y0 = T{x0) or x0= T l{y0) (3.4-2)

r = Rv)

Figure 3.4-2 Monotonic transformations: (a) increasing, and (b) decreasing. [Adapted fromPeebles (1976) with permission of publishers Addison-Wesley, Advanced Book Program.]


where T~! represents the inverse of the transformation T. Now the probability of the event {Y < y0} must equal the probability of the event {X < x0}because of the one-to-one correspondence between X and Y. Thus,

Fy(y0) =P{Y<y0} = P{X < x0} = Fx(x0) (3.4-3)or

J f r ( y ) d y = \ f x ( x ) d x ( 3 . 4 - 4 )- O O " — 0 0

Next, we differentiate both sides of (3.4-4) with respect to y0 using Leibniz'srulet to get

f y ( y o ) = f x [ T - ' ( y 0 ) ] d - 1 ^ ( 3 . 4 - 5 )

Since this result applies for any v0, we may now drop the subscript and write

f y ( y ) = f x [ T - 1 ( y ) ] ^ ^ ( 3 . 4 - 6 ).',

or, more compactly,dx

f r ( y ) = f x ( x ) T y ( 3 . 4 - 7 )In (3.4-7) it is understood that x is a function of y through (3.4-2).

A consideration of Figure 3.4-2b for the decreasing transformationverifies that

Fr(yQ) =P{Y<y0} = P{X > x0} = 1 - Fx(x0). (3.4-8)A repetition of the steps leading to (3.4-6) will again produce (3.4-6) exceptthat the right side is negative. However, since the slope of T~i(y) is alsonegative, we conclude that for either type of monotonic transformation

fr(y)=fx[T-1(y)] d T ' \ y )dy (3.4-9)

t Leibniz's rule, after the great German mathematician Gottfried Wilhelm von Leibniz(1646-1716), states that, if H(x, u) is continuous in x and u and

G{u) = I H(x, «) dx

then the derivative of the integral with respect to the parameter u is

dGU') m, > ,#(«) „r , x -,da(") tmdH(x,u)—U = H[p(u), u] -^ - H[a(u), u] -£l + —LLJ dxd u d u d u • ' , „ . d u!(»)


or simply

My)=fx(x] dxdy (3.4-10)

Example 3.4-1 If we take T to be the linear transformationY = T(X) = aX + b, where a and b are any real constants, thenX = T-1{Y)={Y- b)/a and dx/dy = l/a. From (3.4-9)

fv(y)=fx y - b

If X is assumed to be gaussian with the density function given by (2.4-1),we get

A(V)=-•

.e-l(y-b)la-ax]2/2<TX2

which is the density function of another gaussian random variablehaving

_ p-[y-(aax + b)l2/2fl2<T.v2

aY = aax + b and o\ = a2o\Thus, a linear transformation of a gaussian random variable producesanother gaussian random variable. A linear amplifier having a randomvoltage X as its input is one example of a linear transformation.

Nonmonotonic Transformations of a Continuous Random Variable

A transformation may not be monotonic in the more general case. Figure3.4-3 illustrates one such transformation. There may now be more than oneinterval of values of X that correspond to the event {Y < y0}. For the valueof y0 shown in the figure, the event {Y < y0) corresponds to the event{X < x, and x2 < X < x3}. Thus, the probability of the event {Y < y0) nowequals the probability of the event {x values yielding Y < y0}, which we shallwrite as {x \ Y < y0). In other words

Fy(y0)=P{Y<yo}=P{x\Y<y0}= f fx(x) dx (3.4-11)•U|y<yol

Formally, one may differentiate to obtain the density function of Y:d

fy(yo) = 7- \ fx(x)dx (3.4-12)


y = T(x)

Figure 3.4-3 A nonmonotonic transformation. [Adapted from Peebles (1976) with permission ofpublishers Addison-Wesley. Advanced Book Program.]

Although we shall not give a proof, the density function is also given by(Papoulis, 1965, p. 126)

fy(y) = I fx(xn)dT(x)

dx(3.4-13)

where the sum is taken so as to include all the roots xn, n = 1,2,..., whichare the real solutions of the equationt

y = T { x ) ( 3 . 4 - 1 4 )We illustrate the above concepts by an example.

Example 3.4-2 We find fY(y) for the square-law transformationy = T{X) - cX2

shown in Figure 3.4-4, where c is a real constant c > 0. We shall use boththe procedure leading to (3.4-12) and that leading to (3.4-13).

In the former case, the event {Y < y) occurs when { — y/y/c < x <s/y/c] = {x | Y < y), so (3.4-12) becomes

fy(y) = r\ " fx(*)dx y^°dy--sryTc

t If v = T(x) has no real roots for a given value of y, then/r(y) = 0.


Figure 3.4-4 A square-law transformation. [Adapted from Peebles (1976) with permission ofpublishers Addison-Wesley, Advanced Book Program.]

Upon use of Leibniz's rule we obtain

fy{y)=MJjRfisM .M.^c)d(-^)dyfx(JyTc)+fx(-JyTc)

dy

y > 02y/cy

In the latter case where we use (3.4-13), we have X = ±^jYjc,Y > 0, so xx = —s/y/c and x2 = \Jy/c. Furthermore, dT(x)/dx = 2cxso

- dT(x)dx

dT(x)

= 2cxx = —2c - = -2yfcy

dx= 2y/cy

x = x2

From (3.4-13) we again have

m=um+M^M y>02Jcy

Transformation of a Discrete Random VariableIf X is a discrete random variable while Y = T(X) is a continuous transformation, the problem is especially simple. Here

/*(*) = I P(x„)<5(x-x„)n

Fx(x) = I P(x>(x - x„)

(3.4-15)

(3.4-16)


where the sum is taken to include all the possible values x„, n = 1,2, ...,o\X.

If the transformation is monotonic, there is a one-to-one correspondence between X and Y so that a set {yn} corresponds to the set {x„} throughthe equation y„ = T(xn). The probability P(yn) equals P(x„). Thus,

f y ( y ) = l P ( y n ) H y - y n ) ( 3 . 4 - 1 7 )n

F y ( y ) = Z P ( y „ M y - y n ) ( 3 . 4 - 1 8 )n

where

y n = T ( x n ) ( 3 . 4 - 1 9 )P O O = P ( x n ) ( 3 . 4 - 2 0 )

If T is not monotonic, the above procedure remains valid except therenow exists the possibility that more than one value x„ corresponds to a valuey„. In such a case P(yn) will equal the sum of the probabilities of the variousx„ for which yn = T(x„).

PROBLEMS

3-1 A discrete random variable X has possible values x, = i2. i= 1, 2, 3, 4. 5, which occur withprobabilities 0.4, 0.25, 0.15, 0.1, and 0.1, respectively. Find the mean value X = E[X] of X.3-2 The natural numbers are the possible values of a random variable A"; that is, x„ = n, n = 1,2, .... These numbers occur with probabilities P(x„)= (Yi)". Find the expected value of A".3-3 If the probabilities in Problem 3-2 are P(xn) = p". 0 < p < 1, show that p = ]/2 is the onlyvalue of p that is allowed for the problem as formulated. (Hint: Use the fact thatl-oo fx(x) dx = 1 is necessary.)3-4 Give an example of a random variable where its mean value might not equal any of itspossible values.3-5 Find:

(a) the expected value, and(b) the variance of the random variable with the triangular density of Figure 2.3-la if

a = 1/a.3-6 Show that the mean value and variance of the random variable having the uniform densityfunction of Problem 2-12 are:

X = E[X] = (a + b)/2

and

c\ = (b - a)2/123-7 A pointer is spun on a fair wheel of chance numbered from 0 to 100 around itscircumference.

(a) What is the average value of all possible pointer positions?(b) What deviation from its average value will pointer position take on the average; that

is, what is the pointer's root-mean-squared deviation from its mean? (Hint: Use results ofProblem 3-6.)


3-8 Find:(a) the mean value, and(b) the variance of the random variable X defined by Problems 2-6 and 2-14 of Chapter 2.

*3-9 For the binomial density of (2.5-1). show that

£[A'] = X = Npand

<t\ =Np(\- p)

3-10 (a) Let resistance be a random variable in Problem 2-11 of Chapter 2. Find the meanvalue of resistance.

(b) What is the output voltage E2 if an average resistor were used in the circuit?(c) For the resistors specified, what is the mean value of £2 ? Does the voltage of part (b)

equal this value? Explain your results.3-11 (a) Use the symmetry of the density function given by (2.4-1) to justify that the parameterax in the gaussian density is the mean value of the random variable: X = ax.

(b) Prove that the parameter ax is the variance. (Hint: Use an equation fromAppendix C.)3-12 Show that the mean value E[X] and variance a2x of the Rayleigh random variable, withdensity given by (2.5-6), are

E[X] = a + JnbJAand

a\ = b(4 - n)/4

3-13 What is the expected lifetime of the system defined in Problem 2-33 of Chapter 2?3-14 Find:

(a) the mean value, and(b) the variance for a random variable with the Laplace density

J X \ i 2 b

where b and m are real constants, b > 0 and - co < m < oo.3-15 Determine the mean value of the Cauchy random variable in Problem 2-34 of Chapter 2.What can you say about the variance of this random variable?

* 3-16 For the Poisson random variable defined in Problem 2-36 of Chapter 2 show that:(a) the mean value is b and(b) the variance also equals b.

3-17 (a) Use (3.2-2) to find the first three moments m,, m2, and m3 for the exponential densityof Example 3.1-2.

(b) Find m1,m2, and m3 from the characteristic function found in Example 3.3-1. Verifythat they agree with those of part (a).3-18 Find expressions for all the moments about the origin and central moments for theuniform density (see Problem 2-12).3-19 Define a function g() of a random variable X by

10 X < x0

where .x0 is a real number — oo < x0 < co. Show that

E[g(X)] = 1 - Fx(x0)


3-20 Show that the second moment of any random variable X about an arbitrary point a isminimum when a = X; that is, show that E[(X - a)2] is minimum for a = X.3-21 For any discrete random variable X with values x, having probabilities of occurrenceP(X(), show that the moments of X are

m„ = £ x?P(x,)i= 1

n* = I (Xi - xyp(Xi)i = i

where N may be infinite for some A".3-22 Prove that central moments «„ are related to moments mk about the origin by

,,=i (:)(-*)•'»..3-23 A random variable X has a density function/v(.v) and moments m„. If the density is shiftedhigher in x by an amount a > 0 to a new origin, show that the moments of the shifted density,denoted m'n, are related to the moments m„ by

<-tC)rt* 3-24 Show that any characteristic function <I>x(a>) satisfies

| < D » | < O x ( 0 ) = l3-25 A random variable X is uniformly distributed on the interval (-5, 15). Another randomvariable Y = e~xls is formed. Find E[Y].3-26 A gaussian voltage random variable X [see (2.4-1)] has a mean value X = ax = 0 andvariance a\ = 9. The voltage X is applied to a square-law, full-wave diode detector with atransfer characteristic Y = 5A2. Find the mean value of the output voltage Y.

* 3-27 For the system having a lifetime specified in Problem 2-33 of Chapter 2, determine theexpected lifetime of the system given that the system has survived 20 weeks.

* 3-28 The characteristic function for a gaussian random variable A", having a mean value of 0. is

(fx((o) = cxp(-aW/2)Find all the moments of A using <D*(a>).

* 3-29 Work Problem 3-28 using the moment generating function

Mx(v) = cxp(a2xv2/2)for the zero-mean gaussian random variable.

* 3-30 A discrete random variable X can have N + 1 values xk = kA, k = 0, I, ..., N, whereA > 0 is a real number. Its values occur with equal probability. Show that the characteristicfunction of A" is

u ' N + 1 sin (eoA/2)

3-31 A random variable A is uniformly distributed on the interval (—n/2, n/2). X is transformed to the new random variable Y = T(X) = a tan (A), where a > 0. Find the probabilitydensity function of Y.3-32 Work Problem 3-31 if A is uniform on the interval (-n, n).


3-33 A random variable A undergoes the transformation Y = a/A", where a is a real number.Find the density function of Y.3-34 A random variable A is uniformly distributed on the interval (-a. a). It is transformed toa new variable y by the transformation Y = cX2 defined in Example 3.4-2. Find and sketch thedensity function of Y.3-35 A zero-mean gaussian random variable A is transformed to the random variable Ydetermined by

y = j c X A > 0! o A < 0

where c is a real constant, c- > 0. Find and sketch the density function of Y.3-36 If the transformation of Problem 3-35 is applied to a Rayleigh random variable witha > 0, what is its effect?

* 3-37 A random variable 0 is uniformly distributed on the interval (0U 02) where 0X and 02 arereal and satisfy

0 < 0t < 02 < n

Find and sketch the probability density function of the transformed random variableY = cos (0).3-38 A random variable X can have values -4, -1.2. 3, and 4, each with probability %. Find:

(a) the density function,(b) the mean, and(c) the variance of the random variable Y = 3A3.

CHAPTER

FOURMULTIPLE RANDOM VARIABLES

4.0 INTRODUCTION

In Chapters 2 and 3, various aspects of the theory of a single randomvariable were studied. The random variable was found to be a powerfulconcept. It enabled many realistic problems to be described in a probabilistic way such that practical measures could be applied to the problem eventhough it was random. For example, we have seen that shell impact positionalong the line of fire from a cannon to a target can be described by a randomvariable (Problem 2-29). From knowledge of the probability distribution ordensity function of impact position, we can solve for such practical measuresas the mean value of impact position, its variance, and skew. These measuresare not, however, a complete enough description of the problem in mostcases.

Naturally, we may also be interested in how much the impact positionsdeviate from the line of fire in, say, the perpendicular (cross-fire) direction. Inother words, we prefer to describe impact position as a point in a plane asopposed to being a point along a line. To handle such situations it is necessary that we extend our theory to include two random variables, one for eachcoordinate axis of the plane in our example. In other problems it may benecessary to extend the theory to include several random variables. Weaccomplish these extensions in this and the next chapter.

77


Fortunately, many situations of interest in engineering can be handledby the theory of two random variables.! Because of this fact, we emphasizethe two-variable case, although the more general theory is also stated inmost discussions to follow.

4.1 VECTOR RANDOM VARIABLES

Suppose two random variables X and Y are defined on a sample space S,where specific values of X and Y are denoted by x and y, respectively. Thenany ordered pair of numbers (x, y) may be conveniently considered to be arandom point in the xy plane. The point may be taken as a specific value of avector random variable or a random vector.̂ Figure 4.1-1 illustrates the mapping involved in going from S to the xy plane.

The plane of all points (x, y) in the ranges of X and Y may be considereda new sample space. It is in reality a vector space where the components ofany vector are the values of the random variables X and Y. The new space

t In particular, it will be found in Chapter 6 that such important concepts as autocorrelation, cross-correlation, and covariance functions, which apply to random processes, are basedon two random variables.

1 There are some specific conditions that must be satisfied in a complete definition of arandom vector (Davenport. 1970. Chapter 5). They are somewhat advanced for our scope andwe shall simply assume the validity of our random vectors.

Sj

-j(A(s,). >'(*,))

Function X

Figure 4.1-1 Mapping from the sample space S to the joint sample space Sj (xy plane).

MULTIPLE RANDOM VARIABLES 79

Figure 4.1-2 Comparisons of events in S with those in Sj.

has been called the range sample space (Davenport, 1970) or the two-dimensional product space. We shall just call it & joint sample space and give itthe symbol Sj.

As in the case of one random variable, let us define an event A by

A = {X < x\

A similar event B can be defined for Y:(4.1-1)

B = {Y<y} (4.1-2)Events A and B refer to the sample space S, while events {X < x} and{Y <y} refer to the joint sample space Sj.i Figure 4.1-2 illustrates thecorrespondences between events in the two spaces. Event A corresponds toall points in Sj for which the X coordinate values are not greater than x.Similarly, event B corresponds to the Y coordinate values in Sj not exceeding y. Of special interest is to observe that the event A n B defined on 5corresponds to the joint event {X < x and Y < y} defined on Sj, which wewrite {X < x, Y < y}. This joint event is shown crosshatched in Figure 4.1-2.

In the more general case where TV random variables Xx, X2,..., XN aredefined on a sample space S, we consider them to be components of anN-dimensional random vector or N-dimensional random variable. The jointsample space S, is now A/-dimensional.

t Do not forget that elements s of S form the link between the two events since by writing{A < x} we really refer to the set of those s such that X(s) < x for some real number x. A similarstatement holds for the event {Y < y}.


4.2 JOINT DISTRIBUTION AND ITS PROPERTIES

The probabilities of the two events A = {X < x} and B = {Y < y} havealready been defined as functions of x and y, respectively, called probabilitydistribution functions:

F x ( x ) = P { X < x } ( 4 . 2 - 1 )F Y ( y ) = P { Y < y } ( 4 . 2 - 2 )

We must introduce a new concept to include the probability of the jointevent {X < x, Y < y}.

Joint Distribution FunctionWe define the probability of the joint event {X < x, Y < y}, which is afunction of the numbers x and y, by a joint probability distribution junctionand denote it by the symbol Fx> Y(x, y). Hence,

F x , Y ( x , y ) = P { X < x , Y < y ] ( 4 . 2 - 3 )It should be clear that P{X < x, Y <y} = P(A n B), where the joint eventA n B is defined on S.

To illustrate joint distribution, we take an example where both randomvariables X and Y are discrete.

Example 4.2-1 Assume that the joint sample space Sj has only threepossible elements: (1, 1), (2, 1), and (3, 3). The probabilities of theseelements are assumed to be P(l, 1) = 0.2, P(2, 1) — 0.3, and P(3,3) = 0.5. WefindFx,r(x, y).

In constructing the joint distribution function, we observe that theevent {X < x, Y < y} has no elements for any x < 1 and/or y < 1. Onlyat the point (1, l)does the function assume a step value. So long as x > 1and y > 1, this probability is maintained so that Fx, y(x, y) has a stairstep holding in the region x > 1 and y > 1 as shown in Figure 4.2-la.For larger x and y, the point (2, 1) produces a second stair step ofamplitude 0.3 which holds in the region x > 2 and y > 1. The secondstep adds to the first. Finally, a third stair step of amplitude 0.5 is addedto the first two when x and y are in the region x > 3 and y > 3. The finalfunction is shown in Figure 4.2-la.

The preceding example can be used to identify the form of the jointdistribution function for two general discrete random variables. Let X haveN possible values x„ and Y have M possible values ym, then

Fx. r(x, y) = £ I P(xn, ym)u(x - xn)u(y - vm) (4.2-4)n = 1 m = 1


/•".v. rix.y)

A. W-v. i> 0.5 :

Figure 4.2-1 A joint distributionfunction (a), and its corresponding joint density function (b) thatapply to Examples 4.2-1 and 4.2-2.

where P(x„, ym) is the probability of the joint event {X — x„, Y = ym} andu(-) is the unit-step function. As seen in Example 4.2-1, some couples (x„,ym)may have zero probability. In some cases N or M, or both, may be infinite.

If Fx y(x, y) is plotted for continuous random variables X and Y, thesame general behavior as shown in Figure 4.2-la is obtained except thesurface becomes smooth and has no stairstep discontinuities.

For N random variables X„,n = 1,2,..., N, the generalization of (4.2-3)is direct. The joint distribution function, denoted by FXl Xl Xtl(xi, x2,...,xN), is defined as the probability of the joint event {Xt < xl5 X2< x2, ...,xs<xNy.FXl,x2 xN(xu x2,...,xN)= P{X, < x„ X2<x2,...,XN< xN] (4.2-5)

For a single random variable X, we found in Chapter 2 that Fx(x) couldbe expressed in general as the sum of a function of stairstep form (due to thediscrete portion of a mixed random variable X) and a function that wascontinuous (due to the continuous portion of X). Such a simple decomposition of the joint distribution when N > 1 is not generally true [Cramer, 1946,Section 8.4]. However, it is true that joint density functions in practice oftencorrespond to all random variables being either discrete or continuous.Therefore, we shall limit our consideration in this book almost entirely tothese two cases when N > 1.


Properties of the Joint DistributionA joint distribution function for two random variables X and Y has severalproperties that follow readily from its definition. We list them:(1) F*.y(-oo, -oo) = 0 Fx,y(-oo,y) = 0 FXt Y{x, - oo) = 0 (4.2-6a)( 2 ) ^ , ( 0 0 , 0 0 ) - ! ( 4 . 2 - 6 6 )( 3 ) 0 < F X t y ( x , y ) < 1 ( 4 . 2 - 6 c )(4) Fx, y(x, y) is a nondecreasing function of both x and y (4.2-6d)(5) FXt Y(x2, y2) + FXt y(x,, y,) - FXi Y(xi, y2) - FXi Y(x2, yj

= P{x, < X < x2, y, < Y < y2} > 0 (4.2-6e)

(6) F.v. y(x, co) = Fx(x) FXi v(co, y) = FY(y) (4.2-6/)The first five of these properties are just the two-dimensional extensions

of the properties of one random variable given in (2.2-2). Properties 1,2, and5 may be used as tests to determine whether some function can be a validdistribution function for two random variables X and Y (Papoulis, 1965, p.169). Property 6 deserves a few special comments.

Marginal Distribution FunctionsProperty 6 above states that the distribution function of one random variable can be obtained by setting the value of the other variable to infinity inFx, y{x, y)- The functions Fx(x) or FY(y) obtained in this manner are calledmarginal distribution functions.

To justify property 6, it is easiest to return to the basic events A and B,defined by A = {X < x) and B = {Y < y], and observe that Fx Y(x, y) =P{X < x, Y < y} = P(A n B). Now if we set y to co, this is equivalent tomaking B the certain event; that is, B = {Y < oo} = S. Furthermore, sinceAnB=AnS = A, then we have Fx, y(x, oo) = P{A n S) = P{A) =P{X < x} = Fx(x). A similar proof can be stated for obtaining FY(y).

Example 4.2-2 We find explicit expressions for Fx Y(x, y), and the marginal distributions Fx(x) and FY(y) for the joint sample space ofExample 4.2-1.

The joint distribution derives from (4.2-4) if we recognize that onlythree probabilities are nonzero:

F*.y(x,y) = P(l, l)u(x-lMy-l)+ P(2, l)u(x-2Hy- 1)+ P(3, 3)u(x - 3)u{y - 3)

■\>


Fx(x)1.0 - 1.0

0.5 _ 0.5

0.2

FY(y)1.0

0.5

(«)

0.5

(.b)

— Figure 4.2-2 Marginal distributionsy applicable to Figure 4.2-1 and Example

4.2-2: (a)Fx(x) and (b) Fy(y).

where P(l, 1) = 0.2, P(2, 1) = 0.3, and P(3, 3) = 0.5. If we set y = oo:

Fx(x) = Fx. y(x, co) = P(l, l)u(x - 1) + P(2, l)u(x - 2)+ P(3, 3)n(x - 3)

= 0.2«(x - 1) + 0.3tv(x - 2) + 0.5u(x - 3)If we set x = oo:

Fy{y) = Fx_ y(oo, y) = 0.2w(y - 1) + 0.3u(y - 1) + 0.5w(y - 3)= 0.5u{y- 1) + 0.5«(y - 3)

Plots of these marginal distributions are shown in Figure 4.2-2.

From an N-dimensional joint distribution function we may obtain ak-dimensional marginal distribution function, for any selected group of k of theN random variables, by setting the values of the other N — k random variables to infinity. Here k can be any integer 1, 2, 3, ..., N - 1.

4.3 JOINT DENSITY AND ITS PROPERTIES

In this section the concept of a probability density function is extended toinclude multiple random variables.


Joint Density FunctionFor two random variables X and Y, the joint probability density junction,denoted fXt Y(x, y), is defined by the second derivative of the joint distribution function wherever it exists:

We shall refer often tofx< Y(x, y) as the joint density function.If X and Y are discrete random variables, FXt Y(x, y) will possess step

discontinuities (see Example 4.2-1 and Figure 4.2-1). Derivatives at thesediscontinuities are normally undefined. However, by admitting impulsefunctions (see Appendix A), we are able to define fXt Y(x, y) at these points.Therefore, the joint density function may be found for any two discreterandom variables by substitution of (4.2-4) into (4.3-1):

fx. y(x, y)=i I P(xn, ym) S(x - xn) S(y - ym) (4.3-2)n= 1 m = 1

An example of the joint density function of two discrete random variables isshown in Figure 4.2-lb.

When N random variables Xlf X2, ..., XN are involved, the joint density function becomes the N-fold partial derivative of the iV-dimensionaldistribution function:

f ( y y v I — - F * i - X 2 x N { x i , x 2 , . . . , x N )■ ' " '■ " ' * « { X u * • • - ; x " > ~ a x . & c - a x , ( 4 - 3 " 3 )

By direct integration this result is equivalent toFXl.x2 xs(xi,x2,...,xs)

= | •••|2 I 'a,.v2 x,(c,,C2,-.-^,v)^i^2---^.v (4.3-4)- o o - 0 0

Properties of the Joint DensitySeveral properties of a joint density function may be listed that derive fromits definition (4.3-1) and the properties (4.2-6) of the joint distributionfunction:

( l ) f x . r ( x , y ) > 0 ( 4 3 - 5 a ). 0 0 , 0 0

( 2 ) I I f x , y ( x , y ) d x d y = l ( 4 . 3 - 5 6 )* - oc ' - 00

(3) Fx, Y(x, y) = f f A. tfu c2) dii dc2 (4.3-5c)- o o - 0 0


(4) Fx(x) = f | °° fXt Y(^, c2) d£2 «, (4.3-54)* - 00 ' — 00

F y ( y ) = f f f x . r & , W ^ i d £ 2 ( 4 . 3 - 5 e )' - oo * - 00

(5) P{x, < X < x2, y, < 7 < y2} = [" f*/*, y(x, y) rfx dy (4.3-5/). 00

( 6 ) A W = j f x A * * y ) * y ( 4 - 3 - 5 # )' - 00

. oo

A W = I A . y ( x , y ) d x ( 4 . 3 - 5 / j )' - 00

Properties 1 and 2 may be used as sufficient tests to determine if somefunction can be a valid density function. Both tests must be satisfied (Pap-oulis, 1965, p. 169).

The first five of these properties are readily verified from earlier workand the reader should go through the necessary logic as an exercise.Property 6 introduces a new concept.

Marginal Density FunctionsThe functions fx{x) and fY(y) of property 6 are called marginal probabilitydensity functions or just marginal density functions. They are the densityfunctions of the single variables X and Y and are defined as the derivatives ofthe marginal distribution functions:

A W = ^ < 4 ' 3 - 6 >

m - * & ( « - 7 )By substituting (4.3-5d) and (4.3-5<?) into (4.3-6) and (4.3-7), respectively, weare able to verify the equations of property 6.

We shall illustrate the calculation of marginal density functions from agiven joint density function with an example.

Example 4.3-1 We find fx(x) and/y(y) when the joint density function isgiven by (Clarke and Disney, 1970, p. 108):

fx,y(x,y)=u(x)u(y)xe-x<y+»


From (4.3-5(7) and the above equation:

fx(x) = | u{x)xe-x{y+1) dy = u{x)xe~x f e'xy dy" 0 " 0

= u(x)xe~x(l/x) = ^(x)^"*

after using an integral from Appendix C.From (4.3-5/i):

JY(y)=Cu(y)xe-x^^dx = -^y2

after using another integral from Appendix C.

For N random variables X,, X2, ..., Xs, the k-dimensional marginaldensity function is defined as the /c-fold partial derivative of the k-dimensional marginal distribution function. It can also be found from thejoint density function by integrating out all variables except the k variablesof interest Xlf X2, ..., Xk\

fxux2 X*(X1> x2,..., xk). 0 0 . 0 0

= | ■•' fxltx2 xN(xi,x2 xN)dxk+1dxk + 2---dxN (4.3-8)

4.4 CONDITIONAL DISTRIBUTION AND DENSITY

In Section 2.6, the conditional distribution function of a random variable X,given some event B, was defined as

F ^ | B ) = P { ^ x | B ) = ^ ^ ( 4 . 4 - , )

for any event B with nonzero probability. The corresponding conditionaldensity function was defined through the derivative

M x \ B ) = d - l M ( 4 4 . 2 )In this section these two functions are extended to include a second randomvariable through suitable definitions of event B.

Conditional Distribution and Density—Point ConditioningOften in practical problems we are interested in the distribution function ofone random variable X conditioned by the fact that a second random var-


iable Y has some specific value y. This is called point conditioning and we canhandle such problems by defining event B by

B = {y - Ay < Y < y + Ay} (4.4-3)where Ay is a small quantity that we eventually let approach 0. For thisevent, (4.4-1) can be written

U*\y - to < Y<y + Ay) = f;-% *';{*f£wi*' ̂ <4-4"4>}y-AyjY\Q) "C

where we have used (4.3-5/) and (2.3-6d).Consider two cases of (4.4-4). In the first case, assume X and Y are both

discrete random variables with values x,-,i= l,2,...,N, and y,J = 1,2,...,M, respectively, while the probabilities of these values are denoted P(x,) andP{y}), respectively. The probability of the joint occurrence of x, and y} isdenoted P(x,, y,). Thus,

f y ( y ) = l P ( y j ) H y - y j ) ( 4 . 4 - 5 )j = i

N M

fx. y(x, y) = I I P{xt, yj) S(x - xt) S(y - y,) (4.4-6)i = l j = l

Now suppose that the specific value of y of interest is yk. With substitutionof (4.4-5) and (4.4-6) into (4.4-4) and allowing Ay -> 0, we obtain

Fit* I r - ft) = I ^yL> «(* - x,) (4.4-7)

After differentiation we have

f x ( x \ Y = y k ) = i % ^ - } 5 ( x - x , . ) ( 4 . 4 - 8 )i = i P(yk

Example 4.4-1 To illustrate the use of (4.4-8) assume a joint densityfunction as given in Figure 4.4-la. Here P(xi5 yl)=Vis, P(x2, yt) — %5,etc. Since P{y3) = (%5) + (5/15) = 9/15, use of (4.4-8) will givefx{x\ Y = y3) as shown in Figure 4.4-16.

The second case of (4.4-4) that is of interest corresponds to X and Yboth continuous random variables. As Ay -* 0 the denominator in (4.4-4)becomes 0. However, we can still show that the conditional densityfx(x \Y = y) may exist. If Ay is very small, (4.4-4) can be written as

VI \ A ^V^ , A \ J -oo hMk lA^ l 2Ay <AAQ\F x ( x \ y - A y < Y < y + A y ) = > j ^ - ^ ( 4 . 4 - 9 )


/.v. >(v. V)

fxix\Y=y3)

.

Figure 4.4-1 A joint density function (a) and a conditional density function (b) applicableto Example 4.4-1.

and. in the limit as Ay -» 0

FMY=y),^i^m (4.4-10)

for every y such that fY(y) =£ 0. After differentiation of both sides of (4.4-10)with respect to x:

M * \ r - y ) = £ ^ ( 4 . 4 - . i )When there is no confusion as to meaning, we shall often write (4.4-11) as

fx, v{x, y)

It can also be shown thatfx(x\y) =

fy(y\x) =

fy(}')

fx. y{x, y)fx(x)

(4.4-12)

(4.4-13)


Example 4.4-2 We find fY(y | x) for the density functions defined inExample 4.3-1. Since

fx.y(x,y)=u(x)u(y)xe-x<> + »and

jx(x)=u(x)e-xare nonzero only for 0 < y and 0 < x,fY{y\x) is nonzero only for 0 < yand 0 < x. It is

jy(y\x)=u(x)u(y)xe-x>-from (4.4-13).

^Conditional Distribution and Density—Interval ConditioningIt is sometimes convenient to define event B in (4.4-1) and (4.4-2) in terms ofa random variable Y by

B = { y a < Y < y b } ( 4 . 4 - 1 4 )

where yfl and yb are real numbers and we assume P(B) = P{ya < Y < yb} =/= 0.With this definition it is readily shown that (4.4-1) and (4.4-2) become

F x { x \ y a < Y < y b ) - F Y ( y h ) _ F r M

_&Ax -» f x . y ( c , y )d l dy& A x - . f x . y { x , y ) d x d y * * • * - " >

and

f(x\v < Y < v \ - ^ ' fx- y(X' y) dy (A A 1MJ x \ x \ y a < r < y b j - t — j ^ — - — . ( 4 . 4 - 1 6 )

These last two expressions hold for X and Y either continuous ordiscrete random variables. In the discrete case, the joint density is given by(4.3-2). The resulting distribution and density will be defined, however, onlyfor ya and yb such that the denominators of (4.4-15) and (4.4-16) are nonzero.This requirement is satisfied so long as the interval ya< y < yb spans at leastone possible value of Y having a nonzero probability of occurrence.

An example will serve to illustrate the application of (4.4-16) when Xand Y are continuous random variables.

Example 4.4-3 We use (4.4-16) to find./A-(x | Y < y) for the joint densityfunction of Example 4.3-1. Since we have here defined B = {Y < y], thenya = - °o and yb = y. Furthermore, since fXi Y(x, y) is nonzero only for


0 < x and 0 < y, we need only consider this region of x and y in findingthe conditional density function. The denominator of (4.4-16) can bewritten as $>!_„ fY(q) dt By using results from Example 4.3-1:

fr(Z) dQ =cy u(Q)dc; ,->• d£U ( C + 1 ) 2 J o K + 1 ) 2 y + i

y >0

and zero for y < 0, after using an integral from Appendix C. The numerator of (4.4-16) becomes

I / j .r fef l t f- | u(x)xe-x«+»dc• - o o - 0

= u(x)xe~x [ e'xi dt;•o

= u{x)e~x{l-e-xy) y>0

and zero for y < 0, after using another integral from Appendix C. Thus

fx(x\Y <y)= u(x)u(y)U^y*(l - e~xy)

This function is plotted in Figure 4.4-2 for several values of y.

fx(x\Y<y)

0 0 . 5 1 . 0 1 . 5 2 . 0 2 . 5 3 . 0

.v

Figure 4.4-2 Conditional probability density functions applicable to Example 4.4-3.


4.5 STATISTICAL INDEPENDENCE

It will be recalled from (1.5-3) that two events A and B are statisticallyindependent if (and only if)

P { A n B ) = P ( A ) P { B ) ( 4 . 5 - 1 )This condition can be used to apply to two random variables X and Y bydefining the events A = {X < x} and B = {Y < y} for two real numbers xand y. Thus, X and Y are said to be statistically independent random variables if (and only if)

P{X <x,Y<y} = P{X< x}P{ Y<y] (4.5-2)From this expression and the definitions of distribution functions, it

follows that

F x . y ( x , y ) = F x ( x ) F Y ( y ) ( 4 . 5 - 3 )if X and Y are independent. From the definitions of density functions, (4.5-3)

f x . y ( x , y ) = f x ( x ) f Y ( y ) ( 4 . 5 - 4 )

by differentiation, if X and Y are independent. Either (4.5-3) or (4.5-4) mayserve as sufficient definitions of, or tests for, independence of two randomvariables.

The form of the conditional distribution function for independent eventsis found by use of (4.4-1) with B = {Y < y}:

By substituting (4.5-3) into (4.5-5), we have

F x ( x \ Y < y ) = F x ( x ) ( 4 . 5 - 6 )In other words, the conditional distribution ceases to be conditional andsimply equals the marginal distribution for independent random variables.It can also be shown that

F Y ( y \ X < x ) = F Y ( y ) ( 4 . 5 - 7 )Conditional density function forms, for independent X and Y, are found

by differentiation of (4.5-6) and (4.5-7):f x ( x \ Y < y ) = f x ( x ) ( 4 . 5 - 8 )f r ( y \ X < x ) = f Y ( y ) ( 4 . 5 - 9 )


Example 4.5-1 For the densities of Example 4.3-1:

fx,y(x,y)=u(x)u(y)xe-x<y+»e~x

fx{x)fY{y) = u{x)u{y) ±fx> Y(x, y)

Therefore the random variables X and Y are not independent.

In the more general study of the statistical independence of N randomvariables Xu X2, ..., XN,wc define events At by

A i = { X i < x j i = l , 2 , . . . , N ( 4 . 5 - 1 0 )where the xt are real numbers. With these definitions, the random variablesXt are said to be statistically independent if (1.5-6) is satisfied.

It can be shown that if X u X2,..., XN are statistically independent thenany group of these random variables is independent of any other group.Furthermore, a function of any group is independent of any function of anyother group of the random variables. For example, with N = 4 randomvariables: X4 is independent of X3 + X2 + Xt; X3 is independent ofX2 + Xlt etc. (see Papoulis, 1965, p. 238).

4.6 DISTRIBUTION AND DENSITY OF A SUM OFRANDOM VARIABLES

The problem of finding the distribution and density functions for a sum ofstatistically independent random variables is considered in this section.

Sum of Two Random VariablesLet W be a random variable equal to the sum of two independent randomvariables X and Y:

W = X + Y ( 4 . 6 - 1 )This is a very practical problem because X might represent a random signalvoltage and Y could represent random noise at some instant in time. Thesum W would represent a signal-plus-noise voltage available to somereceiver.

The probability distribution function we seek is defined by

Fw{w) = P{ W < w) = P{X + Y < w} (4.6-2)Figure 4.6-1 illustrates the region in the xy plane where x -f y < w. Nowfrom (4.3-5/), the probability corresponding to an elemental area dx dy in


■

x Figure 4.6-1 Region in xy plane wherex + y < w.

the xy plane located at the point (x, y) isfXt y(x, y) dx dy. If we sum all suchprobabilities over the region where x + y < w we will obtain Fw(w). Thus

. 0 0 . w — y

Fw{w) = I | fx, Y{x, y) dx dy' - oo X— - 00

and, after using (4.5-4):, o o . w - > •

FwH = I A(y) | fx{x) dx dy" — 0 0 ' x = - 0 0

(4.6-3)

(4.6-4)

By differentiating (4.6-4), using Leibniz's rule, we get the desired densityfunction

. 0 0

fw (w) = I f y {y ) f x {w - y ) dy (4 .6 -5 )* - 00

This expression is recognized as a convolution integral. Consequently, wehave shown that the density function of the sum of two statistically independent random variables is the convolution of their individual density functions.

Example 4.6-1 We use (4.6-5) to find the density of W = X + Y wherethe densities of X and Y are assumed to be

fx(x) = -[u(x)-u(x-a)]

fyiy)=\ [«M- «(?-&)]


fx(x)

Ia

(a)

/y(.i')

fw(w)

b

.

(O .

Figure 4.6-2 Two density functions (a) and (b) and their convolution (c).

with 0 < a < b, as shown in Figure 4.6-2a and b. Now because 0 < Xand 0 < Y, we only need examine the case W = X + Y > 0. From(4.6-5) we write

r00 ifw{w) = J -7 ["(y) - "(y - &)][«(w -y)-u{w-y- a)] dy

= -£ J t1 - "(y - ^)][«(w - y) - "(w - y - a)] ^y

ab u(w — y) dy — u(w — y — a) dy

- | u(y - b)u(w — y)dy + u(y — b)u(w - y - a) dy

All these integrands are unity; the values of the integrals are determined


by the unit-step functions through their control over limits of integration. After straightforward evaluation we get

w / a b 0 < w < a

l / b a < w < bfw{w) = {

(a + b — w)/ab b <w < a + b0 w > a + b

which is sketched in Figure 4.6-2c.

*Sum of Several Random Variables>When the sum Y of N independent random variables Xx, X2,..., XN is to beconsidered, we may extend the above analysis for two random variables. LetYj = Xt + X2. Then we know from the preceding work that fYi(yi) =

fx2{xi) */x,(x1).t Next, we know that X3 will be independent ofYi = X{ + X2 because X3 is independent of both Xx and X2. Thus, byapplying (4.6-5) to the two variables X3 and Y, to find the density functionof Y2 = X3+ Yu we get

fy2 = Xi + x2 + x3{y2) = fx3\x3) *fyl = xl + x2(yi)= fx3(x3) *fx2(x2) *fxi(xl) (4.6-6)

By continuing the process we find that the density function of Y= Xx +X2 + •■• + XN is the (N — l)-fold convolution of the N individual densityfunctions:

My) = fxN{xN) * fxN- l (xs- i ) * - *FXl (x1) (4 .6-7)The distribution function of Y is found from the integral of/y(y) using

(2.3-6c).

PROBLEMS

4-1 Two events A and B defined on a sample space S are related to a joint sample space throughrandom variables X and V and are defined by A = {X < x] and B = {y, < Y < y2}. Make asketch of the two sample spaces showing areas corresponding to both events and the eventA n B = {X < x, yi < Y < y2}.4-2 Work Problem 4-1 for the two events A = {.x^ < X < x2] and B = {y, < Y < y2).4-3 Work Problem 4-1 for the two events A = {x, < X <x2 or x3 < X < x4} andB = {yi<Y<y2}.

t The asterisk denotes convolution.


4-4 Three events A, B, and C satisfy C <= B <= A and are defined by A = {X < xa, Y < >•„},B = {X < xb, Y < yb], and C = {X <, xc, Y < yc} for two random variables X and Y.

(a) Sketch the two sample spaces S and Sj and show the regions corresponding to thethree events.

(b) What region corresponds to the event A n B n C?4-5 A joint sample space for two random variables X and Y has four elements (1, 1), (2, 2),(3, 3), and (4, 4). Probabilities of these elements are 0.1, 0.35, 0.05, and 0.5 respectively.

(a) Determine through logic and sketch the distribution function Fx y(.v, y).(b) Find the probability of the event {X < 2.5, Y < 6}.(c) Find the probability of the event {X < 3}.

4-6 Write a mathematical equation for Fx r(x, y) of Problem 4-5.4-7 The joint distribution function for two random variables X and Y is

Fx. Ax, y) = «(#|i - e-" - r" + e-«**»\where u(-) is the unit-step function and a > 0. Sketch Fx y(x. y).4-8 By use of the joint distribution function in Problem 4-7. and assuming a = 0.5 in each case.find the probabilities:

(a) P{X < 1. Y < 2} (b) P{0.5 < X < 1.5} (c) P{ - 1.5 < X < 2. 1 < Y < 3}.4-9 Find and sketch the marginal distribution functions for the joint distribution function ofProblem 4-5.4-10 Find and sketch the marginal distribution functions for the joint distribution function ofProblem 4-7.4-11 Given the function

Gx,y(x,y) = u(x)u(y)[\ -,-<*♦*>]Show that this function satisfies the first four properties of (4.2-6) but fails the fifth one. Thefunction is therefore not a valid joint probability distribution function.4-12 Random variables X and Y are components of a two-dimensional random vector andhave a joint distribution

0 X < 0 o r y < 0

xy 0 < .v < 1 and 0 < y < 1

Fx. r(x, >') =< x 0 < x < 1 and 1 < y

y 1 < x a n d 0 < y < 11 1 < x a n d 1 < y

(a) Sketch FXi r(x. y).(b) Find and sketch the marginal distribution functions Fx(x) and FY(y).

4-13 Show that the function

r « v P x < yI I x > y

cannot be a valid joint distribution function. [Hint: Use (4.2-6i-).]4-14 A fair coin is tossed twice. Define random variables by: X = "number of heads on thefirst toss" and V = "number of heads on the second toss" (note that X and Y can have onlythe values 0 or 1).

(a) Find and sketch the joint density function of X and V.(b) Find and sketch the joint distribution function.


4-15 A joint probability density function is

) l/ab 0 < x < a and 0 < y < b| 0 e l s e w h e r e

Find and sketch Fx y(x, y).4-16 If a < b in Problem 4-15, find:

(a) P{X + Y < 3a/4} (b) P{Y < 2bX/a}.4-17 Find the joint distribution function applicable to Example 4.3-1.4-18 Sketch the joint density function fx ,(.v, y) applicable to Problem 4-5. Write an equationtocfx,r(x,y).4-19 Determine the joint density and both marginal density functions for Problem 4-7.4-20 Find and sketch the joint density function for the distribution function in Problem 4-12.4-21 (a) Find a constant b (in terms of a) so that the function

Jfeg-(*+J>) 0 < x < a and 0 < y < oo/ * . v ( x , y ) = ) 0 e l s e w h e r e

is a valid joint density function.(b) Find an expression for the joint distribution function.

4-22 (a) By use of the joint density function of Problem 4-21, find the marginal densityfunctions.

(b) What is P{0.5a < X < 0.75a} in terms of a and />?4-23 Determine a constant b such that each of the following are valid joint density functions:

|3xy 0 < x < 1 and 0 < y < bM ' * * * r t - ( o e l s e w h e r e

t b \ r i \ = ( M I - J O 0 < . x < 0 . 5 a n d 0 < y < 11 0 e l s e w h e r e

\ _ \ b ( x 2 + 4 f ) 0 < | x | < l a n d 0 < y < 2| 0 e l s e w h e r e .

*4-24 Given the function

hi**'r+,,!)/8" xi+yi<h1 0 e l s e w h e r e

(a) Find a constant h so that this is a valid joint density function.(b) Find P{0.5b < X2 + Y2 < 0.8b}. (Hint: Use polar coordinates in both parts.)

*4-25 On a firing range the coordinates of bullet strikes relative to the target bull's-eye arerandom variables X and Y having a joint density given by

fx-*ix'y) 5?-Here a2 is a constant related to the accuracy of manufacturing a gun's barrel. What value of a2will allow 80% of all bullets to fall inside a circle of diameter 6 cm? (Hint: Use polarcoordinates.)4-26 Given the function

l b ( x + y f - 2 < . \ < 2 a n d - 3 < y < 3J x M X , y ) ' L e l s e w h e r e


(a) Find the constant b such that this is a valid joint density function.(/>) Determine the marginal density functions fx(x) and/,(y).

4-27 Find the conditional density functions fx(x\yl),fx(x\y2),fY(y\xl), and fy(y\x2) for thejoint density defined in Example 4.4-1.4-28 Find the conditional density function fx(x \ y) applicable to Example 4.4-2.4-29 By using the results of Example 4.4-2, calculate the probability of the event{Y <2\X = 1}.4-30 Random variables X and Y are jointly gaussian and normalized if

fx. r{x, y) = ~—7= , exp2nv'l - pl

x2 — 2pxy + y22(1 -P2)

(a) Show that the marginal density functions are

where — 1 < p < 1

fxix) = -4= exp (-x2/2) fY(y) = -L exp (-y2/2)' l i t v / 2 t c

(Hint: Complete the square and use the fact that the area under a gaussian density isunity.)

(b) Are X and Y statistically independent?4-31 By use of the joint density of Problem 4-30, snow that

(x -py )2f x ( x \ Y = y ) = e x p

7271(1 - p2) 2(1 -P2)4-32 Given the joint distribution function

Fx, *{x, y) = «(x)«(y)[l - e-" - e"" + *"*+*i\find:

(a) The conditional density functions /"x(.x| Y = y) and fy(y\X = x).(b) Are the random variables X and Y statistically independent?

4-33 For two independent random variables A" and Y show that

P{Y<X}= f Fy(x) fx (x)dx

P { y < X } = l - | F x ( y ) f y ( y ) d y

4-34 Two random variables X and Y have a joint probability density function

5,, x2y 0 < y < x < 2

elsewhere

(a) Find the marginal density functions of X and Y.(b) Arc X and V statistically independent?

4-35 Show, by use of (4.4-13), that the area underfy(y\x) is unity.*4-36 Two random variables R and 0 have the joint density function

u(r)[u(0)-u(9-2n)]r _rll2


(a) Find P{0 < R < 1, 0 < 0 < n/2}.(b) Find/R(r|0 = 7r).(c) Find fit(r\@ < n) and compare to the result found in part (/?), and explain the

comparison.4-37 Random variables X and Y have respective density functions

f x ( x )= la [u ( x ) -u ( x -a ) ]

fY(y) = bu(y)e-b>where a > 0 and b > 0. Find and sketch the density function of W = X + Y if X and Y arestatistically independent.4-38 Random variables X and Y have respective density functions

fx(x) = 0.1<5(.x - 1) + 0.26(x - 2) + 0AS(x - 3) + 0.3c5(x - 4)

My) = 0-4<5(y - 5) + 0.55(y - 6) + 0.1% - 7)Find and sketch the density function of W = X + Y if A' and V are independent.4-39 Find and sketch the density function of W = X + Y, where the random variable A" is thatof Problem 4-37 with a = 5 and Y is that of Problem 4-38. Assume X and Y are independent.4-40 Find the density function of W = X + Y, where the random variable X is that ofProblem 4-38 and Y is that of Problem 4-37. Assume X and Y are independent. Sketch thedensity function for b = 1 and b = 4.♦4-41 Three statistically independent random variables A",, X2, and X3 all have the same

density function

/xl(x,) = -[H(xl)-«(x,-a)] i=l,2,3

Find and sketch the density function of Y = X, + A"2 + X3 if a > 0 is constant.

88429283 probability random variables and random signal principles ch 3 4 p peebles

Documents