overview random processes

7/30/2019 Overview Random Processes

1/30

1

Overview of Random Processes


2/30


3/30

3

Let denote the random outcome of an experiment.

To every such outcome, suppose a waveform

is assigned.The collection of such

waveforms form a

stochastic process.

For fixed (the set of

all experimental outcomes),

is a specific time function.

For fixed t,is a random variable.

The ensemble of all such realizations

over time represents the stochastic process X(t).

),(tX

Si

),( 11 itXX =

),( tX

t

1t

2t

),(n

tX

),( ktX

),(2

tX

),(1

tX

M

M

M

),( tX

0

),( tX

Random (stochastic) Processes


4/30

4


For example

where is a uniformly distributed random variable in

represents a stochastic process.

Stochastic processes are everywhere:

Noise, detection and classification problems, pattern recognition,

stock market fluctuations, various queuing systems

all represent stochastic phenomena.

),cos()( 0 += tatX

(0,2 ),


5/30

5


IfX(t) is a stochastic process, then for fixed t, X(t) representsa random variable. Its distribution function (cdf) is given by

Notice that depends on t, since for a different t, we obtaina different random variable. Further

represents the first-order probability density function (pdf) of the

process X(t).

})({),( xtXPtxFX

=

),( txFX

dx

txdFtxf X

X

),(),( =


6/30

6


For t= t1 and t= t2, X(t) represents two different random variables

X1 = X(t1) and X2 = X(t2) respectively. Their joint distribution is

given by

and

represents the second-order density function of the process X(t).

Similarly represents the nth order density

function of the process X(t). Complete specification of the stochastic

process X(t) requires the knowledge of

for all and for all n. (an almost impossible taskin reality).

})(,)({),,,( 22112121 xtXxtXPttxxFX =

),,,,,( 2121 nn tttxxxfX LL

),,,,,( 2121 nn tttxxxfX LL

niti ,,2,1, L=

21 2 1 2

1 2 1 2

1 2

( , , , )( , , , )

X

X

F x x t t f x x t t

x x

=


7/30

7


Mean of a Stochastic Process:

represents the mean value of a process X(t). In general, the mean ofa process can depend on the time index t.

Autocorrelation function of a process X(t) is defined as

and it represents the interrelationship between the random variables

X1 = X(t1) and X2 = X(t2) generated from the process X(t).

Properties:

1.

2.

*

1 2 2 1( , ) ( , )

XX XXR t t R t t=

.0}|)({|),(2

>= tXEttRXX (Average instantaneous power)

( ) { ( )} ( , )X

t E X t x f x t dx+

= =

* *

1 2 1 2 1 2 1 2 1 2 1 2( , ) { ( ) ( )} ( , , , )XX XR t t E X t X t x x f x x t t dx dx= =


8/30

8


The function

represents the autocovariance function of the process X(t).

)()(),(),( 2*

12121 ttttRttC XXXXXX =

Similarly

0

2

00

( ) { ( )} {cos( )}1 cos( ) 0.

2

X t E X t aE t

t d

= = +

= + =

).(cos

2

)}2)(cos()({cos2

)}cos(){cos(),(

210

2

210210

22010

2

21

tta

ttttEa

ttEattRXX

=

+++=

++=

Example ).2,0(~),cos()( 0 UtatX +=


9/30

9

Stationary Random Processes

Stationary processes exhibit statistical properties that are

invariant to shift in the time index.

Thus, for example, second-order stationarity implies that

the statistical properties of the pairs

{X(t1) , X(t2) } and {X(t1+c) , X(t2+c)} are the same for any c.

Similarly first-order stationarity implies that the statistical

properties ofX(ti) and X(ti+c) are the same for any c.


10/30

10


In strict terms, the statistical properties are governed by the

joint probability density function. Hence a process is nth-order

Strict-Sense Stationary (S.S.S) if

for any c, where the left side represents the joint density function of

the random variables andthe right side corresponds to the joint density function of the random

variables

A process X(t) is said to be strict-sense stationary if above eqn. is

true for all

),,,,,(),,,,,( 21212121 ctctctxxxftttxxxf nnnn XX +++ LLLL

)(,),(),( 2211 nn tXXtXXtXX === L

).(,),(),( 2211 ctXXctXXctXX nn +=+=+= L

.and,2,1,,,2,1, canynniti LL ==


11/30

11


For a first-order strict sense stationary process,

for any c.

In particular c= tgives

i.e., the first-order density of X(t) is independent of t.

In that case

),(),( ctxftxfXX

+

)(),( xftxfXX

=

[ ( )] ( ) ,E X t x f x dx a constant.+

= =


12/30

12


i.e., the second order density function of a SSS process depends only

on the difference of the time indices

In that case, the autocorrelation function is given by

i.e., it depends only on the difference of the time indices.

2 1 .t t =

*

1 2 1 2

*

1 2 1 2 2 1 1 2

*

2 1

( , ) { ( ) ( )}

( , , )

( ) ( ) ( ),

XX

X

XX XX XX

R t t E X t X t

x x f x x t t dx dx

R t t R R

=

= =

= = =

Similarly, for a second-order strict-sense stationary process

for any c. For c= t1 we get

),,,(),,,( 21212121 ctctxxfttxxf XX ++

1 2 1 2 1 2 2 1( , , , ) ( , , )X Xf x x t t f x x t t


13/30

13


The basic conditions for the first and second order stationarity areusually difficult to verify.

In that case, we often resort to a looser definition of stationarity,known as Wide-Sense Stationarity (W.S.S).

A process X(t) is said to be Wide-Sense Stationary if

(i)and(ii)

i.e., the mean is a constant and the autocorrelation functiondepends only on the difference between the time indices.

=)}({ tXE

*

1 2 2 1{ ( ) ( )} ( ),XXE X t X t R t t=


14/30

14


Remarks:

1. Notice that these conditions do not say anything about the

nature of the probability density functions, and instead dealwith the average behavior of the process.

2. Strict-sense stationarity always implies wide-sense

stationarity. SSSWSS

However, the converse is not truein general, the only exceptionbeing the Gaussian process.

If X(t) is a Gaussian process, then wide-sense stationarity (w.s.s) strict-sense stationarity (s.s.s).


15/30

15

Ergodic Random Processes

If almost every member of the ensemble shows the same statisticalbehavior as the whole ensemble, then it is possible to determine thestatistical behavior by examining only one typical sample function.

Ergodic process

For ergodic process, the mean values and autocorrelation functioncan be determined by time averages as well as by ensembleaverages, that is,

Ergodic in the mean process:

Ergodic in the autocorrelation process:

These conditions can exist if the process is stationary.

ergodicstationary(not vice versa)

{ }1

( ) ( )2

limT

TT

E X t X t dtT

=

( ) { }* *1

( ) ( ) ( ) ( )2

limT

XXT

T

R E X t X t X t X t dtT

= + = +


16/30

16

Power Spectral Density

Power spectrum of X(t)

Autocorrelation of X(t)

Power spectrum and Autocorrelation function

are Fourier transform pair

Total average power=

2( ) [ ( )] ( ) j fx x x

S f F R R e d

= =

1 2

( ) [ ( )] ( )

j f

x x xR F S f S f e df

= =

(0) ( )x x

R S f df

=


17/30

17

Random Processes as Inputs/Outputs to LTI Sytems

A deterministic system transforms each input waveform intoan output waveform by operating only on thetime variable t.

Thus a set of realizations at the input corresponding to a processX(t)generates a new set of realizations at theoutput associated with a new process Y(t).

),( itX )],([),( ii tXTtY =

)},({ tY

Our goal is to study the output process statistics in terms of the inputprocess statistics and the system function.

][T)(tX

)(tY

t t

),(i

tX ),(

itY


18/30

18


Linear Systems: represents a linear system if

Let represent the output of a linear system.

Time-Invariant System: represents a time-invariant system if

i.e., shift in the input results in the same shift in the output also.

If satisfies both, then it corresponds to a linear time-invariant (LTI)system.

][L

)}({)( tXLtY =

)}.({)}({)}()({ 22112211 tXLatXLatXatXaL +=+

][L

)()}({)}({)( 00 ttYttXLtXLtY ==

][L


19/30

19


LTI

+

+

=

=

)()(

)()()(

dtXh

dXthtYarbitrary

input

t

)(tX

t

)(tY

)(tX )(tY

LTI systems can be uniquely represented in terms of their outputto a delta function

LTI)(t )(th

Impulse

Impulse

response ofthe system

t

)(th

Impulse

response

then


20/30

20


Output Statistics: The mean of the output process is given by

).()()()(

})()({)}({)(

thtdth

dthXEtYEt

XX

Y

==

==

+

+

h(t))(tX )(tY

In particular if X(t) is wide-sense stationary, then we haveso that

XXt =)(

constant.acdhtXXY

,)()( == +


21/30

21


Output Statistics: The autocorrelation function of the output is given by

*( ) ( ) ( ) ( ).YY XX

R R h h =

h() h*(-)( )XYR ( )

YYR ( )

XXR

h*(-) h()

( )YXR ( )YYR

( )XX

R

Thus, Y(t) is w.s.s process.X(t) and Y(t) are jointly w.s.s.

LTI systemh(t)

wide-sensestationary process

wide-sensestationary process.

)(tX )(tY


22/30

22


Output Statistics: The power spectral density function of the output is

2 *( ) ( ) ( ) ( ) ( ) ( )

YY XX XX S f S f H f S f H f H f = =

H(f) H*(f)( )XYS f ( )

YYS f( )

XXS f

H*(f) H(f)( )YXS f ( )

YYS f( )XXS f

{ } { }*( ) ( ) ( ) ( )XY XYS f F R F E X t Y t = = +

{ } { *( ) ( ) ( ) ( )YX YX S f F R F E Y t X t = = +


23/30

23

White Noise

A random process X(t) is called a white process if it

has a flat power spectrum.

If Sx(f) is constant for all f

It closely represents thermal noise

f

Sx(f)

The area is infinite(Infinite power !)

0( )

2n

NS f =


24/30


25/30

25

White Gaussian Noise

The sampled random variables will be statisticallyindependent Gaussian random variables

Sn(f)

N0/2

f

N0/2

Rx()

=0 0


26/30

26

Poisson Random Process

Poisson random variable:

L,2,1,0,!"durationofinterval

aninoccurarrivals" ==

kk

ekPk

=== T

Tnp

0 T

43421arrivalsk


27/30

27

Poisson Random Process

Definition: X(t) = n(0, t) represents a Poisson process if

the number of arrivals n(t1, t2) in an interval (t1, t2) of length

t= t2t1 is a Poisson random variable with parameter

Thus

.t

1221 ,,2,1,0,

!

)(}),({ tttk

k

tekttnP

kt

==== L

ttnEtXE == )],0([)]([

).,min(),( 2121221 ttttttRXX +=

X(t) : not WSS


28/30

28

Poisson Impulse Process

Although X(t) does not represent a wide sense stationaryprocess, its derivative doesrepresent a wide sensestationary process (called Poisson Impulse Process).

)(tX

)(tX )(tXdt

d )(

constantadt

td

dt

tdt X

X,

)()(

===

2

1 2 1 2( ) ( ).X XR t , t t t = +


29/30

29

Gaussian Random Process

A random process X(t) is a Gaussian process if for all

n and for all , the random variableshas a jointly Gaussian density function, which may be

expressed as

where

1 2( , , , )nt t tK

2{ ( ), ( ), , ( )}i nX t X t X tK

1

/ 2 1/ 21 1( ) exp[ ( ) ( )]

2(2 ) [det( )]

T

nf x x m C x m

C

=

2[ ( ), ( ), , ( )]

T

i nx X t X t X t=

K( )m E X=

{ } (( )( ))ij i i j jC c E x m x m= =

: n random variables: mean value vector: nxn covariance matrix


30/30

overview random processes

Documents