EEE 461 1

Chapter 6Chapter 6Random ProcessesRandom Processes

Huseyin BilgekulEEE 461 Communication Systems II

Department of Electrical and Electronic Engineering Eastern Mediterranean University

Description of Random Processes Stationarity and ergodicty Autocorrelation of Random Processes Properties of autocorrelation

EEE 461 2

Homework Homework AssignmentsAssignments

• Return date: November 8, 2005.• Assignments: Problem 6-2 Problem 6-3 Problem 6-6 Problem 6-10 Problem 6-11

EEE 461 3

Random ProcessesRandom Processes• A RANDOM VARIABLE X, is a rule for

assigning to every outcome, of an experiment a number X(. – Note: X denotes a random variable and X( denotes

a particular value. • A RANDOM PROCESS X(t) is a rule for

assigning to every a function X(t, – Note: for notational simplicity we often omit the

dependence on .

EEE 461 4

Ensemble of Sample Ensemble of Sample FunctionsFunctions

The set of all possible functions is called the ENSEMBLE.

EEE 461 5

• A general Random or Stochastic Process can be described as: – Collection of time functions

(signals) corresponding to various outcomes of random experiments.

– Collection of random variables observed at different times.

• Examples of random processes in communications: – Channel noise, – Information generated by a source, – Interference.

t1 t2

Random ProcessesRandom Processes

EEE 461 6

EEE 461 7

EEE 461 8

Collection of Time FunctionsCollection of Time Functions• Consider the time-varying function representing a

random process where i represents an outcome of a random event.

• Example: – a box has infinitely many resistors (i=1,2, . . .) of same

resistance R. – Let i be event that the ith resistor has been picked up from

the box– Let v(t, i) represent the voltage of the thermal noise

measured on this resistor.

EEE 461 9

Collection of Random Collection of Random VariablesVariables

• For a particular time t=to the value x(to,i is a random variable. • To describe a random process we can use collection of random

variables {x(to,1 , x(to,2 , x(to,3 , . . . }. • Type: a random processes can be either discrete-time or continuous-

time. • Probability of obtaining a sample function of a RP that passes

through the following set of windows. Probability of a joint event.

EEE 461 10

Description of Random Description of Random ProcessesProcesses

• Analytical description: X(t) =f(t,) where is an outcome of a random event.

• Statistical description: For any integer N and any choice of (t1, t2, . . ., tN) the joint pdf of {X(t1), X( t2), . . ., X( tN) } is known. To describe the random process completely the PDF f(x) is required.

1 1 1 2

1 2

( ), [ , ,... ]

{ ( ), ( ),.... ( )}N

N

x x t x x x

f f x t x t x t

xx

EEE 461 11

Example: Analytical Example: Analytical DescriptionDescription

• Let where is a random variable uniformly distributed on [0,2.

• Complete statistical description of X(to) is: – Introduce – Then, we need to transform from y to x:

– We need both y1 and y2 because for a given x the equation x=A cos (y) has two solutions in [0,2].

0cos 2X t A f t

0 02Y f t

1 2X Y Yp x dx p y dy p y dy

EEE 461 12

Analytical (continued)Analytical (continued)• Note x and y are actual values of the random

variables X and Y. • Since

and pY is uniform in [2f0t0, 2f0t0 + 2we get

• Using the analytical description of X(t), we obtained its statistical description at any time t.

2 2sindx A y A xdy

2 2

1

0 ElsewhereX

A x Ap x A x

EEE 461 13

Example: Statistical Example: Statistical DescriptionDescription

• Suppose a random process x(t) has the property that for any N and (t0,t1, . . .,tN) the joint density function of {x(ti)} is a jointly distributed Gaussian vector with zero mean and covariance

• This gives complete statistical description of the random

process x(t).

2 min ,ij i jt t

EEE 461 14

Activity: EnsemblesActivity: Ensembles• Consider the random process: x(t)=At+B• Draw ensembles of the waveforms:

– B is constant, A is uniformly distributed between [-1,1]– A is constant, B is uniformly distributed between [0,2]

• Does having an “Ensemble” of waveforms give you a better picture of how the system performs?

B

x(t)

t

Slope Random

2x(t)

t

B intersect is Random

EEE 461 15

StationarityStationarity• Definition: A random process is STATIONARY to the

order N if for any t1,t2, . . . , tN,

fx{x(t1), x(t2),...x(tN)}=fx{x(t1+t0), x(t2+t0),...,x(tN +t0)}

• This means that the process behaves similarly (follows the same PDF) regardless of when you measure it.

• A random process is said to be STRICTLY STATIONARY if it is stationary to the order of N→∞.

• Is the random process from the coin tossing experiment stationary?

EEE 461 16

Illustration of StationarityIllustration of Stationarity

Time functions pass through the corresponding windows at different times with the same probability.

EEE 461 17

Example of First-Order Example of First-Order StationarityStationarity

• Assume that A and 0 are constants; 0 is a uniformly distributed RV from ) t is time.

• From last lecture, recall that the PDF of x(t):

• Note: there is NO dependence on time, the PDF is not a function of t.

• The RP is STATIONARY.

0 0RANDOM PROCESS is sinx t A t

2 2x

1

0 Elsewhere

x Af x A x

x

EEE 461 18

Non-Stationary ExampleNon-Stationary Example

• Now assume that A, 0 and 0 are constants; t is time.

• Value of x(t) is always known for any time with a probability of 1. Thus the first order PDF of x(t) is

• Note: The PDF depends on time, so it is NONSTATIONARY.

0 0sinf x x A t

0 0RANDOM PROCESS is sinx t A t

EEE 461 19

Ergodic ProcessesErgodic Processes• Definition: A random process is ERGODIC if all time averages of any

sample function are equal to the corresponding ensemble averages (expectations)

• Example, for ergodic processes, can use ensemble statistics to compute DC values and RMS values

• Ergodic processes are always stationary; Stationary processes are not necessarily ergodic

Ergodic tationaryS

0

2 2 2 2

[ ( )]

1 [ ( )] Time average

[ ( )] [ ] ( ) Ensmble average

limDC x

T

T

x

RMS x

x x t x t m

x t x t dtT

x t x f x dx m

x x t x m

EEE 461 20

Example: Ergodic ProcessExample: Ergodic Process

• A and 0 are constants; 0 is a uniformly distributed RV from ) t is time.

• Mean (Ensemble statistics)

• Variance

01sin 0

2xm x x f d A t d

2

2 2 20

1sin2 2x

AA t d

0 0RANDOM PROCESS is sinx t A t

EEE 461 21

Example: Ergodic ProcessExample: Ergodic Process• Mean (Time Average) T is large

• Variance

• The ensemble and time averages are the same, so the process is ERGODIC

00

1 sin 0limT

Tx t A t dt

T

2

2 2 200

1 sin2lim

T

T

Ax t A t dtT

EEE 461 22

EXERCISEEXERCISE• Write down the definition of :

– Wide sense stationary– Ergodic processes

• How do these concepts relate to each other?• Consider: x(t) = K; K is uniformly distributed

between [-1, 1]– WSS?– Ergodic?

EEE 461 23

Autocorrelation of Random Autocorrelation of Random ProcessProcess

• The Autocorrelation function of a real random process x(t) at two times is:

_________________

1 2 1 2 1 2 1 2, ,1 2x xR t t x t x t x x f x x dx dx

EEE 461 24

Wide-sense StationaryWide-sense Stationary• A random process that is stationary to order 2 or greater is

Wide-Sense Stationary:• A random process is Wide-Sense Stationary if:

• Usually, t1=t and t2=t+ so that t2- t1 =• Wide-sense stationary process does not DRIFT with time.• Autocorrelation depends only on the time gap but not where the

time difference is.• Autocorrelation gives idea about the frequency response of the

RP.

EEE 461 25

Autocorrelation Function of Autocorrelation Function of RPRP• Properties of the autocorrelation function of wide-sense

stationary processes

Autocorrelation of slowly and rapidly fluctuating random processes.Autocorrelation of slowly and rapidly fluctuating random processes.

EEE 461 26

Cross Correlations of RPCross Correlations of RP• Cross Correlation of two RP x(t) and y(t) is

defined similarly as:

• If x(t) and y(t) are Jointly Stationary processes,

• If the RP’s are jointly ERGODIC,

_________________

xy 1 2 1 2 xy 1 2 1 2, ,1 2R t t x t y t x y f x y dx dy

xy 1 2 xy 2 1 xy 2 1, R t t R t t R t t

_________________

xy ( ) ( )R x t y t x t y t

EEE 461 27

Cross Correlation Properties of Cross Correlation Properties of Jointly Stationary RP’sJointly Stationary RP’s

• Some properties of cross-correlation functions are

• Uncorrelated:

• Orthogonal:

• Independent: if x(t1) and y(t2) are independent (joint distribution is product of individual distributions)