![Page 1: ENSC327 Communications Systems 19: Random …...Definition of Random Process A deterministic process has only one possible 'reality' of how the process evolves under time. In a stochastic](https://reader033.vdocuments.mx/reader033/viewer/2022042111/5e8c0b049a61685a3048ab22/html5/thumbnails/1.jpg)
ENSC327
Communications Systems
19: Random Processes
1
Jie Liang
School of Engineering Science
Simon Fraser University
![Page 2: ENSC327 Communications Systems 19: Random …...Definition of Random Process A deterministic process has only one possible 'reality' of how the process evolves under time. In a stochastic](https://reader033.vdocuments.mx/reader033/viewer/2022042111/5e8c0b049a61685a3048ab22/html5/thumbnails/2.jpg)
Outline
� Random processes
� Stationary random processes
� Autocorrelation of random processes
2
![Page 3: ENSC327 Communications Systems 19: Random …...Definition of Random Process A deterministic process has only one possible 'reality' of how the process evolves under time. In a stochastic](https://reader033.vdocuments.mx/reader033/viewer/2022042111/5e8c0b049a61685a3048ab22/html5/thumbnails/3.jpg)
Definition of Random Process
� A deterministic process has only one possible 'reality' of how the process evolves under time.
� In a stochastic or random process there are some uncertainties in its future evolution described by probability distributions.
3
� Even if the initial condition (or starting point) is known, there are many possibilities the process might go to, but some paths are more probable and others less.
� http://en.wikipedia.org/wiki/Stochastic_process
![Page 4: ENSC327 Communications Systems 19: Random …...Definition of Random Process A deterministic process has only one possible 'reality' of how the process evolves under time. In a stochastic](https://reader033.vdocuments.mx/reader033/viewer/2022042111/5e8c0b049a61685a3048ab22/html5/thumbnails/4.jpg)
Definition of Random Process
� Many time-varying signals are random in nature:
� Noises
� Image, audio: usually unknown to the distant receiver.
� Random process represents the mathematical model of these random signals.
4
random signals.
� Definition: A random process (or stochastic process) is a collection of random variables (functions) indexed by time.
� Notation:
� s: the sample point of the random experiment.
� t: time.
� Simplified notation:
),( stX
)(tX
![Page 5: ENSC327 Communications Systems 19: Random …...Definition of Random Process A deterministic process has only one possible 'reality' of how the process evolves under time. In a stochastic](https://reader033.vdocuments.mx/reader033/viewer/2022042111/5e8c0b049a61685a3048ab22/html5/thumbnails/5.jpg)
Random Processes
� The difference between random variable and random process:
� Random variable: an outcome is mapped to a number.
� Random process: an outcome is mapped to a random waveform that is a function of time
5
� We are interested in the ways that these time functions evolve
� correlation
� spectra
� linear systems
![Page 6: ENSC327 Communications Systems 19: Random …...Definition of Random Process A deterministic process has only one possible 'reality' of how the process evolves under time. In a stochastic](https://reader033.vdocuments.mx/reader033/viewer/2022042111/5e8c0b049a61685a3048ab22/html5/thumbnails/6.jpg)
Cont…
� For a fixed sample point sj, X(t, sj) is a realization or sample function of the random process.
� Simplified Notation:
).( as denoted is ),( txstXjj
6
� For a fixed time tk, the set of numbers
{X(tk, s1), …, X(tk, sn)} = {x1(tk), …, xn(tk)}
is a random variable, denoted by X(tk).
� For a fixed sj and tk, X(tk, sj) is a number.
jj
![Page 7: ENSC327 Communications Systems 19: Random …...Definition of Random Process A deterministic process has only one possible 'reality' of how the process evolves under time. In a stochastic](https://reader033.vdocuments.mx/reader033/viewer/2022042111/5e8c0b049a61685a3048ab22/html5/thumbnails/7.jpg)
Pictorial ViewEach sample point represents a time-varying function.
Ensemble: The set of all time-varying functions.
7
![Page 8: ENSC327 Communications Systems 19: Random …...Definition of Random Process A deterministic process has only one possible 'reality' of how the process evolves under time. In a stochastic](https://reader033.vdocuments.mx/reader033/viewer/2022042111/5e8c0b049a61685a3048ab22/html5/thumbnails/8.jpg)
Examples of Random Processes
� Y: a random variable.
� f: a deterministic function of parameter t.
1.
2.
short.for )(),(or )()(),( tYfstXtfsYstX ==
.)2cos()(or )2cos()(),( QtfAtXQtfsAstX +=+= ππ
8
2.
� A: a random variable.
� : a random variable.
3.
� X(n), T(n): random sequences.
� pn(t): deterministic waveforms.
.)2cos()(or )2cos()(),(00
QtfAtXQtfsAstX +=+= ππ
Q
![Page 9: ENSC327 Communications Systems 19: Random …...Definition of Random Process A deterministic process has only one possible 'reality' of how the process evolves under time. In a stochastic](https://reader033.vdocuments.mx/reader033/viewer/2022042111/5e8c0b049a61685a3048ab22/html5/thumbnails/9.jpg)
Probability Distribution of a Random Process
� For any random process, its probability distribution function is
uniquely determined by its finite dimensional distributions.
� The k dimensional cumulative distribution function of a process is
defined by
9
for any and any real numbers x1, …, x
k.
� The cumulative distribution function tells us everything we need to
know about the process {Xt}.
( ) ( )kkktXtXxtXxtXPxxF
k
≤≤= )(,...,)(,..., 111)(),...,(1
ktt ,...,
1
![Page 10: ENSC327 Communications Systems 19: Random …...Definition of Random Process A deterministic process has only one possible 'reality' of how the process evolves under time. In a stochastic](https://reader033.vdocuments.mx/reader033/viewer/2022042111/5e8c0b049a61685a3048ab22/html5/thumbnails/10.jpg)
Outline
� Random processes
� Stationary random processes
� Autocorrelation of random processes
10
![Page 11: ENSC327 Communications Systems 19: Random …...Definition of Random Process A deterministic process has only one possible 'reality' of how the process evolves under time. In a stochastic](https://reader033.vdocuments.mx/reader033/viewer/2022042111/5e8c0b049a61685a3048ab22/html5/thumbnails/11.jpg)
Stationarity
� In general, the time-dependent N-fold joint pdf’s are needed to
describe a random process for all possible N:
� Very difficult to obtain all pdf’s.
� The analysis can be simplified if the statistics are time
independent.
11
independent.
� The random process is called first-order stationary if
t. timefixed aat X(t) process random theof CDF the:)()( xFtX
![Page 12: ENSC327 Communications Systems 19: Random …...Definition of Random Process A deterministic process has only one possible 'reality' of how the process evolves under time. In a stochastic](https://reader033.vdocuments.mx/reader033/viewer/2022042111/5e8c0b049a61685a3048ab22/html5/thumbnails/12.jpg)
Stationarity
� The mean and variance of the first-order stationary
random process are independent of time:
⇒ timeoft independen is )( pdf the )( xftX
,)()(11τ
µµ+
=tXtX .
2
)(
2
)(11τ
σσ+
=tXtX
12
� Proof:
,)()(11τ
µµ+
=tXtX .)()(
11τ
σσ+
=tXtX
![Page 13: ENSC327 Communications Systems 19: Random …...Definition of Random Process A deterministic process has only one possible 'reality' of how the process evolves under time. In a stochastic](https://reader033.vdocuments.mx/reader033/viewer/2022042111/5e8c0b049a61685a3048ab22/html5/thumbnails/13.jpg)
Stationarity
�The random process is called second-order
stationary if the 2nd order CDF is independent of
time:
13
![Page 14: ENSC327 Communications Systems 19: Random …...Definition of Random Process A deterministic process has only one possible 'reality' of how the process evolves under time. In a stochastic](https://reader033.vdocuments.mx/reader033/viewer/2022042111/5e8c0b049a61685a3048ab22/html5/thumbnails/14.jpg)
Strict Stationarity
� A strictly stationary process (or strongly stationary process, or
stationary process) is a stochastic process whose joint pdf does
not change when shifted in time.
� Definition: a random process X(t) is said to be stationary if, for
all k, for all τ, and for all t1, t2, …, tk,
14
all k, for all τ, and for all t1, t2, …, tk,
![Page 15: ENSC327 Communications Systems 19: Random …...Definition of Random Process A deterministic process has only one possible 'reality' of how the process evolves under time. In a stochastic](https://reader033.vdocuments.mx/reader033/viewer/2022042111/5e8c0b049a61685a3048ab22/html5/thumbnails/15.jpg)
Strict Stationarity
� An example of strictly stationary process is one in which all
X(ti)’s are mutually Independent and Identically Distributed.
� Such a random process is called IID random process.
� In this case,
15
� Since the joint pdf above does not depend on the times {ti},
the process is strictly stationary.
� An example of IID process is white noise (studied later)
� Widely used in communications theory
![Page 16: ENSC327 Communications Systems 19: Random …...Definition of Random Process A deterministic process has only one possible 'reality' of how the process evolves under time. In a stochastic](https://reader033.vdocuments.mx/reader033/viewer/2022042111/5e8c0b049a61685a3048ab22/html5/thumbnails/16.jpg)
Outline
� Random processes
� Stationary random processes
� Autocorrelation of random processes
16
![Page 17: ENSC327 Communications Systems 19: Random …...Definition of Random Process A deterministic process has only one possible 'reality' of how the process evolves under time. In a stochastic](https://reader033.vdocuments.mx/reader033/viewer/2022042111/5e8c0b049a61685a3048ab22/html5/thumbnails/17.jpg)
Correlation of Random Processes
. tand at t X(t) of samples :)( and )(Consider 2121
tXtX
[ ][ ]{ } { }YXYX
XYEYXEYXCov µµµµ −=−−= ),(
�Recall: Covariance of two random variables:
17
variablesrandomboth are )( and )(21tXtX
{ } )()(212121
)()())(),((tXtX
tXtXEtXtXCov µµ−=
So we can also define their covariance:
![Page 18: ENSC327 Communications Systems 19: Random …...Definition of Random Process A deterministic process has only one possible 'reality' of how the process evolves under time. In a stochastic](https://reader033.vdocuments.mx/reader033/viewer/2022042111/5e8c0b049a61685a3048ab22/html5/thumbnails/18.jpg)
Correlation of Random Processes� Recall: autocorrelation of deterministic energy signals:
∫∞
∞−
∆
−= )()( )( *dttxtxR
xττ
� Similarly, the autocorrelation of deterministic power signal is:
18
� The limit is necessary since the energy of the power signal can
be infinite.
∫−
∗
∞→
−=
T
TT
Xdttxtx
TR )()(
2
1)( lim ττ
![Page 19: ENSC327 Communications Systems 19: Random …...Definition of Random Process A deterministic process has only one possible 'reality' of how the process evolves under time. In a stochastic](https://reader033.vdocuments.mx/reader033/viewer/2022042111/5e8c0b049a61685a3048ab22/html5/thumbnails/19.jpg)
Correlation of Random Processes
� The autocorrelation function of a random process:
� For random processes: need to consider probability distributions.
19
� If X(t) is stationary to the 2nd order or higher order, RX(t1,t2) only depends on
the time difference t1 - t2, so it can be written as a single variable function:
� Note: steps to get :
� 1: For each sample function X(t, sj), calculate X(t1, sj) X*(t2, sj).
� 2: Take weighted average over all possible sample functions sj.
� (See Example 1 later)
{ })()(2
*
1tXtXE
![Page 20: ENSC327 Communications Systems 19: Random …...Definition of Random Process A deterministic process has only one possible 'reality' of how the process evolves under time. In a stochastic](https://reader033.vdocuments.mx/reader033/viewer/2022042111/5e8c0b049a61685a3048ab22/html5/thumbnails/20.jpg)
Wide-Sense Stationarity (WSS)
� In many cases we do not require a random process to
have all of the properties of the 2nd order stationarity.
� A random process is said to be wide-sense stationary
20
� A random process is said to be wide-sense stationary
or weakly stationary if and only if
![Page 21: ENSC327 Communications Systems 19: Random …...Definition of Random Process A deterministic process has only one possible 'reality' of how the process evolves under time. In a stochastic](https://reader033.vdocuments.mx/reader033/viewer/2022042111/5e8c0b049a61685a3048ab22/html5/thumbnails/21.jpg)
Property of Autocorrelation
� For real-valued wide-sense stationary X(t), we have:
� 1.
{ } ).()()(),( *stRsXtXEstR
XX−==
{ }.)()0( 2tXER
X=
� 2. RX(τ) is even symmetry: R
X(-τ) = R
X(τ).
� Proof:
21
� Proof:
� 3. RX(τ) is max at the origin τ = 0.
� Proof:
![Page 22: ENSC327 Communications Systems 19: Random …...Definition of Random Process A deterministic process has only one possible 'reality' of how the process evolves under time. In a stochastic](https://reader033.vdocuments.mx/reader033/viewer/2022042111/5e8c0b049a61685a3048ab22/html5/thumbnails/22.jpg)
Property of Autocorrelation
� Example of the autocorrelation function:
{ } ).()()(),( *stRsXtXEstR
XX−==
22
� Symmetric, peak at 0.
� Change slow if X(t) changes slow.
![Page 23: ENSC327 Communications Systems 19: Random …...Definition of Random Process A deterministic process has only one possible 'reality' of how the process evolves under time. In a stochastic](https://reader033.vdocuments.mx/reader033/viewer/2022042111/5e8c0b049a61685a3048ab22/html5/thumbnails/23.jpg)
Example 1
A: constant. Θ: uniform random variable in [0, 2π].
� Find the autocorrelation of X.
� Is X wide-sense stationary?
Solution:
)2cos()( Θ+= ftAtX π
23
Solution:
![Page 24: ENSC327 Communications Systems 19: Random …...Definition of Random Process A deterministic process has only one possible 'reality' of how the process evolves under time. In a stochastic](https://reader033.vdocuments.mx/reader033/viewer/2022042111/5e8c0b049a61685a3048ab22/html5/thumbnails/24.jpg)
Example 1 ∫∞
∞−
= dxxfxgXY
)()(µY = g(X) �
24
![Page 25: ENSC327 Communications Systems 19: Random …...Definition of Random Process A deterministic process has only one possible 'reality' of how the process evolves under time. In a stochastic](https://reader033.vdocuments.mx/reader033/viewer/2022042111/5e8c0b049a61685a3048ab22/html5/thumbnails/25.jpg)
Example 2
A: uniform random variable in [0, 1].
� Find the autocorrelation of X. Is X WSS?
)2cos()( ftAtX π=
Solution:Solution:
25
![Page 26: ENSC327 Communications Systems 19: Random …...Definition of Random Process A deterministic process has only one possible 'reality' of how the process evolves under time. In a stochastic](https://reader033.vdocuments.mx/reader033/viewer/2022042111/5e8c0b049a61685a3048ab22/html5/thumbnails/26.jpg)
Example 2
Note the value of A at time t1 and t2 are same in this example,
because it’s determined by the random experiment.
)2cos()( ftAtX π=
26
![Page 27: ENSC327 Communications Systems 19: Random …...Definition of Random Process A deterministic process has only one possible 'reality' of how the process evolves under time. In a stochastic](https://reader033.vdocuments.mx/reader033/viewer/2022042111/5e8c0b049a61685a3048ab22/html5/thumbnails/27.jpg)
Example 3
� A random process X(t) consists of three possible sample functions:
x1(t)=1, x2(t)=3, and x3(t)=sin(t). Each occurs with equal probability.
Find its mean and auto-correlation. Is it wide-sense stationary?
� Solution:
27