ECE594I notes, M. Rodwell, copyrighted
ECE594I Notes set 5:More Math: Random Processes
Mark RodwellUniversity of California, Santa Barbara y
[email protected] 805-893-3244, 805-893-3262 fax
ECE594I notes, M. Rodwell, copyrighted
References and Citations:
DevicesState-Solid in Noise :Zielder Van PhysicsThermal : Kroemerand Kittel
:Citations / Sources
ng.EngineeriionsCommunicatofsPrinciple:Jacobs&Wozencraftory)(introduct s PrincipleSignal RandomVariables, Randomty, Probabili: PeeblesZ. Peyton
ive)comprehens(hard, Variables Randomandity Probabil:Papoulis
1982circaStanfordHellmanMartin:noteslectureyProbabilit1982circa Stanford, Cover, Thomas :notes lecture theory nInformatio
Designc ElectroniNoise Low :erMotchenbakng.EngineeriionsCommunicatofs Principle:Jacobs & Wozencraft
t dffS t d
circuits. in Noise :Notes nsApplicatio tor LinearSemiconduc National1982circa Stanford,Hellman, Martin:notes lecturey Probabilit
design)receiver (optical Personik & Smith noise), (device by Fukui Papers Kroemer and Kittel Peebles,Jacobs, & Wozencraft Ziel,der Van
study.for references Suggested
Theory nInformatio of Elements:Williams andCover )(!Notes App. Semi. National
ECE594I notes, M. Rodwell, copyrighted
random processesrandom processes
ECE594I notes, M. Rodwell, copyrighted
Random Processes
time. vs. voltageof functions of paper, of sheets separate on graphs, ofset a Draw
can. garbagea into Put them
space. sampley probabilit the called is can garbage This
timeoffunctionrandomourisThisrandom.at sheet oneout Pick
).( is process random The
time.offunction randomour is This
tV v(t)is outcome particular The
ECE594I notes, M. Rodwell, copyrighted
Xvariablerandomaofg(X)functionaofnexpectatiotheofdefiniontheRecall
Time Averages vs. Sample Space Averages
[ ] ∫∞+
∞
== )()()(
Xvariablerandoma ofg(X) functiona ofnexpectatio theof definion theRecall
X gdxxfxgxgE∞−
space. sample over the is average the where, of * valueaverage* theis gg
1 :timeparticularsomeatspacesampleover the average an define can we
,definitionprocess randomour With
t
[ ] ∫∞+
∞−
= 1111
1
))(())(())(())((
:timeparticular someat space sample
V tvdtvftvgtvgE
t
+ 2/
:over time function outcome oneany of average an define also can We
T
[ ] ∫+
−∞→=
2/
2/
))((1lim))((T
TiTi dttvg
TtvgA
ECE594I notes, M. Rodwell, copyrighted
What is Random in a Random Process ?
time".ithrandomly w varies" signal that thenexpectatio thehavesomehow We
random. is whichsample theof selection theisit ,definitionour in Yet,
y.discrepanc thisaddresses Ergodicity
suprising. are nsimplicatio Its
ECE594I notes, M. Rodwell, copyrighted
Ergodic Random Processes
llt ti tith toequal over time averages
has process random ErgodicAn
[ ] [ ] all,allfor ))(())((
spacesamplelstatistica over the averages
11 titvgAtvgE i=[ ] [ ]
made have wesense, some In
,))(())(( 11 gg i
space"sampleover thevariationrandom" toequivalent
time" with variationrandom"
spacesampleover the variationrandom
ECE594I notes, M. Rodwell, copyrighted
Time Samples of Random Processes
).( and )( valueshas )( process random the and at times samples timeWith
21
21
tVtVtVtt
on.distributiy probabilitjoint some have )( and )( 21 tVtVGaussian.jointly be not)might (or might They
v(t)
v(t2 )
t
v(t1 )
t t tt1 t2
ECE594I notes, M. Rodwell, copyrighted
Random Waveforms are Random Vectors
hlii 'i
samplestimespaced-regularlypickweifandd,bandlimite is signal randoma if
theorem,sampling sNyquist' Using
tt vector.randoma into process randomour convert we
,... samplestimespaced-regularlypick weif and 1 ntt
geometry. and analysisvector using signals random analyze thuscan We
v(t)( )
tt1 t2 tn
ECE594I notes, M. Rodwell, copyrighted
timeithnot vary wdoprocessstationaryaofstatisticsThe
Stationary Random Processes
:tystationariorder N
time.ithnot vary wdoprocessstationarya of statistics The
th −( )[ ] ( )[ ]
orderslower ..and )(),...,(),()(),...,(),(
y
2121 τττ +++= tVtVtVfEtVtVtVfE nn
( )[ ] ( )[ ])()()()(:tystationariorder 2 nd
ττ ++=−
tVtVfEtVtVfE ( )[ ] ( )[ ]( )[ ] ( )[ ])()(orderslower
)(),()(),(
11
2121
τττ+=→
++=tVfEtVfE
tVtVfEtVtVfE
v(t)
v(t1 )v(t2 )
tt1 t2
ECE594I notes, M. Rodwell, copyrighted
Restrictions on the random processes we consider
We will make following restrictions to make analysis tractable: The process will be Ergodic. The process will be stationary to any order: all statistical properties are independent of time. Many common processes are not stationary, including integrated white noise and 1/f noise. The process will be Jointly Gaussian. This means that if the values of a random process X(t) are sampled at times t1, t2, etc, to form random variables X1=X(t1), etc, then X1,X2, etc. are a jointly Gaussian random variable. In nature, many random processes result from the sum of a vast number of small underlying random processses. From the central limit theorem, such processes can frequently be expected to be Jointly Gaussian.
ECE594I notes, M. Rodwell, copyrighted
Variation of a random process with time
For the random process X(t) look at X X(t ) and X X(t )For the random process X(t), look at X1=X(t1) and X2=X(t2).
[ ] ∫ ∫+∞+∞
== 21212121 ),(•2121
dxdxxxfxxXXER XXXX ∫ ∫∞− ∞−
To compute this we need to know the joint probability distribution. We have assumed a Gaussian process. The above is called the Autocorrellation function. IF the process is stationary, it is a function only of (t1-t2)=tau, and hence y, y ( 1 2) , RXX τ( ) = E X t( )X t + τ( )[ ]
d ib h idl d l ithis is the autocorrellation function. It describes how rapidly a random voltage varies with time…. PLEASE recall we are assuming zero-mean random processes (DC bias subtracted). Thus g p ( )the autocorrellation and the auto-covariance are the same
ECE594I notes, M. Rodwell, copyrighted
Variation of a random process with time
Note that RXX 0( )= E X t( )X t( )[ ]= σX2 gives the variance of the random process.
Th t l ti f ti i i f th d d th l tiThe autocorrelation function gives us variance of the random process and the correlation between its values for two moments in time. If the process is Gaussian, this is enough to completely describe the process.
)(R )(τxxR
Narrow autocorrelation:Fast variation
τ
Fast variation
Broad autocorrelationSlow variation
)(τxxR
τ
ECE594I notes, M. Rodwell, copyrighted
Autocorrelation is an Estimate of the Variation with Time
If random variables X and Y are Jointly Gaussian, and have zero mean, then knowledge of the value y of the outome of Y results in a best estimate of X as follows:follows:
E X Y = y[ ]= X Y = y =RXY
σY2 y
"The expected value of the random variable X, given that the random variable Y has value y is ..." Hence the autocorrellation function tells us the degree to which the signal at time t isHence, the autocorrellation function tells us the degree to which the signal at time t is related to the signal at time τ+t A narrow autocorrelation is indicative of a quickly-varying random process
ECE594I notes, M. Rodwell, copyrighted
Power spectral densities
The autocorrellation function describes how a random process evolves with time. Find its Fourier transform:
( ) ( ) τωττω djRS XXXX )exp(−= ∫+∞
∞−
This is called the power spectral density of the signal. Remembering the usual Fourier transform relationships, if the power spectrum is broad, the autocorrellation function is narrow, and the signal varies rapidly--it has content at high frequencies, and the voltages of any two points are strongly related only if the two points are close together in time. If the power spectrum is narrow, the autocorrellation function is broad, and the signal
i l l i h l l f i d h l f ivaries slowly--it has content only at low frequencies and the voltages of any two points are strongly related unless if the two points are broadly separated in time.
ECE594I notes, M. Rodwell, copyrighted
Power spectral densities
Rxx(tau)
x(t)
Sxx(omega)
x(t)
tautime
omega
Rxx(tau)
x(t)
Sxx(omega)
x(t)
tau timeomega
ECE594I notes, M. Rodwell, copyrighted
Power Spectral Densities
id itt lth t thR ll
( ) ( )
function ationautocorrel theof ransform Fourier ttheisdensity spectralpower that theRecall
∫∞+
( ) ( )
thatso holds, transforminverse The
)exp(XXXX djRS τωττω ∫∞−
−=
( ) ( )
lifi l
)exp(21
XXXX djSR ωωτωτ ∫∞+
∞−π=
( )21)0(
ly,specifical
2XXXXX dSR ωωσ ∫
∞+
∞π==
power. theus give lldensity wi spectralpower the gintegratin thenprocess, theinpower thecalled is if So, 2
Xσ∞−
density" spectralpower " term,for the ionjustificat theis This
ECE594I notes, M. Rodwell, copyrighted
Back To Ergodicity (1)
:space) sampleover (average ationautocorrel lStatistica
particularaof)subscripts(notefunctionationautocorrelTime
.)]()([)( tXtXERXX ττ +=
)()(1lim)]()([)(
. process random theof )( outcomeparticulara of )subscripts(notefunction ationautocorrel Time
2/
dttxtxtxtxAR
X(t)txT
i
τττ +=+= ∫).( outcomes *all*for )()( then Ergodic,is )( If
)()(lim)]()([)(2/
txRRtX
dttxtxT
txtxAR
iXXxx
iT
iTiixx
ii
ii
ττ
τττ
=
+=+= ∫−
∞→
ECE594I notes, M. Rodwell, copyrighted
Back To Ergodicity (2)
).( outcomes *all*for )()( then Ergodic,is )( If txRRtX iXXxx iiττ =
{ } { })()()()(B t
)()]()([)(
*11 jXjXjSR
RtxtxAR
--
XXiixx iiτττ
FF
=+=
{ } { }).( of ransform Fourier t theis )(problems) (notation where
)()()()(But 11
txjX
jXjXjSR
ii
iixxxx iiii
ω
ωωωτ FF ==
phase.Fourier different a haslikely )( outcome Each.||)(|| agnitude Fourier Msame thehas )( outcomeevery So,
txjXtx
i
i ω
ECE594I notes, M. Rodwell, copyrighted
Correlated Random Processes
l dlli ibT Y(t).and X(t) processes random woConsider t
related.lly statisticabecan processes Two
( ) ( ) ( )[ ]processes theof function oncorrellati-cross theDefine
( ) ( ) ( )[ ]ττ tYtXERXY +=
( ) ( ) τωττω djRS )exp(
:follows asdensity spectral-crossa have They will
∫∞+
=( ) ( )
( ) ( )
τωττω
djSR
djRS XYXY
)(1h fd
)exp(
∫
∫∞+
∞−
−=
( ) ( ) ωωτωτ djSR XYXY )exp(2
thereforeand ∫∞−π
=
ECE594I notes, M. Rodwell, copyrighted
Single-Sided Hz-based Spectral Densities
( ) [ ] ( ) djjStXtXER )(1)()(
DensitiesSpectral Sided-Double
∫∞+
( ) [ ] ( )
( ) ( )
ωωτωττ djjStXtXER XXXX )exp(2
)()(π
=+=
∫
∫∞+
∞−
( ) ( ) τωττω djRjS XXXX )exp(−= ∫∞−
1
DensitiesSpectral based- HzSided-Single∞+
( ) [ ] ( ) τπττ dffjjfStXtXER XXXX )2exp(~21)()( =+= ∫
+
+
∞−
( ) ( ) ττπτ dfjRjfS XXXX )2exp(2~ −= ∫∞+
∞−
ECE594I notes, M. Rodwell, copyrighted
Single-Sided Hz-based Spectral Densities- Why ?
{ },bandwidththeinpower signal The?notation Why this
highlow ff{ }
( ) ( ) ( )~~21~
21Power
,pghighhighlow
highlow
dfjfSdfjfSdfjfS
fff
XX
f
XX
f
XX =+= ∫∫∫−
( )~
22lowlowhigh fff∫∫∫
−
( ) bandwidth signal of per Hz
powersignalofWattshedirectly t is ~ jfSXX→
.frequency the toclose lying sfrequencieat f
ECE594I notes, M. Rodwell, copyrighted
Single-Sided Hz-based Cross Spectral Densities
( ) [ ] ( ) djjStYtXER XYXY )exp(21)()(
DensitiesSpectralCross Sided-Double
ωωτωττπ
=+= ∫∞+
( ) ( ) djRjS XYXY )exp(
2
τωττω −=
π
∫
∫∞+
∞
∞−
DensitiesSpectral Cross based- HzSided-Single
∞−
( ) [ ] ( ) dffjjfStYtXER XYXY )2exp(~21)()( τπττ =+= ∫
∞+
∞+
∞−
( ) ( ) dfjRjfS XYXY )2exp(2~ ττπτ −= ∫+
∞−
( ) XYdfdjfSXY as writtenoften also is ~
ECE594I notes, M. Rodwell, copyrighted
Example: Cross Spectral Densities
( ) ( )( )[ ]
tYtXtV )()()( +=X
RV
P=V2/R
( ) ( )( )[ ]( ) ( ) ( ) ( )RRRR
tYtXtYtXER
YXXYYYXX
VV
)()()()(
+++=++++=τττττττ RY
( ) ( ) ( ) ( ) ( )( ) ( ) ( ){ }jSjSjS
jSjSjSjSjS
XYYYXX
XYXYYYXXVV
Re2
*
⋅++=+++=
ωωωωωωωω
( ) ( ) ( ) ( ){ }~~~~densities spectral sided-single in Or,
( ) ( ) ( ) ( ){ }jfSjfSjfSjfS XYYYXXVV Re2 ⋅++=
ECE594I notes, M. Rodwell, copyrighted
Example: Cross Spectral Densities
[ ] ( )/0/)()( valueexpected has /)( Power The 2
==
VV RRRtVtVERtVP X
RV
P=V2/R
[ ] ( )
, and between bandwidth thein And
)()(
highlow
VV
ff
RY
( )
l )ib d id h( hfi hi
...~∫=high
low
f
fVV dfjfSP
R.in dissipatedpower (expected) total thegivesrelevant)isbandwidthever (over whatfrequency respect to withgIntegratin
relevant. isdensity spectral-cross that theNote
ECE594I notes, M. Rodwell, copyrighted
Our Notation for Spectral Densities and Correlations
)(),()(),(frequencyoffunction)()( timeof function
OutcomeProcessRandom
ωω jvjfvjVjfVtvtV
[ ] [ ][ ] [ ]
⎪⎨
⎧ =
⎨⎧ =
+=+=)()(
)()(
)()()()()()(function ationautocorrel)(),()(),(frequency of function
*
τωτω
ττττωω
RjSRjS
tvtvARtVtVERjvjfvjVjfV
vvvvVVVV
vvVV
FF[ ]
[ ] [ ]+=+=
⎪⎩
⎪⎨
==
⎩⎨⎧
=
)()()()()()(function lationcrosscorre)2/(2)(~)()()(
)2/(2)(~)()(
density spectralpower *
ττττπωωωω
πω
tytvARtYtXERjSjfS
jvjvjSjSjfS
j
xyXY
vvvv
vvVVVV
VVVV
[ ] [ ]
[ ] [ ]
⎪
⎪⎨
⎧==
⎩⎨⎧
==
)2/(2)(~)()()(
)()(
)2/(2)(~)()(
density spectral cross
)()()()()()(
* ωωωτω
πωτω
SfSjyjxjS
RjS
jSjfSRjS
y
xy
xyxy
XYXY
XYXY
xyXY
FF
⎪⎩ =⎩ )2/(2)(
)()(πωjSjfS
jjfxyxy
XYXY
. tesimply wri can we),(or )(her clear whetit makescontext When vjvvtvv ω==
)()()()( and )()()()( processes ergodic stationaryFor
** ωωωωωωωω jyjxjSjSjvjvjSjS xyXYvvVV ====
ECE594I notes, M. Rodwell, copyrighted
Example: Noise passing through filters & linear electrical networks
),(functiontransferand)(responseimpulsehasfiltertheIf ωjhth)()()( ),()(any for then
),(functiontransfer and )(responseimpulse hasfilter theIfωωω
ωjvjhjvtvtv
jhth
inoutoutin =→
)()()()()()(So
2
*** ωωωωωω jvjhjvjhjvjv ininoutout =
)()()(
)()()(2
2
ωωω
ωωω
jSjhjS
jSjhjS
ininoutout
ininoutout
VVVV
vvvv
=
=
)()()()()()()()( **
ωωωωωωωω
jSjhjSjvjvjhjvjv inininout
==
Vin(t) h(t) Vout(t))()()(
)()()(
ωωω
ωωω
jSjhjS
jSjhjS
inininout
inininout
VVVV
vvvv
=
=
densities. spectral based- Hzsided-single tochange to trivialisIt