# ii - random processes - applications to signal ... ?· ... random processes - applications to...

Post on 11-Jul-2018

213 views

Embed Size (px)

TRANSCRIPT

06/19/14 EC3410.SuFY14/MPF - Section II 1

II - Random Processes - Applications to Signal & Information Processing

[p. 3] Random signal/sequence definition [p. 6] Signal mean, variance, autocorrelation & autocovariance sequence, normalized cross-correlation sequence [p. 16] Statistical characterization of random signals

I.I.D. Random process Stationarity Wide sense stationarity (wss) Jointly wide sense stationarity (jointly wss) Correlation & cross-correlation for stationary RPs Signal average Ergodicity Concept of white noise, colored noise, Bernoulli process, Random walk

[p. 51] Application: MA Processes: definitions and pdf properties [p. 57] Random process properties [p. 66] Multiple Random Processes Joint Properties [p. 72] Application to data analysis How to assess signal stationarity [p. 76] Application to data analysis - How to check IID assumption

Autocorrelation -- Lag plot [p. 89] Application: target range detection [p. 91] Introduction to the spectrogram [p. 95] Application: Gas furnace reaction time [p. 98] Application: Evaluating correlation between random signals [p. 101] Application: Evaluating correlation status between random signals [p. 102] Application: Detection of the periodicity of stationary signals in noisy environments [p. 104] Correlation matrix properties for a stationary process [p. 108] How to estimate correlation lags; biased/unbiased estimator issues [p. 116] Frequency domain description for a stationary process

Power spectral density (PSD) definition & properties [p. 127] Principal Component Analysis (PCA, DKLT)

Applications to biometrics (face recognition) Applications to network traffic flow anomaly detection

[p. 158] Appendices [p. 184] References

06/19/14 EC3410.SuFY14/MPF - Section II 2

Examples [p. 7] Example 1 [p. 9] Example 2 [p. 11] Example 3 [p. 28] Example 4 [p. 33] Example 5 [p. 35] Example 6 [p. 37] Example 7 [p. 39] Example 8 [p. 50] Example 9 [p. 58] Example 10 [p. 60] Example 11 [p. 70] Example 12 [p. 75] Example 13; Pack2Data1 [p. 87] Example 14 [p. 88] Example 15; Pack2Data3 [p. 94] Example 16; Pack2Data2

[p. 98] Example 17 [p. 101] Example 18; Pack2Data6 [p. 102] Example 19 [p. 103] Example 20; Pack2Data4 [p. 107] Example 21 [p. 114] Example 22 [p. 117] Example 23 [p. 119] Example 24 [p. 120] Example 25 [p. 123] Example 26

06/19/14 EC3410.SuFY14/MPF - Section II 3

Random Signal/Sequence - definitions

A RP is a mapping function that attributes a function x(t) = x(t,) (for continuous signal case) or x(n)=x(nTs, ) (for discrete signal case) to each outcome of the random experiment

x(t, 1)

x(t, 3)

x(t, 2)

1

2

3

t

t

t

...

...

...

n

n

n x(n, 1)

x(n, 3)

x(n, 2)

1

2

3

Continuous random

signal/process

Discrete random

signal/process

06/19/14 EC3410.SuFY14/MPF - Section II 4

Consider sequence x(n) =x(n,) for a fixed t, x(n) is a Random Variable (RV)

x(n): can be infinite dimensional x(n,) for fixed RV : called realization/trial of the random process

Random Signal/Sequence - definitions, cont

...

...

...

n

n

n x(n, 1)

x(n, 3)

x(n, 2)

1

2

3

06/19/14 EC3410.SuFY14/MPF - Section II 5

...

...

...

n

n

n x(n, 1)

x(n, 3)

x(n, 2)

1

2

3

Example: x(n,) = cos(n/10), where = U[0,1].

For a discrete random signal

06/19/14 EC3410.SuFY14/MPF - Section II 6

Signal mean value (ensemble average):

Signal variance: ( ) ( ){ }( ){ } ( )

22

2 2

( )x x

x

n E x n m n

E x n m n

=

=

n1 n

x[n]

n2

2 1n n= lag

( ) ( ){ }xm n E x n=

Discrete signal case

t1 t

x(t) 2 1t t = Time lag (sec)

t2

Continuous signal case

Note: dimensionless!

06/19/14 EC3410.SuFY14/MPF - Section II 7

x(n,) = cos(n/5), where = U[0,1]. Example 1: Compute process mean and variance

06/19/14 EC3410.SuFY14/MPF - Section II 8

Signal autocorrelation sequence:

n1 n

x[n]

n2

2 1k n n= lag

( ) ( ) ( ) ( ){ }*1 2 1 2 1 2, ,xx xR n n R n n E x n x n= =measures the dependency between values of the process at two different times. Allows to evaluate: 1) How quickly a random signal changes with respect to time, 2) The amount of memory a signal may have, 3) Whether the process has a periodic component and what the expected frequency might be, etc

06/19/14 EC3410.SuFY14/MPF - Section II 9

Let x(n) be a real valued process defined as x(n, )= where is defined as a RV with mean 0 and variance 2x. Compute: Rx(k,n)

Example 2:

x(n, 1)

x(n, 3)

x(n, 2)

1

2

3

06/19/14 EC3410.SuFY14/MPF - Section II 10

06/19/14 EC3410.SuFY14/MPF - Section II 11

x(n,) = cos(n/5+), where = U[0,2]. Example 3: Compute Rx(n1,n2)

06/19/14 EC3410.SuFY14/MPF - Section II 12

( ) ( )( ) ( ) ( )( ){ }( ) ( )

*

1 2 1 1 2 2

*

1 2 1 2

( , )

( , )

xx x x

x x x

C n n E x m x m

R n n m m

n n n n

n n

=

=

Signal autocovariance function (remove impact due to process mean):

Signal normalized correlation function (remove impact due to process mean and normalizes max value to 1):

( ) ( )( ) ( )1 2

1 21 2

,, xx

x x

C n nn n

n n

= ( )1 2| , | 1 !!x n n

06/19/14 EC3410.SuFY14/MPF - Section II 13

Signal cross-correlation function:

( )1 2,xyR n n =

Measures the dependency between values of two processes at two different times. Allows to evaluate whether two processes are related in some linear fashion, or how well their dependence can be approximated by a linear relationship.

Will NOT evaluate nonlinear dependence (as with the correlation coefficient defined earlier for random variables)

Warning: Correlation does NOT imply causation

06/19/14 EC3410.SuFY14/MPF - Section II 14

Signal cross-covariance function:

( ) ( ) ( ) ( )*1 2 1 2 1 2, ,xy xy x yC n n R n n m n m n=

Similar to cross-correlation function: measures the dependency between values of two processes at two different times,

but also

Removes impact of the mean value.

Note: unless there is a good reason to keep the signal

means, best to remove or use covariance based expressions!

06/19/14 EC3410.SuFY14/MPF - Section II 15

Normalized cross-correlation function:

( ) ( )( ) ( )1 2

1 21 2

,, xyxy

x y

C n nn n

n n

= ( )1 2| , | 1 !!xy n n

06/19/14 EC3410.SuFY14/MPF - Section II 16

Random signals are characterized by joint distribution (or density) of samples

Fx(x1, x2, , xk, n1,, nk) = Pr [x(n1) x1, x(nk) xk]

F(.) is highly complex to compute - difficult or impossible to obtain in practice

Statistical Characterization of Random Signals

06/19/14 EC3410.SuFY14/MPF - Section II 17

Independent, Identically Distributed (I.I.D.) Random Process:

A Random Process is said to be:

An independent process (i.e., independent of itself at earlier and/or later times) if for any time index nk :

fx(x1, x2,,xk;n1,,nk) = f1(x1;n1)fk(x2;nk)

A RP process is IID if all RVs obtained for all time indices have the same pdf fx(x)

Note: I.I.D. processes have no memory, where a future value would depend on past values they can be viewed as building blocks for more realistic random signals. Mean of I.I.D. Process:

mx(n) = E{x(n)} =

06/19/14 EC3410.SuFY14/MPF - Section II 18

Independent, Identically Distributed (I.I.D.) RP, cont

Autocovariance of an IID process

{ }{ } { }

{ }

*1 2 1 1 2 2

*1 1 2 2 1 2

21 1 1 2

( , ) ( ( ) ( ))( ( ) ( ))

( ( ) ( )) ( ( ) ( )) ,

| ( ( ) ( )) | ,

=

x x x

x x

x

C n n E x n m n x n m n

E x n m n E x n m n n n

E x n m n n n

=

= =

Autocorrelation of an IID process:

Rx(n1, n2) =

06/19/14 EC3410.SuFY14/MPF - Section II 19

( , ) 0.05* ( ), ~ (0,1), ~ (0,1)x n n w n

N w N

= + +

I.I.D. process ?

[ ( , )]E x n

0 5 10 15 20 25 30 35 40 45 500

0.5

1

1.5

2

2.5

3

Time

E[x(n, ksi) ]

06/19/14 EC3410.SuFY14/MPF - Section II 20

( , ) 0.05* ( ), with ~ (0,1), ( )~N(0,1)x n n w n

N w n

= + +

I.I.D. process ?

[ ( , )]E x n

0 5 10 15 20 25 30 35 40 45 500

5

10RP x(n)=ksi + 0.05*n+w(n)

Time

Tria

l 1

0 5 10 15 20 25 30 35 40 45 50-5

0

5

Time

Tria

l 2

0 5 10 15 20 25 30 35 40 45 50-5

0

5

Time

Tria

l 3

0 5 10 15 20 25 30 35 40 45 50-5

0

5

Time

Tria

l 4

06/19/14 EC3410.SuFY14/MPF - Section II 21

Data Analysis Application What does the I.I.D assumption mean

Recommended