ii - random processes - applications to signal ... ?· ... random processes - applications to...

Download II - Random Processes - Applications to Signal ... ?· ... Random Processes - Applications to Signal…

Post on 11-Jul-2018

213 views

Category:

Documents

0 download

Embed Size (px)

TRANSCRIPT

  • 06/19/14 EC3410.SuFY14/MPF - Section II 1

    II - Random Processes - Applications to Signal & Information Processing

    [p. 3] Random signal/sequence definition [p. 6] Signal mean, variance, autocorrelation & autocovariance sequence, normalized cross-correlation sequence [p. 16] Statistical characterization of random signals

    I.I.D. Random process Stationarity Wide sense stationarity (wss) Jointly wide sense stationarity (jointly wss) Correlation & cross-correlation for stationary RPs Signal average Ergodicity Concept of white noise, colored noise, Bernoulli process, Random walk

    [p. 51] Application: MA Processes: definitions and pdf properties [p. 57] Random process properties [p. 66] Multiple Random Processes Joint Properties [p. 72] Application to data analysis How to assess signal stationarity [p. 76] Application to data analysis - How to check IID assumption

    Autocorrelation -- Lag plot [p. 89] Application: target range detection [p. 91] Introduction to the spectrogram [p. 95] Application: Gas furnace reaction time [p. 98] Application: Evaluating correlation between random signals [p. 101] Application: Evaluating correlation status between random signals [p. 102] Application: Detection of the periodicity of stationary signals in noisy environments [p. 104] Correlation matrix properties for a stationary process [p. 108] How to estimate correlation lags; biased/unbiased estimator issues [p. 116] Frequency domain description for a stationary process

    Power spectral density (PSD) definition & properties [p. 127] Principal Component Analysis (PCA, DKLT)

    Applications to biometrics (face recognition) Applications to network traffic flow anomaly detection

    [p. 158] Appendices [p. 184] References

  • 06/19/14 EC3410.SuFY14/MPF - Section II 2

    Examples [p. 7] Example 1 [p. 9] Example 2 [p. 11] Example 3 [p. 28] Example 4 [p. 33] Example 5 [p. 35] Example 6 [p. 37] Example 7 [p. 39] Example 8 [p. 50] Example 9 [p. 58] Example 10 [p. 60] Example 11 [p. 70] Example 12 [p. 75] Example 13; Pack2Data1 [p. 87] Example 14 [p. 88] Example 15; Pack2Data3 [p. 94] Example 16; Pack2Data2

    [p. 98] Example 17 [p. 101] Example 18; Pack2Data6 [p. 102] Example 19 [p. 103] Example 20; Pack2Data4 [p. 107] Example 21 [p. 114] Example 22 [p. 117] Example 23 [p. 119] Example 24 [p. 120] Example 25 [p. 123] Example 26

  • 06/19/14 EC3410.SuFY14/MPF - Section II 3

    Random Signal/Sequence - definitions

    A RP is a mapping function that attributes a function x(t) = x(t,) (for continuous signal case) or x(n)=x(nTs, ) (for discrete signal case) to each outcome of the random experiment

    x(t, 1)

    x(t, 3)

    x(t, 2)

    1

    2

    3

    t

    t

    t

    ...

    ...

    ...

    n

    n

    n x(n, 1)

    x(n, 3)

    x(n, 2)

    1

    2

    3

    Continuous random

    signal/process

    Discrete random

    signal/process

  • 06/19/14 EC3410.SuFY14/MPF - Section II 4

    Consider sequence x(n) =x(n,) for a fixed t, x(n) is a Random Variable (RV)

    x(n): can be infinite dimensional x(n,) for fixed RV : called realization/trial of the random process

    Random Signal/Sequence - definitions, cont

    ...

    ...

    ...

    n

    n

    n x(n, 1)

    x(n, 3)

    x(n, 2)

    1

    2

    3

  • 06/19/14 EC3410.SuFY14/MPF - Section II 5

    ...

    ...

    ...

    n

    n

    n x(n, 1)

    x(n, 3)

    x(n, 2)

    1

    2

    3

    Example: x(n,) = cos(n/10), where = U[0,1].

    For a discrete random signal

  • 06/19/14 EC3410.SuFY14/MPF - Section II 6

    Signal mean value (ensemble average):

    Signal variance: ( ) ( ){ }( ){ } ( )

    22

    2 2

    ( )x x

    x

    n E x n m n

    E x n m n

    =

    =

    n1 n

    x[n]

    n2

    2 1n n= lag

    ( ) ( ){ }xm n E x n=

    Discrete signal case

    t1 t

    x(t) 2 1t t = Time lag (sec)

    t2

    Continuous signal case

    Note: dimensionless!

  • 06/19/14 EC3410.SuFY14/MPF - Section II 7

    x(n,) = cos(n/5), where = U[0,1]. Example 1: Compute process mean and variance

  • 06/19/14 EC3410.SuFY14/MPF - Section II 8

    Signal autocorrelation sequence:

    n1 n

    x[n]

    n2

    2 1k n n= lag

    ( ) ( ) ( ) ( ){ }*1 2 1 2 1 2, ,xx xR n n R n n E x n x n= =measures the dependency between values of the process at two different times. Allows to evaluate: 1) How quickly a random signal changes with respect to time, 2) The amount of memory a signal may have, 3) Whether the process has a periodic component and what the expected frequency might be, etc

  • 06/19/14 EC3410.SuFY14/MPF - Section II 9

    Let x(n) be a real valued process defined as x(n, )= where is defined as a RV with mean 0 and variance 2x. Compute: Rx(k,n)

    Example 2:

    x(n, 1)

    x(n, 3)

    x(n, 2)

    1

    2

    3

  • 06/19/14 EC3410.SuFY14/MPF - Section II 10

  • 06/19/14 EC3410.SuFY14/MPF - Section II 11

    x(n,) = cos(n/5+), where = U[0,2]. Example 3: Compute Rx(n1,n2)

  • 06/19/14 EC3410.SuFY14/MPF - Section II 12

    ( ) ( )( ) ( ) ( )( ){ }( ) ( )

    *

    1 2 1 1 2 2

    *

    1 2 1 2

    ( , )

    ( , )

    xx x x

    x x x

    C n n E x m x m

    R n n m m

    n n n n

    n n

    =

    =

    Signal autocovariance function (remove impact due to process mean):

    Signal normalized correlation function (remove impact due to process mean and normalizes max value to 1):

    ( ) ( )( ) ( )1 2

    1 21 2

    ,, xx

    x x

    C n nn n

    n n

    = ( )1 2| , | 1 !!x n n

  • 06/19/14 EC3410.SuFY14/MPF - Section II 13

    Signal cross-correlation function:

    ( )1 2,xyR n n =

    Measures the dependency between values of two processes at two different times. Allows to evaluate whether two processes are related in some linear fashion, or how well their dependence can be approximated by a linear relationship.

    Will NOT evaluate nonlinear dependence (as with the correlation coefficient defined earlier for random variables)

    Warning: Correlation does NOT imply causation

  • 06/19/14 EC3410.SuFY14/MPF - Section II 14

    Signal cross-covariance function:

    ( ) ( ) ( ) ( )*1 2 1 2 1 2, ,xy xy x yC n n R n n m n m n=

    Similar to cross-correlation function: measures the dependency between values of two processes at two different times,

    but also

    Removes impact of the mean value.

    Note: unless there is a good reason to keep the signal

    means, best to remove or use covariance based expressions!

  • 06/19/14 EC3410.SuFY14/MPF - Section II 15

    Normalized cross-correlation function:

    ( ) ( )( ) ( )1 2

    1 21 2

    ,, xyxy

    x y

    C n nn n

    n n

    = ( )1 2| , | 1 !!xy n n

  • 06/19/14 EC3410.SuFY14/MPF - Section II 16

    Random signals are characterized by joint distribution (or density) of samples

    Fx(x1, x2, , xk, n1,, nk) = Pr [x(n1) x1, x(nk) xk]

    F(.) is highly complex to compute - difficult or impossible to obtain in practice

    Statistical Characterization of Random Signals

  • 06/19/14 EC3410.SuFY14/MPF - Section II 17

    Independent, Identically Distributed (I.I.D.) Random Process:

    A Random Process is said to be:

    An independent process (i.e., independent of itself at earlier and/or later times) if for any time index nk :

    fx(x1, x2,,xk;n1,,nk) = f1(x1;n1)fk(x2;nk)

    A RP process is IID if all RVs obtained for all time indices have the same pdf fx(x)

    Note: I.I.D. processes have no memory, where a future value would depend on past values they can be viewed as building blocks for more realistic random signals. Mean of I.I.D. Process:

    mx(n) = E{x(n)} =

  • 06/19/14 EC3410.SuFY14/MPF - Section II 18

    Independent, Identically Distributed (I.I.D.) RP, cont

    Autocovariance of an IID process

    { }{ } { }

    { }

    *1 2 1 1 2 2

    *1 1 2 2 1 2

    21 1 1 2

    ( , ) ( ( ) ( ))( ( ) ( ))

    ( ( ) ( )) ( ( ) ( )) ,

    | ( ( ) ( )) | ,

    =

    x x x

    x x

    x

    C n n E x n m n x n m n

    E x n m n E x n m n n n

    E x n m n n n

    =

    = =

    Autocorrelation of an IID process:

    Rx(n1, n2) =

  • 06/19/14 EC3410.SuFY14/MPF - Section II 19

    ( , ) 0.05* ( ), ~ (0,1), ~ (0,1)x n n w n

    N w N

    = + +

    I.I.D. process ?

    [ ( , )]E x n

  • 0 5 10 15 20 25 30 35 40 45 500

    0.5

    1

    1.5

    2

    2.5

    3

    Time

    E[x(n, ksi) ]

    06/19/14 EC3410.SuFY14/MPF - Section II 20

    ( , ) 0.05* ( ), with ~ (0,1), ( )~N(0,1)x n n w n

    N w n

    = + +

    I.I.D. process ?

    [ ( , )]E x n

    0 5 10 15 20 25 30 35 40 45 500

    5

    10RP x(n)=ksi + 0.05*n+w(n)

    Time

    Tria

    l 1

    0 5 10 15 20 25 30 35 40 45 50-5

    0

    5

    Time

    Tria

    l 2

    0 5 10 15 20 25 30 35 40 45 50-5

    0

    5

    Time

    Tria

    l 3

    0 5 10 15 20 25 30 35 40 45 50-5

    0

    5

    Time

    Tria

    l 4

  • 06/19/14 EC3410.SuFY14/MPF - Section II 21

    Data Analysis Application What does the I.I.D assumption mean

Recommended

View more >