Transcript

1 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing

NOTES ON

STATIONARY RANDOM PROCESS AND DIGITAL SIGNAL PROCESSING

Prepared by Le Thai Hoa

2004

2 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing

1. STATIONARY RANDOM PROCESS

1.1. Basic concepts (1) Continuous random process:

)}()}...(),(),({)( 321 txtxtxtxtx Kk , t

Where: { }: Ensemble of sample functions xk(t)

k: Index of sample function (k=1,2,3…K)

t: Time variable

Random process {xk(t)} = Ensemble of sample function xk(t)

(2) The random process is called as the K-variate random process (multi-variate

random process)

Ensemble (sample records) of random signal

(3) For discrete sample function, discrete values of any sample random function are

measured at certain time points t1, t2, t3, … tN (N: number of sampling values of

sample function)

t

t

t t+

x1(t)

kth sample function

1st sample function

xk(t) Time shift

3 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing

)}()}...(),(),()( 321 Nkkkkk txtxtxtxtx : Discrete sample function

1.2. Classification of random process (1) Classification of random process can be widely expressed as follows

Classification of random processes

1.3. Representation of random process: (1) Time-domain representation (as raw formats and sources)

(2) Frequency-domain representation (due to Fourier Transform)

(3) Time-frequency representation (due to Wavelet Transform)

1.4. Characteristics of random process

Basic statistical characteristics of two arbitrary random processes )(txk and

)(tyk

(1) Mean value (Expectation): First-order statistical moment

dttx

NLimtxEt kNkx )(1)]([)(

Random process or Stochastic field

Stationary process

Non-stationary processes

Ergodic process

Non-ergodic signals

Gaussian process

Non-Gaussian process

4 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing

dtty

NLimtyEt kNky )(1)]([)(

(2) Variance and covariance: Second-order moment

)()]([]))()([()( 2222 ttxEttxEt kkxkx : Variance

)()]([]))()([()( 2222 ttyEttyEt ykyky : Variance

)(*)()](*)([))]()())(()([()(

tttytxEttyttxEtC

yxkk

ykxkxy

: Covariance

))]()())(()([()( ttxttxEC xkxkxx : Covariance

))]()())(()([()( ttyttyEC ykykyy : Covariance

Note: Zero mean value process xk(t): 0)( tx

]))([()( 22 txEt kx : Variance

]))([()( 22 tyEt ky : Variance

)()0( 2 tC xxx

)()0( 2 tC yyy

(3) Mean square and root mean square

)]([)( 2 txEtC kxx : Mean square

)]([)( 2 tyEtC kyy : Mean square

Note: Zero mean value process xk(t): 0)( tx

5 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing

Variance )(2 tx = Mean Square Cxx(t)

(4) Correlations: Second-order moment

)](*)([)( txtxER kkxx : Autocorrelation

)](*)([)( tytyER kkyy : Autocorrelation

)](*)([)( tytxER kkxy : Cross-correlation

: arbitrary time (time shift or time lag)

Note: xxxxx RC 2)()(

yyyyy RC 2)()(

yxxyxy RC )()(

Zero mean random process: 0)( tx , 0)( ty

)()( xxxx RC

)()( yyyy RC

)()( xyxy RC

(5) Correlation coefficients

)()(

)()()(

2

xx

xxx

xx

xxxx R

RRC

: Auto-correlation coefficient

)()(

)()(

)(2

yy

yyy

yy

yyyy R

RRC

: Auto-correlation coefficient

)()(

)()(

)(

xy

yxxy

xy

xyxy R

RRC

: Cross-correlation coefficient

6 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing

x

xx

xx

xxxx

CRC

2)(

)()()(

y

yy

yy

yyyy

CRC

2

)()()(

)(

yx

xy

xy

xyxy

CRC

)(

)()(

)(

Note 1:

i) )()( xxxx RC

ii) )0()( xxxx CC

iii) )0()( xxxx RR

iv) 2)0( xxxC and

2)0( yyyC

v) )0(*)0(|)(| 2yyxxxy CCC

vi) 222|)(| yxxyC

vii) )0(*)0(|)(| 2yyxxxy RRR

viii) )()( xxxx RR and )()( yyyy RR

)()( yxxy RR

Note 2: 0

i) 2)0( xxxC : Variance

7 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing

ii) 2)0( yyyC : Variance

iii) 22 )]([)0( xxx txER : Mean square

iv) 22 )]([)0( yyy tyER : Mean square

(6) Power spectral density (PSD) function in frequency-domain

1.5. Power spectral density (PSD) PSD function can be computed by following methods: i) Via correlation function (by

definition), ii) Via Fourier transform and iii) Via filter-squaring-averaging computation

(1) Spectra via correlation (by Fourier Transform of correlation)

By definition of spectral density through the Fourier Transform:

deRfS fjxxxx

2*)()(

: Auto-spectral density function

deRfS fjyyyy

2*)()(

: Auto-spectral density function

deRfS fjxyxy

2*)()(

: Cross-spectral density

function

Inverse Fourier Transform:

dfefSR fjxxxx

2*)()(

: Auto-correlation

dfefSR fjyyyy

2*)()(

: Auto-correlation

dfefSR fjxyxy

2*)()(

: Cross-correlation

8 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing

Sxx(f), Syy(f), Sxy(f): Two-sided spectra, f[-,]

One-sided spectral densities

dfRdfRfS xxxxxx 2cos*)(22cos*)()(0

0

2cos*)(2)( dffSR xxxx

Changing the two-sided spectral density Sxx(f) with f[-,] to the one-sided spectral

density Gxx(f) with f[0,]

)(2)( fSfG xxxx

)(2)( fSfG yyyy

)(2)( fSfG xyxy

Thus,

dfRfG xxxx 2cos*)(4)(0

; f[0,]

dfffGR xxxx 2cos*)()(0

; f[0,]

Real part and imaginary part of one-sided cross-spectral density:

f(Hz)

Spectra

[-,0] [0,]

Gxx(f)=2Sxx(f): One-sided

Sxx(f): Two-sided

9 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing

)()(*)(2)( 2 fiQfCdeRfG xyxyfj

xyxy

)( fCxy : Co-spectrum

)( fQxy : Quadratic spectrum

Writing in standard form: )(|)(|)( fj

xyxyxyefGfG

Where:

)()(|)(| 22 fQfCfG xyxyxy

)()(

tan)( 1

fCfQ

fxy

xyxy

)(cos|)(|)( ffGfC xyxyxy

)(sin|)(|)( ffGfQ xyxyxy

dfffiQffCR xyxyxy ]2sin)(2cos)([)(0

dffCR xyxy

0

)()0(

(2) Spectra via Fourier transform

Fourier Transform (Kinchint-Weiner’s pair):

dtetxfX ftjkk

0

2)()(

dfefXfx ftjkk

0

2)()(

10 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing

Spectral density function:

]|)([|12)( 2fXET

LimfG kTxx

]|)([|12)( 2fYET

LimfG kTyy

)](*)([12)( fYfXET

LimfG kkTxy

1.5. Coherence Coherence plays the same role as the correlation coefficient. The correlation coefficient

is expressed in time domain, whereas the coherence in frequency domain.

(1) Correlation coefficient:

)0(*)0(|)(| 2yyxxxy RRR

)]([*)]([|)](*)([| 222 tyEtxEtytxE

yxyyxxxy CCC 222 *)0(*)0(|)(|

]))([(*]))([(|)])((*))([(| 222yxyx tyEtxEtytxE

Thus,

yx

xyxy

C

*

)()( : Correlation coefficient

1)(0 xy

:0)( xy x(t), y(t) Uncorrelated;

11 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing

:1)( xy x(t), y(t) Correlated;

(2) Coherence:

)(*)(|)(| 2 fGfGfG yyxxxy

)(*)(|)(|

)(2

2

fGfGfG

fyyxx

xyxy

1)(0 2 fxy

(3) Role of coherence function )(2 fxy (constant-parameter linear systems) can be

interpreted as fractional portion of the mean square value at the output y(t) that is

contributed by the input x(t) at frequency value f. In contrast, the quantity

)](1[ 2 fxy is the portion of mean square value of output y(t) not be contributed by

input x(t) at frequency f.

Note: The role of coherence function )(2 fxy is similar to the correlation

coefficient function )(2 xy . In constant-parameter linear systems, the

coherence has some following possibilities:

1) 0)(2 fxy : x(t) and y(t) uncorrelated (unrelated)

2) 1)(2 fxy : x(t) and y(t) correlated (unrelated)

3) 1)(0 2 fxy : x(t) and y(t) some possible situations exist:

a. Extra noise

b. Non-linear system between input x(t) and output y(t)

12 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing

c. SIMO or MISO

)(*)()()(

)(2

2

fGfGfiQfC

fyyxx

xyxyxy

)(*)()(

)(2

2

fGfGfC

fyyxx

xyxy : Coherence

)( fCxy : Co-spectrum (Real part of Cross-spectrum)

)(),( fGfG yyxx : Auto-spectra

13 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing

2. DIGITAL SIGNALS AND CLASSIFICATION

2.1. Digital signals (1) The signals and data measurements are the similar concepts for almost cases in the

physical measurements and experiments. The classification of signals is important

to the digital signal processing (DSP) or data processing, especially, this closely

relates to the digital filters and discrete signal analysis.

2.2. Classifications (1) Signals can be commonly classified in the engineering application by some follows

categories: i) Continuous (analogue) and discrete signals (digital), ii) Deterministic

and random signals

Analogue and discrete signals

Branches of deterministic signals

Electric signals (by data acquisition)

Analogue Signals (Continuous)

Discrete Signals (Digital)

Sampling and A/D conversion

Data Analysis and Post-data processing

Deterministic signals Periodic signals

Nonperiodic signals

Sinusoidal signals (2 cycle)

Complex periodic signals (T cycle)

Almost periodic signals

Transient signals

14 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing

Branches of random signals

Notes and comments:

i) Sinusoidal (harmonic) signals (2-periodic signals): )2()( ntxtx

ii) Complex periodic signals (T-periodic signals): )()( nTtxtx

The complex periodic signals may be expanded by a Fourier series into the

combination of harmonic components (sine and cosine functions) as follow:

)2cos()( 00 t

TXtx (Original signals)

1

0 ]12sin12cos[2

)(n

nn tT

nbtT

naatx (Fourier series)

iii) Almost-periodic signals: can be expressed by the sum of sine functions that

their frequencies are not periodic.

iv) Transient signals: can be defined as totally non-periodic signals (In other

word, the transient signals can be considered as the deterministic signals but

out of any kinds of periodic and almost-periodic signals). Apart from

periodic and almost-periodic signals, however, the spectrum of transient

data only exists under form of continuous spectrum but the discrete spectrum

does not exist.

Random signals Stationary signals

Nonstationary signals

Ergodic signals

Non-ergodic signals

Specific classification of nonstationary signals

15 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing

(2) The characteristics of the random signals can be clarified by some following

quantities:

i) Amplitude distribution: Mean value (expectation) or root mean square

(standard deviation) [first moment]

ii) Correlation functions (auto- and cross-correlations) [joint or second moment]

iii) Power spectral density (PSD) [power contribution of each frequency

components]

(3) For discrete signals, the above-mentioned quantities can be expressed by formulas:

N

iix x

N 1

1 ;

N

iix x

Nrms

1

21

)()(1)( 001

txtxN

R i

N

iixx

Nnkj

N

kix ex

NfS /2

1

1)(

; n=[1,N]

(4) Some hints on types of random processes

i) Stationary signals are that their mean value and correlation of discrete

signals do not vary on time.

ii) Non-stationary signals, by contrast, their mean value and correlation vary on

time.

)(),()(

xxxx

xx

RtRt

iii) Weakly stationary signals (or stationary in the wide sense) are that mean

value and correlation (first and joint moments) are time invariant.

16 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing

)(),()(

xxxx

xx

RtRt

iv) Strongly stationary signals (or stationary in the strict sense) are that all

moments and joint moments (not only first moment but also high-order

moments) are time invariant.

)(),()(

xxxx

xx

RtRt

(first moment and joint moment)

)(),(

)(

xxxxi

xxi

RtR

tth

th

(ith moment and ith joint moment)

v) Ergodic signal is stationary one (Mean value and correlation are time

invariant), moreover, its mean value and correlation are constant with

different samples of signal.

)()()(

)(

xxk

xx

xk

x

RR

k: Index of kth sample of signal

vi) Non-ergodic signal is stationary one (Mean value and correlation are time

invariant), however, its mean value and correlation are differed with

different samples of signal.

)()()(

)(

xxk

xx

xk

x

RR

vii) Gaussian signal (Normal distributed signal) is ergodic stationary one with

zero-mean and standard deviation

17 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing

2)()(

)(

)()(

0

k

xxk

xx

kx

CR

viii) Non-gaussian signal is ergodic stationary one with non zero-mean and

standard deviation

2)(

0)(

xx

xx

Ct

Note 1:

Single random process: Signal of one physical quantity (phenomenon) at one

position

Multi-random processes: Signals of many physical quantities at different

positions

Sampling: Signal of random process at any time ( time interval)

Multi-dimensional process: are multi-variable function

Multi-variate process: Vector of many signals

Multi-variate and multi-dimensional process: Vector of many processes (signals)

in which each individual signal is multi-variable function

Ensemble: the collection of sample functions (any time interval) of one signal.

Thus random signal is the collection of these sample function of one signal

Note 2:

Process (Field): Display and illustration of one physical quantity at one position

(If measurement of the same physical quantity at one position differs from that

at another position due to its distribution and redistribution)

Sample: Display and illustration of a physical quantity at one position at any

time interval

18 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing

Record: Discrete display of a physical quantity at one position and one time

Signal: Electric display (continuous or discrete) of a physical quantity at one

position and one time

Difference between ergodic and non-ergodic processes

Ergodic stationary random process Non-ergodic stationary random process

- Different initial phase angles

- The same amplitude

- The same frequency

- Different initial phase angles

- Different amplitude

- The same frequency

Summary on classifications and definition of random signals

No. Items Definition Note

1 Stationary Mean value and correlation not

vary on time

Time-invariant

2 Stationary-Ergodic Mean value and correlation not

vary on time and sampling

Time-invariant

Sampling-independant

3 Stationary-

Nonergodic

Mean value and correlation not

vary on time, but sampling

Time-invariant

Sampling-dependant

4 Weak stationary Mean value and correlation

(first-order moments) not vary on

time

Time-invariant of only

first moments

5 Strong stationary All first-order and high-order

moments not vary on time

Time-invariant of first

and high-order

moments

6 Non-stationary Mean value and correlation vary

on time

Time-variant

19 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing

(5) The ensemble (collection of sampling records) of random signal can be divided by

two categories: Individual sample records (one process) and Multiple sample

records (many processes). The characteristics of individual and multiple sample

records of one and many processes can be expressed by figure hereinafter:

Characteristics of individual and multiple sample records of random signals

Note 3:

Coherence function: is the relation between the power auto-spectral density and

the power cross-spectral density.

Frequency response function (gain factors and phase factors): is also the linear

relation between the power auto-spectral density and the power cross-spectral

density.

Individual Sample Records Analysis

Multiple Sample Records

Mean Values and Root Mean Square(RMS)

Auto-Correlation Function Analysis

Auto-spectral Density Analysis

Cross-Correlation Function Analysis

Cross-spectral Density Analysis

Coherence Function Analysis

Joint Probability Density Functions

Probability Density Functions

One sample of one process Pairs of two samples of one process or many processes

Frequency Response Function (FRF)

20 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing

(6) Applications of statistical functions

From correlation functions

i) Similarity between 2 signals or 1 signal at different positions and time delays

ii) Prediction of signals in noise, influence of noise

iii) Identification of propagation directions and velocities

iv) Measurements of time delays

From power spectral density (PSD) function

i) Power contribution of each frequency components

ii) Identifications of system properties and input signals from output ones

iii) Identification of noise and energy sources

From coherence function

i) Accuracy of linear input/output systems

ii) Identification of propagation directions and velocities

From frequency response function (FRF)

i) Identification of relationship between input and output signals

2.3. Relationships of input and output signals (1) The input and output signal systems can be commonly classified by: i) Single input

and single output systems (SISO); ii) Single input and multi output systems

(SIMO); iii) Multi input and single output systems (MISO); iv.Multi input and

multi output systems (MIMO).

(2) In the practical applications, many signal channels are simultaneously measured at

various positions or different time delays. For many cases of MIMO systems, many

input and output signals can be correlated or uncorrelated measured simultaneously.

21 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing

SISO system

Independent MIMO system

Hxy(f)

Frequency Response Function (FRF)

Input signal x(t)

Output signal y(t)

Signal noise n(t)

Hxy,1(f) Input signal

x1(t) Output signal

y1(t)

Hxy,2 (f) Input signal

x2(t) Output signal

y2(t)

Hxy,k (f)

Input signal xk(t)

Output signal yk(t)

……………………..

Channel No.1

Channel No.2

Channel No.k

Signal noise n(t)

22 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing

3. DATA ACQUISITION, PROCESSING AND ANALYSIS

3.1. Introduction (1) In order to understand and clarify the physical measurements, signal processing and

analysis for the buffeting experiments and other specified measurements as well in

the wind tunnel, it is important to understand the digital signal processing (DSP)

and measurement procedures and instruments in wind tunnels.

(2) This study hinges on some following points:

1) Instrumental systems their by function: data acquisition, A/D conversion, data

qualification and analysis

2) A/D conversion and sampling theorem for eliminating the aliasing errors

3) Data analysis techniques

3.2. Data processing procedure (1) The digital signal processing procedure can be expressed by the following diagram:

Signal processing procedures for measurement systems

(3) Three following main steps of the digital signal processing procedure will be

overviewed as follows:

Data Acquisition Data Conversion Data Qualification Data Analysis

Transducer

Signal Conditioning

Signal Calibration

A/D Conversion

Aliasing Errors

Quantization Errors

Classification

Validation

Editing

Individual Sample Records

Multiple Sample Records

23 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing

3.2.1. Data acquisition

Transducer: Device and sensor to transform from any physical phenomena

(force, pressure and motions: displacement, velocity and acceleration) to

electric signals. Transducers commonly are employed two kinds of materials:

piezoelectric and strain-sensitive materials.

+) Piezoelectric materials: frm physical quantities to electric charge, such

as naturally polarized crystals like quartz and artificially polarized

ferroelectric ceramics like barium titanate.

+) Strain-sensitive materials: from physical quantities to resistance, such

as metallic like copper-nickel alloy and semiconductor like

monocrystalline silicon.

Signal conditioning: Change from the electric signals (charge and resistance)

to voltage.

3.2.2. Data conversion

Analogue to digital converter (ADC): Transforms from continuous analogue

signal to digital signal

Aliasing errors: Eliminated by sampling theorem: sampling frequency (F) or

sampling time interval (t)

Note: ADC can be stored under two types of codes: binary and ASCII codes

Binary code: By numbers 0 and 1 (8 bytes)

ASCII code: By numbers from 0 to 9 (1byte). However, almost data of

discrete signals (after ADC) has been stored under this ASCII code in

application, because it is easily red by any applied programs such as Matlab

and Fortran for data post-processing.

24 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing

3.2.3. Data qualification

Many imperfections of received signals with noise can be reduced by the

digital or analogue filters.

Data qualification consists of: classification, validation and editing.

3.2.4. Data analysis

Individual sample records (Uni-variate process): Sample collection of

measurement data of one physical phenomenon at one point in various time

intervals.

Multiple sample records (Multi-variate processes): Sample collection of many

individual records of one physical phenomenon at numerous points or of some

physical phenomena at one point in various time intervals.

Fig. 2. Individual and multiple sample record analysis

Individual sample record and uni-variate process

Mean and root mean square (RMS) value computation:

Auto-correlation function computation

Auto-spectral density function computation

Individual Sample Records Analysis

Multiple Sample Records

Mean Values and Root Mean Square(RMS)

Auto-Correlation Function Analysis

Auto-spectral Density Analysis

Cross-Correlation Function Analysis

Cross-spectral Density Analysis

Coherence Function Analysis

25 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing

Multiple sample record and multi-variate processes

Cross-correlation function computation

Cross-spectral density function computation

Coherence function computation

3.3. Data analysis procedures 3.3.1. Data sampling and data preparation

Multi input and multi output system (MIMO) = Combination of independent single

input and single output systems (SISO)

n: Index of ensemble (sample record) or signal or random process

Multi sample record (of multi signals) = Collection of number of individual sample

records (of individual signal). Thus analysis of the multi sample record can be carried

out by analysis of the individual sample record and analysis of the pairs of correlated

two individual sample records.

Discrete valued signal (or random process):

nx n=1,2,3… N (N: Number of samples)

Equally spaced time interval (T: sampling time or sampling period) of samples in

discrete signal:

TnTTn *0 n=1,2,3… N

Continuous Signal A/D Conversion Data Sampling

Discrete Signal Discrete Data Record

Frequency Response Function

Signal Filter Input signal ui (t)

A/D Conversion Data Sampling

Output signal xn(t)

Continuous Data Discrete Data

26 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing

T0: Initial time (T0 = 0) TnTn *

T: Sampling period

NT: Total time length of discrete signal; TN *

F: Sampling frequency (Hz); TF 1

F0: Fundamental sampling frequency; TNF

*1

0

Thus, )*()( 0 TnTxTxx ni n=1,2,3… N

Noting that the Limit sampling frequency (the Nyquist frequency) for eliminating the

Aliasing Errors

TFN *2

1

3.3.2. Data standardization

The purpose of data standardization is to transform the original data record nx to

new type of data record nx' (can be called the fluctuating data record) that has the

zero mean value.

1) Mean value and root mean square of sample record

N

nnx

Nx

1

1 n=1,2,3…N (Mean value)

27 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing

N

nnx x

NSQRTs

1

2 ]1

1[ n=1,2,3…N (Root Mean Square

value)

Note: For the stationary ergodic data record (time-invariant and sampling-

independent), it is very convenient and common to transform the initial data

record nx to the new zero-mean data record nx' .

2) Establishment of the fluctuating data record (Zero-mean data record)

nx' : n=1,2,3…N

xnTxxxx nnn )(' n=1,2,3…N

Having: 0' x (Zero-mean value)

N

nnx x

N 1

2'

2 '1

1 (Mean Square value or Variance)

N

nnxx x

NSQRTs

1

2'' ]'

11[

(Root Mean Square or Standard Deviation)

3) Fourier Transform (Discrete Fourier Transform-DFT):

Fluctuating data record (Zero-mean data record) or standardized data record

nx' n=1,2,3…N

Discrete Fourier Transform

N

nkk TnFjTnxFX

1)]2exp(*)([1)(

28 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing

X(Fk): DFT at computational frequency kF

k: Index of discrete frequencies

Fk: Computational frequency; )( fkFk

Note:

i) The frequency space )( f = The fundamental sampling frequency (F0)

NTFf 1

0

ii) Number of frequency space (K)

0FFFK sc

cF : Cut-off frequency

sF : Starting frequency

iii) Computational frequency Fk:

NTkFfkFF ssk )( ; k=0,2,3…K-1

NTkFk ; k=0,2,3…K-1

Note: Index of computational frequency starts from 0

Frequency value starts from Fs (???)

iv) DFT of x(t) at the computational frequency

29 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing

N

nk Tn

NTkjTnx

NTkXFX

1)]2exp(*)([1)()(

;k=0,2,3…K-1

N

nk NnkjTnx

NTkXFX

1)]/2exp(*)([1)()(

;k=0,2,3…K-1

k=1:

N

ns NnjTnxFXFX

11 )]/2exp(*)([1)()(

k=2:

N

ns NnjTnx

NTFXFX

12 )]/22exp(*)([1)1()(

k=3:

N

ns NnjTnx

NTFXFX

13 )]/32exp(*)([1)2()(

Note: +) In the DFT formula, factor (1/) appears (DFT standardization)

+) In Matlab command, X=FFT(x) (without standardization)

Inverse Discrete Fourier Transform:

N

kNnkj

NTkX

NnTx

1)]/2exp(*)([1)( ; n=0,1,2…N

DFT X(Fk)

f(Hz) F1=Fs k=1

X(F1) X(Fi)

Fi=Fs+i/NT k=i

F1=Fc k=K

30 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing

4) Auto-correlation function

Auto-correlation function can be computed by i) Direct computation by definition and

ii) Indirect computation via FFT. Note that the second method is more efficiency

because of application of FFT algorithm, however, the first one is easier to compute

but more time consuming.

Method 1: Direct computation of auto-correlation function

Fluctuating data record (Zero-mean data record) or standardized data record

nx' n=1,2,3…N

Auto-correlation function of data record nx' with delay s

N

nsnnxx ssxsx

NsR

1)()( )]()([1)( (Auto-correlation functions)

trs *

rN

nrnnxx ssxsx

rNtrR

1)]()([1)*(

5) Auto-spectral density function

Auto-spectral density function can be computed by i) Ensemble Averaging and ii)

Frequency Averaging

Method 1: Ensemble-averaging technique

Fluctuating data record (Zero-mean data record) or standardized data record

nx' n=1,2,3…N

31 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing

Definition of mean power:

NTN

nnxx dttx

NTx

NtxES

0

21

0

22 )(11)]([

1

0

2|)(|1)(N

nnxx fX

NTfS : Two-sided spectral density

)( fX : Fourier transform at frequency f

1

0]/2exp[*1)(

N

nn Nnkjx

NfX ; k=0,1,2…N-1

1

0

2|)(|2)(2)(N

nknkxxkxx fX

NTfSfG : One-sided spectral density

Computational procedures (Ensemble-averaging technique)

1) Data Sampling: N samples of process x(t) are picked out (Data record)

nx , n=1,2,3…N

2) Data Standardization: Compute the mean value of data record ( x ), then

reconstruct the Fluctuating Data Record nx' with zero-mean value

( 0' x )

t

x(t)

T 2T 3T 4T 5T 6T NT (N-1)T

32 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing

nx' , n=1,2,3…N

3) Data Blocking: Taper data record by each data blocks using Window

functions

4) Fourier Transform: Compute the Fourier Transform at frequency fk,

k=1,2,3…N by using FFT technique

)( kfX , k=1,2,3…N

5) Scale Factor Adjustment of )( kfX : Scale factor of )( kfX due to the loss

by tapering operation. (By Hanning tapering: Scale factor by 3/8 )

6) Spectral Density: Estimate the spectral density )( kxx fS from each data

blocks

33 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing

4. CORRELATION FUNCTION

4.1. Introduction (1) The correction functions evaluate the statistical independence between the signals

(time-dependant) or stochastic processes (space-dependant). The cross correlation

function is a measure to tell us how much two processes or two signals are like each

other, whereas the auto correlation function tell us how much a process or a signal at

time t is like itself at time t+ (: time shift or time lag) or how much a process or a

signal at location (x,y,z) is like itself at another location (x+x,y+y,z+z). In addition,

to evaluate how much two processes or two signals are like each other, the correlation

coefficient function also is given.

4.2. The discrete correlation function The discrete correlation function of two processes or two signals x, y can be expressed

as follow:

1

0)()()()( )]()([1][)(

N

nsnnNsnnxy ssysx

NLimyxEsR

E[] : Expected value or mean value

For the number of samples is taken large enough, we have following approximations:

N

nsnnxx ssxsx

NsR

1)()( )]()([1)( (Auto-correlation functions)

N

nsnnxy ssysx

NsR

1)()( )]()([1)( (Cross-correlation

functions)

s : Spatial interval or time shift

34 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing

s: Spatial coordinates or time variable

N: Number of samples

Note: 1) Corresponding values of signals are sampled at same time.

2) Rxx(s)=Rxx(-s) and Rxy(s)=Rxy(-s)

3) Above approximate formulas are accuracy only if N

Computational procedure of correlation function:

Step 1: Setting parameters

+) Number of sample: N

+) Time shift or spatial interval: s

Step 2: Sampling

+) Sampling signals x, y

Step 3: Computing correlation function

Step 4: Plotting Rxx, Rxy versus n [1,N]

4.3. The discrete covariance function The discrete covariance function of two processes or two signals x, y can be expressed

as follow:

Span-wise

s

s ss

s

xyxx RR ,

is

)( ixx sR Rxx, Rxy vs. s

35 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing

1

0

1

0)()()()( ])([1)]([1][][)(

N

n

N

nsnNnNsnnxy ssy

NLimsx

NLimyExEs

For the number of samples is taken large enough, we have following approximations:

N

n

N

nsnnxx ssx

Nsx

Ns

1 1)()( ])([1)]([1)( (Auto-covariance functions)

N

n

N

nsnnxy ssy

Nsx

Ns

1 1)()( )]([1)]([1)( (Cross-covariance

functions)

s : Spatial interval or time shift

s: Spatial coordinates or time variable

N: Number of samples

If x, y are the zero-mean processes or signals, these mean that E[x(s)]=0 and E[y(s)]=0,

then the Root mean square (R.M.S) value must be replaced to the mean value (or

Expected value):

Auto-covariance function:

N

n

N

nsnnxx ssx

NSQRTsx

NSQRTs

1 1)(

2)(

2 ]})([1{)]}([1{)(

Cross-covariance function:

N

n

N

nsnnxy ssy

NSQRTsx

NSQRTs

1 1)(

2)(

2 ]})([1{)]}([1{)(

Computational procedure of covariance function:

Step 1: Setting parameters

+) Number of sample: N

36 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing

+) Time shift or spatial interval: s

Step 2: Sampling

+) Sampling signals x, y

Step 3: Computing covariance function

Step 4: Plotting xx, xy versus n [1,N]

4.4. The discrete correlation coefficient function The discrete correlation coefficient function of two processes or two signals x, y can be

expressed as follow:

)()(

)(ssR

sxx

xxxx

(Auto-correlation coefficient function)

)()(

)(ssR

sxy

xyxy

(Cross-correlation coefficient function)

Note: ]1,1[)( s : 1 Full-correlated; 0 No correlated

4.5. Examples Example 1: Correlation function

0 50 100 150-1

-0.5

0

0.5

1

Number of sample

Initial signals x,y

0 50 100 150-0.2

-0.1

0

0.1

0.2

Number of sample

Cross-correlation Rxy by definition

0 100 200 300-0.2

0

0.2

0.4

0.6

Number of sample

Auto-correlation Rxx,Ryy by Xcorr(x)

0 100 200 300-0.2

-0.1

0

0.1

0.2

Number of sample

Cross-correlation Rxy by Xcorr(x,y)

37 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing

Example 2: Correlation of signal with noise

0 1 2 3

x 10-3

-3

-2.5

-2

-1.5

-1

-0.5

0

0.5

1

1.5

2

Time (s)

Initial sinal+noise

0 0.2 0.4 0.6 0.8-0.2

0

0.2

0.4

0.6

0.8

1

1.2

1.4

Time (s)

Correlation functions of signal with noise

38 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing

5. FOURIER TRANSFORM

5.1. Introduction (1) It is well known that the Fourier Series Transform and Fourier Spectral Transform

have been widely applied for almost kinds of natural and physical phenomena.

Applications and contributions of the Fourier Series and Fourier Transform concentrate

on the problem of the Digital Signal Processing (D.S.P) in the data processing and

analysis of measurements and the buffeting response prediction in which the spectral

representation cant be required.

(2) In summary, the Fourier Series and the Fourier Transform will be studied for such

purposes as follows:

1) Data processing and analysis of measurement processes in the Digital Signal

Processing (D.S.P)

2) Spectral representation in the buffeting response prediction

(3) Data measurements can be expressed under the continuous or discrete processes.

However, almost measured signals have been collected under the discrete signal for the

data post-processing and analysis. The discretization of measured signals is well

known as the sampling processes.

(4) Some following DSP techniques will be studied hereinafter:

1) The discrete Fourier series (DFS)

2) The discrete Fourier transform (DFT)

3) The fast Fourier transform (FFT)

4) Amplitude spectrum and phase spectrum

5) The discrete inverse Fourier transform (IFT)

6) The sampling technique

39 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing

5.2. The Discrete Fourier Series The Fourier series purposes to decompose any periodic or non-periodic signals into

combination of simple harmonic signals (sine and cosine functions). Post processing

and analysis on simple harmonic functions seem to be much easy than the original

signals. Mathematically, the Fourier series is known as the harmonic analysis.

N: Number of samples

T: Period (Time step of sampling) of a sample (s)

F: Frequency of a sample (Hz), T

F 1

NT: Fundamental period of series (s)

Fo: Fundamental frequency of series (Hz) NT

Fo 1

Discrete Fourier series is expressed as the sum of harmonic functions as follow:

)]sin()cos([2 00

1

1

0 TmnbTnmaax m

M

mmn

0 : Fundamental frequency of series (rad/s), NTFo

220

m: Times of fundamental frequency (m=1: fundamental harmonic term with 0 ,

m=2: 2nd harmonic term with 2 0 , mth harmonic term with m 0 )

M: Cutting-off number of frequency

ao, am, bm: Fourier coefficients of series

n: Pointer of samples

)]/2sin()/2cos([2

1

1

0 NmnbNnmaax m

M

mmn

40 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing

Discrete Fourier Coefficients:

)/2cos(2 1

0Nnmx

Na

N

nnm

; 10 Mm

)/2sin(2 1

0Nnmx

Nb

N

nnm

; 11 Mm

Computational procedure:

Step 1: Setting parameters

- Initial signal

- Number of samples N

- Number of series M

Step 2: Sampling

Step 3: Calculating Fourier Coefficients

- am, bm

Step 4: Simulating Fourier series and Plotting Simulated signal vs. number of

samples (n)

5.3. The Discrete Fourier Transform x(t): time-dependant signal or stochastic process

T: Sampling cycle (time interval for a sample)

F: Sampling frequency (F=1/T: number samples per a second)

N: Total samples

NT: Total time for sampling or fundamental period of transform

Fo: Fundamental frequency of transform NT

F 10

41 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing

Discrete Fourier Transform: (to frequency domain)

N

n

Tnjk

kenTxX1

])([)(

N

n

Tnfjk

kenTxfX1

2 ])([21)(

(Note: In origin, n=0N-1)

k: Number of discrete frequencies in range

In Matlab: 0)( kkk [k=1-K]: equal spacing of frequency range

0 : Fundamental frequency (rad/s), NTFo

220

Note: n (pointer of samples) and k (pointer of frequency) have different

meaning

Computational procedure:

Step 1: Setting parameters

- T (cycle sampling), N (numbers of samples)

- Freq. range: fs (starting freq.), fc (cut-off freq.)

- Number of freq. interval: K kfsfc /)(

Step 2:

Loop 1: Frequency

For k=1 to K )( kk

Loop 2: Sampling

For n=1 to N

N

n

nTjkk enTxX

1

)()()(

Step 3: Plotting X(k) versus (k)

42 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing

Putting NTFo

220 into formulas:

N

n

nTjkenTxkX1

0 ])([)( 0

N

n

nTNT

jkenTxFkX

1

2

0 ])([)2(

N

n

NknjenTxkFX1

/20 ])([)(

Fourier Series (in time domain)

K

kkkkk tkbtkaath

1

0 )]sin()cos([2

)(

)}(Re{2 kHN

ak

)}(Im{2 kHN

bk

Note in processing using FFT:

1) )/(2 NT or )/(1 NTF

2) Maximum frequency: )2/(12/max TFF

3) Spectrum calculated at certain frequencies: 0, F, 2F, 3F… Fmax

Leakage Effect: Amplitudes will distribute on adjacent closed frequencies. This effects

occurs in cases that total sampling time NT does not coincide the integer multiple of

the sampling cycle T. To prevent the ‘leakage effect’ by Averaging Method or Window

Functions

43 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing

5.4. The Fast Fourier Transform It is very well-known that the Fast Fourier Transform (FFT) algorithm has been

powerfully used for solving the Discrete Fourier Transform. FFT is not a new

transform itself due to using the same DFT formula, however, its FFT algorithm is

much faster than the conventional DFT. In principle, the FFT algorithm eliminates the

component repetition to make the faster computation.

DFT formula:

1

0

/20 ])([)(

N

n

NknjenTxkFX

1

0

N

n

knNnk WxX ; k=[0,N-1]; Nj

N eW /2

Xk contains N2 components (N of xn and N of WN)

Note: sincos je j = x + jy (Euler’s Formula)

)/2sin()/2cos(/2 NknjNkne Nknj

01

12

21

0

2

Nj

NNj

N

n

Nnj

e

ee

Sum can be expressed in the complex plane of 2=3600 (N=8 for example)

Complex plane (N=8) Complex plane

00 k=0

450 k=1

900 k=2

1350 k=3

1800 k=4

2250 k=5

2700 k=6

3150 k=7

je

0 8/02je

8/22je

8/52je 8/72je

Re

Im

44 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing

Based on above-mentioned notes, we can easily obtain:

),( Nmrem

Nm

N WW ; m[0,N-1]; rem(m,N) is the remainder after dividing m

by N mNjm

N eW )/2( ; m[0,N-1]

))(/2(),( miNNjNmremN eW

; i=1,2,..N

knMax )(

For example, N=8, n=[0:7], k=[0:7], max(kn)=49, components W are expressed by

the complex plane

Complex plane (N=8)

Thus, among 49 components, 8 of these are unique.

Then, suppose N is a multiple of 2, we decompose the samples into two vectors

containing even- and odd-numbered samples as follows:

12/

0

12/

0

212

22

N

n

N

n

knNn

knNnk WxWxX ; k=[0:N-1]

Due to: k

Nk

N WW 22/2/ , thus

12/

0

12/

02/122/2

N

n

N

n

knNn

knNnk WxWxX ;

168

88

08 WWW

178

98

18 WWW 19

811

83

8 WWW

208

128

48 WWW

218

138

58 WWW

228

148

68 WWW

238

158

78 WWW

188

108

28 WWW

45 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing

Number of components reduces to 2)2

( N < 2N of original problem. Depending on N,

efficiency of FFT algorithm is different. For example, if N=2K then complex

components in FFT is NNKN2log

22 (in comparison of N2 components in DFT)

5.5. The Discrete Inverse Fourier Transform The discrete inverse Fourier transform of X(f) can be expressed as follows

N

n

NknjenTxkFX01

/20 ])([)(

; k=[0:N-1]:

1

0

/20 ])([1)(

N

k

NknjekFXN

nTx ; n=[0:N-1]: Inverse Transform

5.6. Amplitude spectrum and phase spectrum The amplitude spectrum is defined as the vector of DFT component amplitudes,

whereas the phase spectrum is vector of DFT component phase angles in radian unit.

5.7. The Discrete Sampling Theory The problem in the sampling technique is to require 2 necessary points:

1) Sampling values are required enough for the data processing and analysis.

2) Sampling the pick-up values from either continuous signals or discrete signals

represents the original one. This means that from the sampling values can

reconstruct the similar original signals.

46 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing

The sampling parameter: Sampling rate or sampling frequency Fs(Hz).

The sampling theorem may be stated as follows: ‘If continuous signals is sampled at a

sampling rate or sampling frequency that is twice higher than their highest frequency

component, then it is possible to recover and reconstruct the original signals from

samples ’

signalsampling FF max,2

5.8. Examples Example 1: D.F.S

Initial signal (impulse function) and signal due to Discrete Fourier Series at various

number of series (M=1,3,5,10,10,50)

0 10 20 30 40 50 60 700

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

number of samples

Num

ber o

f ser

ies

M=1

0 10 20 30 40 50 60 70-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

number of samples

Num

ber o

f ser

ies

M=3

0 10 20 30 40 50 60 70-0.2

0

0.2

0.4

0.6

0.8

1

1.2

number of samples

Num

ber o

f ser

ies

M=5

0 10 20 30 40 50 60 70-0.2

0

0.2

0.4

0.6

0.8

1

1.2

number of samples

Num

ber o

f ser

ies

M=1

0

0 10 20 30 40 50 60 70-0.2

0

0.2

0.4

0.6

0.8

1

1.2

number of samples

Num

ber o

f ser

ies

M=2

0

0 10 20 30 40 50 60 70-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

1.2

number of samples

Num

ber o

f ser

ies

M=5

0

Initial Signal Impulse Function

Signal from D.F.S

47 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing

Example 2: FFT vs. SPECTRUM

Fourier Transform by FFT and Spectrum

Note:

Fourier Transform Spectrum

dtetxjXxCFT tj )()()(

1

0)()(

N

n

nTjnexjXxDFT

dtetxjXxCSPEC tj )()()( 2

1

0

2)()(N

n

nTjnexjXxDSPEC

Amplitude

Noise

f1=8Hz

f1=33Hz

No noise

0 0.2 0.4 0.6 0.8-10

0

10

20

time [t]

signal

0 20 40 600

2

4

6

8

frequency [Hz]

spectra by using FFT

0 0.2 0.4 0.6 0.8-10

0

10

20

time [t]

signal

0 20 40 600

5

10

15

20

25

frequency [Hz]

spectrum by using spectrum

48 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing

Example 3: DFT

Results: +) Fast Fourier Transform (FFT) is not a new form of Fourier transform, the

applied formula is exactly such an expression

N

n

NknjenTxkFX1

/20 ])([)( , however,

the modified algorithm is used for much faster computation of DFT

+) Result by DFT’s defined formula is exactly same to that by FFT’s Matlab

command. Moreover, Magnitudes of DFT and FFT are absolute values (abs(H))

+) Results by DFT and FFT are different from that by Spectrum

Notes: +) DFT definition

N

n

NknjenTxkFX1

/20 ])([)( :

H0=x*exp(-j*2*pi*k'*n/N);

n: pointer of sample, k: pointer of frequency, k must be transposed (k’)

H0=H0/N; H0 must be standardized by dividing by N (number of samples)

0 0.2 0.4 0.6 0.80

0.2

0.4

0.6

0.8

1

Number of samples or time interval

Sig

nal

0 20 40 60-0.4

-0.2

0

0.2

0.4

0.6

Frequency Hz

FFT command

0 20 40 60-0.4

-0.2

0

0.2

0.4

0.6

Frequency Hz

DFT

Mag

nitu

de

DFT by definition

0 20 40 600

2

4

6

8

Frequency Hz

Spectrum command

49 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing

Example 4: Amplitude and phase spectrum

Example 5: Amplitude and phase spectrum

0 0.2 0.4 0.6 0.80

0.2

0.4

0.6

0.8

1

Number of samples

Sig

nal

0 20 40 60-0.4

-0.2

0

0.2

0.4

0.6

Frequency(Hz)

Am

plitu

de

Spectrum

0 20 40 60-4

-2

0

2

4

Frequency(Hz)

Pha

se(ra

d)

Phase spectrum

0 20 40 600

5

10

15

20

Frequency(Hz)

Pha

se(ra

d)

Phase spectrum

fftshift(H)

unwrap() Angle(H)

0 50 100-0.5

0

0.5

1

1.5signal

0 50 1000

1

2

3

4FFT

0 50 100-4

-2

0

2

4phase spectrum

0 50 1000

50

100

150unwrapped p.s

0 50 100-0.5

0

0.5

1

1.5signal

0 50 1000

1

2

3

4swapped FFT

0 50 100-4

-2

0

2

4p.s of swapped FFT

0 50 1000

50

100

150unwrapped p.s

50 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing

6. POWER SPECTRAL DENSITY FUNCTION

6.1. Introduction (1) The random signals can not be determined exactly. It means that the random

signals always differ from each other at the different observations, moreover, the

random signals contain the random parameters that can not be described by the means

of analytical quantities and determinant-parameter methods, but they are only able to

be described by the terms of statistical parameters that can differ from one random

signal to another.

(2) Thus the questions are that what terms of statistical parameters can be able to

describe the characteristics of the random signals. It is well known that the two most-

commonly-used means are:

i) amplitude distribution functions of random signal

ii) power spectral density of random signal (or equivalent as the correlation

function)

(3) In the term of amplitude distribution, the commonly-used statistical quantities are:

i) mean value (expectation); ii) mean square (variance when zero-mean signals) or root

mean square (standard deviation when zero-mean signals).

(4) It is broadly said that the square of DFT magnitudes of any function x(t) is

considered as power contribution of any frequency components in x(t) over the

frequency domain.

6.2. Characteristics of amplitude distribution The most-commonly-used statistical terms of the amplitude distribution characteristics

are defined underneath:

51 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing

- Mean value (Expectation):

1

0

1][N

iix x

NxE

- Mean square :

1

0

222 1][N

iix x

NxE

- Root mean square : )1(][1

0

22

N

iix x

NsqrtxE

- Variance:

21

0

1

0

22222 ]1[]1[][])[(

N

ii

N

iixxx x

Nx

NxExE

- Standard deviation: )][(]))[(( 222xxx xEsqrtxEsqrt

Thus zero-mean random signals ( 0x ) have the following deductions:

Mean square = Variance

Root mean square = Standard deviation

Linear relationship of y and x as y=ax+b, we easily obtain: 222xx

xy

a

ba

Some distribution probability functions of measurements are widely used as

i) uniform probability distribution:

ii) normal (Gaussian) probability distribution:

6.3. The Power Spectral Density (PSD) As above-mentioned, the power of random signals is expressed by the square of DFT

magnitudes of x(t) at any time t, whereas the power spectral density is expressed in the

frequency domain . Some concepts of power have been used:

52 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing

The instantaneous power of signal x(t) at any time t can be defined as: 2|)(| txPower

For the complex signals, we must use the mean power or expected power:

2/

2/

22/

2/

22 )(1)(1)]([T

T

T

TT dttx

Tdttx

TLimtxEpowerMean

(when T is long enough)

NTN

nnxx dttx

NTx

NtxESpowermeanDiscrete

0

21

0

22 )(11)]([

(Definition:This is only approximate estimation, that is why it is called as the

spectral estimation)

We have the discrete inverse Fourier transform:

1

0

/20 ])([1)(

N

k

NknjekFXN

nTx

Thus, the discrete mean power can be written as follows:

1

0

21

0

/20

1

0

2 ])(1[11 N

n

N

k

NknjN

nnxx ekFX

NNx

NS

1

0

1

0

1

0

/)(23

1

0

21

0

/203

1])([1 N

k

N

m

N

n

Nnmkjmk

N

n

N

k

Nknjxx eXX

NekFX

NS

1

0

22

1 N

kkxx X

NS

53 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing

We have:

1

0

21 N

kkxx x

NS

As a result, the relationship between the mean power of signal in the time domain and

power spectrum, or the mean power of signal can be presented in the term of the power

spectrum. This expression has been well known as the Parseval’s Theorem.

1

0

21

0

2 1 N

kk

N

kk X

Nx (Parseval’s Theorem)

In DSP, it is defined the periodogram that has N components given by:

21)( kxx XN

kP ; k=[0,N-1]

)()( kNPkP xxxx

The power spectrum can be obtained:

1

0

1

0

2 )(11 N

kxx

N

kkxx kP

Nx

NS

Note: In the same way that 2kx represents a measure of signal power at a point in

the time domain, Pxx(k) represents the measure of signal power at the point in

the frequency domain.

Therefore, the vector Pxx=[Pxx(k)], k=[0:N-1] is considered as the measure of

estimation of the power spectral density (PSD) of signal x(t).

Some expressions in the Power Spectral Density (PSD) S of signal x(t)

54 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing

Discrete PSD: 1)

1

0

22

1 N

kkxx X

NS (Frequency domain)

Xk is DFT amplitude of x(t) at frequency k

2)

1

0

21 N

kkxx x

NS (Time domain)

xk is discrete value of signal x(t) at time t

3)

1

0

1

0

2 )(11 N

kxx

N

kkxx kP

Nx

NS (Periodogram)

21)( kxx X

NkP ; m=[0,N-1]

Parseval’s Theorem:

1

0

21

0

2 1 N

kk

N

kk X

Nx (Parseval’s Equality)

Continuous PSD: 1) NT

xx dttxNT

S0

2 )(1 (Frequency domain)

2)

T

Txxxx dffPS

2/1

2/1

)( or

T

Txxxx dPS

/

/

)(

The relationship of the power spectrum to the auto-correlation: The discrete auto-

correlation of vector x from sampled values of signal x(t) with extended period has

been defined as follows:

1

0

1)(N

nsnnxx xx

NsR ; s=[0:N-1]

By definition of the mean power, we have:

55 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing

1

0

21)0(N

nxxnxx powerMeanSx

NR

Discrete Fourier transform of the auto-correlation can be calculated by definition:

1

0

1

0

/21)}({N

s

N

n

NskjsnnxxR exx

NsRDFTS

; k=[0:N-1]

1

0

/2N

n

Nknjnk exX

; k=[0:N-1] (Definition of DFT)

)(1)}({ 2 kPXN

sRDFTS xxkxxR ; k=[0:N-1]

In the conclusion, the periodogram of vector x with periodic extension is the DFT of

the auto-correlation function of x, that means:

)(1)}({ 2 kPXN

sRDFTS xxkxxR

Role of the auto-correlation function gives us information on how much

dependence of given sample xn on nearby samples xn+1, xn+2, and so on.

The Cross Spectrum: involves two time-dependant signals (two time series). Suppose x

and y are vectors of length N sampled from two signals x(t) and y(t).

- The cross-periodogram: kkxy YXN

kP '1)( ; k=[0:N-1]

56 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing

- Mean cross-power:

1

0

1 N

nnnxy yx

NS

1

0

1

02 )(1'1)0(

N

k

N

kxynnxyxy kP

NYX

NRS

Role of the cross spectrum gives us information on how the cross

correlation function is distributed over the frequency scale.

Coherence function: The magnitude of the cross periodogram Pxy(k), that is )(kPxy , is

the measure of the coherence of signals x(t) and y(t) at different frequencies.

- Coherence function: )()(

)()(

2

2

fPfPfP

fCohyyxx

xyxy

6.4. Examples Example 1: FFT and SPECTRUM

0 0.5 10

0.2

0.4

0.6

0.8

1Impulse signal

0 500

0.1

0.2

0.3

0.4

0.5DFT

0 500

0.1

0.2

0.3

0.4

0.5

Ampl

itude

FFT

0 0.5 10

0.2

0.4

0.6

0.8

1

Time (s)0 50

0

2

4

6

8x 10

-5

Frequency (Hz)

X2

0 500

2

4

6

8

Frequency (Hz)

Spectrum

Ampl

itudeThe same

Difference in shape? Different in value?

57 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing

Note: 1) Using FFT command

H=FFT(x);

H=[H(1) 2*H(2:N/2)]/N; %Vector length limit and

standardization

2) Using DFT’s definition

H=x*EXP(-j*2*pi*k'*n/N); %DFT by definition

H=[H(1) 2*H(2:N/2)]/N;

3) Using SPECTRUM command

S=SPECTRUM(x);

S=[S(1) 2*S(2:N/2)]; %Vector length limit

4) Using spectrum’s definition

S=ABS(H).^2; %Square of absolute amplitude

S=[S(1) 2*S(2:N/2)]/ (N^2); % Vector length limit and

standardize

Example 2:FFT and SPECTRUM of some types of functions

Time intervals Frequency intervals

Sampling process (Step 2)

FFT (Step 3)

SPECTRUM (Step 4)

Digital Signal or Random Process

58 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing

Example 3: Some built-in filter function (window functions) in FFT

0 5-10

0

10normal signal

0 50

5

10uniform signal

0 5-2

0

2sine signal

0 50

0.5

1impulse signal

0 500

0.2

0.4FFT

0 500

2

4

0 500

0.5

1

1.5

0 500

0.5

1

0 500

20

40spectrum

0 500

20

40

0 500

100

200

0 500

20

40

0 5-20

-10

0

10

20normal signal

0 500

2

4

6

8

10FFT

0 500

2

4

6

8

10FFT with boxcar

0 500

1

2

3

4

5FFT with hamming

0 500

1

2

3

4FFT with bartlett

0 500

1

2

3

4FFT with hann

59 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing

7. COMPUTATION OF COHERENCE FUNCTION

7.1. Introduction (1) Coherence function: Defined as the measure to evaluate the statistical

independence of two stochastic processes or two digital signals (but in the

frequency-domain)

(2) Coherence function: Computed by using the cross-spectrum (two signals) and

auto-spectrum (one signal) of two stochastic processes

(3) Correlation function: Similar to the coherence function, the correlation function

also is the measure to evaluate the statistical independence of two stochastic

processes or two digital signals (but in the time-domain)

7.2. Cross spectrum and coherence function (1) The computation of cross spectra has been used in the field of Digital Signal

Processing (D.S.P) and Discrete Data Processing (D.D.P)

(2) The Auto-spectrum represents the Fourier transform of the Auto correlation

Stochastic process A or digital signal A

Stochastic process B or digital signal B

Correlation function (time domain) Coherence function (frequency domain )

Inter-relation or inter-influence

60 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing

(3) The Cross-spectrum represents the Fourier transform of the Cross correlation

(4) The Cross-spectrum is the complex valued function whose magnitude and phase

are used in the signal processing to indicate the degree of correlation between two

signals

(5) The magnitude of the cross spectrum indicates whether frequency components of

one signal are associated with large or small amplitudes at the same frequency in

the second signal.

(6) The phase of the cross spectrum indicates whether the phase lag (tre) or lead (dan

truoc) of one signal with respect to the second signal for a given frequency

component.

(7) The cross-spectrum is used to determine the coherence function between two

signals

(8) Cross spectrum and coherence function can be determined by the MATLAB as

powerful engineering program for Digital Signal Processing

7.3. Computation of coherence function (1) Let x(t) and y(t) be two discrete data measurements of two signals, the problem is

that to determine i) Cross spectrum and auto spectrum and ii) Coherence function

(2) The Fourier transforms of discrete signals

X(f)=F{x(t)}

61 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing

Y(f)=F{y(t)}

(3) Auto spectrum of x(t) or y(t)

Sxx(f)=X(f) X*(f)

Syy(f)=Y(f) Y*(f)

(4) Cross spectrum of x(t) and y(t)

Sxy(f)=X(f) Y*(f)

Syx(f)=Y(f) X*(f)

(5) Coherence function

)()(

)()(

fSfSfS

fCohyyxx

xyxy (Complex Coherence Function)

)()(|)(|

|)(|2

2

fSfSfS

fCohyyxx

xyxy (Magnitude Squared Coherence Function)

Note: Here two discrete signals are taken into account. In cases, many pairs of two

discrete signals are counted, the summation must be used, called the average cross

spectrum and the average coherence function. Complex cross spectrum and

coherence function consists of two parts: i) magnitude and ii) phase

(6) Some simplified formulas

)()()(

)(21

2212

21

uu

uuuu SS

SCoh

)()()(

)(21

2121

uu

uuuu SS

SCoh

62 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing

)()(

)()(

,0,

,,

yuu

yuyu SS

SCoh

: The span-wise coherence

Where: )(, yuCo is co-spectrum (real part) Fourier-transformed from

correlation.

7.4. Examples Example 1: FFT vs. SPECTRUM

0 0.2 0.4 0.6 0.8-10

0

10

20

time [t]

signal

0 20 40 600

2

4

6

8

frequency [Hz]

spectra by using FFT

0 0.2 0.4 0.6 0.8-10

0

10

20

time [t]

signal

0 20 40 600

5

10

15

20

25

frequency [Hz]

spectrum by using spectrum

0 0.2 0.4 0.6 0.8-10

0

10

20

Time[s]

Input Signal

0 20 40 600

2

4

6

8

Frequency [Hz]

Spectrum with FFT

0 20 40 60-0.5

0

0.5

1

Time[s]

Input Signal


Top Related