part 3. part 3. spectrum spectrum …classweb.ece.umd.edu › enee630.f2012 › slides ›...

ENEE630 PartENEE630 Part--33

Part 3. Part 3. Spectrum Spectrum EstimationEstimation3.1 3.1 Classic Methods for Spectrum EstimationClassic Methods for Spectrum Estimation

Electrical & Computer EngineeringElectrical & Computer Engineering

University of Maryland, College Park

Acknowledgment: ENEE630 slides were based on class notes developed by Profs. K.J. Ray Liu and Min Wu. The slides were made by Prof. Min Wu,

ith d t f M W i H Ch C t t i @ d d

UMD ENEE630 Advanced Signal Processing (ver.1211)

with updates from Mr. Wei-Hong Chuang. Contact: [email protected]

LogisticsLogistics

Last Lecture: lattice predictorcorrelation properties of error processes©

200

3)

– correlation properties of error processes– joint process estimator in lattice– inverse lattice filter structureat

ed b

y M

.Wu

inverse lattice filter structure

Today: S t ti ti b k d d l i l th d63

0 S

lides

(cre

– Spectrum estimation: background and classical methods

CP

ENEE

624/

6

Homework setUM

C

UMD ENEE630 Advanced Signal Processing (ver.1211) Nonparametric spectral estimation [2]

Summary of Related Readings on PartSummary of Related Readings on Part--IIII

2.1 Stochastic Processes and modelingHaykin (4th Ed) 1.1-1.8, 1.12-1.14 Hayes 3.3 – 3.7 (3.5); 4.7

2.2 Wiener filteringHaykin (4th Ed) Chapter 2Hayes 7.1, 7.2, 7.3.1

2 3 2 4 Li di ti d L i D bi i2.3-2.4 Linear prediction and Levinson-Durbin recursionHaykin (4th Ed) 3.1 – 3.3Hayes 7.2.2; 5.1; 5.2.1 – 5.2.2, 5.2.4– 5.2.5

2.5 Lattice predictorHaykin (4th Ed) 3.8 – 3.10

UMD ENEE630 Advanced Signal Processing (ver.1211) Parametric spectral estimation [3]

Hayes 6.2; 7.2.4; 6.4.1

Summary of Related Readings on PartSummary of Related Readings on Part--IIIIII

Overview Haykins 1.16, 1.10

3 1 Non-parametric method3.1 Non-parametric methodHayes 8.1; 8.2 (8.2.3, 8.2.5); 8.3

3 2 P t i th d3.2 Parametric methodHayes 8.5, 4.7; 8.4

3.3 Frequency estimationHayes 8.6

Review – On DSP and Linear algebra: Hayes 2.2, 2.3


– On probability and parameter estimation: Hayes 3.1 – 3.2

Spectrum Estimation: BackgroundSpectrum Estimation: Background

Spectral estimation: determine the power distribution in frequency of a random process©

200

3)

frequency of a random process– E.g “Does most of the power of a signal reside at low or high

frequencies?” “Are there resonances in the spectrum?”

ated

by

M.W

u ©

Applications:– Needs of spectral knowledge in spectrum domain non-causal

Wi fil i i l d i d ki b f i30 S

lides

(cre

a

Wiener filtering, signal detection and tracking, beamforming, etc.– Wide use in diverse fields: radar, sonar, speech, biomedicine,

geophysics, economics, …

CP

EN

EE

624/

63

g p y , ,

Estimating p.s.d. of a w.s.s. process is equivalent to estimate autocorrelation at all lags

UM

C


Spectral Estimation: ChallengesSpectral Estimation: Challenges

When a limited amount of observation data are available– Can’t get r(k) for all k and/or may have inaccurate estimate of r(k)©

200

3)

Can t get r(k) for all k and/or may have inaccurate estimate of r(k)– Scenario-1: transient measurement (earthquake, volcano, …) – Scenario-2: constrained to short period to ensure (approx.)

ated

by

M.W

u ©

stationarity in speech processing

30 S

lides

(cre

a

N

Mkknunukr 10][][1)(ˆ

Ob d d t h b t d b iCP

EN

EE

624/

63

kn

MkknunukN

kr1

,1,0 ],[][)(

Observed data may have been corrupted by noise

UM

C


Spectral Estimation: Major ApproachesSpectral Estimation: Major Approaches

Nonparametric methods– No assumptions on the underlying model for the data

© 2

003)

p y g– Periodogram and its variations (averaging, smoothing, …)– Minimum variance method

ated

by

M.W

u ©

Parametric methods– ARMA, AR, MA models

M i t th d30 S

lides

(cre

a

– Maximum entropy method

Frequency estimation (noise subspace methods)For harmonic processes that consist of a sum of sinusoids orC

P E

NE

E62

4/63

– For harmonic processes that consist of a sum of sinusoids or complex-exponentials in noise

High-order statistics

UM

C


g

Example of Speech SpectrogramExample of Speech Spectrogramu

© 2

002)

eate

d by

M. W

u08

G S

lides

(cre

UM

CP

ENEE

40U


Figure 3 of SPM May’98 Speech Survey

)

8000

“Sprouted grains and seeds are used in salads and dishes such as chop suey”W

ilson

© 2

004)

6000

cy (k

Hz)

by C

arol

Esp

y-W

2000

4000

Freq

uenc

F2

lides

(cre

ated

b

1.0 2.0 3.0 4.0 0.0

Time (sec)

EN

EE

408G

Sl

6000

8000

(kH

z)

“S t d”

UM

CP

2000

4000

requ

ency

(“Sprouted”


0.1 0.3 0.5

Fr

fricative stopconsonantglide vowel stopconsonantvowel

Section 3.1 Classical Nonparametric MethodsSection 3.1 Classical Nonparametric Methods©

200

3) Recall: given a w.s.s. process {x[n]} with

ated

by

M.W

u ©

)(]][][[]][[

krknxnxEmnxE x

30 S

lides

(cre

a

The power spectral density (p.s.d.) is defined as

)(]][][[ krknxnxE

k

fkjekrfP 2)()(

CP

EN

EE

624/

63

21

21

f

k

As we can take DTFT on a specific realization of a random process,What is the relation between the DTFT of a specific signal and the

UM

C ):2or ( f


What is the relation between the DTFT of a specific signal and the p.s.d. of the random process?

Ensemble Average of Squared Fourier MagnitudeEnsemble Average of Squared Fourier Magnitude

p.s.d. can be related to the ensemble average of the squared Fourier magnitude |X()|2

© 2

003)

q g | ( )|

ated

by

M.W

u ©

22][

121)(Consider

M

fnjM enx

MfP

30 S

lides

(cre

a 12 Mn

M M

M M

mnfjemxnx )(2][][1

i.e., take DTFT on (2M+1) samples and CP

EN

EE

624/

63

Mn Mm

emxnxM

][][12

, ( ) pexamine normalized squared magnitude

Note: for each frequency f, PM(f) is a random variable

UM

C


Ensemble Average of Squared Fourier MagnitudeEnsemble Average of Squared Fourier Magnitude

p.s.d. can be related to the ensemble average of the squared Fourier magnitude |X()|2

© 2

003)

q g | ( )|

ated

by

M.W

u ©

22][

121)(Consider

M

fnjM enx

MfP

30 S

lides

(cre

a 12 Mn

M M

M M

mnfjemxnx )(2][][1

CP

EN

EE

624/

63

Mn Mm

emxnxM

][][12

i.e., take DTFT on (2M+1) samples and

UM

C , ( ) pexamine normalized squared magnitude

Note: for each frequency f, PM(f) is a random variable


Ensemble Average of PEnsemble Average of PMM(f)(f)©

200

3)

M M

mnfjM emnr

MfPE )(2)(

121)]([

ated

by

M.W

u © Mn MmM 12

M

fkjekrkMM

22)()12(

121

Mfkjekr

k22)(1

30 S

lides

(cre

a MkM 212

Mk

ekrM2

)(12

1

M

fkM

fk2

22

2 1CP

EN

EE

624/

63

Mk

fkj

Mk

fkj ekrkM

ekr2

2

2

2 )(12

1)(

UM

C


Now, what if M goes to infinity?

Ensemble Average of PEnsemble Average of PMM(f)(f)©

200

3)

M M

mnfjM emnr

MfPE )(2)(

121)]([

ated

by

M.W

u © Mn MmM 12

M

fkjekrkMM

22)()12(

121

30 S

lides

(cre

a

Mfkjekr

k22)(1

MkM 212

CP

EN

EE

624/

63

Mk

ekrM2

)(12

1

M

fkM

fk2

22

2 1

UM

C

Mk

fkj

Mk

fkj ekrkM

ekr2

2

2

2 )(12

1)(


Now, what if M goes to infinity?

P.S.D. and Ensemble Fourier MagnitudeP.S.D. and Ensemble Fourier Magnitude©

200

3) If the autocorrelation function decays fast enough s.t.

ated

by

M.W

u ©

)for rapidly 0 (i.e., )(

kr(k)krkk

fkj 2

30 S

lides

(cre

a

then

k

fkjMM

fPekrfPE )()()]([lim 2

p.s.d.

)( ][12

1lim)(2

2

M

Mn

fnj

Menx

MEfP

CP

EN

EE

624/

63

Thus

Mn

UM

C


P.S.D. and Ensemble Fourier MagnitudeP.S.D. and Ensemble Fourier Magnitude©

200

3) If the autocorrelation function decays fast enough s.t.

ated

by

M.W

u ©

th

fkj 2

)for rapidly 0 (i.e., )(

kr(k)krkk

30 S

lides

(cre

a then

k

fkjMM

fPekrfPE )()()]([lim 2

p.s.d.

CP

EN

EE

624/

63

Thus )( ][12

1lim)(2

2

M

Mn

fnj

Menx

MEfP

UM

C Mn


3.1.1 Periodogram Spectral Estimator3.1.1 Periodogram Spectral Estimator©

200

3) (1) This estimator is based on (**)

ated

by

M.W

u ©

Given an observed data set {x[0], x[1], …, x[N-1]},the periodogram is defined as

30 S

lides

(cre

a p g21

2PER ][1)(

N

fnjenxN

fP

CP

EN

EE

624/

63

0)(

nNf

UM

C


3.1.1 Periodogram Spectral Estimator3.1.1 Periodogram Spectral Estimator©

200

3) (1) This estimator is based on (**)

ated

by

M.W

u ©

Given an observed data set {x[0], x[1], …, x[N-1]},the periodogram is defined as

30 S

lides

(cre

a 21

0

2PER ][1)(

N

fnjenxN

fP

CP

EN

EE

624/

63 0nN

UM

C


An Equivalent Expression of PeriodogramAn Equivalent Expression of Periodogram20

04) The periodogram estimator can be given in terms of )(kr

N 1

d by

M.W

u ©

2

fkjN

NkekrfP 2

1

)1(PER )()(

0for )()( ];[][1)(1

0

kkrkrknxnxN

krkN

nSlid

es (c

reat

ed

where0n

– The quality of the estimates for the higher lags of r(k) may be poorer since they involve fewer terms of lag products in theM

CP

ENEE

630

poorer since they involve fewer terms of lag products in the averaging operation

Exercise: to show this from the periodogram definition in last page

UM



An Equivalent Expression of PeriodogramAn Equivalent Expression of Periodogram20

04) The periodogram estimator can be given in terms of )(kr

N 1

d by

M.W

u ©

2

fkjN

NkekrfP 2

1

)1(PER )()(

Slid

es (c

reat

ed

where 0for )()( ];[][1)(1

0

kkrkrknxnxN

krkN

n

– The quality of the estimates for the higher lags of r(k) may be poorer since they involve fewer terms of lag products in theM

CP

ENEE

630 0n

poorer since they involve fewer terms of lag products in the averaging operation


UM


p g p g

(2) Filter Bank Interpretation of Periodogram(2) Filter Bank Interpretation of Periodogram©

200

3)

For a particular frequency of f0:21

2 ][1)( 0

N

kfj kfP

ated

by

M.W

u ©

0

20PER ][)( 0

k

kfj kxeN

fP

21 N

30 S

lides

(cre

a

00

][][

nk

kxknhN

where

h i0

;0,1),...,1(for )2exp(1][ 0 Nnnfj

Nnh

CP

EN

EE

624/

63 where

otherwise0

– Impulse response of the filter h[n]: a windowed version of a

UM

C


complex exponential

(2) Filter Bank Interpretation of Periodogram(2) Filter Bank Interpretation of Periodogram©

200

3)

For a particular frequency of f0:21

2 ][1)( 0

N

kfj kfP

ated

by

M.W

u ©

0

20PER ][)( 0

k

kfj kxeN

fP

21 N

30 S

lides

(cre

a

00

][][

nk

kxknhN

where

CP

EN

EE

624/

63

h i0

;0,1),...,1(for )2exp(1][ 0 Nnnfj

Nnh where

– Impulse response of the filter h[n]: a windowed version of a

UM

C otherwise0


complex exponential

Frequency Response of h[n]Frequency Response of h[n]©

200

3) )()1(exp)(i)(sin)( 0

0 ffNjffNffNfH

ated

by

M.W

u ©

)()(p)(sin

)( 00

ffjffN

f

sinc-like function centered at f0:

H(f) is a bandpass filterC f i f30

Slid

es (c

rea 0:

– Center frequency is f0

– 3dB bandwidth 1/N

CP

EN

EE

624/

63U

MC


Periodogram: Filter Bank PerspectivePeriodogram: Filter Bank Perspective

Can view the periodogram as an estimator of power spetrum that has a built-in filterbank©

200

3)

p– The filter bank ~ a set of bandpass filters– The estimated p.s.d. for each frequency f0 is the power of one

ated

by

M.W

u ©

p q y 0 poutput sample of the bandpass filter centering at f0

30 S

lides

(cre

aC

P E

NE

E62

4/63

21

0PER ][][)(

N

kxknhNfP

UM

C

00

n

k


E.g. White Gaussian ProcessE.g. White Gaussian Process[Lim/Oppenheim Fig.2.4] Periodogram of zero-mean white Gaussian noise using N-point data record: N=128, 256, 512, 1024

© 2

003)

ated

by

M.W

u ©

30 S

lides

(cre

aC

P E

NE

E62

4/63

The random fluctuation (measured by variance) of the i d d t d ith i i N

UM

C


periodogram does not decrease with increasing N periodogram is not a consistent estimator

(3) How Good is Periodogram for Spectral Estimation?(3) How Good is Periodogram for Spectral Estimation?00

3/20

04)

?)( p.s.d. will, If PER fPPN

Estimation: Tradeoff between bias and varianced by

M.W

u ©

20

Slid

es (c

reat

ed

For white Gaussian process, we can show that at fk = k/N

ENE

E62

4/63

0 U

MC

P E


Performance of Periodogram: SummaryPerformance of Periodogram: Summary

The periodogram for white Gaussian process is an unbiased estimator but not consistent

© 2

003)

– The variance does not decrease with increasing data length– Its standard deviation is as large as the mean (equal to the quantity

to be estimated)ated

by

M.W

u ©

to be estimated)

Reasons for the poor estimation performance– Given N real data points, the # of unknown parameters {P(f0), … 30

Slid

es (c

rea

p , p { ( 0),P(fN/2)} we try to estimate is N/2, i.e. proportional to N

Similar conclusions can be drawn for processes withCP

EN

EE

624/

63

Similar conclusions can be drawn for processes with arbitrary p.s.d. and arbitrary frequencies– Asymptotically unbiased (as N goes to infinity) but inconsistent

UM

C


3.1.2 Averaged Periodogram3.1.2 Averaged Periodogram

As one solution to the variance problem of periodogram– Average K periodograms computed from K sets of data records

© 2

003)

g p g p

ated

by

M.W

u ©

1 (m)

PERPER AV )(1)(K

fPfP

30 S

lides

(cre

a 0

V )()(m

fK

f

where 212

(m)

PER ][1)(

L

fnjenxfP

CP

EN

EE

624/

63

0PER ][)(

n

m enxL

fP

And the K sets of data records are

UM

C

... ;10 ],[ ];1[ ..., ],0[{ 100 LnnxLxx}10 ],1[{ 1 LnnxK


}],[{ 1K

Performance of Averaged PeriodogramPerformance of Averaged Periodogram

– If K sets of data records are uncorrelated with each other,we have: ( fi = i/L )

© 2

003)

( fi )

ated

by

M.W

u ©

1

30 S

lides

(cre

a

)(1 2ifP

K

CP

EN

EE

624/

63

KK for 0Varand,Var i.e., KK for 0Var and ,Var i.e.,

UM

C

KK for 0Varand,Var i.e., ,,i.e., consistent estimatei.e., consistent estimate


,,i.e., consistent estimate

Performance of Averaged PeriodogramPerformance of Averaged Periodogram

– If K sets of data records are uncorrelated with each other,we have : ( fi = i/L )

© 2

003)

( fi )

ated

by

M.W

u ©

1

30 S

lides

(cre

a

)(1 2ifP

K

CP

EN

EE

624/

63U

MC

KK for 0Var and ,Var i.e.,i e consistent estimate


i.e., consistent estimate

Practical Averaged PeriodogramPractical Averaged Periodogram

Usually we partition an available data sequence of length Ninto K non overlapping blocks each block has length L (i e N=KL)©

200

3)

– into K non-overlapping blocks, each block has length L (i.e. N=KL)

ated

by

M.W

u ©

i.e. 1 ..., ,1 ,0 ,][][ LnmLnxnxm

110 K

Since the blocks are contiguous, the K sets of data records 30 S

lides

(cre

a 1...,,1,0 Km

gmay not be completely uncorrelated– Thus the variance reduction factor is in general less than K

CP

EN

EE

624/

63

Periodogram averaging is also known as the Bartlett’s method

UM

C


Averaged Periodogram for Fixed Data SizeAveraged Periodogram for Fixed Data Size

Given a data record of fixed size N, will the result be better if we segment the data into more and more subrecords?

© 2

003)

We examine for a real-valued stationary process:

ated

by

M.W

u ©

1 1 )(K m

30 S

lides

(cre

a )(ˆ)(1)( )0(PER

0

)(

PERPER AV fPEfPK

EfPEm

CP

EN

EE

624/

63

identical stat. mean for all mNote

1

2)0()0(PER )(ˆ)(ˆ

LfljelrfP

UM

C

where

)1(Ll

lL

lnxnxlr1

)0( ][1)(ˆ

an equivalent expression to definition in terms of x[n]


n

lnxnxL

lr0

][)( of x[n]

Mean of Averaged PeriodogramMean of Averaged Periodogram©

200

3)at

ed b

y M

.Wu

©30

Slid

es (c

rea

CP

EN

EE

624/

63U

MC


Mean of Averaged Periodogram (cont’d)Mean of Averaged Periodogram (cont’d)©

200

3) fkrkwfPE )}(][{DTFT)](ˆ[ PER AV 1

multiplication in time

ated

by

M.W

u ©

dPfW )()(21

21

)( fP

convolution in frequency

Biased estimate (both averaged and regular periodogram)– The convolution with the window function w[k] lead to the mean of30

Slid

es (c

rea )( fP

– The convolution with the window function w[k] lead to the mean of the averaged periodogram being smeared from the true p.s.d

Asymptotic unbiased as L

CP

EN

EE

624/

63

– To avoid the smearing, the window length L must be large enough so that the narrowest peak in P(f) can be resolved

This gives a tradeoff between bias and variance

UM

C


gSmall K => better resolution (smaller smearing/bias) but larger variance

NonNon--parametric Spectrum Estimation: Recapparametric Spectrum Estimation: Recap

Periodogram– Motivated by relation between p.s.d. and squared magnitude of DTFT

of a finite-size data record– Variance: won’t vanish as data length N goes infinity ~ “inconsistent”– Mean: asymptotically unbiased w r t data length N in generalMean: asymptotically unbiased w.r.t. data length N in general

equivalent to apply triangular window to autocorrelation function(windowing in time gives smearing/smoothing in freq. domain)

unbiased for white Gaussian (flat spectrum) unbiased for white Gaussian (flat spectrum)

Averaged periodogram– Reduce variance by averaging K sets of data record of length L eachReduce variance by averaging K sets of data record of length L each– Small L increases smearing/smoothing in p.s.d. estimate thus higher

bias equiv. to triangular windowing


Windowed periodogram: generalize to other symmetric windows

Case Study on NonCase Study on Non--parametric Methodsparametric Methods Test case: a process consists of narrowband components

(sinusoids) and a broadband component (AR)– x[n] = 2 cos(1 n) + 2 cos(2 n) + 2 cos(3 n) + z[n]

where z[n] = a1 z[n-1] + v[n], a1 = 0.85, 2 = 0.1 /2 = 0 05 /2 = 0 40 /2 = 0 421/2 = 0.05, 2/2 = 0.40, 3/2 = 0.42

– N=32 data points are available periodogram resolution f = 1/32

Examine typical characteristics of various non-parametric pspectral estimators

(Fig.2.17 from Lim/Oppenheim book)


3.1.3 Periodogram with Windowing3.1.3 Periodogram with Windowing

Review and Motivation

© 2

003)

The periodogram estimator can be given in terms of )(k

ated

by

M.W

u © The periodogram estimator can be given in terms of )(kr

1

2PER )()(

NfkjekrfP

30 S

lides

(cre

a )1( Nk

where )()( ];[][1)(1

kN

krkrknxnxN

kr

– The higher lags of r(k), the poorer estimates since the estimates

CP

EN

EE

624/

63

)()(][][)(0nN 0for k

g g ( ) pinvolve fewer terms of lag products in the averaging operation

Solution: weigh the higher lags less

UM

C


– Trade variance with bias

WindowingWindowing Use a window function to weigh the higher lags less

© 2

003)

ated

by

M.W

u ©

30 S

lides

(cre

a

w(0)=1 preserves variance r(0)

CP

EN

EE

624/

63

Effect: periodogram smoothing – Windowing in time Convolution/filtering the periodogram

UM

C


g g p g– Also known as the Blackman-Tukey method

Common Lag WindowsCommon Lag Windows Much of the art in non-parametric spectral estimation is in

choosing an appropriate window (both in type and length)

© 2

003)

ated

by

M.W

u ©

30 S

lides

(cre

aC

P E

NE

E62

4/63

UM

C


Table 2.1 common lag window (from Lim-Oppenheim book)

Discussion: Estimate r(k) via Time AverageDiscussion: Estimate r(k) via Time Average Normalizing the sum of (N-k) pairs

by a factor of 1/N ? v.s. by a factor of 1/(N-k) ?Biased (low variance) Unbiased (may not non-neg. definite)

• Hints on showing the non-negative definiteness: using )(kr

definiteness: using to construct correlation matrix

)(1 kr


HW#8)(For 2 kr

Discussion: Estimate r(k) via Time AverageDiscussion: Estimate r(k) via Time Average Normalizing the sum of (N-k) pairs

by a factor of 1/N ? v.s. by a factor of 1/(N-k) ?Biased (low variance) Unbiased (may not non-neg. definite)

• Hints on showing the non-negative definiteness: using )(kr

HW#8

definiteness: using to construct correlation matrix

)(1 kr


HW#8)(For 2 kr

3.1.43.1.4 Minimum Variance Spectral Estimation Minimum Variance Spectral Estimation (MVSE)(MVSE)

Recall: filter bank perspective of periodogram– The periodogram can be viewed as estimating the p s d byThe periodogram can be viewed as estimating the p.s.d. by

forming a bank of narrowband filters with sinc-like response– The high sidelobe can lead to “leakage” problem:

large output power due to p.s.d outside the band of interest

MVSE designs filters to minimize the leakage from out-of-MVSE designs filters to minimize the leakage from out ofband spectral components– Thus the shape of filter is dependent on the frequency of interest

d d t d tiand data adaptive(unlike the identical filter shape for periodogram)


– MVSE is also referred to as the Capon spectral estimator

Main Steps of MVSE MethodMain Steps of MVSE Method

1. Design a bank of bandpass filters Hi(f) with center frequency fi so thati

– Each filter rejects the maximum amount of out-of-band power– And passes the component at frequency fi without distortion

2. Filter the input process { x[n] } with each filter in the filter bank and estimate the power of each output processbank and estimate the power of each output process

3. Set the power spectrum estimate at frequency fi to be the power estimated above divided by the filter bandwidth


Formulation of MVSEFormulation of MVSE

The MVSE designs a filter H(f) for each f f i t t ffrequency of interest f0

minimize the output power

dffPfH )()( 221

1

minimize the output power

dffPfH )()(21

subject to 1)( 0 fH

(i.e., to pass the components at f0 w/o distortion)


Formulation of MVSEFormulation of MVSE

The MVSE designs a filter H(f) for each f f i t t ffrequency of interest f0

minimize the output powerminimize the output power

dffPfH )()( 221

1

subject to

dffPfH )()(21

1)( 0 fH

(i.e., to pass the components at f0 w/o distortion)


Deriving MVSE SolutionsDeriving MVSE Solutions


Output Power From H(f) filterOutput Power From H(f) filter

From the filter bank perspective of periodogram:0

)1(

2][)(Nn

fnjenhfH

Thus

dffPelhekh fljfkj )(][][0

221

1

02

0 0

21

)(2)(][][ klfj dfefPlhkh

ffNlNk

)(][][)1(2

1)1(

)1( )1( 2

1 )(][][Nk Nl

dfefPlhkh

0 0

)(][][ klrlhkhEquiv. to filter r(k) with { h(k) h*(-k) } and evaluate at

)1( )1(

)(][][Nk Nl

klrlhkh


and evaluate at output time k=0

Output Power From H(f) filterOutput Power From H(f) filter

From the filter bank perspective of periodogram:0

)1(

2][)(Nn

fnjenhfH

Thus

dffPelhekh fljfkj )(][][0

221

1

02 ff

NlNk)(][][

)1(21

)1(

0 0

21

)(2)(][][ klfj dfefPlhkh

Equiv. to filter r(k) with { h(k) h*(-k) } and evaluate at

Equiv. to filter r(k) with { h(k) h*(-k) } and evaluate at

)1( )1( 2

1 )(][][Nk Nl

dfefPlhkh

0 0

)(][][ klrlhkh


and evaluate at output time k=0and evaluate at output time k=0

)1( )1(

)(][][Nk Nl

klrlhkh

MatrixMatrix--Vector Form of MVSE FormulationVector Form of MVSE Formulation

Define

The constraint can be written in vector form as 1ehH

)( 0fH

Thus the problem becomes

hRh THmin subject to 1ehH

hj


MatrixMatrix--Vector Form of MVSE FormulationVector Form of MVSE Formulation

Define

The constraint can be written in vector form as 1ehH

)( 0fH

Thus the problem becomes

hRh THmin subject to 1ehH


hj

Solving MVSESolving MVSE )1(2Re ehhRhJ HTHdef

Use Lagrange multiplier approach for solving the constrained optimization problem

– Define real-valued objective function s.t. the stationary condition can be derived in a simple and elegant way based on the theorem for complex derivative/gradient operatorsp g p

)1()1(min*

,ehehhRhJ HHTH

h

)1()1( * heehhRh HHTH

00either * ehRJ T 1

00or

00either

**

*

eRhJ

ehRJTTH

h

h

1and

1

eh

eRhH

T


00 ehRehR THT 1 and eh

Solution to MVSESolution to MVSE *

,)1()1(min ehehhRhJ HHTH

h

)( 1 0or H*

ehJ

)( )(0 0or 1*

eRhehRJ TT

hh

The optimal filter and its output power:

T 1 RTH 11

Bring () into ():

e

eReRh

TH

T

MV 1

1

Filter’s output power:

eRe T

eRe TH 11

1eRRhhRh TTHTH


MVSE: SummaryMVSE: Summary

If choosing the bandpass filters to be FIR of length q, its 3dB-b.w. is approximately 1/q

Thus the MVSE ismatrix ncorrelatio is ˆ qqR

eReqfP

TH 1MVˆ

)(

)2exp(

1fj

e

(i.e. normalize by filter b.w.)

))1(2exp( qfj

MVSE is a data adaptive estimator and provides improved resolution and reduced variance over periodogram– Also referred to as “High-Resolution Spectral Estimator”


Also referred to as High Resolution Spectral Estimator– Doesn’t assume a particular underlying model for the data

MVSE vsMVSE vs. . PeriodogramPeriodogram

MVSE is a data adaptive estimator and provides improved resolution and reduced variance over periodogramp g

Periodogram MVSEEquivalent Bandpass Filter

h

e e

eReR

TH

T

1

1

hFilter is “universal” data-independent

Filter adapts to observation data via R

E i l t

Equivalent spectrum estimate eRe

qTH 1ˆ

)( fP eReq TH ˆ


Recall: Case Study on NonRecall: Case Study on Non--parametric Methods parametric Methods Test case: a process consists of narrowband components

(sinusoids) and a broadband component (AR)– x[n] = 2 cos(1 n) + 2 cos(2 n) + 2 cos(3 n) + z[n]

where z[n] = a1 z[n-1] + v[n], a1 = 0.85, 2 = 0.1 /2 = 0 05 /2 = 0 40 /2 = 0 421/2 = 0.05, 2/2 = 0.40, 3/2 = 0.42

– N=32 data points are available periodogram resolution f = 1/32

Examine typical characteristics of various non-parametric pspectral estimators

(Fig.2.17 from Lim/Oppenheim book)


Ref. on Derivative and Gradient Operators for Ref. on Derivative and Gradient Operators for C lC l V i bl F tiV i bl F tiComplexComplex--Variable FunctionsVariable Functions

Ref: D.H. Brandwood, “A complex gradient operator and its application i d ti th ” i IEE P l 130 P t F d H 1in adaptive array theory,” in IEE Proc., vol. 130, Parts F and H, no.1, Feb. 1983. (downloadable from IEEEXplorer)

– Solving constrained optimization ith l l d bj ti f ti f l i blwith real-valued objective function of complex variables,

subject to constraint function of complex variables As seen in minimum variance spectral estimation and other As seen in minimum variance spectral estimation and other

array/statistical signal processing context.

UMD ENEE630 Advanced Signal Processing (ver.1211) Discussions [64]

Liu

© 2

002)

ReferenceReferenceby M

.Wu

& R

.L

ReferenceReference

Slid

es (c

reat

ed b

P E

NE

E40

8G S

UM

CP


Recall: Filtering a Random ProcessRecall: Filtering a Random Process


ChiChi--Squared DistributionSquared Distribution


ChiChi--Squared Distribution (cont’d)Squared Distribution (cont’d)


Periodogram of White Gaussian ProcessPeriodogram of White Gaussian Process

See proof in Appendix 2 1 in Lim Oppenheim Book:


See proof in Appendix 2.1 in Lim-Oppenheim Book:- Basic idea is to examine the distribution of real and imaginary part of the DFT, and take the magnitude

Liu

© 2

002)

Preview & WarmPreview & Warm upupby M

.Wu

& R

.L

Preview & WarmPreview & Warm--upup

Slid

es (c

reat

ed b

P E

NE

E40

8G S

UM

CP


Model or Not?Model or Not? Implicit assumption by classical methods

– Classical methods use Fourier transform on either windowed d t i d d t l ti f ti (ACF)data or windowed autocorrelation function (ACF)

– Implicitly assume the unobserved data or ACF outside the window are zero => not true in reality

– Consequence of windowing: smeared spectral estimate(leading to low resolution)

If prior knowledge about the process is available If prior knowledge about the process is available– Can use prior knowledge and select a good model to

approximate the process– Usually need to estimate fewer model parameters (than non-

parametric approaches) using the limited data points we have– The model may allow to better describe the process outside


The model may allow to better describe the process outside the window (instead of assuming zeros)

General Procedure of Parametric MethodsGeneral Procedure of Parametric Methods

Select a model (based on prior knowledge)

Estimate the parameters of the assumed model

Obtain the spectral estimate implied by the model (with the estimated parameters)


Spectral Estimation using AR, MA, ARMA ModelsSpectral Estimation using AR, MA, ARMA Models

Physical insight: the process is generated/approximated by filtering white noise with an LTI filter of rational transfer func H(z)

Use observed data to estimate a few lags of r(k)– Larger lags of r(k) can be implicitly extrapolated by the model

Relation between r(k) and filter parameters {ak} and {bk}PARAMETER EQUATIONS f S ti 2 1 2(6) i thi– PARAMETER EQUATIONS from Section 2.1.2(6) review this

– Solve the parameter equations to obtain filter parameters– Use the p.s.d. implied by the model as our spectral estimatep p y p

Deal with nonlinear parameter equationsTry to “convert” or relate them to AR models that has linear equations


– Try to convert or relate them to AR models that has linear equations

Review: Parameter EquationsReview: Parameter Equations

Yule-Walker equations (for AR process)

ARMA model MA model


Spectrum Spectrum Estimation with Estimation with AR ModelingAR Modeling

• Use Levinson-Durbin recursion and solve for

where

– Approximate the observed data sequence {x[0] x[N]} with an ARApproximate the observed data sequence {x[0], …, x[N]} with an AR model (consider real-valued process here for simplicity)

– Use biased ACF estimate here to ensure nonnegative definiteness and ll i th bi d ti t (di idi b N k)


smaller variance than unbiased estimate (dividing by N-k)

MA MA Spectral EstimationSpectral Estimation

An MA(q) modelqq

q

k

kk

q

kk zbzBknvbnx

00)( ][][

can be used to define an MA spectral estimator2

22 1)(ˆ q

fkjbfP

Recall: ( ) f f { ( )} f

1

22MA 1)(

k

fkjkebfP

(1) The problem of solving for bk given {r(k)} is to solve a set of nonlinear equations;

(2) An MA process can be approximated by an AR process of


( ) p pp y psufficiently high order.

Basic Idea to Avoid Solving Nonlinear EquationsBasic Idea to Avoid Solving Nonlinear Equations

Consider two processes:

Process#1: we observed N samples and need to perform Process#1: we observed N samples, and need to perform spectral estimate– We first model it as a high-order AR process, generated by 1/A(z) filter

Process#2: an MA-process generated by A(z) filter– Since we know A(z), we can know process#2’s autocorrelation function;


– We model process#2 as an AR(q) process => the filter would be 1/B(z)

Complex Exponentials in Additive NoiseComplex Exponentials in Additive Noise

UMD ENEE630 Advanced Signal Processing Frequency estimation [78]

Correlation Matrix for the ProcessCorrelation Matrix for the Process

– Determine autocorrelation function

– Rs = ? Rw = ? Rx = ?

– Rank of correlation matrices?



E[ x( ) w( ) ] = E[x( )] E[w( )] = 0


E[ x( ) w( ) ] = E[x( )] E[w( )] = 0this crosscorr term vanish because of uncorrelated *and* zero mean for either x( ) or w( ).

part 3. part 3. spectrum spectrum …classweb.ece.umd.edu › enee630.f2012 › slides ›...

Documents