on two different signal processing modelshome.iitk.ac.in/~kundu/indo-russian-2015.pdf · first...

On Two Different Signal Processing

Models

Debasis Kundu

Department of Mathematics & StatisticsIndian Institute of Technology Kanpur

January 15, 2015

Debasis Kundu On Two Different Signal Processing Models

First ModelBasic Formulation

Different Estimation ProceduresSecond ModelCollaborators

References

Outline

1 First Model

2 Basic Formulation

3 Different Estimation Procedures

4 Second Model

5 Collaborators

6 References




References

Introduction

We observe periodic phenomena everyday in our lives. For examplethe number of tourists visiting the famous Taj Mahal, the dailytemperature of Delhi or the ECG data of a normal human beingclearly follow periodic pattern. Sometimes the data may not beexactly periodic but it is nearly periodic.Our aim is to analyze such periodic/ nearly periodic data.




References

Question?

1 What is a periodic data?

2 Why do we care to analyze?




References

What is a periodic data?

We do not give the formal definition. But informally speaking

1 It shows a repeated (periodic) pattern in one dimension.

2 It shows a symmetric (periodic) pattern in higher dimension.




References

Why do we want to analyze?

1 Theoretical reason.

2 Prediction purposes.

3 Compression purposes.




References

Example: Airlines Passenger Data

0 20 40 60 80

200

300

400

500

600

Airline passengers data

t −−−−>

x(t)

−−

−−

>




References

Example: Brightness of Variable Star Data

t

y(t)

0

5

10

15

20

25

30

35

0 100 200 300 400 500 600




References

Example: Vowel Sound Data ’uuu’

t

y(t)

−3000

−2000

−1000

0

1000

2000

3000

0 100 200 300 400 500 600




References

Example: ECG Data of a Normal Human

0 100 200 300 400 500 600−200

−100

0

100

200

300

400

500

600

700Original Signal

m

y(m)




References

Example: Two Dimension Periodic Data

..Debasis Kundu On Two Different Signal Processing Models



References

Example: Three Dimension Periodic Data

..Debasis Kundu On Two Different Signal Processing Models



References

Outline

1 First Model

2 Basic Formulation


4 Second Model

5 Collaborators

6 References




References

Simplest Periodic Function

The simplest periodic function is the sinusoidal function, and it canbe written in the following form:

y(t) = A cos(ωt) + B sin(ωt)

The period of y(t) is the shortest time taken for y(t) to repeatitself, and it is 2π/ω.




References

Smooth Periodic Function

In general a smooth periodic function (mean adjusted) with period2π/ω, can be written in the form:

y(t) =∞∑

k=1

[Ak cos(ωkt) + Bk sin(ωkt)] ,

and it is well known as the Fourier expansion of y(t).




References

Extracting Parameters

From y(t), Ak and Bk can be obtained uniquely.

∫ 2π/ω

0cos(jωt)y(t)dt =

πAj

ω if j ≥ 1

2πA0ω if j = 0

and

∫ 2π/ω

0sin(jωt)y(t)dt =

πBj

ω.




References

Noisy Periodic Function

Most of the times y(t) is corrupted with noise, so we observe thefollowing:

y(t) =∞∑

k=1

[Ak cos(ωt) + Bk sin(ωt)] + X (t),

where X (t) is the noise component.




References

Practical Model

It is impossible to estimate infinite number of parameters. Hencethe model is approximated by the following model:

y(t) =

p∑

k=1

[Ak cos(ωkt) + Bk sin(ωkt)] + X (t),

for some p < ∞.




References

Model

The model has two components,

1 Deterministic component

2 Random component




References

Aim

The aim is to extract (estimate) the deterministic component µ(t),where

µ(t) =

p∑

k=1

[Ak cos(ωkt) + Bk sin(ωkt)] ,

in presence of the random error component X (t), based on theavailable data y(t), t = 1, . . . ,N.




References

Problem Formulation

Based on the available data {y(t); t = 1, . . . ,N},1 Deterministic Component

Determine (estimate) pDetermine (estimate) A1, . . . ,Ap, B1, . . . ,Bp

Determine (estimate) ω1, . . . , ωp.

2 Random Component

Estimate X (t)




References

Procedure

1 Assume certain structure on X (t)

2 Estimate the deterministic component µ(t)

3 Estimate the error X (t)

4 Verify the assumption.

5 If the assumption is satisfied then stop the process, otherwisego back to step 1.




References

Outline

1 First Model

2 Basic Formulation


4 Second Model

5 Collaborators

6 References




References

Periodogram Estimators

The most used and popular estimation procedure is theperiodogram estimators. The periodogram at a particularfrequency is defined as

I (θ) =

(1

N

N∑

t=1

y(t) cos(θt)

)2

+

(1

N

N∑

t=1

y(t) sin(θt)

)2

≈(

1

N

N∑

t=1

µ(t) cos(θt)

)2

+

(1

N

N∑

t=1

µ(t) sin(θt)

)2




References

Periodogram Estimator

Consider the following sinusoidal signal: Sinusoidal Example 1:

y(t) = 3.0(cos(0.2πt)+sin(0.2πt))+3.0(cos(0.5πt)+sin(0.5πt))+X (t)

Here X (t)’s are i.i.d. N(0,0.5)




References

Examples: Sinusoidal Signal

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5

0 0.5 1 1.5 2 2.5 3




References

Periodogram Estimator

Consider the following sinusoidal signal: Sinusoidal Example 2:

y(t) = 3.0(cos(0.2πt)+sin(0.2πt))+0.25(cos(0.5πt)+sin(0.5πt))+X (t)

Here X (t)’s are i.i.d. N(0,2.0)




References

Examples: Sinusoidal Signal

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

0 0.5 1 1.5 2 2.5 3




References

Least Squares Estimators

Assuming p is known, the most natural estimators will be the leastsquares estimators and they can be obtained as follows:

n∑

t=1

(y(t)−

[p∑

k=1

Ak cos(ωkt) + Bk sin(ωkt)

])2




References

Numerical Issues

1 It is a highly non-linear problem. The least squares surfacehas several local minima.

2 Most of the time the standard Newton-Raphson algorithmmay not converge.

3 Even if they converge, often it converges to the localminimum rather than the global minimum.

4 If p is large, it becomes a higher dimensional optimizationproblem, extremely accurate initial guesses are required forany iterative procedure to work well.




References

Theoretical Issues

1 It can be treated as a standard non-linear regression problemas follows:

y(t) = ft(θ) + X (t)

2 Unfortunately it does not satisfy the standard sufficientconditions of Wu (1981) or Jennrich (1969) for theconsistency of the least squares estimators.

3 It can be shown that the least squares estimators areconsistent.

4 Ak and Bk have the convergence rate n−1/2, where as ωk hasthe convergence rate n−3/2




References

Sequential Estimation Procedures

It is based on the facts that the components are orthogonal and itworks like thisFirst minimize

n∑

t=1

(y(t)− A cos(ωt)− B sin(ωt))2

with respect to A, B and ω.

Take out their effect from y(t), i .e. consider

y(t) = y(t)− A cos(ωt)− B sin(ωt)

Repeat the procedure p times.Debasis Kundu On Two Different Signal Processing Models



References

Advantage

It reduces the computational burden significantly. For example if p= 25, instead of solving a 25 dimensional optimization problem, weneed to solve 25 one dimensional optimization problems. It doesnot have any problem about initial guess or convergence.

It produces the same accuracy as the least squares estimators.




References

Super Efficient Estimators

When p = 1, the Newton-Raphson algorithm will be of thefollowing form:

ω(j+1) = ω(j) − Q ′(ω)

Q ′′(ω)

It has been suggested

ω(j+1) = ω(j) − 1

4

Q ′(ω)

Q ′′(ω)

It not only converges, it produces estimators which are better thanthe least squares estimators.




References

Main Theoretical Results

1 Least squares estimators are consistent under mildassumptions on the errors.

2 Least squares estimators have the convergence rate N−3/2.

3 Sequential estimators have the same convergence rate as theleast squares estimators.

4 Asymptotic variances of the super efficient estimators aresmaller than the least squares estimators.

5 Periodogram estimators are consistent, but it has theconvergence rate N−1/2.




References

Outline

1 First Model

2 Basic Formulation


4 Second Model

5 Collaborators

6 References




References

Chirp Signal Model

It has the following mathematical form:

Simple Chirp Model

y(t) = A cos(αt + βt2) + B sin(αt + βt2) + X (t)

General Chirp Model

y(t) =

p∑

k=1

{Ak cos(αkt + βkt

2) + Bk sin(αkt + βkt2)}+X (t)




References

Several Applications

1 Chirps are naturally encountered in many audio signals,ranging from bird songs, music to animal vocalization andspeech.

2 Radar and Sonar systems: Chirp signals are also commonlyobserved in natural sonar systems.

3 Biology and Medicine: Chirp models have been used toanalyze EEG signals.




References

Theoretical Consideration

1 Least squares estimators are consistent under finiteness of thefourth order moment conditions on the error random variables.

2 Least squares estimators of the amplitude has the convergencerate N−1/2.

3 Least squares estimators of the frequencies have theconvergence rate N−3/2.

4 Least squares estimators of the chirp parameters have theconvergence rate N−5/2.

5 Least squares estimators are asymptotically normallydistributed.




References

One Number Theory Result:

If (θ1, θ2) ∈ (0, π)× (0, π), then except for countable number ofpoints the following results are true:

limN→∞

1

N

N∑

n=1

cos(θ1n + θ2n2) = 0

limN→∞

1

Nt+1

N∑

n=1

nt cos2(θ1n + θ2n2) =

1

2(t + 1).

limN→∞

1

Nt+1

N∑

n=1

nt sin(θ1n + θ2n2) cos(θ1n + θ2n

2) = 0.




References

Sequential Estimator

Based on the above number theory results it can be shown thatthe sequential estimators are consistent.




References

One Number Theory Conjecture

If θ1, θ2, θ′

1, θ′

2 ∈ (0, π), then except for countable number of pointsthe following results are true:

limN→∞

1√NNt

N∑

n=1

nt cos(θ1n + θ2n2) sin(θ′1n + θ′2n

2) = 0




References

Asymptotic Distribution of the Sequential

Estimators

Based on the above conjecture, it can be shown that the sequentialestimators and the least squares estimators are equivalent.




References

Two dimensional Chirp Signals

y(m, n) = A cos(α1m + β1m2 + α2n + β2n

2) +

B sin(α1m + β1m2 + α2n + β2n

2) + X (m, n)

Results can be extended to two dimensional chirp signals models.




References

Outline

1 First Model

2 Basic Formulation


4 Second Model

5 Collaborators

6 References




References

Collaborators

Z.D. Bai

Li Bai

Swagata Nandi

Ananya Lahiri

Amit Mitra

Anurag Prasad




References

Outline

1 First Model

2 Basic Formulation


4 Second Model

5 Collaborators

6 References




References

References

Kundu, D. and Nandi, S. (2008), “Parameter estimation onchirp signals in presence of stationary noise”, Statistica Sinica.

Nandi, S. and Kundu, D. (2004), “Asymptotic properties ofthe least squares estimators of the parameters of the chirpsignals’, Annals of the Institute Statistical Mathematics.

Kundu, D., Bai. Z.D., Nandi, S. and Bai, L. (2011), “Superefficient frequency estimation”, Journal of Statistical Planningand Inference.




References

References

Ananya Lahiri, D. Kundu , and A. Mitra (2012) “Efficientalgorithm for estimating the parameters of chirp signal”Journal of Multivariate Analysis.

Ananya Lahiri, D. Kundu , and A. Mitra “Estimating theparameters of multiple chirp signals” Journal of Multivariate

Analysis.Ananya Lahiri, D. Kundu , Amit Mitra (2014), “On leastabsolute deviation estimator of one dimensional chirp model”,Statistics.




References

References

B.G. Quinn and E.J. Hannan (2001), The estimation and

tracking of frequency, Cambridge University Press.

D. Kundu and S. Nandi (2012), Statistical signal processing:Frequency Estimation.




References

THANK YOU


on two different signal processing modelshome.iitk.ac.in/~kundu/indo-russian-2015.pdf · first...

Documents