On Two Different Signal Processing
Models
Debasis Kundu
Department of Mathematics & StatisticsIndian Institute of Technology Kanpur
January 15, 2015
Debasis Kundu On Two Different Signal Processing Models
First ModelBasic Formulation
Different Estimation ProceduresSecond ModelCollaborators
References
Outline
1 First Model
2 Basic Formulation
3 Different Estimation Procedures
4 Second Model
5 Collaborators
6 References
Debasis Kundu On Two Different Signal Processing Models
First ModelBasic Formulation
Different Estimation ProceduresSecond ModelCollaborators
References
Outline
1 First Model
2 Basic Formulation
3 Different Estimation Procedures
4 Second Model
5 Collaborators
6 References
Debasis Kundu On Two Different Signal Processing Models
First ModelBasic Formulation
Different Estimation ProceduresSecond ModelCollaborators
References
Introduction
We observe periodic phenomena everyday in our lives. For examplethe number of tourists visiting the famous Taj Mahal, the dailytemperature of Delhi or the ECG data of a normal human beingclearly follow periodic pattern. Sometimes the data may not beexactly periodic but it is nearly periodic.Our aim is to analyze such periodic/ nearly periodic data.
Debasis Kundu On Two Different Signal Processing Models
First ModelBasic Formulation
Different Estimation ProceduresSecond ModelCollaborators
References
Question?
1 What is a periodic data?
2 Why do we care to analyze?
Debasis Kundu On Two Different Signal Processing Models
First ModelBasic Formulation
Different Estimation ProceduresSecond ModelCollaborators
References
What is a periodic data?
We do not give the formal definition. But informally speaking
1 It shows a repeated (periodic) pattern in one dimension.
2 It shows a symmetric (periodic) pattern in higher dimension.
Debasis Kundu On Two Different Signal Processing Models
First ModelBasic Formulation
Different Estimation ProceduresSecond ModelCollaborators
References
Why do we want to analyze?
1 Theoretical reason.
2 Prediction purposes.
3 Compression purposes.
Debasis Kundu On Two Different Signal Processing Models
First ModelBasic Formulation
Different Estimation ProceduresSecond ModelCollaborators
References
Example: Airlines Passenger Data
0 20 40 60 80
200
300
400
500
600
Airline passengers data
t −−−−>
x(t)
−−
−−
>
Debasis Kundu On Two Different Signal Processing Models
First ModelBasic Formulation
Different Estimation ProceduresSecond ModelCollaborators
References
Example: Brightness of Variable Star Data
t
y(t)
0
5
10
15
20
25
30
35
0 100 200 300 400 500 600
Debasis Kundu On Two Different Signal Processing Models
First ModelBasic Formulation
Different Estimation ProceduresSecond ModelCollaborators
References
Example: Vowel Sound Data ’uuu’
t
y(t)
−3000
−2000
−1000
0
1000
2000
3000
0 100 200 300 400 500 600
Debasis Kundu On Two Different Signal Processing Models
First ModelBasic Formulation
Different Estimation ProceduresSecond ModelCollaborators
References
Example: ECG Data of a Normal Human
0 100 200 300 400 500 600−200
−100
0
100
200
300
400
500
600
700Original Signal
m
y(m)
Debasis Kundu On Two Different Signal Processing Models
First ModelBasic Formulation
Different Estimation ProceduresSecond ModelCollaborators
References
Example: Two Dimension Periodic Data
..Debasis Kundu On Two Different Signal Processing Models
First ModelBasic Formulation
Different Estimation ProceduresSecond ModelCollaborators
References
Example: Three Dimension Periodic Data
..Debasis Kundu On Two Different Signal Processing Models
First ModelBasic Formulation
Different Estimation ProceduresSecond ModelCollaborators
References
Example: Three Dimension Periodic Data
..Debasis Kundu On Two Different Signal Processing Models
First ModelBasic Formulation
Different Estimation ProceduresSecond ModelCollaborators
References
Outline
1 First Model
2 Basic Formulation
3 Different Estimation Procedures
4 Second Model
5 Collaborators
6 References
Debasis Kundu On Two Different Signal Processing Models
First ModelBasic Formulation
Different Estimation ProceduresSecond ModelCollaborators
References
Simplest Periodic Function
The simplest periodic function is the sinusoidal function, and it canbe written in the following form:
y(t) = A cos(ωt) + B sin(ωt)
The period of y(t) is the shortest time taken for y(t) to repeatitself, and it is 2π/ω.
Debasis Kundu On Two Different Signal Processing Models
First ModelBasic Formulation
Different Estimation ProceduresSecond ModelCollaborators
References
Smooth Periodic Function
In general a smooth periodic function (mean adjusted) with period2π/ω, can be written in the form:
y(t) =∞∑
k=1
[Ak cos(ωkt) + Bk sin(ωkt)] ,
and it is well known as the Fourier expansion of y(t).
Debasis Kundu On Two Different Signal Processing Models
First ModelBasic Formulation
Different Estimation ProceduresSecond ModelCollaborators
References
Extracting Parameters
From y(t), Ak and Bk can be obtained uniquely.
∫ 2π/ω
0cos(jωt)y(t)dt =
πAj
ω if j ≥ 1
2πA0ω if j = 0
and
∫ 2π/ω
0sin(jωt)y(t)dt =
πBj
ω.
Debasis Kundu On Two Different Signal Processing Models
First ModelBasic Formulation
Different Estimation ProceduresSecond ModelCollaborators
References
Noisy Periodic Function
Most of the times y(t) is corrupted with noise, so we observe thefollowing:
y(t) =∞∑
k=1
[Ak cos(ωt) + Bk sin(ωt)] + X (t),
where X (t) is the noise component.
Debasis Kundu On Two Different Signal Processing Models
First ModelBasic Formulation
Different Estimation ProceduresSecond ModelCollaborators
References
Practical Model
It is impossible to estimate infinite number of parameters. Hencethe model is approximated by the following model:
y(t) =
p∑
k=1
[Ak cos(ωkt) + Bk sin(ωkt)] + X (t),
for some p < ∞.
Debasis Kundu On Two Different Signal Processing Models
First ModelBasic Formulation
Different Estimation ProceduresSecond ModelCollaborators
References
Model
The model has two components,
1 Deterministic component
2 Random component
Debasis Kundu On Two Different Signal Processing Models
First ModelBasic Formulation
Different Estimation ProceduresSecond ModelCollaborators
References
Aim
The aim is to extract (estimate) the deterministic component µ(t),where
µ(t) =
p∑
k=1
[Ak cos(ωkt) + Bk sin(ωkt)] ,
in presence of the random error component X (t), based on theavailable data y(t), t = 1, . . . ,N.
Debasis Kundu On Two Different Signal Processing Models
First ModelBasic Formulation
Different Estimation ProceduresSecond ModelCollaborators
References
Problem Formulation
Based on the available data {y(t); t = 1, . . . ,N},1 Deterministic Component
Determine (estimate) pDetermine (estimate) A1, . . . ,Ap, B1, . . . ,Bp
Determine (estimate) ω1, . . . , ωp.
2 Random Component
Estimate X (t)
Debasis Kundu On Two Different Signal Processing Models
First ModelBasic Formulation
Different Estimation ProceduresSecond ModelCollaborators
References
Procedure
1 Assume certain structure on X (t)
2 Estimate the deterministic component µ(t)
3 Estimate the error X (t)
4 Verify the assumption.
5 If the assumption is satisfied then stop the process, otherwisego back to step 1.
Debasis Kundu On Two Different Signal Processing Models
First ModelBasic Formulation
Different Estimation ProceduresSecond ModelCollaborators
References
Outline
1 First Model
2 Basic Formulation
3 Different Estimation Procedures
4 Second Model
5 Collaborators
6 References
Debasis Kundu On Two Different Signal Processing Models
First ModelBasic Formulation
Different Estimation ProceduresSecond ModelCollaborators
References
Periodogram Estimators
The most used and popular estimation procedure is theperiodogram estimators. The periodogram at a particularfrequency is defined as
I (θ) =
(1
N
N∑
t=1
y(t) cos(θt)
)2
+
(1
N
N∑
t=1
y(t) sin(θt)
)2
≈(
1
N
N∑
t=1
µ(t) cos(θt)
)2
+
(1
N
N∑
t=1
µ(t) sin(θt)
)2
Debasis Kundu On Two Different Signal Processing Models
First ModelBasic Formulation
Different Estimation ProceduresSecond ModelCollaborators
References
Periodogram Estimator
Consider the following sinusoidal signal: Sinusoidal Example 1:
y(t) = 3.0(cos(0.2πt)+sin(0.2πt))+3.0(cos(0.5πt)+sin(0.5πt))+X (t)
Here X (t)’s are i.i.d. N(0,0.5)
Debasis Kundu On Two Different Signal Processing Models
First ModelBasic Formulation
Different Estimation ProceduresSecond ModelCollaborators
References
Examples: Sinusoidal Signal
0
0.5
1
1.5
2
2.5
3
3.5
4
4.5
5
0 0.5 1 1.5 2 2.5 3
Debasis Kundu On Two Different Signal Processing Models
First ModelBasic Formulation
Different Estimation ProceduresSecond ModelCollaborators
References
Periodogram Estimator
Consider the following sinusoidal signal: Sinusoidal Example 2:
y(t) = 3.0(cos(0.2πt)+sin(0.2πt))+0.25(cos(0.5πt)+sin(0.5πt))+X (t)
Here X (t)’s are i.i.d. N(0,2.0)
Debasis Kundu On Two Different Signal Processing Models
First ModelBasic Formulation
Different Estimation ProceduresSecond ModelCollaborators
References
Examples: Sinusoidal Signal
0
0.5
1
1.5
2
2.5
3
3.5
4
4.5
0 0.5 1 1.5 2 2.5 3
Debasis Kundu On Two Different Signal Processing Models
First ModelBasic Formulation
Different Estimation ProceduresSecond ModelCollaborators
References
Least Squares Estimators
Assuming p is known, the most natural estimators will be the leastsquares estimators and they can be obtained as follows:
n∑
t=1
(y(t)−
[p∑
k=1
Ak cos(ωkt) + Bk sin(ωkt)
])2
Debasis Kundu On Two Different Signal Processing Models
First ModelBasic Formulation
Different Estimation ProceduresSecond ModelCollaborators
References
Numerical Issues
1 It is a highly non-linear problem. The least squares surfacehas several local minima.
2 Most of the time the standard Newton-Raphson algorithmmay not converge.
3 Even if they converge, often it converges to the localminimum rather than the global minimum.
4 If p is large, it becomes a higher dimensional optimizationproblem, extremely accurate initial guesses are required forany iterative procedure to work well.
Debasis Kundu On Two Different Signal Processing Models
First ModelBasic Formulation
Different Estimation ProceduresSecond ModelCollaborators
References
Theoretical Issues
1 It can be treated as a standard non-linear regression problemas follows:
y(t) = ft(θ) + X (t)
2 Unfortunately it does not satisfy the standard sufficientconditions of Wu (1981) or Jennrich (1969) for theconsistency of the least squares estimators.
3 It can be shown that the least squares estimators areconsistent.
4 Ak and Bk have the convergence rate n−1/2, where as ωk hasthe convergence rate n−3/2
Debasis Kundu On Two Different Signal Processing Models
First ModelBasic Formulation
Different Estimation ProceduresSecond ModelCollaborators
References
Sequential Estimation Procedures
It is based on the facts that the components are orthogonal and itworks like thisFirst minimize
n∑
t=1
(y(t)− A cos(ωt)− B sin(ωt))2
with respect to A, B and ω.
Take out their effect from y(t), i .e. consider
y(t) = y(t)− A cos(ωt)− B sin(ωt)
Repeat the procedure p times.Debasis Kundu On Two Different Signal Processing Models
First ModelBasic Formulation
Different Estimation ProceduresSecond ModelCollaborators
References
Advantage
It reduces the computational burden significantly. For example if p= 25, instead of solving a 25 dimensional optimization problem, weneed to solve 25 one dimensional optimization problems. It doesnot have any problem about initial guess or convergence.
It produces the same accuracy as the least squares estimators.
Debasis Kundu On Two Different Signal Processing Models
First ModelBasic Formulation
Different Estimation ProceduresSecond ModelCollaborators
References
Super Efficient Estimators
When p = 1, the Newton-Raphson algorithm will be of thefollowing form:
ω(j+1) = ω(j) − Q ′(ω)
Q ′′(ω)
It has been suggested
ω(j+1) = ω(j) − 1
4
Q ′(ω)
Q ′′(ω)
It not only converges, it produces estimators which are better thanthe least squares estimators.
Debasis Kundu On Two Different Signal Processing Models
First ModelBasic Formulation
Different Estimation ProceduresSecond ModelCollaborators
References
Main Theoretical Results
1 Least squares estimators are consistent under mildassumptions on the errors.
2 Least squares estimators have the convergence rate N−3/2.
3 Sequential estimators have the same convergence rate as theleast squares estimators.
4 Asymptotic variances of the super efficient estimators aresmaller than the least squares estimators.
5 Periodogram estimators are consistent, but it has theconvergence rate N−1/2.
Debasis Kundu On Two Different Signal Processing Models
First ModelBasic Formulation
Different Estimation ProceduresSecond ModelCollaborators
References
Outline
1 First Model
2 Basic Formulation
3 Different Estimation Procedures
4 Second Model
5 Collaborators
6 References
Debasis Kundu On Two Different Signal Processing Models
First ModelBasic Formulation
Different Estimation ProceduresSecond ModelCollaborators
References
Chirp Signal Model
It has the following mathematical form:
Simple Chirp Model
y(t) = A cos(αt + βt2) + B sin(αt + βt2) + X (t)
General Chirp Model
y(t) =
p∑
k=1
{Ak cos(αkt + βkt
2) + Bk sin(αkt + βkt2)}+X (t)
Debasis Kundu On Two Different Signal Processing Models
First ModelBasic Formulation
Different Estimation ProceduresSecond ModelCollaborators
References
Several Applications
1 Chirps are naturally encountered in many audio signals,ranging from bird songs, music to animal vocalization andspeech.
2 Radar and Sonar systems: Chirp signals are also commonlyobserved in natural sonar systems.
3 Biology and Medicine: Chirp models have been used toanalyze EEG signals.
Debasis Kundu On Two Different Signal Processing Models
First ModelBasic Formulation
Different Estimation ProceduresSecond ModelCollaborators
References
Theoretical Consideration
1 Least squares estimators are consistent under finiteness of thefourth order moment conditions on the error random variables.
2 Least squares estimators of the amplitude has the convergencerate N−1/2.
3 Least squares estimators of the frequencies have theconvergence rate N−3/2.
4 Least squares estimators of the chirp parameters have theconvergence rate N−5/2.
5 Least squares estimators are asymptotically normallydistributed.
Debasis Kundu On Two Different Signal Processing Models
First ModelBasic Formulation
Different Estimation ProceduresSecond ModelCollaborators
References
One Number Theory Result:
If (θ1, θ2) ∈ (0, π)× (0, π), then except for countable number ofpoints the following results are true:
limN→∞
1
N
N∑
n=1
cos(θ1n + θ2n2) = 0
limN→∞
1
Nt+1
N∑
n=1
nt cos2(θ1n + θ2n2) =
1
2(t + 1).
limN→∞
1
Nt+1
N∑
n=1
nt sin(θ1n + θ2n2) cos(θ1n + θ2n
2) = 0.
Debasis Kundu On Two Different Signal Processing Models
First ModelBasic Formulation
Different Estimation ProceduresSecond ModelCollaborators
References
Sequential Estimator
Based on the above number theory results it can be shown thatthe sequential estimators are consistent.
Debasis Kundu On Two Different Signal Processing Models
First ModelBasic Formulation
Different Estimation ProceduresSecond ModelCollaborators
References
One Number Theory Conjecture
If θ1, θ2, θ′
1, θ′
2 ∈ (0, π), then except for countable number of pointsthe following results are true:
limN→∞
1√NNt
N∑
n=1
nt cos(θ1n + θ2n2) sin(θ′1n + θ′2n
2) = 0
Debasis Kundu On Two Different Signal Processing Models
First ModelBasic Formulation
Different Estimation ProceduresSecond ModelCollaborators
References
Asymptotic Distribution of the Sequential
Estimators
Based on the above conjecture, it can be shown that the sequentialestimators and the least squares estimators are equivalent.
Debasis Kundu On Two Different Signal Processing Models
First ModelBasic Formulation
Different Estimation ProceduresSecond ModelCollaborators
References
Two dimensional Chirp Signals
y(m, n) = A cos(α1m + β1m2 + α2n + β2n
2) +
B sin(α1m + β1m2 + α2n + β2n
2) + X (m, n)
Results can be extended to two dimensional chirp signals models.
Debasis Kundu On Two Different Signal Processing Models
First ModelBasic Formulation
Different Estimation ProceduresSecond ModelCollaborators
References
Outline
1 First Model
2 Basic Formulation
3 Different Estimation Procedures
4 Second Model
5 Collaborators
6 References
Debasis Kundu On Two Different Signal Processing Models
First ModelBasic Formulation
Different Estimation ProceduresSecond ModelCollaborators
References
Collaborators
Z.D. Bai
Li Bai
Swagata Nandi
Ananya Lahiri
Amit Mitra
Anurag Prasad
Debasis Kundu On Two Different Signal Processing Models
First ModelBasic Formulation
Different Estimation ProceduresSecond ModelCollaborators
References
Outline
1 First Model
2 Basic Formulation
3 Different Estimation Procedures
4 Second Model
5 Collaborators
6 References
Debasis Kundu On Two Different Signal Processing Models
First ModelBasic Formulation
Different Estimation ProceduresSecond ModelCollaborators
References
References
Kundu, D. and Nandi, S. (2008), “Parameter estimation onchirp signals in presence of stationary noise”, Statistica Sinica.
Nandi, S. and Kundu, D. (2004), “Asymptotic properties ofthe least squares estimators of the parameters of the chirpsignals’, Annals of the Institute Statistical Mathematics.
Kundu, D., Bai. Z.D., Nandi, S. and Bai, L. (2011), “Superefficient frequency estimation”, Journal of Statistical Planningand Inference.
Debasis Kundu On Two Different Signal Processing Models
First ModelBasic Formulation
Different Estimation ProceduresSecond ModelCollaborators
References
References
Ananya Lahiri, D. Kundu , and A. Mitra (2012) “Efficientalgorithm for estimating the parameters of chirp signal”Journal of Multivariate Analysis.
Ananya Lahiri, D. Kundu , and A. Mitra “Estimating theparameters of multiple chirp signals” Journal of Multivariate
Analysis.Ananya Lahiri, D. Kundu , Amit Mitra (2014), “On leastabsolute deviation estimator of one dimensional chirp model”,Statistics.
Debasis Kundu On Two Different Signal Processing Models
First ModelBasic Formulation
Different Estimation ProceduresSecond ModelCollaborators
References
References
B.G. Quinn and E.J. Hannan (2001), The estimation and
tracking of frequency, Cambridge University Press.
D. Kundu and S. Nandi (2012), Statistical signal processing:Frequency Estimation.
Debasis Kundu On Two Different Signal Processing Models
First ModelBasic Formulation
Different Estimation ProceduresSecond ModelCollaborators
References
THANK YOU
Debasis Kundu On Two Different Signal Processing Models