Download - 1-Adaptive Signal Processing (1)
-
8/13/2019 1-Adaptive Signal Processing (1)
1/37
AGC
DSP
Professor A G Constantinides 1
Adaptive Signal Processing Problem: Equalise through a FIR filter the distorting
effect of a communication channel that may bechanging with time.
If the channel were fixedthen a possible solutioncould be based on the Wiener filterapproach
We need to know in such case the correlation matrixof the transmitted signal and the cross correlation
vector between the input and desired response. When the the filter is operating in an unknown
environment these required quantities need to be
found from the accumulated data.
-
8/13/2019 1-Adaptive Signal Processing (1)
2/37
AGC
DSP
Professor A G Constantinides 2
Adaptive Signal Processing The problem is particularly acute when not
only the environment is changingbut also the
data involved are non-stationary In such cases we need temporally to follow
the behaviour of the signals, and adaptthecorrelation parameters as the environment ischanging.
This would essentially produce a temporallyadaptive filter.
-
8/13/2019 1-Adaptive Signal Processing (1)
3/37
AGC
DSP
Professor A G Constantinides 3
Adaptive Signal ProcessingA possible framework is:
][nd][ nd]}[{ nx
][ne
w:Filter
Adaptive
Algorithm
-
8/13/2019 1-Adaptive Signal Processing (1)
4/37
AGC
DSP
Professor A G Constantinides 4
Adaptive Signal Processing Applications are many
Digital Communications
Channel Equalisation Adaptive noise cancellation
Adaptive echo cancellation
System identification
Smart antenna systems
Blind system equalisation
And many, many others
-
8/13/2019 1-Adaptive Signal Processing (1)
5/37
AGC
DSP
Professor A G Constantinides 5
Applications
-
8/13/2019 1-Adaptive Signal Processing (1)
6/37
AGC
DSP
Professor A G Constantinides 6
Adaptive Signal Processing Echo Cancellers in Local Loops
-
+
-
+
Rx1
Rx2
Tx1 Rx2
Echo canceller Echo canceller
Adaptive AlgorithmAdaptive Algorithm
HybridHybrid
Local Loop
-
8/13/2019 1-Adaptive Signal Processing (1)
7/37
-
8/13/2019 1-Adaptive Signal Processing (1)
8/37
AGC
DSP
Professor A G Constantinides 8
Adaptive Signal Processing System Identification
Unknown System
Signal
-
+
FIR filter
Adaptive Algorithm
-
8/13/2019 1-Adaptive Signal Processing (1)
9/37
AGC
DSP
Professor A G Constantinides 9
Adaptive Signal Processing System Equalisation
Unknown System
Signal
-
+
FIR filter
Adaptive Algorithm
Delay
-
8/13/2019 1-Adaptive Signal Processing (1)
10/37
AGC
DSP
Professor A G Constantinides 10
Adaptive Signal ProcessingAdaptive Predictors
Signal
-
+
FIR filter
Adaptive Algorithm
Delay
-
8/13/2019 1-Adaptive Signal Processing (1)
11/37
AGC
DSP
Professor A G Constantinides 11
Adaptive Signal ProcessingAdaptive Arrays
Linear Combiner
Interference
-
8/13/2019 1-Adaptive Signal Processing (1)
12/37
AGC
DSP
Professor A G Constantinides 12
Adaptive Signal Processing Basic principles:
1) Form an objective function (performance
criterion) 2) Find gradient of objective function with
respect to FIR filter weights
3) There are several different approaches
that can be used at this point 3) Form a differential/difference equation
from the gradient.
-
8/13/2019 1-Adaptive Signal Processing (1)
13/37
AGC
DSP
Professor A G Constantinides 13
Adaptive Signal Processing Let the desired signal be
The input signal
The output Now form the vectors
So that
][nd][nx
][ny
Tmnxnxnxn ]1[.]1[][][ x
Tmhhh ]1[.]1[]0[ h
hx Tnny ][][
-
8/13/2019 1-Adaptive Signal Processing (1)
14/37
AGC
DSP
Professor A G Constantinides 14
Adaptive Signal Processing The form the objective function
where
}][][{)( 2nyndEJ w
Rhhphhpw TTT
dJ 2)(
}][][{ TnnE xxR
]}[][{ ndnE xp
-
8/13/2019 1-Adaptive Signal Processing (1)
15/37
AGC
DSP
Professor A G Constantinides 15
Adaptive Signal Processing We wish to minimise this function at the
instant n
Using Steepest Descentwe write
But
][
])[(
2
1][]1[
n
nJnn
h
hhh
Rhph
h22
)(
J
-
8/13/2019 1-Adaptive Signal Processing (1)
16/37
AGC
DSP
Professor A G Constantinides 16
Adaptive Signal Processing So that the weights update equation
Since the objective function is quadratic thisexpression will converge in m steps
The equation is not practical If we knew and a priori we could find
the required solution (Wiener) as
])[(][]1[ nnn Rhphh
pR
pRh 1opt
-
8/13/2019 1-Adaptive Signal Processing (1)
17/37
AGC
DSP
Professor A G Constantinides 17
Adaptive Signal Processing However these matrices are not known
Approximate expressions are obtained by
ignoring the expectations in the earliercomplete forms
This is very crude. However, because theupdate equation accumulates such quantities,progressive we expect the crude form toimprove
Tnnn ][][][ xxR ][][][ ndnn xp
-
8/13/2019 1-Adaptive Signal Processing (1)
18/37
AGC
DSP
Professor A G Constantinides 18
The LMS Algorithm Thus we have
Where the error is
And hence can write
This is sometimes called the stochasticgradientdescent
])[][][]([][]1[ nnndnnn Thxxhh
])[][(])[][][(][ nyndnnndne T hx
][][][]1[ nennn xhh
-
8/13/2019 1-Adaptive Signal Processing (1)
19/37
AGC
DSP
Professor A G Constantinides 19
ConvergenceThe parameter is the step size, and it
should be selected carefully
If too small it takes too long toconverge, if too large it can lead toinstability
Write the autocorrelation matrix in theeigen factorisation form
QQR T
-
8/13/2019 1-Adaptive Signal Processing (1)
20/37
AGC
DSP
Professor A G Constantinides 20
Convergence Where is orthogonal and is
diagonal containing the eigenvalues
The error in the weights with respect totheir optimal values is given by (usingthe Wiener solution for
We obtain
Q
])[(][]1[ nnn optoptopt RhRhhhhh
p
][][]1[ nnn hhh Reee
-
8/13/2019 1-Adaptive Signal Processing (1)
21/37
AGC
DSP
Professor A G Constantinides 21
Convergence Or equivalently
I.e.
Thus we have
Form a new variable
][)1(]1[ nn hh eQQe T
][)(
][)1(]1[
n
nn
h
hh
eQQQQ
eQQQQe
T
T
][)1(]1[ nn hh QeQe
][][ nn hQev
-
8/13/2019 1-Adaptive Signal Processing (1)
22/37
AGC
DSP
Professor A G Constantinides 22
Convergence So that
Thus each element of this new variable isdependent on the previous value of it via ascaling constant
The equation will therefore have an
exponential form in the time domain, and thelargest coefficient in the right hand side willdominate
][)1(]1[ nn vv
-
8/13/2019 1-Adaptive Signal Processing (1)
23/37
AGC
DSP
Professor A G Constantinides 23
Convergence We require that
Or
In practice we take a much smallervalue than this
11 max
max
20
-
8/13/2019 1-Adaptive Signal Processing (1)
24/37
AGC
DSP
Professor A G Constantinides 24
Estimates Then it can be seen that as the
weight update equation yields
And on taking expectations of both sides of it
we have
Or
n
]}[{]}1[{ nEnE hh
])}[][][]([{]}[{]}1[{ nnndnEnEnE
T
hxxhh
])}[][][][][{(0 nnnndnE Thxxx
-
8/13/2019 1-Adaptive Signal Processing (1)
25/37
AGC
DSP
Professor A G Constantinides 25
Limiting forms This indicates that the solution
ultimately tends to the Wiener form
I.e. the estimate is unbiased
-
8/13/2019 1-Adaptive Signal Processing (1)
26/37
AGC
DSP
Professor A G Constantinides 26
Misadjustment The excess mean square error in the
objective function due to gradient noise
Assume uncorrelatedness set
Where is the variance of desired
response and is zero when uncorrelated. Then misadjustment is defined as
optT
dJ hp 2
min 2d
opth
minmin /))(( JJJJ LMSXS
-
8/13/2019 1-Adaptive Signal Processing (1)
27/37
AGC
DSP
Professor A G Constantinides 27
Misadjustment It can be shown that the misadjustment
is given by
m
i i
iXS JJ
1min
1/
-
8/13/2019 1-Adaptive Signal Processing (1)
28/37
AGC
DSP
Professor A G Constantinides 28
Normalised LMS To make the step size respond to the
signal needs
In this case
And misadjustment is proportional tothe step size.
][][][1
2][]1[
2 nen
nnn x
xhh
10
-
8/13/2019 1-Adaptive Signal Processing (1)
29/37
AGC
DSP
Professor A G Constantinides 29
Transform based LMS
][nd][ nd]}[{ nx
][new:Filter
Adaptive
AlgorithmTransform
Inverse Transform
-
8/13/2019 1-Adaptive Signal Processing (1)
30/37
AGC
DSP
Professor A G Constantinides 30
Least Squares Adaptive with
We have the Least Squares solution
However, this is computationally veryintensive to implement.
Alternative forms make use of recursiveestimates of the matrices involved.
n
i
Tiin1
][][][ xxR
n
indnn
1][][][ xp
][][][ 1 nnn pRh
-
8/13/2019 1-Adaptive Signal Processing (1)
31/37
AGC
DSP
Professor A G Constantinides 31
Recursive Least Squares Firstly we note that
We now use the Inversion Lemma (or the
Sherman-Morrison formula) Let
][][]1[][ ndnnn xpp
Tnnnn ][][]1[][ xxRR
-
8/13/2019 1-Adaptive Signal Processing (1)
32/37
AGC
DSP
Professor A G Constantinides 32
Recursive Least Squares (RLS) Let
Then
The quantity is known as the Kalmangain
][]1[][1
][]1[][ 1
1
nnn
nnn T xRx
xRk
1][][ nn RP
]1[][][]1[][ nnnnn T PxkRP
][nk
-
8/13/2019 1-Adaptive Signal Processing (1)
33/37
AGC
DSP
Professor A G Constantinides 33
Recursive Least Squares Now use in the computation of
the filter weights
From the earlier expression for updates wehave
And hence
][][][ nnn xPk
])[][]1[]([][][][ ndnnnnnn xpPpPh
][nP
]1[]1[][][]1[]1[]1[][ nnnnnnnn T
pPxkpPpP
])1[][][]([]1[][ nnndnnn T
hxkhh
-
8/13/2019 1-Adaptive Signal Processing (1)
34/37
AGC
DSP
Professor A G Constantinides 34
Kalman Filters Kalman filter is a sequential estimation
problem normally derived from
The Bayes approach The Innovations approach
Essentially they lead to the same equations
as RLS, but underlying assumptions aredifferent
-
8/13/2019 1-Adaptive Signal Processing (1)
35/37
AGC
DSP
Professor A G Constantinides 35
Kalman Filters The problem is normally stated as:
Given a sequence of noisy observations to
estimate the sequence of state vectors of a linearsystem driven by noise.
Standard formulation
][][]1[ nnn wAxx
][][][][ nnnn xCy
-
8/13/2019 1-Adaptive Signal Processing (1)
36/37
AGC
DSP
Professor A G Constantinides 36
Kalman Filters Kalman filters may be seen as RLS with the
following correspondenceSate space RLS
Sate-Update matrix
Sate-noise variance
Observation matrix
Observations
State estimate
][nA
][nx
Tn][x][nC
][ny
][nh
}][][{][ T
nnEn wwQ
I0
][nd
-
8/13/2019 1-Adaptive Signal Processing (1)
37/37
AGC
DSP
Cholesky Factorisation In situations where storage and to some
extend computational demand is at a
premium one can use the Choleskyfactorisation tecchnique for a positive definitematrix
Express , where is lower
triangular
There are many techniques for determiningthe factorisation
TLLR L