1-adaptive signal processing
TRANSCRIPT
-
8/3/2019 1-Adaptive Signal Processing
1/37
AGC
DSP
AGC
DSP
Professor A G Constantinides 1
Adaptive Signal Processing Problem: Equalise through a FIR filter the distorting
effect of a communication channel that may be
changing with time. If the channel were fixed then a possible solution
could be based on the Wiener filter approach
We need to know in such case the correlation matrixof the transmitted signal and the cross correlation
vector between the input and desired response. When the the filter is operating in an unknown
environment these required quantities need to be
found from the accumulated data.
-
8/3/2019 1-Adaptive Signal Processing
2/37
AGC
DSP
AGC
DSP
Professor A G Constantinides 2
Adaptive Signal Processing The problem is particularly acute when not
only the environment is changing but also the
data involved are non-stationary
In such cases we need temporally to followthe behaviour of the signals, and adaptthecorrelation parameters as the environment ischanging.
This would essentially produce a temporallyadaptive filter.
-
8/3/2019 1-Adaptive Signal Processing
3/37
AGC
DSP
AGC
DSP
Professor A G Constantinides 3
Adaptive Signal ProcessingA possible framework is:
][nd][ nd]}[{ nx
][ne
w:Filter
Adaptive
Algorithm
-
8/3/2019 1-Adaptive Signal Processing
4/37
AGC
DSP
AGC
DSP
Professor A G Constantinides 4
Adaptive Signal Processing Applications are many
Digital Communications
Channel Equalisation
Adaptive noise cancellation
Adaptive echo cancellation
System identification
Smart antenna systems
Blind system equalisation
And many, many others
-
8/3/2019 1-Adaptive Signal Processing
5/37
AGC
DSP
AGC
DSP
Professor A G Constantinides 5
Applications
-
8/3/2019 1-Adaptive Signal Processing
6/37
AGC
DSP
AGC
DSP
Professor A G Constantinides 6
Adaptive Signal Processing Echo Cancellers in Local Loops
-
+
-
+
Rx1
Rx2
Tx1 Rx2
Echo canceller Echo canceller
Adaptive Algorithm Adaptive Algorithm
Hybrid Hybrid
Local Loop
-
8/3/2019 1-Adaptive Signal Processing
7/37
-
8/3/2019 1-Adaptive Signal Processing
8/37
AGC
DSP
AGC
DSP
Professor A G Constantinides 8
Adaptive Signal Processing System Identification
Unknown System
Signal
-
+
FIR filter
Adaptive Algorithm
-
8/3/2019 1-Adaptive Signal Processing
9/37
AGC
DSP
AGC
DSP
Professor A G Constantinides 9
Adaptive Signal Processing System Equalisation
Unknown System
Signal
-
+
FIR filter
Adaptive Algorithm
Delay
-
8/3/2019 1-Adaptive Signal Processing
10/37
AGC
DSP
AGC
DSP
Professor A G Constantinides 10
Adaptive Signal ProcessingAdaptive Predictors
Signal
-
+
FIR filter
Adaptive Algorithm
Delay
-
8/3/2019 1-Adaptive Signal Processing
11/37
AGC
DSP
AGC
DSP
Professor A G Constantinides 11
Adaptive Signal ProcessingAdaptive Arrays
Linear Combiner
Interference
-
8/3/2019 1-Adaptive Signal Processing
12/37
AGC
DSP
AGC
DSP
Professor A G Constantinides 12
Adaptive Signal Processing Basic principles:
1) Form an objective function (performance
criterion) 2) Find gradient of objective function with
respect to FIR filter weights
3) There are several different approaches
that can be used at this point 3) Form a differential/difference equation
from the gradient.
-
8/3/2019 1-Adaptive Signal Processing
13/37
AGC
DSP
AGC
DSP
Professor A G Constantinides 13
Adaptive Signal Processing Let the desired signal be
The input signal
The output Now form the vectors
So that
][nd][nx
][ny
? AT
mnxnxnxn ]1[.]1[][][ !x
? ATmhhh ]1[.]1[]0[ !h
hxTnny ][][ !
-
8/3/2019 1-Adaptive Signal Processing
14/37
AGC
DSP
AGC
DSP
Professor A G Constantinides 14
Adaptive Signal Processing The form the objective function
where
? A }][][{)(2
nyndEJ !w
RhhphhpwTTT
dJ !2)( W
}][][{ TnnE xxR!
]}[][{ ndnE xp !
-
8/3/2019 1-Adaptive Signal Processing
15/37
AGC
DSP
AGC
DSP
Professor A G Constantinides 15
Adaptive Signal Processing We wish to minimise this function at the
instantn
Using Steepest Descentwe write
But][
])[(
2
1][]1[
n
nJnn
h
hhh
x
x! Q
Rhph
h22
)(!
x
xJ
-
8/3/2019 1-Adaptive Signal Processing
16/37
AGC
DSP
AGC
DSP
Professor A G Constantinides 16
Adaptive Signal Processing So that theweights update equation
Since the objective function is quadratic thisexpression will converge in m steps
The equation is not practical If we knew and a priori we could find
the required solution (Wiener) as
])[(][]1[ nnn Rhphh ! Q
pR
pRh1!
opt
-
8/3/2019 1-Adaptive Signal Processing
17/37
AGC
DSP
AGC
DSP
Professor A G Constantinides 17
Adaptive Signal Processing However these matrices are not known
Approximate expressions are obtained by
ignoring the expectations in the earliercomplete forms
This is very crude. However, because theupdate equation accumulates such quantities,progressive we expect the crude form toimprove
Tnnn ][][][ xxR ! ][][][ ndnn xp !
-
8/3/2019 1-Adaptive Signal Processing
18/37
AGC
DSP
AGC
DSP
Professor A G Constantinides 18
The LMS Algorithm Thus we have
Where the error is
And hence can write
This is sometimes called the stochasticgradientdescent
])[][][]([][]1[ nnndnnn Thxxhh ! Q
])[][(])[][][(][ nyndnnndne T !! hx
][][][]1[ nennn xhh Q!
-
8/3/2019 1-Adaptive Signal Processing
19/37
AGC
DSP
AGC
DSP
Professor A G Constantinides 19
ConvergenceThe parameter is the step size, and it
should be selected carefully
If too small it takes too long toconverge, if too large it can lead toinstability
Write the autocorrelation matrix in theeigen factorisation form
Q
QQRT!
-
8/3/2019 1-Adaptive Signal Processing
20/37
AGC
DSP
AGC
DSP
Professor A G Constantinides 20
Convergence Where is orthogonal and is
diagonal containing the eigenvalues
The error in the weights with respect totheir optimal values is given by (usingthe Wiener solution for
We obtain
Q
])[(][]1[ nnnoptoptopt RhRhhhhh ! Q
p
][][]1[ nnnhhh
Reee Q!
-
8/3/2019 1-Adaptive Signal Processing
21/37
AGC
DSP
AGC
DSP
Professor A G Constantinides 21
Convergence Or equivalently
I.e.
Thus we have
Form a new variable
][)1(]1[ nnhheQQe
TQ!
][)(
][)1(]1[
n
nn
h
hh
eQQQQ
eQQQQe
T
T
Q
Q
!
!
][)1(]1[ nnhh
QeQe Q!
][][ nnh
Qev !
-
8/3/2019 1-Adaptive Signal Processing
22/37
AGC
DSP
AGC
DSP
Professor A G Constantinides 22
Convergence So that
Thus each element of this new variable isdependent on the previous value of it via ascaling constant
The equation will therefore have an
exponential form in the time domain, and thelargest coefficient in the right hand side willdominate
][)1(]1[ nn vv Q!
-
8/3/2019 1-Adaptive Signal Processing
23/37
AGC
DSP
AGC
DSP
Professor A G Constantinides 23
Convergence We require that
Or
In practice we take a much smallervalue than this
11 max
QP
max
20
PQ
-
8/3/2019 1-Adaptive Signal Processing
24/37
AGC
DSP
AGC
DSP
Professor A G Constantinides 24
Estimates Then it can be seen that as the
weight update equation yields
And on taking expectations of both sides of itwe have
Or
gpn
]}[{]}1[{ nEnE hh !
])}[][][]([{]}[{]}1[{ nnndnEnEnET
hxxhh ! Q
])}[][][][][{(0 nnnndnE Thxxx ! Q
-
8/3/2019 1-Adaptive Signal Processing
25/37
AGC
DSP
AGC
DSP
Professor A G Constantinides 25
Limiting forms This indicates that the solution
ultimately tends to the Wiener form
I.e. the estimate is unbiased
-
8/3/2019 1-Adaptive Signal Processing
26/37
AGC
DSP
AGC
DSP
Professor A G Constantinides 26
Misadjustment The excess mean square error in the
objective function due to gradient noise
Assume uncorrelatedness set
Where is the variance of desired
response and is zero when uncorrelated. Then misadjustment is defined as
optT
dJ hp!2
min W2dW
opth
minmin /))(( JJJJ LMSXS g!
-
8/3/2019 1-Adaptive Signal Processing
27/37
AGC
DSP
AGC
DSP
Professor A G Constantinides 27
Misadjustment It can be shown that the misadjustment
is given by
!!
m
ii
i
XS JJ1
min1
/QP
QP
-
8/3/2019 1-Adaptive Signal Processing
28/37
AGC
DSP
AGC
DSP
Professor A G Constantinides 28
Normalised LMS To make the step size respond to the
signal needs
In this case
And misadjustment is proportional tothe step size.
][][][1
2][]1[
2nen
nnn x
xhh
!
Q
10 Q
-
8/3/2019 1-Adaptive Signal Processing
29/37
AGC
DSP
AGC
DSP
Professor A G Constantinides 29
Transform based LMS
][nd][ nd]}[{ nx
][new:Filter
Adaptive
AlgorithmTransform
Inverse Transform
-
8/3/2019 1-Adaptive Signal Processing
30/37
AGC
DSP
AGC
DSP
Professor A G Constantinides 30
Least Squares Adaptive with
We have the Least Squares solution
However, this is computationally veryintensive to implement.
Alternative forms make use of recursiveestimates of the matrices involved.
!!
n
i
Tiin
1
][][][ xxR
!!
n
i
ndnn1
][][][ xp
][][][ 1 nnn pRh !
-
8/3/2019 1-Adaptive Signal Processing
31/37
AGC
DSP
AGC
DSP
Professor A G Constantinides 31
Recursive Least Squares Firstly we note that
We now use the Inversion Lemma (or the
Sherman-Morrison formula) Let
][][]1[][ ndnnn xpp !
Tnnnn ][][]1[][ xxRR !
-
8/3/2019 1-Adaptive Signal Processing
32/37
AGC
DSP
AGC
DSP
Professor A G Constantinides 32
Recursive Least Squares (RLS) Let
Then
The quantity is known as the Kalmangain
][]1[][1][]1[][ 1
1
nnnnnn
TxRx
xRk
!
1][][ ! nn RP
]1[][][]1[][ ! nnnnn T PxkRP
][nk
-
8/3/2019 1-Adaptive Signal Processing
33/37
AGC
DSP
AGC
DSP
Professor A G Constantinides 33
Recursive Least Squares Now use in the computation of
the filter weights
From the earlier expression for updates wehave
And hence
][][][ nnn xPk !
])[][]1[]([][][][ ndnnnnnn xpPpPh !!][nP
]1[]1[][][]1[]1[]1[][ ! nnnnnnnnT
pPxkpPpP
])1[][][]([]1[][ ! nnndnnnThxkhh
-
8/3/2019 1-Adaptive Signal Processing
34/37
AGC
DSP
AGC
DSP
Professor A G Constantinides 34
Kalman Filters Kalman filter is a sequential estimation
problem normally derived from
The Bayes approach
The Innovations approach
Essentially they lead to the same equations
as RLS, but underlying assumptions aredifferent
-
8/3/2019 1-Adaptive Signal Processing
35/37
AGC
DSP
AGC
DSP
Professor A G Constantinides 35
Kalman Filters The problem is normally stated as:
Given a sequence of noisy observations to
estimate the sequence of state vectors of a linearsystem driven by noise.
Standard formulation
][][]1[ nnn wAxx !
][][][][ nnnn xCy !
-
8/3/2019 1-Adaptive Signal Processing
36/37
-
8/3/2019 1-Adaptive Signal Processing
37/37
AGC
DSP
AGC
DSP
Professor A G Constantinides 37
Cholesky Factorisation In situations where storage and to some
extend computational demand is at a
premium one can use the Choleskyfactorisation tecchnique for a positive definitematrix
Express , where is lower
triangular
There are many techniques for determiningthe factorisation
TLLR! L