linear predictive coding - ut · pdf filelinear predictive coding ... interpretation of the...
TRANSCRIPT
Linear predictive coding
This method combines linear processing with scalar quantization.
The main idea of the method is to predict the value of the current
sample by a linear combination of previous already reconstructed
samples and then to quantize the difference between the actual
value and the predicted value.
Linear prediction coefficients are weighting coefficients used in
linear combination.
A simple predictive quantizer or differential pulse-coded
modulator is shown in Fig. 5.1.
If the predictor is simply the last sample and the quantizer has
only one bit, the system becomes a delta-modulator. It is shown in
Fig. 5.2.
Differential pulse-coded modulator
)( snTxQUANTIZER
DEQUANTIZER
LINEAR
PREDICTOR
)(ˆ snTx
)( snTe)( snTq
)( snTd
)( sR nTx
Delta-modulator
COMPARATOR
Staircase
function
former
)( snTx )( snTx )(ˆ snTx
)( snTx )(ˆ snTx
1
0
Linear predictive coding
The main feature of the quantizers shown in Figs 5.1, 5.2 is
that they exploit not all advantages of predictive coding.
•Prediction coefficients used in these schemes are not optimal
•Prediction is based on past reconstructed samples and not
true samples
•Usually coefficients of prediction are chosen by using some
empirical rules and are not transmitted
For example quantizer in Fig.5.1 instead of actual value of error
e uses reconstructed values d and instead of true sample
values their estimates obtained via x
Rx .d
Linear predictive coding
The most advanced quantizer of linear predictive type
represents a basis of the so-called Code Excited Linear
Predictive (CELP) coder.
•It uses the optimal set of coefficients or in other words
linear prediction coefficients of this quantizer are
determined by minimizing the MSE between the current
sample and its predicted value.
•It is based on the original past samples
•Using the true samples for prediction requires the
“looking-ahead” procedure in the coder.
•The predictor coefficients are transmitted
Linear predictive coding
Assume that quantizer coefficients are optimized for each sample
and that the original past samples are used for prediction. Let
be a sequence of samples at the quantizer input.
Then each sample is predicted by the previous samples
according to the formula
),...2(),( ss TxTx
)( snTx
,)()(ˆ1
m
k
ssks kTnTxanTx
where
)(ˆ snTx is the predicted value,
ka are prediction
coefficients,
m denotes the order of prediction. The prediction
error is
).(ˆ)()( sss nTxnTxnTe
(5.1)
Linear predictive coding
Prediction coefficients are determined by minimizing the sum
of squared errors over a given interval
1
0
)(2n
nn
snTeE (5.2)
Inserting (5.1) into (5.2) we obtain
1
0
2
1 ))(...)()((n
nn
ssmsss mTnTxaTnTxanTxE
1
0
1
01
2)()(2)(
n
nn
m
j
n
nn
sssjs jTnTxnTxanTx
m
j
n
nn
ssss
m
k
kj kTnTxjTnTxaa1 1
1
0
).()( (5.3)
Linear predictive coding
Differentiating (5.3) over ,ka
mk ,...,2,1 yields
m
j
n
nn
ssssj
n
nn
sssk jTnTxkTnTxakTnTxnTxaE1
1
0
1
0
0)()()()(/
Thus we obtain a system of linear equations with m m
unknown quantities maaa ,...,, 21
,1
m
j
okjkj cca mk ,...,2,1
where .)()(1
0
n
nn
sssskjjk kTnTxjTnTxcc
(5.4)
(5.5)
The system (5.4) is called the Yule-Walker equations.
Linear predictive coding
If
maaa ,...,, 21 are solutions of (5.4) then we can find the
minimal achievable prediction error. Insert (5.5) into (5.3).
We obtain that
.21 1 1
000
m
k
m
k
m
j
jkjkkk caacacE (5.6)
Using (5.3) we reduce (5.6) to
m
k
kk cacE1
000
Interpretation of the Yule-Walker equations like a
digital filter
Eq. (5.1) describes the th order predictor with transfer
function equal to
m
)(
)(ˆ)(
zX
zXzP
m
k
k
k za1
.
z-transform for the prediction error is
m
k
k
k zzXazXzE1
.)()()(
The prediction error is an output signal of the discrete-time
filter with transfer function
m
k
k
k zazX
zEzA
1
.1)(
)()(
The problem of finding the optimal set of prediction
coefficients = problem of constructing th order FIR filter. m
Interpretation of the Yule-Walker equations like a
digital filter
Another name of the linear prediction (5.1) is the
autoregressive model of signal It is assumed that the
signal can be obtained as the output of the
autoregressive filter with transfer function
).( snTx)( snTx
,
1
1)(
1
km
k
k za
zH
that is can be obtained as the output of the filter which is
inverse with respect to the prediction filter. This filter is a
discrete-time IIR filter.
Methods of finding coefficients , 0,1,..., , 1,2,...,ijc i m j m
In order to solve the Yule-Walker eq. (5.4) it is
necessary first to evaluate values , 0,1,..., , 1,2,...,ijc i m j m
There are two approaches to estimating these values:
The autocorrelation method and the covariance method.
The complexity of
solving (5.4) is
proportional to 2m
The complexity of
solving (5.4) is
proportional to 3m
Autocorrelation method
The values are computed as ijc
.)()(1
0
i
in
ssssjiij jTnTxiTnTxcc
We set
10 , ii and
0)( snTx if ,,0 Nnn
where N is called the interval of analysis.
(5.7)
In this case we can simplify (5.7)
).()()()(1
0
1
0
N
n
s
jiN
n
ssssssij TjinTxnTxjTnTxiTnTxc
Normalized by N they coincide with estimates of entries of
).()(/1/)(ˆ1
0
ss
jiN
n
sij TjinTxnTxNNcjiR
(5.8)
covariance matrix for )( snTx
Autocorrelation method
Autocorrelation method
The Yule-Walker equations for autocorrelation method have
the form
m
i
i mjjRjiRa1
.,...,2,1),(ˆ)(ˆ (5.9)
Eq.(5.9) can be given by matrix equation
,bRa
where ),,...,,( 21 maaaa
)),(ˆ),...,2(ˆ),1(ˆ( mRRRb
.
)0(ˆ)...2(ˆ)1(ˆ
.......................................
)2(ˆ )...0(ˆ )1(ˆ
)1(ˆ )...1(ˆ )0(ˆ
RmRmR
mRRR
mRRR
R
Autocorrelation method
It is said that (5.9) relates the parameters of the autoregressive
model of th order with the autocorrelation sequence.
Matrix of the autocorrelation method has two important
properties.
•It is symmetric, that is
•It has Toeplitz property, that is
m
R
).(ˆ),(ˆ jiRjiR
),(ˆ),(ˆ ijRjiR
The Toeplitz property of R makes it possible to reduce the
computational complexity of solving (5.4). The fast
Levinson-Durbin recursive algorithm requires only 2m
operations.
Covariance method
We choose 0 0i 1 1i N and and signal is
)( snTx
not constrained in time. In this case we have
.)()(1
0
N
n
ssssij jTnTxiTnTxc
Set ink
(5.10)
then (5.10) can be rewritten as
1
.,...0,,...,1),)(()(iN
ik
sssij mjmiTjikTxkTxc
(5.11) resembles (5.8) but it has other range of definition for .k
(5.11)
•It uses signal values out of range
•The method leads to the cross-correlation function between
two similar but not exactly the same finite segments of
,10 Nk
).( skTx
Covariance method
1
0
).)(())((/1/),(ˆN
n
ssij TjnxTinxNNcjiR
The Yule-Walker equations for the covariation method
are
m
i
i mjjRjiRa1
.,...,2,1),,0(ˆ),(ˆ (5.12)
Eq. (5.12) can be given by the matrix equation
,cPa
where ),,...,,( 21 maaaa )),,0(ˆ),...,2,0(ˆ),1,0(ˆ( mRRRc
.
.
),(ˆ)...2,(ˆ)1,(ˆ
...............................
),2(ˆ)...2,2(ˆ)1,2(ˆ
),1(ˆ)...2,1(ˆ)1,1(ˆ
mmRmRmR
mRRR
mRRR
P
Covariance method
Unlike the matrix of autocorrelation method the
matrix is symmetric ( ) but it is not
Toeplitz .
R
P
),(ˆ),(ˆ ijRjiR
Since computational complexity of solving an arbitrary
system of linear equations of order is equal to
then in this case to solve (5.12) it is necessary
operations.
m3m
3m
Algorithms for the solution of the Yule-Walker
equations
The computational complexity of solving the Yule-
Walker equations depends on the method of evaluating
values .ijc
Assume that ijc are found by the autocorrelation method.
In this case the Yule-Walker equations has the form (5.9) and
the matrix is symmetric and the Toeplitz matrix. RThese properties make it possible to find the solution of
(5.9) by fast methods requiring operations. 2m
There are a few methods of this type: the Levinson-
Durbin algorithm, the Euclidean algorithm and the
Berlekamp-Massey algorithm.
The Levinson-Durbin algorithm
It was suggested by Levinson in 1948 and then was
improved by Durbin in 1960. Notice that this algorithm
works efficiently if matrix of coefficients is
simultaneously symmetric and Toeplitz. The Berlekamp-
Massey and the Euclidean algorithms do not require the
matrix of coefficients to be symmetric.
R
We sequentially solve equations (5.9) of order .,...,1 ml
Let denote the solution for the
system of the th order. Given we find the solution
for the th order.
),...,,( )()(
2
)(
1
)( l
l
lll aaaa
l )(la
)1( l At each step of the algorithm we
evaluate the prediction error of the th order system.
lE l
The Levinson-Durbin algorithm
Initialization: .0),0(ˆ,0 )0(
0 aREl
Recurrent procedure:
For ml ,...,1 compute
,/))(ˆ)(ˆ(1
1
1
)1()(
l
i
l
l
i
l
l EilRalRa
,)1()()1()(
l
jl
l
l
l
j
l
j aaaa ,11 lj
).)(1( 2)(
1
l
lll aEE
When ml we obtain the solution
.,),...,,( )(
21 m
m
m EEaaa aa
Example
1m ),1()0( )1(
1 RaR
),0(/)1()1(
1 RRa ),0(/)1()1(
1 RRa
),0(/)1(1 RRk
)1()0( )1(
11 RaRE ).0(/))1()0(( )2(2
1 RRRE
2m
),2()0()1(
)1()1()0()2(
2
)2(
1
)2(
2
)2(
1
RaRaR
RaRaR
)1(
1
)2(
2
)1(
1
)2(
2
)2(
1 )1()0(
)1(aaaa
R
Ra
1
)1(
1
)1(
1
)1(
1)2(
2
)1()2(
)1()0(
)1()2(
E
aRR
RaR
aRRa
Example
))(1( 2)2(
212 aEE
))(1))(1()0(()2()1()0( 2)2(
2
)1(
1
)2(
2
)2(
12 aRaRRaRaRE
).)(1( 2)2(
21 aE
))1()0(/())2()1()0()1(( 22)2(
1 RRRRRRa
)).1()0(/())1()2()0(( )2(22)2(
2 RRRRRa