rational bases for system identification

Rational bases for system identification

Adhemar Bultheel, Patrick Van guchtDepartment Computer Science

Numerical Approximation and Linear Algebra Group (NALAG)

K.U.Leuven, Belgium

adhemar.bultheel.cs.kuleuven.ac.be

http://www.cs.kuleuven.ac.be/∼ade/

March 2001

Approximation & Optimization in the Caribbean, Guatemala, March 27, 2001

1/25

Systems

We consider discrete time systems

u −→ G −→ y

u is input → U(z) =∑k ukz

1−k

y is output → Y (z) =∑k ykz

1−k

G transfer function

Y (z) = G(z)U(z)

Approximation & Optimization in the Caribbean, Guatemala, March 27, 2001 A. Bultheel

2/25

Frequency Domain Identification

Estimate G as G such that we minimize (L2(T)-norm)

‖Y − Y ‖ = ‖GU − GU‖ = ‖(G− G)U‖ = ‖G− G‖w

min1

2π

∫ +π

−π|G(z)− G(z)|2w(z)dω, z = eiω, w(z) = |U(z)|2

♣ ♣ ♣Measurements G(zj)Nj=1, zj ∈ T, with variance σjNj=1.

min ‖G− G‖w = min

N∑j=1

|G(zj)− G(zj)|2

σ2j

1/2

, wj = σ−2j ,


3/25

Frequency Domain Identification(2)

or

‖Y − Y ‖ = ‖(G− G)U‖ = ‖G− G‖w

=

N∑j=1

|G(zj)− G(zj)|2wj

1/2

, wj = |U(zj)|2/σ2j

or

‖G− G‖ =∥∥∥∥YU − BA

∥∥∥∥=

∥∥∥∥Y A−BUAU

∥∥∥∥ = ‖Y A−BU‖w, w = 1/|AU |2


4/25

Linear/Nonlinear problem

Since the approximant G is a rational function, we have a mixedlinear/nonlinear approximation problem.

G(z) =B(z)A(z)

, A,B ∈ Πn

linear B(z) =n∑k=0

bkz−k

nonlinear A(z) =n∑k=0

akz−k =

n∏k=1

(1− αk/z).

Given an estimate for A,finding B is a weighted but linear least squares problem.


5/25

Nonlinear problem

Given a stable and efficient method to solve the linear parameters λ asa function of the nonlinear parameters ν, we can consider the problem ofminimizing a cost function C(ν) = K(ν, λ(ν)).

Need system stability. If ν = α (system poles), then need |αk| < 1. Sohave to solve a constrained nonlinear weighted least squares problem in C.

Solve nonlinear problem with standard routine.How to solve the linear problem?


6/25

Orthogonal Rational Functions

G(z) =n∑k=0

λkφk(z),

with for given αk

φk(z) ∈ Lk =

pk(z)∏k

j=1(1− αj/z): pk ∈ Πk

orthogonal basis functions with respect to an appropriate inner product.The inner product can be discrete or continuous, but is in general withrespect to a weight (measure) on T.

〈f, g〉µ =∫

T

f(z)g(z)dµ(z) or 〈f, g〉w =N∑j=1

f(zj)g(zj)wj.


7/25

ORF recurrence

Forward recursion[φn(z)φ∗n(z)

]= en

z − αn−1

z − αn

[1 LnLn 1

] [ζn−1(z) 0

0 1

] [φn−1(z)φ∗n−1(z)

]

Ln = −

⟨φk,

1−αn−1zz−αn φn−1

⟩⟨φk,

z−αn−1z−αn φ

∗n−1

⟩ , en =(

1− |αn|2

1− |αn−1|21

1− |Ln|2

)1/2

ζk(z) =1− αkzz − αk

, Bn = ζ1 · · · ζn, φ∗n(z) = Bn(z)φn(1/z).

Backward recursion = Nevanlinna-Pick algorithm


8/25

Inner product evaluation

If data are available in z = zjNj=1 ⊂ T with corresponding weights

w = wjNj=1 then choose the discrete inner product.

For a continuous inner product, choose points on the circle, e.g.equidistant zj = exp2ijπ/N, j = 1, . . . , N and evaluate weightwj = w(zj) and use discrete inner product.

For example w(z) = |U(z)|2, and zj equidistant on T, then given timedomain data uk, this can be computed very efficiently by FFT.

We can evaluate the moments cl in |U(z)|2 =∑l∈Z

clzl by convolution

and use a Nevanlinna-Pick type algorithm on the PR function Ω(z) =c0/2 +

∑∞l=1 clz

l. (approximately)


9/25

The linear least squares problem

So we can compute

Φ(α) = Φ(z; α) = [φ0(z) | φ1(z) | · · · | φn(z)] ∈ CN×n+1

W1/2 = diag(w)1/2, λ = λknk=0, G = G(z)

Setting Gn =∑nk=0 λkφk, solve in least squares sense

W1/2Φ(α)λ(α) = W1/2G

by orthogonality however, λ(α) = Φ(α)HWG.


10/25

Related work

Work by Ninnes, Van den Hof, Hueberger, Bokor, and others:

Use ORF with respect to Lebesgue measure. These have explicitexpressions

φn(z) =

√1− |αn|2 zz − αn

Bn−1(z)

but that does not help for the condition number of ΦHWΦ in the linearleast squares problem.

And/or use a finite number of αk that are cyclically repeated.


11/25

Nonlinear problem

K(α) =∑j

|G(zj)−n∑k=0

λk(α)φk(zj; α)|2wj ⇒ minα⊂D

K(α)

= ‖(I−Φ(α)Φ(α)HW)G‖2w

constraint: set αj = rjeiωj with −1 ≤ rj ≤ 1, j = 1, . . . , n.

αj can be forced to be real or complex conjugate.


12/25

Robot arm

100 data points, 200 Hz, condensed in the beginning, order [6/6]

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.810

−4

10−2

100

102

frequency response fct

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.810

−10

10−5

100

105

variance

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.810

−4

10−2

100

102

relative error

poles

0.0665± 0.0690i0.8770± 0.4765i0.9733± 0.2266i

zeros

1.9350, − 0.86491.0164± 0.1056i0.6905± 0.1238i

−1 −0.5 0 0.5 1 1.5 2−1

−0.8

−0.6

−0.4

−0.2

0

0.2

0.4

0.6

0.8

1


13/25

Band pass filter

50 data points, 20 kHz, in lower half, order [6/6]

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.60

0.5

1

1.5frequency response fct

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.610

−12

10−11

10−10

10−9

variance

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.610

−2

100

102

104 relative error

poles zeros

0.0487 - 0.4006i 9.67660.0487 + 0.4006i 1.47420.6070 - 0.4944i 1.09550.6070 + 0.4944i 0.90320.7356 - 0.3739i -0.65070.7356 + 0.3739i 0.4337

−1 −0.5 0 0.5 1 1.5 2

−1

−0.8

−0.6

−0.4

−0.2

0

0.2

0.4

0.6

0.8

1


14/25

Band pass filter

50 data points, 20 kHz, in lower half, order [22/22]

0 0.5 1 1.5 210

−12

10−11

10−10

10−9 variance

0 0.5 1 1.5 20

0.5

1

1.5frequency response fct

0 0.5 1 1.5 2−4

−2

0

2

4frequency response fct

0 0.5 1 1.5 210

−10

10−5

100

relative error

−1 −0.5 0 0.5 1 1.5

−1

−0.8

−0.6

−0.4

−0.2

0

0.2

0.4

0.6

0.8

1


15/25

Electrical Machine

110 data points, 4 kHz, condensed in the beginning, order [5/5]

0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.410

−2

10−1

100

frequency response fct

0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.410

−12

10−10

10−8

10−6 variance

0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.410

−4

10−3

10−2

10−1

relative error

poles zeros0.1875 0.54610.3886 0.92390.9609 0.96560.9966 0.99690.9970 0.9990

−1 −0.8 −0.6 −0.4 −0.2 0 0.2 0.4 0.6 0.8 1−1

−0.8

−0.6

−0.4

−0.2

0

0.2

0.4

0.6

0.8

1


16/25

Sensitivity, Electrical Machine

−1 −0.8 −0.6 −0.4 −0.2 0 0.2 0.4 0.6 0.8 110

−3

10−2

relative error, variable parameters 5


17/25

Polynomial method

min∥∥∥∥YU − BA

∥∥∥∥2

= min ‖Y A− UB‖2|AU |−2

The weight depends on the solution.

Estimate A so that we have an estimate for the weight 1/|AU |,

Set w = [Y/AU − U/AU ], P = [A B]T ⇒

minN∑j=1

P (zj)HWjP (zj), Wj = w(zj)Hw(zj)

need orthogonal block polynomials φk and write P =∑nk=0 φkλk,

φk ∈ Π2×2k , λk ∈ C

2×1. Constraint: P “monic”.


18/25

Monic solution

Φ = [φ0(z) . . . φn(z)] ∈ C2N×2(n+1), W = diag(Wj) ∈ C

2N×2N

min λHΦHWΦλ, Φλ monic

orthogonal polynomial ΦHWΦ = I ⇒ P = φnλn.

Szego-type recurrence

φnσn = zφn−1 + φ∗n−1γn

φ∗nσHn = zφn−1γ

Hn + φ∗n−1

σn, γn ∈ C2×2 (block Schur parameters)


19/25

Hessenberg matrix

w1 z1w2 z2

. . .wN zN

unitary

similarity⇒

transf.

BlockUpper

Hessenberg

× × × × × × × ×× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×

|

× × × × × × × × ×× × × × × × × ×× × × × × × ×× × × × × ×× × × × ×× × × ×× × ×


20/25

Efficient algorithm

Store the block upper Hessenberg matrix in factored form H = G1G2 · · ·Gm

Gk =

I2(k−1)

−γk σkσk γk

I...

Chasing the elements to block upper Hessenberg form by similaritytransformations percolating through the product requires operations on3× 3 or 5× 5 blocks ⇒ fast algorithm.


21/25

Nonlinear problem

Once P = [A B]T has been found, use A to modify the weight w = 1/|AU |and reiterate.

or

Use nonlinear method to improve the estimate∑nk=0 φkλk. Weight depends

on A ⇒ loss of orthogonality, but condition number of Jacobian increasesonly slightly.


22/25

CD’s radial servo system

sampling frequency 9.7 kHz, order [5/5]

Relative Freq

0.1 0.2 0.30-60

-40

-20

0

20

40M

agni

tude

FR

F (d

B)

error

frf


23/25

Conclusion

• Rational approximation in weighted discrete least squares sense.

• Linear and nonlinear parameters. Solve for linear in terms of nonlinear.Use orthogonal basis to minimize the condition number.

• Either direct orthogonal rational basis or linearized vector polynomial asa combination of orthogonal block polynomials.


24/25

References ORF

[1] A. Bultheel, P. Gonzalez-Vera, E. Hendriksen, and O. Njastad.Orthogonal rational functions. Cambridge University Press, 1999.

[2] P. Van gucht, A. Bultheel. Using orthogonal rational functions for systemidentification, Report TW314, Dept. Computer Science, K.U.Leuven,September 2000.

[3] P. Van gucht, A. Bultheel. Matlab routines for system identificationusing orthogonal rational functions.http://www.cs.kuleuven.ac.be/˜nalag/research/software/ORF/ORFidentification.html.


25/25

References OPV

[1] A. Bultheel, M. Van Barel, Y. Rolain. Robust rational approximationfor identification, 2001, Submitted

[2] M. Van Barel, A. Bultheel, Discrete linearized least squaresapproximation on the unit circle, J. Comput. Appl. Math. 50 (1994)965-972.

[3] A. Bultheel, M. Van Barel. Vector orthogonal polynomials and leastsquares approximation, SIAM J. Matrix Anal. Appl. 16 (1995) 863-885.

[4] M. Van Barel, A. Bultheel, Orthogonal polynomial vectors and leastsquares approximation for a discrete inner product, Electron. Trans.Numer. Anal. 3 (1995) 1-23.


rational bases for system identification

Documents