Computing ill-Conditioned Eigenvaluesand Polynomial Roots

Zhonggang ZengNortheastern Illinois University

International Conference on Matrix Theory and its Applications -- Shanghai

Can you solve (x-1.0 )100 = 0 Can you solve

x100-100 x99 +4950 x98 - 161700 x97+3921225x96 - ... - 100 x +1 = 0

1 01 1 1 1 ... ... 1 1

0 1 1

A = X X-1

Eigenvalues of

The Wilkinson polynomial

p(x) = (x-1)(x-2)...(x-20) = x20 - 210 x19 + 20615 x18 + ...

Wilkinson wrote in 1984:

Speaking for myself I regard it as the most traumatic experience in my career as a numerical analyst.

Myths on multiple eigenvalues/roots:

- multiple e’values/roots are ill-conditioned, or even intractable

- extension of machine precision is necessary to calculate multiple roots

- there is an “attainable precision” for multiple eigenvalues/roots: machine precision attainable precision = ----------------------------- multiplicity

Example: for a 100-fold eigenvalue, to get 5 digits right

500 digits in machine precision 5 digits precision = -----------------------------------------

100 in multiplicity

The backward error: 5 x 10-10

The forward error: 5

Conclusion: the problem is “bad”

-- method is good!

-- Ouch! Who’s responsible?

If the answer is highly sensitive to perturbations, you

have probably asked the wrong question.Maxims about numerical mathematics, computers, science and life, L. N. Trefethen. SIAM News

Who is asking a wrong question?

What is the wrong question?

A: “Customer”

B: Numerical analyst

A: The polynomial or matrix

B: The computing objective

Kahan’s pejorative manifolds

xn + a1 xn-1+...+an-1 x + an <=> (a1 , ..., an-1 , an )

All n-polynomials having certain multiplicity structure form a pejorative manifold

Example: ( x-t )2 = x2 + (-2t) x + t2

Pejorative manifold: a1= -2t a2= t2

Pejorative manifolds of 3-polynomials

( x - s )( x - t )2 = x3 + (-s-2t) x2 + (2st+t2) x + (-st2)

( x - s )3 = x3 + (-3s) x2 + (3s2) x + (-s3)

Pejorative manifold of multiplicity structure [1,2]

a1= -s-2ta2= 2st+t2

a3= -st2

Pejorative manifold ofmultiplicity structure [ 3 ]

a1 = -3sa2 = 3s2

a3 = -s3

Pejorative manifolds of 3-polynomials

The wings: a1= -s-2t a2= 2st+t2

a3= -st2

The edge: a1 = -3s a2 = 3s2

a3 = -s3

General form ofpejorative manifolds

u = G(z)

W. Kahan, Conserving confluence curbs ill-condition, 1972

1. Ill-condition occurs when a polynomial/matrix is near a pejorative manifold.

2. A small “drift” of the problem on that pejorative manifold does not cause large forward error to the multiple roots, except

3. If a multiple root/eigenvalue is sensitive to small perturbation on the pejorative manifold, then the polynomial/matrix is near a pejorative submanifold of higher multiplicity.

Ill-condition is caused by solving polynomialequations on a wrong manifold

Pejorative manifolds of 3-polynomials

The wings: a1= -s-2t a2= 2st+t2

a3= -st2

The edge: a1 = -3s a2 = 3s2

a3 = -s3

Given a polynomial p(x) = xn + a1 xn-1+...+an-1 x + an

The wrong question: Find ( z1, ..., zn ) such thatp(x) = ( x - z1 )( x - z2 ) ... ( x - zn )

because you are asking for simple roots!

/ / / / / / / / / / / / / / / / / // / / / / / / / / / / / / / / / / /

The right question:Find distinct z1, ..., zm such thatp(x) = ( x - z1 1 x - z2 )2 ... ( x - zm )m

m = n, m < n

do it on the pejorative manifold!

For ill-conditioned polynomial p(x)= xn + a1 xn-1+...+an-1 x + an ~ a = (a1 , ..., an-1 , an )

The objective: find u*=G(z*) that is nearest to p(x)~a

Let ( x - z1 1 x - z2 )2 ... ( x - zm )m =

xn + g1 ( z1, ..., zm ) xn-1+...+gn-1 ( z1, ..., zm ) x + gn ( z1, ..., zm )

Then, p(x) = ( x - z1 1 x - z2 )2 ... ( x - zm )m <==>

g1 ( z1, ..., zm ) =a1

g2( z1, ..., zm ) =a2

... ... ...

gn ( z1, ..., zm ) =an

I.e. An over determined polynomial system

G(z) = a

(m<n)n

m

tangent plane P0 :

u = G(z0)+J(z

0)(z- z0)

initial iterate

u0 =

G(z

0 )

pejorative root

u* =

G(z

* )

The polynomiala

Project to tangent plane

u 1 = G(z 0

)+J(z 0)(z 1

- z 0)

~

new iterate

u1 =

G(z

1 )

Pejora

tive m

anifo

ld

u = G

( z )

Solve G( z ) = a for nonlinear least squares solution z=z*

Solve G(z0)+J(z0)( z - z0 ) = a for linear least squares solution z = z1

G(z0)+J(z0)( z - z0 ) = aJ(z0)( z - z0 ) = - [G(z0) - a ] z1 = z0 - [J(z0)+] [G(z0) - a]

Theorem: Let u*=G(z*) be nearest to p(x)~a, if1. z*=(z*1, ..., z*m) with z*1, ..., z*m distinct;2. z0 is sufficiently close to z*;3. a is sufficiently close to u*

then the iteration converges with a linear rate.

Further assume that a = u* , then the convergence is quadratic.

Theorem: If z=(z1, ..., zm) with z1, ..., zm distinct, then the Jacobian J(z) of G(z) is of full rank.

zi+1=zi - J(zi )+[ G(zi )-a ], i=0,1,2 ...

The “pejorative” condition number

u = G(y)

v = G(z)

|| u - v ||2 = backward error|| y - z ||2 = forward error

u - v = G(y) - G(z) = J(z) (y - z) + h.o.t.

|| u - v ||2 = || J(z) (y - z) ||2 > y - z ||2

y - z ||2 < (1/) u - v ||2

1/ is the pejorative condition numberwhere is the smallest singular value of J(z) .

Example (x-0.9)18(x-1.0)10(x-1.1)16 = 0 Step z1 z2 z3

--------------------------------------------------------------------0 .92 .95 1.121 .87 1.05 1.102 .92 .95 1.113 .88 1.01 1.104 .90 .97 1.125 .901 .992 1.1016 .89993 .9998 1.10027 .9000003 .999998 1.10000078 .899999999997 .999999999991 1.1000000000099 .900000000000006 .99999999999997 1.10000000000001

forward error: 6 x 10-15

backward error: 8 x 10-16

Pejorative condition: 58

Even clustered multiple roots are pejoratively well conditioned

Example (x-.3-.6i)100 (x-.1-.7i) 200 (x - .7-.5i) 300 (x-.3-.4i) 400 =0

Scary enough? Round coefficients to 6 digits.

Z1 z2 z3 z4

.289 +.601i .100 +.702i .702 +.498i .301 +.399i

.309 +.602i .097 +.698i .698 +.499i .299 +.401i

.293 +.596i .101 +.7003i .7002 +.5005i .3007 +.4003i

.300005 +.600006i .099998 +.6999992i .69999992+.4999993i .2999992 +.3999992i

.3000002+.60000005i .09999995+.69999998i .69999997+.49999998i .29999997+.400000002i

Roots are correct up to 7 digits!

Pejorative condition: 0.58

Example: The Wilkinson polynomial

p(x) = (x-1)(x-2)...(x-20) = x20 - 210 x19 + 20615 x18 + ...

There are 605 manifolds in total. It is near some manifolds, but which ones?

Multiplicity backward error condition Estimated structure number error------------------------------------------------------------------------[1,1,1,1,1,1,1,1,1...,1] .000000000000003 550195997640164 1.6[1,1,1,1,2,2,2,4,2,2,2] .000000003 29269411 .09[1,1,1,2,3,4,5,3] .0000001 33563 .003[1,1,2,3,4,6,3] .000001 6546 .007[1,1,2,5,7,4] .000005 812 .004[1,2,5,7,5] .00004 198 .008[1,3,8,8] .0002 25 .005[2,8,10] .003 6 .02[5,15] .04 1 .04[20] .9 .2 .2

What are the roots of the Wilkinson polynomial?

Choose your poison!

The “right” question for ill-conditioned eigenproblem

Given a matrix A

Find a structured Schur form S and a matrix U such that

AU - US = 0U*U - I = 0

A ~

3 1 3

2 S =

+

+ + ++ + +

2 1 2

+

Over-determined!!!

Minimize || AU - US ||F2 + || U*U - I ||F2

---- nonlinear least squares problem

Example: A + E, where ||E|| A 1.0e-7

Step ------------------------------------------ 0 4.0 1.1 1 2.99 2.01 2 3.0006 1.9998 3 3.000001 1.9999997 4 3.00000001 1.99999992

The pejorative condition number: 22.8

UT(A+E)U =

+

+ + ++ + +

+

+ O(10 -7)

Conclusion

1. Ill-condition is cause by a wrong “identity”

4. To calculate ill-conditioned eigenvalues/roots, onehas to figure out the pejorative structure (how?)

2. Multiple eigenvalues/roots are pejoratively well conditioned, thereby tractable.

3. Extension of machine precision is NOT needed,a change in computing concept is.