<^ Pf^Zl2 3

pi 7 h 1*^-

UCRL-51186

LINEAR MULTISTEP METHODS FOR ORDINARY

DIFFERENTIAL EQUATIONS: METHOD FORMULATIONS,

STABILITY, AND THE METHODS OF NORDSIECK AND GEAR

A. C. Hindmarsh

March 20, 1972

Prepared for US. Atomic Energy Commission under contract No. W-7405-Eng-48

Hr LAWRENCE LIVERMORE LABORATORY University of California/Livermore

&%im\imH OF THIS DOCt]M£HT IS !J^»i!lTI8

DISCLAIMER

This report was prepared as an account of work sponsored by an agency of the United States Government. Neither the United States Government nor any agency Thereof, nor any of their employees, makes any warranty, express or implied, or assumes any legal liability or responsibility for the accuracy, completeness, or usefulness of any information, apparatus, product, or process disclosed, or represents that its use would not infringe privately owned rights. Reference herein to any specific commercial product, process, or service by trade name, trademark, manufacturer, or otherwise does not necessarily constitute or imply its endorsement, recommendation, or favoring by the United States Government or any agency thereof. The views and opinions of authors expressed herein do not necessarily state or reflect those of the United States Government or any agency thereof.

DISCLAIMER Portions of this document may be illegible in electronic image products. Images are produced from the best available original document.

TID-4500, UC-32 Mathematics and

Computers

m LA\/VRENCE UVERMORE LABORATORY

University ot Caffomiet/livermore, CaliforrK^/94550

UCRL-51186

LINEAR MULTISTEP METHODS FOR ORDINARY DIFFERENTIAL EQUATIONS: METHOD FORMULATIONS,

STABILITY, AND THE METHODS OF NORDSIECK AND GEAR

A. C. Hindmarsh

MS. date: March 20, 1972

- N O T I C E -This report was prepared as an account of work sponsored by the United States Government. Neither the United States nor the United States Atomic Energy Commission, nor any of their employees, nor any of their contractors, subcontractors, or their employees, makes any warranty, express or implied, or assumes any legal liability or responsibility for the accuracy, com­pleteness or usefulness of any information, apparatus, product or process disclosed, or represents that its use would not infringe privately owned rights.

DISTRIBUTION OF THIS DOCUMENT IS UHLIMII

Contents

Abstract 1

Basic Concepts 1

Conventional Formulation . . . . . . . . . . 1

Matrix Formulation 4

Equivalence and the Method of Nordsieck 6

Equivalence and Multivalue Methods 6

Nordsieck 's Method 9

Order 10

Stability 13

Conventional Definitions 13

Matrix Definitions 15

Gear ' s Method 18

Stiffness and Stiff Stability 18

Derivation of Method Coefficients 19

Other Features 22

References 24

- i i i -

LINEAR MULTISTEP METHODS FOR ORDINARY DIFFERENTIAL EQUATIONS: METHOD

FORMULATIONS, STABILITY, AND THE METHODS OF NORDSIECK AND GEAR

Abstract

This report deals with linear multistep methods for the numerical solution of

systems of ordinary differential equations. Rather than survey the entire subject,

however, the report emphasizes those aspects of the theory necessary for an under­

standing of the methods of C. W. Gear. These methods were devised over the last

few years for the case of stiff systems, where stiffness is typified by widely varying

time constants, but they are applicable generally. The report is in part a summary

of certain existing l i terature, authored chiefly by Gear, and in part an augmentation

of that l i tera ture in an attempt to fill certain gaps and to clarify the development of the

methods.

The basic notions of the classical theory of linear multistep methods are

summarized briefly and then restated in the matrix reformulation of these methods.

The methods of Nordsieck, which were p recursors of those of Gear in many respects ,

a re presented in t e rms of this matrix formulation and of the idea of equivalence of

methods. The notion of numerical stability is then discussed in both the classical and

matr ix fornaulations. Finally, we give a motivated derivation of the methods of Gear,

and we discuss their implementation in a practical algorithm.

Basic Concepts

CONVENTIONAL FORMULATION

We are concerned here with the following system of ordinary differential equations

(ODE's)

y = f(y,t), (1)

where y and f a re vectors of length N= l, t is the independent variable, and y = dy/dt.

For this ODE system we consider the initial value problem, where y(t) is to be

computed for t„ s t £ T, w h e r e t^ < T ^ oo. (The value T might not be known in

a d v a n c e . ) To fac i l i t a t e the a n a l y s i s , it i s r e q u i r e d that f at l e a s t sa t i s fy a L ipsch i t z

condit ion

\i(y^. t) - f(y2, t ) | ^ c IYI - ygl

w h e r e |* | deno tes any su i t ab le n o r m if N > 1.

The c l a s s of l i n e a r m u l t i s t e p m e t h o d s for th i s p rob l em i s u sua l ly d e s c r i b e d as

fol lows: A p p r o x i m a t e solut ion va lues a r e ca lcu la ted at t = t^., t . , t , . . . , w h e r e

t = t 1 + h, with s o m e s t ep s i z e h, a c c o r d i n g to a fo rmula of the fo rm n n - 1 ' f > &

Ki Kg

y = y a. y . + y jS. h y ., (2)

j=l j=o

w h e r e y, = y(t, ), y, = y (t. ) = fCy .. t, ), and the a. and p. a r e fixed coeff ic ients a s s o c i a t e d

with the p a r t i c u l a r me thod . Equat ion (2) i s used to ca l cu l a t e y when p r e v i o u s a p p r o x i ­

m a t e va lues of y and y a r e known. Spec ia l c o n s i d e r a t i o n s , not d i s c u s s e d h e r e , a r e

n e c e s s a r y to obta in the s e v e r a l va lues needed at the beginning to m a k e Eq. (2) app l i cab l e .

A c l a s s i c a l e x a m p l e i s that of the A d a m s m e t h o d s , w h e r e K = 1 and 0 = 1 .

We say Eq. (2) i s of o r d e r q if, whenever it i s appl ied to a given (sufficiently

smooth) p r o b l e m us ing exac t pa s t va lues , it y ie lds a y which is exac t within an e r r o r

(the t r unca t i on e r r o r ) that i s ©(h*^ ) a s h -• 0, but that i s g e n e r a l l y not 0(h*' ). Th i s

can be t r a n s l a t e d into expl ic i t ana ly t i ca l t e r m s by c o n s i d e r i n g the p r o b l e m y = Xy, y(0) = 1

with solut ion y = e . We define po lynomia ls

Ki ^2

p ( x ) = x ^ - y Q. x^~\ CT(X) = - ^ /3. x ^ ' ^ , K = m a x ( K ^ , K 2 )

j=l j=o

(3)

Then if x = e , the exac t pas t va lues a r e y, = e = x , y = X x . When t h e s e a r e

i n s e r t e d into Eq . (2), and the o r d e r i s a s s u m e d to be q, we get

(y^ - x") (1 - JSQ Xh) + x " " ^ [p{K) + hXa (x)] = 0 , y^ " x " = 0 (h ' l^^ ) ,

p(x) +logxCT(x) = j ^ ^ ^ 0 (h ' l ^^ ) = 0 ( h ' ^ ^ ^ = o [ ( x - l ) ' ^ ' ^ M , as x - 1 . (4) X -'

In o t h e r w o r d s , if we expand p(l + x) + log (1 + X)CT (1 + x) in a power s e r i e s in x, the

t e r m s th rough x'^ m u s t vanish and the x^ t e r m does not, if the method is of o r d e r q.

It can be shown ( Ref. 1, pp. 244-246) that th i s condit ion is a l so sufficient for the

method to be of o r d e r q.

- 2 -

Notice that Eq. (3) involves K^ + K„ + 1 coefficients, so that this is the maximum

number of t e rms one could expect to make vanish in Eq. (4). Thus, the maximum order

attainable is K + K„. The minimum order allowed is one. The condition q ?= 1 is

referred to as consistency, and merely means that Eq. (2) is to give exact resul ts in

the case f(y, t) = constant.

The method given by Eq. (2) is said to be explicit if / - =0 and implicit if |S =t 0.

In the implicit case the term /3f hy makes Eq. (2) a system of (generally nonlinear)

equations to be solved. Experience has shown, and we will see later, that the cost of

solving this system is usually more than offset by the increase in accuracy and numeri­

cal stability that can be gained over explicit methods. Therefore we will think in te rms

of implicit methods from here on, although the special case [3 = 0 is not ruled out.

In the implicit case, Eq. (2) is generally solved by making a first guess at

(predicting) y , say y ,„,, with an explicit method, and then using an iterative method

(correcting) to find the t rue solution of Eq. (2). Two basic methods, functional iteration

and Newton's method, for the correction phase will be considered here, although a few

other methods for solving this problem are known. Denote the right hand side of Eq. (2)

by g(y ), so that the quantity y = y is to solve the equation y = g(y). [Notice that g(y)

involves y only in the te rm ;8 h f(y, t ). ] If the i tera tes are denoted y , >, then the

method of functional iteration is based on the formula

y(m+l) " Sfy(m)J'

while Newton's method is based on

y(m+l) = y(m) - [ l - I f | y ( ^ J | ^(m) " ^^^(m)] }

where 1 denotes the N X N identity matr ix . In most practical applications of Newton's method, the matrix whose inverse

appears above, namely 1 - l3„h-TT- [y , \, t ], is not reevaluated as m changes. ^^ ' •^ ' 0 9y n, (m)' n ^ Rather, the value at [y ,^..,1 ], or possibly at some ear l ier point, usually suffices.

Inaccuracies in this matr ix do not affect the limit of the i tera tes if convergence occurs,

only the ra te of convergence.

In the case of Newton's method, let W denote the inverse matr ix used, and in the

case of functional iteration, let W = I. Then, both methods take the form

y(m+l) =y(m)+wjg[y(^)] -y^^)} (5)

As will be seen more clearly later, choosing the more costly method (Newton's) again

gives benefits that are often well worth its cost.

- 3 -

MATRIX FORMULATION

Both the londerstandingand the implementation of these ODE methods can be

aided by a matr ix reformulation of them. This reformulation was constructed by

Gear and is ,summarized below.

In the calculation of y from Eq. (2), we must have saved, in some form, the

L = Kj + Kg pieces of information y^^.^, y^ g' • • -^n-K ' ^"'^ ^ ^ n - l ' ^ ^ n - 2 ' * " " ^ ^ n - K o " Let y . be the vector of length L consisting of these quantities (which a re themselves vectors of length N). F rom this history vector, the updated vector

T

yn = (yn' V l ' - ' - ' ^ n + l - K i ' ^^n'^^n-V ' ' ' ' ^yn+l -Kg) ^ ^

(T = transpose) must be constructed. In Eq. (6), only the components y and hy

require calculation, the others merely being t ransferred from y _ i .

The prediction phase gives y ,^, as a linear combination of the components of

2 1 . Suppose we next define hy ,„,, a prediction of hy , by the relation

^ 1 ^2

^n, (0) = S ' j ^n-j + ^0 ^^n. (0) + S ^j^^n-y j=l j=l

With y i^'. and hy ,„>, we may form a vector y ,„. which predicts y and satisfies

for some L X L matr ix B. (The matr ix B will be independent of h as a resul t of using

hy, ra ther than y, as vector components.)

We now i tera te to get corrected values y , .. As we do, suppose we also

calculate hy , > according to ''n, (m) *=

^n, (m) = 1 '^j^n-j ^^oh^n , (m) + 1 ^J^^n-j ' ^ ^ ]=1 3=1

Next, let the vector y , . be as in Eq. (6), but with y / ^ in place of y , and

hy / \ in place of hy . This vector is then a solution of Eq. (2) in the sense of • n, (m) '' •'n ^ Eq. (8), for m = 0, 1, 2, . . . . Assuming that the y , . converge as m -* oo, we know

from the construction of y / \ by way of Eq. (5) that the limit must be a solution y

of Eq. (2) with y = f(y , t ) there . By taking l imits in Eq. (8) it follows that the y , .

converge to y^ . Thus we may take y; = rn^^oo ^n, (m)' °^ ^" practice y ^ = y^^ ^j^^ for

a sufficiently large M. If we define a function

- 4 -

G[y , A = hf [y / , t ] - hy , ^ , (9) -n , (m) - n, (m)' n -^n, (m) ' ^

and use Eq. (8), we find that Eq. (5) can be written

V (m+1) - ^n, (m) = 0 ^ n ^ ^ ^ n , (m)^'

and that

h^n, (m+1) - ^^^n, (m) = ^ ^ ^ f l n , (m) ^'

To express this more compactly, let k be the vector of length L with i3„ as its

first component, 1 as its(K^ + 1) st component, and 0 for all other components. Then

we have

y I ,1^ = y / . + k W G[y , J •^n, (m+1) -^n, (m) — n - n, (m) ^

for m = 0, 1,. . . . Equations (7), (9), and (10), together with

(10)

W

for functional iteration

[i - S„h - ^ ly , „ , ] ^ for Newton's method |_ ^0 ^ '• n, (0) J

(11)

and

Zn " ^n, (M) (12)

constitute the matrix formulation of the method. [Sometimes, a final evaluation of f

is performed, to give hy = hf[y /ivD' n^ instead of using Eq. (10) as i s . For

simplicity, we will exclude this in the presentation here . ]

The formalism so far allows the case K„ =0, L = K^, where the 1 would not

appear in k. However, this situation is r a re ly a practical one since y i is used in

the predictor if not the corrector , and so must be stored anyway. We will therefore

require K. a 1, K„ a 1 henceforth.

Also, it will often be useful to have a relation obtained by summing in Eq. (10).

M-1

If we set H =} ^^^n Tn ] ^"^^ ^^^ ^^' (12), we obtain

m=0

y = y /r.\ + k w H . ^n ^n, (0) — n n

We may eliminate W H by use of component K^ + 1 above and get

- 5 -

B y n - l + ^ f h y n - h y n . ( 0 ) J (13)

Conversely, if a method is given in its matrix formulation, we may obtain its

conventional form shown in Eq. (2) from the first equation in Eq. (13) and the equation

for hy^^ Q in Eq. (7).

Equivalence and the Method of Nordsieck

EQUIVALENCE AND MULTIVALUE METHODS

The choice of the vector y ear l ier to represent the relevant past history of the

calculation was somewhat a rb i t ra ry . We could just as well have taken , instead of the

L quantities listed in Eq. (6), some set of L different linear combinations of those

quantities, the combinations being chosen for convenience in e r r o r estimation or from

roundoff considerations, etc. For any such choice, however, the original L quantities

must be obtainable from those saved if information is not to be i r revers ib ly lost.

Following Gear ' s t reatment 2,3 we consider saving a vector z of length L given by

z = Qy , - n ^^n ' (14)

where Q is any nonsingular L X L matrix, and y is as before.

The Eqs. (7), (9), (10), and (12) can be restated in t e rms of vectors having the

form of £ as follows: If A = Q B Q " , F{z) = G(Qy) and S_ = Qk, then the new description

of the method is

^ , (0) " ^ ^n-V

z , , i x = z , > f i W Ffz , J , —n, (m+1) —n, (m) — n '-n, (m)' '

Ffz , .] = hf[y , ,, t 1 - hy , ,, ^~n, (m)J ^•'n, (m)' n' - n, (m)'

and

-n " -n , (M)-

(15)

The analog of Eq, (13) is

z = Az , + i [hy - hy / „ J . —n —n-1 — ^ ' n •^n, (0)^ (16)

- 6 -

For ease in calculating F, it is convenient to make y and hy (or y ) two of the & . -'n - n ' n

components of £ , as they were for y . For definiteness, we will number the com­

ponents of £ from 0 to L - 1, with y as component 0 and hy as component 1. (The

reason for numbering from 0 will be seen shortly.)

The mathematical equivalence of the two descriptions means if a step is taken

under each of the two, starting from equivalent past values (via Q), then the resulting

values y and z will also be equivalent [ i . e . , related by Eq. (14)], except for round­

off e r r o r . Methods of the form of Eq. (15) are called either multivalue or L-value 2 3 methods by Gear ' as it is the number L of values saved that is significant, not the

number max (K^,K„) of steps involved. Other than trivial variations of y , there are two forms of the vector z_ in

^n —n common usage. One is the vector of successive backward differences of values of

y and y . Thus, for example, the implicit Adams method of order q, where K^ = 1

and K„ = q - 1, is often implemented with

z = ( y , h y , h V y , . , . , h V y ) —n Y'n' - n' - n' ' ^ nj

as the history vector, 4

A radically different choice, first proposed by Nordsieck, is that of approximate successive derivatives of y at t , scaled so as to become the t e rms of a finite Taylor

• n' ser ies

- iT , q = L - 1. (17) z —n

y hy ^ yn h y^^ n - n'

These derivatives are generally not exact, but they a re uniquely defined by reference

to the qth order polynomial represented by the vector in question in the case of a

single ODE (N = 1). That is, the vector y uniquely defines such a polynomial p for

which p(t. ) = y, and p'di^) = y, for the components of y . The value £ is then defined

to have components h p^* '(t )/kI, where k = 0, 1, . . . , q. This definition t ransla tes

into z = Qy and in this form applies for general N.

So far, the algorithm of Eq. (15) has been just a reformulation of that in Eqs.

(6) through (12). However, it can now be regarded as a basis for constructing new

methods. That is, once a choice of the history vector £ is made, we can construct

predictor mat r ices A and vectors i_ without reference to y , B, and k. Thereby we

obtain algorithms that would not (or possibly could not) have been thought of in the

conventional form.

In choosing i^ for the construction of new methods, there are some important

res t r ic t ions , however. In Eq. (10), it was necessary that k had components / Q

and 1 in the positions corresponding to y and hy in y . Under an equivalence , rp n. n n

transformation to z = (y . hy . . . , ) , rows 0 and 1 of Q will be rows of the identity

matrix, and so 1_ will have ^Q = iS„, ^ = 1. If we were now to consider methods based

on arb i t ra ry S_ in Eq, (15), we would get an analog of Eq, (16), where the first equation

would read

yn = yn, ( 0 + 1 7 f^f^^n-y '"^^nAO)^-

Now we find that the i terative solution of this equation for y , using functional iteration

or Newton's method, is not generally the algorithm of Eq. (15). In particular, for

Newton's method with f(y, t) l inear in y, the method converges in one iteration, but

Eq. (15) does not—unless jf, = 1 and JP„ = „ . Having no reason to abandon this

desirable convergence feature, we therefore require that S.^ = \, and in the definition

of W by Eq. (11), we define /3- = i_ . In case of functional iteration, - i = 1 has the

convenient consequence that f ', '•, = f[y / \, t ]. ^ •'n, (m+1) '•'n, (m)' n The following example shows that Eq. (15) allows the formation of methods that

Eqs. (6) through (12) do not. Consider Eq. (15) with L = 3 and the Nordsieck vector Q rp rp

- n "" ^^n' ^^n'*^ ^n^^' ' ^ ^ (arbitrarily) let ^ = (1, 1, 1) , The choice of A and of iteration method is immater ia l . If this method is equivalent to one of the form of Eqs, (6) through (12), the lat ter must use a vector v of the form

(a) yn=(vyn- l '^ynr

or

(b) I^-{yn'^^n-^K-lf-

( rp

y , y ^ - i ' y 9) . regarded as inadmissible for other reasons , could also be considered and similarly eliminated.] To determine Q for case (a), we may

2 consider a quadratic y(t) = c„ + c-t + c„t at points t = 0, t _.. = -h. Then

Zn =fy^O)' y(-h)' hy(O)]'^ = (-Q CQ-hc^+h2c2, h c j " ^ , while z^ = (CQ, hc^, h^Cg)'^. Writing ^ = Q z , we obtain

Q-^

Similarly, in case (b), we find

' 1 0 0

Q'^ = I 0 1 0

\Q 1 -2

The vector k = Q i is then given by

(a') k =( 1 , and (b') k =( 1 )•

Both of these are inadmissible, as k was to have 0 for the second component in (a')

and for the third in (b'). Thus the original method (not particularly valuable otherwise)

has no equivalent three-value method in the form of Eqs. (6) through (12). (There is,

however, a two-step, four-value conventional method to which it is equivalent, in a

slightly broader sense.)

From here on, we will work, so far as possible, in the framework of Eq. (15)

and the la rger class of methods given thereby. We may then define two such L-value

methods to be equivalent if their history vectors y and £ are related by a nonsingular

L X L matrix Q; if the quantities B, G, and k for the first method are related to the A,

F, and S^ of the second by way of Q as before; and if the choice of iteration method

(i. e . , of W ) is the same for both.

NORDSIECK'S METHOD

What was possibly the ear l ies t nontrivialuse of an equivalent reformulation of a 4

linear multistep method was made by Nordsieck. His ODE method is the equivalent form

of theimplicit Adams method that uses the history vector of Eq. (17), where the length of

the vector is L for the method of order L. Nordsieck uses a predictor of order q = L - 1

(the equivalent of the classical explicit Adams formula) and functional iteration for the

cor rec tor . [Nordsieck did not express his method in the matr ix formulation of Eq. (15),

but ra ther in a system of equations written more compactly in Eq. (16) .]

The pr imary advantage that Nordsieck gained from his choice of history vector

is in the situation where the stepsize h must be changed. If it is changed by a factor

Q, one simply multiplies component i in z by a , where 0 s i * q, and proceeds with

the new sizestep oh. In the context of Eq. (2), this corresponds to estimating the

needed values of y and y at times t , t - ah, t - 2Q'h, . . . by an interpolation from the J J n' n n - " saved values at t imes t , t . , , . . .

n' n-h Nordsieck's method was programmed also by Gear (Ref. 3, Sec 9, 3, and Refs. 5

and 6), but Gear uses a z of length L = q + 1 for the method of order q, and a predictor

of the same order . Thus, at the cost of a slight increase in storage for a given order.

Gear reduces the amount of correction necessary. Also, he saves an estimate of

h ^ y ^ , a derivative of one higher order than that saved by Nordsieck, making it

considerably easier to estimate ti-unca+ion e r r o r s accurately.

In either of these two implementations, the method of Nordsieck uses a predictor

matrix A that is of a particularly elegant form. In fact, this form re l ies only on the

choice [Eq. (17) ] of history vector, not on the relation to the Adams method. The

prediction y /(-.,, being a linear combination of the L independent components oi z_ _-.,

- 9 -

can n e v e r be of o r d e r g r e a t e r than q = L - 1. Suppose it i s of o r d e r q, and that in

fact a l l componen t s of £ ,„^ a r e c o r r e c t to o r d e r q w h e n e v e r £ ^ i s c o r r e c t to o r d e r q,

(This i s the c a s e when the method in ques t ion i s equivalent to a convent ional one with an

o r d e r q p r e d i c t o r and a c o r r e c t o r of o r d e r q o r m o r e . ) It then t u r n s out that A m u s t be

t he uppe r t r i a n g u l a r P a s c a l t r i a n g l e m a t r i x

^ 4 i 3 t e j = 0 ' a . .=( j ) (=Oif i>j ) . (18)

This i s b e c a u s e the r e l a t i o n s

^ n X / n \ ^ • i ( 0 ^ - = - ^ J = l

a r e exac t in the c a s e s y(t) = t , w h e r e 0 s k s q and the m a t r i x A i s uniquely d e t e r ­

mined by the e x a c t n e s s of i t s i m a g e s in t h e s e L c a s e s .

The p r e m u l t i p l i c a t i o n of a vec to r by the A in Eq. (18) can be a c c o m p l i s h e d with

r e p e a t e d addi t ions only, thus avoiding both c o m p u t e r mu l t i p l i c a t i ons and m e m o r y

s t o r a g e for A. Th i s fact was o b s e r v e d by G e a r and u t i l ized in h i s p r o g r a m .

ORDER

As ind ica ted in the p r e c e d i n g sec t ion , the o r d e r s of the p r e d i c t o r and c o r r e c t o r

a r e m o s t e a s i l y d i s c u s s e d in the c a s e of N o r d s i e c k ' s h i s t o r y v e c t o r . By equ iva lence ,

the d i s c u s s i o n in that context then c a r r i e s ove r to the c a s e of an a r b i t r a r y h i s t o r y

v e c t o r .

The o r d e r of the p r e d i c t o r , which i s at m o s t q = L - 1 for an L - v a l u e method, i s

u sua l l y defined in t e r m s of the a c c u r a c y of y /„>. However , it i s n a t u r a l now to

define an o r d e r for the p r e d i c t o r m a t r i x A a s the l a r g e s t i n t e g e r q for which a l l

componen t s of z ._> a r e a c c u r a t e to o r d e r q wheneve r z , i s a c c u r a t e to o r d e r q . —n, (0; ^p —n-1 p By following p r e v i o u s r e a s o n i n g , we find that q i s the l a r g e s t i n t ege r for which the

f i r s t q + 1 co lumns of A a g r e e with t hose of the m a t r i x in Eq. (18). Since Eq. (18)

m a k e s q = q, and t h e r e would s e e m to be no jus t i f ica t ion for us ing a p r e d i c t o r of l e s s

than m a x i m a l o r d e r , we m a y a s wel l r e s t r i c t o u r s e l v e s to the c a s e w h e r e A i s given

by Eq . (18). Th i s s i tua t ion , o r i t s equiva lent f o r m for the c a s e of s o m e o t h e r h i s t o r y

v e c t o r of length L, i s a l m o s t a lways what o c c u r s in p r a c t i c e .

T u r n i n g now to the o r d e r of the o v e r a l l method , and a s s u m i n g that the N o r d s i e c k

v e c t o r and p r e d i c t o r m a t r i x of Eq. (18) a r e used, we find that the o r d e r of the method

i s at l e a s t q no m a t t e r what i_ i s . (Even the r e s t r i c t i o n £_^ = 1 i s not n e c e s s a r y for th i s

r e s u l t . ) In o t h e r w o r d s , the c o r r e c t e d solut ion i s a lways a c c u r a t e to a t l e a s t the o r d e r

of the p r ed i c t ed so lu t ion . To s e e th i s , c o n s i d e r a p r o b l e m y = f(y, t) , w h e r e y e C

and f h a s a L ipsch i t z cons tan t C with r e s p e c t to y. We apply Eq. (15) with exac t pa s t

- 1 0 -

v a l u e s , and we know that £ ,Q^ i s c o r r e c t within 0 ( h ). Hence the e r r o r in z^,

say 6 = (6.)5, s a t i s f i e s ' i 'O'

6 = 0 ( h ^ ) + i [hy^ - h j ^ , (0) ]

by Eq. (16). E l im ina t i ng the l a s t e x p r e s s i o n .

But

Hence

6 Q = V l - ^ O ( h L ) .

6, = hy - hy (t ) 1 ' 'n • n

= hf(y , t ) - hf(y - 6r>. t ) •'n' n " n 0' n

| 6 ^ | s h C | 6 Q | .

6Q| - I i p I h C | 6Q | + 0 ( h ^ )

| 6 J ^ i 0(h '") = 0 ( h h , 0' hC IJPQI

a s a s s e r t e d . It follows a l so that al l componen t s of £ a r e a c c u r a t e within 0 ( h ), and

in fact |6^| = 0(h-^^-^).

Now we m a y ask what condi t ions on i would p e r m i t the o r d e r of the method to

exceed q, tha t i s , to be q + 1 = L . F o r L > 2, however , we find that the notion of

o r d e r , even for the s ing le quant i ty y , i s no longer wel l -def ined b e c a u s e the notion

of "exac t pas t v a l u e s " i s not we l l -de f ined . If we r e g a r d the z of Eq. (17) to be /. \ —n exac t when the y a r e exac t d e r i v a t i v e s of o r d e r i s q and we look at the e r r o r in

• 'n ^'

y when z . i s exact , we get one definit ion of o r d e r . But if we r e g a r d the method

a s an equiva lent fo rm of one that u s e s a convent ional y a s in Eq. (6), and if we define

z to be exac t when z = Qy with exact y, and y, in y , then we get ano ther definition.

F u r t h e r m o r e , if we fo rmu la t e the l a t t e r definition for each of the L = 1 poss ib le fo rms

of y al l of the r e s u l t i n g definit ions will be di f ferent .

G e a r ' s explanat ion of th i s ambigu i ty (Ref, 3, Sec . 10.4) i s that we s o m e t i m e s

al low z to differ f rom the vec to r z of exact sca led d e r i v a t i v e s by an e r r o r vector n - n L ~ " L (L)

that i s 0 ( h ), but we s t i l l r e g a r d z a s exac t . If we w r i t e r = h y (t , ) / L ! and L+1 ^ ^~

ri_= er + 0 ( h ), then a given notion of e x a c t n e s s c o r r e s p o n d s to a p a r t i c u l a r vec to r e of c o n s t a n t s . Th i s vec to r wi l l have e„ = e , = 0 b e c a u s e y and hy a r e independent

° ^ n e " L+1 of that cho ice . We a r e then defining z to be exact when z = z + e r + 0 ( h ). Now ^ —n —n —n — the ques t ion of o r d e r i s whe the r the method gives exact z_ f rom exact z , , in the

above s e n s e , a s s u m i n g y e C To s e t t l e th i s ques t ion , we f i r s t o b s e r v e that , f rom Tay lo r s e r i e s cons ide r a t i ons ,

® - dr + O ( h ^ Az.! . = z ! - dr + o ( h ^ ' ^ ) . .e _ e —n-1 —n

d = (d.

• 1 1 -

We then have

^ £ n - l " Az^_^ + Aer +0(h^'^^)

= £n ^ (A£ - d ) r + 0(h^"*"^.

If we define §_ = z - £ as before and use Eq. (16), we get

6_ = (Ae - d) r + i [hy^ - hy^^ ^Q^] + 0(h^^^),

while exactness requires

6 = e r + 0(h^^^) .

Defining £ = (c.)jJ = Ae - £ - d, and eliminating the bracketed expression, we obtain

c. - i . c , =0 all i, or 1 1 1

q q q je . - L. (19)

The equation for i = q gives 0 = L + c..i , or c = - L / i provided i 7 0. The equations

for 1 s q - 1 then provided a nonsingular linear system for e . . . , e in t e rms of i , ^ q —

When it is solved, the i = 1 equation in that system will be identical to the one defining

c. in Eq. (19), so that the solution will be consistent. The equation from 1 = 0 in

Eq. (19) is then satisfied if and only if

'^Mi e. - 11= V^ |1 -> e, ). (20)

Thus, Eq. (20) and Jl =/= 0 constitute necessary conditions on l^ such that the order be L. T.

A reve r sa l of the logic shows that they are also sufficient to make 6 of the required

form. However, for a given ^, we may no longer have the same £ we started with, and

the original ambiguity appears again. To eliminate it, in the manner of Gear, we

simply say that if any e exists for which e„ = e^ =0 and Eq. (19) holds, then the given

method has order L. This in effect allows methods to be of order L that would not

have been by our ear l ier definitions. For example, for q = 2 and L = 3, Eq. (19)

becomes

-12 -

^0 + (2 6^ - 3) 1

^2

and yields, for i „ =1= 0.

^ 2 = 1 ( - l / V - ^ " ^ ^ ^ 0 = 1 - ^2/6-

Thus there is a whole one-parameter farhily of three-value methods of order three. The T two cases arr ived at by reference to the vectors (a) y = (y , y , , hy ) and '' „ - n - n - ' n - r - n

(b) y = (y , hy , hy _ ) as given ear l ie r correspond to e = -1 and -3/2 , respectively. IX XX XX IX X ^

The lat ter case is Nordsieck's method of order three .

It is still t rue, however, that the order of an L-value method cannot exceed L for

any reasonable definition of order . Thus the L-value method of Eq. (15) with the

predictor of Eq. (18) has an order of either q or q + 1 depending on i (q = L - 1). After

desired order conditions, if any, are met, any remaining free parameters in _? can be

set on the basis of other c r i te r ia . The most important other cri terion (often more so

than order) is stability, the subject of the next section.

S t a b i l i t y

CONVENTIONAL DEFINITIONS

We begin with a review of the concepts of stability, as defined for the conventional

formulation of a linear multistep method. For this purpose, it is considered sufficient

to study only the simple linear ODE y = Xy. The nonlinear case is not generally

amenable to stability analysis, other than in locally linearized form, and the case of

linear systems almost always reduces to that of N uncoupled single ODE's by a

preliminary linear transformation. Because of the latter rationale, we must allow X

to be complex, even though the ODE's actually solved m a y b e rea l .

Recall the formulation of Eq. (2) and the associated polynomials of Eq. (3).

Applied to y = Xy, Eq. (2) becomes a finite difference equation with constant coefficients.

The theory of such equations then gives

y , = f c. x;^, n = l , 2 , . . . , (21) i = l

where the x. are roots of the polynomial equation of degree K = max (K^ K„),

p(x) +hXa (x) = 0,

-13 -

provided t h e s e r o o t s a r e d i s t inc t . When they a r e not, Eq. (21) m u s t be modified to k n

inc lude t e r m s of the form n x. , w h e r e the i n t ege r k i s nonnegat ive and l e s s than the

mul t ip l i c i ty of the roo t . The c. a r e cons tan t s obta ined by so lv ing the (nonsingular)

l i n e a r s y s t e m that c o n s i s t s of Eq. (21) for n = 1, 2, . . . , K. Th i s g ives c. as a l i n e a r

combina t ion of y^, . . . , y „ , and hence a l so of the L =K^ + K„ in i t ia l va lues r e q u i r e d ,

name ly , the componen t s of the vec to r ^n i" Eq. (6) (with n = 0 t h e r e ) .

The notion of s t ab i l i ty h a s to do with the way in which e r r o r s in the in i t ia l va lues

a r e p ropaga ted in the ca lcu la ted solut ion as n-»oo . F r o m the l i n e a r i t y of the c. in

Eq. (21), we m a y w r i t e th i s p ropaga ted e r r o r in y^ a s

K

' y n - 1 6 c. x ^ (22)

1 = 1

w h e r e 6c. depends l i n e a r l y on the L in i t ia l e r r o r s ( a s sumed to v a r y independent ly

without r e f e r e n c e to the ODE). T h u s , the e r r o r p r o p a g a t e s in a m a n n e r d e t e r m i n e d

by the r o o t s x., which in t u r n depends on hX. (This 6y does not include the effects of

t runca t ion e r r o r s in t roduced dur ing the ca lcu la t ion , a s th i s i s a m a t t e r of a c c u r a c y

r a t h e r than s tabi l i ty . )

One of t h e s e r o o t s , s ay x . , i s ca l led the p r inc ipa l root , in that c^x.. i s an

^t hX n hX

app rox ima t ion to the c o r r e c t solut ion function y„e " = y^e . That i s , x^ = e

At hX = 0, we have x = 1 f rom the 0th o r d e r condit ion p(l) = 0. All o the r r o o t s ,

ca l led nonpr inc ipa l o r e x t r a n e o u s , give r i s e to s o - c a l l e d p a r a s i t i c so lu t ions of the

d i f fe rence equat ion, which a r e r e g a r d e d as s o u r c e s of e r r o r . T h e s e t e r m s a r e

t o l e r a b l e if each nonpr inc ipa l x. is l e s s than x^ in magni tude , for then | 6c .x . | << |c -x^ |

a s n -• 00, and s i m i l a r l y in the c a s e of mul t ip l e r o o t s . If |x . | = |x^ | but x. i s a s i m p l e

roo t , it i s s t i l l t o l e r a b l e , b e c a u s e we wil l have | 6c . | < < | c . | if the in i t ia l va lues a r e

a c c u r a t e . We a r e thus led to cons ide r what i s t e r m e d the r eg ion of r e l a t i v e s t ab i l i ty

in the complex hX plane

S = |hX : Ix. |< Ix, I , o r X. i s s i m p l e and |x. i = Ix, I, for i = 2, , . . , K L r ' ' i ' ' l ' ' 1 ^ ' i ' ' l " '

The r e g i o n S is u sua l ly quite difficult to d e t e r m i n e , and so ano ther concept of

s t ab i l i ty which is m o r e m a n a g e a b l e , ye t s t i l l meaningful , i s c o m m o n l y used . This i s

the concept of abso lu te s tabi l i ty , w h e r e b y a p a r a s i t i c solut ion is c o n s i d e r e d t o l e r a b l e

if i t m e r e l y t ends to 0 a s n-* oo . Of c o u r s e i t i s u n r e a s o n a b l e to a sk for th i s condi t ion

when Re(hX) > 0, s ince then the solut ion g rows exponent ia l ly . F o r such X, hX m u s t

be kept r a t h e r s m a l l and we can usua l ly count on the fact that 0 e S to g u a r a n t e e that

h X e S a l so (because u sua l ly 0 is a l so an i n t e r i o r point of S ). F o r Re(hX)< 0, however ,

the solut ion decays exponent ia l ly , and i t i s c e r t a i n l y r e a s o n a b l e to r e q u i r e that |x . | < 1

for both p r i nc ipa l and nonpr inc ipa l r o o t s . Thus we define the r eg ion of abso lu te

s t ab i l i ty a s

S = |hX: Re(hX) < 0, |x . | < I for a l l i } .

- 1 4 -

It is this region that we will be concerned with hereafter . The reason it is more

tractib'

that is

tractible than S is that its boundary is given by the relations p(x) + hX a (x) =0, | x | = 1,

hX = -p(x)/a (x) I ,Q,0^e<2Tr, x = e

or ra ther a portion of this curve together with a part of the imaginary axis.

An important point to be seen here is that explicit methods have finite absolute

stability regions. For in the explicit case, i3_ = 0, the degree of a (x) is less than that

of p(x), which is K, and we have the identity

(x - X )(x - x„) . . . (x - x„) = p(x) + hXa (x).

If the leading term of a (x) is -/3.x •', and hXeS , then the coefficient of x •'

above is -a. - hX/3. from the right-hand side, while it is bounded in magnitude by j .1, T i I I \ /

in view of (xJ < 1 in the left-hand side. Hence we obtain a finite bound on |hX | . This fact is a major justification for the use of implicit methods, where, as we shall see, S need not be bounded. ' a

MATRIX DEFINITIONS

Since we are committed to the matrix formulation of all methods here, we need

to have a definition of stability in t e rms of that formulation. Following Gear (Ref. 3,

Sec. 9. 1. 1), we therefore consider in Eq. (15) an e r r o r vector 6 , in z , , and ask ^ —n-1 —n-1

how this is propagated into z , z . . , . . . , where we are still looking only at the problem y = Xy. Thus F(z , .) = hXy / \ - hy , s, and W is either 1 or (1 - f„hX)" . •^ ^ -n , (m)' - n, (m) -'n, (m)' ^ n T Denote hX by p for short, and let v = (p, - 1 , 0, . . . , 0) . Then Ffz . > ] = v z . ., and we may take differences in Eq. (15) to get

6 /„. = A6 1 —n, (0) —n-1

^n, (m+1) = -^n, (m) + 1 ^ ^ ^ i n , (m)

i n = ^n, (M)

for the e r r o r s 6 , > in z , , . We then have -n , (m) -n, (m)

M T 6 =r^^A6 1, J = I + W ! t v , —n —n-1 n '

where I is the L X L identity matr ix .

-15-

In the case of Newton's method, as we would expect from its convergence prop-M ,

er t ies , we have J = J for all M sr i provided only that jP„p 7= 1. In the case of functional iteration, we find that

r M I +

U^P) M

1- V jf V

and so convergence occurs in Eq. (15) if and only if | i„p |< 1, or | h | < l / | i.X | . M u u

Assuming this, we replace J by its limiting value, which is the same as the J for Newton's method. We than have 6 = T6 , with an e r r o r amplification matrix

—n —n-1

T = T(p) = [I + (1 i „ p ) ' ^ i v ^ ] A . v'^ = (P, 1. 0. , 0) . (23)

(In methods where M is small and Newton's method is not used, it may be necessary

to study J A ra ther than T, but we will not do this here.) If T has eigenvalues

1* • form

and these are distinct, then the components of 6 •n

T 6Q will be of the

6 = y c.X- .1 = 1

(24)

with c.. depending linearly on the components of 6_. In the case of multiple eigenvalues, ^''k n t e rms n X. must be included, as given by the Jordan canonical form of T.

Two things should be noticed about this resul t . Firs t , if the above process is

repeated for any equivalent method based on y , B, k with £ n = Qy , giving an e r r o r

amplification matrix S, then it is easily verified that S = Q'^TQ. Therefore S and T

have the same eigenvalues, and the analog of Eq. (24) for y is the same except for

the coefficients c... This is to be expected from equivalence. Second, suppose we

a re working with a method which is equivalent to a conventional one [Eq. (2)]. Then

Eq. (24) for i = 0 must be identical to Eq. (22), as both a re formulas for the same,

uniquely determined, quantity, provided we identify 6_^ by way of Q and the initial e r r o r s

leading to Eq. (22). The same must hold for the modified forms of Eqs. (22) and (24)

where distinctness of roots and eigenvalues is not assumed. It follows that the set of

nonzero X. is the same as the set of nonzero x., identically in p = hX, and that the

multiplicities of nonzero elements of that set also agree . This can be restated simply

as

(1 - ^Qp)det [ x l - T(p)] = K[p{K) + pa(x)]. k = L - K = min(K^, K (25)

- 1 6 -

i den t i ca l ly in x and p. Th i s r e l a t i o n i s proved d i r e c t l y by G e a r .

The m e a n i n g of Eq. (25) i s tha t we m a y equal ly wel l define the s t ab i l i ty r e g i o n s

in t e r m s of the e igenva lues of T(p), r a t h e r than as we did e a r l i e r . Thus

S = I p: Re(p )< 0 and all e igenva lues of T (p) l ie in | x | < 1 | . (26)

N o r d s i e c k p e r f o r m s an e r r o r p ropaga t ion a n a l y s i s s i m i l a r to G e a r ' s ; but does

not r e g a r d the L componen t s 6 a s independent . R a t h e r , in the p r o b l e m y = Xy, he

a s s u m e s that 6y = X6 , o r 6 = p6 for all n. Thus the vec to r 6 = ( 6 , 6 , . . . , 6^) of length "'n n n ^ n —n n n n *= q = L = 1 is s tudied and found to be M6 _^ for a q X q m a t r i x M; and the e igenva lues of

M a r e used to define s t ab i l i ty . The m a t r i x M can be obtained f rom the r e l a t i o n s

6 ' - = y t . . 6 ^ , = ( t . „ + p t . j 6 ° , + y t..6'i ,, n ZL 1] n - 1 lO " i l n - 1 Z_, i] n - 1 '

j=0 j=2

in t e r m s of T(p) = ( t . . ) . ( N o r d s i e c k ' s a n a l y s i s u s e s the p a r t i c u l a r choice of h i s t o r y

vec to r shown in Eq. (17), but i t is j u s t a s e a s i l y done in genera l . )

To r e l a t e the e igenva lues of M and T, we f i r s t o b s e r v e that t . . = p t „ . . Th i s

follows e a s i l y f rom

t . . = a. . + (1 - i„p) ' -^ £. ( p a „ . - a, .) (27) 1] 13 0* ' i'^^ 0] 1]

a s given by Eq. (23), We then exannine xl - T and p e r f o r m c e r t a i n o p e r a t i o n s that

p r e s e r v e i t s d e t e r m i n a n t . Adding p t i m e s column 1 to co lumn 0 gives a m a t r i x with

e l e m e n t s

(i, j) : x6 . . - t . . (j "> 0, 6 = K r o n e c k e r 6)

(i, 0): x6.Q - t.Q + p(x6.^ - t.^) (an i ) .

In t h i s m a t r i x we s u b t r a c t p t i m e s row 0 f rom row 1 and obta in a m a t r i x with e l e m e n t s

( i , j ) : x 6 . . - t . . ( j > 0, i ^ 1 )

(1,0): x6.Q - t.Q - pt.^ (i i=l)

(1, j ) : x 6 ^ . - t^. - P ( X 6 Q . - IQ.) = x6^^ (j > 0)

(1, 0): -t^Q + p(x - t^^) - p(x - IQQ - P I Q ^ ) 0.

If I i s the q X q ident i ty m a t r i x , we s e e that the above m a t r i x a g r e e s with xl - M in

pos i t ions (i, j) for i = 1, whi le it h a s x6^ . in the (1, j) pos i t ion . Hence

- 1 7 -

det (xl - T) = X det (xl' - M),

and the eigenvalues of M are simply those of T, with the eigenvalue 0 possibly omitted.

A definition of S based on M is thus identical to Eq. (26).

In summary, the three definitions of stability—based on (a) the root set in the

conventional context, (b) the e r ro r amplification matr ix T of Gear, and (c) the e r r o r

amplification matr ix M of Nordsieck — are all equivalent, whether it is absolute or

relat ive stability that is of interest . As we discuss Gear ' s methods next, it is natural

that we use choice (b), with the region of Eq. (26), in our stability considerations.

Gear's Method

STIFFNESS AND STIFF STABILITY

The kind of ODE problem for which the classical methods (e, g., the Adams

methods, Milnes' method, etc.) generally fail is one in which the solution asymptotically

approaches a very smooth function, so that transient contributions decay to zero — some 5 7 possibly more rapidly than others . As Gear ' observed, this phenomenon can be seen

in a single ODE problem

y = X [y - F(t)] +F ' ( t ) , y(0) = F(0),

where F is smooth and X < 0. The solution is y = F, but various e r r o r s are bound to

introduce a contribution ce as well. Since the added term F ' - XF in y drops out in

the stability analysis of the last section, the matr ix T = T(hX) of Eq. (23) applies. When

X « 0, the ODE itself is very stable [i. e . , y - F -• 0 rapidly regardless of any initial

e r ro r in y(0) ] . But we still may have a very unstable numerical solution in that hX may

not belong to the absolute stability region S . Put another way, if we choose h so

small that hXeS , then the time scale of interest , as determined by F, may require

so many steps that it is not feasible to finish the problem.

Another situation posing the same difficulty is the linear system y = My, where

M is a constant matr ix with eigenvalues X., . . , , X^, all negative, but widely differing

in magnitude. Here, all of the hX. must lie in S if numerical stability is to occur. 1 cL

Yet, the stability of the system itself is determined by the smallest of the h | X. |, say

h/j = h max X.. That is, we would want to take steps of a size h on the order of 1/ |A( |,

the largest (in magnitude) "time constant" present, in order to reach the equilibrium

state in a reasonable number of steps, but we a re not allowed to use such an h if some

other hX. ^ S ,

-18-

S y s t e m s y = f whose t i m e cons t an t s have wide ly differ ing nega t ive r e a l p a r t s a r e

ca l led stiff. T i m e cons t an t s a r e defined g e n e r a l l y a s l /X. in t e r m s of the e igenva lues 3f ^

X. of the Jacob ian -3- . Of c o u r s e , s t i f fness i s a m a t t e r of d e g r e e , a s i s the notion of

a " r e a s o n a b l e n u m b e r of s t e p s above.

C l a s s i c a l m e t h o d s fail on the above e x a m p l e s s i m p l y b e c a u s e only a bounded

por t ion of the nega t ive r e a l ax i s l i e s in S . The difficulty does not o c c u r for a method

in which S conta ins e i t h e r the whole nega t ive r e a l ax i s , o r , b e t t e r yet , a neighborhood

of the r e a l ax i s p lus a half p lane Re(p) < D, w h e r e D s 0. If th i s i s the c a s e with D = 0, so that S i s the e n t i r e nega t ive r e a l halfplane, then the method is ca l led A - s t a b l e , a a o t e r m in t roduced by Dahlqu i s t . However , Dahlquis t showed that th is condit ion is

7 unach ievab le for m u l t i s t e p m e t h o d s of o r d e r l a r g e r than two. G e a r defined a notion

of stiff s t ab i l i t y which r e q u i r e s that S conta in the r e g i o n d e s c r i b e d above for s o m e

D < 0. It r e q u i r e s a l so that S , the r eg ion of r e l a t i v e s tab i l i ty , include a ne ighbor ­

hood of the o r ig in to the r igh t of the i m a g i n a r y a x i s .

The m e t h o d s tha t a r e advocated by G e a r , and named af te r h im, a r e stiffly s t ab le

and a r e e a s i l y mot iva ted by the d i s c u s s i o n up to th is point . The p r o b l e m is to i n s u r e

that the T(p) in Eq . (23) i s a s t ab l e m a t r i x (all e igenva lues in |x | < 1) for va lues of p

r a n g i n g ove r m o s t of the nega t ive r e a l ha l fp lane . We know that such methods m u s t

n e c e s s a r i l y be imp l i c i t . If we w e r e conce rned only about a neighborhood of p = 0, we

would be t empted to a s k that T(0) be as s t ab le as pos s ib l e , i . e . , that it have al l

e igenva lues equal to 0 except one, which m u s t be 1; th i s g ives the A d a m s m e t h o d s .

But a s we a r e conce rned with an unbounded s e t of va lues of p, ano the r choice i s

n e c e s s a r y . It i s then n a t u r a l to c o n s i d e r the r e q u i r e m e n t that T(oo) [= lirn T(p)] be as

s t ab le a s pos s ib l e , i, e . , that a l l i t s e igenva lues be 0. If ach ievab le , th i s will at l e a s t

a s s u r e s tab i l i ty of T(p) for al l p in a "ne ighborhood of 00, which would inc lude a hal f -

plane Re(p) < D for s o m e D < 0. We can hope that it wil l a l so happen to give s tab i l i ty

on the whole nega t ive r e a l ax i s , al though t h e r e i s no g u a r a n t e e of t h i s .

DERIVATION O F METHOD COEFFICIENTS

The r e q u i r e m e n t on T(oo) that we a r e mak ing can be s t a t ed a s det [xl - T(oo)] = x

for an L - v a l u e me thod . Shifting back to the convent ional fo rmula t ion for a momen t ,

we know by Eq . (25) that t h i s i s equiva len t to the s t a t e m e n t tha t

^ L __ ^ . ^ x ^ [ p ( x ) + p a ( x ) ] ^ _^k ^ (^^/^

CT (x) = - /3 Q X ,

p rovided that the method in ques t ion is in fact equivalent to one in the convent ional

fo rm of Eq. (2).

- 1 9 -

Now le t us r e v e r s e the viewpoint, and cons ide r c o n s t r u c t i n g a method of the f o r m

of Eq . (2) with the above a (x), and then c o n s i d e r equiva lent m a t r i x r e f o r m u l a t i o n s of i t .

Since we r e q u i r e K„ ^ 1, but a (x) has only the one t e r m , we n a t u r a l l y take K„ = k = 1,

and so K^ = K = L - 1 - q. Then Eq. (2) b e c o m e s

q

y = V Q-.y . + jS- hy . (28) n /_! J n - j 0 - n

j = l

F o r m a l l y , t h e r e i s a l so a p a r a m e t e r jS which van i shes by definit ion for the c o r r e c t o r

fo rmula [Eq. (28)] , but will be n e c e s s a r i l y n o n z e r o for the p r e d i c t o r . We would

expect to a t ta in o r d e r q but no h ighe r in Eq. (28), as t h e r e a r e only q + 1 = L f ree

p a r a m e t e r s . The p r e d i c t o r , hav ing ^ = 0 but jS f ree , will a l so be of o r d e r q. The

o r d e r equat ions for Eq. (28) then take the fo rm

p(l + x) - j3 (1 + x)'^ l o g d + x) = 0(x'^^^), 0

q q- j

1

p(x) = x'^ - y Q-.X"

and can in fact be used to d e t e r m i n e i3_ and the a. uniquely. The so lvabi l i ty of th i s

p rob lem is e a s i l y s een in that the l i n e a r a l g e b r a i c s y s t e m for (/3 a . . ., a ) has a

nons ingu la r t r i a n g u l a r m a t r i x . This at l e a s t p r o v e s that a method e x i s t s with a

T(oo) a s d e s i r e d , and that it e x i s t s in the convent ional f o r m .

Gea r , however , p r e f e r s the unconvent ional form in which the N o r d s i e c k h i s t o r y

vec to r , Eq. (17), i s used . The advan tages of that cho ice have a l r e a d y been d i s c u s s e d .

It i s t h e r e f o r e not p a r t i c u l a r l y d e s i r a b l e , and in fact not n e c e s s a r y , to ca l cu l a t e the

convent ional coeff icients as above and t r a n s f o r m . R a t h e r we will cons ide r the

condit ion on T(oo) d i r e c t l y and d e t e r m i n e ^ f rom it.

In the chosen formula t ion , the m a t r i x T(p) i s given by Eq . (27) as

Hj -yi)+\(p-i)/(^ - v > .

and so T(oo) = T is given by

(29) '.rC)-V'o = (0 - ' . If we define r = ( r . )^ = ig i , and u = (1, 1, . . . , 1) , then T = A - r u . The condit ion

that T have al l z e r o e igenva lues is equivalent to the s i m p l e r s t a t e m e n t T ^ = 0, the

z e r o m a t r i x .

The d e t e r m i n a t i o n of r i s fu r ther s impl i f ied by the u s e of gene ra t i ng po lynomia l s .

q

With any X = (x.)^, we will a s s o c i a t e the polynomial x(t) = ^ x.t of d e g r e e = q. Th i s

0

- 2 0 -

i s j u s t N o r d s i e c k ' s i n t e rpo la t ing polynomia l a s s o c i a t e d with Eq. (17) in the c a s e h = 1,

t = 0 . We w r i t e x «—•x(t), the c o r r e s p o n d e n c e being o n e - t o - o n e and l i n e a r . We then

know that Ax c o r r e s p o n d s to the s a m e polynomial but with the o r ig in shifted by h:

T V~* T Ax ^-* x(t + 1). Since £ x = > x. = x ( l ) , we m a y w r i t e r u x *-* x ( l ) r ( t ) , w h e r e

r * — r ( t ) . Thus we have Tx * '^x( t + 1) - x ( l ) r ( t ) when x —* x(t) .

Now suppose we choose x to sa t i s fy x( l ) = 0, so that Tx —* x(t + 1). Then 2

T X « ^ x ( t + 2) - x(2) r ( t ) . Let us a l so r e q u i r e that x(2) = 0, and in fact x(3) = 0,

Then T''x • -»x( t + j) for j = 0, 1, . . . , q. F ina l ly we m a y take x(L) = 1, and w r i t e

0 = T ^ x = T ( T \ )

*-» x(t + q + 1) - x(q +1) r( t)

= x(t + L) - r ( t ) .

Now the r e q u i r e m e n t s on x l ead uniquely to

x(t) = (t - l ) ( t - 2) • •• (t - q ) /q l .

Hence we have

r «-*r(t) = x(t + L) = (t + l)(t + 2) • . . (t + q ) / q l

P a r t i c u l a r coeff ic ients of i n t e r e s t a r e

q

r. = 1. r = ^ i , r 1 0 "' "" 1 zL i ' " q "q~l

1

Since th is r is d e t e r m i n e d uniquely f rom Eq. (29) and the r e l a t i o n T = 0 , and

we know in advance that a solut ion to th i s equat ion e x i s t s , we do not need to ver i fy

d i r e c t l y that r_ i s indeed a solut ion. We know that it i s , and we p roceed to t ake

i = r r , u s ing the r e l a t i o n r^ = l/-^r,. That i s ,

i = ( i . )^ ^ Ht) - S Jl.t' - (t + l)(t + 2)- - - ( t + q) (30) — 1 U L^ \ q

0 '•1\

1

The n u m e r i c a l va lues of t h e s e v e c t o r s l_ a r e given by G e a r (Ref. 3, Sec . 1 1 . 1 . 1 ,

and Ref. 7) for q = 1, 2, . . . , 6. To i l l u s t r a t e the use of Eq . (30) to get them, we give

the c a s e q = 2

m (t + l )( t + 2) _ t^ f 3t + 2 2(1 + 1/2)

- 2 1 -

OTHER FEATURES

The methods of Gear use the above coefficient vectors i_, together with the

L X L Pascal triangle matrix A for the prediction. Before proceeding, however, we

must turn to the verification of two other requirements on T (p), which is now known for

each q = 1, 2, . . . . The first of these is convergence, and, as stated ear l ier , it

shall be met in the s t r ic t sense, namely that |x | < 1 for all eigenvalues of T(0) except

for the simple eigenvalue x = l . Verification that this is the case must be done

numerically. The second condition we require is that of stiff stability. This also was 7

done numerically by Gear for 1 s q ^ 6. The computed region S contains a 3.

horizontal s t r ip covering the entire negative rea l axis in all six cases , as required.

But the vertical extent of that str ip becomes steadily smal ler as q increases . For

q = 7, the boundary curve c rosses the axis, making the method not stiffly stable for

this or l a rger q. As these facts about convergence and stiff stability require numerical

calculation, we will say no more about them here .

The last important specification necessary is that of the corrector iteration

method. As far as the two basic approaches considered here are concerned, this is

easily settled by recalling that the use of functional iteration on the problem y = - y

requi res that | -ff P I < 1 for p = hX, in order that the corrector converge. This would

destroy the advantage gained by achieving stiff stability, and hence we reject functional

i teration. The methods of Gear therefore include the use of Newton's method, in the

modified form described, as a fundamental part of the algorithm. The expense of

calculating the Jacobian matrix -^ can be further offset by neglecting to re-evaluate

it at every step, unless the existing value of this matrix fails to produce convergence.

In Gear ' s program , re-evaluation occurs only when the order q is changed, or when

the convergence of £ ,^x has not yet been achieved with M = 3. Convergence is determined by a relative difference test on the i terates y , .. •^ - n, (m)

This completes the basic description of the algorithm for the numerical solution

of ODE systems by Gear ' s method. Of course, there are a number of details involved

in its implementation which have not been discussed. While a complete treatment of

these is out of place here, a few comments are appropriate.

Two crucial mat te rs in the implementation of any multistep or multivalue method

a re the startup procedure and the choice of the stepsize h. A third which is often

relevant is the choice of the order of the method when a ser ies of methods of different

o rders is available, as with Gear ' s methods. In Gear ' s program, all of these mat ters

a re handled by a single algorithm which allows q and h to vary in an automatic and

dynamic manner throughout the problem. At the start , q is simply set to 1 and allowed

to increase to whatever value (s6) is found to be optimal.

The basis of this algorithm is the fact that the truncation e r r o r can be approxi­

mated at any given time by C i h* y " , if higher order t e rms are neglected and

the current order is q. This expression is in turn approximated by saving, at each

step, the quantity

- 2 2 -

M-1

\~-l ^n ^" -n. (n ) ^'-^^n- hyn, (0)' m=o

which satisfies

z = Az , + iE . —n —n-1 — n

In part icular

i E =h^y<'lVqI - h ^ y ^ , / q : -h^-^V^^+^Vq'. . q n •'n ' ^ -^n- l '^ -'n ' ^

[The e r r o r s in z_ and £ . of order q + 1, discussed in the section Order (p. 10) cancel

out he re . ] Hence, e = C .^ql i E is a readily available estimate of the desired •' ' q q+1^ q n " quantity. Now h is chosen so that |e | is smaller than some specified quantity 6, but

not too much smal ler . A new value h which makes it equal to 6 is given by

( h q / h ) ' l + l = 6 / | e q | .

Similarly, stepsizes h _ and h , can be computed for which the truncation e r ro r

would be approximately 6 in norm if the method of order q - 1 or q + 1, respectively,

were used. The lat ter requires an estimate of y ^ , which is obtained by way of

E - E 1. F rom these three stepsizes, a decision is easily made as to whether to n n-1 ^ ' •^ use order q - 1, q, or q + 1 on the next step, and what value of h to use. The

decision is based on maximizing h, but it is purposely biased slightly in favor of

making no change in order, so as not to make computational overhead costs too high.

In the case of an order increase, the vector £ is augmented by the component i E / L = h^y^^^/U . q n' -^n '

Another phase of the calculation that can require considerable attention is the

l inear algebra problem of calculating W F'[z j (M)]- For sizable systems it is not

efficient to invert the N X N matrix P = 1 - ^f^hi£ and to multiply by the inverse

W . Rather, the most efficient approach is to perform what is called an LU ^ - 1 - 1 - 1

decomposition of P, P = LU, and calculate P v as U (L v) for vectors v. Here

L and U are lower and upper triangular matr ices , respectively. The gain in efficiency

is made all the greater by the fact that for the same value of P, P v is needed for

many values of v. One program that performs this decomposition and back-substitution

is given by Forsythe and Moler (Ref. 9, pp. 68-69).

- 2 3 -

References

1. P, Henrici, Discrete Variable Methods in Ordinary Differential Equations (John

Wiley and Sons, New York, 1962).

2. C. W. Gear, "The Numerical Integration of Ordinary Differential Equations,"

Math, of Computations 21, 146 (1967).

3. C. W. Gear, Numerical Initial Value Problems in Ordinary Differential Equations

(Prentice-Hall, Englewood Cliffs, N. J . , 1971).

4. A. Nordsieck, "On Numerical Integration of Ordinary Differential Equations,"

Math, of Computations 16, 22 (1962).

5. C. W. Gear, "The Automatic Integration of Ordinary Differential Equation,"

Commun. Ass . Computing Machinery 14, 176 (1971).

6. C. W. Gear, "Algorithm 407: DIFSUB for Solution of Ordinary Differential Equations,

Commun. Ass . Computing Machinery 14, 185 (1971).

7. C. W. Gear, "The Automatic Integration of Stiff Ordinary Differential Equations,"

in Proc. Int. Fed. Inform. P rocess . Congr. (Humanities P r e s s Inc . , New York,

1968), p. A-81 .

8. G. Dahlquist, "Convergence and Stability in the Numerical Integration of Ordinary

Differential Equations," Mathematica Scandinavica 4, 3 3 (1956).

9. G. E. Forsythe and C. B. Moler, Computer Solution of Linear Algebraic Systems

(Prentice-Hall, Englewood Cliffs, N. J . , 1967).

- 2 4 -

Distribution

LLL Internal Distribution External Distribution

Roger E. Batzel

J. H. Bolstad

J. Chang

E. G. Gorman

J. R. Creighton

R. P. Dickinson

S. Fernbach

F . N. Fr i t sch

R. J . Gelinas

M. A. Harr ison

R. F . Hausman

N. W. Hetherington

A. C. Hindmar.sh

B. C. Howard

B. M. Johnston

M. C. MacCracken

R. L. Pexton

K. D. Pimental

R. E. Shafer

N. G. Smiriga

T. Suyehiro

J. P, Vajk

R, E, von Holdt

A. M. Wins low

M. Wirth

LBL Library

TID File

15

R. E. Huddleston R. A. McHugh Sandia Laboratories Livermore, California

B. L. Hulme L. F . Shampine H. A. Watts Sandia Laboratories Albuquerque, New Mexico

R. F . Sincovec Kansas State University Manhattan, Kansas

D. Kahaner M. Klein J . Sopka C D . Sutherland Los Alamos Scientific Laboratories Los Alamos, New Mexico

C. Anderson Institute for Space Studies New York, New York

H. Levy, II Smithsonian Astronomical Observatory Cambridge, Massachusetts

C. W, Gear Computer Science Dept. University of Illinois Urbana, Illinois

TID-4 500 Distribution, UC-32, Mathematics and Computers 208

30

MSG/edas/lc

- 2 5 -