1

In chapter 6 we develop quantitative methods. In this chapter we ”discretize” the problem and seek, instead of an analytical solution, the numerical values of the dependent variable at a discrete set of values of the independent variable so that the result is a table or graph, with those values determined approximately (but accurately), rather than exactly.

Chapter 6 Quantitative Methods: Numerical Solution of Differential Equations

6.1 Introduction

Consider the problem

6.2 Euler’s Method

With the exact solution

2' 2 ; (0) 1 (0 )y y x x y x (2)

2( ) .xy x x e (3)

2

In Fig. 1 we display the direction field defined by f(x, y) ＝ y+2x-x2, as well as the exact solution of

Let us denote the y values at Q, R, …as y1, y2,…. They are computed as y1 = y0 + f(x0, y0)h, y2 = y1 + f (x1, y1)h,…,

Expressed as a numerical algorithm, the Euler method is therefore as follows:

1 ( , ) , ( 0,1,2,...)n n n ny y f x y h n (4)

Fig. 1 Direction field motivation of Euler’s method

Euler’s method is also known as the tangent-line method because the first straight-line segment of the approximate solution is tangent to the exact solution y(x) at P. Apparently, the greater the step size the less accurate the results (Table 1).

2( ) .xy x x e

3

Scanning each row of the tabulation, we can see that the approximate solution appears to be converging to the exactsolution as h → 0.

Does the method converge to the exact solution as h → 0 and, if so, how fast? By a method being convergent we mean that for any fixed x value in the x interval of interest the sequence of y values, obtained using smaller and smaller step size h, tends to the exact solution y(x) as h → 0.

There are two sources of error in the numerical solution. ● The tangent-line approximation upon which the method is based;● The accumulation of numerical roundoff errors within the computing machine since a machine can carry only a finite number of significant figures.

4

It is important to understand that the Euler method (6) amounts to retaining only the first two terms of the Taylor series in (5). Thus, it replaces the actual function by its tangent-line approximation.

Let us denote the exact solution at xn as y(xn) and the approximate numerical solution at xn as yn. By the Taylor series

211 1 1 1

''( )( ) ( ) '( )( ) ( )

2!n

n n n n n n n

y xy x y x y x x x x x

…

211 1

''( )( ) '( )

2!n

n n

y xy x y x h h

… (5)

and the Euler algorithm

1 1 1( , ) ,n n n ny y f x y h (6)

Local Truncation error

5

The latter expression for en is awkward to apply since we don’t know ξ, except that Since We see

that as h → 0 we have ξ → xn-1, so (8) given

as h → 0, where C is a constant. Accordingly, we say that en is of order h2 and write as h → 0.

We ask how large the error en ≡ y(xn) – yn is after making that single step using the Euler method, from xn-1 to xn.

21 1

''( )( ) ( ) '( ) ,

2!n n n

yy x y x y x h h

(7)

Subtracting (6) from (7),2''( ).

2n

ye h

(8)

*1 .n nx x 1 1 .n nx x h

2 21''( )~

2n

n

y xe h Ch

2( )ne O h (9)

6

Accumulated truncation error and convergence.

The accumulated truncation error that has accumulated over all of the preceding steps since that error is the difference between the exact solution and the computed solution at any given x

n. We denote is as En ≡ y(xn) – yn and call it the accumulated truncation error.

Since the error en is due to truncation of the Taylor series it is called the truncation error-more specifically, the local truncation error.

We can estimate En as the local truncation error en times the number of steps n. Since en ＝ O(h2), this idea gives

2 2 2 00( ) ( ) ( ) ( ) ( ) ( ).n

n n

x xnhE O h n O h O h O h x x O h

h h

(10)

( )nE O h (11)

7

which result tells us how fast the numerical solution converges to the exact solution as h → 0.

The accumulated truncation error En is the sum of the n single-step errors. However, that is not quite true. As shown in Fig. 2 that E2 is e2+, not the sum of the single-step errors e2+e1. and is not identical to e1.

The difference between and e1 is the result of the slightly different slopes of L1 and L2 acting over the short distance h.

8

In summary, Eq. (11) shows that the Euler method (4) is convergent because the accumulated truncation error tends to zero as h → 0. More generally if , for a given method, En

＝ O(hp) as h → 0, then the method is convergent if p ＞ 0, and we say that it is of order p. Thus, the Euler method is a first-order method.

Second, besides the truncation error that we have discussed there is also machine roundoff error, and that error can be expected to grow with the number of calculations.

Euler’s method is usually too inaccurate for serious computation because it is only a first-order method. To increase the accuracy, we need to make h extremely small. Why can’t we do that? There are two important reasons: One is too long the computation time and expense.

9

If we neglect the O(h2) term, then we have the Euler method

6.3 Improvements: Midpoint Rule and Runge-Kutta6.3.1. Midpoint rule.

Consider the Taylor’s formula with remainder,

2''( )( ) ( ) '( )( ) ( ) .

2!

yy x y a y a x a x a

(1)

Where ξ is some point in [a, x].

21

''( )( ) ( ) '( ) .

2!n n n

yy x y x y x h h

(2)

21( ) ( ) ( , ( )) ( ).n n n ny x y x f x y x h O h

(3)

1( ) ( ) ( , ( )) .n n n ny x y x f x y x h (4)

Let x =xn+1, a =xn, and x – a =xn+1 – xn =h

One way to obtain a higher-order method is to retain more terms in the Taylor;s formula, For instance,

10

If we truncate (6) by dropping the O(h3) term,2

1

1( , ) [ ( , ) ( , ) ( , )]

2n n n n x n n y n n n ny y f x y h f x y f x y f x y h (7)

With a local truncation error that is O(h3) and an accumulated truncation error that is O(h2); that is, we now have a second-order method.

3 3 3 2 2( ) ( ) ( ) ( ) ( ).nn n

xnhE O h n O h O h O h x O h

h h

2 31

''( ) '''( )( ) ( ) '( )

2 6n

n n n

y x yy x y x y x h h h

(5)

'' ( , ( )) ' ,x y x y

dy f x y x f f y f f f

dx

1( ) ( ) ( , ( ))n n n ny x y x f x y x h 2 31

[ ( , ( )) ( , ( )) ( , ( ))] ( ).2 x n n y n n n nf x y x f x y x f x y x h O h

(6)

since

Why second-order?

11

If the local truncation error is O(hp), with p ＞ 1, then the accumulated truncation error is O(hp-1), and one has a (p - 1)th-order method.

Although the second-order convergence of (7) is an improvement over the first-order convergence of Euler’s method. However, almost threefold increase in the computing time per step since it requires three function evaluations (f, fx, fy) per step whereas Euler’s method requires only one (f).

2 31

''( ) '''( )( ) ( ) '( ) ,

2 6n

n n n

y x yy x y x y x h h h

2 31

''( ) '''( )( ) ( ) '( ) ,

2 6n

n n n

y x yy x y x y x h h h

(8a)

(8b)

Let us return to Taylor’s formula. If we replace h by –h, that amounts to making a backward step so the term on the left will be y (xn-1) instead of y(xn+1). And we have,

12

31 1

'''( ) '''( )( ) ( ) 2 '( )

6n n n

y yy x y x y x h h

3( ) ( ) 2 ( , ( )) ( ).n n n ny x y x f x y x h O h

Subtracting Eq. (8a) from (8b), we have

or

Finally, if we drop the O(h3) term and change y(xn+1), y(xn-1), y(xn) to yn+1, yn-1, yn, respectively, we have

1 1 ( , )(2 )n n n ny y f x y h (9)

which method is known as the midpoint rule. Like Eq.(7), the midpoint rule is a second-order method but, unlike (7), it requires only one function evaluation per step. It is an example of a multi-step method because it uses information from more than one of the preceding points.

13

Example 1

Consider the equation

22 ; y(0)=1 (0 x< )y y x x (10)

with the exact solution y(x)=x2 +ex

Table 1

X Euler Midpoint Exact0.10 1.10000 1.11358 1.115170.20 1.22900 1.26072 1.261400.30 1.38790 1.43772 1.439860.40 1.57769 1.65026 1.651820.50 1.79946 1.89577 1.89872

(using h=0.1)

14

6.3.2. Second-order Runge-Kutta.

Observe that the Low-order Euler method yn+1＝ yn＋ f(xn, yn)h amounts to an extrapolation away from the initial point (xn, yn) using the slope f(xn,yn) at that point. Expecting an average slope to give greater accuracy, one might try the algorithm

1 1 1

1[ ( , ) ( , )] ,

2n n n n n ny y f x y f x y h (15)

which uses an average of the slopes at the initial and final

points. Since yn+1 appears both on the left-hand side and in the argument of f(xn+1, yn+1). Replace yn+1 by yn＋ f(xn, yn)h.

Then the revised version of Eq.(15) is

1 1

1{ ( , ) [ , ( , ) ]} .

2n n n n n n n ny y f x y f x y f x y h h (16)

15

and choosing the adjustable parameters a, b, α, β so as to make the order of the method (17) as high as possible; α, β determine the second slope location, and a, b determine the “weight” of the two slopes.

Guided initially by intuition we can put the idea on a rigorous basis by considering a method of the form

1 { ( , ) [( , ( , ) ]}n n n n n n n ny y af x y bf x h y f x y h h (17)

That is, we seek a, b, α, β so that the left- and right-hand

sides of

1( ) ( ) { [ , ( )]n n n ny x y x af x y x [ , ( ) [ , ( )] }n n n nbf x h y x f x y x h h

(18)

agree to as high a degree in h as possible.

16

2''( )( ) '( )

2n

n n

y xLHS y x y x h h

21( )

2 x yy fh f f f h (19)

2( ) ( ) .x yRHS y a b fh f ff bh (20)

1 11

2 2a b b and b (20, 21)

These are the Runge-Kutta methods of second order.For instance, with a＝ b＝ 1/2 and α＝ β＝ 1 we have

1 1

1{ ( , ) [ , ( , ) ]} ,

2n n n n n n n ny y f x y f x y f x y h h (22)

1 1 2

1 2 1 1

1( ),

2( , ), ( , )

n n

n n n n

y y k k

k hf x y k hf x y k

(23)

1 { ( , ) [( , ( , ) ]}n n n n n n n ny y af x y bf x h y f x y h h (17)

17

For this reason (23) is also known as the improved Euler

method. a＝ b＝ 1, α＝ β＝ 1/2, gives what is known as the

modified Euler method.

To understand this result, note that Euler’s method would give yn+1＝ yn＋ f(xn, yn)h. If we denote that Euler estimate as , then (22) can be expressed as

1Eulerny

1 11

( , ) ( , ).

2

Eulern n n n

n n

f x y f x yy y h

18

6.3.3. Fourth-order Runge-Kutta.

Using this idea of a weighted average of slopes at various points in the x, y plane, with the weights and locations determined so as to maximize the order of the method, one can derive higher-order Runge-Kutta methods as well. One of the most commonly used is the fourth-order Runge-Kutta method:

which that give without derivation.

1 1 2 3 4

1 2 1

3 2 4 1 3

1( 2 2 )

61

( , ), ( , ),2 2

1( , ), ( , ),

2 2

n n

n n n n

n n n n

y y k k k k

hk hf x y k hf x y k

hk hf x y k k hf x y k

(25)