1

Numerical analysis

2

Introduction

Numerical analysis is the study of algorithms that use

numerical approximation (as opposed to general symbolic

manipulations) for the problems of continuous mathematics

(as distinguished from discrete mathematics).

Numerical solution for differential equations

It is well known that it is too difficult to obtain the exact

solution for non linear differential equations. We overcome

this problem by using numerical methods for solving linear or

non linear differential equations approximately and in this

chapter; we will study Taylor, Picard and Euler methods for

solving first and higher order differential equations.

Taylor method

In this section, we will study Taylor method which is one of

numerical methods used for solving first and higher order

differential equations approximately.

3

1. Taylor for first order differential equation

y` = f(x,y) , y(x0) = y0 (1)

To solve the above differential equation using Taylor method,

we have to get at least the first three derivatives with respect

to x and put x = x0, y= y0 in y(x), y`(x), y``(x),… then

substitute in the equation

2 30 0 0

0 0 0 0(1) (2) (3)x x (x x ) (x x )

y(x) y y y y1

= + + - - -

(2)

Example 1: Solve the following differential equations

a) y` = Ln (x+y) , y(0) = 0.5

b) y` = 0.1(x3+ y2) , y (0) = 1

Solution

a) (1)0y Ln 0.5 = -0.693147 , x0 = 0 , y0 = 0.5

y``= 1+ yx y

+,

0 0

(1)(2) 00

1 yy

x y

+, therefore

(2)0

1 0.693147y 0.6137060 0.5

-

4

(2) (1)0 0 0 0

0 0

22 y (x y )- (1 y )y (x y)- (1 y )y(x y) (x y )

2 2++

+++ +

, therefore

2

2(3)0

0.613706(0 0.5 (1 0.693147 )y 0.85078(0 0.5)

)- -+

+

Substitute in equation (2) , we will get :

0

0

) )(x -0) (x - )y(x) 1.5 ( 0.693147 (0.6137061 2

(x - ) 0.85078)3

(

2

3

= - +

+

b) 0(1) 3 20 0 0 00.1y 0.1(x + y ) 0.1(0 +1) x 0 y 1 ,= ,

y`` = 0.1(3x2+2y y`), (2)0y = 0.1(3 2

0x +2 y0(1)0y ) , therefore

(2)0y = 0.1[3(0)+2(1)(0.1)]=0.02

y``` = 0.1(6x+2 (y`)2+ 2 y y``), (3)0y = 0.1(6x0+2 ( (1)

0y )2+ 2 y0

(2)0y ) , therefore (3)

0y = 0.1[0+2 (0.1)2+ 2(1)(0.02)] = 0.006

Substitute in equation (2) , we will get :

(x -0) (x -0) (x -0)y(x) =1 (0.1) (0.02) (0.006)1 2 3

2 3

+ +

5

2. Taylor for higher order differential equation

Consider g(y(n) , y(n-1) , y(n-2) , …., y`) = f(x,y), and its

boundary condition is (i)0

i0,y (x ) y i 0,1,..., n 1 , where x0

and i0y are given constants. To solve the higher order

differential equation , we have to convert it to a system of n-

first order differential equation , by putting y` = z , y`` = z` =

w , y```= z`` = w` = p and so on, then find at least the first

three derivatives for y, z, w, p,…

Example 2: Solve the following differential equations

y`` +3 xy` - 6 y = 0, y(0) = 1, y`(0) = 0.1

Solution

Put y` = z , y`` = z` and y0 = 1, (1)0y = z0 = 0.1, x0 = 0 such

that y` = z, z`= -3 xy` + 6 y and y`` = z`, thus (2)0y =

(1)0z = -3 x0

(1)0y + 6 y0 = 6, y```= z``= 3 y`- 3 xy``, hence (3)

0y =

(2)0z = 3 (1)

0y - 3x0(2)0y = 0.3. Substitute in equation (2) , we

will get :2 30 0 0) ) )(x - ) (x - ) (x - )y(x) 1 (0.1 (6 (0.3

1 2 3 = + +

6

Picard method (method of successive approximation)

In this section, we will study Picard method which is

successive approximation method used in getting first,

second, third, approximations for first and higher order

differential equations.

1. Picard for first order differential equation

Consider the first order differential equation y` = f(x,y) ,

y(x0) = y0 , from this D.E., we have dy = f(x,y) dx and by

integration, we will get:0 0

y x

y xdy f(x y) dx ,

y - y0 =0x

xf(x y) dx , , therefore y = y0 +

0x

xf(x y) dx , , (3)

If (x- x0) is small enough, y will not be much different from

y0 in that integral. We can as a first approximation, substitute

y0 for y in the integrand: y1 = y0 + 0

0

x

xf(x y ) dx ,

7

Substitute y = y1 in the integral formula (3) to get the second

approximation y2 , such that: y2 = y0 + 1

0

x

xf(x y ) dx ,

Repeat this process to obtain the third, the

fourth,…approximations until the required accuracy is

obtained. In general we can write the (n+1)th approximations

in the form: yn+1 = y0 + n

0

x

xf(x y ) dx , (4)

Example 3: Solve by Picard’s method the D.E. y` = x2+ y2 ,

y (0) = 0 up to third approximation.

Solution

Since x0 = 0 , y0 = 0 , f(x,y) = x2+ y2 and according to

formula (4), such that: yn+1 = y0 + n

0

x

x

2 2)(x + y dx

Put n=0 to obtain first approximation, so that:

y1 = y0 + 0

0

x

xf(x y ) dx , = 0 +

x

0

0f(x ) dx , =x

2 2

0

)(x + 0 dx =3x

3

8

Put n=1 to obtain second approximation, so that:

y2 = y0 +0

1

x

xf(x y ) dx , = 0+

3x

0

xf(x ) dx3 , =

62

x

0)x(x + dx

9 =3x

3

+7x

63

Put n = 2 to obtain third approximation, so that:

y3 = y0+0

2

x

xf(x y ) dx , =

3 7x

0+x xf(x ) dx

3 63 ,

=3 7

2 2x

0)x x(x + ( + ) dx

3 63 =6 10 14

2x

0

x 2x x(x )dx9 189 3969 + + +

=3 7 11 15x x 2x 13x

3 63 2079 218295+ + +

2. Picard for higher order differential equation

Consider g(y(n) , y(n-1) , y(n-2) , …., y`) = f(x,y), and its

boundary condition (i) i0 0,y (x ) y i 0,1,..., n 1 , where x0

and i0y are given constants. To solve the higher order

9

differential equation using Picard successive approximation,

we have to apply the following steps

1- Convert the differential equation of order n to system of n-

first order D.E. as we put y`= z and y``= z`= w and so on,

such that (1)0 0y (x ) z and (2)

0 0y (x ) w and so on.

2- Arrange the system of n – first order differential equations

such that

y`= z , y``= z`= w, …..., y(n) = z(n-1) = w(n-2) =…= R(x, y, z,

w,…)

3- Modify formula (4) to be used for multi variables such

that: yn+1 = y0 +0

n

x

xz dx

To evaluate first approximation as we put n=0, then

y1 = y0 +0

0

x

xz dx , where (1)

0 0y (x ) z is a given constant, but

to evaluate the second approximation y2 as we put n=1, we

10

have to calculate the first approximation z1 by putting n = 0

in the following formula:

zn+1 = z0 +0

n

x

xw dx , therefore to obtain (n+1)th

approximation yn+1 , we have to get nth approximation zn.

Example 4: Use Picard’s method up to third approximation

the following differential equation y`` = x3 (y`+ y ), y(0) = 1,

y`(0) = ½

Solution

Put y`= z, therefore y``= z` = x3 (z + y) = f(x, y, z), given that

x0 = 0, y0 = 1, y`(0) = z0 = ½.

According to the following two formulas, we can get the first

three approximations yn+1 = y0 +0

n

x

xz dx ,

zn+1 = z0 +00

3n n n n

x x

xxf(x, y , z ) dx x (y + z ) dx =

Put n=0 to obtain y1, z1 such that:

11

y1 = y0 +0

0

x

xz dx = 1 +

x

0

dx2 =1+ x

2

And

z1 = z0+0

30 0 0 0

x x

x 0

1f(x, y , z ) dx x (y + z ) dx2 =

3x

0

1 1x (1 + ) dx2 2 = = 1

2+

43x8

Put n=1 to obtain y2, z2 such that

y2 = y0 +0

1

x

xz dx = 1+

4x

0( )3x 1 dx

8 2 = 1+ x

2+

53x40

And

z2 = z0 +0

31 1 1 1

x x

x 0x1f(x, y , z ) dx (y z ) dx

2 + =

43

x

0 ( + )1 x 1 3xx 1 + dx

2 2 2 8 = = 1

2+

43x8

+5x

10+

83x64

,

12

Therefore y3 = y0 +0

2

x

xz dx = 1+

4x

0(3x 1 ) dx

8 2

= 1+4 5 8x

0( )3x 1 x 3x dx

8 2 10 64

= 1+53x

40+ x

2+

6x60

+9x

192

Euler method

The Euler method is a first-order numerical procedure for

solving ordinary differential equations (ODEs) with a given

initial value. It is the most basic explicit method for

numerical integration of ordinary differential equations and is

the simplest Runge–Kutta method. The Euler method is

named after Leonhard Euler

13

1. Euler for first order differential equation

Let’s start with a general first order IVP:

dy f(t,y)dt , y (t0) = y0 (5)

Where f(t,y) is a known function and the values in the initial

condition are also known numbers. If f and fy are continuous

function then there is a unique solution to the IVP in some

interval surrounding t = t0 . So, let’s assume that everything

is nice and continuous so that we know that a solution will in

fact exist. We want to approximate the solution to (5) near

t = t0. First, we know the value of the solution at t = t0 from

the initial condition. Second, we also know the value of the

derivative at t = t0 . We can get this by plugging the

initial condition into f(t,y) into the differential equation

itself. So, the derivative at this point is0t=t

dydt

= f(t0, y0), so

the tangent line to the solution at t = t0 is

y = y0 + f(t0, y0)( t - t0)

14

If t1 is close enough to t0 then the point y1 on the tangent line

should be fairly close to the actual value of the solution at t1,

or y(t1). Finding y1 is easy enough. All we need to do is plug

t1 in the equation for the tangent line.

y1= y0 + f(t0, y0)( t1 - t0)

So, let’s hope that y1 is a good approximation to the solution

and construct a line through the point (t1, y1) that has slope f

(t1, y1). This gives

y= y1 + f(t1, y1)( t – t1)

Now, to get an approximation to the solution at t = t2 we will

hope that this new line will be fairly close to the actual

solution at t2 and use the value of the line at t2 as an

approximation to the actual solution. This gives

y2= y1 + f(t1, y1)( t2 – t1)

We can continue in this fashion. Use the previously

computed approximation to get the next approximation. So,

15

y3= y2 + f(t2, y2)( t3 – t2)

etc

In general, if we have tn and the approximation to the solution

at this point, yn, and we want to find the approximation at tn+1

all we need to do is use the following.

yn+1 = yn + f(tn, yn)( tn+1 – tn) (6)

Often, we will assume that the step sizes between the points

t0 , t1 , t2 , …, tn are of a uniform size of h. In other words, we

will often assume that tn+1 – tn= h

This doesn’t have to be done and there are times when it’s

best that we not do this. However, if we do the formula for

the next approximation becomes.

yn+1 = yn + h f(tn, yn) (7)

Example 5: Find y(0.3) for the D.E. y` = 3x+ y2, y (0) = 1

using Euler method, h=0.1.

16

Solution: Given x0 = 0, y0 = 1, h=0.1, xi+1 = xi + h and Euler

equation state that n n nn+1 )y y h f( x y , .

So that we can modify Euler equation according to the

example such that 2n n nn+1 )y y 0.1(3x y +

From which we get the first approximation at n = 0 such that

2 21 0 0 0) )y y 0.1(3x y =1+ 0.1(3(0) +1 1.1 y(0.1) +

Similarly we can get the second approximation at n = 1 such

that

2 22 1 1 1 ) )y y 0.1(3x y =1+ 0.1(3(0.1) + (1.1) 1.251 y(0.2) +

Thus the third approximation y(0.3) at n = 2 will be obtained

as follows

22 2

3 2 2) = )y y 0.1(3x y 1+ 0.1(3(0.2) + (1.251) 1.467 y(0.3) +

17

2. Euler for higher order differential equation

Consider g(y(n) , y(n-1) , y(n-2) , …., y`) = f(x,y), and its

boundary condition (i) i0 0,y (x ) y i 0,1,..., n 1 , where x0

and i0y are given constants. To solve the higher order

differential equation using Euler method, we have to apply

the following steps

1- Convert the differential equation of order n to system of n-

first order D.E. as we put y`= z and y``= z`= w and so on,

such that (1)0 0y (x ) z and (2)

0 0y (x ) w and so on.

2- Arrange the system of n – first order differential equations

such that y`= z , y``= z`= w,…, y(n) = z(n-1) = w(n-2) =…= R(x,

y, z, w,…)

3-According to Euler method: n nn+1y y h z ,

n nn+1z z h w and so on

18

Example 6: Use Euler method to find y(1.2) for the

following differential equation y`` = x3 (y`+ y ) , y(1) = 1 ,

y`(1) = ½ , h = 0.1.

Solution

Put y`= z, therefore y``= z` = x3 (z + y) = f(x, y, z), given that

x0 = 1, y0 = 1, y`(0) = z0 = ½, therefore n nn+1y y 0.1z &

3n n nnn+1z z 0.1x (z + y ) , thus y1 = y(1.1) = y0 + 0.1 z0 = 1.05,

and 30 0 01 0z z 0.1x (z + y ) =0.65, y2 = y(1.2) = y1 + 0.1 z1 =

1.115

Runge-Kutta Method

These methods have high - order local truncation error while

eliminating the need to compute and evaluate the derivatives.

1. Runge-Kutta for first order differential equation

For any ordinary differential equation of first order in the

form y` = f(x,y), y(x0) = y0, a< x< b, h = (b-a)/N can be

solved using Runge-Kutta methods and it is classified into

different types.

19

1. Mid point Method

i+1 i i i i iy y hf (x +h/2,y +(h/2)f(x ,y )) (8)

2. Modified Euler Method

i+1 i i i i+1 i i iy y (h/2)[f(x ,y ) f (x ,y +hf(x ,y ))] (9)

3. Heun’s Method

i+1 i i i i i i iy y (h/4)[f(x ,y ) 3f (x +(2/3)h,y +(2/3)hf(x ,y ))]

(10)

Both are classified as Runge-Kutta methods of order two, the

order of the local truncation error.

4. Runge-Kutta Order Four

k1 = hf(xi, yi), k2 = hf(xi+h/2, yi+ k1/2),

k3 = hf(xi+h/2, yi+ k2/2), k4 = hf(xi+1, yi+ k3),

1i+1 i 2 3 4y y (1/6)[k +2k +2k +k ]

(11)

20

Example 7:

Solve the differential equation y` = y - x2 + 1, y(0) = 0.5,

h = 0.2. Apply Runge-Kutta methods of order two to get

y(0.4).

Solution:

Let y - x2 + 1 = f(x,y), x0 = 0, y0 = 0.5, h=0.2, xi+1 = xi + h &

mid point method: i+1 i i i i+iy y hf (x +h/2,y (h/2)f(x ,y )) ,

modified Euler method:

i+1 i i i i+1 i i iy y (h/2)[f(x ,y ) f (x ,y +hf(x ,y ))] ,

And Heun’s method:

i+1 i i i i i i iy y (h/4)[f(x ,y ) 3f (x +(2/3)h,y +(2/3)hf(x ,y ))]

i xi xi+h/2 yi+h/2 i if(x , y ) f[xi+h/2, yi+h/2 i if(x , y ) ] yi

0 0 0.1 0.65 1.64 0.5

1 0.2 0.3 1.0068 1.9168 0.828

2 0.4 0.5 1.21136

Solution of D.E. (y` = y - x2 + 1) using mid point method

21

i xi xi+1 yi+h i if(x ,y ) f(xi+1, yi+h i if(x ,y ) yi

0 0 0.2 0.8 1.76 0.5

1 0.2 0.4 1.1832 2.0232 0.826

2 0.4 0.6 1.20692

Solution of D.E. (y` = y - x2 + 1) using modified Euler

i xi xi+2h/3 yi+2h i if(x , y ) /3 f(xi+2h/3, yi+2h i if(x , y ) /3 yi

0 0 0.1333 0.7 1.6822 0.5

1 0.2 0.3333 1.0656 1.9545 0.8273

2 0.4 0.5333 1.2098

Solution of D.E. (y` = y - x2 + 1) using Heun’s method

We can solve the problem more faster by substituting

y-x2 + 1 = f(x,y) in the mid point formula such that:

i+1 i i i i iy y hf (x +h/2,y +(h/2)f(x ,y ))

= 2i i i i iy 0.2f (x +0.1,y +0.1[y -x + 1])

= 2 2i i i iy 0.2[[1.1y -0.1x +0.1] -[x +0.1] 1]

22

Similarly if we substitute y - x2 + 1 = f(x,y) in the modified

Euler formula such that:

i+1 i i i i+1 i i iy y (h/2)[f(x ,y ) f (x ,y +hf(x ,y ))]

= 2 2i i i i+1 i i iy 0.1[y -x + 1 f (x ,y +0.2[y -x + 1])]

= 2 2 2i i i i i i+1y 0.1[y -x + 1 [1.2y 0.2x 0.2 x 1]]

If we substitute y-x2 + 1 = f(x,y) in the Heun’s formula, we

get:

i+1 i i i i i i iy y (h/4)[f(x ,y ) 3f (x +(2/3)h,y +(2/3)hf(x ,y )] =

2 2i i i i i i iy (0.05)[y -x + 1 3f (x +(0.4/3),y +(0.4/3)[y -x + 1])]

= 2 2i i i i iy (0.05)[y -x + 1 3 [(3.4/3)y -(0.4/3)x (0.4/3)

2i-[x +(0.4/3)] 1]]

The above formulas based on the differential equation.

23

Example 8:

Repeat the above problem by applying Runge-Kutta of order

four

Solution:

k1 = hf(xi, yi) = 0.2[ 2i iy - x +1],

k2 = hf(xi+h/2, yi+ k1/2) = 0.2[yi+ k1/2- [xi+0.1]2 + 1],

k3 = hf(xi+h/2, yi+ k2/2) = 0.2[yi + k2/2- [xi+0.1]2 + 1],

k4 = hf(xi+1, yi+ k3) = 0.2[yi + k3 - [xi+1]2 + 1],

1i+1 i 2 3 4y y (1/6)[k +2k +2k +k ]

i xi k1 k2 k3 k4 yi

0 0 0.3 0.328 0.3308 0.35816 0.5

1 0.2 0.3579 0.3836 0.3862 0.4111 0.8293

2 0.4 1.2141

Solution of D.E. (y` = y - x2 + 1) using Runge-Kutta of

order four

24

2. Runge-Kutta for higher order differential equation

To solve the higher order differential equation using Runge-

Kutta methods, we have to convert the differential equation

of order n to system of n- first order D.E. as we put y`= z

and y``= z`= w and so on.

1. Mid point Method

i+1 i i i i i

i+1 i i i i i i i i

y y h[z +h/2w(x ,y ,z )],

z z hw(x +h/2,y +(h/2)z ,z +h/2w(x ,y ,z ))

and so on (12)

2-Modified Euler Method

i+1 i i i i i

i+1 i i i i i+1 i i i i i i

y y (h/2)[2 z +hw(x ,y ,z )]

z z (h/2)[w(x ,y ,z ) w(x ,y +hz ,z +hw(x ,y ,z ))]

and so on (13)

25

3-Heun’s Method

i+1 i i i i i

i+1 i i i i

i i i i i i i

y y (h/4)[4z 2hw(x ,y ,z )]]

z z (h/4)[w(x ,y ,z )

3w(x +(2/3)h,y +(2/3)hz ,z +(2/3)hw(x ,y ,z )) ]

and so on (14)

4- Runge-Kutta Order Four

k1y =hzi, k1z = hw(xi, yi, zi), k2y = h[zi+ k1z/2],

k2z =hw(xi+h/2, yi +k1y/2, zi+ k1z/2), k3y=h[zi + k2z/2],

k3z = hw(xi+h/2, yi+ k2y/2, zi+ k2z/2) k4y = h[ zi+ k3z],

k4z = hw(xi+1, yi+ k3y, zi+ k3z),

1yi+1 i 2y 3y 4yy y (1/6)[k +2k +2k +k ] ,

1zi+1 i 2z 3z 4zz z (1/6)[k +2k +2k +k ] . (15)

26

Example 9: Find y(0.2) for the following differential

equation using Runge- Kutta of 2nd order and 4th order, given

h=0.1.

y`` +3 xy` - 6 y = 0, y(0) = 1, y`(0) = 0.1

Solution:

Put y` = z , y`` = z` and y0 = 1, (1)0y = z0 = 0.1 , x0 = 0 such

that y` = z, z`= -3 xy` + 6 y = -3 xz + 6 y = w(x,y,z).

First we will use mid point method such that:

i+1 i i i i iy y h[z +h/2w(x ,y ,z )]

= i i i i iy 0.1[z +0.05(-3 x z + 6 y ) ]

= i i i1.03y 0.1z (1-0.15x )

i+1 i i i i i i i iz z hw(x +h/2,y +(h/2)z ,z +(h/2)w(x ,y ,z ) )

= i i i i i i i iz 0.1w(x +0.05,y +0.05z ,z +0.05(-3 x z + 6 y ) )

= i i i i i i i iz 0.3( x +0.05)[z +0.05(-3 x z +6 y )] + 0.6 (y +0.05z )

From the above derived formula depending on the differential

equation, we will get y(0.2) more easy and fast such that:

27

1y = 0 0 01.03y 0.1z (1-0.15x ) = 1.03 0.01= 1.04 = y(0.1)

1z = 0 0 0 0 0 0 0z 0.3( x +0.05)[0.3y +(1-0.15x )z ]+0.6 (y +0.05z )

= 0.1 -0.015[0.3+0.1] + 0.6(1+0.005) = 0.697

2 1 1 1y =1.03y 0.1z (1-0.15x )

= 1.03(1.04) (0.0697)(1-0.015) 1.14 = y(0.2)

Second, we will use modified Euler method such that:

i+1 i i i i iy y (h/2)[2z hw(x ,y ,z )]

= i i i i iy 0.05 [2z 0.1(-3 x z + 6 y )]

= 1.03 iy +0.1 i iz (1-0.15x )

i+1 i i i i i+1 i i i i i i,z z (h/2)[w(x ,y ,z ) w(x ,y +hz z +hw(x ,y ,z ) )]

= i i i i i+1 i i i i i i,z 0.05 [-3x z +6y w(x ,y +0.1z z +0.1(-3x z +6 y ))]

= i i i i i+1 i i i i i iz 0.05[-3x z +6y -3x [z +0.1(-3 x z +6y )]+6[y +0.1z ]]

At i = 0, 1y = 0 0 01.03y 0.1z (1-0.15x ) = 1.03 0.01= 1.04

= y(0.1)

28

1 0 0 0 0 1 0

0 0 0 0 0

z z 0.05 [-3 x z +6 y -3x [z

+0.1(-3x z +6 y )]+6[y +0.1z ]]

0.1 0.05 [6 -0.3 [0.1+0.6] + 6 [1+0.01]] = 0.6925

2 1 1 1y =1.03y 0.1z (1-0.15x )

= 1.03(1.04) (0.06925)(1-0.015) 1.14 = y(0.2)

Third, we will use Heun’s method such that:

i+1 i i i i iy y (h/4)[4z 2hw(x ,y ,z )]

= i i i i iy 0.025[4z 0.2(-3x z +6y )]

= 1.03 1y + 0.1 i iz (1-0.15x )

i+1 i i i i

i i i i i i i

z z (h/4)[w(x ,y ,z )

3w(x +(2/3)h,y +(2/3)hz ,z +(2/3)hw(x ,y ,z ))]

i i i i

i i i i i i i

z 0.025 [(-3x z +6y )

3w(x +0.067,y +0.067z ,z +0.067(-3x z +6y ))]

29

i+1 i i i i

i i i i i i i

z z 0.025[(-3x z +6y )

3 [-3(x +0.067)[z +0.067(-3x z +6y )]+6[y +0.067z ]]]

At i = 0,

1y = 0 0 01.03y 0.1z (1-0.15x ) =1.03 0.01= 1.04 = y(0.1)

1 0 0 0 0

0 0 0 0 0

0 0

z z 0.025 [(-3x z +6y )

3 [-3(x +0.067)[z +0.067(-3x z +6y )]

+ 6 [y +0.067z ]]]

1 0z z 0.025 [6-(0.603)(0.502)+18(1.0067)] 0.6954

2 1 1 1y =1.03y 0.1z (1-0.15x )

=1.03(1.04) (0.06954)(1-0.015) 1.14 = y(0.2)

30

Problems on numerical solution of D.E.

1) Solve the following differential equations using Taylor

method

i) y` = x2- y2 , y (0) = 0 ,

ii) y` = xy+ y2 , y (0) = 1

iii) y` = x y , y (2) = 4, then find y (2.5)

iv) y`` + xy` - 2y = 2, y (1) = 1, y` (1) = 2

2) Solve the following differential equations using Picard

method up to third approximation.

i) y` = xy+ y2 , y (0) = 1

ii) y` = x y y = 0.81 when x = 0.36

iii) y` = y +1x

, y =3 at x = 2, then find y (2.8)

iv) y`` - xy` - y = 2, y (0) = 0, y` (0) = 1

31

3) Solve the following differential equations using Euler

method and Runge-Kutta methods of order two and four.

i) y` = exy, y (0) = 1, h=0.2, find y (1).

ii) y` = Ln( x y )2+ , y (0.5) = 1.5 , h=0.05, find y (0.7) .

iii) y` + 2 y = 2 – e-4t , y (0) = 1

iv) y` = 50 (y-1)2 (y-5), y(0) = 1.1

v) y` - y = -0.5 et/2 sin(5t) + 5 et/2 cos(5t), y(0) = 0

vi) 2t2y`` + ty` - 3y = 0, y(0) = 1, y(0) = 0.1

Use Euler’s Method with a step size of h = 0.1 to find

approximate values of the solution at t = 0.1, 0.2, 0.3, 0.4,

and 0.5. Compare them to the exact values of the solution as

these points.

32

References

1) www.math.jmu.edu/~jim/expository.pdf

2) livetoad.org/Courses/Documents/.../picard_iterates.p

3) www4.ncsu.edu/eos/users/w/white/.../chap7.1.PDF

4) math.fullerton.edu/mathews/.../TaylorDEMod.html

5) www.mcs.sdsmt.edu/.../ugr_TaylorSeriesODE.pdf

6) www.cliffsnotes.com/.../Taylor-Series.topicArticleId

7) en.wikipedia.org/wiki/Euler_method

8) tutorial.math.lamar.edu/Classes/.../EulersMethod.asp

9) en.wikipedia.org/wiki/Runge–Kutta_methods

33

Finding Roots

What is a "root"? A root is a value for which a given

function equals zero. When that function is plotted on a

graph, the roots are points where the function crosses the

x-axis. For a function, f(x), the roots are the values of x for

which f(x)=0. For example, with the function f(x)=2-x, the

only root would be x = 2, because that value produces f(x)=0.

Of course, it's easy to find the roots of a trivial problem like

that one, but what about something crazy like this:

(2x 3)(x 3)yx(x 2)

Steps to find roots of rational functions

Set each factor in the numerator to equal zero.

Solve that factor for x.

Check the denominator factors to make sure you aren't

dividing by zero!

34

Numerator Factors

Remember that a factor is something being multiplied or

divided, such as (2x-3) in the above example. So, the two

factors in the numerator are (2x-3) and (x+3). If either of

those factors can be zero, then the whole function will be

zero. It won't matter (well, there is an exception) what the

rest of the function says, because you're multiplying by a

term that equals zero.

So, the point is, figure out how to make the numerator zero

and you've found your roots (also known as zeros, for

obvious reasons!). In this example, we have two factors in the

numerator, so either one can be zero. Let's set them both

equal to zero and then solve for the x values:

2x - 3 = 0 2x = 3 x = 3/2 and x + 3 = 0 x = -3

So, x = 3/2 and x = -3 become our roots for this function.

They're also the x-intercepts when plotted on a graph,

because y will equal 0 when x is 3/2 or -3.

35

Denominator Factors

Just like with the numerator, there are two factors being

multiplied in the denominators. They are x and x-2. Let's set

them both equal to zero and solve them:

x = 0 and x - 2 = 0 x = 2

Those are not roots of this function. Look what happens

when we plug in either 0 or 2 for x. We get a zero in the

denominator, which means division by zero. That means the

function does not exist at this point. In fact, x = 0 and x = 2

become our vertical asymptotes (zeros of the denominator).

So, there is a vertical asymptote at x = 0 and x = 2 for the

above function.

A root-finding algorithm is a numerical method, or

algorithm, for finding a value x such that f(x) = 0, for a given

function f. Such an x is called a root of the function f.

This article is concerned with finding scalar, real or complex

roots, approximated as floating point numbers. Finding

integer roots or exact algebraic roots are separate problems,

36

whose algorithms have little in common with those discussed

here. (See: Diophantine equation as for integer roots)

Finding a root of f(x) − g(x) = 0 is the same as solving the

equation f(x) = g(x). Here, x is called the unknown in the

equation. Conversely, any equation can take the canonical

form f(x) = 0, so equation solving is the same thing as

computing (or finding) a root of a function.

Numerical root-finding methods use iteration, producing a

sequence of numbers that hopefully converge towards a limit

(the so called "fixed point") which is a root. The first values

of this series are initial guesses. The method computes

subsequent values based on the old ones and the function f.

The behavior of root-finding algorithms is studied in

numerical analysis. Algorithms perform best when they take

advantage of known characteristics of the given function.

Thus an algorithm to find isolated real roots of a low-degree

polynomial in one variable may bear little resemblance to an

algorithm for complex roots of a "black-box" function which

is not even known to be differentiable. Questions include

37

ability to separate close roots, robustness in achieving reliable

answers despite inevitable numerical errors, and rate of

convergence.

Example 10 : Find the roots of f(x) = x2 - 5x + 4 .

Solution: Since we can factor f as f(x)=(x-4)(x-1), we see

that the solutions of f(x)=0 are exactly x=4 and x=1. These

are the desired roots.

Example11: Find the roots of f(x) = x2 - 6x + 4 .

Solution: While f does not factor "nicely", we do have the

quadratic formula at our disposal. The two roots of f are

given by

2( 6) ( 6) 4(1)(4) 6 20x 3 522(1)

So our two roots can be written as x = 3+ 5 and x = 3- 5

Before leaving this example, one question should be

considered. What does 5 look like? What is its decimal

representation? The answer is that it has an infinite, non

38

repeating decimal representation. Hence, we can at best write

down an approximation of 5 as a decimal number. We will

consider such a task below. Admittedly, factoring and the

quadratic formula are valuable tools for finding roots when

they can be used. However, there are many functions whose

roots cannot be found by applying these two tools. Take, for

example, p(x) = x7 + 9 x 5 - 13 x - 17.

How does one find (or approximate) the roots of this

function?

Bisection method

One of the two methods for finding roots that will be

discussed here is the bisection method. We illustrate this

method by considering the above-mentioned polynomial

p(x) = x 7 + 9 x 5 - 13 x - 17.

Note that p(0)=-17 and p(2)=373. Therefore, since p(x) is a

continuous function (i.e., its graph has no ``breaks''), we

know that there must be a root, say r, in the interval (0, 2).

39

To close in on r, we now evaluate p at the midpoint of (0, 2)

which is 1) p(1) = -20. Now we see that r must actually lie in

the interval (1, 2) since p switches signs from negative to

positive as x ranges from 1 to 2 So we have reduced the

interval under consideration from (0, 2) to (1, 2). We have cut

the length of our interval in half, or bisected it.

We look next at the midpoint of (1, 2), namely 1.5. p(1.5)

48.929 . Thus, r must be in the interval (1, 1.5). We continue

this procedure until a desired accuracy has been achieved.

The following table summarizes the results of iterating this

technique several times.

Left Endpoint Right Endpoint Midpoint p(Midpoint)0111.251.251.251.251.251.2578125

221.51.51.3751.31251.281251.2656251.2656251.26171875

11.51.251.3751.31251.281251.2656251.25781251.261718751.259765625

-2048.929-1.01518.6517.7013.0860.974-0.0350.4650.213

40

Note that after these iterations are performed, we see that

1.2578125 < r < 1.259765625

Hence, we have been able to find r accurate to the hundredths

place.

One question that should be raised is the following: Does this

method always converge to a root? The answer is yes,

provided the function under consideration is continuous and

we begin with two values a and b such that p(a) < 0 and

p(b) > 0. (The Intermediate value Theorem guarantees that a

root exists under these conditions.)

Newton’s method

Newton's method assumes the function f to have a continuous

derivative. Newton's method may not converge if started too

far away from a root. However, when it does converge, it is

faster than the bisection method, and is usually quadratic.

Newton's method is also important because it readily

generalizes to higher-dimensional problems. Newton-like

methods with higher order of convergence are the

41

Householder's methods. The first one after Newton's method

is Halley's method with cubic order of convergence.

In numerical analysis, Newton's method (also known as the

Newton–Raphson method), named after Isaac Newton and

Joseph Raphson, is a method for finding successively better

approximations to the roots (or zeroes) of a real-valued

function. The algorithm is first in the class of Householder's

methods, succeeded by Halley's method. The method can also

be extended to complex functions and to systems of

equations.

The Newton-Raphson method in one variable is implemented

as follows:

Given a function ƒ defined over the reals x, and its derivative

ƒ ', we begin with a first guess x0 for a root of the function f.

Provided the function is reasonably well-behaved a better

approximation x1 is

01 0

0

f (x )x x

f '(x )

or 01 0

0

f (x )x x

f '(x )

42

Iterating this yields the general term

nnn+1

n

f (x )x x f '(x )

, n = 0, 1, 2, …

Example 12: Consider p(x) = x7 + 9x5 - 13x - 17 once again.

We have seen from the bisection method that one root of p

lies in the interval (1, 2). Using x0 = 1 as our seed value, we

can generate the following table via Newton's method.

n xn p(xn ) p'(xn ) xn+1

0

1

2

3

4

5

1

1.512820513

1.340672368

1.269368397

1.258332053

1.258092767

-20

52.78287188

12.33751268

1.46911353

0.03053547

0.00001407

39

306.6130739

173.0270062

133.1159618

127.6107243

127.4932403

1.512820513

1.340672368

1.269368397

1.258332053

1.258092767

1.258092657

Note that after 6 iterations of Newton's method, the root r can

be approximated as r = 1.258092657, accurate to 6 decimal

places.

43

Conclusion

In comparing the results of the previous two sections, we see

that Newton's method appears to converge to the root r much

more rapidly than the bisection method. In general, this is

indeed the case. So why even consider the bisection method?

The answer is that Newton's method does not always

converge to the root, while the bisection method always does

(so long as the function f is continuous and one begins with

values a and b such that f(a) and f(b) are of different sign). A

"poor" choice for the seed value x0 may cause Newton's

method to "miss" the root or run away from it. A quick

example is in order here.

Example 13: We begin with the function f(x) =2

ln xx

Note that f has one root at x=1 since ln 1 = 0 . The following

table gives the results of applying Newton's method to this

function with seed value x0 = 2 :

44

n xn f(xn ) f'(xn ) xn+1

0

1

2

3

4

2

5.588699452

9.527574062

15.64919197

25.21226310

0.1732867952

0.05509287184

0.02483281062

0.1123091313

0.00507714751

-0.0482867952

-0.01398695752

-0.004056576382

-0.00117440443

-0.000340355109

5.588699452

9.527574062

15.64919197

25.21226310

40.12946981

Note that the estimates for the root are getting larger and

larger. They will, in fact, continue to do this. The root r=1 is

being completed missed by this. The issue of determining

exactly when Newton's method does converge to a root has

been studied and documented by many. I refer the reader to a

first-year calculus text for such discussion.

Iteration method

Suppose that you can bring an equation g(x)=0 in the form

x = f(x). We'll show that you can solve this equation, on

certain conditions, using iteration.

Start with an approximation x0 of the root. Calculate

x0,x1,...,xn,... such that x1=f(x0); x2 = f(x1) ; x3 = f(x2); ...

45

Theorem1:

If the sequence x0,x1,...,xn, ... belongs to an interval I, where

|f '(x)| < k < 1 , then the sequence has a limit L and L is the

only root of x=f(x) in the interval I.

Proof:

Appealing on Lagrange's theorem we can write:

x2 - x 1 = f(x 1) - f(x 0) = (x1 - x0).f'(c1) with c1 between x0 &

x1 and x3 - x2 = f(x 2) - f(x 1) = (x2 - x 1).f '(c2) with c1 between

x1 and x2, similarly xn+1- xn = f(xn) - f(xn-1)=( xn - xn-1).f'(cn)

with cn between xn and x n-1.

Since |f '(x)| < k < 1, therefore | x2 - x1 | < k.| x1 - x0| and

|x3 - x2 | < k.|x2 - x1|, similarly |x n+1- xn| < k.|xn - xn-1|,

Multiplying each side | x n+1- x n| < kn . | x1 - x0|.

|x n+p- x n| = |xn+p - x n+p-1 + x n+p-1 - x n+p-2 + ... + x n+1- x n |=>

|xn+p-xn| = < |xn+p - xn+p-1| + |xn+p-1 - xn+p-2| + ... + |xn+1-xn |,

therefore |xn+p-xn| < |x1 - x0|.(kn+p-1 + kn+p-2+...+kn), thus |xn+p -

xn| < |x1 - x0|. (kn - kn+p)/(1-k) => |xn+p-xn| < |x1 - x0|. kn/(1-k).

46

Since k< 1 is kn/(1-k) smaller than each strictly positive

number e if n is sufficiently large and p = 1,2,3, ...

Therefore we conclude with the Criterion of Cauchy that the

sequence {xn} converges. Say the limit is L.

xn = f(xn-1) => lim xn = lim f(xn-1) => L = f(L)

Thus, L is a root of the equation x = f(x).

If M is a second root of x = f(x) in interval I, then M - L =

f(M) - f(L) = (M - L).f'(c) with c in I, then f'(c) = 1 and this

gives a contradiction.

Example 14: Solve x = 2 + sin(x)/2 by the iteration method

Solution:

Let f(x) =2 + sin(x)/2 and f`(x) = cos(x)/2 and the condition

|f`(x)| < k < 1 is satisfied for all x. Starting with x0 = 2, we

calculate x1,x2,... as shown by the following table

47

xn 2 + sin(xn)/22 2.31708861973148

2.31708861973148 2.367105575318652.36710557531865 2.349674770623532.34967477062353 2.355850929047432.35585092904743 2.353674836932842.35367483693284 2.354443099310472.35444309931047 2.354172058221862.35417205822186 2.354267704728952.35426770472895 2.354233955421632.35423395542163 2.354245864389032.35424586438903 2.354241662170942.35424166217094 2.354243144978352.35424314497835 2.354242621751152.35424262175115 2.354242806378522.35424280637852 2.35424274123042

The root is 2.35424275822278

Example 15 : Solve x tanh(x) =1 by the iteration method

Solution:

x = coth(x), let f(x) = coth(x) such that |f '(x)| = | -1/sinh2(x)|

< 1/x2, so that |f '(x)| < 1 for x > 1. Starting with x0 = 1 we

calculate x1, x2,...

48

xn coth(xn)1 1.31303528549933

1.31303528549933 1.156014018113951.15601401811395 1.219904026111621.21990402611162 1.191006666387691.19100666638769 1.203527567425641.20352756742564 1.197995858954451.19799585895445 1.200419260800651.20041926080065 1.199353627191561.19935362719156 1.199821451048241.19982145104824 1.199615924385251.19961592438525 1.199706188950351.19970618895035 1.199666540477331.19966654047733 1.199683954907391.19968395490739 1.199676305925221.19967630592522 1.19967966556677

The root is 1.1996786402

Problems on methods of finding roots

1) Starting with the interval [1,2], find square root of (2) to

within two decimal places using bisection method.

(Hint: let f(x) = x2 – 2)

2) Find the root of f(x) = e-x (3.2 sin(x) - 0.5 cos(x)) on the

interval [3, 4] using bisection method.

49

3) Solve f(x) = x - e-x with initial interval [0,1] using

bisection method.

4) Solve f(x) = x - cos(x), with initial interval [0,1] using

bisection method.

5) Use Newton's method to find one of the three roots of the

cubic polynomial 4x3 -15x2 + 17x - 6= 0.

6) Find the solution to ex – cosx = 0 using Newton's method.

7) Find the solution to 1-10x+25x2 = 0 using Newton'smethod

8) Find the solution to arctan(x) = 0 using Newton's method

9) Find the cubic root of seven using iteration method.

References

1) www.damtp.cam.ac.uk/lab/people/sd/.../roots.htm

2) reference.wolfram.com/.../NumericalRootFinding.ht...

50

Interpolation

In the mathematical subfield of numerical analysis,

interpolation is a method of constructing new data points

within the range of a discrete set of known data points.

In engineering and science one often has a number of data

points, as obtained by sampling or experimentation, and tries

to construct a function which closely fits those data points.

This is called curve fitting or regression analysis.

Interpolation is a specific case of curve fitting, in which the

function must go exactly through the data points.

A different problem which is closely related to interpolation

is the approximation of a complicated function by a simple

function. Suppose we know the function but it is too complex

to evaluate efficiently. Then we could pick a few known data

points from the complicated function, creating a lookup table,

and try to interpolate those data points to construct a simpler

function. Of course, when using the simple function to

calculate new data points we usually do not receive the same

result as when using the original function, but depending on

51

the problem domain and the interpolation method used the

gain in simplicity might offset the error.

It should be mentioned that there is another very different

kind of interpolation in mathematics, namely the

"interpolation of operators". The classical results about

interpolation of operators are the Riesz–Thorin theorem and

the Marcinkiewicz theorem. There are also many other

subsequent results.

Lagrange interpolation

The Lagrange interpolating polynomial is the polynomial

P(x) of degree ≤ n-1 that passes through the n points and is

given by:

P(x) = j

n

j=1P (x) where k

j jj k

n

k 1

x -xP (x) y

x -x

The formula was first published by Waring (1779),

rediscovered by Euler in 1783, and published by Lagrange in

1795 (Jeffreys and Jeffreys 1988). Lagrange interpolating

polynomials are implemented in Mathematica as

52

Interpolating Polynomial. They are used, for example, in the

construction of Newton-Cotes formulas. When constructing

interpolating polynomials, there is a tradeoff between having

a better fit and having a smooth well-behaved fitting

function. The more data points that are used in the

interpolation, the higher the degree of the resulting

polynomial, and therefore the greater oscillation it will

exhibit between the data points. Therefore, a high-degree

interpolation may be a poor predictor of the function between

points, although the accuracy at the data points will be

"perfect."

Theoretical example

Find Lagrange interpolating polynomial that fit the following

data (x0 , y0) , (x1 , y1) , (x2 , y2) , (x3 , y3)

Solution:

1 2 3 0 2 33 0 1

0 1 0 2 0 3 1 0 1 2 1 3

0 1 3 0 1 22 3

2 0 2 1 2 3 3 0 3 1 3 2

(x x )(x x )(x x ) (x x )(x x )(x x )P y y

(x x )(x x )(x x ) (x x )(x x )(x x )

(x x )(x x )(x x ) (x x )(x x )(x x )y y

(x x )(x x )(x x ) (x x )(x x )(x x )

- - - - - -- - - - - -

- - - - - -- - - - - -

53

Example 16: (2,-3) , (-1,7) , (4,6) , (12,-4)

Find cubic Lagrange interpolating polynomial satisfy the

above data.

Solution:

3(x 1)(x 4)(x 12) (x 2)(x 4)(x 12)P ( 3) (7)(2 1)(2 4)(2 12) ( 1 2)( 1 4)( 1 12)

(x 2)(x 1)(x 12) (x 2)(x 1)(x 4)(6) ( 4)(4 2)(4 1)(4 12) (12 2)(12 1)(12 4)

- - - - -- - - - -

- - - -- - - -

Inverse Lagrange interpolation

The inverse Lagrange interpolating polynomial is the

polynomial x = P(y) of degree ≤ n-1 that passes through the

n points and is given by:

x= P(y) = j

n

j=1P (y) where k

j jj k

n

k 1

y- yP (y) x

y - y .

This method used to obtain a root of the interpolating

polynomials, i.e. we can obtain any x related to its given y.

54

Example 17: Find a root of the cubic Lagrange interpolating

polynomial of the above example.

Solution: To get a root of the cubic Lagrange interpolating

polynomial, we have to obtain inverse Lagrange interpolating

polynomial satisfies the above data points such that:

( ) ( 1)(y 7)(y 6)(y 4) (y 3)(y 6)(y 4)x 2

(-3 7)(-3 6)(-3 4) (7 3)(7 6)(7 4)

(y 3)(y 7)(y 4) (y 3)(y 7)(y 6)(4) (12)(6 3)(6 7)(6 4) (-4 3)(-4 7)(-4 6)

- -- -

At y = 0, we will obtain the root.

Lagrange Interpolation has a number of disadvantages

• The amount of computation required is large

• Interpolation for additional values of requires the same

amount of effort as the first value (i.e. no part of the previous

calculation can be used)

• When the number of interpolation points is changed

(increased/decreased), the results of the previous

computations can not be used

55

• Error estimation is difficult (at least may not be convenient)

• Use Newton Interpolation which is based on developing

difference tables for a given set of data points.

• The nth degree interpolating polynomial obtained by fitting

(n+1) data points will be identical to that obtained using

Lagrange formulae!

Newton interpolation is simply another technique for

obtaining the same interpolating polynomial as was obtained

using the Lagrange formulae

Newton forward interpolation on equi spaced points

Newton's forward difference formula is a finite difference

identity giving an interpolated value between tabulated points

in terms of the first value y0 and the powers of the forward

difference . If we have set of data points (x0, y0), (x1, y1),

(x2, y2), (x3, y3), then the interpolating polynomial that fit the

above data using Newton method will be as follows

56

x y y y y

x0 y0

y1 - y0 = A

x1 y1 B-A= Dy2 - y1 = B E-D = F

x2 y2 C-B= E y3 – y2 = C

x3 y3

Where D = y2 - 2y1 + y0 , E = y3 - 2y2 + y1 , & F = y3 - 3y2 +

3y1 - y0 , x1 - x0 = x2 – x1 = x3 – x2 = h.

Since the interpolating polynomial is expressed by

3

20 0 13 0 2

30 1 2

x x (x x )(x x )P y y y

h 2 h

(x x )(x x )(x x )y

3 h

- - -

- - -

+

+

0 0 1 0 1 23 0 2 3

x x (x x )(x x ) (x x )(x x )(x x )P y A D F

h 2 h 3 h

- - - - - -

+ +

57

Example 18: Find interpolating polynomial that fit the

following data using Newton forward

(1 , 3) , (1.5 , 7) , (2 , 13) , (2.5 , 20)

Solution:

x y y y y

1 37-3=4

1.5 7 6-4= 213-7=6 1-2 = -1

2 13 7-6=1 20-13=7

2.5 20

3 2 3(x 1)(x 1.5) (x 1)(x 1.5)(x 2)x 1P 3 (4) (2) (-1)

0.5 2 0.5 0.5

- - - - -- + +

Newton backward interpolation on equi spaced points

Newton's backward difference formula is a finite difference

identity giving an interpolated value between tabulated points

in terms of the last value yn and the powers of the backward

58

difference∇. If we have set of data points (x0 , y0) , (x1 , y1)

, (x2 , y2) , (x3 , y3), then the interpolating polynomial that fit

the above data using Newton method will be as follows

x y ∇y ∇2y ∇3y

x0 y0

y1 - y0 = A

x1 y1 B-A= Dy2 - y1 = B E-D = F

x2 y2 C-B= E y3 – y2 = C

x3 y3

D = y2 -2y1 + y0 , E = y3 -2y2 + y1, & F = y3 -3y2 +3y1 - y0 ,

x1 - x0 = x2 – x1 = x3 – x2 = h.

23 3 23 3 2

33 2 13

x x (x x )(x x )P y y y

h 2 h

(x x )(x x )(x x )y

3 h

- - -

- - -

+

+

3 3 2 3 2 13 3 2 3

x x (x x )(x x ) (x x )(x x )(x x )P y C E F

h 2 h 3 h

- - - - - -

+ +

59

Example 19: Find interpolating polynomial that fit the

following data using Newton backward method

(1, 3) , (1.5 , 7) , (2 , 13) , (2.5 , 20)

Solution:

x y ∇y ∇2y ∇3y

1 3 7-3=41.5 7 6-4= 2

13-7=6 1-2 = -12 13 7-6=1

20-13=72.5 20

3 2 3P

(x 2.5)(x 2) (x 2.5)(x 2)(x 1.5)x 2.520 (7) (1) (-1)0.5 2 0.5 0.5

- - - - -- + +

Problems on interpolation

1- (1,3) , (5,-7) , (-13,4) , (2,47) , (-6,15)

Find Lagrange interpolating polynomial that fit the above

data.

60

2- Find Lagrange interpolating polynomial that fit the

following data, then find P(5).

x -1 3 9 13 20

y 5 7 -15 4 9

3- Construct Newton interpolation polynomial satisfy the

following data

x 3.5 3.55 3.6 3.65 3.7y 33.115 34.413 36.598 38.475 40.447

4- Find Newton interpolating polynomial passes through (-1,2)

(0,1) , (1,3)

5- Find interpolating polynomial that fit the following data

using Newton backward method

x 1.1 1.2 1.3y -3.143 1.326 5.974

61

References

1) Mathworld.wolfram.com ‹ ... ‹ Interpolation

2) www.math-linux.com/spip.php?article71

3) mat.iitm.ac.in/home/.../interpolation/lagrange.html

4) en.wikipedia.org/wiki/Newton_polynomial

5) www.math-linux.com/spip.php?article72

Curve fitting

Curve fitting is the process of constructing a curve, or

mathematical function, which has the best fit to a series of

data points, possibly subject to constraints. Curve fitting can

involve either interpolation, where an exact fit to the data is

required, or smoothing, in which a "smooth" function is

constructed that approximately fits the data. A related topic is

regression analysis, which focuses more on questions of

statistical inference such as how much uncertainty is present

in a curve that is fit to data observed with random errors.

Fitted curves can be used as an aid for data visualization, to

infer values of a function where no data are available, and to

62

summarize the relationships among two or more variables.

Extrapolation refers to the use of a fitted curve beyond the

range of the observed data, and is subject to a greater degree

of uncertainty since it may reflect the method used to

construct the curve as much as it reflects the observed data.

Let's start with a first degree polynomial equation:

y = ax + b

This is a line with slope a. We know that a line will connect

any two points. So, a first degree polynomial equation is an

exact fit through any two points.

If we increase the order of the equation to a second degree

polynomial, we get:

y = ax2 + bx + c

This will exactly fit a simple curve to three points.

If we increase the order of the equation to a third degree

polynomial, we get:

63

y = ax3 + b x2 + cx + d

This will exactly fit four points.

Suppose we wish to fit a first degree polynomial y = ax + b

to an approximate set of data (xi , yi) such that yi = axi + b

and we want to determine the best values for a and b so that

y’s predict the function values that correspond to x- values

where xi is a particular value of the variable assumed free of

error.

Let e = Yexact – Yapp , where Yapp= axi + b which is the value

from the equation and Yexact is the experimental value,

Yexact = yi. The least- squares criterion requires that

S= 2 2i i i

N N

i=1 i=1e (y ax b) - be a minimum, N is the number of

x,y- pairs. At a minimum for S, the two partial derivatives

S S,a b will be zero such that:

i i i

N

i=1

S 2(y ax b)( x ) = 0a - , i i

N

i=1

S 2(y ax b)( 1) = 0b -

64

Dividing each of these equations by -2 and expanding the

summation, we will get

2i i ii

N N N

i=1 i=1 i=1x y a x b x + , i i

i=1

N N

i=1y a x bN +

The above equation can be expressed in matrix form as

follows

i i

2i i ii

N N

i=1 i=1N N N

i=1 i=1 i=1

N x yb

ax x x y

Given N, i

N

i=1x , 2

i

N

i=1x , i

N

i=1y , i i

N

i=1x y , thus by Cramer rule we

get a and b such that:

a =

i

N N N

i i i ii=1 i=1 i=1

N N2

ii=1 i=1

2

N x y x y

N x ( x )

and b =

i

i

N N N N2

i i i ii=1 i=1 i=1 i=1

N N2

ii=1 i=1

2

y x x y x

N x ( x )

Example 20: Find a straight line that best fit the data (-1,3),

(1,7), (3,2)

65

Solution: Let the straight line is y = ax+b , we have to

i xi2i

x yi xi yi

1 -1 1 3 -32 1 1 7 73 3 9 2 6

Sum i

3

i=1x =3 2

i

3

i=1x =11 i

3

i=1y = 12 i i

3

i=1x y = 10

Therefore a = 3(10) 3(12)3(11)-9

= -1/4, b = 12(11) 10(3)3(11)-9

= 17/4

Similarly if we want to fit second degree polynomial

Y= ax2 + bx + c to an approximate set of data (xi , yi) . We

have to determine the best values for a, b and c so that y’s

predict the function values that correspond to x- values where

xi is a particular value of the variable assumed free of error.

Let iY = appY is the value from the equation where

2i ii

Y ax bx c + + and exactY = yi .The least- squares criterion

requires that:

S=N N N2 2 2 2

appexacti i iii =1 i =1 i=1e (Y - Y ) (y ax bx c) - -

66

be a minimum, N is the number of x,y - pairs. At a minimum

for S, the three partial derivatives S S S, ,a cb will be zero

such that:

Sa

2 2i i i i

N

i =12(y ax bx c (-x ) 0) - - ,

2i i i i

N

i=1

S 2(y ax bx c)(-x )b - - = 0,

2i i i

N

i=1

S 2(y ax bx c)(-1)c - - = 0

Dividing each of these equations by -2 and expanding the

summation, we will get:

2 4 3 2i i i i i

N N N N

i=1 i=1 i=1i=1x y a x b x c x + + ,

3 2i i i i i

N N N N

i=1 i=1 i=1i=1x y a x b x c x + +

2i i i

N N N

i=1 i=1i=1y a x b x cN + +

67

The above equation can be expressed in matrix form asfollows

2i ii

2 3i i ii i

2 3 4 2ii i i i

N N N

i=1 i=1 i=1N N NN

i=1 i=1 i=1 i=1N N N N

i=1 i=1 i=1 i=1

N x x yc

x x x b x y

ax x x x y

Given N , i

N

i=1x , 2

i

N

i=1x , 3

i

N

i=1x , 4

i

N

i=1x , i

N

i=1y , i i

N

i=1x y , 2

ii

N

i=1x y ,

thus by Cramer rule we get a , b and c such that: a = a ,

b = b , c = c

, where

2i i

2 3i i i

2 3 4i i i

N N

i=1 i=1N NN

i=1 i=1 i=1N N N

i=1 i=1 i=1

N x x

x x x

x x x

,

68

a =i

i i

N

ii=1

N N2

ii=1 i=1

N N2 3

i=1 i=1

i

i i

2ii

N

i=1N

i=1N

i=1

N x y

x x x y

x x x y

, b =

i

i

i i

N2

i=1

N N3

ii=1 i=1

N N2 4

i=1 i=1

i

i i

2ii

N

i=1N

i=1N

i=1

N y x

x x y x

x x y x

,

c =

i

i i

i i

N N2

ii=1 i=1

N N2 3

i=1 i=1

N N3 4

i=1 i=1

i

i i

2ii

N

i=1N

i=1N

i=1

y x x

x y x x

x y x x

Example 21: Find a quadratic polynomial that best fit the

data (-2,2), (-1, 1), (0,1), (2,2)

Solution: Let the approximated quadratic polynomial is

y = ax2+bx + c and we have to evaluate constants a, b and c.

N = 4, i

4

i=1x =-1, 2

i

4

i=1x = 9, 3

i

4

i=1x = -1, 4

i

4

i=1x = 33, i

4

i=1y = 6,

i i

4

i=1x y = -1, 2

ii

4

i=1x y = 17 (Explained by the following table)

69

i xi2i

x 3i

x 4i

x yi xi yi2i

x yi

1 -2 4 -8 16 2 -4 82 -1 1 -1 1 1 -1 13 0 0 0 0 1 0 04 2 4 8 16 2 4 8

Sum -1 9 -1 33 6 -1 17

By using above determinants, we will get:

4 1 9

1 9 1

9 1 33

, a =

4 1 6

1 9 1

9 1 17

,

b =

4 6 9

-1 -1 -1

9 17 33

, c =

9 1 9

1 9 1

17 1 33

Fitting exponential curves to data points

To find an exponential curve y = aebx that best fit set of

given data, we have first to modify the exponential curve to

first degree polynomial by taking ln to both sides such that ln

y = bx + ln a and we can express this equation in simplest

70

form such that Y = Ax+ B where Y = ln y , A = b and

B = ln a . Now we search for the best approximated values of

A and B which satisfy the following matrix form :

i

2

i i

i i i

N N

i=1 i=1N N N

i=1 i=1 i=1

N x YB

Ax x x Y

Given a set of data consist of four x,y- pairs (x1, y1), (x2, y2),

(x3, y3), (x4, y4), i.e. N = 4, hence i

N

i=1x ,

i

2N

i=1x , i

N

i=1Y ,

i i

N

i=1x Y will be obtained according to the following table

i xi yi i

2x Yi = ln yi xi Yi

1 x1 y1 1

2x Y1= ln y1 x1 Y1

2 x2 y222x Y2= ln y2 x2 Y2

3 x3 y3 3

2x Y3= ln y3 x3 Y3

4 x4 y4 4

2x Y4= ln y4 x4 Y4

Sum i

4

i=1x i

24

i=1x i

4

i=1Y i i

4

i=1x Y

71

A=

i

2

i i i i

2i

4 4 4

i=1 i=1 i=14 4

i=1 i=1

4 x Y x Y

4 x ( x )

= b & B =

i

i

2

2

i i i i

2i

4 4 4 4

i=1 i=1 i=1 i=14 4

i=1 i=1

Y x x Y x

4 x ( x )

=

ln a, therefore a = eB , there is a restriction on the data points

(xi , yi), such that yi .

Generally if we want to find a transcendental function

(y = c abx), for best fit a set of given data, we modify this

equation to first degree polynomial by taking loga to both

sides such that loga y = bx + loga c and we can express this

equation in simplest form such that Y = Ax + B where

Y = loga y , A = b and B = loga c . Now we search for the

best approximated values of A and B which satisfy the

following matrix form:

i

2

i i

i i i

N N

i=1 i=1N N N

i=1 i=1 i=1

N x YB

Ax x x Y

By using Cramer rule, we will get:

72

A=

i

2

i i i i

2i

N N N

i=1 i=1 i=1N N

i=1 i=1

N x Y x Y

N x ( x )

= b

And

B=i

i

2

2

i i i i

2i

N N N N

i=1 i=1 i=1 i=1N N

i=1 i=1

Y x x Y x

N x ( x )

= loga c,

Where a must be given constant.

If we consider N = 5, i.e. there are five x,y- pairs (x1, y1),

(x2, y2), (x3, y3) , (x4, y4) , (x5, y5) and i

5

i=1x ,

i

25

i=1x , i

5

i=1Y ,

i i

5

i=1x Y will be obtained according to the following table

73

i xi yi i

2x Yi = loga yi xi Yi

1 x1 y121

x Y1= loga y1 x1 Y1

2 x2 y222

x Y2= loga y2 x2 Y2

3 x3 y323

x Y3= loga y3 x3 Y3

4 x4 y424

x Y4= loga y4 x4 Y4

5 x5y4

25

x Y5= loga y5 x5 Y5

Sum i

5

i=1x i

25

i=1x i

5

i=1Y i i

5

i=1x Y

From the data of the above table, A and B can be calculated,

hence b = A and c = aB

Example 22: Find an exponential curve in the form y = aebx

that best fit the data (2,1), (1, 3), (0,1)

Solution: we have to get the approximated values for a and b

that best fit the above data, given N = 3, then we put the

above exponential form as a first degree polynomial so that

the equation will be Y = Ax+ B, where Y = ln y, A = b and

B = ln a.

74

A =

i

2

i i i i

2i

3 3 3

i=1 i=1 i=13 3

i=1 i=1

3 x Y x Y

3 x ( x )

= b

And

B =i

i

2

2

i i i i

2i

3 3 3 3

i=1 i=1 i=1 i=13 3

i=1 i=1

Y x x Y x

3 x ( x )

= ln a a = eB

According to the above three points, we can

calculate i

3

i=1x ,

i

23

i=1x , i

3

i=1Y , i i

3

i=1x Y using the following table

i xi yi i

2x Yi = ln yi xi Yi

1 2 1 4 Y1= ln 1=0 02 1 3 1 Y2= ln 3 ln33 0 1 0 Y3= ln 1=0 0

Sum i

3

i=1x =3

i

23

i=1x = 5 i

3

i=1Y = ln3 i i

3

i=1x Y = ln3

75

Fitting rational function to data points

To find the constants of the rational function y = 1ax b+

that

best fit set of given data, we have first to modify the rational

function to first degree polynomial by putting Y = 1y

and we

can express this equation in simplest form such that Y= ax+b.

Now we search for the best approximated values of a and b

which satisfy the following matrix form:

i

2

i i

i i i

N N

i=1 i=1N N N

i=1 i=1 i=1

N x Yb

ax x x Y

Given a set of data consist of x,y- pairs (x1, y1), (x2, y2) ,

(x3, y3) , …, (xm, ym), i.e. N = m, hence i

m

i=1x ,

i

2m

i=1x , i

m

i=1Y ,

i i

m

i=1x Y will be obtained according to the following table

76

i xi yi i

2x Yi =i

1y xi Yi

1 x1 y121

x Y1=1

1y x1 Y1

2 x2 y222

x Y2=2

1y x2 Y2

3 x3 y323

x Y3=3

1y x3 Y3

m xm ym2m

x Ym=m

1y xmYm

Sum i

m

i=1x i

2m

i=1x i

m

i=1Y i i

m

i=1x Y

a =

i

2

i i i i

2i

m m m

i=1 i=1 i=1m m

i=1 i=1

m x Y x Y

m x ( x )

and b =

i

i

2

2

i i i i

2i

m m m m

i=1 i=1 i=1 i=1m m

i=1 i=1

Y x x Y x

m x ( x )

,

there is a restriction on the data points (xi , yi), such that yi 0

Example 23: Find the constants of the rational function

y = 1ax b+

that best fit the data (2,1/3), (1, 1), (0,1/2)

77

Solution: Referring to the following table to get i

3

i=1x ,

i

23

i=1x ,

i

3

i=1Y , i i

3

i=1x Y .

i xi yi i

2x Yi =i

1y xi Yi

1 2 1/3 4 Y1=1

1y

= 3 6

2 1 1 1 Y2=2

1y

= 1 1

3 0 1/2 0 Y3=3

1y

=2 0

Sum i

3

i=1x =3

i

23

i=1x = 5 i

3

i=1Y = 6 i i

3

i=1x Y = 7

Substitute in the above two formulas such that

a = 23(7) 3(6)3(5)-(3)

= 1/2 , b = 26(5) 7(3)3(5)-(3)

= 3/2.

Fitting general function to data points

To find the constants of the curve y = a x) + b that best fit

set of given data where x) is any given continuous

function, we have first to modify the above function to first

78

degree polynomial by putting X = x) and we can express

this equation in simplest form such that y = aX + b . Now we

search for the best approximated values of a and b which

satisfy the following matrix form:

i i

2i i ii

N N

i=1 i=1N N N

i=1 i=1 i=1

N X yb

aX X X y

Given a set of data consist of x,y- pairs (x1, y1), (x2, y2) , (x3,

y3) , …, (xm, ym), i.e. N = m, hence i

m

i=1X ,

i

2m

i=1X , i

m

i=1y ,

i i

m

i=1X y will be obtained according to the following table

i xi Xi= ix ) yi i

2X Xi yi

1 x1 X1= 1x ) y1 1

2X X1 y1

2 x2 X2= 2x ) y2 2

2X X2 y2

3 x3 X3= 3x ) y3 3

2X X3 y3

m xm Xm= mx ) ym m2X Xm ym

Sum i

m

i=1X i

m

i=1y i

2m

i=1X i i

m

i=1X y

79

a =

i

2

i i i i

2i

m m m

i=1 i=1 i=1m m

i=1 i=1

m X y X y

m X ( X )

& b =

i

i

2

2

i i i i

2i

m m m m

i=1 i=1 i=1 i=1m m

i=1 i=1

y X X y X

m X ( X )

Example 24: Find the function y = a sin(x) + b that best fit

the data (1,0), (2, 3), (5,1)

Solution: Referring to the following table to get i

3

i=1X ,

i

23

i=1X , i

3

i=1y , i i

3

i=1X y .

i xi Xi= isin x ) yi i

2X Xi yi

1 1X1= sin 1)

=0.8410 0.7073 0

2 2X2=sin 2)

=0.913 0.8281 2.73

3 5X3= sin 5)

= -0.9591 0.9197 -0.959

Sum i

3

i=1X = 0.792 i

3

i=1y = 4 i

23

i=1X

= 2.4551

i i

3

i=1X y

= 1.771

80

Therefore a =

i

2

i i i i

2i

3 3 3

i=1 i=1 i=13 3

i=1 i=1

3 X y X y

3 X ( X )

=0.318 and

b =i

i

2

2

i i i i

2i

3 3 3 3

i=1 i=1 i=1 i=13 3

i=1 i=1

y X X y X

3 X ( X )

=1.249

General formula of curve fitting

If we consider the function y = a0 x) + a1 x) +

a2 x) +….+ an n x) that best fit set of given data points

(xi,, yi) and we have to calculate a0, a1, a2, a3, …., an to obtain

best approximation.

Let e = Yexact –Yapp , where Yapp= a0 ix ) + a1 ix ) +

a2 ix ) +….+ an n ix ) which is the value from the equation

and Yexact is the experimental value, Yexact = yi .

The least squares criterion requires that:

81

S = i2 2

n ni i i i i

N N

i=1 i =1e (y a x ) a x )-a x )-...-a x )) -

be a minimum, N is the number of x,y- pairs. At a minimum

for S, the partial derivativesn0 1 2

, ,....,S S S S,a a a a will be

zero such that:

i in ni i i i0

N

i =1

S 2(y a x ) a x )-a x )-...-a x ))(- x )) = 0a -

i in ni i i i1

N

i =1

S 2(y a x ) a x )-a x )-...-a x ))(- x )) = 0a -

Similarly

i in n ni i i in

N

i =1

S 2(y a x ) a x )-a x )-...-a x ))(- x )) = 0a -

Dividing each of these equations by -2 and expanding the

summation, we will get

82

i i i i

i i i i

i

n n

N N N

i =1 i =1 i =1N N

i =1 i =1

y x ) = a x ) a x ) x )

+a x ) x )+...+ a x ) x )

+

i i i i

i i i i

i

n n

N N N

i =1 i =1 i =1N N

i =1 i =1

y x ) = a x ) x ) a x )

+a x ) x )+...+ a x ) x )

+

i i i i i

i i i

i

n n

N N N

i =1 i =1 i =1N N

i =1 i =1

y x ) = a x ) x ) a x ) x )

+a x )+...+ a x ) x )

+

Similarly

i i i i i

i i i

n n ni

n n n

N N N

i =1 i =1 i =1N N

i =1 i =1

y x ) = a x ) x ) a x ) x )

+a x ) x )+...+ a x )

+

To obtain a0, a1, a2,…., an , we have to arrange the above

equations in matrix form:

83

0ni i i i i

1ni i i i i

n ni i i i in n

N N N

i =1 i =1 i =1N N N

i =1 i =1 i =1

N N N

i =1 i =1 i =1

ax ) x ) x ) ... x ) x )

ax ) x ) x ) ... x ) x )

x ) x ) x ) x ) ... x ) a

i i

i i

ni i

N

i =1N

i =1

N

i =1

y x )

y x )

y x )

By Cramer’s rule, we can get a0, a1, a2,.., an

Problems on curve fitting

1- Find the linear equation that best fit the following data

a) (1,1), (-1,7)

b) (-1,7), (1,1), (2,-5)

c) (-1,7), (1,1), (2,-5), (3,15). Discuss your conclusion.

2- Find the parabolic equation that best fit the following data

a) (1,7), (3,13), (2,25)

b) (-1,7), (1,3), (8,-5), (13,15)

84

3- Find the exponential curve y = aebx that best fit the

following data

a) (11,7), (23,13), (32,25)

b) (21,17), (41,3), (68,5), (93,15)

4- Find the curve y = a cos(x)+bx that best fit the followingdata

a) (31,17), (23,13), (12,15)

b) (21,19), (35,23), (68,5), (93,15)

References

en.wikipedia.org/wiki/Curve_fitting

www.synergy.com/Tools/curvefitting.pdf

www.ctr.unican.es/asignaturas/instrumentacion_5_IT/curvefit.pdf