che306
DESCRIPTION
Engineering Thermodynamics - Numerical Analysis : Made easy to everyone.TRANSCRIPT
CHE 306 NOTES 1
COURSE OUTLINES
A. ITERATIVE METHODS – Solution of Equation of f(x) = 0
1. Bisection Method
2. Fixed-Point Iterative (FPI) Method
3. Jacobi Iteration
4. Gauss-Seidel Iteration
B. INTERPOLATION AND POLYNOMIAL APPROXIMATION
1. Linear Interpolation
2. Quadratic Interpolation
3. Lagrange interpolation
4. Divided Difference Interpolation – Divided differencesand polynomials
5. Equispaced Interpolations
- Difference operators and difference Tables
- Forward, backward and central differences
C. MUMERICAL INTEGRATION AND DIFFERENTIATION
1. Numerical Differentiation
2. Difference Notation and Operators
3. Numerical Integration
- Trapezoidal Rule
- Simpson‟s Rule
- Mid-Point Rule
- Romberg Integration
D. NUMERICAL SOLUTION OF INITIAL VALUE PROBLEMS
1. Euler Method
2. Runge-Kutta Methods
3. Predictor-Corrector Methods.
INTRODUCTION
In the process of solving problems in Science, Engineering, Economics, etc., a physical
situation is first converted into a mathematical model. This is often called formulation of the
problem. This mathematical model often gives rise to mathematical problems which are too
difficult to solve in a neat closed form e.g.
(i.) Integration: Find 2
1
0
xe dx
(ii.) Nonlinear Equation: Solve cos x x
(iii.) Linear Algebra: Find the eigenvalues of a large matrix.
(iv.) Differential equations: Solve a system of nonlinear differential equations.
When such problem arises, numerical analysis is then used for developing techniques to find
a solution or approximate solution of the mathematical equations describing the model.
A numerical method (or a combination of numerical methods) which can be to solve a
problem is often called an algorithm.
An algorithm is a complete and unambiguous set of procedures leading to the solution of a
mathematical problem.
The results obtained for the solution of a problem will be affected by various source of error.
Numerical analysts must consider how much accuracy is required, estimate the magnitude of
round-off and discretization errors, determine an appropriate step-size or the number of
iterations required, provide for checks on the accuracy and make allowance for corrective
action in cases of non-convergence.
The efficiency of any numerical method (or algorithm) must also be considered. An
algorithm would be of no practical use if it required the largest computer error built to obtain
a useful answer.
The final phase in solving a problem is programming. Programming is the transformation of
the algorithm into a set of unambiguous step-by-step instructions for the computer.
In this segment of the course, we will look at the design (formulation) and analysis of various
numerical methods and assess them in terms of accuracy, efficiency and computer effort.
This will involve some mathematical analysis and some practical work using MATLAB.
Mathematical Modelling:
Example 1.1
Where = angle of elevation.
v0= muzzle speed.
From kinematic and projectile motion, the distance travelled by the canon ball is obtained as
follows.
0 cosx V t (1.1)
2
0 sin 1/ 2y V t gt (1.2)
..
0
2
0
cos
sin 1/ 2
V txxV t gty
(1.3)
..
..
..
0xx
gy
(1.4)
Analysis:
When 0,y
2
0 sin 1/ 2 0V t gt
2
0( sin 1/ 2 )t V t gt
2
00, sin 1/ 2 0t V t gt
2
0 sin 1/ 2 0V t gt
02 sinVt
g
(1.5)
Distance travelled to hit the target,
0 cosx v t
0
Cannon Target
d
Figure 1.1: Aiming a Cannon
2
02 sin cosvx
g
(1.6)
The distance travelled by the canon ball is2
02 sin cosv
g
, where g= acceleration due to
gravity.
In order to find the correct elevation to hit the target requires satisfying the equation
2
02 sin cosvd
g
(1.7)
So, we require to solve ( ) 0f . Where
2
02 sin cos( )
vf d
g
(1.8)
Note the following observations:
(i) The modelling process gives an idealisation: Some features have been ignored
e.g. air resistance, length of the muzzle. They may be significant.
(ii) The nonlinear equation may not have a solution. The maximum range is 2
0v
g
when / 4 .So if 2
0vd
g , the target is out of range.
(iii) The nonlinear equation may have many solutions. If * is a solution, then
so is * 2k for any integer k. These are trivial rotated solutions. If a
solution / 2 then so is */ 2 .
(iv) The equation can be rearranged and solved easily as
2
02 sin cosvd
g
(1.9)
2 2
0 0(2sin cos ) (sin 2 )v vd
g g
(1.10)
2
0
sin(2 )dg
v
1
2
0
1sin
2
dg
v
(1.11)
Normally, in such problems, a closed form solution is not possible and so an approximate
solution is sought from a numerical method.
00
2 sincos .
vv t
g
Example 1.2
A sphere of radius r and density is floating in water of unit density. It is required to
determine the depth h of the sphere that is below the waterline.
Solution
Mass of the sphere = Volume x Density
34
3r (1.12)
Displaced water mass: Applying Archimede‟s Principle, a body partially submerged in a fluid
is buoyed up by a force equal to the weight of the displaced fluid.
Mass of displaced water 21/ 3 (3 ) 1h r h equating the two weight masses, we have
3 24 1(3 ) 1
3 3r h r h
3 24 (3 )r h r h
2 3
3 3
34
rh h
r r
2 3
4 3h h
r r
Define x = r
2 34 3x x
3 23 4 0x x (1.13)
Equation (1.13) is a cubic polynomial equation with three zeros. If 0.6 , to what depth
does the sphere sink as a fraction of its radius? This example can be visualized as finding the
values of x for which the graph f(x) = 0 i.e. touches the x-axis.
r
h
SOLUTION OF EQUATIONS OF ONE VARIABLE
We have the problem of finding values of x that satisfy ( ) 0f x for a given function f. A
solution is called a zero of f or a root of ( ) 0f x .
The need for numerical methods arises since equation of the form ( ) 0f x rarely provide
closed-form expressions for the roots. A well-known equation having a closed form
expression for its roots is the quadratic equation.
2 0ax bx c (1.14)
The roots of (2) are defined explicitly by
2 4, 0
2
b b acx a
a
(1.15)
And simply require the substitution of values for a, b and c into (1.15).
LOCATING A ROOT
To develop a numerical method for finding the rots of ( ) 0f x , it is useful to first determine
an interval a x b that contains at least one root. So, we need to ask. What properties must
a function „f of x‟ satisfy on the interval a x b to guarantee at least one root in this
interval?
Example 1.3:
Consider the polynomial curve, 3 23 2x x x (shown below). The curve cuts the x-axis at
0, 1x x and 2x , indicating that the cubic equation 3 23 2 0x x x has three
solutions.
y
-1 1 2 3
1
2
3
-1
-2
x
A root of ( ) 0f x will be denoted by p or x*. So ( *)f x has value 0 or ( )f p .
An important application of iteration is in the problem of finding zero(s) of a function of one
variable x, the problem is written as:
Find all x: ( ) 0f x .
A root of equation ( ) 0f x is a value of x which when substituted into the L.H.S of the
equation gives zero. The value is then a zero of the function f. We consider several numerical
methods for this problem.
1.1 BISECTION METHOD
Is a numerical method for solving ( ) 0f x . The underlying mathematics for this methodis
contained Intermediate Value Theorem (IVT). If the function f is continuous on [a,b] and k is
any number lying between ( )f a and ( )f b then there is a point C somewhere in (a,b) such
that ( ) .f c k
y = f(x) f(a)
f(b)
k
c a b x
y
Figure 2: Initial value Theorem
y
-1 1 2 3
0.5
1.0
1.5
-0.5
-1.0
x
For the equation ( ) 0,f x we use 0.k then the IVT tells us that if f is continuous on [a,b]
and ( )f a and (b)f have different signs, then there is a solution of ( ) 0f x between a and b.
however, there might be more than one.
Figure 3: Solutions using IVT methods
So, ( ), ( )f a f b opposite signs, then at least one solution exists.
If ( )f a and (b)f have the same sign, then the theorem does not apply, so we cannot say
whether is a solution or not.
Figure 4: IVT with signs changes
PROCEDURES
1. Starting with an interval [a,b] (obtained by lucky or informed guess) with
( ) 0f a f b then there is at least one root a or b, in the interval [a,b].
Note: i. That if ( ) 0,f a then a is a root or if (b) 0,f then b is a root.
ii. An interval [a,b] with ( ), ( ) 0f a f b is called a Nontrivial Bracket for a root of
( ) 0f x .
2. Suppose [a,b] is a nontrivial bracket for a root of ( ) 0,f x then the value at the
midpoint 1/ 2( )c a b gives 3 possibilities.
(i) ( ) 0,f c then c is a root and the problem is solved (unlikely in general).
(ii) ( ) 0f c and ( ). ( ) 0,f c f b then [c, b] is a nontrivial bracket for f(x) = 0.
(iii) ( ) 0 ( ). ( ) 0,f c and f c f b then [a, c] is a nontrivial bracket for f(x) = 0.
With case (ii) or (iii), the resulting nontrivial bracket is half the size of the original.
solution
y = f(x) y
x b a
solution
y
y =f(x)
b a
solution solution
solution solution
x
y
x b a
y
x b a
3. This process is repeated until the interval containing the root is small enough or any
convergence is decided upon (used).
Example 1: (Use your calculator here) Find a root of 2 0.5 0x , using the Bisection method
starting with the interval [0, 1]. Use 4 steps (i.e. calculate 4 subintervals).
n a c b f(a) f(c) f(b)
0 0 0.5 1
1
2
3
4
Some Notes with using University Calculator
1. REPLAY FUNCTION: is used to safe retyping
2. ANS: holds the last number calculated.
3. STO: stores a number in a register A – F, X, Y, M. e.g.
stores the last number calculated in A.
4. RCL: recalls a number from a register. e.g. recalls the number in register A.
5. Changing mode from rad to degree etc. then note small
in the display.
n a c b f(a) f(c) f(b)
0 0 0.5 1 - - +
1 0.5 0.75 1 - + +
2 0.5 0.625 0.75 - - +
3 0.625 0.6875 0.75 - - +
4 0.6875 0.71875 0.75 - +
Approximate root c4= 0.71875
The root lies in [0.6875, 0.75]
The error in the approximation is 4 41/ 2( ) 1/ 2(0.75 0.6875)b a
0.03125
So 4 0.03125p c
The root cannot be further from C4 than 4 41/ 2( )b a but it may be closer.
ANS STO A
RCL A
Mode Mode
1 D
Figure 5: Bisection Method through IVT solution
The root is at x = p.
Thus the bisection method generates subintervals [an, bn], n = 1, 2, 3, …all containing the
root p with [a0, b0] = [a, b]. The corresponding midpoints c0, c1,c2, c3, …are a sequence of
approximations for p.
BISECTION METHOD ALGORITHM
INPUT: Nontrivial Bracket - [a, b], Tolerance – ε.
OUTPUT: Approximate solution, c.
STEP 1: WHILE a b do STEPS 2-4.
STEP 2: SET 1
( )2
c a b .
STEP 3: IF f(c) = 0, THEN OUTPUT (c),
STOP
STEP 4: IF f(a) x f(c) < 0 THEN
SET b = c
ELSE
SET a = c
STEP 5: SET 1
( )2
c a b ,
OUTPUT (c),
STOP
y
x
b0 a0 c0
b1 a1 c1
b2 a2 c2
a3 b3 c3
y = f(x)
p
BISECTION METHOD ALGORITHM (MATLAB PROGRAM)
NOTES on Bisection MATLAB Program
1. LINE 1: begins with keyword function followed by output argument c and the “=”
symbol. This is followed by the input arguments within round brackets.
2. LINE 2: called the H1 (help 1) line. It should be a comment line of special form:
beginning with %, followed without space by the function name in capitals, followed
by one or more spaces and then a brief description. The description should begin with
a capital, end with a period “.” and omit the words „the‟ and „a‟.
3. All comments lines from H1 up to the first non-comment line (usually a blank line for
readability) are displayed when “ helpfunction_Name” is typed in the command
window after the “ >> “ symbol. These lines should describe the function and its
arguments (capitalized by convention).
Note that the function name must be the same as the name of the m-file in which it is stored.
So here “bisect” and “bisect.m” for the m-file.
function c = bisect(a, b, epsilon)
%BISECT - Bisection method for f(x) = 0.
% BISECT (A, B, EPSILON) is an approximate solution on the interval [a, b].
% The final subinterval has length < = EPSILON
SAMPLE OF MATLAB CODES
SAMPLE OF A FUNCTION m-FILE
function c = bisect(a,b,epsilon)
%BISECT Bisection method for f(x) = 0.
% BISECT(A,B,EPSILON) is an approximate solution on the
% interval [A,B]. The final subinterval has length <= EPSILON.
disp(' a b c fa fc fb');
fa = f(a);
fb = f(b);
% check initial bracket
if fa*fb>= 0
error('fa*fb is non-negative! The bracket is out of range!!!')
end
% bisection loop
while abs(b-a) > epsilon
c = (a + b)*0.5;
fc = f(c);
disp([a,b,c,fa,fc,fb]);
if fc == 0
return
end
if fa*fc < 0 % root is to left of mid-point
b = c;
fb = fc;
else % root is to right of mid-point
a = c;
fa = fc;
end
end
INTERVAL LENGTH
If the original bracket has length 0L i.e 0L b a , then the bracket length after k steps
(calculation of k brackets after the original) is 0
2k k
LL . This formula allows us to calculate
the number of steps required to make the bracket lengthkL E . Thus,
kL E implies
0
0
0
ln(2
2
2
)
k
k L
E
Lln
L
E
E
k
Example 2: If 0 1L and 610E , calculate the number of iterations required to make
kL E .
This requires
6
6
1ln
10
ln(2)
ln 1019.93
ln(2)
k
Thus 20k (k is an integer) iterations are required.
Note: useful numbers: 0 610 3 2~10 ,2 ~2 10 .
function y = f(x)
%F(X) Function of X.
% Y = F(X) computes given function of X.
% F(X) = x^2 - 0.5.
y = x^2 - 0.5;
Note: the „error‟ in the mid-point as an approximation to the root introduces an extra ½,
1/ 2k kC P L .
ERROR ANALYSIS
The Bisection algorithm was stopped when b a E . This gave us an error estimate for the
approximate root. If the approximate root is ≈1, then an error estimate of 610 , say, is
reasonable. If however, the approximate root is ≈ 710 , then an error estimate of 610 is very
bad.
ABSOLUTE ERROR
If y approximates x, then y x is the absolute error.
RELATIVE ERROR
If y approximates x, then y x
x
is the relative error. This definition requires 0x . Often
relative error is multiplied by 100 to give percentage error.
Class Work
If y approximates x, then compute the (i) Absolute Error, (ii) Relative Error and
(iii) Percentage Error for the following:
1. 1 10.3000x10 , 0.3100x10x y .
2. 3 30.3000x10 , 0.3100x10x y
3. 4 40.3000x10 , 0.3100x10x y
Solutions
1. 1 10.3000x10 , 0.3100x10x y
Absolute Error 0.1 .
Relative error 10.3333x10
2. 3 30.3000x10 , 0.3100x10x y
Absolute Error 40.1x10
Relative error 10.3333x10
3. 4 40.3000x10 , 0.3100x10x y
Absolute Error 30.1x10 . Relative error 10.3333x10
Observation: Widely varying absolute errors for the same relative error.
Generally, relative error is more meaningful while absolute error can be misleading.
STOPPING CRITERIA
In the Bisection algorithm, the computation is „stopped‟ when b a E . Then the current
midpoint c is an approximation to the root p with
Absolute Error c P 1/ 2 a b
i.e. half the current interval length. Thus this stopping criterion gives an approximation with
absolute error 1/ 2E .
ADVANTAGES AND DISADVANTAGES OF THE BISECTION METHOD
Disadvantages
1. Slow convergence, a large number of iterations may be required to make the absolute
error, c P , small.
2. Good intermediate approximations may be discarded when the root is close to the
midpoint.
Advantages
1. Always converge to a solution. This makes it an attractive technique for finding a
„starting value‟ (initial guess) for faster methods.
2. Does not require derivative of a given function.
1.2 FIXED-POINT ITERATIVE METHODS
1. First-Order Fixed-Point Iteration.
In this section, we explore a class of numerical methods for solving the equation f(x) = 0,
which given a first term x0, generates a sequence {xn} such that:
(a) xn+1 is obtained from xn (first-order recurrence relation) and
(b) nx as n , where α is a root of the equation f(x) = 0.
The requirements are to (i) generate a sequence based on the equation f(x) = 0 and (ii) to
ensure that nx (xn approaches α) as n (n tends to ∞) i.e. convergence is reached.
Notation of a Fixed-Point Iteration
It is first necessary to develop the notation of a fixed-point.
Example 1.2: Let a function, f, be defined by 2( ) 2f x x x . The roots of f(x) = 0 are x = 0
and x = 0.5. Consider the sequence generated by the recurrence relation:
10.1, 2 (1 ), 0,1,2,...n n nx x x x n (1.2.1)
n 0 1 2 3 4 5 6
xn 0.100 0.180 0.295 0.416 0.486 0.5000 0.5000
From the Table, the sequence approaches the root x = 0.5 where “it gets stuck”. On dropping
subscripts from equation 1.2.1, the equation 2 (1 )x x x is obtained.
In the figure (b), it is seen that the graph of y = x and 2 (1 )y x x intersect at x = 0.5. Thus
when RHS of equation 1.2.1 is evaluated at x = 0.5, the value of xn+1 = 0.5 is obtained. All
subsequent terms in the sequence, 2 3, , ...n nx x
etc. will equal to fixed value 0.5.
Definition
The above example suggest that if a sequence generated by a recurrence relation of the form
xn+1 = g(xn)
has a limiting value, α, then α is a fixed point of the function g.
Formulation of Fixed-Point recurrence Relation
We are set here to find a value of : ( ) 0x f x is satisfied. To utilize recurrence relation, the
equation must be written in the form ( )x g x .
This can be accomplished by adding x to each side of f(x) = 0.
i.e. Given
y= x y
x 0.25
-0.2
0.5 0.75 1.0
-0.1
a:Roots
y
x
y= 2x(1-x)
b: Fixed-Point
The value x is a fixed point of the function gif α =
g(α).
( ) 0f x (1.2.2)
Adding x to both sides of equation (1.2.2), we have
( )x f x x (1.2.3)
Rearranging (1.2.3), we have
( )x x f x (1.2.4)
x = g(x) (1.2.5)
There are therefore infinity ways of arranging (1.2.2) in the form of (1.2.5).
Example 1.2.2: For the equation ( ) 0f x , where 3 2( ) 4 10f x x x
The following are the three possible arrangements in the form of x = g(x):
(i) 3 2
110 4 ( )x x g x
(ii) 3
2
110 ( )
2x x g x
(iii) 2
10( )
4x g x
x
The systematic iterative approach required to determine a zero off follows the following
steps:
1. Write f(x) = 0 in the form of x = g(x).
2. Define the first-order recurrence relation, xn+1 = g(xn)
3. Given x0, use the recurrence relation to generate the sequence {xn}.
Example 1.2.3. Find the roots of 3 24 10 0x x .
Applying steps 1-3 above, we have
Step 1: 3 2( ) 4 10f x x x
Step 2: defining the 1st-orde recurrence relation, we have
231 110 4 ( )n n nx x g x (1.2.6)
3
1 2
110 ( )
2n n nx x g x (1.2.7)
1 3
10( )
4n n
n
x g xx
(1.2.8)
Step 3: Starting with x0 = 1.5, we obtain the following sequences:
n g1(xn) g2(xn) g3(xn)
0 1 1.2870 1.3484
2 1.8171 1.4025 1.3674
3 -1.4748 1.3455 1.3656
4 1.0914 1.3752 1.3653
5 1.7364 1.3601 1.3652
6 -1.2725 1.3678 1.3652
7 1.5215 1.3639
Alr
eady C
onver
ged
8
No C
onver
gen
t
. . .
9
10
11
12
13
14 1.3652
The sequences show a range of behaviour. The first sequence does not approach a limiting
value. The second sequence attains a limiting value after 14 iterations. The third sequence
attains a limiting value after just 5 iterations.
Then the question to ask is:
What property does g(x) require near a fixed-point α for α to be the limiting value of a
sequence generated by xn+1 = g(xn)?
The answer leads us to another topic.
Fixed-Point Convergence
Convergence of the sequence {xn} generated by xn+1 = g(xn) occurs if
( ) 1g x
Smaller ( )g x implies faster convergence.
Classification of Fixed-Points
Class Condition Bahaviour
Attracting ( ) 1if g Convergent
Indifferent ( ) 1if g Uncertain
Repelling ( ) 1if g Divergent
So if the slope of g is positive, then monotonic convergence or divergence results. If the slope
of g is negative, then oscillatory convergence or divergence results.
Initial Guess
Fixed-Points methods require a good initial value x0. A simple solution is to find an interval
[a, b] in which the original function f satisfies the sign property and then use the mid-point
0
1( )
2x a b as the initial value. In other words, bisection method can be used in evaluating
an approximate starting value.
Example 1.2.4:
Find a root of 2( ) 5 4 0f x x x .
The possible arrangements are:
(i) 2
1
4( )
5
xx g x
(ii) 2 ( ) 5 4x g x x
(iii) 2
3( ) 4 4x g x x x
Obtaining convergence condition, we have
(a) 1
2( )
5
xg x
(b) 2
5( )
2 5 4g x
x
(c) 3 ( ) 2 4g x x
For (a): Solving 1 ( ) 1g x , we have
21
5
x
241
25
x
2 250
4x
5 5( )( ) 0
2 2x x
5 5
2 2x
(a guess of 0 1.5x will be appropriate)
So the derivative is less than 1 in magnitude for all x such that5 5
2 2x
. Moreover, if
( )g x is near zero in the entire region, the iteration converges quickly; if the derivative is
near 1 in magnitude, the iteration converges quite slowly.
For (b): Solving 2 ( ) 1g x , we have
251
4(5 4)x
25 4(5 4)x
20 9x 0.45x
(a guess of 0 1.0x will be appropriate)
For (c): solving 3 ( ) 1g x , we have
2 4 1x
2(2 4) 1x
2 4 1 2 4 1x or x
5 3
2 2x or x
3 5. .
2 2i e x
(a guess of 0 2.0x will be appropriate)
1.3 ACCELERATED CONVERGENCE – AITKEN’S METHOD
From the Fixed-Point processes seen so far it is clear that the choice of g is instrumental in
defining the convergence properties of the sequence {xn} generated by 1 ( )n nx g x . We now
investigate how to accelerate the convergence of {xn}. The first approach uses the sequence
{xn}itself and simply modifies the term xn to improve convergence.
The Aitken‟s acceleration scheme is given as
2*
1 1 2
1
( )nn n
n
xx x
x
(1.2.9)
Where
1n n nx x x
2
1 1 12n n n nx x x x
Equation (1.2.9) is sometimes referred to as Aitken‟s Delta Squared method.
Example 1.2.5: In Example 1.2.3, fixed-point iteration was used to find the real root of the
equation 3 24 10 0x x , specifically, for the sequence
3
1
110
2n nx x , the process took
14 iterations to converge to 4 d.p. i.e. 14 1.3652x . The Table below shows the associated
Aitken‟ sequence {xn*} obtained by applying Equation (1.2.9). Convergence to 4 d.p. is now
achieved after just 6 iterations.
n xn xn*
0 1.5 -
1 1.2870 -
2 1.4025 1.3619
3 1.3455 1.3643
4 1.3752 1.3650
5 1.3601 1.3652
6 1.3678 1.3652
For n = 1,
2* * * 2 1
1 1 1 2 2
2 1 0
( )
2n
x xx x x x
x x x
2*
2
(1.4025 1.2870)1.4025 1.3619
1.4025 2(1.2870) 1.50x
For n =2,
2*
3
(1.3455 1.4025)1.3455 1.3643
1.3455 2(1.4025) 1.2870x
For n =3,
2
*
4
(1.3752 1.3455)1.3752 1.3650
1.3752 2(1.3455) 1.4025x
For n =4,
2
*
5
(1.3601 1.3752)1.3601 1.3652
1.3601 2(1.3752) 1.3455x
For n =5,
2
*
6
(1.3678 1.3601)1.3678 1.3652
1.3678 2(1.3601) 1.3752x
1.4 ACCELERATED CONVERGENCE – The ‘g’- Factor
A second approach to accelerating convergence is based upon the choice of the function g.
Earlier several ideas were established regarding the convergence of a sequence {xn}
generated by 1 ( )n nx g x , to a fixed-point α of g, that
(a) ( ) 1 andnx if g
(b) Convergence is accelerated for smaller values of ( )g .
Thus this suggests the following questions:
(i) Can g always be chosen such that ( )g < 1?
(ii) Can the value of ( )g be optimised (minimised)?
To answer the first question, let us add the quantity x to each side of the equation ( )x g x ,
where 1 , to obtain
( )x x g x x
Rearranging, we have
(1 ) ( )x g x x
( )( )
(1 )
g x xx G x
(1.30)
Equation (1.30) has the same root(s) as ( )x g x .
Example 1.2.6: The quadratic equation 2 3 2 0x x has 2 real roots, x =1 and x = 2. In the
form ( )x g x , i.e. 2( 2) / 3x x , then
1 2(1) 1
3g
4 2(2) 2
3g
And x = 1 and x = 2 are fixed-points of the expression 2 2
3
x .
If, for example, 7x is added to both sides and then arranged to the form equation (1.2.6), we
obtain
x
xx x x
G x
2
2
2
37
1 7
21 2
24
G(1) = 1 and G(2) = 2. Thus, x = 1 and x = 2 are fixed point of G, and hence the roots of x =
G(2). To find a fixed point α of G, the relation xn + 1 = G(xn) could be used to generate a
sequence {xn} with limiting value α. To be efficient ( )G should be small, ideally close to
zero.
G xg x x
1
, 11
g xG x
(1.31)
G Equals zero if 0,g i.e. g
Typically, if a root is sought in the vicinity of x0, then λ is taken as 0g x . The better the
initial guess, the better the value of λ.
Example1.2.7: consider again the equation x x3 24 10 0 . Then acceleration scheme is
applied to all three iterative processes of example 1.6.
i.e. x g xn i n 1 , I = 1, 2, 3 where gi
115.
Compare the convergence of the sequences in the table below (with 11, 5 and 3 iterations
respectively) with the original table under example 1.6 (divergence, 4 and 6 iterations).
λ 4.000 0.6556 0.1226
n xn
0 1.5000 1.5000 1.5000
1 1.4000 1.3713 1.3653
2 1.3785 1.3657 1.3652
3 1.3705 1.3653 1.3652
4 1.3674 1.3652
5 1.3661 1.3652
6 1.3656
7 1.3654
8 1.3653
9 1.3653
10 1.3652
11 1.3652
1.2.5 Optimized Accelerated Convergence – Newton Raphson Iteration.
Let the sequence {xn} generated by the modified relation xn+1 = G(x) converge to a limiting
value equal to the fixed point α of the equation x = g(x). It is reasonable to assume that each
term xn+1 will be an improved estimate to α when compared with the estimate xn.
To further accelerate the sequence, we might update the value of λ at each step using λ =
( )ng x in place of the fixed value λ = 0( )g x (xn should be closer to α than x0 for a
convergent sequence). The effect of this modification is now investigated and it results in the
new iterative scheme – Newton-Raphson iteration.
A recurrence relation needs to be developed:
x G xg x x
n n
n n
1 1
1
n n n
n
g x g x x
g x
(1.28)
The function g is related to function f appearing in the original equation f(x) = 0. So by simple
manipulation, adding x to each side yields.
x f x x
x x f x g x
Hence, g(x) = x – f(x)
g x f xi1 1
Then equation (1.28) becomes
xx f x f x x
f xn
n n
i
n n
i
n
1
1
1 1
x f x x f x x
f x
n n n
i
n n
i
n
f x x f x
f xx
f x
f x
i
n n n
i
n
n
n
i
n
1
n
n n i
n
f xx x
f x (1.29)
Equation (1.2.9) is called Newton-Raphson iteration scheme.
Example 1.2.8: For the equation f(x) = 0 with
f x x x 3 24 10 ;
23 8f x x x .
The recurrence relation (1.29) becomes:
x xf x
f xx
x x
x xn n
n
i
n
n
n n
n n
1
3 2
2
4 10
3 4
2 4 10
3 8
3 2
2
x x
x x
n n
n n
Stating with x0 = 1.5
n = 0; x0 = 1.500
x xx x
x x0 1 1
0
3
1
2
0
2
1
2 4 10
3 813733
.
n =1; xi = 1.3733
x xx x
x x1 1 2
1
3
1
2
1
2
1
2 4 10
3 813653
.
n = 2; x2 = 1.3653
x xx x
x x2 1 3
2
3
2
2
2
2
2
2 4 10
3 813652
.
n = 3; x3 = 1.3652
x x
x x
x x3 1 4
3
3
3
2
3
2
3
2 4 10
3 813652.
Convergence of Newton-Raphson Method.
The general relation is
x g xn n 1
So it could be deduced from equation (1.2.9) that
g x x
f x
f xi
Recall that the derivative of g is the key descriptor of convergence points. Here
g x
f x f x f x f x
f x
i
i i ii
i
1 2
1 2 2
f x f x
f x
f x f x
f x
i i
i
ii
i
f x f x
f x
ii
i 2
At a root x = α, the value f(α) is zero by definition. Hence, for the Newton-Raphson scheme,
0g x - „optimal‟ convergence. So the condition for convergence:
1g x holds.
2
f x f x
f x
< 1
2.0 INTERPOLATION AND POLYNOMIAL APPROXIMATION
Engineers and scientists commonly assume that relationships between variables in a
physical problem can be approximately reproduced from the data given by the problem.
The ultimate goal might be to determine the values at intermediate points to approximate the
integral or derivative of the underlying function, or to simply give a smooth or
continuous representation of the variables in the problem.
Interpolation refers to determining a function that exactly represents a collection of data.
The most elementary type of interpolation consists of fitting polynomials, so they are a
natural choice for approximating derivatives and integrals. We will see here how
polynomials are easily constructed, the result of which implies that there are
polynomials that are arbitrarily close to any continuous function.
2.1 Langrange Polynomials
Here, we find approximating polynomials that can be determined simply by specifying
certain points in the plane through which they must pass. Determining a polynomial of
degree that passes through the distinct points (x0, y0) and (xi, yi) in the same as approximating
a function f for which f(x0) = y0 and f(x1) = y1 by means of a first – degree polynomial
interpolating, or agreeing with, the values of f at the given points. We first define the
functions
L xx x
x xL x
x x
x x0
1
0 1
1
0
1 0
,
and then define
P x L x f x L x f x 0 0 1 1
since
L x L x L x0 0 0 1 1 01 0 0 , , , and L x1 1 1 ,
we have
P x f x f x f x y0 0 1 0 01 0 . . and
P x f x f x f x y1 0 1 1 10 1 . .
So, P is the unique linear function passing through (x0, y0) and (x1, y1),
To generalize the concept of linear interpolation to higher degree polynomials, consider the
construction of a polynomial of degree at most n that passes through the n + 1 points.
x f x x f x x f xn n0 0 1 1, , , ,....., , . See the figure below.
In this case, we need to construct, for each k = 0, 1, … , n, a function Ln, k(x) with the point
that Ln, k (xi) =0 when i k and Ln, k(xk) = 1. To satisfy Ln, k (xi) =0 for each i = k requires
that the numerator of Ln, k(x) contains the term.
x x x x x x x x x xk k n 0 1 1 1...... ......
To satisfy Ln, k(xk) =1, the dominator of Ln, k(x) must be equal to this term evaluated at
x = xk. Thus
0 1 1
0 1 1
- ... ..., ( ) =
... ...
k k n
n k
k k k k k k n
x x x x x x x xL x
x x x x x x x x
A sketch of the graph of a typical Ln, k is shown below
y1 = f(x1)
y0 = f(x0)
x0 x1 x
y y = f(x)
y = P(x)
y
x xn xn - 1 x2 x0 x1
f
P
The interpolating polynomial is easily described now that the form of Ln, k (x) is
known. This polynomial is called the nth Lagrange interpolating polynomial.
P x f x L x f x L x f x L xn n o n n n n n kk
n
00
, , ,......
Where
L x
x x x x x x x x x x
x x x x x x x x x xn k
k k n
k k k k k k k n
,
..... ......
...... ......
0 1 1 1
0 1 1 1
For each k = 0, 1, ………., n.
If x0, x1,..., xn are (n + 1) distinct numbers and f is a function whose values are given at
these numbers, then Pn(x) is the unique polynomial of degree at most n that agrees
with f(x) at x0, x1, …, xn.
The notation for describing Lagrange interpolating polynomial Pn(x) is rather
complicated. To reduce this somewhat, we will write Ln, k(x) simply as Lk(x).
Example 2.1: Using the numbers, or node, x0 = 2, x1 = 2.5, and x2 = 4 to find the
second interpolating polynomial for f(x) = 1/x.
Solution: this requires that we first determine the coefficient polynomials L0, L1 and
L2.
L x
x xx x0
2 5 4
2 2 5 2 465 10
.
..
L x
x x x x1
2 4
2 5 2 2 5 4
4 24 32
3
. .
And
L x
x x x x2
2 2 5
4 2 4 2 5
4 5 5
3
.
.
.
Then f(x) = 1/x
So, f x f0 2 1 2 05 .
Ln, k(x)
x x0 x1 xk -1 xk +1 xk xn -1 xn
f x f1 25 1 25 04 . . . and
f x f2 4 1 4 025 .
We have
P x f x L xx k kk
0
2
f x L x f x L x f x L x0 0 1 1 2 2
05 65 10 0 44 24 32
30 25
4 5 5
3. . . .
.x x
x x x x
0 05 0 425 115. . .x x
An approximation to f(3) = 1/3 is
f P3 3 0325 .
Example 2.2: The table below lists values of a function at variable points. compare
the approximation to f(1.5) obtained by Lagrange polynomials of define 2 and 3.
x f(x)
1.0 0.7651977
1.3 0.6200860
1.6 0.4554022
1.9 0.2818186
2.2 0.1103623
Since 1.5 is between 1.3 and 1.6, then the most approximate linear polynomial uses
x0 = 1.3 and x1 = 1.6. The value of interpolating polynomial at 1.5 is
P1 15
15 16
13 160 6200860
15 13
16 130 4554022.
. .
. ..
. .
. ..
=0.5102968
Two polynomials of degree 2 could also be used, one by letting
x x x0 1 213 16 19 . , . , . , which gives
2
1.5 1.6 1.5 1.9 0.6200860 1.5 1.3 1.5 1.91.5 0.4554022
1.3 1.6 1.3 1.9 1.6 1.3 1.6 1.9
1.5 1.3 1.5 1.60.2818186
1.9 1.3 1.9 1.6
P
= 0.5112857
In the third – degree core these are two reasonable choices for the polynomial. One
uses:
(i) x0 = 1.3, x1 = 1.6, x2 = 1.9, x3 = 2.2, which gives
P3 15
15 16 15 19 15 2 2
13 16 13 19 13 2 20 6200860.
. . . . . .
. . . . . ..
15 13 15 19 15 2 2
16 13 16 19 16 2 20 4554022
. . . . . .
. . . . . ..
15 13 15 16 15 2 2
19 13 19 16 19 2 20 2818186
. . . . . .
. . . . . ..
15 13 15 16 15 19
2 2 13 2 2 16 2 2 190 403623
. . . . . .
. . . . . ..
= 0.5118302
Although P3 15. is the most accurate approximate .
2.2 Langrange Polynomial Error Formula.
It has the same form as the error formula for the Taylor Polynomials.
f x P xf x
nx x x x x xn
n
n
1
0 11
!...
For some x between x0, x1, …, xn and x. A practical difficulty with Langrage
interpolating is that since the error term in difficult to apply, the degree of polynomial
needed for the desired accuracy is generally not known until computations are
determined. The usual practical is to compute the results given from various
polynomials until appropriate agreement is obtained, as was done in the previous
example.
Recursively generated Langrage Polynomials. Let f be defined at x0, x1, …xk and xj,
xi, be two numbers in this set. If
P x
x x P x x x P x
x x
j j j k i i i k
i j
0 1 1 1 0 1 1 1, ,..., , ,... , ,..., , ,...
Then P(x) is the kth Lagrange polynomial that interpolates f at the kth points x0, x1,
…,xk. To test the correctness of the recursive formula,
Let Q P i i k 0 1 1 1, ,..., , ,... and
Q P j j k 0 1 1 1, ,..., , ,... since Q(x) and Q(x) are polynomials of degree at most k – 1
P x
x x Q x x x Q x
x x
j i
i j
Must be of degree at most k if 0 ≤ r ≤ k with r ≠ j, then Q(xr) = Q(xr) = f(xr), so
P x x x Q x x x Q xx x
x xf x f xr r j r r i r
i j
i j
r r
This result implies that the approximations from the interpolating polynomials can be
generated recursively in the manner shown in the table below:
This procedure is referred to as Neville‟s method.
Example 2.3: In Example 2.2, values of various interpolating polynomials at x = 1.5 were
obtaining using the data shown in the first two columns of the table below. Suppose that we
want to use Neville‟s method to calculate the approximate to f(1.5). If
x x x x and x0 1 2 3 410 13 16 19 2 2 . , . , . , . . , then f(1.0) =
Q0,0, f(1.3) = Q1,0, f(1.6) = Q2,0, f(1.9) = Q3,0, and f(2.2) = Q4,0, so these are the 5 polynomials
of degree zero (constant) that approximate f(1.5). Calculating:
x0 P0 = Q0, 0
x1 P1 = Q1, 0 P0,1 = Q1,1
x2 P2 = Q2, 0 P1, 2 = Q2, 1 P 0,1, 2 = Q2, 2
x3 P3 = Q3, 0 P2, 3 = Q3, 1 P 1,2, 3 = Q3, 2 P 0,1, 2,3 = Q3, 3
x4 P4 = Q4, 0 P3, 4 = Q4, 1 P 2, 3,4 = Q4, 2 P 0,1, 2,3,4 = Q4, 3 P 0,1, 2,3,4 = Q4, 4
Calculating the approximation Q1,1(1.5) gives
Q
Q Q1 1
1 0 0 015
15 10 15 13
13 10,
, ,.
. . . .
. .
05 0 6200860 0 2 0 7651977
0 30523349
. . . .
..
Similarly,
Q2 1 15
15 13 0 4554022 15 16 0 6200860
16 13, .. . . . . .
. .
= 0.5102968
Q3 1 15 05132634, . . and Q4 1 15 05104270, . .
The higher degree approximates are generated in a similar manner and are shown in
the table below:
The best linear approximate is expected to be Q2,1 since 1.5 is between x1 = 1.3 and x2
=1.6
i xi fi = Qi, 0
0 1.0 0.7651977
1 1.3 0.6200860 Q1,1
2 1.6 0.4554022 Q2,1 Q2,2
3 1.9 0.2818186 Q3,1 Q3,2 Q3,3
4 2.2 0.1103623 Q4,1 Q4,2 Q4,3 Q4,4
1.0 0.7651977
1.3 0.6200860 0.5233449
1.6 0.4554022 0.5102968 0.5124715
1.9 0.2818186 0.5132634 0.5112857 0.5118127
2.2 0.1103623 0.5104270 0.5137361 0.5118302 0.5118200
3.0 THE FINITE DIFFERENCE CALCULUS
In conventional calculus the operation of differentiation of a function is a well – defined
formal procedure which the operations highly dependent on the form of the function
involved. Thus we need a technique for differentiating functions by employing only
arithmetic operations. The finite difference calculus satisfies this need.
3.1 Divided differences
Iterated interpolating was used in the previous section to generate successful higher degree
polynomial approximations at a specific point. Divided – difference methods introduced here
are used to successfully generate the polynomials themselves. We first need to introduce the
notation. The zeroth divided difference of the function f w.r.t. xi f xiis simply the value of f
at xi
f x f xi i
The remaining divided differences are defined inductively. The first divided difference of f
w. r. t. x1 and xi+1 is denoted by f x xi i, 1and is defined as
f x xf x f x
x xi i
i i
i i
,
1
1
1
3.2 Forward and Backward Differences.
Consider a function f(x) which is analytic (can be expanded in a Taylor series) in the
neighbourhood of a point x. We find f(x + h) by expanding f(x) in a Taylor series about x:
2 3
...2! 3!
h hf x h f x hf x f x f x (3.1)
Solving equation (3.1) for ( )f x yields
2
....2 6
f x h f x h hf x f x f x
h
(3.2)
Using the Error notation,
f x h f x
f x hh
(3.3)
In words, equation (3.3) states that we have found an expression for the 1st derivative of f w.
r. t. x which is accurate to within an error of order of h. we shall employ the subscript
notation.
f x h f j i (3.4)
jf x f (3.5)
Using the notation, (3.3) becomes
1j jf ff x h
h
(3.6)
We define the first forward difference of f at j as
1j j jf f f (3.7)
The expression for ( )f x may now be written as
jff x h
h
(3.8)
The term f hj / is called a first forward difference approximate of error order h to ( )f x .
Gradually, the expression f f hj j 1 approximates the slope of the function f at the pt x by
the slope of the straight line passing through f(x +h) and f(x).
We now use the Taylor series expansion of f(x) about x to determine f(x – h):
2 3
...2! 3!
h hf x h f x hf x f x f x (3.9)
Solving for ( )f x , we have
2
...2 6
f x f x h h hf x f x f x
h
(3.10)
Or
f x f x h
f x hh
(3.11)
Using the subscript notation,
1j jf ff x h
h
(3.12)
The first backward difference of f at j is defined as
1j j jf f f (3.13)
So that the expression (3.12) for ( )f x may be written as
jff x h
h
(3.14)
The term f hj is called a first background difference approximation of error order h to
( )f x . The geometric interpretation of the approximate is that of the slope of the straight line
connectivity f(x) and f(x – h).
3.3 Approximations to Higher Order Derivatives
Considering Taylor series expansion (3.1) for f(x + h),
2 3
...2! 6
h hf x h f x hf x f x f x (3.15)
Performing a similar expansion about x, f(x + 2h) is found as
3
2 42 2 2 ...
3
hf x h f x hf x h f x f x (3.16)
Multiplying (3.15) by 2, and subtracting (3.15) from (3.16), we have
2
2 2...
f x h f x h f xf x hf x
h
(3.17)
Or, employing the subscript notation,
2 1
2
2j j jf f ff x h
h
(3.18)
We have now found an expression for the second derivative of f w.r.t. x which is accurate to
within an error order of h. The 2nd
forward difference of f at j is defined as
2 12j j j jf f f f
(3.19)
We may rewrite (3.18) for ( )f x as
2
2
jff x h
h
(3.20)
By using the backward expansion (3.9) to obtain f(x - h) and a similar expansion about x to
obtain f(x – 2h), we can find backward difference expression for ( )f x which is accurate to
h :
1 2
2
2j j jf f ff x h
h
(3.21)
The 2nd
backward difference of f at j as defined as
2
2 1 22j j j jf f f f (3.22)
Equation (3.21) may then be written as
2
2
jff x h
h
(3.23)
We may now define the procedure for finding any higher forward and backward differences
of order, say n. Any forward or backward difference may be obtained starting from the 1st
forward and backward difference (3.1) and (3.13) by using the following recurrence
formulas.
1n n
j jf f (3.24)
1n n
j jf f (3.25)
e.g. the second backward difference of f at j may be found as
2
1 1f f f f f fj j j j j j
f f f f f f fj j j j j j j1 1 2 1 22
Forward and backward difference of expression for derivatives of any order are given by
nn
j
xjn n
fd fh
dx h
(3.26)
and
nn
j
xjn n
fd fh
dx h
(3.27)
Forward and backward difference expressions of h are tabulated below for derivatives of
up to 4th
order. It may be a convenient memory aid to note that the coefficients of the forward
difference experience expressions for the nth derivative starting form j and proceeding
forward are given by the coefficient of (- 1)n(a - b)
n in order, while those for the backward
difference expressions starting from j and proceeding backward are given by the coefficients
of (a - b)n in order.
fj fj +1 fj + 2 fj + 3 fj +4
( )hf x -1 1
2 ( )h f x 1 -2 1
3 ( )h f x -1 3 -3 1
4 ( )ivh f x 1 -4 6 -4 1
(a) Forward difference representations
h
fj - 4 fj - 3 fj - 2 fj - 1 fj
( )hf x -1 1
2 ( )h f x 1 -2 1
3 ( )h f x -1 3 -3 1
4 ( )ivh f x 1 -4 6 -4 1
(b) Backward difference representations
3.4 Higher Order Forward and Backward Difference Expressions
The difference expressions for derivatives which we have thus far obtained are of h .
More accurate expressions may be found by simply taking more for example, the series in
equation (3.1) for f(x + h).
2 3
...2! 3!
h hf x h f x hf x f x f x (3.28)
As before, solving for ( )f x yields
2
...2 6
f x h f x h hf x f x f x
h
(3.29)
From (3.17) we have a forward difference expression for ( )f x complete which its error
term. Substituting this expression into (3.29), we obtain
2
2
2 2...... ...
2 6
f x h f x f x h f x h f xh hf x hf x f x
h h
(3.30)
Collecting terms,
22 4 3
...2 3
f x h f x h f x hf x f x
h
(3.31)
Or in subscript notation,
2 1 24 3
2
j j jf f ff x h
h
(3.32)
We have thus found a forward difference representation for the first derivative which is
accurate to h2. Note that the expression is exact for a parabola since the error term
involves only 3rd
and higher derivatives. A similar backward difference expression of h2
could be obtained by using the backward Taylor series expansion of f(x - h) and replacing
h
( )f x by the backward difference expression of h form equation (3.21). Forward and
backward difference expressions of h2 for derivatives of up to 4
th order are tabulated
below.
fj fj + 1 fj + 2 fj + 3 fj + 4 fj + 5
2 ( )jhf x -3 4 -1
2 ( )jh f x 2 -5 4 -1
32 ( )jh f x -5 18 -24 14 -3
4 ( )iv
jh f x 3 -14 26 -24 11 -2
(a) Forward difference representations
fj - 5 fj - 4 fj - 3 fj - 2 fj - 1 fj
2 ( )jhf x 1 -4 3
2 ( )jh f x -1 4 -5 2
32 ( )jh f x 3 -14 24 -18 5
4 ( )iv
jh f x -2 11 -24 26 -14 3
(b)Backward difference representations
Higher order forward and backward difference representations, although rarely used in
practice, can be obtained by replacing successively more terms in the Taylor series
expansions by difference representations of h .
3.5 Central Differences
Consider again the analytic function f(x); the forward and backward Taylor series expansions
about x are respectively.
2 3
...2! 3!
h hf x h f x hf x f x f x (3.33)
2 3
...2! 3!
h hf x h f x hf x f x f x (3.34)
h2
h2
Equation (3.34) – equation (3.33), yields
3
2 ...3!
hf x h f x h hf x f x (3.35)
Or solving for f x ,
2
...2 6
f x h f x h hf x f x
h
(3.36)
Or
2 ...2
f x h f x hf x h
h
(3.37)
Employing subscript notation,
1 1 2 ...2
j jf ff x h
h
(3.38)
Thus, difference representation is called a central difference representation and is accurate to
h2. Note that the expression is exact for polynomials of degree 2 (parabolas) and lower.
An expression of h2for f x is readily obtainable from (3.33) and (3.34) by adding
these equation and solving for f x to yield.
1 1 2
2
2j j jf f ff x h
h
(3.39)
The central difference expressions of h2for derivatives up to 4
th order are tabulated
below. A convenient memory aid for these central difference expressions of h2in terms of
ordinary forward and backward differences is given by
2 2 2
2,
2
n nnj n j n
n
f fd fh n even
dx h
(3.40)
1 2 1 2 2 ,2
n nnj n j n
n n
f fd fh n odd
dx h
(3.41)
fj – 2 fj – 1 fj fj + 1 fj + 2
2 ( )jhf x -1 0 1
2 ( )jh f x 1 -2 1
2 ( )jhf x -1 2 0 -2 1
4 ( )iv
jh f x 1 -4 6 -4 1
Representation of h2
3.6 Difference and Polynomials
Difference expressions for derivatives and polynomials have some distinct relationships
which can be very useful. Thus, if we consider a polynomial of order n, the nth difference
representation taken anywhere along this polynomial will be constant and exactly equal to the
nth derivative regardless of the mesh spacing h (since all of the error terms will be zero).
This knowledge may be used to get some idea of how well a given polynomial will fit data
obtained at a series of equally-spaced points on the independent variable. For example, if the
3rd
differences taken at various values of the independent variable are approximately equal
and the 4th
differences are close to zero, then a cubic polynomial should fit the date relatively
well.
Example 3.1: find the fifth backward difference representation which is of h .
Solution
From the recurrence scheme for difference, the fifth backward difference can be expressed as
5 4f fj j
f f f f f f f f f fj j j j j f j j j j4 6 4 4 6 41 2 3 4 1 2 3 4 5
f f f f f fj j j j j j5 10 10 51 2 3 4 5
and
d f
dx
f
hh
j5
5
5
5
Example 3.2: Given the function tabulated at the points j, j +1, and j + 2 shown in the figure
below, find a three points difference representation for jf .
h2
Solution: We pass the parabola f x Ax Bx C 2 through the points xj = 0,
xj + 1 = h, xj + 2 =3h and solve for (0)f :
jf C (a)
2
1jf Ah Bh C b
2
2 9 3jf Ah Bh C c
2f x Ax B
0f B
Now 0f B , so solving for B yields
2
1j jf Ah Bh f (d)
2
2 9 3j jf Ah Bh f (e)
Eqn (e) – 9 x Eqn (d) yields
f f Bh fj j j 2 19 6 8
Bf f f
h
j j j9 8
6
1 2
1 28 90
6
j j jf f ff
h
Example 3.3:
Find a central difference representation of h2for d f dx5 5/ . From example 3.1,
5
5 1 2 3 4 510 10 5f f f f f f fj j j j j j j
f(x) h 2h
j j +1 j +2 x
5 f j can be found in a similar manner as
5 4f fj j
f f f f f f f f f fj j j j j j j j j j5 4 3 2 1 4 3 2 14 6 4 4 6 4
f f f f f fj j j j j j5 4 3 2 15 10 10 5
Applying equation (3.41) directly
d f
dx
f f
hh
j j5
5
5
2
5
2
2
2
2
Or
5
2 1 1 2 35
5 2
3 2 1 1 2
{ 5 10 10 5
[ 5 10 10 5 ]}/ 2 ( )
j j j j j j
j j j j j j
d ff f f f f f
dx
f f f f f f h h
f f f f f f
hh
j j j j j j3 2 1 1 2 3
5
24 5 5 4
2
Example 3.4: Given the following equally spaced data:
Find (0)f , (2)f , (4)f and (0)f using difference representations which are of h2
.
Solution:
At x = 0, a forward difference must be used since no points are available in the backward
direction.
22 4 1 3 00 1
2 1
f f ff
228 4 33 3 30
0 7 12
f to
At x = 2, we have a choice of several representations. We arbitrary select a central difference
representation of h2
23 1
2 12
f ff
x 0 1 2 3 4
f(x) 30 33 28 12 -22
12 33
2105 1
2. to
At x = 4, a backward difference representation must be employed:
23 4 4 3 24 1
2 1
f f ff
23 22 4 12 2843 1
2to
At x = 0, the forward difference representation of h2
is given as
2
2
2 0 5 1 4 2 30
1
f f f ff h
3 30 5 33 4 28 12
1
2 h
5 12
to
4.0 DIFFERENCE EQUATION
Definition
Suppose U1, U2, U3,…, Un are the terms of a sequence. We now introduce a difference
operator such that the result of operating with on Ur, is defined by
1r r rU U U (4.1)
Where r = 1, 2, 3, …, (n - 1), hence
1 2 1U U U (4.2)
2 3 2U U U (4.3)
And so on. There expression are usually called the first finite difference of U1, U2, U3, . . .,
Un. In the same way the second finite differences 2Ur are defined by
2
1U U U Ur r r r
U U U Ur r r r2 1 1 (4.4)
2 12r r rU U U
Consequently,
2
1 3 2 12U U U U (4.5)
2
2 4 3 22U U U U (4.6)
Higher order finite differences 3 4, ,..., n
r r rU U U may be defined in a similar way.
Any equation which expresses a relation between finite differences
2, ,..., m
r r rU U U is called a difference equation, the order of the equation being
equal to the higher order finite difference term contained in it. e.g.
U Ur r 5 3 (is of order 1)
2 23 2r r rU U U r (is of order 2)
3 2 0U r U Ur r
r
r (is of order 3)
The general mth order linear difference equation therefore has the form.
1 2
0 1 2 1......m m m
r r r m r m ra U a U a U a U a U f r
(4.7)
Where f(n) and a0, a1, …, am are given functions of r. by analogy which the
terminology of o.d.e (4.7) is said constant coefficient type if a0, a1, ………am are
numbers in dependent of r.
Difference equations may always be written in a form involving successive values of
Ur by substituting the definition of 2, ,...r rU U
e.g. using equation (4.1) the equation,
U Ur r 5 3
becomes
1
1
5 3
6 3
r r r
r r
U U U
U
(4.8)
Also the equation,
2 23 2U U U rr r r
becomes
U U U U U U rr r r r r r 2 1 1
22 3 2
2
2 15 6r r rU U U r (4.9)
When written in this form, difference equations are often called recurrence relations.
4.2 Formation of Difference Equations
The following examples illustrate the formation of difference equations from
expressions defining Ur as a function of r.
Example 4.1:
If
U Ar
r 4
where A is an arbitrary constant, then
1
1 4 .4.4 4. 4 4r r r
r rU A A A U
Hence we see that the elimination of the arbitrary constant A leads to a first order difference
equation.
In general, if the expression for Ur contains m arbitrary constants, it must satisfy mth
order
difference equations.
Example 4.2: if
(3)r
rU A B (4.10)
where A and B are arbitrary constants, then
1
1 (3)r
rU A B
(4.11)
And
2
2 (3)r
rU A B
(4.12)
Hence eliminating B from (4.11) and (4.12) using (4.10), we have
1
1 3 3 23
r rr rr
U AU A U A
(4.13)
And
2
2 3 9 83
r rr rr
U AU A U A
(4.14)
Finally, eliminating A from (4.13) and (4.14) we get
2 14 3 0r r rU U U (4.15)
Consequently, (4.10) which contain 2 arbitrary constants must satisfy a 2nd
order difference
equation. It should be clear that the general solution of an mth order linear difference
equation must contain m arbitrary constants.
4.3 Solution of Difference Equations
1. Homogeneous Linear 1st order Equation
1 0r r rU a U (4.16)
Where ar is a given function of r.
By putting r = 1, 2, 3, …, in (4.16), we have
2 1U aU (4.17)
3 2 2 2 1 1U a U a aU (4.18)
4 3 3 3 2 1 1U a U a a aU (4.19)
And so on, giving finally
1 2... 3 2 1 1r r rU a a a a aU (4.20)
Hence, if U1 has a known value, say c, the solution of (4.16) may be written as
1
1
r
r
p
U c ap
(4.21)
2. Inhomogeneous Linear 1st order equation
The equation
1r r r rU a U b (4.22)
Where ar, br are given functions of r, we define the general solution of (4.16) as
r r rU U w (4.23)
Where Vr is the several solution of the homogeneous equation
1 0r r rV a v (4.24)
And where wr is any particular solution of the inhomogeneous equation
1r r r rW a w b (4.25)
Example 4:3: Solve the homogeneous equation
1 4 0r
r rU U (4.26)
Given that 1 2U
We write
2 14 ,U U (4.27)
U U U3
2
2
2
14 4 4
U U U4
3
3
3 2
14 4 4 4
Consequently,
1 2 3 2
14 4 ...4 4 4r r
rU U
1
1
2 4r
p
p
Example 4:4: To solve the inhomogeneous equation
1 2r rU U r (4.28)
The homogeneous equation is given by
1 2 0r rU V (4.29)
and the inhomogeneous equating is
1 2r rW W r (4.30)
The general solution, therefore, is
U V Wr r r
Equation (4.29) is similar to (4.16), so the solution is given by
U V
V V
r r
1
2 1
2
2
V V V3 2 12 2 2
V V V4 3 12 2 2 2
V Vr
r 2 1
1
V r
1
12
where V1 is arbitrary. Hence the solution of (4.28) now depends on our ability to find any one
solution of (4.30). To do this we adopt a trial solution of the form
rW r (4.31)
where α and are to be determined such that (4.31) satisfies (4.30) we find
1 2r v r (4.32)
which gives comparing coefficients
0, (coefficient of r0)
1, (coefficient of r)
Hence 1,
W rr 1
Consequently the general solution is
1
12 1r
rU V r (4.33)
If in addition we are given that U C1 then from (4.33)
U C V1 1
02 1 1
C V 1 2
V C1 2
U c rr
r2 2 11
5.0 NUMERICAL INTEGRATION
Here we are concerned with obtaining estimates of the value of the definite integral
b
a
I f x dx (5.1)
Where f is a function of one real variable x; a and b define the lower and upper limits of the
domain of integration on the x – axis. A formal evaluation of equation (5.1) requires finding a
primitive F which, when differentiated, gives the original function f, i.e.
dF
dxf
In this case,
b
a
dFI dx F b f a
dx (5.2)
A major problem with integration is finding a primitive F, and for this reason, the numerical
integration is required in order to obtain numerical estimates to the value of I. A useful first
step is to develop a visual appreciation of integration. It is often stated that the integral of a
function f between limits x = a and x = b is equal to the area under the graph of y = f(x) on the
interval a x b .
The value of the integral (5.1) equals the area under the graph y = f(x).
Numerical integration can be thought of as the development of numerical methods for
estimating the area of the shaded region.
y
a b x
b
af x dx
y = f(x)
5.1 THE TRAPEZOIDAL RULE
We wish to evaluate the integral
I f x dxa
b
For an integreable function f(x) on the interval a x b . We divide the interval a x b into
n equal subintervals each of width x where
xb a
n
Thus, the trapezoidal rule is presented as
1
2
0
1
22
n
n j
j
xI f f f x
(5.3)
The rule is thus termed, a second order method of numerical integration.
For most reasonably well–behaved functions, it is possible to obtain a much improved
integration technique by estimating the error term involved in (5.3) and this result in
21
0
1
2 5.42 12
n
n j
j
xxI f f f f b f a
Equation (5.4) is called the trapezoidal rule with end correction.
Example 5.1: use trapezoidal rule to evaluate
I xdx sin0
using 3 panels and then include the end correction.
Solution:
f(x) = sin x
n = 3;
xb a
n
0
3 3
i xi sin(xi)
0 0 0.00000
1 3 0.866025
2 2 3 0.866025
3 0.00000
The trapezoidal rule yields
Ix
f f f f
220 3 1 2
3
20 00000 0 0000 2 0866025 0866025. . . .
64 0866025 1813799. .
The end correction is
2
12
xf b f a
cosf x x
23
012
f f
3
120
2
cos cos
3
122 0182770
2
.
Adding the end correction to the previously obtained trapezoidal rule value yields
I 1813799 0182771 199659. . .
This value is indeed closer to the analytical solution (I = 2) than the previous results.
5.2 SIMPSON’S RULE
Simpson‟s rule is a numerical integration technique which is based on the use of parabolic
arcs to approximate f(x) instead of the straight lines employed as the interpolating
polynomials with the trapezoidal rule. The method is given as
1 2
6
0
1 2
4 23
n n
n j j
j j
xI f f f f x
(5.5)
Equation (5.5) is Simpson‟s one – third rule for the entire interval. It is a fourth order method.
The Simpson‟s 3/8th
rule is obtained by substituting the factor x
3with
3
8
xin (5.5) with
which we obtain
1 2
0
1 2
34 2
8j odd j even
n n
n j j
j j
xI f f f f
(5.6)
With the ends correction taken into considering equation 5.5 becomes
2 1
0
2 1
114 16 5.7
15 2j even j odd
n n
n j j
j j
xI f f f f x f a f b
Example 5.2: Using the same integrand in example 5.1, evaluate I using Simpson‟s 1/3rd
and 3/8th
Rule with 3 panels and compared the results.
Solution:
I f x dx xdx 0 0
sin
x
3
Applying the Simpson‟s 1/3rd
rule.
Ix
f f f f
34 20 3 1 2
3
30 0 4 0866025 2 0866025. .
96 0 866025.
31732050.
18145292.
The 3/8th
Rule:
Ix
f f f f 3
84 20 3 1 2
86 0866025.
2 041344.
5.3 ROMBERG INTEGRATION
This powerful and efficient integration technique is based on the use of trapezoidal rule
combines with Richardson extrapolation. In order to describe the algorithm in detail, we
adopt a new notation. The trapezoidal rule estimates of the integral will be denoted as
.
1
22
l
l k
j
xT f a f b f a j x
(5.8)
Where
x b a k 2 1 and l k 2 11 . The number of panels, n, involves in Tl,k is 2k – 1
,
i.e. n = 2k -1
. Thus
Tb a
f a f b1 1 2,
Tb a
f a f b f ab a
1 2 42
2,
Tb a
f a f b f ab a
f ab a
f ab a
1 3 82
22
22
3
4,
e.t.c.
NOTE THAT:
TT b a
f ab a
1 2
1 1
2 2 2,
,
T
T b af a
b af a
b a1 3
1 1
2 4 4
3
4,
,
e.t.c.
The extrapolating is carried out according to
1
, 1, 1 1,1
14
4 1
l
l k l k l klT T T
(5.9)
For example, for l = 2,
T T T2 1 1 2 1 1
1
34, , ,
T T T2 2 1 3 1 2
1
34, , ,
Now for l = 3,
T T T3 1 2 2 2 1
1
1516, , ,
These results can conveniently be arranged in tabular form.
Example 5.3: Consider the integral
Ix
x x dx
5
84 2 1
4
3
0
8
The analytical solution of the integrand is I = 72. Romberg extrapolation should yield this
exact answer in only a few extrapolations.
f xx
x x 5
84 2 1
4
3
So,
f(0) = 1
f(8) = 2560 - 2048 + 16 + 1= 529
and
b –a = 8 – 0 = 8
Trapezoidal approximate with 1 and 2 panels are
T1 1
8
21 529 2120,
T1 1,
T1 2, T2 1,
T1 3, T2 2, T3 1,
T1 4, T2 3, T3 2, T4 1,
. . . . .
. . . . . .
. . . . . . Tl1 1,
T l1, T l2 1, T l3 2, . . . Tl1 2, Tl ,1
Increasin
gly accurate Trap
ezoid
al rule valu
e
T f1 2
2120
2
8
24 1060 4 160 250 8 1 712,
Extrapolating these 2 values yields,
T2 1
1
34 712 2120 242
2
3,
The trapezoidal rule with four panels gives
T f f1 3
712
2
8
42 6 356 2 17 41 240,
Extrapolating T1,2 and T1,3 yields
T2 2
1
34 240 712 82
2
3,
By extrapolating T2,1 and T2,2 according to (5.9) we obtain
T3 1
1
1516 82
2
3242
2
372,
. (which is the exact answer)
The Romberg table obtained so far, is
2120
712 242
2
3
240 822
3 72
The best available trapezoidal rule value of 240 using four panels is still very far from correct
and the greatly accelerated convergence along the diagonal should be apparent. In general of
course, we would not know that the exact answer had been obtained, so another line of the
table would have to be computed. After this computation, the table would be
2120
712 2422
3
240 822
3 72
1281
2 72
2
3 72 72
6.0 NUMERICAL SOLUTION OF ORDINARY DIFFERENTIAL EQUATIONS
All problem involving ordinary differential equations fall into two categories: the initial value
problem and boundary value problems. Initial value problems are those for which conditions
are specified at only one value of the independent variable. These conditions are termed
initial conditions. A typical IVP might be of the form.
Ad y
dtB
dy
dtCy g t y y
dy
dtV
2
2 0 00 0 , ,
On the other hand, Boundary value problems are those for which conditions are specified at
two values of the independent variable. a typical BVP might be of the form.
d y
dxD
dy
dxEy h x y y y L YL
2
2 00 , ,
The problem is a boundary value problem if any conditions are specified at two different
values of the independent variable. Thus,
d y
dxA f xy
4
4
y ydy
dxW
d y
dxV y L yL0 0 00 0
2
2 0 , ,,
is a boundary value problem.
6.1 The General IVP
Any IVP can be represented as a set of one of more coupled first – order ordinary differential
equations, each which an initial condition. For example, the simple harmonic oscillator
described by
2
2
d y dyA B Cy g t
dt dt (6.1)
00y y (6.2)
00dy
Vdt
(6.3)
Can be related by making the substitution
dyZ
dt (6.4)
The differential equation (6.1) can now be written as
dz
A Bz Cy g tdt
(6.5)
With some rearrangement, the problem separated by equations (6.1) – (6.3) can now be
written as
dyz
dt (6.6)
tdz B Cz y g
dt A A A (6.7)
With initial conditions (i.cs.):
00y y (6.8)
00z V (6.9)
Any nth order differential equations can similarly be reduced to a system of n first–order
differential equations.
The general form of any IVP can thus be stated as
11 1 2, ,..., ,n
dyf y y y t
dt (6.10)
22 1 2, ,..., ,n
dyf y y y t
dt
. .
. .
. .
1 2, ,..., ,nn n
dyf y y y t
dt
Subject to the initial conditions:
1 10
2 20
0
0
y y
y y
. (6.11).
.
y yn n0 0
Since any IVP can be expressed as a set of first–order ordinary differential equations, our
concern now will be to consider numerical methods for the solution of first – ordinary
differential equations. We will be concerned with two classes of methods. The first of these
consists of the formulas of the Runge–Kutta type. In these formulas, the desired solution y j1
is obtained in terms of , ,j j jy f y t and f y t, evaluated for various estimated values of y
between t j and t j1 . Thesemethods are self starting because solution is carried directly from
t j to t j1 without requiring values of y or f y t, for t t j .
A second class of methods consists of formulas of the multistep types. These formulas, in
general, requires information for t t j . (see Fig below)
The solution for yj +1 might require the value of yj and values of f(y, t) at each of the points tj,
tj -1, tj – 2 and tj – 3. These multi step formulas are obviously not self – starting.
tj
y
Solution Known to here
Δt
yj Solution desired here
tj + 1 t
●
●
Δt
tj + 1 t
●
● ●
●
Δt Δt
●
Δt
tj tj - 1 tj - 2 tj - 3
yj - 3 yj - 2
yj - 1 yj
yj + 1
y
6.2 THE EULER METHOD
Consider again the 1st – order IVP
0
,
0
dtf y t
dy
y y
(6.12)
The numerical solution could be obtained by replacing dy
dtby a simple forward difference
representation. Thus (6.12) can be approximate by
1,
j j
j j
y yf y t
t
(6.13)
Solving for y j1 yields
1 x ,j j j jy y t f y t (6.14)
Equation (6.14) is called the Euler formula.
Example: 6.1: stating at t = 0 and Δt = 0.1, solve the ordinary differential equation
2dyy
dt (6.15)
The problem can be solved numerically by applying Euler recurrence formula (6.14).
2
1j j jy y t y (6.16)
Stating at t = 0 (j = 0) as Δt = 0.1, we can find y at t = 0.1.
y1
21 01 1 0 9 . .
The exact solution at these points is
yexact 011
1 010 9090909.
..
At t = 0.2
y2
20 9 01 0 9 0819 . . . .
The exact solution is
yexact 0 21
1 0 20833333.
..
After 10 steps of Δt = 0.1, we find
y10 0 4627810 .
10.5
1 1exacty
Although this numerical method is very simple, it is obviously not extremely accurate.
However, because it is so simple, it is convenient to use as an introduction to numerical
technique for ordinary difference equations.
6.3 RUNGE – KUTTA TYPE FORMULAS
These formulas are among the most widely used formula for the numerical solution of
ordinary differential equations.
Their advantages include:
they are easy to program
the step size can be changed as desired without any complication.
they are self – starting
they have good stability characteristics
Their disadvantages are:
they requires significantly more computer time than other methods of
comparable accuracy.
local error estimates are somewhat difficult to obtain.
For a n–step Runge – Kutta process, we may write
1
,n
i i ij j
j
k f x C h y h a k
, i = 1, 2, …, n (6.17)
and
1
n
i i
l
y y h b k
(6.18)
The coefficients of the common procedures are listed in the table below
c1 a11 a12 . . . a n1
c2 a21 a22 . . . a n2
. . . . . . .
. . . . . . .
. . . . . . .
cn an1 an2 . . . ann
b1 b2 . . . bn
0 0
1
0 0
1 1
0.5 0.5
0 0
0.5 0.5
0.1
0 0
0.5 0.5
0.5 0 0.5
1 0 0 1
1
6
1
3
1
3
1
6
0 0
0.5 0.5
1 -1 2 2
1
6
2
3
1
6
(a)
(b)
Euler method
e = 0(h)
(c)
improved Euler
method
e = 0(h2) (d)
Modified Euler
method
e = 0 (h2)
(e)
Runge-Kutta Method
e = 0 (h3) (f)
Runge-Kutta Method
e = 0 (h4)
Employing equations (6.17) and (6.18), we can formulate the formulas presented in table (a)
to (f), e.g.
(a) Euler Method
1 10. , 0.k f x h y h k
f x y,
.y y h k 1 1
y hf x y,
y y hf x yi i i i 1 ,
(b) Improved Euler method
k f x y1 ,
k f x h y k h2 1 ,
. .y y h k k 05 051 2
yh
k k2 1 2
(c) Modified Euler method
k f x y1 ,
k f x h y k h2 1
1
2
1
2
,
y y h b k b k 1 1 2 2
y h b k2 2,
y hk2
(d) Standard Runge – Kutta method
k f x y1 ,
k f x h y hk2 1
1
2
1
2
,
k f x h y h k k3 1 2
1
2
1
20
1
2
, .
f x h y hk
1
2
1
2 2,
k f x h y hk4 3 ,
y yh
k k k k 6
2 21 2 3 4