numerical analysis eciv 3306 chapter 6

The Islamic University of Gaza

Faculty of Engineering

Civil Engineering Department

Numerical Analysis

ECIV 3306

Chapter 6

Open Methods & System of Non-linear Eqs

Associate Prof. Mazen Abualtayef Civil Engineering Department, The Islamic University of Gaza

PART II: ROOTS OF EQUATIONS

Roots of Equations

Bracketing Methods

Bisection method

False Position Method

Open Methods

Simple fixed point iteration

Newton Raphson

Secant

Modified Newton Raphson

System of Nonlinear Equations

Roots of polynomials

Muller Method

Open Methods

• Bracketing methods are based on assuming an interval of the function which brackets the root.

• The bracketing methods always converge to the root.

• Open methods are based on formulas that require only a single starting value of x or two starting values that do not necessarily bracket the root.

• These method sometimes diverge from the true root.

Open Methods- Convergence and Divergence Concepts

Converging increments

f(x)

xi xi+1 x

f(x)

xi xi+1 x

Diverging increments

6.1 Simple Fixed-Point Iteration

• Rearrange the function f(x)=0 so that x is on the left side of the equation:

)(

)(0)(

1 ii xgx

xxgxf

• Bracketing methods are “convergent”.

• Fixed-point methods may sometime “diverge”, depending on the stating point (initial guess) and how the function behaves.

6.1 Simple Fixed-Point Iteration

Examples: 1.

2. f(x) = x 2-2x+3 x = g(x)=(x2+3)/2

3. f(x) = sin x x = g(x)= sin x + x

3. f(x) = e-x- x x = g(x)= e-x

xxg

or

xxg

or

xxg

xxxxf

21)(

2)(

2)(

02)(

2

2

Simple Fixed-Point Iteration Convergence

• x = g(x) can be expressed as a pair of equations:

y1= x

y2= g(x)…. (component equations)

• Plot them separately.

Simple Fixed-Point Iteration Convergence

1

1

( )

( )  1

Suppose that the true root:

Subtracting 1 from 2

( ) ( ) (3)

  2r

i i

r i r i

r

x g x

x x x

x

x

g

g

x

g

to compute a new estimate xi+1 as expressed by the iterative formula

Simple Fixed-Point Iteration Convergence

Derivative mean value theorem:

If g(x) are continuous in [a,b] then there exist at least one value of x= within the interval such that:

i.e. there exist one point where the slope parallel to the line joining (a & b)

'

g b g ag

b a

Simple Fixed-Point Iteration Convergence

1

'

'

'

1

'

, 1 ,

'

'

( ) ( ) 

Let and

1.0 theerror decreases with each iteration

1.0 theerror increases with each iteration

r i r i

i r

r i

r i

r i r i

r i r i

t i t i

x x g x g x

a x b x

g x g xg

x x

g x g x x x g

then x x x x g

E g E

If g

If g

'

g b g ag

b a

Simple Fixed-Point Iteration Convergence

• Fixed-point iteration converges if:

( ) 1 (slope of the line ( ) )g x f x x

• When the method converges, the error is roughly proportional to or less than the error of the previous step, therefore it is called “linearly convergent.”

Simple Fixed-Point Iteration-Convergence

1. f(x) is manipulated so that we

get x=g(x) g(x) = e-x 2. Thus, the formula predicting the

new value of x is: xi+1 = e-xi

3. Guess x0 = 0

4. The iterations continues till the

approx. error reaches a certain limiting value

f(x)

Root x

f(x)

x

f(x)=e-x - x

f(x) = e-x - x

Example 6.1: Simple Fixed-Point Iteration

Example 6.1: Simple Fixed-Point Iteration

i xi g(xi) ea% et% 0 0 1.0 1 1.0 0.367879 100.0 76.3 2 0.367879 0.692201 171.8 35.1 3 0.692201 0.500473 46.9 22.1 4 0.500473 0.606244 38.3 11.8 5 0.606244 0.545396 17.4 6.89 6 0.545396 0.579612 11.2 3.83 7 0.579612 0.560115 5.90 2.2 8 0.560115 0.571143 3.48 1.24 9 0.571143 0.564879 1.93 0.705 10 0.564879 1.11 0.399

g(x) = e-x Recall true root is 0.56714329

Flow Chart – Fixed Point

Start

Input: xo , es, maxi

i=0 ea=1.1es

1

Stop

1

while ea< es & i >maxi

i=1 or

xn=0

x0=xn

100%n o

a

n

x x

xe

Print: xo, f(xo) ,ea , i 0

1

nx g x

i i

False

True

6.2 The Newton-Raphson Method

• Most widely used method.

• Based on Taylor series expansion:

)(

)(

)(0

g,Rearrangin

0)f(x when xof value theisroot The

...!2

)()()()(

1

1

1i1i

2

1

i

iii

iiii

iiii

xf

xfxx

xx)(xf)f(x

xxfxxfxfxf

Solve for

Newton-Raphson formula

• A tangent to f(x) at the initial point xi is extended till it meets the x-axis at the improved estimate of the root xi+1.

• The iterations continues till the approx. error reaches a certain limiting value.

f(x)

Root x

xi xi+1

f(x) Slope f /(xi)

f(xi)

)(

)(

)()(

/

/

i

ii1i

1ii

ii

xf

xfxx

xx

0xfxf

6.2 The Newton-Raphson Method

Example 6.3: The Newton Raphson Method

11)(

)(/1

x

x

ix

x

i

i

iii

e

xex

e

xex

xf

xfxx

• Find the root of f (x) = e-x-x= 0 f(xi) = e-x-x and f`(xi)= -e-x-1; thus

Iter. Xi+1 et%

0 0 100

1 0.5 11.8

2 0.566311003 0.147

3 0.567143165 0.00002

4 0.567143290 <10-8

Recall true root is 0.56714329

Flow Chart – Newton Raphson

Start

Input: xo , es, maxi

i=0 ea=1.1es

1

Stop

1

while ea >es & i <maxi

i=1 or

xn=0

x0=xn

100%n o

a

n

x x

xe

Print: xo, f(xo) ,ea , i

0

0 '

0

1

n

f xx x

f x

i i

False

True

Pitfalls of The Newton Raphson Method

6.3 The Secant Method

The derivative is replaced by a backward finite divided difference

)()(

))((

i1i

i1iii1i

xfxf

xxxfxx

Thus, the formula predicting the xi+1 is:

/ 1

1

( ) ( )( ) i i

i

i i

f x f xf x

x x

/ ( )if x

)(

)(/1

i

iii

xf

xfxx

6.3 The Secant Method

•Requires two initial estimates of x , e.g, xo, x1. However, because f(x) is not required to change signs between estimates, it is not classified as a “bracketing” method.

•The secant method has the same properties as Newton’s method. Convergence is not guaranteed for all xo, x1, f(x).

6.3 Secant Method: Example

• Use the Secant method to find the root of e-x-x=0; f(x) = e-x-x and xi-1=0, xi=1 to get xi+1 of the first iteration using:

Iter xi-1 f(xi-1) xi f(xi) xi+1 et%

1 0 1.0 1.0 -0.632 0.613 8.0

2 1.0 -0.632 0.613 -0.0708 0.5638 0.58

3 0.613 -0.0708 0.5638 0.00518 0.5672 0.0048

)()(

))((

1

11

ii

iiiii

xfxf

xxxfxx

xi-1 xi xi+1

Recall true root is 0.56714329

Comparison of convergence of False Position and Secant Methods

False Position Secant Method

Use two estimate xl and xu

Use two estimate xi and xi-1

f(x) must changes signs between xl and xu

f(x) is not required to change signs between xi and xi-1

Xr replaces whichever of the original values yielded a function value with the same sign as f(xr)

Xi+1 replace xi

Xi replace xi-1

Always converge May be diverge

1

1

1

( )( )

( ) ( )

i i i

i i

i i

f x x xx x

f x f x

( )( )

( ) ( )

u l u

r u

l u

f x x xx x

f x f x

Comparison of convergence of False Position and Secant Methods

• Use the false-position and secant methods to find the root of f(x) = lnx. Start computation with xl= xi-1=0.5, xu=xi = 5.

1. False position method

2. Secant method

False Position and Secant Methods

xi-1

xi xu

xl

Although the secant

method may be

divergent, when it

converges it usually

does so at a quicker

rate than the false

position method

• Comparison of

the true percent

relative Errors Et

for the methods to

the determine the

root of

f(x)=e-x-x

Flow Chart – Secant Method

Start

Input: x-1 , x0,es, maxi

i=0 ea=1.1es

1

Stop

1

while ea >es & i < maxi

i=1 or

Xi+1=0

Xi-1=xi

Xi=xi+1

1

1

100%i i

a

i

x x

xe

Print: xi , f(xi) ,ea , i 1

1

1

( )( )

( ) ( )

1

i i i

i i

i i

f x x xx x

f x f x

i i

False

True

6.3.3 Modified Secant Method

Rather than using two initial values, an alternative approach is using a fractional perturbation of the independent variable to estimate

1

( )

( ) ( )

i i

i i

i i i

x f xx x

f x x f x

is a small perturbation fraction

/ ( ) ( )( ) i i i

i

i

f x x f xf x

x

/ ( )if x

Modified Secant Method: Example 6.8

• Use modified secant method to find the root of f(x) = e-x-x, x0=1 and = 0.01. Recall true root is 0.56714329

1

( )

( ) ( )

i i

i i

i i i

x f xx x

f x x f x

6.5 Multiple Roots

x

f(x)= (x-3)(x-1)(x-1)

= x3- 5x2+7x -3

f(x)

1 x

3

Double roots

f(x)= (x-3)(x-1)(x-1)(x-1)

= x4- 6x3+ 125 x2- 10x+3

f(x)

1 3

triple roots

6.5 Multiple Roots

•“Multiple root” corresponds to a point

where a function is tangent to the x-axis.

•Difficulties

- Function does not change sign with double

(or even number of multiple root), therefore,

cannot use bracketing methods.

- Both f(x) and f′(x)=0, division by zero with

Newton’s and Secant methods which may

diverge around this root.

Modified Newton-Raphson Method for Multiple Roots

• Another alternative is introduced such new u(x)=f(x)/f /(x);

• Getting the roots of u(x) using Newton-Raphson technique:

)()()(

)()(

)]([

)()()()()(

)(

)(

//2/

/

1

2/

/////

/1

iii

iiii

i

iiiii

i

iii

xfxfxf

xfxfxx

xf

xfxfxfxfxu

xu

xuxx

This function has roots

at all the same locations

as the original function

Differentiate u(x)=f(x)/f /(x)

Using the Newton-Raphson and Modified Newton-Raphson to evaluate the multiple roots of f(x)= x3-5x2+7x-3 with an initial guess of x0=0

)106)(375()7103(

)7103)(375(

)()()(

)()(

2322

223

//2/

/

1

iiiiii

iiiiii

iii

iiii

xxxxxx

xxxxxx

xfxfxf

xfxfxx

7x10x3

3x7x5xx

xf

xfxx

2

i

i

2

i

3

ii

i

ii1i

)(

)(/

•Newton Raphson formula:

•Modified Newton Raphson formula:

Example 6.10 Modified Newton-Raphson Method for Multiple Roots

Newton Raphson Modified Newton-Raphson

Iter xi et% iter xi et%

0 0 100 0 0 100

1 0.4286 57 1 1.10526 11

2 0.6857 31 2 1.00308 0.31

3 0.83286 17 3 1.000002 00024

4 0.91332 8.7

5 0.95578 4.4

6 0.97766 2.2

• Newton Raphson technique is linearly converging

towards the true value of 1.0 while the Modified Newton

Raphson is quadratically converging.

• For simple roots, modified Newton Raphson is less

efficient and requires more computational effort than the

standard Newton Raphson method.

Modified Newton Raphson Method: Example

6.6 Systems of Nonlinear Equations

• Roots of a set of simultaneous equations:

f1(x1,x2,…….,xn)=0

f2 (x1,x2,…….,xn)=0

. .

fn (x1,x2,…….,xn)=0

• The solution is a set of x values that

simultaneously get the equations to zero.

6.6 Systems of Nonlinear Equations

Example: x2 + xy = 10 and y + 3xy2 = 57

u(x,y) = x2+ xy -10 = 0

v(x,y) = y+ 3xy2 -57 = 0

• The solution will be the value of x and y which makes

u(x,y)=0 and v(x,y)=0

• These are x=2 and y=3

• Numerical methods used are extension of the open

methods for solving single equation; Fixed point

iteration and Newton-Raphson.

1. Use an initial guess x =1.5 and y =3.5

2. The iteration formulae:

xi+1=(10-xi2)/yi and yi+1=57-3xiyi

2

3. First iteration,

x=(10-(1.5)2)/3.5 = 2.21429

y=(57-3(2.21429)(3.5)2 = -24.37516

4. Second iteration:

x=(10-2.214292)/-24.37516 = -0.209

y=57-3(-0.209)(-24.37516)2 = 429.709

5. Solution is diverging so try another iteration formula

6.6 Systems of Nonlinear Equations 1. Fixed Point Iteration

x2 + xy = 10

y + 3xy2 = 57

6.6 Systems of Nonlinear Equations 1. Fixed Point Iteration

1. Using iteration formula:

xi+1=(10-xiyi)1/2 and yi+1=[(57-yi)/3xi]

1/2

First guess: x=1.5 and y=3.5

2. 1st iteration:

x=(10-(1.5)(3.5))1/2=2.17945

y=((57-(3.5))/3(2.17945))1/2=2.86051

3. 2nd iteration:

x=(10-(2.17945)(2.86051))1/2 = 1.94053

y=((57-(2.86051))/3(1.94053))1/2 = 3.04955

4. The approach is converging to true root, x=2 and y=3

x2 + xy = 10

y + 3xy2 = 57

The sufficient condition for convergence for the two-equation case (u(x,y)=0 and v(x,y)=0) are:

1

1

u v

x x

and

u v

y y

6.6 Systems of Nonlinear Equations 1. Fixed Point Iteration

MS Excel: Solver u(x,y)= x2+xy-10 =0

v(x,y)=y+3xy2-57=0