numerical methods - akademik personel · 2019-03-17 · lecture 4: roots of equations - open...

Numerical Methods

Mustafa YILMAZSchool of Mechanical Engineering

University of Marmara

Lecture 5 –Roots of Equations Open Methods

MATH259 Numerical AnalysisLecture 4: Roots of Equations - Open 2

The following root finding methods will be introduced:

A. Bracketing MethodsA.1. Bisection MethodA.2. Regula Falsi

B. Open MethodsB.1. Fixed Point IterationB.2. Newton Raphson's MethodB.3. Secant Method


B. Open Methods

(a) Bisection method(b) Open method (diverge)(c) Open method (converge)

To find the root for f(x) = 0, we constructa magic formulae

xi+1 = g(xi)to predict the root iteratively until xconverge to a root. However, x may diverge!

They require either only one initial starting value or two that though do not necessarily bracket the root


What you should know about Open Methods

How to construct the magic formulae g(x)?

How can we ensure convergence?

What makes a method converges quickly or diverge?

How fast does a method converge?


B.1. Fixed Point Iteration

• Also known as one-point iteration or successivesubstitution

• To find the root for f(x) = 0, we rearrange f(x) = 0 so that there is an x on one side of the equation.

xxgxf =⇔= )(0)(• If we can solve g(x) = x, we solve f(x) = 0.• We solve g(x) = x by computing

until xi+1 converges to xi.given with)( 01 xxgx ii =+


Fixed Point Iteration –Example

032)( 2 =−+= xxxf

23)(

2332032

2

1

222

iii

xxgx

xxxxxx

−==⇒

−=⇒−=⇒=−+

+

Reason: When x converges, i.e. xi+1 xi

0322

32

3

2

22

1

=−+⇒

−=→

−=+

ii

ii

ii

xx

xxxx


Example

Find root of f(x) = e-x - x = 0. (Answer: α= 0.56714329)

ixi ex −+ =1putWe

i xi εa (%) εt (%)0 0 100.01 1.000000 100.0 76.32 0.367879 171.8 35.13 0.692201 46.9 22.14 0.500473 38.3 11.85 0.606244 17.4 6.896 0.545396 11.2 3.837 0.579612 5.90 2.208 0.560115 3.48 1.249 0.571143 1.93 0.70510 0.564879 1.11 0.399


Two Curve Graphical Method

Demo

The point, x, where the two curves,f1(x) = x andf2(x) = g(x),

intersect is the solution to f(x) = 0.

http://www.apropos-logic.com/nc/FixedPointIteration.html


Fixed Point Iteration

032)( 2 =−−= xxxf

23)(

2332032

2

2

2

2

−=⇒

−=⇒

−=⇒

=−−

xxg

xx

xxxx

23)(

23

03)2(0322

−=⇒

−=⇒

=−−⇒=−−

xxg

xx

xxxx

32)(

32

32032

2

2

+=⇒

+=⇒

+=⇒

=−−

xxg

xx

xxxx

• There are infinite ways to construct g(x) from f(x).

For example,

So which one is better?

(ans: x = 3 or -1)

Case a: Case b: Case c:


Case a

𝑥𝑥𝑖𝑖+1 = 2𝑥𝑥𝑖𝑖 + 3

1. x0 = 42. x1 = 3.316623. x1 = 3.103754. x1 = 3.034395. x1 = 3.011446. x1 = 3.00381

Case b

𝑥𝑥𝑖𝑖+1 =3

𝑥𝑥𝑖𝑖 − 2

Case c

𝑥𝑥𝑖𝑖+1 =𝑥𝑥𝑖𝑖2 − 3

2

1. x0 = 42. x1 = 1.53. x1 = -64. x1 = -0.3755. x1 = -1.2631586. x1 = -0.9193557. -1.027628. -0.9908769. -1.00305

1. x0 = 42. x1 = 6.53. x1 = 19.6254. x1 = 191.070

Converge!

Converge, but slower

Diverge!


How to choose g(x)?

• Can we know

which function

g(x) would

converge to

solution before

we do the

computation?


Convergence of Fixed Point Iteration

By definition 𝐸𝐸𝑖𝑖 = 𝑥𝑥𝑟𝑟 − 𝑥𝑥𝑖𝑖 (1)𝐸𝐸𝑖𝑖+1 = 𝑥𝑥𝑟𝑟 − 𝑥𝑥𝑖𝑖+1 (2)

Fixed point iteration

𝑥𝑥𝑟𝑟 = 𝑔𝑔(𝑥𝑥𝑟𝑟) (3)&

𝑥𝑥𝑖𝑖+1 = 𝑔𝑔(𝑥𝑥𝑖𝑖) (4)

(3) − (4) ⇒ 𝑥𝑥𝑟𝑟 − 𝑥𝑥𝑖𝑖+1 = 𝑔𝑔(𝑥𝑥𝑟𝑟) − 𝑔𝑔(𝑥𝑥𝑖𝑖) (5)Sub (2) in (5) ⇒ 𝐸𝐸𝑖𝑖+1 = 𝑔𝑔(𝑥𝑥𝑟𝑟) − 𝑔𝑔(𝑥𝑥𝑖𝑖) (6)


Convergence of Fixed Point Iteration

According to the derivative mean-value theorem, if a g(x) and g'(x) are continuous over an interval xi ≤ x ≤ α, there exists a value x = ξ within the interval such that

𝑔𝑔𝑔(𝑥𝑥) =𝑔𝑔(𝑥𝑥𝑟𝑟) − 𝑔𝑔(𝑥𝑥𝑖𝑖)

𝑥𝑥𝑟𝑟 − 𝑥𝑥𝑖𝑖

• Therefore, if |g'(x)| < 1, the error decreases with each iteration. If |g'(x)| > 1, the error increase.

• If the derivative is positive, the iterative solution will be monotonic.• If the derivative is negative, the errors will oscillate.

From (1) & (6), we have 𝛿𝛿𝑖𝑖 = 𝛼𝛼 − 𝑥𝑥𝑖𝑖 & 𝛿𝛿𝑖𝑖+1 = 𝑔𝑔(𝛼𝛼) − 𝑔𝑔(𝑥𝑥𝑖𝑖)

Thus(7) ⇒ 𝑔𝑔𝑔(𝑥𝑥) =𝐸𝐸𝑖𝑖+1𝐸𝐸𝑖𝑖

⇒ 𝐸𝐸𝑖𝑖+1 = 𝑔𝑔𝑔(𝑥𝑥)𝐸𝐸𝑖𝑖

(7)


Demo

(a) |g'(x)| < 1, g'(x) is +ve⇒ converge, monotonic

(b) |g'(x)| < 1, g'(x) is -ve⇒ converge, oscillate

(c) |g'(x)| > 1, g'(x) is +ve⇒ diverge, monotonic

(d) |g'(x)| > 1, g'(x) is -ve⇒ diverge, oscillate

http://www.apropos-logic.com/nc/FixedPointIteration.html


Fixed Point Iteration Impl. (as m function) with “eval”

% Fixpointfx=' (exp(x) + 1)/x^2 ';x=2;err=1;xim1=2;i=0;disp('iter xr err(%)');disp('............................')while err > 10^-3;x = eval(fx);aerr=abs((x-xim1)/x)*100;fprintf('%3u %9.6f %13.6f \n', i+1,x,err);xim1=x;i=i+1;end


Fixed Point Iteration Impl. (as m function) with function

% Fixpointfunction fixpointx=65;err=1;xim1=2;i=0;disp('iter xr err(%)');disp('............................')while err > 10^-3;x = fn(x);err=abs((x-xim1)/x)*100;fprintf('%3u %9.6f %13.6f \n', i+1,x,err);xim1=x;i=i+1;end

function f=fn(x)% f=(exp(x) + 1)/x^2;f=14*35/(9.8*(1-exp(-(14*7/x))));

MATH259 Numerical Analysis

Derivation of Newton’s Method

𝐺𝐺𝐺𝐺𝐺𝐺𝐺𝐺𝐺𝐺: 𝑥𝑥𝑖𝑖an initial guess of the root of 𝑓𝑓(𝑥𝑥) = 0𝑄𝑄𝑄𝑄𝐺𝐺𝑄𝑄𝑄𝑄𝐺𝐺𝑄𝑄𝐺𝐺: How do we obtain a better estimate 𝑥𝑥𝑖𝑖+1?____________________________________Taylor Therorem:𝑓𝑓(𝑥𝑥 + ℎ) ≈ 𝑓𝑓(𝑥𝑥) + 𝑓𝑓𝑔(𝑥𝑥)ℎFind ℎ such that 𝑓𝑓(𝑥𝑥 + ℎ) = 0.

⇒ ℎ ≈ −𝑓𝑓(𝑥𝑥)𝑓𝑓𝑔(𝑥𝑥)

A new guess of the root: 𝑥𝑥𝑖𝑖+1 = 𝑥𝑥𝑖𝑖 −𝑓𝑓(𝑥𝑥𝑖𝑖)𝑓𝑓𝑔(𝑥𝑥𝑖𝑖)

Lecture 4: Roots of Equations - Open 18


B.2. Newton-Raphson Method

Use the slope of f(x) to predict the location of the root.

xi+1 is the point where the tangent at xi intersects x-axis.

𝑓𝑓𝑔(𝑥𝑥𝑖𝑖) =𝑓𝑓(𝑥𝑥𝑖𝑖) − 0𝑥𝑥𝑖𝑖 − 𝑥𝑥𝑖𝑖+1

⇒ 𝑥𝑥𝑖𝑖+1 = 𝑥𝑥𝑖𝑖 −𝑓𝑓(𝑥𝑥𝑖𝑖)𝑓𝑓𝑔(𝑥𝑥𝑖𝑖)


Newton-Raphson Method

What would happen when f '(xr ) = 0?

For example, f(x) = (x-1)2 = 0

𝑥𝑥𝑖𝑖+1 = 𝑥𝑥𝑖𝑖 −𝑓𝑓(𝑥𝑥𝑖𝑖)𝑓𝑓𝑔(𝑥𝑥𝑖𝑖)


Error Analysis of Newton-Raphson Method

By definition 𝐸𝐸𝑖𝑖 = 𝑥𝑥𝑟𝑟 − 𝑥𝑥𝑖𝑖 (1)𝐸𝐸𝑖𝑖+1 = 𝑥𝑥𝑟𝑟 − 𝑥𝑥𝑖𝑖+1 (2)

Newton-Raphson method

𝑥𝑥𝑖𝑖+1 = 𝑥𝑥𝑖𝑖 −𝑓𝑓 𝑥𝑥𝑖𝑖𝑓𝑓′ 𝑥𝑥𝑖𝑖

⇒ 𝑓𝑓 𝑥𝑥𝑖𝑖 = 𝑓𝑓′ 𝑥𝑥𝑖𝑖 𝑥𝑥𝑖𝑖 − 𝑥𝑥𝑖𝑖+1⇒ 𝑓𝑓 𝑥𝑥𝑖𝑖 + 𝑓𝑓′ 𝑥𝑥𝑖𝑖 −𝑥𝑥𝑖𝑖 = 𝑓𝑓′ 𝑥𝑥𝑖𝑖 −𝑥𝑥𝑖𝑖+1⇒ 𝑓𝑓(𝑥𝑥𝑖𝑖) + 𝑓𝑓𝑔(𝑥𝑥𝑖𝑖)(𝑥𝑥𝑟𝑟 − 𝑥𝑥𝑖𝑖) = 𝑓𝑓𝑔(𝑥𝑥𝑖𝑖)(𝑥𝑥𝑟𝑟 − 𝑥𝑥𝑖𝑖+1) (3)


Suppose xr is the true value (i.e., 𝑓𝑓(𝑥𝑥𝑟𝑟) = 0).Using Taylor's series

Error Analysis of Newton-Raphson Method

𝑓𝑓(𝑥𝑥𝑟𝑟) = 𝑓𝑓(𝑥𝑥𝑖𝑖) + 𝑓𝑓𝑔(𝑥𝑥𝑖𝑖)(𝑥𝑥𝑟𝑟 − 𝑥𝑥𝑖𝑖) +𝑓𝑓𝑓(𝜉𝜉)

2 (𝑥𝑥𝑟𝑟 − 𝑥𝑥𝑖𝑖)2

⇒ 0 = 𝑓𝑓(𝑥𝑥𝑖𝑖) + 𝑓𝑓𝑔(𝑥𝑥𝑖𝑖)(𝑥𝑥𝑟𝑟 − 𝑥𝑥𝑖𝑖) +𝑓𝑓𝑓(𝜉𝜉)

2 (𝑥𝑥𝑟𝑟 − 𝑥𝑥𝑖𝑖)2

⇒ 0 = 𝑓𝑓𝑔(𝑥𝑥𝑖𝑖)(𝑥𝑥𝑟𝑟 − 𝑥𝑥𝑖𝑖+1) +𝑓𝑓𝑓(𝜉𝜉)

2(𝑥𝑥𝑟𝑟 − 𝑥𝑥𝑖𝑖)2 (from(3))

⇒ 0 = 𝑓𝑓𝑔(𝑥𝑥𝑖𝑖)(𝐸𝐸𝑖𝑖+1) +𝑓𝑓𝑓(𝜉𝜉)

2 (𝐸𝐸𝑖𝑖)2 (from(1)and(2))

⇒ 𝐸𝐸𝑖𝑖+1 =−𝑓𝑓𝑓(𝜉𝜉)2𝑓𝑓𝑔(𝑥𝑥𝑖𝑖)

𝐸𝐸𝑖𝑖2 ≅−𝑓𝑓𝑓(𝑥𝑥𝑟𝑟)2𝑓𝑓′ 𝑥𝑥𝑟𝑟

𝐸𝐸𝑖𝑖2When xi and xr are very close to each other, ξ is between xi and xr.

The iterative process is said to be of second order.


Convergence AnalysisRemarks

When the guess is close enough to a simple root of

the function then Newton’s method is guaranteed to

converge quadratically.

Quadratic convergence means that the number of

correct digits is nearly doubled at each iteration.



Error of the Newton-Raphson Method

Each error is approximately proportional to the square of the previouserror. This means that the number of correct decimal places roughlydoubles with each approximation.

Example: Find the root of 𝑓𝑓 𝑥𝑥 = 𝐺𝐺−𝑥𝑥 − 𝑥𝑥 = 0(Ans: xr = 0.56714329)

𝑥𝑥𝑖𝑖+1 = 𝑥𝑥𝑖𝑖 −𝐺𝐺−𝑥𝑥𝑖𝑖 − 𝑥𝑥𝑖𝑖−𝐺𝐺−𝑥𝑥𝑖𝑖 − 1

Error Analysis

𝑓𝑓𝑔(𝑥𝑥𝑟𝑟) = −𝐺𝐺−𝑥𝑥𝑟𝑟 − 1 = −1.56714329𝑓𝑓𝑓(𝑥𝑥𝑟𝑟) = 𝐺𝐺−𝑥𝑥𝑟𝑟 = 0.56714329


Error Analysis

𝐸𝐸𝑖𝑖+1 ≅𝑓𝑓𝑓(𝑥𝑥𝑟𝑟)

2𝑓𝑓′ 𝑥𝑥𝑟𝑟𝐸𝐸𝑖𝑖2

≅0.56714329

2(−1.56714329)𝐸𝐸𝑖𝑖2

= −0.18095𝐸𝐸𝑖𝑖2

i xi εt (%) |Ei| estimated |Ei+1|

0 0 100 0.56714329 0.0582

1 0.500000000 11.8 0.06714329 0.008158

2 0.566311003 0.147 0.0008323 0.000000125

3 0.567143165 0.0000220 0.000000125 2.83x10-15

4 0.567143290 < 10-8


Newton-Raphson vs. Fixed Point Iteration

Find root of f(x) = e-x - x = 0.(Answer: xr = 0.56714329) 𝑥𝑥𝑖𝑖+1 = 𝐺𝐺−𝑥𝑥𝑖𝑖

i xi εa (%) εt (%)0 0 100.01 1.000000 100.0 76.32 0.367879 171.8 35.13 0.692201 46.9 22.14 0.500473 38.3 11.85 0.606244 17.4 6.896 0.545396 11.2 3.837 0.579612 5.90 2.208 0.560115 3.48 1.249 0.571143 1.93 0.705

10 0.564879 1.11 0.399

i xi εt (%) |δi|

0 0 100 0.56714329

1 0.500000000 11.8 0.06714329

2 0.566311003 0.147 0.0008323

3 0.567143165 0.0000220 0.000000125

4 0.567143290 < 10-8

Newton-Raphson

Fixed Point Iteration with


Pitfalls of the Newton-Raphson Method

• Sometimes slowiteration x

0 0.51 51.652 46.4853 41.83654 37.652855 33.8877565

… …Infinity 1.0000000

𝑓𝑓(𝑥𝑥) = 𝑥𝑥10 − 1



Figure (a)An infection point (f"(x)=0) at the vicinity of a root causes divergence.

Figure (b)A local maximum or minimum causes oscillations.



Figure (c)It may jump from one location close to one root to a location that is several roots away.

Figure (d)A zero slope causes division by zero.


Overcoming the Pitfalls?

• No general convergence criteria for Newton-Raphsonmethod.

• Convergence depends on function nature andaccuracy of initial guess.– A guess that's close to true root is always a better choice– Good knowledge of the functions or graphical analysis can

help you make good guesses• Good software should recognize slow convergence or

divergence.– At the end of computation, the final root estimate should

always be substituted into the original function to verifythe solution.


B.3. Secant Method

Newton-Raphson method needs to compute the derivatives.

The secant method approximate the derivatives by finite divided

difference.

𝑓𝑓𝑔(𝑥𝑥𝑖𝑖) ≅𝑓𝑓(𝑥𝑥𝑖𝑖−1) − 𝑓𝑓(𝑥𝑥𝑖𝑖)

𝑥𝑥𝑖𝑖−1 − 𝑥𝑥𝑖𝑖


𝑥𝑥𝑖𝑖+1 ≅ 𝑥𝑥𝑖𝑖 −𝑓𝑓(𝑥𝑥𝑖𝑖)(𝑥𝑥𝑖𝑖−1 − 𝑥𝑥𝑖𝑖)𝑓𝑓(𝑥𝑥𝑖𝑖−1) − 𝑓𝑓(𝑥𝑥𝑖𝑖)

𝑥𝑥𝑖𝑖+1 = 𝑥𝑥𝑖𝑖 −𝑓𝑓(𝑥𝑥𝑖𝑖)(𝑥𝑥𝑖𝑖−1 − 𝑥𝑥𝑖𝑖)𝑓𝑓(𝑥𝑥𝑖𝑖−1) − 𝑓𝑓(𝑥𝑥𝑖𝑖)

From Newton-Raphson method


Secant Method

The same as the

Newton-Raphson

method except that

the derivative f’(x)

is calculated using

the backward

approximation:


Secant Method -Flowchart



𝐺𝐺 = 𝐺𝐺 + 1

𝑥𝑥0, 𝑥𝑥1, 𝐺𝐺 = 1

εa < εs StopNO Yes


Secant Method – Example

Find root of f(x) = e-x - x = 0 with initial estimate of x-1 = 0 and x0 = 1.0. (Answer: α= 0.56714329)


i xi-1 xi f(xi-1) f(xi) xi+1 εt

0 0 1 1.00000 -0.63212 0.61270 8.0 %1 1 0.61270 -0.63212 -0.07081 0.56384 0.58 %2 0.61270 0.56384 -0.07081 0.00518 0.56717 0.0048 %

Again, compare this results obtained by the Newton-Raphson method and simple fixed point iteration method.


Comparison of the Secant and False-position method

• Both methods use the same expression to compute xr.

• They have different methods for the replacement of the initial values by the new estimate. (see next page)

Secant: 𝑥𝑥𝑖𝑖+1 = 𝑥𝑥𝑖𝑖 −𝑓𝑓(𝑥𝑥𝑖𝑖)(𝑥𝑥𝑖𝑖−1 − 𝑥𝑥𝑖𝑖)𝑓𝑓(𝑥𝑥𝑖𝑖−1) − 𝑓𝑓(𝑥𝑥𝑖𝑖)

Falseposition: 𝑥𝑥𝑟𝑟 = 𝑥𝑥𝑢𝑢 −𝑓𝑓(𝑥𝑥𝑢𝑢)(𝑥𝑥𝑙𝑙 − 𝑥𝑥𝑢𝑢)𝑓𝑓(𝑥𝑥𝑙𝑙) − 𝑓𝑓(𝑥𝑥𝑢𝑢)


xi and xi-1 always

bracket the root in the

false-position method,

but not necessarily in

the secant method.

f(x) = ln x

Secant method diverges

False-position method converges




xef(x) x −= −


Modified Secant Method

• Replace xi-1 - xi by δxi and approximate f'(x) as

• From Newton-Raphson method,

𝑓𝑓𝑔(𝑥𝑥𝑖𝑖) =𝑓𝑓(𝑥𝑥𝑖𝑖 + 𝛿𝛿 ⋅ 𝑥𝑥𝑖𝑖) − 𝑓𝑓(𝑥𝑥𝑖𝑖)

𝛿𝛿 ⋅ 𝑥𝑥𝑖𝑖

𝑥𝑥𝑖𝑖+1= 𝑥𝑥𝑖𝑖 −𝑓𝑓(𝑥𝑥𝑖𝑖)𝑓𝑓𝑔(𝑥𝑥𝑖𝑖)

⇒ 𝑥𝑥𝑖𝑖+1 = 𝑥𝑥𝑖𝑖 −𝛿𝛿 ⋅ 𝑥𝑥𝑖𝑖𝑓𝑓(𝑥𝑥𝑖𝑖)

𝑓𝑓(𝑥𝑥𝑖𝑖 + 𝛿𝛿 ⋅ 𝑥𝑥𝑖𝑖) − 𝑓𝑓(𝑥𝑥𝑖𝑖)

• Needs only one instead of two initial guess points


Modified Secant Method

i xi-1 xi f(xi-1) f(xi) xi+1 εt

0 0 1 1.00000 -0.63212 0.61270 8.0 %1 1 0.61270 -0.63212 -0.07081 0.56384 0.58 %2 0.61270 0.56384 -0.07081 0.00518 0.56717 0.0048 %

i xi xi+δxi f(xi) f(xi+δxi) xi+1

0 1 1.01 -0.63212 -0.64578 0.5372631 0.537263 0.542635 0.047083 0.038579 0.567012 0.56701 0.567143 0.000209 -0.00867 0.567143

Find root of f(x) = e-x - x = 0 with initial estimate of x0 = 1.0 and δ=0.01. (Answer: α= 0.56714329)

Compared with the Secant method


Modified Secant Method – About δ

If δ is too small, the method can be swamped by round-offerror caused by subtractive cancellation in the denominator of

If δ is too big, this technique can become inefficient and evendivergent.

If δ is selected properly, this method provides a goodalternative for cases when developing two initial guess isinconvenient.

𝑥𝑥𝑖𝑖+1 = 𝑥𝑥𝑖𝑖 −𝛿𝛿 ⋅ 𝑥𝑥𝑖𝑖𝑓𝑓(𝑥𝑥𝑖𝑖)

𝑓𝑓(𝑥𝑥𝑖𝑖 + 𝛿𝛿 ⋅ 𝑥𝑥𝑖𝑖) − 𝑓𝑓(𝑥𝑥𝑖𝑖)





Can they handle multiple roots?


Multiple Roots

• A multiple root corresponds to a point where a function is tangent to the x axis.

• For example, this function has a double root.

f(x) = (x – 3)(x – 1)(x – 1)

= x3 – 5x2 + 7x - 3

• For example, this function has a triple root.

f(x) = (x – 3)(x – 1)(x – 1) (x – 1)

= x4 – 6x3 +12x2 - 10x + 3


Multiple Roots

• Odd multiple roots cross the axis.

(Figure (b))

• Even multiple roots do not cross the

axis. (Figure (a) and (c))

– The function does not change sign at

even multiple roots – the bracket

methods can not be used, only open

methods can be used.


Difficulties when we have multiple roots

• Bracketing methods do not work for even multiple roots.

• f(xr) = f'(xr) = 0, so both f(xi) and f'(xi) approach zero near the root. This could result in division by zero. A zero check for f(x) should be incorporated so that the computation stops before f'(x) reaches zero.

• For multiple roots, Newton-Raphson and Secant methods converge linrearly, rather than quadratic convergence.


Problems with Newton’s Method

• If the initial guess of the root is far from the root the method may not converge.

• Newton’s method converges linearly near multiple zeros { f(r) = f’(r) =0 }.

• In such a case, modified algorithms can be used toregain the quadratic convergence.



Modified Newton-Raphson Methods for Multiple Roots

• Suggested Solution 1:

Define 𝑓𝑓 = 𝑓𝑓1/𝑚𝑚, 𝑚𝑚 is the multiplicity of the root𝑓𝑓 𝛼𝛼 = 0 and 𝛼𝛼 is a single root.

𝑥𝑥𝑖𝑖+1 = 𝑥𝑥𝑖𝑖 −𝑓𝑓 𝑥𝑥𝑖𝑖𝑓𝑓′ 𝑥𝑥𝑖𝑖

𝑥𝑥𝑖𝑖+1 = 𝑥𝑥𝑖𝑖 −𝑓𝑓1/𝑚𝑚(𝑥𝑥𝑖𝑖)

1𝑚𝑚𝑓𝑓

1𝑚𝑚−1(𝑥𝑥𝑖𝑖)𝑓𝑓𝑔(𝑥𝑥𝑖𝑖)

𝑥𝑥𝑖𝑖+1 = 𝑥𝑥𝑖𝑖 − 𝑚𝑚𝑓𝑓 𝑥𝑥𝑖𝑖𝑓𝑓′ 𝑥𝑥𝑖𝑖

Disadvantage: work only when m is known.



• Suggested Solution 2:

Define 𝑄𝑄(𝑥𝑥) =𝑓𝑓(𝑥𝑥)𝑓𝑓𝑔(𝑥𝑥)

𝑄𝑄(𝑥𝑥)has roots at all the same locations as 𝑓𝑓(𝑥𝑥).(1)

𝑥𝑥𝑖𝑖+1 = 𝑥𝑥𝑖𝑖 −𝑄𝑄 𝑥𝑥𝑖𝑖𝑄𝑄′ 𝑥𝑥𝑖𝑖

(2)

Differentiate(1) ⇒ 𝑄𝑄𝑔(𝑥𝑥) =𝑓𝑓𝑔(𝑥𝑥𝑖𝑖)𝑓𝑓𝑔(𝑥𝑥) − 𝑓𝑓(𝑥𝑥)𝑓𝑓𝑓(𝑥𝑥)

[𝑓𝑓𝑔(𝑥𝑥)]2(3)

Sub 1 & 3 into 2 ⇒ 𝑥𝑥𝑖𝑖+1 = 𝑥𝑥𝑖𝑖 −𝑓𝑓 𝑥𝑥𝑖𝑖 𝑓𝑓′ 𝑥𝑥𝑖𝑖

[𝑓𝑓𝑔(𝑥𝑥)]2 − 𝑓𝑓(𝑥𝑥𝑖𝑖)𝑓𝑓𝑓(𝑥𝑥𝑖𝑖)


• Original Newton Raphson method

𝑓𝑓(𝑥𝑥) = (𝑥𝑥 − 3)(𝑥𝑥 − 1)(𝑥𝑥 − 1)𝑓𝑓(𝑥𝑥) = 𝑥𝑥3 − 5𝑥𝑥2 + 7𝑥𝑥 − 3

𝑥𝑥𝑖𝑖+1 = 𝑥𝑥𝑖𝑖 −𝑓𝑓(𝑥𝑥)𝑓𝑓𝑔(𝑥𝑥)

𝑥𝑥𝑖𝑖+1 = 𝑥𝑥𝑖𝑖 −𝑥𝑥3 − 5𝑥𝑥2 + 7𝑥𝑥 − 3

3𝑥𝑥2 − 10𝑥𝑥 + 7

i xi εt (%)

0 0 100

1 0.4285714 57

2 0.6857143 31

3 0.8328654 17

4 0.9133290 8.7

5 0.9557833 4.4

6 0.9776551 2.2

The method is linearly convergent toward the true value of 1.0.



• For the modified algorithm

𝑓𝑓(𝑥𝑥) = (𝑥𝑥 − 3)(𝑥𝑥 − 1)(𝑥𝑥 − 1)𝑓𝑓(𝑥𝑥) = 𝑥𝑥3 − 5𝑥𝑥2 + 7𝑥𝑥 − 3


[𝑓𝑓𝑔(𝑥𝑥)]2 − 𝑓𝑓(𝑥𝑥𝑖𝑖)𝑓𝑓𝑓(𝑥𝑥𝑖𝑖)

𝑥𝑥𝑖𝑖+1 = 𝑥𝑥𝑖𝑖 −(𝑥𝑥𝑖𝑖3 − 5𝑥𝑥𝑖𝑖2 + 7𝑥𝑥𝑖𝑖 − 3)(3𝑥𝑥𝑖𝑖2 − 10𝑥𝑥𝑖𝑖 + 7)

(3𝑥𝑥𝑖𝑖2 − 10𝑥𝑥𝑖𝑖 + 7) − (𝑥𝑥𝑖𝑖3 − 5𝑥𝑥𝑖𝑖2 + 7𝑥𝑥𝑖𝑖 − 3)(6𝑥𝑥𝑖𝑖 − 10)

i xi εt (%)0 0 1001 1.105263 112 1.003082 0.313 1.000002 0.00024



• How about their performance on finding the single root?

i Standard εt (%) Modified εt (%)0 4 33 4 331 3.4 13 2.636364 122 3.1 3.3 2.820225 6.03 3.008696 0.29 2.961728 1.34 3.000075 0.0025 2.998479 0.055 3.000000 2x10-7 2.999998 7.7x10-5



• What's the disadvantage of the modified Newton-

Raphson Methods for multiple roots over the original

Newton-Raphson method?

• Note that the Secant method can also be modified in a

similar fashion for multiple roots.



Summary of Open Methods

• Unlike bracketing methods, open methods do not always

converge.

• Open methods, if converge, usually converge more quickly

than bracketing methods.

• Open methods can locate even multiple roots whereas

bracketing methods cannot. (why?)


Study Objectives

• Understand the graphical interpretation of a root

• Understand the differences between bracketing methods and

open methods for root location

• Understand the concept of convergence and divergence

• Know why bracketing methods always converge, whereas open

methods may sometimes diverge

• Realize that convergence of open methods is more likely if the

initial guess is close to the true root.


Study Objectives

• Understand what conditions make a method converge quickly or

diverge

• Understand the concepts of linear and quadratic convergence and

their implications for the efficiencies of the fixed-point-iteration

and Newton-Raphson methods

• Know the fundamental difference between the false-position and

secant methods and how it relates to convergence

• Understand the problems posed by multiple roots and the

modifications available to mitigate them


Analysis of Convergent Rate

𝑔𝑔(𝑥𝑥𝑟𝑟) = 𝑔𝑔(𝑥𝑥𝑖𝑖) + 𝑔𝑔𝑔(𝑥𝑥𝑖𝑖)(𝑥𝑥𝑟𝑟 − 𝑥𝑥𝑖𝑖) +𝑔𝑔𝑓(𝑥𝑥𝑖𝑖)

2! (𝑥𝑥𝑟𝑟 − 𝑥𝑥𝑖𝑖)2+. . .

⇒ 𝑔𝑔(𝑥𝑥𝑟𝑟) − 𝑔𝑔(𝑥𝑥𝑖𝑖) = 𝑔𝑔𝑔(𝑥𝑥𝑖𝑖)(𝑥𝑥𝑟𝑟 − 𝑥𝑥𝑖𝑖) + 𝑔𝑔𝑓(𝑥𝑥𝑖𝑖)2!

(𝑥𝑥𝑟𝑟 − 𝑥𝑥𝑖𝑖)2 + ⋯ (1)

Suppose g(x) converges to the solution xr, then the Taylor series of g(xr) about xi can be expressed as

By definition

𝑥𝑥𝑟𝑟 = g(𝑥𝑥𝑟𝑟), 𝑥𝑥𝑖𝑖+1 = 𝑔𝑔(𝑥𝑥𝑖𝑖), 𝐸𝐸𝑖𝑖+1 = 𝑥𝑥𝑟𝑟 − 𝑥𝑥𝑖𝑖+1, 𝐸𝐸𝑖𝑖 = 𝑥𝑥𝑟𝑟 − 𝑥𝑥𝑖𝑖

21

21

"( )'( )2!

"( )'( ) (2)2!

iir i i i

iii i i

g xx x g x E E

g xE g x E E

+

+

− = + +

⇒ = + +

Thus (1) becomes


Analysis of Convergent Rate

When xi is very close to the solution, we can rewrite (2) as

Suppose g(n) exists and the nth term is the first non-zero term, then

Thus to analyze the convergent rate, we can find the smallest n such that

g(n)(xr) ≠ 0.

(3)2 3

1"( ) ( )'( )2! 3!

r ri ii r i

g x g xE g x E E E+ ≈ + + +

( )

1( )!

nn

iigE E

nξ

+ ≈ Where ξ is between xi and xr


Systems of Nonlinear Equations

𝑓𝑓1(𝑥𝑥1, 𝑥𝑥2, . . . , 𝑥𝑥𝑛𝑛) = 0𝑓𝑓2(𝑥𝑥1, 𝑥𝑥2, . . . , 𝑥𝑥𝑛𝑛) = 0

.

.

.𝑓𝑓𝑛𝑛(𝑥𝑥1, 𝑥𝑥2, . . . , 𝑥𝑥𝑛𝑛) = 0

Example: �𝑓𝑓1(𝑥𝑥,𝑦𝑦) = 𝑥𝑥2 + 𝑥𝑥𝑦𝑦 − 10 = 0𝑓𝑓2(𝑥𝑥,𝑦𝑦) = 𝑦𝑦 + 3𝑥𝑥𝑦𝑦2 − 57 = 0


Systems of Nonlinear Equations

𝑋𝑋𝑖𝑖+1 = 𝑋𝑋𝑖𝑖 −𝐹𝐹(𝑋𝑋𝑘𝑘)𝐹𝐹𝑔(𝑋𝑋𝑘𝑘)

𝐹𝐹(𝑋𝑋) =𝑓𝑓1(𝑥𝑥1, 𝑥𝑥2, . . . )𝑓𝑓2(𝑥𝑥1, 𝑥𝑥2, . . . )

⋮,𝐹𝐹𝑔(𝑋𝑋) =

𝜕𝜕𝑓𝑓1𝜕𝜕𝑥𝑥1


⋮



⋮


�𝑄𝑄(𝑥𝑥,𝑦𝑦) = 0𝐺𝐺(𝑥𝑥,𝑦𝑦) = 0

Newton-Raphson Method for Systems of Nonlinear Equations

xv

yu

yv

xu

xvu

xuv

yy

xv

yu

yv

xu

yuv

yvu

xx

iiii

ii

ii

ii

iiii

ii

ii

ii

∂∂

∂∂

−∂∂

∂∂

∂∂

−∂∂

−=

∂∂

∂∂

−∂∂

∂∂

∂∂

−∂∂

−=

+

+

1

1


Example

• Solve the following system of equations:


𝑦𝑦 + 𝑥𝑥2 − 0.5 − 𝑥𝑥 = 0𝑥𝑥2 − 5𝑥𝑥𝑦𝑦 − 𝑦𝑦 = 0

𝐹𝐹 = 𝑦𝑦 + 𝑥𝑥2 − 0.5 − 𝑥𝑥𝑥𝑥2 − 5𝑥𝑥𝑦𝑦 − 𝑦𝑦

,𝐹𝐹𝑔 = 2𝑥𝑥 − 1 12𝑥𝑥 − 5𝑦𝑦 −5𝑥𝑥 − 1 ,𝑋𝑋0 = 1

0

Initial guess𝑥𝑥 = 1,𝑦𝑦 = 0


Example


Iteration 1:

𝐹𝐹 = 𝑦𝑦 + 𝑥𝑥2 − 0.5 − 𝑥𝑥𝑥𝑥2 − 5𝑥𝑥𝑦𝑦 − 𝑦𝑦

= −0.51 =,𝐹𝐹𝑔 = 2𝑥𝑥 − 1 1

2𝑥𝑥 − 5𝑦𝑦 −5𝑥𝑥 − 1 = 1 12 −6

𝑋𝑋1 = 10 − 1 1

2 −6−1 −0.5

1 = 1.250.25

Iteration2:

𝐹𝐹 = 0.0625−0.25 =,𝐹𝐹𝑔 = 1.5 1

1.25 −7.25

𝑋𝑋2 = 1.250.25 − 1.5 1

1.25 −7.25−1 0.0625

−0.25 = 1.23320.2126


Example


• Solve the following system of equations:

𝑦𝑦 + 𝑥𝑥2 − 1 − 𝑥𝑥 = 0𝑥𝑥2 − 2𝑦𝑦2 − 𝑦𝑦 = 0

𝐹𝐹 = 𝑦𝑦 + 𝑥𝑥2 − 1 − 𝑥𝑥𝑥𝑥2 − 2𝑦𝑦2 − 𝑦𝑦

,𝐹𝐹𝑔 = 2𝑥𝑥 − 1 12𝑥𝑥 −4𝑦𝑦 − 1 , 𝑋𝑋0= 0

0

Initial guess𝑥𝑥 = 0,𝑦𝑦 = 0

𝒊𝒊 0 1 2 3 4 5𝑋𝑋𝑘𝑘 0

0−10

−0.60.2

−0.52870.1969

−0.52570.1980

−0.52570.1980


Fixed-Point Iteration for Systems of Nonlinear Equations

Example: � 𝑓𝑓1(𝑥𝑥,𝑦𝑦) = 𝑥𝑥2 + 𝑥𝑥𝑦𝑦 − 10 = 0𝑓𝑓2(𝑥𝑥,𝑦𝑦) = 𝑦𝑦 + 3𝑥𝑥𝑦𝑦2 − 57 = 0

Solution 1: xi+1 = (10 – xi2)/yi

yi+1 = 57 – 3xiyi2

x0 = 1.5, y0 = 3.5

x1 = 2.21429

y1 = -24.37516

x2 = -0.20910y2 = 429.709

diverges

Solution 2: xi+1 = (10 – xiyi )0.5

yi+1 = ((57 – yi)/3xi)0.5

x0 = 1.5, y0 = 3.5

x1 = 2.17945

y1 = 2.86051

x2 = 1.94053y2 = 3.04955

converges


Summary


Method Pros ConsBisection - Easy, Reliable, Convergent

- One function evaluation per iteration- No knowledge of derivative is needed

- Slow- Needs an interval [a,b] containing the root, i.e., f(a)f(b)<0

Newton - Fast (if near the root)- Two function evaluations per iteration

- May diverge- Needs derivative and an initial guess x0 such that f’(x0) is nonzero

Secant - Fast (slower than Newton)- One function evaluation per iteration- No knowledge of derivative is needed

- May diverge- Needs two initial points guess x0, x1 such that f(x0)- f(x1) is nonzero


Home Work of Chapter 6

From the text book:Problem 6.4, 6.12, 6.22

Additional:None.

numerical methods - akademik personel · 2019-03-17 · lecture 4: roots of equations - open...

Documents