Numerical Analysis

1

EE, NCKU

Tien-Hao Chang (Darby Chang)

In the previous slide Rootfinding

multiplicity

Bisection method

Intermediate Value Theorem

convergence measures

False position

yet another simple enclosure method

advantage and disadvantage in comparison with bisection method

2

In this slide Fixed point iteration scheme

what is a fixed point?

iteration function

convergence

Newtons method

tangent line approximation

convergence

Secant method

3

Rootfinding Simple enclosure

Intermediate Value Theorem

guarantee to converge

convergence rate is slow

bisection and false position

Fixed point iteration

Mean Value Theorem

rapid convergence

loss of guaranteed convergence

4

2.3

5

Fixed Point Iteration Schemes

6

7

There is at least one point on the graph at which the tangent

lines is parallel to the secant line

Mean Value Theorem

= ()

We use a slightly different

formulation

() =

An example of using this theorem

proof the inequality

sin sin

8

9

Fixed points Consider the function sin

thought of as moving the input value of

6

to the output value 1

2

the sine function maps 0 to 0

the sine function fixes the location of 0

= 0 is said to be a fixed point of the

function sin

10

11

Number of fixed points

According to the previous figure, a

trivial question is

how many fixed points of a given

function?

12

13

14

< 1

Only sufficient conditions

Namely, not necessary conditions

it is possible for a function to violate one or more of the hypotheses, yet still have a (possibly unique) fixed point

15

Fixed point iteration

16

Fixed point iteration

If it is known that a function has a

fixed point, one way to approximate

the value of that fixed point is fixed

point iteration scheme

These can be defined as follows:

17

18

In action

http://thomashawk.com/hello/209/1017/1024/Jackson%20Running.jpg

19

20

Any Questions?

21

About fixed point iteration

Relation to rootfinding

Now we know what fixed point

iteration is, but how to apply it on

rootfinding?

More precisely, given a rootfinding

equation, f(x)=x3+x2-3x-3=0, what is its

iteration function g(x)?

22

hint

Iteration function Algebraically transform to the form

=

= 3 + 2 3 3

= 3 + 2 2 3

=3+23

3

Every rootfinding problem can be transformed into any number of fixed point problems

(fortunately or unfortunately?)

23

24

25

In action

http://thomashawk.com/hello/209/1017/1024/Jackson%20Running.jpg

26

Analysis #1 iteration function converges

but to a fixed point outside the interval 1,2

#2 fails to converge

despite attaining values quite close to #1

#3 and #5 converge rapidly

#3 add one correct decimal every iteration

#5 doubles correct decimals every iteration

#4 converges, but very slow

27

Convergence This analysis suggests a trivial question

the fixed point of is justified in our previous

theorem

28

29

30

31

11 0 demonstrates

the importance of the parameter

when 0, rapid

when 1, dramatically slow

when 1

2, roughly the same as the

bisection method

32

Order of convergence of fixed point iteration schemes

33

All about the derivatives,

34

35

36

37

38

Stopping condition

39

40

Two steps

41

The first step

lim

+1

=

lim

+1

= lim

1

lim

= 0

lim

+1

= 0 when > 1

42

The second step

+1

=

+1+

+1

+1

+ +1

lim

+1

= 0

1 0 lim

+1

1 + 0

lim

+1

= 1 when > 1

43

Any Questions?

44

2.3 Fixed Point Iteration Schemes

2.4

45

Newtons Method

46

47

Newtons Method

Definition

48

49

In action

http://thomashawk.com/hello/209/1017/1024/Jackson%20Running.jpg

50

In the previous example

Newtons method used 8 function

evaluations

Bisection method requires 36

evaluations starting from (1,2)

False position requires 31

evaluations starting from (1,2)

51

52

Any Questions?

53

Initial guess

Are these comparisons fair?

= tan 6

0 = 0.48, converges to 0.4510472613

after 5 iterations

0 = 0.4, fails to converges after 5000

iterations

0 = 0, converges to 697.4995475 after 42

iterations

54

example

answer

Initial guess

Are these comparisons fair?

= tan 6

0 = 0.48, converges to 0.4510472613

after 5 iterations

0 = 0.4, fails to converges after 5000

iterations

0 = 0, converges to 697.4995475 after 42

iterations

55

answer

Initial guess

Are these comparisons fair?

= tan 6

0 = 0.48, converges to 0.4510472613

after 5 iterations

0 = 0.4, fails to converges after 5000

iterations

0 = 0, converges to 697.4995475 after 42

iterations

56

0 in Newtons method Not guaranteed to converge

0 = 0.4, fails to converge

May converge to a value very far

from 0

0 = 0, converges to 697.4995475

Heavily dependent on the choice of

0

57

Convergence analysis for Newtons method

58

59

The simplest plan is to apply the general fixed point iteration

convergence theorem

Analysis strategy

To do this, it is must be shown that

there exists such an interval, ,

which contains the root , for which

60

61

62

63

64

Any Questions?

65

Newtons Method

Guaranteed to Converge?

Why sometimes Newtons method

does not converge?

This theorem guarantees that

exists

But it may be very small

66

hint

answer

Newtons Method

Guaranteed to Converge?

Why sometimes Newtons method

does not converge?

This theorem guarantees that

exists

But it may be very small

67

answer

Newtons Method

Guaranteed to Converge?

Why sometimes Newtons method

does not converge?

This theorem guarantees that

exists

But it may be very small

68

69

Oh no! After these annoying analyses, the Newtons method is

still not guaranteed to converge!?

http://img2.timeinc.net/people/i/2007/startracks/071008/brad_pitt300.jpg

Dont worry Actually, there is an intuitive method

Combine Newtons method and

bisection method

Newtons method first

if an approximation falls outside current

interval, then apply bisection method to

obtain a better guess

(Can you write an algorithm for this

method?)

70

Newtons Method

Convergence analysis

At least quadratic

=

2

= 0, since = 0

Stopping condition

1 <

71

72 http://www.dianadepasquale.com/ThinkingMonkey.jpg

Recall that

73

Is Newtons method always faster?

74

75

In action

http://thomashawk.com/hello/209/1017/1024/Jackson%20Running.jpg

76

Any Questions?

77

2.4 Newtons Method

2.5

78

Secant Method

Secant method Because that Newtons method

2 function evaluations per iteration

requires the derivative

Secant method is a variation on either false position or Newtons method

1 additional function evaluation per iteration

does not require the derivative

Lets see the figure first

79

answer

80

Secant method Secant method is a variation on

either false position or Newtons

method

1 additional function evaluation per

iteration

does not require the derivative

does not maintain an interval

+1 is calculated with and 1

81

82

83

84

85

Any Questions?

86

2.5 Secant Method