solutions of equations in one variable [0.125in]3.375in0 ... · numerical analysis (chapter 2)...

92
Solutions of Equations in One Variable Newton’s Method Numerical Analysis (9th Edition) R L Burden & J D Faires Beamer Presentation Slides prepared by John Carroll Dublin City University c 2011 Brooks/Cole, Cengage Learning

Upload: others

Post on 27-Mar-2021

4 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Solutions of Equations in One Variable [0.125in]3.375in0 ... · Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 5 / 33. Derivation Example Convergence Final

Solutions of Equations in One Variable

Newton’s Method

Numerical Analysis (9th Edition)R L Burden & J D Faires

Beamer Presentation Slidesprepared byJohn Carroll

Dublin City University

c© 2011 Brooks/Cole, Cengage Learning

Page 2: Solutions of Equations in One Variable [0.125in]3.375in0 ... · Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 5 / 33. Derivation Example Convergence Final

Derivation Example Convergence Final Remarks

Outline

1 Newton’s Method: Derivation

2 Example using Newton’s Method & Fixed-Point Iteration

3 Convergence using Newton’s Method

4 Final Remarks on Practical Application

Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 2 / 33

Page 3: Solutions of Equations in One Variable [0.125in]3.375in0 ... · Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 5 / 33. Derivation Example Convergence Final

Derivation Example Convergence Final Remarks

Outline

1 Newton’s Method: Derivation

2 Example using Newton’s Method & Fixed-Point Iteration

3 Convergence using Newton’s Method

4 Final Remarks on Practical Application

Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 2 / 33

Page 4: Solutions of Equations in One Variable [0.125in]3.375in0 ... · Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 5 / 33. Derivation Example Convergence Final

Derivation Example Convergence Final Remarks

Outline

1 Newton’s Method: Derivation

2 Example using Newton’s Method & Fixed-Point Iteration

3 Convergence using Newton’s Method

4 Final Remarks on Practical Application

Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 2 / 33

Page 5: Solutions of Equations in One Variable [0.125in]3.375in0 ... · Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 5 / 33. Derivation Example Convergence Final

Derivation Example Convergence Final Remarks

Outline

1 Newton’s Method: Derivation

2 Example using Newton’s Method & Fixed-Point Iteration

3 Convergence using Newton’s Method

4 Final Remarks on Practical Application

Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 2 / 33

Page 6: Solutions of Equations in One Variable [0.125in]3.375in0 ... · Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 5 / 33. Derivation Example Convergence Final

Derivation Example Convergence Final Remarks

Outline

1 Newton’s Method: Derivation

2 Example using Newton’s Method & Fixed-Point Iteration

3 Convergence using Newton’s Method

4 Final Remarks on Practical Application

Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 3 / 33

Page 7: Solutions of Equations in One Variable [0.125in]3.375in0 ... · Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 5 / 33. Derivation Example Convergence Final

Derivation Example Convergence Final Remarks

Newton’s Method

ContextNewton’s (or the Newton-Raphson) method is one of the most powerfuland well-known numerical methods for solving a root-finding problem.

Various ways of introducing Newton’s methodGraphically, as is often done in calculus.As a technique to obtain faster convergence than offered by othertypes of functional iteration.Using Taylor polynomials. We will see there that this particularderivation produces not only the method, but also a bound for theerror of the approximation.

Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 4 / 33

Page 8: Solutions of Equations in One Variable [0.125in]3.375in0 ... · Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 5 / 33. Derivation Example Convergence Final

Derivation Example Convergence Final Remarks

Newton’s Method

ContextNewton’s (or the Newton-Raphson) method is one of the most powerfuland well-known numerical methods for solving a root-finding problem.

Various ways of introducing Newton’s methodGraphically, as is often done in calculus.

As a technique to obtain faster convergence than offered by othertypes of functional iteration.Using Taylor polynomials. We will see there that this particularderivation produces not only the method, but also a bound for theerror of the approximation.

Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 4 / 33

Page 9: Solutions of Equations in One Variable [0.125in]3.375in0 ... · Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 5 / 33. Derivation Example Convergence Final

Derivation Example Convergence Final Remarks

Newton’s Method

ContextNewton’s (or the Newton-Raphson) method is one of the most powerfuland well-known numerical methods for solving a root-finding problem.

Various ways of introducing Newton’s methodGraphically, as is often done in calculus.As a technique to obtain faster convergence than offered by othertypes of functional iteration.

Using Taylor polynomials. We will see there that this particularderivation produces not only the method, but also a bound for theerror of the approximation.

Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 4 / 33

Page 10: Solutions of Equations in One Variable [0.125in]3.375in0 ... · Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 5 / 33. Derivation Example Convergence Final

Derivation Example Convergence Final Remarks

Newton’s Method

ContextNewton’s (or the Newton-Raphson) method is one of the most powerfuland well-known numerical methods for solving a root-finding problem.

Various ways of introducing Newton’s methodGraphically, as is often done in calculus.As a technique to obtain faster convergence than offered by othertypes of functional iteration.Using Taylor polynomials. We will see there that this particularderivation produces not only the method, but also a bound for theerror of the approximation.

Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 4 / 33

Page 11: Solutions of Equations in One Variable [0.125in]3.375in0 ... · Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 5 / 33. Derivation Example Convergence Final

Derivation Example Convergence Final Remarks

Newton’s Method

Derivation

Suppose that f ∈ C2[a, b]. Let p0 ∈ [a, b] be an approximation to psuch that f ′(p0) 6= 0 and |p − p0| is “small.”Consider the first Taylor polynomial for f (x) expanded about p0and evaluated at x = p.

f (p) = f (p0) + (p − p0)f ′(p0) +(p − p0)

2

2f ′′(ξ(p)),

where ξ(p) lies between p and p0.Since f (p) = 0, this equation gives

0 = f (p0) + (p − p0)f ′(p0) +(p − p0)

2

2f ′′(ξ(p)).

Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 5 / 33

Page 12: Solutions of Equations in One Variable [0.125in]3.375in0 ... · Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 5 / 33. Derivation Example Convergence Final

Derivation Example Convergence Final Remarks

Newton’s Method

DerivationSuppose that f ∈ C2[a, b]. Let p0 ∈ [a, b] be an approximation to psuch that f ′(p0) 6= 0 and |p − p0| is “small.”

Consider the first Taylor polynomial for f (x) expanded about p0and evaluated at x = p.

f (p) = f (p0) + (p − p0)f ′(p0) +(p − p0)

2

2f ′′(ξ(p)),

where ξ(p) lies between p and p0.Since f (p) = 0, this equation gives

0 = f (p0) + (p − p0)f ′(p0) +(p − p0)

2

2f ′′(ξ(p)).

Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 5 / 33

Page 13: Solutions of Equations in One Variable [0.125in]3.375in0 ... · Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 5 / 33. Derivation Example Convergence Final

Derivation Example Convergence Final Remarks

Newton’s Method

DerivationSuppose that f ∈ C2[a, b]. Let p0 ∈ [a, b] be an approximation to psuch that f ′(p0) 6= 0 and |p − p0| is “small.”Consider the first Taylor polynomial for f (x) expanded about p0and evaluated at x = p.

f (p) = f (p0) + (p − p0)f ′(p0) +(p − p0)

2

2f ′′(ξ(p)),

where ξ(p) lies between p and p0.

Since f (p) = 0, this equation gives

0 = f (p0) + (p − p0)f ′(p0) +(p − p0)

2

2f ′′(ξ(p)).

Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 5 / 33

Page 14: Solutions of Equations in One Variable [0.125in]3.375in0 ... · Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 5 / 33. Derivation Example Convergence Final

Derivation Example Convergence Final Remarks

Newton’s Method

DerivationSuppose that f ∈ C2[a, b]. Let p0 ∈ [a, b] be an approximation to psuch that f ′(p0) 6= 0 and |p − p0| is “small.”Consider the first Taylor polynomial for f (x) expanded about p0and evaluated at x = p.

f (p) = f (p0) + (p − p0)f ′(p0) +(p − p0)

2

2f ′′(ξ(p)),

where ξ(p) lies between p and p0.Since f (p) = 0, this equation gives

0 = f (p0) + (p − p0)f ′(p0) +(p − p0)

2

2f ′′(ξ(p)).

Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 5 / 33

Page 15: Solutions of Equations in One Variable [0.125in]3.375in0 ... · Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 5 / 33. Derivation Example Convergence Final

Derivation Example Convergence Final Remarks

Newton’s Method

0 = f (p0) + (p − p0)f ′(p0) +(p − p0)

2

2f ′′(ξ(p)).

Derivation (Cont’d)

Newton’s method is derived by assuming that since |p − p0| issmall, the term involving (p − p0)

2 is much smaller, so

0 ≈ f (p0) + (p − p0)f ′(p0).

Solving for p gives

p ≈ p0 −f (p0)

f ′(p0)≡ p1.

Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 6 / 33

Page 16: Solutions of Equations in One Variable [0.125in]3.375in0 ... · Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 5 / 33. Derivation Example Convergence Final

Derivation Example Convergence Final Remarks

Newton’s Method

0 = f (p0) + (p − p0)f ′(p0) +(p − p0)

2

2f ′′(ξ(p)).

Derivation (Cont’d)Newton’s method is derived by assuming that since |p − p0| issmall, the term involving (p − p0)

2 is much smaller, so

0 ≈ f (p0) + (p − p0)f ′(p0).

Solving for p gives

p ≈ p0 −f (p0)

f ′(p0)≡ p1.

Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 6 / 33

Page 17: Solutions of Equations in One Variable [0.125in]3.375in0 ... · Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 5 / 33. Derivation Example Convergence Final

Derivation Example Convergence Final Remarks

Newton’s Method

0 = f (p0) + (p − p0)f ′(p0) +(p − p0)

2

2f ′′(ξ(p)).

Derivation (Cont’d)Newton’s method is derived by assuming that since |p − p0| issmall, the term involving (p − p0)

2 is much smaller, so

0 ≈ f (p0) + (p − p0)f ′(p0).

Solving for p gives

p ≈ p0 −f (p0)

f ′(p0)≡ p1.

Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 6 / 33

Page 18: Solutions of Equations in One Variable [0.125in]3.375in0 ... · Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 5 / 33. Derivation Example Convergence Final

Derivation Example Convergence Final Remarks

Newton’s Method

p ≈ p0 −f (p0)

f ′(p0)≡ p1.

Newton’s Method

This sets the stage for Newton’s method, which starts with an initialapproximation p0 and generates the sequence {pn}∞n=0, by

pn = pn−1 −f (pn−1)

f ′(pn−1)for n ≥ 1

Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 7 / 33

Page 19: Solutions of Equations in One Variable [0.125in]3.375in0 ... · Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 5 / 33. Derivation Example Convergence Final

Derivation Example Convergence Final Remarks

Newton’s Method

p ≈ p0 −f (p0)

f ′(p0)≡ p1.

Newton’s MethodThis sets the stage for Newton’s method, which starts with an initialapproximation p0 and generates the sequence {pn}∞n=0, by

pn = pn−1 −f (pn−1)

f ′(pn−1)for n ≥ 1

Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 7 / 33

Page 20: Solutions of Equations in One Variable [0.125in]3.375in0 ... · Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 5 / 33. Derivation Example Convergence Final

Derivation Example Convergence Final Remarks

Newton’s Method: Using Successive Tangents

xx

y

(p0, f (p0))

(p1, f (p1))

p0

p1p2

pSlope f 9(p0)

y 5 f (x)Slope f 9(p1)

Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 8 / 33

Page 21: Solutions of Equations in One Variable [0.125in]3.375in0 ... · Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 5 / 33. Derivation Example Convergence Final

Derivation Example Convergence Final Remarks

Newton’s AlgorithmTo find a solution to f (x) = 0 given an initial approximation p0:

1. Set i = 0;2. While i ≤ N, do Step 3:

3.1 If f ′(p0) = 0 then Step 5.3.2 Set p = p0 − f (p0)/f ′(p0);3.3 If |p − p0| < TOL then Step 6;3.4 Set i = i + 1;3.5 Set p0 = p;

4. Output a ‘failure to converge within the specified number ofiterations’ message & Stop;

5. Output an appropriate failure message (zero derivative) & Stop;6. Output p

Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 9 / 33

Page 22: Solutions of Equations in One Variable [0.125in]3.375in0 ... · Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 5 / 33. Derivation Example Convergence Final

Derivation Example Convergence Final Remarks

Newton’s AlgorithmTo find a solution to f (x) = 0 given an initial approximation p0:

1. Set i = 0;2. While i ≤ N, do Step 3:

3.1 If f ′(p0) = 0 then Step 5.3.2 Set p = p0 − f (p0)/f ′(p0);3.3 If |p − p0| < TOL then Step 6;3.4 Set i = i + 1;3.5 Set p0 = p;

4. Output a ‘failure to converge within the specified number ofiterations’ message & Stop;

5. Output an appropriate failure message (zero derivative) & Stop;6. Output p

Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 9 / 33

Page 23: Solutions of Equations in One Variable [0.125in]3.375in0 ... · Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 5 / 33. Derivation Example Convergence Final

Derivation Example Convergence Final Remarks

Newton’s AlgorithmTo find a solution to f (x) = 0 given an initial approximation p0:

1. Set i = 0;2. While i ≤ N, do Step 3:

3.1 If f ′(p0) = 0 then Step 5.

3.2 Set p = p0 − f (p0)/f ′(p0);3.3 If |p − p0| < TOL then Step 6;3.4 Set i = i + 1;3.5 Set p0 = p;

4. Output a ‘failure to converge within the specified number ofiterations’ message & Stop;

5. Output an appropriate failure message (zero derivative) & Stop;6. Output p

Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 9 / 33

Page 24: Solutions of Equations in One Variable [0.125in]3.375in0 ... · Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 5 / 33. Derivation Example Convergence Final

Derivation Example Convergence Final Remarks

Newton’s AlgorithmTo find a solution to f (x) = 0 given an initial approximation p0:

1. Set i = 0;2. While i ≤ N, do Step 3:

3.1 If f ′(p0) = 0 then Step 5.3.2 Set p = p0 − f (p0)/f ′(p0);

3.3 If |p − p0| < TOL then Step 6;3.4 Set i = i + 1;3.5 Set p0 = p;

4. Output a ‘failure to converge within the specified number ofiterations’ message & Stop;

5. Output an appropriate failure message (zero derivative) & Stop;6. Output p

Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 9 / 33

Page 25: Solutions of Equations in One Variable [0.125in]3.375in0 ... · Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 5 / 33. Derivation Example Convergence Final

Derivation Example Convergence Final Remarks

Newton’s AlgorithmTo find a solution to f (x) = 0 given an initial approximation p0:

1. Set i = 0;2. While i ≤ N, do Step 3:

3.1 If f ′(p0) = 0 then Step 5.3.2 Set p = p0 − f (p0)/f ′(p0);3.3 If |p − p0| < TOL then Step 6;

3.4 Set i = i + 1;3.5 Set p0 = p;

4. Output a ‘failure to converge within the specified number ofiterations’ message & Stop;

5. Output an appropriate failure message (zero derivative) & Stop;6. Output p

Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 9 / 33

Page 26: Solutions of Equations in One Variable [0.125in]3.375in0 ... · Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 5 / 33. Derivation Example Convergence Final

Derivation Example Convergence Final Remarks

Newton’s AlgorithmTo find a solution to f (x) = 0 given an initial approximation p0:

1. Set i = 0;2. While i ≤ N, do Step 3:

3.1 If f ′(p0) = 0 then Step 5.3.2 Set p = p0 − f (p0)/f ′(p0);3.3 If |p − p0| < TOL then Step 6;3.4 Set i = i + 1;

3.5 Set p0 = p;

4. Output a ‘failure to converge within the specified number ofiterations’ message & Stop;

5. Output an appropriate failure message (zero derivative) & Stop;6. Output p

Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 9 / 33

Page 27: Solutions of Equations in One Variable [0.125in]3.375in0 ... · Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 5 / 33. Derivation Example Convergence Final

Derivation Example Convergence Final Remarks

Newton’s AlgorithmTo find a solution to f (x) = 0 given an initial approximation p0:

1. Set i = 0;2. While i ≤ N, do Step 3:

3.1 If f ′(p0) = 0 then Step 5.3.2 Set p = p0 − f (p0)/f ′(p0);3.3 If |p − p0| < TOL then Step 6;3.4 Set i = i + 1;3.5 Set p0 = p;

4. Output a ‘failure to converge within the specified number ofiterations’ message & Stop;

5. Output an appropriate failure message (zero derivative) & Stop;6. Output p

Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 9 / 33

Page 28: Solutions of Equations in One Variable [0.125in]3.375in0 ... · Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 5 / 33. Derivation Example Convergence Final

Derivation Example Convergence Final Remarks

Newton’s AlgorithmTo find a solution to f (x) = 0 given an initial approximation p0:

1. Set i = 0;2. While i ≤ N, do Step 3:

3.1 If f ′(p0) = 0 then Step 5.3.2 Set p = p0 − f (p0)/f ′(p0);3.3 If |p − p0| < TOL then Step 6;3.4 Set i = i + 1;3.5 Set p0 = p;

4. Output a ‘failure to converge within the specified number ofiterations’ message & Stop;

5. Output an appropriate failure message (zero derivative) & Stop;6. Output p

Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 9 / 33

Page 29: Solutions of Equations in One Variable [0.125in]3.375in0 ... · Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 5 / 33. Derivation Example Convergence Final

Derivation Example Convergence Final Remarks

Newton’s AlgorithmTo find a solution to f (x) = 0 given an initial approximation p0:

1. Set i = 0;2. While i ≤ N, do Step 3:

3.1 If f ′(p0) = 0 then Step 5.3.2 Set p = p0 − f (p0)/f ′(p0);3.3 If |p − p0| < TOL then Step 6;3.4 Set i = i + 1;3.5 Set p0 = p;

4. Output a ‘failure to converge within the specified number ofiterations’ message & Stop;

5. Output an appropriate failure message (zero derivative) & Stop;

6. Output p

Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 9 / 33

Page 30: Solutions of Equations in One Variable [0.125in]3.375in0 ... · Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 5 / 33. Derivation Example Convergence Final

Derivation Example Convergence Final Remarks

Newton’s AlgorithmTo find a solution to f (x) = 0 given an initial approximation p0:

1. Set i = 0;2. While i ≤ N, do Step 3:

3.1 If f ′(p0) = 0 then Step 5.3.2 Set p = p0 − f (p0)/f ′(p0);3.3 If |p − p0| < TOL then Step 6;3.4 Set i = i + 1;3.5 Set p0 = p;

4. Output a ‘failure to converge within the specified number ofiterations’ message & Stop;

5. Output an appropriate failure message (zero derivative) & Stop;6. Output p

Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 9 / 33

Page 31: Solutions of Equations in One Variable [0.125in]3.375in0 ... · Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 5 / 33. Derivation Example Convergence Final

Derivation Example Convergence Final Remarks

Newton’s Method

Stopping Criteria for the AlgorithmVarious stopping procedures can be applied in Step 3.3.

We can select a tolerance ε > 0 and generate p1, . . . , pN until oneof the following conditions is met:

|pN − pN−1| < ε (1)

|pN − pN−1||pN |

< ε, pN 6= 0, or (2)

|f (pN)| < ε (3)

Note that none of these inequalities give precise information aboutthe actual error |pN − p|.

Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 10 / 33

Page 32: Solutions of Equations in One Variable [0.125in]3.375in0 ... · Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 5 / 33. Derivation Example Convergence Final

Derivation Example Convergence Final Remarks

Newton’s Method

Stopping Criteria for the AlgorithmVarious stopping procedures can be applied in Step 3.3.We can select a tolerance ε > 0 and generate p1, . . . , pN until oneof the following conditions is met:

|pN − pN−1| < ε (1)

|pN − pN−1||pN |

< ε, pN 6= 0, or (2)

|f (pN)| < ε (3)

Note that none of these inequalities give precise information aboutthe actual error |pN − p|.

Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 10 / 33

Page 33: Solutions of Equations in One Variable [0.125in]3.375in0 ... · Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 5 / 33. Derivation Example Convergence Final

Derivation Example Convergence Final Remarks

Newton’s Method

Stopping Criteria for the AlgorithmVarious stopping procedures can be applied in Step 3.3.We can select a tolerance ε > 0 and generate p1, . . . , pN until oneof the following conditions is met:

|pN − pN−1| < ε (1)

|pN − pN−1||pN |

< ε, pN 6= 0, or (2)

|f (pN)| < ε (3)

Note that none of these inequalities give precise information aboutthe actual error |pN − p|.

Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 10 / 33

Page 34: Solutions of Equations in One Variable [0.125in]3.375in0 ... · Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 5 / 33. Derivation Example Convergence Final

Derivation Example Convergence Final Remarks

Newton’s Method

Stopping Criteria for the AlgorithmVarious stopping procedures can be applied in Step 3.3.We can select a tolerance ε > 0 and generate p1, . . . , pN until oneof the following conditions is met:

|pN − pN−1| < ε (1)

|pN − pN−1||pN |

< ε, pN 6= 0, or (2)

|f (pN)| < ε (3)

Note that none of these inequalities give precise information aboutthe actual error |pN − p|.

Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 10 / 33

Page 35: Solutions of Equations in One Variable [0.125in]3.375in0 ... · Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 5 / 33. Derivation Example Convergence Final

Derivation Example Convergence Final Remarks

Newton’s Method as a Functional Iteration Technique

Functional Iteration

Newton’s Method

pn = pn−1 −f (pn−1)

f ′(pn−1)for n ≥ 1

can be written in the form

pn = g (pn−1)

withg (pn−1) = pn−1 −

f (pn−1)

f ′(pn−1)for n ≥ 1

Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 11 / 33

Page 36: Solutions of Equations in One Variable [0.125in]3.375in0 ... · Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 5 / 33. Derivation Example Convergence Final

Derivation Example Convergence Final Remarks

Newton’s Method as a Functional Iteration Technique

Functional IterationNewton’s Method

pn = pn−1 −f (pn−1)

f ′(pn−1)for n ≥ 1

can be written in the form

pn = g (pn−1)

withg (pn−1) = pn−1 −

f (pn−1)

f ′(pn−1)for n ≥ 1

Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 11 / 33

Page 37: Solutions of Equations in One Variable [0.125in]3.375in0 ... · Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 5 / 33. Derivation Example Convergence Final

Derivation Example Convergence Final Remarks

Newton’s Method as a Functional Iteration Technique

Functional IterationNewton’s Method

pn = pn−1 −f (pn−1)

f ′(pn−1)for n ≥ 1

can be written in the form

pn = g (pn−1)

withg (pn−1) = pn−1 −

f (pn−1)

f ′(pn−1)for n ≥ 1

Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 11 / 33

Page 38: Solutions of Equations in One Variable [0.125in]3.375in0 ... · Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 5 / 33. Derivation Example Convergence Final

Derivation Example Convergence Final Remarks

Newton’s Method as a Functional Iteration Technique

Functional IterationNewton’s Method

pn = pn−1 −f (pn−1)

f ′(pn−1)for n ≥ 1

can be written in the form

pn = g (pn−1)

withg (pn−1) = pn−1 −

f (pn−1)

f ′(pn−1)for n ≥ 1

Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 11 / 33

Page 39: Solutions of Equations in One Variable [0.125in]3.375in0 ... · Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 5 / 33. Derivation Example Convergence Final

Derivation Example Convergence Final Remarks

Outline

1 Newton’s Method: Derivation

2 Example using Newton’s Method & Fixed-Point Iteration

3 Convergence using Newton’s Method

4 Final Remarks on Practical Application

Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 12 / 33

Page 40: Solutions of Equations in One Variable [0.125in]3.375in0 ... · Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 5 / 33. Derivation Example Convergence Final

Derivation Example Convergence Final Remarks

Newton’s Method

Example: Fixed-Point Iteration & Newton’s MethodConsider the function

f (x) = cos x − x = 0

Approximate a root of f using (a) a fixed-point method, and (b)Newton’s Method

Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 13 / 33

Page 41: Solutions of Equations in One Variable [0.125in]3.375in0 ... · Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 5 / 33. Derivation Example Convergence Final

Derivation Example Convergence Final Remarks

Newton’s Method & Fixed-Point Iteration

y

x

y 5 x

y 5 cos x

1

1 q

Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 14 / 33

Page 42: Solutions of Equations in One Variable [0.125in]3.375in0 ... · Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 5 / 33. Derivation Example Convergence Final

Derivation Example Convergence Final Remarks

Newton’s Method & Fixed-Point Iteration

(a) Fixed-Point Iteration for f (x) = cos x − x

A solution to this root-finding problem is also a solution to thefixed-point problem

x = cos x

and the graph implies that a single fixed-point p lies in [0, π/2].The following table shows the results of fixed-point iteration withp0 = π/4.The best conclusion from these results is that p ≈ 0.74.

Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 15 / 33

Page 43: Solutions of Equations in One Variable [0.125in]3.375in0 ... · Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 5 / 33. Derivation Example Convergence Final

Derivation Example Convergence Final Remarks

Newton’s Method & Fixed-Point Iteration

(a) Fixed-Point Iteration for f (x) = cos x − xA solution to this root-finding problem is also a solution to thefixed-point problem

x = cos x

and the graph implies that a single fixed-point p lies in [0, π/2].

The following table shows the results of fixed-point iteration withp0 = π/4.The best conclusion from these results is that p ≈ 0.74.

Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 15 / 33

Page 44: Solutions of Equations in One Variable [0.125in]3.375in0 ... · Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 5 / 33. Derivation Example Convergence Final

Derivation Example Convergence Final Remarks

Newton’s Method & Fixed-Point Iteration

(a) Fixed-Point Iteration for f (x) = cos x − xA solution to this root-finding problem is also a solution to thefixed-point problem

x = cos x

and the graph implies that a single fixed-point p lies in [0, π/2].The following table shows the results of fixed-point iteration withp0 = π/4.

The best conclusion from these results is that p ≈ 0.74.

Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 15 / 33

Page 45: Solutions of Equations in One Variable [0.125in]3.375in0 ... · Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 5 / 33. Derivation Example Convergence Final

Derivation Example Convergence Final Remarks

Newton’s Method & Fixed-Point Iteration

(a) Fixed-Point Iteration for f (x) = cos x − xA solution to this root-finding problem is also a solution to thefixed-point problem

x = cos x

and the graph implies that a single fixed-point p lies in [0, π/2].The following table shows the results of fixed-point iteration withp0 = π/4.The best conclusion from these results is that p ≈ 0.74.

Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 15 / 33

Page 46: Solutions of Equations in One Variable [0.125in]3.375in0 ... · Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 5 / 33. Derivation Example Convergence Final

Derivation Example Convergence Final Remarks

Newton’s Method & Fixed-Point Iteration

Fixed-Point Iteration: x = cos(x), x0 = π4

n pn−1 pn |pn − pn−1| en/en−11 0.7853982 0.7071068 0.0782914 —2 0.707107 0.760245 0.053138 0.6787193 0.760245 0.724667 0.035577 0.6695254 0.724667 0.748720 0.024052 0.6760645 0.748720 0.732561 0.016159 0.6718266 0.732561 0.743464 0.010903 0.6747537 0.743464 0.736128 0.007336 0.672816

Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 16 / 33

Page 47: Solutions of Equations in One Variable [0.125in]3.375in0 ... · Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 5 / 33. Derivation Example Convergence Final

Derivation Example Convergence Final Remarks

Newton’s Method

(b) Newton’s Method for f (x) = cos x − x

To apply Newton’s method to this problem we need

f ′(x) = − sin x − 1

Starting again with p0 = π/4, we generate the sequence defined,for n ≥ 1, by

pn = pn−1 −f (pn−1)

f (p′n−1)= pn−1 −

cos pn−1 − pn−1

− sin pn−1 − 1.

This gives the approximations shown in the following table.

Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 17 / 33

Page 48: Solutions of Equations in One Variable [0.125in]3.375in0 ... · Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 5 / 33. Derivation Example Convergence Final

Derivation Example Convergence Final Remarks

Newton’s Method

(b) Newton’s Method for f (x) = cos x − xTo apply Newton’s method to this problem we need

f ′(x) = − sin x − 1

Starting again with p0 = π/4, we generate the sequence defined,for n ≥ 1, by

pn = pn−1 −f (pn−1)

f (p′n−1)= pn−1 −

cos pn−1 − pn−1

− sin pn−1 − 1.

This gives the approximations shown in the following table.

Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 17 / 33

Page 49: Solutions of Equations in One Variable [0.125in]3.375in0 ... · Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 5 / 33. Derivation Example Convergence Final

Derivation Example Convergence Final Remarks

Newton’s Method

(b) Newton’s Method for f (x) = cos x − xTo apply Newton’s method to this problem we need

f ′(x) = − sin x − 1

Starting again with p0 = π/4, we generate the sequence defined,for n ≥ 1, by

pn = pn−1 −f (pn−1)

f (p′n−1)= pn−1 −

cos pn−1 − pn−1

− sin pn−1 − 1.

This gives the approximations shown in the following table.

Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 17 / 33

Page 50: Solutions of Equations in One Variable [0.125in]3.375in0 ... · Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 5 / 33. Derivation Example Convergence Final

Derivation Example Convergence Final Remarks

Newton’s Method

(b) Newton’s Method for f (x) = cos x − xTo apply Newton’s method to this problem we need

f ′(x) = − sin x − 1

Starting again with p0 = π/4, we generate the sequence defined,for n ≥ 1, by

pn = pn−1 −f (pn−1)

f (p′n−1)= pn−1 −

cos pn−1 − pn−1

− sin pn−1 − 1.

This gives the approximations shown in the following table.

Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 17 / 33

Page 51: Solutions of Equations in One Variable [0.125in]3.375in0 ... · Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 5 / 33. Derivation Example Convergence Final

Derivation Example Convergence Final Remarks

Newton’s Method

Newton’s Method for f (x) = cos(x)− x , x0 = π4

n pn−1 f (pn−1) f ′ (pn−1) pn |pn − pn−1|1 0.78539816 -0.078291 -1.707107 0.73953613 0.045862032 0.73953613 -0.000755 -1.673945 0.73908518 0.000450963 0.73908518 -0.000000 -1.673612 0.73908513 0.000000044 0.73908513 -0.000000 -1.673612 0.73908513 0.00000000

An excellent approximation is obtained with n = 3.Because of the agreement of p3 and p4 we could reasonablyexpect this result to be accurate to the places listed.

Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 18 / 33

Page 52: Solutions of Equations in One Variable [0.125in]3.375in0 ... · Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 5 / 33. Derivation Example Convergence Final

Derivation Example Convergence Final Remarks

Newton’s Method

Newton’s Method for f (x) = cos(x)− x , x0 = π4

n pn−1 f (pn−1) f ′ (pn−1) pn |pn − pn−1|1 0.78539816 -0.078291 -1.707107 0.73953613 0.045862032 0.73953613 -0.000755 -1.673945 0.73908518 0.000450963 0.73908518 -0.000000 -1.673612 0.73908513 0.000000044 0.73908513 -0.000000 -1.673612 0.73908513 0.00000000

An excellent approximation is obtained with n = 3.

Because of the agreement of p3 and p4 we could reasonablyexpect this result to be accurate to the places listed.

Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 18 / 33

Page 53: Solutions of Equations in One Variable [0.125in]3.375in0 ... · Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 5 / 33. Derivation Example Convergence Final

Derivation Example Convergence Final Remarks

Newton’s Method

Newton’s Method for f (x) = cos(x)− x , x0 = π4

n pn−1 f (pn−1) f ′ (pn−1) pn |pn − pn−1|1 0.78539816 -0.078291 -1.707107 0.73953613 0.045862032 0.73953613 -0.000755 -1.673945 0.73908518 0.000450963 0.73908518 -0.000000 -1.673612 0.73908513 0.000000044 0.73908513 -0.000000 -1.673612 0.73908513 0.00000000

An excellent approximation is obtained with n = 3.Because of the agreement of p3 and p4 we could reasonablyexpect this result to be accurate to the places listed.

Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 18 / 33

Page 54: Solutions of Equations in One Variable [0.125in]3.375in0 ... · Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 5 / 33. Derivation Example Convergence Final

Derivation Example Convergence Final Remarks

Outline

1 Newton’s Method: Derivation

2 Example using Newton’s Method & Fixed-Point Iteration

3 Convergence using Newton’s Method

4 Final Remarks on Practical Application

Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 19 / 33

Page 55: Solutions of Equations in One Variable [0.125in]3.375in0 ... · Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 5 / 33. Derivation Example Convergence Final

Derivation Example Convergence Final Remarks

Convergence using Newton’s Method

Theoretical importance of the choice of p0

The Taylor series derivation of Newton’s method points out theimportance of an accurate initial approximation.The crucial assumption is that the term involving (p − p0)

2 is, bycomparison with |p − p0|, so small that it can be deleted.This will clearly be false unless p0 is a good approximation to p.If p0 is not sufficiently close to the actual root, there is little reasonto suspect that Newton’s method will converge to the root.However, in some instances, even poor initial approximations willproduce convergence.

Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 20 / 33

Page 56: Solutions of Equations in One Variable [0.125in]3.375in0 ... · Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 5 / 33. Derivation Example Convergence Final

Derivation Example Convergence Final Remarks

Convergence using Newton’s Method

Theoretical importance of the choice of p0

The Taylor series derivation of Newton’s method points out theimportance of an accurate initial approximation.

The crucial assumption is that the term involving (p − p0)2 is, by

comparison with |p − p0|, so small that it can be deleted.This will clearly be false unless p0 is a good approximation to p.If p0 is not sufficiently close to the actual root, there is little reasonto suspect that Newton’s method will converge to the root.However, in some instances, even poor initial approximations willproduce convergence.

Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 20 / 33

Page 57: Solutions of Equations in One Variable [0.125in]3.375in0 ... · Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 5 / 33. Derivation Example Convergence Final

Derivation Example Convergence Final Remarks

Convergence using Newton’s Method

Theoretical importance of the choice of p0

The Taylor series derivation of Newton’s method points out theimportance of an accurate initial approximation.The crucial assumption is that the term involving (p − p0)

2 is, bycomparison with |p − p0|, so small that it can be deleted.

This will clearly be false unless p0 is a good approximation to p.If p0 is not sufficiently close to the actual root, there is little reasonto suspect that Newton’s method will converge to the root.However, in some instances, even poor initial approximations willproduce convergence.

Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 20 / 33

Page 58: Solutions of Equations in One Variable [0.125in]3.375in0 ... · Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 5 / 33. Derivation Example Convergence Final

Derivation Example Convergence Final Remarks

Convergence using Newton’s Method

Theoretical importance of the choice of p0

The Taylor series derivation of Newton’s method points out theimportance of an accurate initial approximation.The crucial assumption is that the term involving (p − p0)

2 is, bycomparison with |p − p0|, so small that it can be deleted.This will clearly be false unless p0 is a good approximation to p.

If p0 is not sufficiently close to the actual root, there is little reasonto suspect that Newton’s method will converge to the root.However, in some instances, even poor initial approximations willproduce convergence.

Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 20 / 33

Page 59: Solutions of Equations in One Variable [0.125in]3.375in0 ... · Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 5 / 33. Derivation Example Convergence Final

Derivation Example Convergence Final Remarks

Convergence using Newton’s Method

Theoretical importance of the choice of p0

The Taylor series derivation of Newton’s method points out theimportance of an accurate initial approximation.The crucial assumption is that the term involving (p − p0)

2 is, bycomparison with |p − p0|, so small that it can be deleted.This will clearly be false unless p0 is a good approximation to p.If p0 is not sufficiently close to the actual root, there is little reasonto suspect that Newton’s method will converge to the root.

However, in some instances, even poor initial approximations willproduce convergence.

Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 20 / 33

Page 60: Solutions of Equations in One Variable [0.125in]3.375in0 ... · Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 5 / 33. Derivation Example Convergence Final

Derivation Example Convergence Final Remarks

Convergence using Newton’s Method

Theoretical importance of the choice of p0

The Taylor series derivation of Newton’s method points out theimportance of an accurate initial approximation.The crucial assumption is that the term involving (p − p0)

2 is, bycomparison with |p − p0|, so small that it can be deleted.This will clearly be false unless p0 is a good approximation to p.If p0 is not sufficiently close to the actual root, there is little reasonto suspect that Newton’s method will converge to the root.However, in some instances, even poor initial approximations willproduce convergence.

Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 20 / 33

Page 61: Solutions of Equations in One Variable [0.125in]3.375in0 ... · Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 5 / 33. Derivation Example Convergence Final

Derivation Example Convergence Final Remarks

Convergence using Newton’s Method

Convergence Theorem for Newton’s Method

Let f ∈ C2[a, b]. If p ∈ (a, b) is such that f (p) = 0 and f ′(p) 6= 0.Then there exists a δ > 0 such that Newton’s method generates asequence {pn}∞n=1, defined by

pn = pn−1 −f (pn−1)

f (p′n−1)

converging to p for any initial approximation

p0 ∈ [p − δ, p + δ]

Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 21 / 33

Page 62: Solutions of Equations in One Variable [0.125in]3.375in0 ... · Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 5 / 33. Derivation Example Convergence Final

Derivation Example Convergence Final Remarks

Convergence using Newton’s Method

Convergence Theorem for Newton’s Method

Let f ∈ C2[a, b]. If p ∈ (a, b) is such that f (p) = 0 and f ′(p) 6= 0.

Then there exists a δ > 0 such that Newton’s method generates asequence {pn}∞n=1, defined by

pn = pn−1 −f (pn−1)

f (p′n−1)

converging to p for any initial approximation

p0 ∈ [p − δ, p + δ]

Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 21 / 33

Page 63: Solutions of Equations in One Variable [0.125in]3.375in0 ... · Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 5 / 33. Derivation Example Convergence Final

Derivation Example Convergence Final Remarks

Convergence using Newton’s Method

Convergence Theorem for Newton’s Method

Let f ∈ C2[a, b]. If p ∈ (a, b) is such that f (p) = 0 and f ′(p) 6= 0.Then there exists a δ > 0 such that Newton’s method generates asequence {pn}∞n=1, defined by

pn = pn−1 −f (pn−1)

f (p′n−1)

converging to p for any initial approximation

p0 ∈ [p − δ, p + δ]

Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 21 / 33

Page 64: Solutions of Equations in One Variable [0.125in]3.375in0 ... · Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 5 / 33. Derivation Example Convergence Final

Derivation Example Convergence Final Remarks

Convergence using Newton’s Method

Convergence Theorem (1/4)

The proof is based on analyzing Newton’s method as thefunctional iteration scheme pn = g(pn−1), for n ≥ 1, with

g(x) = x − f (x)

f ′(x).

Let k be in (0, 1). We first find an interval [p− δ, p + δ] that g mapsinto itself and for which |g′(x)| ≤ k , for all x ∈ (p − δ, p + δ).Since f ′ is continuous and f ′(p) 6= 0, part (a) of Exercise 29 inSection 1.1 Ex 29 implies that there exists a δ1 > 0, such thatf ′(x) 6= 0 for x ∈ [p − δ1, p + δ1] ⊆ [a, b].

Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 22 / 33

Page 65: Solutions of Equations in One Variable [0.125in]3.375in0 ... · Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 5 / 33. Derivation Example Convergence Final

Derivation Example Convergence Final Remarks

Convergence using Newton’s Method

Convergence Theorem (1/4)The proof is based on analyzing Newton’s method as thefunctional iteration scheme pn = g(pn−1), for n ≥ 1, with

g(x) = x − f (x)

f ′(x).

Let k be in (0, 1). We first find an interval [p− δ, p + δ] that g mapsinto itself and for which |g′(x)| ≤ k , for all x ∈ (p − δ, p + δ).Since f ′ is continuous and f ′(p) 6= 0, part (a) of Exercise 29 inSection 1.1 Ex 29 implies that there exists a δ1 > 0, such thatf ′(x) 6= 0 for x ∈ [p − δ1, p + δ1] ⊆ [a, b].

Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 22 / 33

Page 66: Solutions of Equations in One Variable [0.125in]3.375in0 ... · Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 5 / 33. Derivation Example Convergence Final

Derivation Example Convergence Final Remarks

Convergence using Newton’s Method

Convergence Theorem (1/4)The proof is based on analyzing Newton’s method as thefunctional iteration scheme pn = g(pn−1), for n ≥ 1, with

g(x) = x − f (x)

f ′(x).

Let k be in (0, 1). We first find an interval [p− δ, p + δ] that g mapsinto itself and for which |g′(x)| ≤ k , for all x ∈ (p − δ, p + δ).

Since f ′ is continuous and f ′(p) 6= 0, part (a) of Exercise 29 inSection 1.1 Ex 29 implies that there exists a δ1 > 0, such thatf ′(x) 6= 0 for x ∈ [p − δ1, p + δ1] ⊆ [a, b].

Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 22 / 33

Page 67: Solutions of Equations in One Variable [0.125in]3.375in0 ... · Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 5 / 33. Derivation Example Convergence Final

Derivation Example Convergence Final Remarks

Convergence using Newton’s Method

Convergence Theorem (1/4)The proof is based on analyzing Newton’s method as thefunctional iteration scheme pn = g(pn−1), for n ≥ 1, with

g(x) = x − f (x)

f ′(x).

Let k be in (0, 1). We first find an interval [p− δ, p + δ] that g mapsinto itself and for which |g′(x)| ≤ k , for all x ∈ (p − δ, p + δ).Since f ′ is continuous and f ′(p) 6= 0, part (a) of Exercise 29 inSection 1.1 Ex 29 implies that there exists a δ1 > 0, such thatf ′(x) 6= 0 for x ∈ [p − δ1, p + δ1] ⊆ [a, b].

Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 22 / 33

Page 68: Solutions of Equations in One Variable [0.125in]3.375in0 ... · Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 5 / 33. Derivation Example Convergence Final

Derivation Example Convergence Final Remarks

Convergence using Newton’s Method

Convergence Theorem (2/4)

Thus g is defined and continuous on [p − δ1, p + δ1]. Also

g′(x) = 1− f ′(x)f ′(x)− f (x)f ′′(x)

[f ′(x)]2=

f (x)f ′′(x)

[f ′(x)]2,

for x ∈ [p − δ1, p + δ1], and, since f ∈ C2[a, b], we haveg ∈ C1[p − δ1, p + δ1].By assumption, f (p) = 0, so

g′(p) =f (p)f ′′(p)

[f ′(p)]2= 0.

Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 23 / 33

Page 69: Solutions of Equations in One Variable [0.125in]3.375in0 ... · Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 5 / 33. Derivation Example Convergence Final

Derivation Example Convergence Final Remarks

Convergence using Newton’s Method

Convergence Theorem (2/4)Thus g is defined and continuous on [p − δ1, p + δ1]. Also

g′(x) = 1− f ′(x)f ′(x)− f (x)f ′′(x)

[f ′(x)]2=

f (x)f ′′(x)

[f ′(x)]2,

for x ∈ [p − δ1, p + δ1], and, since f ∈ C2[a, b], we haveg ∈ C1[p − δ1, p + δ1].

By assumption, f (p) = 0, so

g′(p) =f (p)f ′′(p)

[f ′(p)]2= 0.

Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 23 / 33

Page 70: Solutions of Equations in One Variable [0.125in]3.375in0 ... · Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 5 / 33. Derivation Example Convergence Final

Derivation Example Convergence Final Remarks

Convergence using Newton’s Method

Convergence Theorem (2/4)Thus g is defined and continuous on [p − δ1, p + δ1]. Also

g′(x) = 1− f ′(x)f ′(x)− f (x)f ′′(x)

[f ′(x)]2=

f (x)f ′′(x)

[f ′(x)]2,

for x ∈ [p − δ1, p + δ1], and, since f ∈ C2[a, b], we haveg ∈ C1[p − δ1, p + δ1].By assumption, f (p) = 0, so

g′(p) =f (p)f ′′(p)

[f ′(p)]2= 0.

Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 23 / 33

Page 71: Solutions of Equations in One Variable [0.125in]3.375in0 ... · Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 5 / 33. Derivation Example Convergence Final

Derivation Example Convergence Final Remarks

Convergence using Newton’s Method

g′(p) =f (p)f ′′(p)

[f ′(p)]2= 0.

Convergence Theorem (3/4)

Since g′ is continuous and 0 < k < 1, part (b) of Exercise 29 inSection 1.1 Ex 29 implies that there exists a δ, with 0 < δ < δ1,and

|g′(x)| ≤ k , for all x ∈ [p − δ, p + δ].

It remains to show that g maps [p − δ, p + δ] into [p − δ, p + δ].If x ∈ [p − δ, p + δ], the Mean Value Theorem MVT implies that forsome number ξ between x and p, |g(x)−g(p)| = |g′(ξ)||x −p|. So

|g(x)− p| = |g(x)− g(p)| = |g′(ξ)||x − p| ≤ k |x − p| < |x − p|.

Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 24 / 33

Page 72: Solutions of Equations in One Variable [0.125in]3.375in0 ... · Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 5 / 33. Derivation Example Convergence Final

Derivation Example Convergence Final Remarks

Convergence using Newton’s Method

g′(p) =f (p)f ′′(p)

[f ′(p)]2= 0.

Convergence Theorem (3/4)Since g′ is continuous and 0 < k < 1, part (b) of Exercise 29 inSection 1.1 Ex 29 implies that there exists a δ, with 0 < δ < δ1,and

|g′(x)| ≤ k , for all x ∈ [p − δ, p + δ].

It remains to show that g maps [p − δ, p + δ] into [p − δ, p + δ].If x ∈ [p − δ, p + δ], the Mean Value Theorem MVT implies that forsome number ξ between x and p, |g(x)−g(p)| = |g′(ξ)||x −p|. So

|g(x)− p| = |g(x)− g(p)| = |g′(ξ)||x − p| ≤ k |x − p| < |x − p|.

Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 24 / 33

Page 73: Solutions of Equations in One Variable [0.125in]3.375in0 ... · Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 5 / 33. Derivation Example Convergence Final

Derivation Example Convergence Final Remarks

Convergence using Newton’s Method

g′(p) =f (p)f ′′(p)

[f ′(p)]2= 0.

Convergence Theorem (3/4)Since g′ is continuous and 0 < k < 1, part (b) of Exercise 29 inSection 1.1 Ex 29 implies that there exists a δ, with 0 < δ < δ1,and

|g′(x)| ≤ k , for all x ∈ [p − δ, p + δ].

It remains to show that g maps [p − δ, p + δ] into [p − δ, p + δ].

If x ∈ [p − δ, p + δ], the Mean Value Theorem MVT implies that forsome number ξ between x and p, |g(x)−g(p)| = |g′(ξ)||x −p|. So

|g(x)− p| = |g(x)− g(p)| = |g′(ξ)||x − p| ≤ k |x − p| < |x − p|.

Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 24 / 33

Page 74: Solutions of Equations in One Variable [0.125in]3.375in0 ... · Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 5 / 33. Derivation Example Convergence Final

Derivation Example Convergence Final Remarks

Convergence using Newton’s Method

g′(p) =f (p)f ′′(p)

[f ′(p)]2= 0.

Convergence Theorem (3/4)Since g′ is continuous and 0 < k < 1, part (b) of Exercise 29 inSection 1.1 Ex 29 implies that there exists a δ, with 0 < δ < δ1,and

|g′(x)| ≤ k , for all x ∈ [p − δ, p + δ].

It remains to show that g maps [p − δ, p + δ] into [p − δ, p + δ].If x ∈ [p − δ, p + δ], the Mean Value Theorem MVT implies that forsome number ξ between x and p, |g(x)−g(p)| = |g′(ξ)||x −p|. So

|g(x)− p| = |g(x)− g(p)| = |g′(ξ)||x − p| ≤ k |x − p| < |x − p|.

Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 24 / 33

Page 75: Solutions of Equations in One Variable [0.125in]3.375in0 ... · Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 5 / 33. Derivation Example Convergence Final

Derivation Example Convergence Final Remarks

Convergence using Newton’s Method

Convergence Theorem (4/4)

Since x ∈ [p − δ, p + δ], it follows that |x − p| < δ and that|g(x)− p| < δ. Hence, g maps [p − δ, p + δ] into [p − δ, p + δ].All the hypotheses of the Fixed-Point Theorem Theorem 2.4 are nowsatisfied, so the sequence {pn}∞n=1, defined by

pn = g(pn−1) = pn−1 −f (pn−1)

f ′(pn−1), for n ≥ 1,

converges to p for any p0 ∈ [p − δ, p + δ].

Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 25 / 33

Page 76: Solutions of Equations in One Variable [0.125in]3.375in0 ... · Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 5 / 33. Derivation Example Convergence Final

Derivation Example Convergence Final Remarks

Convergence using Newton’s Method

Convergence Theorem (4/4)Since x ∈ [p − δ, p + δ], it follows that |x − p| < δ and that|g(x)− p| < δ. Hence, g maps [p − δ, p + δ] into [p − δ, p + δ].

All the hypotheses of the Fixed-Point Theorem Theorem 2.4 are nowsatisfied, so the sequence {pn}∞n=1, defined by

pn = g(pn−1) = pn−1 −f (pn−1)

f ′(pn−1), for n ≥ 1,

converges to p for any p0 ∈ [p − δ, p + δ].

Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 25 / 33

Page 77: Solutions of Equations in One Variable [0.125in]3.375in0 ... · Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 5 / 33. Derivation Example Convergence Final

Derivation Example Convergence Final Remarks

Convergence using Newton’s Method

Convergence Theorem (4/4)Since x ∈ [p − δ, p + δ], it follows that |x − p| < δ and that|g(x)− p| < δ. Hence, g maps [p − δ, p + δ] into [p − δ, p + δ].All the hypotheses of the Fixed-Point Theorem Theorem 2.4 are nowsatisfied, so the sequence {pn}∞n=1, defined by

pn = g(pn−1) = pn−1 −f (pn−1)

f ′(pn−1), for n ≥ 1,

converges to p for any p0 ∈ [p − δ, p + δ].

Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 25 / 33

Page 78: Solutions of Equations in One Variable [0.125in]3.375in0 ... · Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 5 / 33. Derivation Example Convergence Final

Derivation Example Convergence Final Remarks

Outline

1 Newton’s Method: Derivation

2 Example using Newton’s Method & Fixed-Point Iteration

3 Convergence using Newton’s Method

4 Final Remarks on Practical Application

Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 26 / 33

Page 79: Solutions of Equations in One Variable [0.125in]3.375in0 ... · Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 5 / 33. Derivation Example Convergence Final

Derivation Example Convergence Final Remarks

Newton’s Method in Practice

Choice of Initial Approximation

The convergence theorem states that, under reasonableassumptions, Newton’s method converges provided a sufficientlyaccurate initial approximation is chosen.It also implies that the constant k that bounds the derivative of g,and, consequently, indicates the speed of convergence of themethod, decreases to 0 as the procedure continues.This result is important for the theory of Newton’s method, but it isseldom applied in practice because it does not tell us how todetermine δ.

Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 27 / 33

Page 80: Solutions of Equations in One Variable [0.125in]3.375in0 ... · Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 5 / 33. Derivation Example Convergence Final

Derivation Example Convergence Final Remarks

Newton’s Method in Practice

Choice of Initial ApproximationThe convergence theorem states that, under reasonableassumptions, Newton’s method converges provided a sufficientlyaccurate initial approximation is chosen.

It also implies that the constant k that bounds the derivative of g,and, consequently, indicates the speed of convergence of themethod, decreases to 0 as the procedure continues.This result is important for the theory of Newton’s method, but it isseldom applied in practice because it does not tell us how todetermine δ.

Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 27 / 33

Page 81: Solutions of Equations in One Variable [0.125in]3.375in0 ... · Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 5 / 33. Derivation Example Convergence Final

Derivation Example Convergence Final Remarks

Newton’s Method in Practice

Choice of Initial ApproximationThe convergence theorem states that, under reasonableassumptions, Newton’s method converges provided a sufficientlyaccurate initial approximation is chosen.It also implies that the constant k that bounds the derivative of g,and, consequently, indicates the speed of convergence of themethod, decreases to 0 as the procedure continues.

This result is important for the theory of Newton’s method, but it isseldom applied in practice because it does not tell us how todetermine δ.

Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 27 / 33

Page 82: Solutions of Equations in One Variable [0.125in]3.375in0 ... · Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 5 / 33. Derivation Example Convergence Final

Derivation Example Convergence Final Remarks

Newton’s Method in Practice

Choice of Initial ApproximationThe convergence theorem states that, under reasonableassumptions, Newton’s method converges provided a sufficientlyaccurate initial approximation is chosen.It also implies that the constant k that bounds the derivative of g,and, consequently, indicates the speed of convergence of themethod, decreases to 0 as the procedure continues.This result is important for the theory of Newton’s method, but it isseldom applied in practice because it does not tell us how todetermine δ.

Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 27 / 33

Page 83: Solutions of Equations in One Variable [0.125in]3.375in0 ... · Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 5 / 33. Derivation Example Convergence Final

Derivation Example Convergence Final Remarks

Newton’s Method in Practice

In a practical application . . .

an initial approximation is selectedand successive approximations are generated by Newton’smethod.These will generally either converge quickly to the root,or it will be clear that convergence is unlikely.

Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 28 / 33

Page 84: Solutions of Equations in One Variable [0.125in]3.375in0 ... · Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 5 / 33. Derivation Example Convergence Final

Derivation Example Convergence Final Remarks

Newton’s Method in Practice

In a practical application . . .

an initial approximation is selected

and successive approximations are generated by Newton’smethod.These will generally either converge quickly to the root,or it will be clear that convergence is unlikely.

Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 28 / 33

Page 85: Solutions of Equations in One Variable [0.125in]3.375in0 ... · Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 5 / 33. Derivation Example Convergence Final

Derivation Example Convergence Final Remarks

Newton’s Method in Practice

In a practical application . . .

an initial approximation is selectedand successive approximations are generated by Newton’smethod.

These will generally either converge quickly to the root,or it will be clear that convergence is unlikely.

Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 28 / 33

Page 86: Solutions of Equations in One Variable [0.125in]3.375in0 ... · Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 5 / 33. Derivation Example Convergence Final

Derivation Example Convergence Final Remarks

Newton’s Method in Practice

In a practical application . . .

an initial approximation is selectedand successive approximations are generated by Newton’smethod.These will generally either converge quickly to the root,

or it will be clear that convergence is unlikely.

Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 28 / 33

Page 87: Solutions of Equations in One Variable [0.125in]3.375in0 ... · Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 5 / 33. Derivation Example Convergence Final

Derivation Example Convergence Final Remarks

Newton’s Method in Practice

In a practical application . . .

an initial approximation is selectedand successive approximations are generated by Newton’smethod.These will generally either converge quickly to the root,or it will be clear that convergence is unlikely.

Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 28 / 33

Page 88: Solutions of Equations in One Variable [0.125in]3.375in0 ... · Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 5 / 33. Derivation Example Convergence Final

Questions?

Page 89: Solutions of Equations in One Variable [0.125in]3.375in0 ... · Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 5 / 33. Derivation Example Convergence Final

Reference Material

Page 90: Solutions of Equations in One Variable [0.125in]3.375in0 ... · Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 5 / 33. Derivation Example Convergence Final

Exercise 29, Section 1.1

Let f ∈ C[a, b], and let p be in the open interval (a, b).

Exercise 29 (a)Suppose f (p) 6= 0. Show that a δ > 0 exists with f (x) 6= 0, for all x in[p − δ, p + δ], with [p − δ, p + δ] a subset of [a, b].

Return to Newton’s Convergence Theorem (1 of 4)

Exercise 29 (b)Suppose f (p) = 0 and k > 0 is given. Show that a δ > 0 exists with|f (x)| ≤ k , for all x in [p − δ, p + δ], with [p − δ, p + δ] a subset of [a, b].

Return to Newton’s Convergence Theorem (3 of 4)

Page 91: Solutions of Equations in One Variable [0.125in]3.375in0 ... · Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 5 / 33. Derivation Example Convergence Final

Mean Value Theorem

If f ∈ C[a, b] and f is differentiable on (a, b), then a number c existssuch that

f ′(c) =f (b)− f (a)

b − a

y

xa bc

Slope f 9(c)

Parallel lines

Slopeb 2 a

f (b) 2 f (a)

y 5 f (x)

Return to Newton’s Convergence Theorem (3 of 4)

Page 92: Solutions of Equations in One Variable [0.125in]3.375in0 ... · Numerical Analysis (Chapter 2) Newton’s Method R L Burden & J D Faires 5 / 33. Derivation Example Convergence Final

Fixed-Point TheoremLet g ∈ C[a, b] be such that g(x) ∈ [a, b], for all x in [a, b]. Suppose, inaddition, that g′ exists on (a, b) and that a constant 0 < k < 1 existswith

|g′(x)| ≤ k , for all x ∈ (a, b).

Then for any number p0 in [a, b], the sequence defined by

pn = g(pn−1), n ≥ 1,

converges to the unique fixed point p in [a, b]. �

Return to Newton’s Convergence Theorem (4 of 4)