lecture 2

E. T. S. I. Caminos, Canales y Puertos 1

Lecture 2

EngineeringComputation


Smearing Occurs when individual terms are larger than summation itself. Consider the exponential series with x = -10

Consider formulas such as:

With 7-decimal-digit accuracy:exact answer = 4.54 10-05

computed answer = – 6.26 10-05

(45 terms) wrong sign !

Largest intermediate terms are:9th = –2,755.732 & 10th = 2,755.732

2 3 4 5x x x x x

e 1 x2! 3! 4! 5!

Round-off Error due to Arithmetic Operations


Error caused by the nature of the numerical technique employed to approximate the solution.

Example:

Maclaurin series expansion of ex

2 3 4 5x x x x x

e 1 x2! 3! 4! 5!

2x x

e 1 x2!

3 4 5x x x

3! 4! 5!

If we use a truncated version of the series:

Then the Truncation Error is:

Truncation Error


Approximations and Rounding Errors

• Precautions:– Sums of large and small numbers: due to equaling the

exponent. They are common in sums of infinite series where the individual terms are very small when compared with the accumulated sum. This error can be reduced by summing first the small terms and using double precision.

– Cancellation of the subtraction: The subtraction of very similar numbers.

– Smearing: The individual terms are larger than the total sum.

– Inner products: They are prone to rounding errors. Thus, it is convenient to use double precision in this type of calculations.

n

1inn2211ii yxyxyxyx


Error PropagationErrors which appear because we are basing current calculations on previous calculations which also incurred some form of error

Stability and Condition NumberNumerically Unstable: Computations which are so sensitive to round-off errors that errors grow uncontrollably during calculations.Condition: sensitivity to such uncertainty; "well conditioned" vs. "ill conditioned"Condition Number: measure of the condition; i.e., extent to which uncertainty in x is amplified by ƒ(x)

C.N. 1 ===> "well-conditioned" C.N. >> 1 ===> "ill-conditioned"


Basic Idea:

Predict the value of a function, ƒ, at a point xi+1 based on the value of the function and all of its derivatives, ƒ, ƒ', ƒ",… at a neighboring point xi

Given xi, ƒ(xi), ƒ'(xi), ƒ"(xi), ... ƒn+1(xi),

we can predict or approximate ƒ(xi+1)

Taylor Series Expansion


General Form:

h = "step size" = xi+1 – xi

Rn = remainder to account for all other terms

= O (hn+1) with x not exactly known "on the order of hn+1 "

Note: f(x) must be a function with n+1 continuous derivatives

2 3 nn

i 1 i i i i i nh h h

f (x ) f (x ) hf (x ) f (x ) f (x ) f (x ) R2! 3! n!

n 1n 1h

f ( )(n 1)!

with xi xi+1



0th order T.S. approx. (n = 0): f(xi+1) = f(xi) + O (h1)

1st order T.S. approx. (n = 1): f(xi+1) = f(xi) + hf '(xi) + O (h2)

2nd order T.S. approx. (n = 2):

nth order T.S. approximation will be exact for an nth order polynomial

2 nn n 1

i 1 i i i ih h

f (x ) f (x ) h f (x ) f (x ) f (x ) (h )2! n!

O

2n 1

i 1 i i ih

f (x ) f (x ) h f (x ) f (x ) (h )2!

O



Zero orderFirst orderSecond order

f(xi )f(xi+1 ) f(xi )

f(xi+1 ) f(xi )+f '(xi )h

f(xi+1 ) f(xi )+f '(xi )h+ )+f "(xi )h2/2!

f(xi+1 )

True

f(x )

xi+1xi

h

x



Objective:Evaluate the derivatives of function, ƒ(xi), without doing it analytically.

When would we want to do this?1. function is too complicated to differentiate analytically:

2. function is not defined by an equation,

i.e., given a set of data points (xi, ƒ(xi)), i=1,…,n

i 0 1 2 3 4

xi 1.0 3.0 5.0 7.0 9.0ƒ(xi) 2.3 4.1 5.5 5.7 5.9

0.5x2 cos(1 x )e

1 0.5x

Numerical Differentiation from Taylor Series Expansion



– First derivative with backward difference.

)xx)(x('f)x(f)x(f 1iiii1i

)xx()x(f)x(f

)x('f1ii

1iii


Backward Difference Approx.:

First Derivative:

first backward difference

2

i 1 i i 1 i ih

f (x ) f (x ) (x x )f '(x ) f "( )2

2

i 1 i ih

f (x ) f (x ) hf '(x ) f "( )2

Letting h = xi - xi-1

2

i i i 1h

hf '(x ) f (x ) f (x ) f "( )2

i i 1f (x ) f (x )f '(x) (h)

h

O



Using data below calculate ƒ'(x1) :

i 0 1 2 3 4xi 1.0 3.0 5.0 7.0 9.0

ƒ(xi) 2.3 4.1 5.5 5.7 5.9

First Backward Finite-Divided-Difference at x1:

1 0f (x ) f (x )f '(x) (h)

h

O

14.1 2.3

f '(x ) (h)2

O

f ' (x1) 0.9 { + O (h) }

Example of 1st Backward FDD


Second Derivative:

2i 2 i

i 2 i i 2 i ix x

f (x ) f (x ) x x f '(x) f "(x )2!

+ O([xi-2– xi]3)

with h = xi– xi-1 and 2h = xi – xi-2

The 2nd order approximation to ƒ(xi-2) becomes:

ƒ(xi-2) = ƒ(xi) – 2hƒ'(xi) + 2h2 ƒ"(xi) +O (h3) [1]

2nd order approximation to ƒ(xi-1):2

3i 1 i i

h(x ) (x ) h '(x) "(x ) (h )

2!f f f f O [2]

Backward Difference Approximation


Subtracting 2*[2] from [1] yields:

f(xi-2) – 2f(xi-1) = –f(xi) + h2f"(xi) + O (h3)

Rearranging:

h2ƒ"(xi) = f(xi) – 2f(xi-1) + f(xi-2) + O (h3)

Second backward difference

3i i 1 i 2

i 2

f (x ) 2f (x ) f (x ) O(h )f "(x )

h

i i 1 i 2i 2

f (x ) 2f (x ) f (x )f "(x ) O(h)

h

Backward Difference Approximation


Using data below calculate ƒ"(x2) :i 0 1 2 3 4

xi 1.0 3.0 5.0 7.0 9.0ƒ(xi) 2.3 4.1 5.5 5.7 5.9

Second Backward Finite-Divided-Difference at x2:

2 1 02 2

f (x ) 2f (x ) f (x )f "(x ) (h)

h

O

2 2

5.5 2*4.1 2.3f "(x ) (h)

2

O

f " (5.0) - 0.1 { + O (h) }

Example of 2nd Backward FDD


What points are used for each form?

Backward:

…, ƒ(xi-2), ƒ(xi-1), ƒ(xi), ƒ(xi+1), ƒ(xi+2), …

Forward:


Centered:


Other Forms of Numerical Differentiation


Taylor Series and Truncation errors

- Higher order divided differences.

2iii1i

2iii1i

h!2

)x(''fh)x('f)x(f)x(f

h!2


- Second finite central divided difference


Forward:

i 1 ii

(x ) (x )f '(x ) (h)

h

O

i 2 i 1 ii 2

(x ) 2 (x ) (x )f "(x ) (h)

h

O

Centered:

2i 1 i 1i

(x ) (x )f '(x ) (h )

2h

O

2i 1 i i 1i 2

(x ) 2 (x ) (x )f "(x ) (h )

h

O

22 1i

- ( ) 4 ( ) -3 ( )f '(x ) = + O(h )

2hi i if x f x f x

Other Forms of Numerical Differentiation



• Use of the Taylor series to calculate derivatives.– First derivative with forward difference.

)xx)(x('f)x(f)x(f i1iii1i

)xx()x(f)x(f

)x('fi1i

i1ii



– First derivative with central differences.

h2)x(f)x(f

)x('f 1i1ii

2iii1i

2iii1i

h!2


h!2


3i

)3(

i1i1i h!3

)x(fh)x('f2)x(f)x(f


Questions:

• Which is a better approximation? Forward, Centered, or Backward?

• Why?

• When would you use which?

Note:

We also can get higher order forward, centered, and backward difference derivative approximations

[C&C Chapter 23, tabulated in Figs. 23.1-3]



Determine h to minimize the total error of a forward finite-divided difference approximation for:

i 1 if (x ) f (x )f '(x)

h

xi xi+1

• Round-off Error:

• Truncation Error:

i 1 if (x ) f (x ) hf '(x) f "( )

h 2

ˆx x x f f f with = machine epsilon.

. .i i(x h) (1 ) (x ) (1 ) hˆ "( )

h 2

f ' =

i i i(x h) (x ) 2 (x )RoundoffError h h

As a result:

Example Combining Roundoff and Truncation Error


Total error = truncation error + roundoff error

E = | Total Error | h

f "( )2

NOTE: Truncation error decreases as h decreases Round-off error increases as h decreases

i2 f (x )

h

+



To minimize total error E with respect to h, set the first derivative to zero:

i2

f "( ) 2 f (x )dE0

dh 2 h

Solve for h and approximate f "() as f

"(xi):

i

i

4 f (x )h

f "(x )



Using the first forward-divided-difference approximation with

error O(h) and a 5-decimal-digit machine:

e = b1-t = 101-5 = 10-4 = 0.0001

f '(x) = ; f "(x) = 0

i

i

4 f (x )h infinity

f "(x )

Linear Application:

Determine h that will minimize total error for calculating f’(x) for

f(x) = x at x = 1



f(x+h)= {exact: 3.1415} h (x+h) f(x+h)-f(x) [f(x+h)-f(x)]/h0 3.1415

0.000001 3.1415 0 00.00001 3.1416 0.0001 100.0001 3.1419 0.0004 4.00.001 3.1447 0.0032 3.20.01 3.1730 0.0315 3.150.1 3. 4557 0.3142 3.1421 6.2831 3.1416 3.146

Underlined digits are subject to round-off error. They are likely to be in error by ± one or two units. This does not cause much problem when h = 1, but causes large errors in the final result when h < 10-4.



Nonlinear Application:

Determine h for minimizing the total error for computing f’(x) for

ƒ(x) = ex at x = 3

Using the first forward-divided-difference approximation with

error O(h) and a 5-decimal-digit machine:

e = b1-t = 101-5 = 10-4 = 0.0001

f(x) = f '(x) = f "(x) = ex = 20.0855;

i

i

4 (x )h 0.02

"(x )

or about 0.01



Underlined digits subject to roundoff error.Bold digits in error due to truncation.

full precisionh f(x+h)=ex+h f(x+h)-f(x) [f(x+h)-f(x)] [f(x+h)-f(x)]

h h0 20.085 {exact = 20.085}

0.00001 20.085 0.0 0 20.0860.0001 20.087 0.002 20 20.0860.001 20.105 0.020 20 20.0960.01 20.287 0.202 20.2 20.180.1 22.198 2.113 21.13 21.121 54.598 34.513 34.523 34.512

Roundoff Truncation


lecture 2

Documents

fxi f xi hfxi

fxi f xi h f xi h22

canales y puertosround

fxi hf

neighboring point xigiven

exponential series

individual terms

small terms