1

Chapter 1 Chapter 1 Introductory Introductory

Concepts and Concepts and Calculus ReviewCalculus Review

2

IntroductionIntroduction

The subjectsThe subjects The derivation of the algorithmsThe derivation of the algorithms The implementation of the algorithmsThe implementation of the algorithms Analyze the algorithms mathematicallyAnalyze the algorithms mathematically

Accuracy, efficiency, and stabilityAccuracy, efficiency, and stability

3

1.1 Basic Tools of Calculus1.1 Basic Tools of Calculus

1.1.1 Taylor’s Theorem1.1.1 Taylor’s Theorem

Integral mean value

theorem

4

Three particular expansions of Three particular expansions of Taylor’s TheoremTaylor’s Theorem

...!3

)0(

!2

)0(

!1

)0(

!0

)0( 03

02

01

00

ex

ex

ex

ex

ex

where x0= ?

5

Three particular expansions of Three particular expansions of Taylor’s TheoremTaylor’s Theorem

where x0= 0

where x0= 0

6

Example : eExample : exx

]1,1[x

Finally, n can be found! (here n = 9)

If we wantthen

Let x0= 0

7

Example : eExample : exx

p2 (x)

p9 (x)

exp(x)

8

Example : eExample : exx

9

Example : eExample : exx

The result tells usThe result tells us We can approximate the exponential function We can approximate the exponential function

to within to within 1010-6-6 accuracy using a specific accuracy using a specific polynomial, and this accuracy holds for all polynomial, and this accuracy holds for all xx in in a specified interval.a specified interval.

10

Example 1.1Example 1.1 Let Let f f ((xx) = () = (xx+1)+1)1/21/2,, then the second-order Taylor then the second-order Taylor

polynomial (computed about polynomial (computed about xx00= 0= 0) is computed ) is computed as follows:as follows:

22

11

Example 1.2: sinExample 1.2: sin Function:Function: Accuracy:Accuracy:

12

Example 1.3: arctanExample 1.3: arctan Function:Function:

http://zh.wikipedia.org/wiki/File:Atan_acot_plot.svg

13

Example 1.3: arctanExample 1.3: arctan Function:Function:

Error term

Let

and

14

Example 1.3 : arctanExample 1.3 : arctan Please determine the error in a Please determine the error in a ninthninth-degree -degree

Taylor approximation to the arctangent function.Taylor approximation to the arctangent function. Since Since 22n n +1 = 9+1 = 9 implies that implies that n n = 4= 4, we have, we have

15

Taylor’s Theorem ExpansionTaylor’s Theorem Expansion

Let x x + h and x0 x

16

1.1.2 Mean Value and Extreme Value 1.1.2 Mean Value and Extreme Value TheoremsTheorems

http://en.wikipedia.org/wiki/Mean_value_theorem

17

1.1.2 Mean Value and Extreme Value 1.1.2 Mean Value and Extreme Value TheoremsTheorems

W

18

1.1.2 Mean Value and Extreme Value 1.1.2 Mean Value and Extreme Value TheoremsTheorems

M m

Critical point

Critical point

19

1.1.2 Mean Value and Extreme Value 1.1.2 Mean Value and Extreme Value TheoremsTheorems

The integral mean value theorem (a corollary of the intermediate value theorem) states that a function continuous on an interval takes on its average value somewhere in the interval. More exactly, if  is continuous on       , then there exists  in        such that                         .

20

1.1.2 Mean Value and Extreme Value 1.1.2 Mean Value and Extreme Value TheoremsTheorems

21

1.2 Error, Approximate Equality, and 1.2 Error, Approximate Equality, and Asymptotic Order NotationAsymptotic Order Notation

1.2.1 Error1.2.1 Error AA : a quantity( : a quantity( 數量數量 ) we want to compute) we want to compute AAhh: an approximation(: an approximation( 近似值近似值 ) to that quantity) to that quantity

Relative error (Relative error ( 相對誤差相對誤差 ) is better.) is better. These errors are both computational errors.These errors are both computational errors.

22

1.2.2 Notation: Approximate 1.2.2 Notation: Approximate EqualityEquality

Approximate equalityApproximate equality

It is an equivalence relation, and satisfy the folIt is an equivalence relation, and satisfy the following properties:lowing properties: Transitive(Transitive( 遞移性遞移性 ):): Symmetric(Symmetric( 對稱性對稱性 ):): Reflexive(Reflexive( 反身性反身性 ):):

23

1.2.3 Notation: Asymptotic Order 1.2.3 Notation: Asymptotic Order (Big O)(Big O)

24

Example 1.4Example 1.4

LetLet

Simple calculus shows that Simple calculus shows that

so that we haveso that we have HereHere

25

1.2.3 Notation: 1.2.3 Notation: Asymptotic Order (Big O) Asymptotic Order (Big O)

26

Example 1.6Example 1.6

27

28

1.3 A Primer on Computer Arithmetic1.3 A Primer on Computer Arithmetic

Computer arithmetic is generally Computer arithmetic is generally inexact.inexact. While the errors are very small, they can While the errors are very small, they can

accumulate and dominate the calculation.accumulate and dominate the calculation. Example: floating-point arithmeticExample: floating-point arithmetic

Reference: An Introduction to Computer Science, Reference: An Introduction to Computer Science, Chapter 3, Excess System (Excess_127 or Chapter 3, Excess System (Excess_127 or Excess_1023)Excess_1023)

is the sign of the number, f is the fraction (0 <= f <= 1),is the base of the internal number system

IEEE standards for floating-point representation

（底數 尾數）（底數 尾數）

Example Example

Show the representation of the normalized number + 26 x 1.01000111001

SolutionSolution

The sign isThe sign is positivepositive. The Excess_127 representation of . The Excess_127 representation of the exponent is the exponent is 133133. You add extra 0s on the . You add extra 0s on the right right to to make it 23 bits. The number in memory is stored as:make it 23 bits. The number in memory is stored as:

00 1000010110000101 0100011100101000111001000000000000000000000000

31

ErrorsErrors

Rounding error v.s. chopping errorRounding error v.s. chopping errorRounding: Rounding: 四捨五入四捨五入Chopping: Chopping: 無條件捨去無條件捨去Discussion:Discussion:

Rounding is more accurate but chopping is fastRounding is more accurate but chopping is faster.er.

The chopping error is indeed lager than the rouThe chopping error is indeed lager than the rounding error.nding error.

32

ExampleExample

Rounding errorRounding error

Chopping errorChopping error

33

Subtractive CancellationSubtractive Cancellation

If If aa and and bb are accurate to 16 decimal digits. are accurate to 16 decimal digits. What about their difference What about their difference cc = = aa - - bb ? ?

Example: Example:

The result The result cc is accurate to 12 digits. is accurate to 12 digits. This is because we were subtracting two This is because we were subtracting two

nearly equalnearly equal numbers. numbers.

0050009999990000.02)1000/1( eb

9983339999000049.02)100/1( ea

34

ExampleExample

Function :Function :

We know that :We know that :

Taylor’s Theorem : Taylor’s Theorem :

35

…………

36

1.5 Simple Approximations1.5 Simple Approximations

Error function: Error function:

(probability theory)(probability theory) It isIt is notnot possible to evaluate this integral by possible to evaluate this integral by

means of the fundamental theorem of calculus.means of the fundamental theorem of calculus. Use Taylor’s Theorem to approximate.Use Taylor’s Theorem to approximate.

wherewhere

37

Substitution: Substitution:

DefineDefine

So that we haveSo that we have

SetSet

where where cc depends on depends on tt andand

38

Apply the Integral Mean Value Theorem:Apply the Integral Mean Value Theorem:

The structured form:The structured form:

wherewhere

39

Use the big Use the big OO notation: notation:

Use the approximate equality notation:Use the approximate equality notation:

Simplify: Simplify: if the values of if the values of xx between between 00 and and 22

if if k k >=1>=1

thusthus

40

Fundamental IdeaFundamental Idea

When confronted with a computation that When confronted with a computation that cannot be done exactly, we often replace cannot be done exactly, we often replace that relevant function with something that relevant function with something simpler that approximates it, and carry out simpler that approximates it, and carry out the computation exactly on the simple the computation exactly on the simple approximation.approximation.