Numerical Analysis, lecture 1: Introduction
• Course info
• What is numerical analysis? (textbook §1.1-2)
function example2[t,Y] = ode45(@rocket,[0 40],[0 0]);plot(t,Y(:,1),'linewidth',3)xlabel('t'), ylabel('h','rot',0), box off function dy = rocket(t,y)v = y(2); m = max(180-10*t,0);M = 120+m;dy = [v ((5000+10*v)*(t<=18)-0.1*v*abs(v))/M - 9.81];
0 5 10 15 20 25 30 35 400
500
1000
1500
2000
2500
3000
t
h
Numerical Analysis, lecture 1, slide ! 2
Operational info for course MAT-31102/31107
Lecturer
Lectures (in Finnish)
Textbook (read it!)
Tutorials
Bring your laptop with Matlab or Octave to the tutorials
Lars Eldén, Linde Wittmeyer-Koch, Hans Bruun Nielsen: Introduction to Numerical Computation, Analysis and Matlab Illustrations, Studentlitteratur, 2004.
43!
prof. Robert Pichéautomation science and [email protected]
http://www.tut.fi/~piche/numa
see POP (schedule) & home page (exercises & model quiz questions & solutions)
see POP (schedule) & home page (slides)
Numerical Analysis, lecture 1, slide !
Home page
3
http://www.tut.fi/~piche/numa
MAT-31102 / 31107 Numeerinen analyysi / Numerical analysis
Perehdytään tieteellisen ja teknillisen laskennan perusmenetelmien teoriaaan ja käyttöön. Theory and practical application of essential numerical methods for scientific and engineering problem solving.
Opettajat
luennoitsija/lecturer Robert Piché, assari/teaching assistant Jari Niemi
Sisältö / Contents
• Johdanto, Virheanalyysi / Intro, Error analysis [slides, harjoitus/exercises, solutions]!• Tietokonearitmetiikka / Computer Arithmetic [slides, harjoitus/exercises, solutions]!• Epälineaarisen yhtälön ratkaiseminen / Solving nonlinear equations [slides,!harjoitus/exercises, solutions bisection illinois newtonraphson secant ]• Funktioiden interpolointi! / Function interpolation [slides, harjoitus/exercises, solutions, neville, osculate]• Numeerinen integrointi / Numerical integration [slides,!harjoitus/exercises, solutions]• Funktioiden approksimointi / Function approximation [slides, harjoitus/exercises, solutions, lspolyfit]• Tavallisten differentiaaliyhtälöiden alkuarvotehtävät / ODE initial value problems [slides, harjoitus/exercises, solutions]
Oppikirja / Textbook
Lars Eldén, Linde Wittmeyer-Koch, Hans Bruun Nielsen: Introduction to Numerical Computation, Analysis and Matlab Illustrations, Studentlitteratur, 2004. [errata]
Pikakokeet / Quizzes
Harjoitustunnilla annetaan pikakokeet, josta saa bonuspisteitä. The weekly tutorial sessions include quizzes that give you bonus points.
Tentti / Exam
Tavallinen tai graafinen/ohjelmoitava laskin ja yksi A4 kaksipuolinen sivu käsinkirjoitetut muistiinpanot sallittu. MallikysymyksiäOrdinary or graphing/programmable calculator and one A4 two-sided page of handwritten notes allowed. Model exam
Numerical Analysis, lecture 1, slide !
Exercise sets & PC labs
4
theory problems
problems from the textbook
Matlab/Octave questions (bring your laptop!)
modelling problems
answers! (complete solutions are posted after one week)
Numerical Analysis, Exercises for lectures 1–2 [§1.1–2, 2.1–3]1 a. Determine the absolute error and the relative error of 106/39 as an approximation of
the natural logarithm base e.b. How many correct decimals does x̄ = 0.9951 have as an approximation of x =
0.9949? How many significant digits?c. Some books say that x̄ has t correct decimals if x̄ rounded to t decimals is equal
to x rounded to t decimals. By this definition, how many correct decimals doesx̄ = 0.9951 have as an approximation of x = 0.9949?
2 How accurately do we need to know π in order to compute√
π with 4 correct decimals?[textbook page 37 exercise E1]
3 Derive the error propagation formula ∆ff � α1
∆x1x1
+ α2∆x2x2
+ α3∆x3x3
for the functionf(x1, x2, x3) = xα1
1 xα22 xα3
3 . [page 38 exercise E3]
4 Reformulate or approximate the following Matlab/Octave expressions to avoid cancellationfor x � 0, and compute their values for the specified x value. [page 38 exercise E5]
a. exp(x)-exp(-x), x = 10−5
b. 1-cos(x), x = 10−5
c. 1/(sqrt(1+x^2)-sqrt(1-x^2)), x = 10−2
5 For any x0 > −1, the sequence recursively defined by the formula
xn+1 = 2n+1��
1 + 2−nxn − 1�
converges to ln(1 + x0 + 14x2
0). However, computing the sequence for x0 = 4 in Mat-lab/Octave gives completely incorrect results. Try it:
x=4; for n=1:100, x=2^(n+1)*(sqrt(1+x/2^n)-1); end; x
Explain the problem, and fix it.
6 A gas exerts a pressure of 0.892 atm (all decimals correct) in a 5.00 L container (all decimalscorrect) at 15± 1 degrees Celsius. How much gas is there? (Hint: look up the ideal gaslaw.)
7 The following code computes the zeros of the quadratic polynomial ax2 + bx + c using thewell-known formula:
d = sqrt(b^2-4*a*c);r1 = -(b-d)/(2*a);r2 = -(b+d)/(2*a);
Try it with the polynomial x2 − 109x + 1, which has 2 positive real zeros. Which zerois not computed accurately, and why? Try again, but with the last two lines replacedby:
r1 = -(b+sign(b)*d)/(2*a);r2 = c/(a*r1);
Answers
1. (a) ∆x � −3.3 · 10−4, ∆xx � −1.2 · 10−4. (b) 3 correct decimals, 3 significant digits.
(c) 1 and 3 but not 2
2. |π̄ − π| ≤ 1.8 · 10−4
4. (a) 2.00000000003333e-05 (b) 4.99999999995833e-11 (c) 9.99999998750000e+03
5. use xn+1 = 2xn/(√
1 + 2−nxn + 1)
6. 0.1886 ± .001 mol
Numerical Analysis, lecture 1, slide !
Miniquizzes give bonus points
5
quiz during exercise period
Numerical Analysis 1 — Quiz questions for lectures 7-8
Find the elements of the first column of the following Romberg table for� 1
0 e−x2dx.
Answer:h T1(h) T2(h) T3(h)
1 0.683940.5 0.73137 · · ·0.25 0.74298 · · · · · ·
Solution: We can determine the elements of the first column of the Romberg tablewith the trapezoidal rule for a corresponding step length. With the step length h = 1we need values of function f(x) = e−x2
at f(0) = f0 = 1.0000 and f(1) = f1 = 0.3679.Now
T1(1) = h
�f0
2+
f1
2
�= 1 ·
�1.0000
2+
0.3679
2
�= 0.68394
The recursive formula T (h) = 12T (2h)+h(f1 + f3 + f5 · · ·+ fm−1) presented on page 172
gives us the other elements.
T1(0.5) = 0.5 · T1(1) + h(f(0.5)) = 0.5 · 0.68395 + 0.5 · 0.77880 = 0.73137
T1(0.25) = 0.5 · T1(0.5) + h(f(0.25) + f(0.75))
= 0.5 · 0.73137 + 0.25 · (0.93941 + 0.56978) = 0.74298
Compute the integral
� 0.1
0
cos x
x13
dx using the approximation cosx ≈ 1− x2
2
Answer:
� 0.1
0
cos x
x13
dx ≈ 0.3228
Solution:
� 0.1
0
cos x
x13
dx ≈� 0.1
0
1− x2
2
x13
dx =
� 0.1
0
�x−
13 − 1
2x
53
�dx =
� 0.1
0
3
2x
23 − 3
16x
83 ≈ 0.3228
Using Simpson’s rule with the step length h = 14 find an approximation for
� 1
0 e−x2dx.
Answer:� 1
0 e−x2dx ≈ 0.7468
Solution:
S(h) = S
�1
4
�=
h
3(f(0) + 4f(0.25) + 2f(0.5) + 4f(0.75) + f(1))
=1
12(1.0000 + 4 · 0.9394 + 2 · 0.7788 + 4 · 0.5698 + 0.3679) = 0.7468
1
Numerical Analysis 1 — Quiz questions for lectures 7-8
Find the elements of the first column of the following Romberg table for� 1
0 e−x2dx.
Answer:h T1(h) T2(h) T3(h)
1 0.683940.5 0.73137 · · ·0.25 0.74298 · · · · · ·
Solution: We can determine the elements of the first column of the Romberg tablewith the trapezoidal rule for a corresponding step length. With the step length h = 1we need values of function f(x) = e−x2
at f(0) = f0 = 1.0000 and f(1) = f1 = 0.3679.Now
T1(1) = h
�f0
2+
f1
2
�= 1 ·
�1.0000
2+
0.3679
2
�= 0.68394
The recursive formula T (h) = 12T (2h)+h(f1 + f3 + f5 · · ·+ fm−1) presented on page 172
gives us the other elements.
T1(0.5) = 0.5 · T1(1) + h(f(0.5)) = 0.5 · 0.68395 + 0.5 · 0.77880 = 0.73137
T1(0.25) = 0.5 · T1(0.5) + h(f(0.25) + f(0.75))
= 0.5 · 0.73137 + 0.25 · (0.93941 + 0.56978) = 0.74298
Compute the integral
� 0.1
0
cos x
x13
dx using the approximation cosx ≈ 1− x2
2
Answer:
� 0.1
0
cos x
x13
dx ≈ 0.3228
Solution:
� 0.1
0
cos x
x13
dx ≈� 0.1
0
1− x2
2
x13
dx =
� 0.1
0
�x−
13 − 1
2x
53
�dx =
� 0.1
0
3
2x
23 − 3
16x
83 ≈ 0.3228
Using Simpson’s rule with the step length h = 14 find an approximation for
� 1
0 e−x2dx.
Answer:� 1
0 e−x2dx ≈ 0.7468
Solution:
S(h) = S
�1
4
�=
h
3(f(0) + 4f(0.25) + 2f(0.5) + 4f(0.75) + f(1))
=1
12(1.0000 + 4 · 0.9394 + 2 · 0.7788 + 4 · 0.5698 + 0.3679) = 0.7468
1
• study the model questions beforehand!• closed-book quiz, one question per week• grading: zero or one• total bonus 0-6 points (exam is 30 points)
Numerical Analysis, lecture 1, slide !
What previous students say about this course
6
Luennoitsijalle palautetta: vaikka opiskelijat ovatkin huonoja nauramaan luennoilla, älä anna tämän hämätä sinua: heitit luennoilla hyvää läppää, jatka samaan malliin! Tämän takia ajattelin hiukan positiivisemmin "aikaista" kouluunlähtöä.
Harjoitukset tukivat hyvin luentoja ja asiat aukesi oikeastaan harkkojen myötä vasta kunnolla. Hyödyllinen kurssi, olisi kannattanut jo aikaisemmin käydä.
Opetus oli hyvää mutta aihe niin kuivaa että kurkku on vieläkin kipeänä
Ihan hyvä kokonaisuus. Enemmän laskuesimerkkejä luennoilla, jos mahdollista.
Tässä kurssissa oli mielestäni järjestelyt ja muut systeemit kohdallaan.
Palautettahan ei pitäisi antaa, enne kun näkee arvosanansa, että voi sitten sopivalla kitkeryydellä asennoitua tämän palautteen kirjottamiseen.
do y! have any questions?
Numerical Analysis, lecture 1, slide ! 7
Numerical simulations are important in engineering & natural sciences…
Numerical Analysis, lecture 1, slide ! 8
“The objective of numerical analysis is to construct and analyse methods to solve practical computational problems”
• computational problems
- 2 small examples
• NA topics
• issues
• history
Numerical Analysis, lecture 1, slide ! 9
a computational problem is solved in 4 stages
• collect data and formulate a mathematical model
get(x) x > 0? disp(x)
x = -(x+1)^2
• find an algorithm (method)
• compute
5
1
25
10= 20
0
efficiencyratio
00
• look at results
Numerical Analysis, lecture 1, slide ! 10
A small example: how high does a thrown ball go? (page 2)
Model & data
Algorithm
Computation & Result>> v0 = 25; >> g = 9.81; >> h = (v0^2)/2/g
h =
31.8552
12mv0
2 = mgh, g = 9.81m ! s"2 , v0 = 25m ! s"1
h = v02
2g
Numerical Analysis, lecture 1, slide ! 11
Another example:how high does a rocket go? (p. 2)
Model & data
Algorithm
Computation & Resultfunction example2[t,Y] = ode45(@rocket,[0 40],[0 0]);plot(t,Y(:,1)) function dy = rocket(t,y)v = y(2); m = (180-10*t)*(t<=18);M = 120+m;Mp =-10*(t<=18);T = 5000*(t<=18);d = 0.1*v*abs(v);dy = [ v (T-9.81*M-d-Mp*v)/M ];
Use an adaptive Runge-Kutta method to solve the ODE IVP
!v =T " Mg " d " !M v
M, !h = v, v(0) = 0, h(0) = 0
0 5 10 15 20 25 30 35 400
500
1000
1500
2000
2500
3000
t
h
fuel mass m = 180 !10t, thrust T = 5000 (0 " t " 18), rocket mass M = 120 + m, drag d = 0.1v v , Newton's law (Mv #) = T ! Mg ! d
Numerical Analysis, lecture 1, slide ! 12
This course will introduce most of the main topics of numerical analysis
• understand floating point computation• solve nonlinear equations• approximate functions• compute integrals• solve ordinary differential equations
NA topics are also covered in other courses: • matrix analysis MAT-31090/31096
• optimization MAT-41120
• intro tech. comp. with Matlab MAT-45700/45706
Numerical Analysis, lecture 1, slide ! 13
A numerical analyst studiesthe mathematical properties of algorithms (p.6)
• how many operations does the algorithm need?
• how fast does the iteration converge?
• how accurate is the approximation?
Solving Ax = b with Gaussian elimination requires 13 n
3 +O(n2 ) multiplications.
!f (x) =f (x + h) " f (x " h)
2h+O(h2 )
(quadratic convergence)In Newton-Raphson method, limn!"
xn+1 # x$
xn # x$ 2 = K
Numerical Analysis, lecture 1, slide !
We will frequently use Taylor series
14
f (x) = 1n!f (n)(c)(x ! c)n
n=0
N
"SN
! "### $###+
1(N +1)!
f (N+1)(#)(x ! c)N+1
RN! "##### $#####
Taylor polynomial and remainder
truncation error
standard functions
ex = 1+ x + 12!x2 +
13!x3 +!
sin x = x ! 13!x3 +
15!x5 !!
cos x = 1! 12!x2 +
14!x4 !!
ln(1+ x) = x ! 12x2 +
13x3 !!
(1+ x)p = 1+ px + p(p !1)2!
x2 +p(p !1)(p ! 2)
3!x3 +!
Numerical Analysis, lecture 1, slide ! 15
what happened, what’s next
Computational problems are solved in 4 stages: build a mathematical model, devise an algorithm, compute, and look at results.
Numerical analysis is concerned mainly with stage 2, i.e. inventing and analysing numerical algorithms.
Core topics: solving equations, approximating functions, computing integrals, and solving differential equations.
Typical questions: how accurate is it? how fast does it converge? how many operations does it require?
Next lecture: error analysis (§2.1-3)
Numerical Analysis, lecture 2: Error Analysis(textbook sections 2.1–3)
• sources of error
• error characterizations
• sensitivity & cancellation
• the right way to solve
!f (x )"x !f
!x
ax2 + bx + c = 0
Numerical Analysis, lecture 2, slide ! 2
There are many kinds of “error” (p. 9-10)
!
"
###
$
###
this course
human error bugs and blunders
model error simplifications and idealizations
data error measurements of physical quantities
truncation error approximations of infinite sequences, integrals, limits, etc.
rounding error rounding of input data & in arithmetic operations
Numerical Analysis, lecture 2, slide ! 3
There are many ways to describe the errorof an approximation of a real number (p. 11, 13)
absolute & relativea : exact, a : approximateabsolute error of a : a ! a (=:"a)relative error of a :"a a (provided a # 0)
correct decimalsa has t correct decimals if !a " 0.5 #10$t
bounding interval a ![a " #,a + #] is written as a = a ± #
for example:
a = 200 = 14.14213562!
a = 14.14 = 1.414 !101
"a = # 0.00213562!
"aa
= #0.0001510!
"a $ 0.005
a = 14.14 ± 0.005
a has 2 correct decimals
a has 4 significant digits
significant digits
a = d0.d1d2!!10E (d0!0)
has s significant digits if "a # 0.5 $101+E%s
Numerical Analysis, lecture 2, slide ! 4
These examples illustrate the various ways of describing error
a = ! = 3.14159265!
a =355113
"a =
"aa
=
"a # 0.5 $10
a = 355113
± 0.5 $10
a has correct decimals
a has significant digits
a = 0.001234 ± 0.5 !10"5
a has 5 correct decimals and 3 significant digits
a = 56.789 ± 0.5 !10"3
a has 3 correct decimals and 5 significant digits
a = 210000 ± 5000a has 2 significant digits
a = 1.789 ± 0.005
a has correct decimals and significant digits
p. 13:
2.667.10-7
8.49 .10-8
-6
-6
6
7 2 3
Numerical Analysis, lecture 2, slide ! 5
Rounding increases approximation error (p. 11-12)
a Matlab function to round to t decimals>> Round = @(x,t) round(x*10^t)/10^t;>> Round( [1.23767, 0.774501, 6.3225, 6.3235], 3)
ans =
1.238 0.775 6.323 6.324
“round to nearest even” would give 6.322 !
rounding error boundsif a = (a rounded to t decimals) then a = a ± 0.5 !10"t
if b = b ± # and b = (b rounded to t decimals) then b = b ± (# + 0.5 !10"t )
example (p. 12)
rounding b = 11.2376 ± 0.1 to 1 decimal gives b = 11.2 ± 0.15
proof: b ! b = b ! b + b ! b " b ! b + b ! b
Numerical Analysis, lecture 2, slide ! 6
In addition, absolute errors accumulate;in multiplication, relative errors (p. 13-16)
Addition!(x1 + x2 ) " !x1 + !x2
!(x1 " x2 ) # !x1 + !x2
Subtraction
Examplex1 = 123.4 ± 0.05 = 123.4 ! (1± 0.0004052)x2 = 122.1± 0.05 = 122.1 ! (1± 0.0004095)
x1 " x2 = 1.3 ± 0.1
x1x2
= 1.010647 ! (1± 0.0008147)
= 1.010647 ± 0.00082Division
!(x1 x2 )x1 x2
<!!x1x1
+!x2x2
Multiplication!(x1x2 )x1x2
"!x1x1
+!x2x2
+!x1x1
#!x2x2
$ !x1x1
+!x2x2
!(x1 + x2 ) = (x1 + x2 ) " (x1 + x2 )
= (x1 " x1) + (x2 " x2 )
# x1 " x1 + x2 " x2
proof
Numerical Analysis, lecture 2, slide ! 7
Which solar collector is most efficient? (p.14, 17)
! = K QTdIEfficiency:
Collector 1! = 0.76, "Q
Q= 1.5%, "Td
Td= 1%, "I
I= 3.6%
"!!
# "!!
+"TdTd
+"II
= 6.1%
! = 0.76 % (1± 0.061) = 0.76 ± 0.046 & 0.714 ' ! ' 0.806
Collector 2! = 0.70, "Q
Q= 0.5%, "Td
Td= 1%, "I
I= 2%
"!!
# "!!
+"TdTd
+"II
= 3.5%
! = 0.70 % (1± 0.035) = 0.70 ± 0.025 & 0.675 ' ! ' 0.725
Numerical Analysis, lecture 2, slide ! 8
!f (x )"x !f
!x
We can approximate the error that ispropagated by a univariate function… (p. 14-15)
Formula:
!f = f (x ) " f (x) = #f ($)!x for some $ %[x, x ]Proof using Mean Value Theorem:
f (x) = f (x ! "x) = f (x ) ! #f (x )"x + 12 ##f ($)("x)2
Proof using Taylor’s formula:
a = 2.05 ± 0.01, f (a) = a = ?Example (p.15)
!f " #f (x )!x
!f (a) = 12 a
, "f <!0.01
2 2.05# 0.0035
$ a = 2.05 ± 0.0035, i.e. a %[1.42828,1.435329]
verify: a ! 2.04, 2.06"# $% = [1.4282856857,1.4352700094]
Numerical Analysis, lecture 2, slide ! 9
…and by a multivariate function (p. 17-18)
!f " #f
#x1(x ) $ !x1 +
#f#x2
(x ) $ !x2 +!
Example (p.18)
x1 = 0.75 ± 0.01, x2 = 0.413 ± 0.003, y = sin(x12x2 ) = ?
y = sin(x12x2 ) = 0.230229
!y <! 2x1x2 cos(x12x2 ) " !x1 + x12 cos(x1
2x2 ) " !x2 # 0.0077
$ y = 0.23 ± 0.008
∆ f (x1,x2, . . . ,xn) = ?
Numerical Analysis, lecture 2, slide ! 10
Cancellation is the loss of significant digitsdue to subtraction of nearly equal values (p. 19-20)
Example (slide 7)
x1 = 123.4 ± 0.05, x2 = 122.1± 0.05 (4 significant digits)x1 ! x2 = 1.3 ± 0.1 (1 significant digit)
loga ! logb = log ab
To avoid cancellation, reformulate using identities
1+ x ! 1! x = 1+ x ! 1! x( ) " 1+ x + 1! x1+ x + 1! x
=2x
1+ x + 1! x
or approximate using a Taylor polynomial1− cosx
x≈ x
2− x3
24
Numerical Analysis, lecture 2, slide ! 11
There is a wrong way and a right way to compute quadratic roots
Example (p.19)
(3 significant digits)
(5 significant digits)
Solve x2 !18x +1 = 0
x1 = 9 + 80 " 9 + 8.9443 = 17.9443
x2 = 9 ! 80 " 9 ! 8.9443 = 0.0557
x2 =1
9 + 80" 1
17.9443 " 0.0557280
Vieta's formula is obtained by substituting x = 0 into ax2 + bx + c = a(x ! x1)(x ! x2 )
The roots of ax2 + bx + c = 0 are x1,2 =
!b ± b2 ! 4ac2a
" cancellation when 4ac ! b2
A better formula is: x1 =! b + sign(b) " b2 ! 4ac( )
2a, x2 =
cax1
Numerical Analysis, lecture 2, slide ! 12
what happened, what’s next
• Sensitivity analysis studies how error intervals propagate in calculations.
• Rounding adds an absolute error.
• Absolute error accumulates in addition & subtraction, relative error in multiplication & division
• Taylor formula for error propagation
• Cancellation can sometimes be avoided by changing the formula.
Next lecture: computer arithmetic (§2.4-8)