math20602 numerical analysis 1 - school of …mlotz/teaching/math20602-old/... · 2014-01-27 ·...

M\crManchester Numerical Analysis

MATH20602Numerical Analysis 1

Martin Lotz

School of MathematicsThe University of Manchester

Manchester, January 27, 2014

Outline

General Course Information

Introduction to Numerical Analysis

Prerequisites from Calculus

Outline

Organisation

The course website can be found under

http://www.maths.manchester.ac.uk/~mlotz/teaching/math20602/

I Tutorials start in second week (on February 4).

I Problem sets in week n are discussed in week n+ 1.

I All the material presented in the lecture will be made availableonline as the course progresses.

I Lecture notes will appear on the website towards the end ofeach week.

I Contact: martin.lotz@manchester.ac.uk

1 / 24

Organisation

1 / 24

Organisation

1 / 24

Organisation

1 / 24

Organisation

1 / 24

Organisation

1 / 24

Example classes

Example classes are there to deepen the understanding of thecourse material.

I Problems sheets should ideally be looked at before theexample class.

I In the example classes,I you will be given some time to work on the problems,I you should ask questions if some parts are not clear,I you will get feedback on your attempts at the problems,I we will go through a selection of problems and their solutions.

2 / 24

Example classes

2 / 24

Example classes

2 / 24

Important dates

Reserve these dates:

I Wed, March 12: Revision class for midterm test

I Mon, March 17: Midterm (coursework) test (1 hour)

I Wed, May 14: Revision class for final exam

I Date of final exam will be announced when available!

3 / 24

MATLAB

I Matlab, or an equivalent system, is an indispensable tool forthis course.

I Availability:I On campus computersI Student version available at low cost

I The course page contains links to Matlab documentation

I Alternatives: Python, Octave, Mathematica, Sage

I An easy to parse introduction to Matlab is available here:

http://www.guettel.com/teaching/matlab/

4 / 24

MATLAB

4 / 24

MATLAB

4 / 24

MATLAB

4 / 24

MATLAB

4 / 24

MATLAB

4 / 24

Course Structure

The lecture consists of the following parts

1 Introduction and foundational material (Lectures 1-2)

2 Interpolation (Lectures 3-6)

3 Numerical Integration/Quadrature (Lectures 7-9)

4 Numerical Linear Algebra (Lectures 10-15)

5 Nonlinear Equations (Lectures 16-19)

5 / 24

Course Structure

5 / 24

Course Structure

5 / 24

Course Structure

5 / 24

Course Structure

5 / 24

Course Structure

5 / 24

Learning Outcomes

After the lecture you should have

I practical knowledge of a range of iterative techniques forsolving linear and nonlinear systems of equations, theoreticalknowledge of their convergence properties;

I an appreciation of how small changes in the data affect thesolutions and experience with key examples arising in thesolution of differential equations;

I practical knowledge of polynomial interpolation, its numericalimplementation and theoretical knowledge of associatedapproximation properties;

I practical knowledge of quadrature schemes and theoreticalknowledge of their associated approximation properties;

I acquired numerical problem solving skills that can be appliedto problems from the whole range of applied mathematics.

6 / 24

Learning Outcomes

6 / 24

Learning Outcomes

6 / 24

Learning Outcomes

6 / 24

Learning Outcomes

6 / 24

Learning Outcomes

6 / 24

Outline

What is Numerical Analysis?

Numerical analysis is concerned with the development and analysisof algorithms for solving problems based on continuousmathematics.

I This includes computing integrals, solving differentialequations, approximating functions based on data samples,minimising functions.

I Applications can be found in any field in science andengineering, such as aircraft design, chemical processes,weather forecasting, signal processing, Google, ...

7 / 24

Top Ten Algorithms

1 Monte Carlo method or Metropolis algorithm, devised by John von Neumann,Stanislaw Ulam, and Nicholas Metropolis;

2 simplex method of linear programming, developed by George Dantzig;

3 Krylov Subspace Iteration method, developed by Magnus Hestenes, EduardStiefel, and Cornelius Lanczos;

4 Householder matrix decomposition, developed by Alston Householder;

5 Fortran compiler, developed by a team lead by John Backus;

6 QR algorithm for eigenvalue calculation, developed by J Francis;

7 Quicksort algorithm, developed by Anthony Hoare;

8 Fast Fourier Transform, developed by James Cooley and John Tukey;

9 Integer Relation Detection Algorithm, developed by Helaman Ferguson andRodney Forcade;

10 fast Multipole algorithm, developed by Leslie Greengard and Vladimir Rokhlin;

(List assembled by Dongarra and Sullivan)

8 / 24

Numerical Analysis and Manchester: The Baby

The world’s first stored-program computer, the Small Scale ExperimentalMachine (SSEM), also called The Baby, was developed at Manchester

University in 1948. (Picture credit: MOSI)

9 / 24

Numerical Analysis and Manchester: Alan Turing

Alan Turing not only developed the mathematical foundations of

programming, he also did pioneering work in numerical analysis by

introducing the concept of Condition Number.

10 / 24

The Challenges of Numerical Analysis

A particular challenge for Numerical Analysis is the fact thatcomputers are finite devices.

I Most numerical problems can’t be solved in a finite amount oftime.

I Most real numbers can’t be represented in a finite amount ofspace.

Since none of the numbers which we take out fromlogarithmic and trigonometric tables admit of absoluteprecision, but are all to a certain extent approximate only,the results of all calculations performed by the aid ofthese numbers can only be approximately true.- C.F. Gauss (1809)

11 / 24

The E word

An unfortunate fact in numerical computation is that we haveaccept the presence of errors. The are many types of errors toconsider...

I Modelling errors;

I Measurement errors;

I Rounding errors (most numbers cannot be stored exactly tocomputer precision);

I Truncation or approximation errors.

http://ta.twi.tudelft.nl/users/vuik/wi211/disasters.html

12 / 24

The E word

I Modelling errors;

12 / 24

The E word

I Modelling errors;

12 / 24

The E word

I Modelling errors;

12 / 24

The E word

I Modelling errors;

12 / 24

The E word

I Modelling errors;

12 / 24

The E word

A further source of error... photocopiers!

Scan on a Xerox Workcentre 7535. More information can be found on

Prof. Nick Higham’s blog.

13 / 24

The E word

A further source of error... photocopiers!

Scan on a Xerox Workcentre 7535. More information can be found on

Prof. Nick Higham’s blog.

13 / 24

Types of errors

I Modelling and measurement errors fall outside the scope ofnumerical analysis, but we need to be aware of their presence!

I Rounding errors arise due to the fact that computers operatewith limited storage space, and results of calculations arealways rounded to the neares representable number (seefloating point numbers).

I Truncation errors arise when replacing functions or equationswith an approximations. For example, the Taylorapproximation of a function gives

f(x) ≈f(x0) + f ′(x0)(x− x0) +1

2f ′′(x0)(x− x0)2 + · · ·

+f (n)(x0)

n!(x− x0)n.

I Errors can accumulate!

14 / 24

Types of errors

f(x) ≈f(x0) + f ′(x0)(x− x0) +1

2f ′′(x0)(x− x0)2 + · · ·

+f (n)(x0)

n!(x− x0)n.

14 / 24

Types of errors

f(x) ≈f(x0) + f ′(x0)(x− x0) +1

2f ′′(x0)(x− x0)2 + · · ·

+f (n)(x0)

n!(x− x0)n.

14 / 24

Types of errors

f(x) ≈f(x0) + f ′(x0)(x− x0) +1

2f ′′(x0)(x− x0)2 + · · ·

+f (n)(x0)

n!(x− x0)n.

14 / 24

Types of errors

f(x) ≈f(x0) + f ′(x0)(x− x0) +1

2f ′′(x0)(x− x0)2 + · · ·

+f (n)(x0)

n!(x− x0)n.

14 / 24

Measuring errors

In order to quantify errors in our solutions we need to define ameasure for the error

I If x̂ is an approximation to a quantity x then the absoluteerror is defined by

|x− x̂|

I The relative error is defined by

|x̂− x||x|

, x 6= 0

15 / 24

Measuring errors

In order to quantify errors in our solutions we need to define ameasure for the error

I If x̂ is an approximation to a quantity x then the absoluteerror is defined by

|x− x̂|

I The relative error is defined by

|x̂− x||x|

, x 6= 0

15 / 24

Relative errors

Example

Relative vs absolute measures

An error of 1cm makes a big difference for small objects, but is not considered an error at all on a cosmic scale.

16 / 24

Significant Digits

When doing calculations by hand, we will be using the concept ofsignificant figures.

I Starting with the first non-zero digit on the left, count all thefigures on the right of it, including trailing zeros if they are tothe right of the decimal point.

I 1.2048, 12.040 and 0.012040 all have 5 significant figures.

I Concept can be made more precise using the normalisedscientific notation.

I An approximation x̂ of a number x is correct to n significantfigures, if both number round to the same number to ksignificant digits.

17 / 24

Significant Digits

17 / 24

Significant Digits

17 / 24

Significant Digits

17 / 24

Significant Digits

17 / 24

Floating Point Arithmetic

I On a computer, numbers are stored as a bounded sequence ofbits, or 0 and 1 digits, usually using 32 bits (single precision)or 64 bits (double precision) per number.

I In double precision, a number is represented as

x = ±f × fe,

whereI f is a number in [0, 1], represented using 52 bits,I e is an exponent, represented using 11 bits.I The remaining bit is used as a sign.

I The largest representable number is of the order ±10308, thesmallest of the order 10−308.

I Floating point numbers form a finite subset of the realnumbers and are not uniformly spaced!

18 / 24

x = ±f × fe,

18 / 24

x = ±f × fe,

18 / 24

x = ±f × fe,

18 / 24

x = ±f × fe,

18 / 24

Big O notation

Our measure of running time of an algorithm is the number ofarithmetic operations needed to perform a calculation.

I Running time is measured as a function of a parameter n thatdescribes the problem.

I We are usually more interested in the order of magnitude thanexact operation count.

I We normally don’t care if an algorithm uses 0.5n2 or 20n2

operations, but will consider a difference between n log(n) orn4 operation significant.

I Given function f(n) and g(n), we say that