mathematical principles for scientiﬁc computing and

Mathematical Principles for

Scientific Computing and Visualization

Chapter 2: Computational Basics

Gerald Farin & Dianne Hansford

CRC Press, Taylor & Francis Group, An A K Peters Bookwww.farinhansford.com/books/scv

c©2008

Farin & Hansford Scientific Computing and Visualization 1 / 22

Outline

1 Introduction to Computational Basics

2 Algorithms

3 Floating Point Numbers

4 Errors

5 Case Study: 1991 Scud Attack


Computational Basics

Scientific Computing / Computational Science

solves scientific problems by

• Constructing good mathematical models

• Using efficient computational tools

Key:

• Algorithms• Floating point numbers


Algorithms: Defintion

Algorithm = set of instructions to solve a problem

• unambiguous

• terminates

• complete

• provable

Ambiguous sentence fun: Link

Ambiguous “Dangling else”

if (a > b) then if (c > d) then print(“1”) else print(“2”)


http://www.byrdseed.com/ambiguous-sentences/

Algorithms: Errors → dangerous consequences

WW III Almost:1980 NORAD software reported US under attackCause: Algorithm didn’t cover case of faulty circuit

Undetected Exposure:NASA software missed hole in ozone layer for possibly 5 yearsCause: Algorithm did not report outlier data

Crash and Burn:1999 Mars Climate Orbiter flew too close to Mars and disintegrated,costing $378 M and 1 year of travelCause: incorrect mix of metric and English units

Deadly Radiation Therapy:Therac-25 delivered up to 100x too much radiation, killing 3 peopleCause: Bug in user-interaction software(software testers did not use software as technicians)

More examples: Link


http://royal.pingdom.com/2009/03/19/10-historical-software-bugs-with-extreme-consequences/

Algorithm Complexity: Big-O

Worst case execution time (or space)

Function of the amount of input data n

O(1) Access array index

O(logn) Binary search

O(n) Traverse array

O(nlogn) Merge sort

O(n2) Traverse a 2D arrray

O(n3) Solve n× n linear system

O(2n) Recursive Fibonacci (exponential)

See AlgorithmComplexity.nb for plots


Algorithm Complexity: O(logn)Binary Search: O(logn)

Where does the log come from?

Problem: Find the position of a target value in a sorted array of n values

Example: Find 12 in the array

1 3 4 6 7 9 10 12 15

Let x be the number of times the array is divided until down to 1 element

n

2x= 1

2x = n

log2(2x ) = log2(n)

x = log(n)


Algorithm Complexity: Hard Problems

Hard problems: efficient algorithms do not exist

Hard 6= Poorly designed algorithm

Not hard ≡ bounded by a polynomial

Not hard: Solve n linear equations with n unknownsGauss elimination with n3 computations

Hard: Solve nonlinear system of equationsNo bound on the number of computations

Hard: How many shapes are able to tile the plane?15th convex pentagon found in 2015 Link

Approximate solutions used to solve hard problems• Hueristics: greedy algorithms, e.g, Bees’ garden rounds• Linearization: piecewise linear, e.g., FEM


http://www.npr.org/sections/thetwo-way/2015/08/14/432015615/with-discovery-3-scientists-chip-away-at-an-unsolvable-math-problem

Algorithmic Advances

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

1617 18

19

20

Voronoi diagram of a point set

Innovative techniques andNew mathematical machinery

Example: Computer simulation of acluster of 1000 stars

For each time step:

Attraction of each pair⇒ 999,000 computations

Instead: neighboring stars only:

6 neighbors/star average

6 attractions × 1000 stars⇒ 6000 computations


Floating Point Numbers I

IEEE standard forsingle precision (“float”):

±0.x1x2x3 . . . x9 ± e1e2 range: 10±38

sign, significand/mantissa, exponent

double precision:

±0.x1x2x3 . . . x16 ± e1e2e3 range: 10±308

Advantages of FPN:

Omit leading/trailing zeros (scientific notation)

Extend range of numbers representable

Maximum precision


Floating Point Numbers II

FPN approximates a real number

• significand provides precision

• exponent provides range

Fixed point 32 bit number: 231 − 1 ≈ 109 max value

Floating point 32 bit number:

1 sign bit + 23 significand bits + 8 exponent bits

8 exponent bits: 28/2 → range [−128, 127]

2128 ≈ 1038 ⇒ max value ≈ 1038

Floating point 64 bit number: 1 sign + 52 sig + 11 exprange [−1022, 1023] ⇒ max value ≈ 10308


Floating Point Numbers III

Significand stored in powers of 2−i

2−1 = 0.5 2−2 = 0.25 2−3 = 0.125

Only a finite number of real numbers are representable!

−10308 −10−308 0 10−308 10308overflow usable underflow usable overflow

Zoom in on usable range: representable numbers not uniformly distributed|||| | | | | | | | | | |

“single precision”: 32 bit with 7 (ave) significant digits“double precision”: 64 bit with 16 (ave) significant digits

Resource: IEEE.org See number-rep.nb


Floating Point Numbers IV

Storage and manipulation of FPNs requires computing power

Special chip: Floating Point Unit (FPU) – “math coprocessor”

IEEE standards ensure program transportability and consistency of resultsfrom machine to machine

Supercomputers simulating complex physical phenomena– Billions of variables

FLOPS = Floating Point Operation Per Second

June 2016: fastest at 93 PFLOPS (Peta FLOPS = 1015 FLOPS)


Floating Point Errors

Finite number of digits

0.1E1 + 0.1E − 20 = 0.1E1

Non-associativity of addition:(x + y) + z 6= x + (y + z)

(See number-rep.nb)



Never usex == y

Instead use a tolerance

abs(x − y) < 0.5E − 08


Truncation Error

-6 -4 -2 2 4 6

-1.0

-0.5

0.5

1.0

Approximation with five terms

sin x =

∞∑

i=1

(−1)i−1

(2i − 1)!x2i−1

See TruncationError.nb


Cancellation Error

-10 -5 5 10

0.1

0.2

0.3

0.4

0.5

-4.´10-8-2.´10-8 2.´10-8 4.´10-8

0.2

0.4

0.6

0.8

1.0

f (x) =1− cos x

x2

See Cancellation subtractive.nb



Decimal ≃ Binary2 ≃ 103 ≃ 119 ≃ 10010.5 ≃ 0.1

0.1 ≃ 0× 1/2 + 0× 1/4 + 0× 1/8 + 1× 1/16 + 1× 1/32 + 0× 1/64 +0× 1/128 + 1× 1/256 + +1× 1/512 + . . .

= .000110011 . . .Non-terminating binary expansion

Conversion / truncation error: 0.1 × 10 6= 1

Result depends on precision


Case Study: 1991 Scud Attack

1st Gulf war 1991

Iraq attacking US troopsusing Scud Missiles

US defense: Patriot missiles


Patriot Clock

Start: 0 seconds

Tracking computer’s time counter: integer units indicating 1/10 second

Time converted to 24 bit fixed point binary

Since 1/10 has a non-terminating binary expansion

⇒ each time increment off by 9.5E-8 sec (times reduced by 0.0001%)


Patriot Tracking

Figure credit: University of Sydney

Object detection(position + velocity)

Position prediction after 1 second– Based on ballistic trajectory

Object at predicted position? Fire!

Original system built in early 1970s

System updated for Gulf War to handle ballistic technology

System updated to 48 bit floating point

Problem: Error in software update – not all clocks updated


Scud Disaster

Patriot system up for 100 hours

Timing off by 0.34 seconds

100 hr × 60 min/hr × 60 sec/min × 10 inc/sec = 3,600,000 inc

Error = 9.5E-8 sec × 3.6E6 = 0.34 sec

Scud traveling approx. 1600 meters/sec

⇒ Scud position missed by 500+ meters

Patriot not fired

28 fatalities and 100 injuries

Source: Robert Skeel, Roundoff Error and the Patriot Missile, SIAM News,Volume 25, No. 4, page 11, July 1992.


mathematical principles for scientiﬁc computing and

Documents