week 7 numerical methodsen1811/15s1/lectures/week07.pdf · methods: • bisection • interpolation...
TRANSCRIPT
ENGG1811 © UNSW, CRICOS Provider No: 00098G1 W7 slide 1
Week 7
Numerical Methods
ENGG1811 Computing for Engineers
ENGG1811 © UNSW, CRICOS Provider No: 00098G W7 slide 2
Prologue
Many engineering programs will include a
serious coverage (say a third of a course or
more) on numerical methods.
This lecture covers only a small fraction of the
topic, but delves into some interesting aspects
in detail.
The emphasis this week is on algorithms rather
than coding. However implementation is still
important, and the lab work next week will use
or extend a couple of examples presented here.
ENGG1811 © UNSW, CRICOS Provider No: 00098G W7 slide 3
Numerical Methods
Numerical Methods are algorithms that provide exact or
approximate solutions to problems that are difficult or
impossible to solve analytically.
They include
• Root finding
• Optimisation (max/min or Solver-type)
• Area of a polygon
• Area under a curve (approximating definite integrals)
• Curve fitting
• Solving systems of linear equations
• Solving differential equations
We will look at some examples of the first four of these.
Curve fitting and solving linear equations are what
Matlab is good at, and ODEs require maths > 1st year.
ENGG1811 © UNSW, CRICOS Provider No: 00098G W7 slide 4
Root Finding
We used Calc’s Goal Seek tool in the SS part of the
course. How does it work?* (we’ll tell you later)
Goal: find a value of x such that f (x) = 0
• it can also be f (x) = k for some constant k by defining
g (x) = f (x) – k and finding one of its roots
Methods:
• Bisection
• Interpolation (“regula falsi”)
• Brent’s method (composite, not covered)
• Fixed-point iteration (not covered)
* Who cares, you ask? Well, knowing how any tool, system or
technique works, be it software, car suspension, stock markets or a
truss bridge, allows you to understand its strengths and limitations so
you can make more effective use of it professionally.
• Newton-Raphson
• Secant method
ENGG1811 © UNSW, CRICOS Provider No: 00098G W7 slide 5
Bisection and Interpolation
The simpler methods start with two points x1 and x2 on
either side of the root such that f (x1) < 0 and f (x2) > 0
They pick a new value between x1 and x2 that’s likely to
be much closer to the root
Depending on the sign of the result, replace one or other
bound, continue until the bounds are close enough
Pseudocode: choose x1 and x2 such that f(x1) < 0 and f(x2) > 0 While Abs(x1-x2) > EPSILON pick x3 between x1 and x2 If f(x3) < 0 Then x1 = x3 Else x2 = x3 End If Wend
Bisection: mean of x1 and x2
Interpolation: where the line
intersecting the function at x1
and x2 crosses the x axis
Bisection
ENGG1811 © UNSW, CRICOS Provider No: 00098G W7 slide 6
x
f (x)
x1
x2
x3
x4
lo hi
x1 x2
x3 x2
x3 x4
Slow but steady
convergence
Interpolation
ENGG1811 © UNSW, CRICOS Provider No: 00098G W7 slide 7
x
f (x)
x1
x2
lo hi
x1 x2
x3 x2
x4 x2
x3
x4
One end tends to
get stuck while the
other is very close
Interpolation: where
the line intersecting the
function at x1 and x2
crosses the x axis
Modifed Regula Falsi (Interpolation 2)
ENGG1811 © UNSW, CRICOS Provider No: 00098G W7 slide 8
x
f (x)
x1
x2
x3
x4
For the end that
doesn’t move,
divide the function
value by 2
½ f (x2)
What matters is not
how close x4 is to the
root, but how small
the endpoint
difference is and how
quickly it converges
Interpolation 2 algorithm
• Same structure as bisection
• If a and b are the current estimates, the
interpolated point is given by
• To avoid repeated function
evaluations, keep variables fa and fb as well as a and b:
fa = f(a) : fb = f(b)
While Abs(fa – fb) > EPSILON
interp = (a*fb – b*fa) / (fb – fa)
fx = f(interp)
If sign of fx = sign of fa Then
a = interp : fa = fx : fb = fb/2
Else
b = interp : fb = fx : fa = fa/2
ENGG1811 © UNSW, CRICOS Provider No: 00098G W7 slide 9
)()(
)()(
afbf
afbbfa
convergence
criteria options
next slide
…
Convergence criteria
We’ve used two ways of deciding what’s “close enough”,
four are common. a and b are the two most recent
estimates:
ENGG1811 © UNSW, CRICOS Provider No: 00098G W7 slide 10
)(.4
)()(.3
.2
.1
af
bfaf
a
ba
ba Absolute: significant figures vary with magnitude
Relative: won’t work close to zero
OK but sensitive to slope at the root
best overall choice, not sensitive to slope or singularities
• The demo workbook has a GoalSeekers sheet
• These two and the remaining two algorithms coded with a specified iteration stopping point (as a demo)
• Function is defined on the sheet
– we’re solving which has one positive root,
the golden ratio (phi)
• Spin button sets iteration count
• Initial values specified (no evaluation if missing)
• Try the first two examples, these are called first-order solutions to the problem
– estimate range roughly halves (interp is a bit better) on each iteration
– need about log2 10 = 3.3 iterations per decimal place
Evaluation Sheet
ENGG1811 © UNSW, CRICOS Provider No: 00098G W7 slide 11
012 xx
618034.12
51
• A smarter technique uses a single estimate x1 and
draws a tangent from that point on the curve.
• In most cases, where the tangent crosses the x axis is
much closer to the root
• Tangent is the function derivative f ′ (x)
• Iterate this process until successive estimates are
close enough
Example: calculate the square root of some value b > 0
(this is the way basic calculators do square roots)
ENGG1811 © UNSW, CRICOS Provider No: 00098G W7 slide 12
Newton-Raphson (NR)
)(
)(1
i
iii
xf
xfxx
i
i
i
iii
x
bx
x
bxxxxxfbxxf
2
1
2;2)(;)(
2
12
Newton-Raphson
ENGG1811 © UNSW, CRICOS Provider No: 00098G W7 slide 13
x
f (x)
x1
x2
x3
Converges rapidly if
the function is
smooth and the
derivative doesn’t
change sign
22)( 3 xxxf
ENGG1811 © UNSW, CRICOS Provider No: 00098G W7 slide 14
NR swings and roundabouts
Advantages: converges rapidly under good conditions
Disadvantages:
– need the derivative as well as the function
– near a local min/max the derivative becomes very small and
the algorithm can diverge (below right, near x = ± 0.8165)
– approaching a cusp (left) the slope is ± inf, also diverges
– for high-order polynomials the algorithm can get caught in a
box if a poor starting point is chosen
x1=0, f (x1)=2
x2=1, f (x2)=1
x3=0, ...
Source: Wikipedia
ENGG1811 © UNSW, CRICOS Provider No: 00098G W7 slide 15
NR Implementation
' Find x such that f(x) = 0, starting with estimate cur Dim fcur As Double, fd As Double ' f(cur) and f'(cur) Dim fprev As Double ' previous fcur for convergence Dim converged As Boolean Dim numIt As Long converged = False While Not converged
fprev = fcur ' save for convergence test fcur = f(cur) : fd = fderiv(cur)
numIt = numIt + 1 If numIt >= ITERATION_LIMIT Or Abs(fd) <= EPSILON Then ' report that NR can’t find a solution Exit Sub End If
cur = cur – fcur / fd converged = Abs(fcur – fprev) <= EPSILON
Wend ' cur is the NR best estimate
• Apparent variation on interpolation but much better
• Starts with two points, need not be either side of
root, draw line (called a secant) through them to
intersect X axis [so far same as interp]
• Gives next x, but secant is always drawn between
the last two estimates
• Iteration formula is
• Looks a bit like NR, in fact as the x values get close,
the secant approaches the tangent, so the secant
method incorporates numeric differentiation too!
• Has similar limitations to NR but no need for explicit
derivative function
Secant method
ENGG1811 © UNSW, CRICOS Provider No: 00098G W7 slide 16
)()(
))((
1
11
ii
iiiii
xfxf
xxxfxx
Secant
ENGG1811 © UNSW, CRICOS Provider No: 00098G W7 slide 17
x
f (x)
x1
x3
x4
Like NR, a second-
order algorithm that
converges rapidly
under favourable
conditions
x2
ENGG1811 © UNSW, CRICOS Provider No: 00098G W7 slide 18
But wait, there’s more
Don’t worry, that’s enough detail for this topic
Goalseekers in specialised software tend to be adaptive,
that is, they will try either
• second-order methods until they show signs of
oscillation or divergence, then switch to first-order, or
• multiple first-order algorithms at each step, choosing
whichever appears to be approaching the root faster
Brent’s method tries secant and quadratic (rather than
linear) interpolation, with a fallback to bisection.
Calc’s Goalseek uses linear interpolation according to the
source code listing at http://opengrok.libreoffice.org/xref/core/sc/source/core/data/documen4.cxx#51
ENGG1811 © UNSW, CRICOS Provider No: 00098G W7 slide 19
Optimisation
Complete courses could be devoted to this topic, we’ll
just look at one small algorithm that can find a local
max or min value for a range of non-linear functions*
One approach is to find a root of the derivative function
(if available) using one of the root-finding schemes.
* Calc’s Goal Seek can’t do max/min and Calc’s Solver can’t do
non-linear optimisation, so this fills a genuine need.
f ′(x) < 0
f ′(x) > 0
f ′(x) = 0
Of course to use
NR you’ll need
the second
derivative too
ENGG1811 © UNSW, CRICOS Provider No: 00098G W7 slide 20
Finding the minimum
Consider a function that’s unimodal (slope changes sign
exactly once) in a region, and has a minimum there.
We pick two points that bracket the minimum, and a
third between them, and evaluate the function there:
y4a
y1
y2
y3
a b
c
y4b
Now choose a point
x4 in the larger
region x2|x3
If y4 < y2 as shown,
the min must be in
the region x2|x3
x1 x2 x3
But if y4 > y2 (blue),
the min must be in
the region x1|x4 x4
Minimum
• So in case 1 the region is reduced from a+b to b,
and in case 2 from a+b to a+c. For maximum
efficiency these should be equal. a+c = b ...(1)
• Now we have to decide how x2 was placed. For the
algorithm to work the subintervals on each side
must maintain the original ratio r = b/a. Thus
ENGG1811 © UNSW, CRICOS Provider No: 00098G W7 slide 21
x3 c x1 x2 x4
a b
01,01
1,01
1,1
:Invert
so,(1)frombut 2case
2
rrr
rra
b
ra
ab
rab
aabcr
c
a
Golden Section Search
r2 – r – 1 = 0 ?
Hang on, we just solved that, r must be equal to , the
Golden Ratio (in fact, interval subdivision like this is one
way to define phi). For this reason the algorithm is
called the Golden Section Search.
It was discovered (remember good algorithms already
exist, they just have to be discovered) by Jack Kiefer in
1953.
The algorithm itself iterates the interval subdivision
principle, choosing one new point each time. An
analogous one to bisection would have to choose 2
points to decide which subinterval to follow.
ENGG1811 © UNSW, CRICOS Provider No: 00098G W7 slide 22
Golden Section Search algorithm
Note that whichever subdivision we take, the
equivalent of x2 is already placed, we just have to
worry about x4.
However we need to know whether the larger portion is
to the left or right, or equivalently, whether x2 is right
or left of the midpoint, respectively
Larger right: x4 = x1 + b = x1 + (x3 – x2)
Larger left: x4 = x3 – b = x3 – (x2 – x1) = x1 + (x3 – x2)
OMG they’re the same!! That’s a sign of a truly elegant
algorithm (IMHO).
Cases 1 and 2 must be handled separately to establish
the new subintervals, as different end-points move
ENGG1811 © UNSW, CRICOS Provider No: 00098G W7 slide 23
ENGG1811 © UNSW, CRICOS Provider No: 00098G W7 slide 24
GSS Solution
' Find local minimum of f(x), lying between x=lo and x=hi ' the subinterval boundary is mid (x2 in the analysis) ' the new x value is probe (x4 in the analysis) ' fLo, fHi, fMid and fProbe are corresponding function values ' PHI is the Golden Ratio mid = lo + (hi – lo) * (1 + 1/PHI) ' the only use of PHI ! While Abs(fHi – fLo) <= EPSILON
probe = lo + (hi – mid) ' x4 fProbe = f(probe) ' only need one func eval per iteration If probe > mid Then ' right is larger
If fProbe < fMid Then ' Case 1
lo = mid : fLo = fMid mid = probe : fMid = fProbe Else ' Case 2 hi = probe : fHi = fProbe ' mid is OK End If
Else ' left is larger: symmetric If statement, with lo hi End If
Wend ' mid is the GSS best estimate
Practicalities
• This algorithm is first-order, about 1/log10 = 4.8
iterations per significant digit, or around 60 to get a highly precise answer
• To turn this into a practical tool we need two things
1. A means of supplying the function and initial estimates, and
2. A funky name for it.
• #2 is easy, it’s MiniMe (apologies to Mike Myers)
• The Optimisers sheet in the demo workbook has a similar test setup to the Goalseekers sheet.
ENGG1811 © UNSW, CRICOS Provider No: 00098G W7 slide 25
Area of a Polygon
• Given the coordinates in 2D space of the N vertices of a polygon, a neat formula due to Gauss calculates the area
ENGG1811 © UNSW, CRICOS Provider No: 00098G W7 slide 26
x0 (1,2)
x1 (2,3)
x2 (4,4)
x3 (6,3)
xN = x0
x4 (4,3)
x5 (3,1.5)
1
0
112
1N
i
iiii yxyxA
i xi yi term
0 1 2 3 – 4 = –1
1 2 3 8 – 12 = –4
2 4 4 12 – 24 = –12
3 6 3 18 – 12 = 6
4 4 3 6 – 9 = –3
5 3 1.5 6 – 1.5 = 4.5
6 1 2 sum = –9.5 The cross product form suggests the name Shoelace Formula
How does it work?
Note: this is the principle on which the algorithm works, though each term doesn’t correspond directly to a trapezium
ENGG1811 © UNSW, CRICOS Provider No: 00098G W7 slide 27
1. Project vertical lines from the vertices to the X axis. These form trapeziums.
2. Add the areas of the trapeziums in clockwise order, noting that as you process the lower vertices, the areas are negative.
3. Eventually the areas under the polygon are subtracted, leaving just the polygon area
Benefits and limitations
• Can be fairly easily adapted to surveying data (distance/bearing for each leg of a closed traverse)
• Can be extended to calculate the centroid (centre of mass if the polygon was a plate of uniform density): search for ‘centroid’ in Paul Bourke’s
http://paulbourke.net/geometry/polygonmesh/
• Need coordinates, so it doesn’t directly give the answer to a question like: what’s the area of a regular pentagon with each side 100 metres?
• Works for polygons with holes, provided holes are traversed in the opposite order to the perimeter
ENGG1811 © UNSW, CRICOS Provider No: 00098G W7 slide 28
• Does not work for self-intersecting polygons like these:
Numerical Integration
• Definite integrals represent the area under the
curve described by f (x)
• Sometimes the
function integral doesn’t have a closed form
• Sometimes the function is only sampled, or is
empirically determined
• In both cases, the area can be approximated using
suitable algorithms
ENGG1811 © UNSW, CRICOS Provider No: 00098G W7 slide 29
b
a
dxxfA )(
f (x)
a b
A square units x
ENGG1811 © UNSW, CRICOS Provider No: 00098G W10 slide 30
Trapezoidal Rule
The simplest of the numerical integration algorithms is
the Trapezoidal Rule, which uses thin vertical slices
with straight lines at the top approximating the function
• Slices are adjacent trapeziums, area easily calculated
• If the panel width is small enough, the calculated area
is asymptotically correct, but...
– area is underestimated if f is convex as viewed from below
– area is overestimated if f is concave
a
f (x)
b
Integral of f (x)
from a to b
is approximated by
the sum of the areas
inside the red trapeziums
Visualisation
A nice animation is available at
http://math.fullerton.edu/mathews/a2001/Animations/Quadrature/Trapezoidal/Trapezoidalaa.html
(numerical integration is also known as quadrature)
The integral of the sample function +1
doesn’t have a closed form*, and the definite integral
from x=0 to x=2 can only be approximated using these
kinds of methods.
Its value is about 2.016
* Matlab’s symbolic integration toolbox can’t integrate it, anyway
ENGG1811 © UNSW, CRICOS Provider No: 00098G W7 slide 31
3/28sin)( xexf x
Derivation of the TR
The area of a trapezium with base
and heights h1 and h2 is just
The heights are function values, and
the panels are all of equal width , hence
So the outside points are used once and all the
interior points are used twice.
ENGG1811 © UNSW, CRICOS Provider No: 00098G W7 slide 32
h1 h2 )(
221 hh
)()(2...)2(2)(2(2
)()(2
...)2()(2
)()(2
)(
bfbfafafaf
bfbfafafafafdxxf
b
a
½(h1+ h2)
ENGG1811 © UNSW, CRICOS Provider No: 00098G W10 slide 33
Trapezoidal Rule, theory
Assuming there are n equally spaced panels, the formula
in terms of successive x values reduces to
*/)( ianabiaxwhere i
)()(2)(2)(2)(2
)( 1210 nn
b
a
xfxfxfxfxfdxxf
Twice the middle values
Panel width
n
ab
End-points counted
once only
ENGG1811 © UNSW, CRICOS Provider No: 00098G W10 slide 34
Trapezoidal Rule, pseudocode
We can easily convert this to pseudocode:
set sum to f (a) + f (b) as these are the odd ones out
set delta to the panel width (b-a)/n
For p = 1 To n-1
add 2*f(a + delta*p) to sum
Next p
area = sum * delta/2
ENGG1811 © UNSW, CRICOS Provider No: 00098G W10 slide 35
Trapezoidal Rule, implementation
' Estimates the area under the curve of f(x) from x=a ' to x=b using the Trapezoidal Rule with n panels. ' Uses FuncEvalTrap(x) to obtain function values from ' the active sheet.
Function TrapAreaFunc(a As Double, b As Double, _
numPanels As Long) As Double
' Estimates the area under the curve of f(x) ' expressed as a series of sampled values on the ' active sheet starting with the given row and column ' Uses the Trapezoidal Rule.
Function TrapAreaSampled( _ rowStart As Long, colStart As Long, _ numSamples As Long) As Double
Each of these has a wrapper subprogram that’s linked to a
button on the sheet (so the functions are only called when the
data is stable)
ENGG1811 © UNSW, CRICOS Provider No: 00098G W7 slide 36
TR1 Wrapper
' First wrapper (second is similar but more complex validity check) Sub CalculateTRbyFormula() Dim a As Double, b As Double Dim n As Long ' fetch problem parameters a = ActiveSheet.Range("TR1a") b = ActiveSheet.Range("TR1b") n = ActiveSheet.Range("TR1n") ' clear result cell during calculations ActiveSheet.Range("TR1Area") = "" ' check limit validity If a >= b Then ActiveSheet.Range("TR1valid") = False Exit Sub End If ActiveSheet.Range("TR1valid") = True ActiveSheet.Range("TR1Area") = TrapAreaFunc(a, b, n) End Sub
Adding buttons for action
Add buttons to the sheet to run the integrators
– View – Toolbars – Form controls
– Design Mode on (all icons visible)
– Pick the button icon, draw on sheet
– Right-click, select Control...
– General tab, relabel appropriately
– Events tab, choose Execute Action, press
ENGG1811 © UNSW, CRICOS Provider No: 00098G W7 slide 37
Adding a button, continued
• Assign macro by navigating to the Standard module in the workbook, select CalculateTRbyFormula (in this case)
• When it’s all OK, turn Design Mode off again to activate the button
ENGG1811 © UNSW, CRICOS Provider No: 00098G W7 slide 38
Putting it all together
Use the sample function, but also try quarter, half and full cycles of
For the sampled version, first use evenly spaced calculated values, then irregularly spaced ones.
Finally use the tool to solve the ellipse perimeter and river flow problems overleaf
ENGG1811 © UNSW, CRICOS Provider No: 00098G W7 slide 39
Integration problems
1. The orbit of the demoted former planet Pluto is elliptical, with axes
a = 5.9065e+9 km and
b = 5.7208e+9 km.
The perimeter of an ellipse is:
determine Pluto’s orbital distance.
2. Estimate the cross-sectional area of a river whose depth
has been measured at 12m intervals:
ENGG1811 © UNSW, CRICOS Provider No: 00098G W7 slide 40
a
b
a
bakdkaP
222/
0
22 wheresin14
1 11 16 20 22 20 18 12 19 21 20 19 9 10 4 (m)
Other integrators
• Simpson’s Rule – uses quadratic segments rather than straight lines
• Gauss quadrature
• Romberg integration
• etc etc (you don’t need to know this)
ENGG1811 © UNSW, CRICOS Provider No: 00098G W7 slide 41
Summary
• There are often multiple ways to approximate numeric solutions to tough analytical problems
• Root finding can be achieved with
– simple, reliable but slow first-order algs
– clever but sometimes unreliable second-order algs
• Minimisation can be found through
– direct slicing means (bisection, golden section),
– by finding roots of the derivative
– by clever means that follow the gradient better (we didn’t get to that)
• Polygon area can be calculated using Gauss’ (or Shoelace) formula
• Numeric integration divides the area into geometric segments and uses their areas to approximate the solution
ENGG1811 © UNSW, CRICOS Provider No: 00098G W7 slide 42