ch7
TRANSCRIPT
Instructor
Walid Morsi Ibrahim
Part II: Roots and Optimization
Fall 2010ENGR 1400 Information Technology for Engineers1
Roots
Fall 2010ENGR 1400 Information Technology for Engineers2
Quadratic formula
These values of x are called the roots or zeros of the equation.
Many functions are not solved so easily, thus we will look at the numerical
methods, for example: Bungee Jumper
If you want to solve for m, you cant bring m to the left hand side.
Subtract v(t) from both sides
And then find the roots, i.e. the value of m that makes f(m) = 0
Solution:
Optimization
Fall 2010ENGR 1400 Information Technology for Engineers3
Is the determination of optimal values.
For the figure shown, values of x that maximizes or
minimizes f(x)
Finding the Roots
Fall 2010ENGR 1400 Information Technology for Engineers4
Graphical methods: plot the function and see where the zero
crossings are.
We can zoom in on the plot for increased precision but there
is a great deal of human intervention.
Therefore it is very slow. More automated methods are
needed.
General Rules for Finding the Roots
Graphically
Fall 2010ENGR 1400 Information Technology for Engineers5
Consider xl and xu are the lower and upper limit points.
If both are positive or negative no root
Even number of roots
If they have different sign odd number of roots
Exceptions
Fall 2010ENGR 1400 Information Technology for Engineers6
Bracketing Methods and Initial Guess
Fall 2010ENGR 1400 Information Technology for Engineers7
Bracketing methods are based on two initial guesses, one on
either side of the root.
Advantage: always works
Disadvantage: converges slowly
Open methods have an initial guess but do not need to
bracket the root.
Advantage: more quicker
Disadvantage: may not work if the solution diverges
Increment Search
Fall 2010ENGR 1400 Information Technology for Engineers8
This method searches for an interval with a sign change.
It sates that: if f(x) is real and continuous in the interval from
xl to xu and f(xl) and f(xu) have opposite sign that is:
f(xl) f(xu) < 0
Then there is at least one real root between xl and xu
Limitations
Fall 2010ENGR 1400 Information Technology for Engineers9
If the increment length is too small then the search becomes
time consuming.
If the increment length is too large then some roots may be
missed.
Bisection Method
Fall 2010ENGR 1400 Information Technology for Engineers10
Start with initial bounds and cut the problem in half again
and again.
Ex: use bisection method to determine the drag coefficient
needed so that an 65 kg bungee jumper has a velocity of 35
m/s after 4.5 s of free fall. g = 9.81 m/s2. start with xl =
0.2, xu = 0.3 and iterate until < 2%.
Solution
Fall 2010ENGR 1400 Information Technology for Engineers11
Solution Cont.
Fall 2010ENGR 1400 Information Technology for Engineers12
Example on Bisection Method
Fall 2010ENGR 1400 Information Technology for Engineers13
Looking at the graph the root
Can be estimated to be at 145
Now try the same problem
Fall 2010ENGR 1400 Information Technology for Engineers14
f(50) = -ve and f(200) = +ve
Bracketing [50,200]
xl has to be updated. New interval
xu has to be updated. New interval
Run M-file bisection to get final value of the
root at the 8th iteration 142.7376
Graphical representation of Bisection
Method
Fall 2010ENGR 1400 Information Technology for Engineers15
See M-file bisectnew: Given xl = 40 and xu = 200, Sol mass = 142.7376
False Position Method
Fall 2010ENGR 1400 Information Technology for Engineers16
Rather than bisecting the interval, it locates the root by
joining f(xl) and f(xu) with a straight line. The intersection of
this line with the x axis represents an improved estimate of
the root.
Derivation of the FP Method
Fall 2010ENGR 1400 Information Technology for Engineers17
Referring to the figure:
[f(xu) - f(xl)]/[xu - xl ]= m [f(xu) - f(xl)]/[xu - xl ] = f(xu) /[xu - xr ]
And [f(xu) - 0]/[xu - xr ]= m
[xu - xr ] = [f(xu) [xu - xl ]]/ [f(xu) - f(xl)]
- xr = - xu + [f(xu) [xu - xl ]]/ [f(xu) - f(xl)]
xr = xu - [f(xu) [xu - xl ]]/ [f(xu) - f(xl)]
xr = xu - [f(xu) [xl - xu]]/ [f(xl) - f(xu)]
Disadvantages
Fall 2010ENGR 1400 Information Technology for Engineers18
The shape of the function influences the new root estimate
The figure shows slow convergence of the FP method
f(x)
root
x
Fall 2010ENGR 1400 Information Technology for Engineers19
Open Methods
Open Methods
Fall 2010ENGR 1400 Information Technology for Engineers20
In bracketing methods the root has to be located between an
upper and lower bound: xl and xu
In open methods the starting value (initial estimate) does not
need to bracket the root.
Although the solution converges more quickly , sometimes it
may diverge.
Comparison
Fall 2010ENGR 1400 Information Technology for Engineers21
Simple Fixed-point Iteration
Fall 2010ENGR 1400 Information Technology for Engineers22
Rearranging the function f(x) = 0 so that x is in the left hand
side of the equation: x = g(x)
Then given an initial estimate xi the new value of x can be
calculated: xi+1 = g(xi)
The process continue until the error criteria is met:
= abs([xi+1 – xi]/[xi+1]) x 100%
Example
Fall 2010ENGR 1400 Information Technology for Engineers23
Pros and Cons
Fall 2010ENGR 1400 Information Technology for Engineers24
Simple and easy formula
Slow convergence
Convergence depends mainly on the location of the initial
guess.
Divergence may occur
Newton-Raphson
Fall 2010ENGR 1400 Information Technology for Engineers25
The most widely used.
Considering the figure:
Example
Fall 2010ENGR 1400 Information Technology for Engineers26
Example
Fall 2010ENGR 1400 Information Technology for Engineers27
Four Cases of Poor Convergence
Fall 2010ENGR 1400 Information Technology for Engineers28
Function file Newton Raphson
Fall 2010ENGR 1400 Information Technology for Engineers29
Secant Method
Fall 2010ENGR 1400 Information Technology for Engineers30
If the derivative used in NR is not available then a finite
difference derivative can be used
Secant method requires two initial values xi and xi-1 to yield a
new value xi+1
An alternative approach uses one initial value and a small
perturbation fraction (modified secant method)
Convergence and Divergence
Fall 2010ENGR 1400 Information Technology for Engineers31
Example
Fall 2010ENGR 1400 Information Technology for Engineers32
Example Cont.
Fall 2010ENGR 1400 Information Technology for Engineers33
Trade-off in choosing the perturbation
value
Fall 2010ENGR 1400 Information Technology for Engineers34
If the perturbation value is chosen too small, round off error
could be increased especially in the denominator.
If the perturbation value is chosen too large, then the
solution may diverges and the algorithm becomes inefficient
MATLAB Function: fzero
Fall 2010ENGR 1400 Information Technology for Engineers35
Methods that we have seen so far areeither:
1. Slow but reliable (bracketing methods)
2. Fast but possibly unreliable (openmethods)
As a trade off, better results could beobtained using the fzero function.
Fzero is a combination of the bisection(reliable), secant (fast) and inversequadratic interpolation (very fast).
Note the inverse quadratic interpolationuses parabola instead of straight line.
Secant method
Inverse quadratic
interpolation
MATLAB Function: fzero
Fall 2010ENGR 1400 Information Technology for Engineers36
Syntax:
Disadvantages: root has to cross the x axis. If it touches the x
axis then it is not valid.
Ex: y = x.^2
Initial guess
Guesses that bracket a sign change
Polynomials
Fall 2010ENGR 1400 Information Technology for Engineers37
Are nonlinear algebraic equations of the general form:
MATLAB function: roots
Syntax:
Is a column vector
Containing the rootsIs a row vector containing
The polynomial coefficients
Example
Fall 2010ENGR 1400 Information Technology for Engineers38
Use MATLAB to find the roots of the following polynomial:
Sol:
Evaluate the polynomial at x = 1as substitute by x =1 in f5(x)
Polynomials
Fall 2010ENGR 1400 Information Technology for Engineers39
To get the polynomial from its roots:
Fall 2010ENGR 1400 Information Technology for Engineers40
Optimization
What is optimization?
Fall 2010ENGR 1400 Information Technology for Engineers41
It is the process of creating something that is as effective as possible.
Some examples of optimization:
1. Finding the minimum or maximum
2. Minimizing the fuel consumption
3. Maximizing the data rate
One popular method for optimization is to solve for the root of the problem:
Since at minimum or maximum the slope should be zero
Is it maximum or minimum?
Fall 2010ENGR 1400 Information Technology for Engineers42
We need to check the sign of that root:
1. If the sign is negative, then it is maximum
2. If the sign is positive then it is minimum.
Optimization types
Fall 2010ENGR 1400 Information Technology for Engineers43
One dimensional: One dependent variable f(x) based on one
independent variable x and we are searching for x*
Two dimensional: one dependent variable f(x,y) based on two
independent variables x and y and we are searching for x* and
y*
One Dimensional Optimization
Fall 2010ENGR 1400 Information Technology for Engineers44
Global optimum represents the very best solution
Local optimum is not the very best
Since optimization is based on finding roots then the
optimization methods are similar to the idea of bracketing
used in bisection method
Golden Section Search
Fall 2010ENGR 1400 Information Technology for Engineers45
Euclid’s definition of the golden ratio:
Select l1 and l2 such that:
[l1 + l2]/ l1 = l1 / l2Multiplying by
l1 / l2 = Ф gives:
Ф2 - Ф – 1 = 0
Solving for Ф
Ф = [1 + ]/2 = 1.618033 golden ratio
Like bisection method we need a lower and an upper bound of an interval containing a single minimum (unimodal)
l1 + l2
l1 l2
How to proceed with the Golden
Search?
Fall 2010ENGR 1400 Information Technology for Engineers46
We need to identify two points
between xl and xu say x1 and x2
if f(x2) < f(x1) therefore the
minimum lies between x1 and xu
Example:
Fall 2010ENGR 1400 Information Technology for Engineers47
Example Cont.
Fall 2010ENGR 1400 Information Technology for Engineers48
Since
For next iteration: new values (or updated values)
The process is repeated until the 8th iteration, optimum value
is:
MATLAB Function fminbnd
Fall 2010ENGR 1400 Information Technology for Engineers49
Similar algorithm to fzero
Find the minimum of a function of one variable within fixed
interval.
fminbnd for optimization (one dimensional) combines slow
dependable (golden section) and faster possible unreliable
(parabolic interpolation).
Syntax:
Location value of the min of f(x) Bounds of the interval being searched
Limitations: limited to one dimensional-only handles real
variables-slow convergence
Multi-dimensional Optimization
Fall 2010ENGR 1400 Information Technology for Engineers50
MATLAB function fminsearch
Syntax:
Unconstrained optimization
Start with an initial estimates
Only handles real values
If complex then it has to be split into two parts.