solving optimization problems with matlab · 93 global optimization toolbox solvers globalsearch,...
Post on 26-Jul-2020
41 Views
Preview:
TRANSCRIPT
1© 2015 The MathWorks, Inc.
Solving Optimization Problems with
MATLAB
LOREN SHURE
2
Introduction
Least-squares minimization
Nonlinear optimization
Mixed-integer programming
Global optimization
Topics
3
Optimization Problems
Minimize Risk
Maximize Profits
Maximize Fuel Efficiency
4
Design Process
Initial
Design
Variables
System
Modify
Design
Variables
Optimal
DesignObjectives
met?
No
Yes
5
Why use Optimization?
Manually (trial-and-error or iteratively)
Initial
Guess
6
Why use Optimization?
Automatically (using optimization techniques)
Initial
Guess
7
Why use Optimization?
Finding better (optimal) designs
Faster Design Evaluations
Useful for trade-off analysis
Non-intuitive designs may be found
Antenna Design Using Genetic Algorithmhttp://ic.arc.nasa.gov/projects/esg/research/antenna.htm
8
Introduction
Least-squares minimization
Nonlinear optimization
Mixed-integer programming
Global optimization
Topics
9
Curve Fitting Demo
Given some data:
Fit a curve of the form:
teccty 21
)(
t = [0 .3 .8 1.1 1.6 2.3];
y = [.82 .72 .63 .60 .55 .50];
0 0.5 1 1.5 2 2.50.5
0.55
0.6
0.65
0.7
0.75
0.8
0.85
0.9
t
y
10
How to solve?
Ecc
cey
eccty
t
t
2
1
21
1
)(
As a linear system of equations:
2
2min yEc
Ecy
c
Can’t solve this exactly
(6 eqns, 2 unknowns)
An optimization problem!
2
1
3.2
6.1
1.1
8.0
3.0
0
1
1
1
1
1
1
50.0
55.0
60.0
63.0
72.0
82.0
c
c
e
e
e
e
e
e
11
Introduction
Least-squares minimization
Nonlinear optimization
Mixed-integer programming
Global optimization
Topics
12
Nonlinear Optimization
min𝑥
log 1 + 𝑥1 −4
3
2
+ 3 𝑥1 + 𝑥2 − 𝑥13 2
13
Nonlinear Optimization - Modeling Gantry Crane
Determine acceleration profile
that minimizes payload swing
s25s4
s20s1
s20s1
:sConstraint
2
1
21
f
p
p
ppf
t
t
t
ttt
14
Functions for analytical computations
Conveniently manage & document symbolic computations
in MuPAD Notebook
• Math notation, embedded text, graphics
• Share work as pdf
Perform exact computations using familiar MATLAB
syntax in MATLAB
Integrate with numeric computing – MATLAB,
Simulink and Simscape language
Perform Variable-precision arithmetic
Symbolic Math Toolbox
Integration Differentiation Solving equations Transforms
Simplification
15
Introduction
Least-squares minimization
Nonlinear optimization
Mixed-integer programming
Global optimization
Topics
16
Mixed-Integer Programming
Many things exist in discrete amounts:
– Shares of stock
– Number of cars a factory produces
– Number of cows on a farm
Often have binary decisions:
– On/off
– Buy/don’t buy
Mixed-integer linear programming:
– Solve optimization problem while enforcing that certain
variables need to be integer
17
Continuous and integer variables
𝑥1 ∈ 0, 100 𝑥2 ∈ 1,2,3,4,5
Linear objective and constraints
min𝑥
−𝑥1 − 2𝑥2
𝑥1 + 4𝑥2 ≤ 20𝑥1 + 𝑥2 = 10
such that
Mixed-Integer Linear Programming
18
Traveling Salesman Problem
Problem
How to find the shortest path through a series of points?
Solution
Calculate distances between all combinations of points
Solve an optimization problem where variables correspond to trips
between two points
1
11
0
1
1
0
0
00
19
Introduction
Least-squares minimization
Nonlinear optimization
Mixed-integer programming
Global optimization
Topics
20
Example Global Optimization Problems
Why does fmincon have a hard time finding the
function minimum?
0 5 10
-10
-5
0
5
10
x
Starting at 10
0 5 10
-10
-5
0
5
10
x
Starting at 8
0 5 10
-10
-5
0
5
10
x
Starting at 6
x s
in(x
) +
x c
os(2
x)
0 5 10
-10
-5
0
5
10
x
Starting at 3
0 5 10
-10
-5
0
5
10
x
Starting at 1
0 5 10
-10
-5
0
5
10
x
Starting at 0x s
in(x
) +
x c
os(2
x)
21
Example Global Optimization Problems
Why didn’t fminunc find the maximum efficiency?
Revolutions Per Minute, RPM
Manifold
Pre
ssure
Ratio
Peak VE Value = 0.96144
Start
End
1000 2000 3000 4000 5000 60000
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
22
Example Global Optimization Problems
Why didn’t nonlinear regression find a good fit?
0 20 40 60 80 100 1200
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
t
c
c=b1e-b
4t+b
2e-b
5t+b
3e-b
6t
23
Global Optimization
Goal:
Want to find the lowest/largest value of
the nonlinear function that has many local
minima/maxima
Problem:
Traditional solvers often return one of the
local minima (not the global)
Solution:
A solver that locates globally optimal
solutionsRastrigin’s Function
24
Global Optimization Solvers Covered Today
Multi Start
Global Search
Simulated Annealing
Pattern Search
Genetic Algorithm
25
MultiStart Demo – Nonlinear Regression
lsqcurvefit solution MultiStart solution
26
MULTISTART
27
What is MultiStart?
Run a local solver from
each set of start points
Option to filter starting
points based on feasibility
Supports parallel
computing
28
MultiStart Demo – Peaks Function
29
GLOBAL SEARCH
30
What is GlobalSearch?
Multistart heuristic algorithm
Calls fmincon from
multiple start points to try
and find a global minimum
Filters/removes non-
promising start points
31
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
GlobalSearch Overview Schematic Problem
Peaks function
Three minima
Green, z = -0.065
Red, z= -3.05
Blue, z = -6.55
x
y
32
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
GlobalSearch Overview – Stage 0Run from specified x0
x
y
33
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
GlobalSearch Overview – Stage 1
3
6
0
00
4
0
-2
x
y
34
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
GlobalSearch Overview – Stage 1
3
6
0
00
4
0
-2
x
y
35
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
GlobalSearch Overview – Stage 1
x
y
36
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
GlobalSearch Overview – Stage 2
x
y
37
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
GlobalSearch Overview – Stage 2
x
y
38
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
GlobalSearch Overview – Stage 2
x
y
39
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
GlobalSearch Overview – Stage 2
x
y
40
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
GlobalSearch Overview – Stage 2
6
Current penalty
threshold value : 4
x
y
41
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
GlobalSearch Overview – Stage 2
x
y
42
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
GlobalSearch Overview – Stage 2
Current penalty
threshold value : 4
-3
x
y
43
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
GlobalSearch Overview – Stage 2Expand basin of attraction if minimum already found
Current penalty threshold value : 2
-0.1
x
y Basins can overlap
44
GlobalSearch Demo – Peaks Function
45
SIMULATED ANNEALING
46
What is Simulated Annealing?
A probabilistic metaheuristic
approach based upon the
physical process of annealing
in metallurgy.
Controlled cooling of a metal
allows atoms to realign from
a random higher energy state
to an ordered crystalline
(globally) lower energy state
47
Simulated Annealing Overview – Iteration 1Run from specified x0
x
y
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
0.9
48
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
Simulated Annealing Overview – Iteration 1
3
x
y 0.9
Possible New Points:
Standard Normal N(0,1) * Temperature
Temperature = 1
49
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
Simulated Annealing Overview – Iteration 1
3
x
y 0.9
Temperature = 1
11.01
1/)9.03(
Taccepte
P
50
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
Simulated Annealing Overview – Iteration 1
3
x
y 0.9
Temperature = 1
0.3
51
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
Simulated Annealing Overview – Iteration 1
3
x
y 0.9
Temperature = 1
0.3
52
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
Simulated Annealing Overview – Iteration 2
3
x
y 0.9
Temperature = 1
0.3
53
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
Simulated Annealing Overview – Iteration 2
3
x
y 0.9
Temperature = 0.75
0.3
-1.2
54
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
Simulated Annealing Overview – Iteration N-1
3
x
y 0.9
Temperature = 0.1
0.3
-1.2
-3
55
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
Simulated Annealing Overview – Iteration NReannealing
3
x
y 0.9
Temperature = 1
0.3
-1.2
-3
-2
56
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
Simulated Annealing Overview – Iteration NReannealing
3
x
y 0.9
Temperature = 1
0.3
-1.2
-3
-2
27.01
1/))3(2(
Taccepte
P
57
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
Simulated Annealing Overview – Iteration NReannealing
3
x
y 0.9
Temperature = 1
0.3
-1.2
-3
-2
58
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
Simulated Annealing Overview – Iteration N+1
3
x
y 0.9
Temperature = 0.75
0.3
-1.2
-3
-2
-3
59
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
Simulated Annealing Overview – Iteration N+1
3
x
y 0.9
Temperature = 0.75
0.3
-1.2
-3
-2
-3
60
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
Simulated Annealing Overview – Iteration …
3
x
y 0.9
Temperature = 0.75
0.3
-1.2
-3
-2
-3
-6.5
61
Simulated Annealing – Peaks Function
62
PATTERN SEARCH
(DIRECT SEARCH)
63
What is Pattern Search?
An approach that uses a
pattern of search directions
around the existing points
Expands/contracts around
the current point when a
solution is not found
Does not rely on gradients:
works on smooth and
nonsmooth problems
64
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
Pattern Search Overview – Iteration 1Run from specified x0
x
y
3
65
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
Pattern Search Overview – Iteration 1Apply pattern vector, poll new points for improvement
x
y
3
Mesh size = 1
Pattern vectors = [1,0], [0,1], [-1,0], [0,-1]
0_*_ xvectorpatternsizemeshPnew
0]0,1[*1 x1.6
0.4
4.6
2.8
First poll successful
Complete Poll (not default)
66
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
Pattern Search Overview – Iteration 2
x
y
3
Mesh size = 2
Pattern vectors = [1,0], [0,1], [-1,0], [0,-1]
1.6
0.4
4.6
2.8
-4
0.3-2.8
Complete Poll
67
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
Pattern Search Overview – Iteration 3
x
y
3
Mesh size = 4
Pattern vectors = [1,0], [0,1], [-1,0], [0,-1]
1.6
0.4
4.6
2.8
-4
0.3-2.8
68
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
Pattern Search Overview – Iteration 4
x
y
3
Mesh size = 4*0.5 = 2
Pattern vectors = [1,0], [0,1], [-1,0], [0,-1]
1.6
0.4
4.6
2.8
-4
0.3-2.8
69
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
Pattern Search Overview – Iteration NContinue expansion/contraction until convergence…
x
y
31.6
0.4
4.6
2.8
-4
0.3-2.8
-6.5
70
Pattern Search – Peaks Function
71
Pattern Search Climbs Mount Washington
72
PARTICLE SWARM
73
What is Particle Swarm Optimization?
A collection of particles
move throughout the region
Particles have velocity and
are affected by the other
particles in the swarm
Does not rely on gradients:
works on smooth and
nonsmooth problems
74
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
Particle Swarm Overview – Iteration 1Initialize particle locations and velocities, evaluate all
locations
x
y
75
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
Particle Swarm Overview – Iteration NUpdate velocities for each particle
x
yPrevious
Velocity
Best Location
for this Particle
Best Location for
Neighbor Particles
76
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
Particle Swarm Overview – Iteration NUpdate velocities for each particle
x
y
New
Velocity
77
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
Particle Swarm Overview – Iteration NMove particles based on new velocities
x
y
78
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
Particle Swarm Overview – Iteration NContinue swarming until convergence
x
y
79
Particle Swarm – Peaks Function
80
GENETIC ALGORITHM
81
What is a Genetic Algorithm?
Uses concepts from
evolutionary biology
Start with an initial generation
of candidate solutions that are
tested against the objective
function
Subsequent generations
evolve from the 1st through
selection, crossover and
mutation
82
How Evolution Works – Binary Case
Selection
– Retain the best performing bit strings from one generation to the next.
Favor these for reproduction
– parent1 = [ 1 0 1 0 0 1 1 0 0 0 ]
– parent2 = [ 1 0 0 1 0 0 1 0 1 0 ]
Crossover
– parent1 = [ 1 0 1 0 0 1 1 0 0 0 ]
– parent2 = [ 1 0 0 1 0 0 1 0 1 0 ]
– child = [ 1 0 0 0 0 1 1 0 1 0 ]
Mutation
– parent = [ 1 0 1 0 0 1 1 0 0 0 ]
– child = [ 0 1 0 1 0 1 0 0 0 1 ]
83
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
Genetic Algorithm – Iteration 1Evaluate initial population
x
y
84
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
Genetic Algorithm – Iteration 1Select a few good solutions for reproduction
x
y
85
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
Genetic Algorithm – Iteration 2Generate new population and evaluate
x
y
86
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
Genetic Algorithm – Iteration 2
x
y
87
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
Genetic Algorithm – Iteration 3
x
y
88
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
Genetic Algorithm – Iteration 3
x
y
89
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
Genetic Algorithm – Iteration NContinue process until stopping criteria are met
x
y
Solution found
90
Genetic Algorithm – Peaks Function
91
Genetic Algorithm – Integer Constraints
Mixed Integer Optimization
s.t. some x are integers
Examples
Only certain sizes of components
available
Can only purchase whole shares of stock
)(min xfx
92
Application: Circuit Component Selection
Thermistor Circuit 6 components to size
Only certain sizes
available
Objective:
– Match Voltage vs.
Temperature curve
300 Ω
330 Ω
360 Ω
…
180k Ω
200k Ω
220k Ω
Thermistors:
Resistance varies
nonlinearly with
temperature
?
𝑅𝑇𝐻 =𝑅𝑇𝐻,𝑁𝑜𝑚
𝑒𝛽𝑇−𝑇𝑁𝑜𝑚𝑇∗𝑇𝑁𝑜𝑚
93
Global Optimization Toolbox Solvers
GlobalSearch, MultiStart
– Well suited for smooth objective and constraints
– Return the location of local and global minima
ga, simulannealbnd
– Many function evaluations to sample the search space
– Work on both smooth and nonsmooth problems
patternsearch
– Fewer function evaluations than ga, simulannealbnd
– Does not rely on gradient calculation like GlobalSearch and
MultiStart
– Works on both smooth and nonsmooth problems
94
Speeding-up with Parallel Computing
Global Optimization Algorithms that support
Parallel Computing:– ga: Members of population evaluated in parallel at
each iteration
– patternsearch: Poll points evaluated in parallel at
each iteration
– MultiStart: Start points evaluated in parallel
Optimization Algorithms that support Parallel
Computing:– fmincon: parallel evaluation of objective function for
finite differences
– fminimax, fgoalattain: same as fmincon
Parallel Computing can also be used in the
Objective Function– parfor
95
Speed up parallel applications
Take advantage of GPUs
Prototype code for your cluster
Parallel Computing Toolbox for the Desktop
Simulink, Blocksets,
and Other Toolboxes
Local
Desktop Computer
MATLAB
……
96
Scale Up to Clusters and Clouds
Cluster
Scheduler
Computer Cluster
…
…
…
…
…
…
… … …
Simulink, Blocksets,
and Other Toolboxes
Local
Desktop Computer
MATLAB
……
97
Learn More about Optimization with MATLAB
MATLAB Digest: Using Symbolic
Gradients for Optimization
Recorded webinar: Optimization
in MATLAB: An Introduction to
Quadratic Programming
Optimization Toolbox Web demo:
Finding an Optimal Path using MATLAB
and Optimization Toolbox
MATLAB Digest: Improving
Optimization Performance with
Parallel Computing
98
Key Takeaways
Solve a wide variety of optimization problems in MATLAB
– Linear and Nonlinear
– Continuous and mixed-integer
– Smooth and Nonsmooth
Find better solutions to multiple minima and non-smooth problems
using global optimization
Use symbolic math for setting up problems and automatically
calculating gradients
Using parallel computing to speed up optimization problems
99© 2015 The MathWorks, Inc.
Questions?
top related