zeit4700 – s1, 2015 mathematical modeling and optimization school of engineering and information...
DESCRIPTION
Classical optimization techniques Bracketing (Exhaustive search) / Region elimination methods Simplex based search Gradient based search Nelder Mead simplex method (Image source : Interval halving method (Image source : K.Deb, Optimization for engineering design)TRANSCRIPT
![Page 1: ZEIT4700 – S1, 2015 Mathematical Modeling and Optimization School of Engineering and Information Technology](https://reader036.vdocuments.mx/reader036/viewer/2022082723/5a4d1af37f8b9ab05997f933/html5/thumbnails/1.jpg)
ZEIT4700 – S1, 2015Mathematical Modeling and Optimization
School of Engineering and Information Technology
![Page 2: ZEIT4700 – S1, 2015 Mathematical Modeling and Optimization School of Engineering and Information Technology](https://reader036.vdocuments.mx/reader036/viewer/2022082723/5a4d1af37f8b9ab05997f933/html5/thumbnails/2.jpg)
Optimization - basics
Maximization or minimization of given objective function(s), possibly subject to constraints, in a given search space
Minimize f1(x), . . . , fk(x) (objectives) Subject to gj(x) < 0, i = 1, . . . ,m (inequality constraints) hj(x) = 0, j = 1, . . . , p (equality constraints)
Xmin1 ≤ x1 ≤ Xmax1 (variable / search space)Xmin2 ≤ x2 ≤ Xmax2
. .
![Page 3: ZEIT4700 – S1, 2015 Mathematical Modeling and Optimization School of Engineering and Information Technology](https://reader036.vdocuments.mx/reader036/viewer/2022082723/5a4d1af37f8b9ab05997f933/html5/thumbnails/3.jpg)
Classical optimization techniques Bracketing (Exhaustive search) / Region elimination methods Simplex based search Gradient based search
Nelder Mead simplex method(Image source : http://upload.wikimedia.org/wikipedia/commons/9/96/Nelder_Mead2.gif)
Interval halving method(Image source : K.Deb, Optimization for engineering design)
![Page 4: ZEIT4700 – S1, 2015 Mathematical Modeling and Optimization School of Engineering and Information Technology](https://reader036.vdocuments.mx/reader036/viewer/2022082723/5a4d1af37f8b9ab05997f933/html5/thumbnails/4.jpg)
Classical optimization techniques (cntd.)Gradient based (Cauchy’s steepest descent method)
Image source : K. Deb, Multi-objective optimization using Evolutionary Algorithms, John Wiley and Sons, 2002.
![Page 5: ZEIT4700 – S1, 2015 Mathematical Modeling and Optimization School of Engineering and Information Technology](https://reader036.vdocuments.mx/reader036/viewer/2022082723/5a4d1af37f8b9ab05997f933/html5/thumbnails/5.jpg)
Drawbacks of classical optimization methods
• Gradient based methods : Assumptions on continuity / derivability
• Region elimination methods : For single variable problem only
• Search for local optimum only
• Constraint handling is not inherently included
• No provisions for handling multiple objectives
![Page 6: ZEIT4700 – S1, 2015 Mathematical Modeling and Optimization School of Engineering and Information Technology](https://reader036.vdocuments.mx/reader036/viewer/2022082723/5a4d1af37f8b9ab05997f933/html5/thumbnails/6.jpg)
Optimization – Heuristics/meta-heuristics
A heuristic is a technique which seeks good (i.e., near optimal) solutions at a reasonable computational cost without being able to guarantee either feasibility or optimality, or even in many cases to state how close to optimality a particular feasible solution is. - Reeves, C.R.: Modern Heuristic Techniques for Combinatorial Problems. Orient Longman (1993)
![Page 7: ZEIT4700 – S1, 2015 Mathematical Modeling and Optimization School of Engineering and Information Technology](https://reader036.vdocuments.mx/reader036/viewer/2022082723/5a4d1af37f8b9ab05997f933/html5/thumbnails/7.jpg)
Simple “Hill climb”Start from random X
(while termination criterion not met){Perturb X to get a new point X’If F(X’) > F(X), move to X’, else not}
Maximize f(x)
X X’
F(x)
X X’• “Greedy”• Local
![Page 8: ZEIT4700 – S1, 2015 Mathematical Modeling and Optimization School of Engineering and Information Technology](https://reader036.vdocuments.mx/reader036/viewer/2022082723/5a4d1af37f8b9ab05997f933/html5/thumbnails/8.jpg)
Simulated AnnealingStart from random X
(while termination criterion not met){Perturb X to get a new point X’ If F(X’) > F(X), move to X’, else Calculate P = exp(-(F(X) – F(X’))/T) move to X’ with probability P}
Maximize f(x)
X X’
F(x)
X X’
Attempts to escape local minimaby accepting occasional ‘worse’ moves
![Page 9: ZEIT4700 – S1, 2015 Mathematical Modeling and Optimization School of Engineering and Information Technology](https://reader036.vdocuments.mx/reader036/viewer/2022082723/5a4d1af37f8b9ab05997f933/html5/thumbnails/9.jpg)
Genetic / Evolutionary algorithmsFrom point-to-point methods to population based methods..
• EAs are nature inspired optimization methods which search for the optimum solution(s) by evolving a population of solutions.
• Require no assumptions on differentiability / continuity of functions, hence can handle much more complex functions as compared to classical optimization techniques.
• Can deliver the whole Pareto Optimal Front in a single run as opposed to conventional methods.
• Its an Intelligent hit and trial !
![Page 10: ZEIT4700 – S1, 2015 Mathematical Modeling and Optimization School of Engineering and Information Technology](https://reader036.vdocuments.mx/reader036/viewer/2022082723/5a4d1af37f8b9ab05997f933/html5/thumbnails/10.jpg)
Evolutionary Algorithms (EA)
Initialization (population of
solutions)Parent selection Recombination /
Crossover
Mutation
Ranking (parent+child pop)Reduction
Termination criterion met
? Yes
No
Output best solution obtained
“Evolve”childpop
Evaluate childpop
![Page 11: ZEIT4700 – S1, 2015 Mathematical Modeling and Optimization School of Engineering and Information Technology](https://reader036.vdocuments.mx/reader036/viewer/2022082723/5a4d1af37f8b9ab05997f933/html5/thumbnails/11.jpg)
Gen 1 Gen 25
Gen 50 Gen 100
Evolutionary Algorithms (contd.)
![Page 12: ZEIT4700 – S1, 2015 Mathematical Modeling and Optimization School of Engineering and Information Technology](https://reader036.vdocuments.mx/reader036/viewer/2022082723/5a4d1af37f8b9ab05997f933/html5/thumbnails/12.jpg)
Evolutionary Algorithm (cntd)
Minimize f(x) = (x-6)^20 ≤ x ≤ 31
Binary GA Real Parameter GA
Representation Binary RealParent selection Binary
tournatment/Roulett wheel
Binary tournatment/Roulett wheel
Crossover One point/multi-point
SBX,PCX …
Mutation Binary flip Polynomial
![Page 13: ZEIT4700 – S1, 2015 Mathematical Modeling and Optimization School of Engineering and Information Technology](https://reader036.vdocuments.mx/reader036/viewer/2022082723/5a4d1af37f8b9ab05997f933/html5/thumbnails/13.jpg)
Resources
Course material and suggested reading can be accessed at http://seit.unsw.adfa.edu.au/research/sites/mdo/Hemant/design-2.html