![Page 1: SCILAB WORKSHOP...approximation for pour Hessian π β’ Οπ pour control convergence Caracteristics β’ High memory need to store π: O(nΒ²) (O(n) in case of limited method)](https://reader035.vdocuments.mx/reader035/viewer/2022071603/613ecf65c500cf75ab36208b/html5/thumbnails/1.jpg)
1www.esi-group.com
Copyright Β© ESI Group, 2017. All rights reserved.Copyright Β© ESI Group, 2017. All rights reserved.
www.esi-group.com
SCILAB WORKSHOP
Optimization in SCILAB
SCILAB TEAM
17th, October 2019
![Page 2: SCILAB WORKSHOP...approximation for pour Hessian π β’ Οπ pour control convergence Caracteristics β’ High memory need to store π: O(nΒ²) (O(n) in case of limited method)](https://reader035.vdocuments.mx/reader035/viewer/2022071603/613ecf65c500cf75ab36208b/html5/thumbnails/2.jpg)
2www.esi-group.com
Copyright Β© ESI Group, 2017. All rights reserved.
Advanced post-processingAirfoil shape optimization
![Page 3: SCILAB WORKSHOP...approximation for pour Hessian π β’ Οπ pour control convergence Caracteristics β’ High memory need to store π: O(nΒ²) (O(n) in case of limited method)](https://reader035.vdocuments.mx/reader035/viewer/2022071603/613ecf65c500cf75ab36208b/html5/thumbnails/3.jpg)
3www.esi-group.com
Copyright Β© ESI Group, 2017. All rights reserved.
Quick introduction to Numerical Optimization
![Page 4: SCILAB WORKSHOP...approximation for pour Hessian π β’ Οπ pour control convergence Caracteristics β’ High memory need to store π: O(nΒ²) (O(n) in case of limited method)](https://reader035.vdocuments.mx/reader035/viewer/2022071603/613ecf65c500cf75ab36208b/html5/thumbnails/4.jpg)
4www.esi-group.com
Copyright Β© ESI Group, 2017. All rights reserved.
Optimization problem
maxπ₯βπ
π π₯
3 Steps:
1/ Identification of the decision variables: x
2/ Set-up of constraints associated to the variables: X
3/ Definition of a cost/objective function: f
![Page 5: SCILAB WORKSHOP...approximation for pour Hessian π β’ Οπ pour control convergence Caracteristics β’ High memory need to store π: O(nΒ²) (O(n) in case of limited method)](https://reader035.vdocuments.mx/reader035/viewer/2022071603/613ecf65c500cf75ab36208b/html5/thumbnails/5.jpg)
5www.esi-group.com
Copyright Β© ESI Group, 2017. All rights reserved.
Two formulations, One problem
Solving the problem
minπ₯βπ
π(π₯)
Corresponds to solve the problem
maxπ₯βπ
βπ π₯
Minimize or Maximize a function?
![Page 6: SCILAB WORKSHOP...approximation for pour Hessian π β’ Οπ pour control convergence Caracteristics β’ High memory need to store π: O(nΒ²) (O(n) in case of limited method)](https://reader035.vdocuments.mx/reader035/viewer/2022071603/613ecf65c500cf75ab36208b/html5/thumbnails/6.jpg)
6www.esi-group.com
Copyright Β© ESI Group, 2017. All rights reserved.
Numerical optimization
Initialization
Evaluation of cost
function
Choice of direction
New element
Optimal element
!
![Page 7: SCILAB WORKSHOP...approximation for pour Hessian π β’ Οπ pour control convergence Caracteristics β’ High memory need to store π: O(nΒ²) (O(n) in case of limited method)](https://reader035.vdocuments.mx/reader035/viewer/2022071603/613ecf65c500cf75ab36208b/html5/thumbnails/7.jpg)
7www.esi-group.com
Copyright Β© ESI Group, 2017. All rights reserved.
1/ Initial approximationβ Influence convergence
2/ Number of iterationsβ Recursive process
3/ Convergence speed
4/ Stopping criteria
β’ Maximum number of iterations, β¦
β’ Cost function value under a given threshold, β¦
π(π₯π) < π1
β’ Difference between two consecutive approximations under a giventhreshold, β¦
π₯π+1 β π₯π < π2
Classical set-up parameters for optimization solvers
![Page 8: SCILAB WORKSHOP...approximation for pour Hessian π β’ Οπ pour control convergence Caracteristics β’ High memory need to store π: O(nΒ²) (O(n) in case of limited method)](https://reader035.vdocuments.mx/reader035/viewer/2022071603/613ecf65c500cf75ab36208b/html5/thumbnails/8.jpg)
8www.esi-group.com
Copyright Β© ESI Group, 2017. All rights reserved.
General remarks
β’ Local extrema are easy to find!
β’ Some algorithm are performinglocal search only!
Take care on initial
approximation!
WARNING : Different types of extrema
![Page 9: SCILAB WORKSHOP...approximation for pour Hessian π β’ Οπ pour control convergence Caracteristics β’ High memory need to store π: O(nΒ²) (O(n) in case of limited method)](https://reader035.vdocuments.mx/reader035/viewer/2022071603/613ecf65c500cf75ab36208b/html5/thumbnails/9.jpg)
9www.esi-group.com
Copyright Β© ESI Group, 2017. All rights reserved.
Direct search
Search for an optimal position usingcost function evaluations, without anycomputation of the function exact orapproximate derivatives.
Interior Points method, simplex
Maximum search methods
Derivatives computation
Search for an optimal position usingcost function derivatives (Jacobian forsteepest ascent direction, Hessian forcritical point type) exact or approximatecomputation.
Gradient method, Newton method
How to make design space exploration?
![Page 10: SCILAB WORKSHOP...approximation for pour Hessian π β’ Οπ pour control convergence Caracteristics β’ High memory need to store π: O(nΒ²) (O(n) in case of limited method)](https://reader035.vdocuments.mx/reader035/viewer/2022071603/613ecf65c500cf75ab36208b/html5/thumbnails/10.jpg)
10www.esi-group.com
Copyright Β© ESI Group, 2017. All rights reserved.
Optimization problem typology
![Page 11: SCILAB WORKSHOP...approximation for pour Hessian π β’ Οπ pour control convergence Caracteristics β’ High memory need to store π: O(nΒ²) (O(n) in case of limited method)](https://reader035.vdocuments.mx/reader035/viewer/2022071603/613ecf65c500cf75ab36208b/html5/thumbnails/11.jpg)
11www.esi-group.com
Copyright Β© ESI Group, 2017. All rights reserved.
Travelling salesman problem
Continuous vs Discrete
DISCRETE (Combinatorial Analysis)
Variables are taking values in a finite setor states (IN, ON/OFF, β¦).
Classical examples: Travelling salesmanproblem which consists in finding theshortest path binding a given set of cities.
![Page 12: SCILAB WORKSHOP...approximation for pour Hessian π β’ Οπ pour control convergence Caracteristics β’ High memory need to store π: O(nΒ²) (O(n) in case of limited method)](https://reader035.vdocuments.mx/reader035/viewer/2022071603/613ecf65c500cf75ab36208b/html5/thumbnails/12.jpg)
12www.esi-group.com
Copyright Β© ESI Group, 2017. All rights reserved.
Mono vs Multi objective
MULTI OBJECTIVE
minπ₯βπ
(π1 π₯ , π2 π₯ ,β¦ , ππ π₯ )
Several cost, potentially conflicting,functions minimization.
Found solution is therefore not unique.The complete set of solutions is calledPareto Front.
Choice is about trade-offAn example would be to ensuremaximum efficiency, and minimuminvestment at the same time.
![Page 13: SCILAB WORKSHOP...approximation for pour Hessian π β’ Οπ pour control convergence Caracteristics β’ High memory need to store π: O(nΒ²) (O(n) in case of limited method)](https://reader035.vdocuments.mx/reader035/viewer/2022071603/613ecf65c500cf75ab36208b/html5/thumbnails/13.jpg)
13www.esi-group.com
Copyright Β© ESI Group, 2017. All rights reserved.
Constrained vs Non Constrained
CONSTRAINTS
Constraints are referring to the limit of the design space we impose
β’ Linear
πΆπ₯ = π
β’ Non Linear
π₯ππΆπ₯ = π
β’ EqualitiesπΆπ₯ = π
β’ InequalitiesπΆπ₯ β₯ π
β’ Lower and Upper boundsπ1 β€ π₯ β€ π2
![Page 14: SCILAB WORKSHOP...approximation for pour Hessian π β’ Οπ pour control convergence Caracteristics β’ High memory need to store π: O(nΒ²) (O(n) in case of limited method)](https://reader035.vdocuments.mx/reader035/viewer/2022071603/613ecf65c500cf75ab36208b/html5/thumbnails/14.jpg)
14www.esi-group.com
Copyright Β© ESI Group, 2017. All rights reserved.
Problem size
Let n be the number of design variables
β’ n < 10 : Small problem (s)
β’ 10 < n < 100 : Medium problem (m)
β’ 100 < n < 1000 : Large problem (l)
Solver speed and memory needsare depending on the problem size
![Page 15: SCILAB WORKSHOP...approximation for pour Hessian π β’ Οπ pour control convergence Caracteristics β’ High memory need to store π: O(nΒ²) (O(n) in case of limited method)](https://reader035.vdocuments.mx/reader035/viewer/2022071603/613ecf65c500cf75ab36208b/html5/thumbnails/15.jpg)
15www.esi-group.com
Copyright Β© ESI Group, 2017. All rights reserved.
Optimization solvers in SCILAB
![Page 16: SCILAB WORKSHOP...approximation for pour Hessian π β’ Οπ pour control convergence Caracteristics β’ High memory need to store π: O(nΒ²) (O(n) in case of limited method)](https://reader035.vdocuments.mx/reader035/viewer/2022071603/613ecf65c500cf75ab36208b/html5/thumbnails/16.jpg)
16www.esi-group.com
Copyright Β© ESI Group, 2017. All rights reserved.
How to choose SCILAB optimization solver
Convex
SemidefiniteLeast squares
Quadratic
Linear
Objective Constraints
[Bnds, Eq, Ineq]
Size Gradient Solver
Linear Y Y Y m karmarkar
Quadratic Y Y Y L qpsolve
SemiDef N Y Y L lmisolver
SemiDef Y N Y L semidef
N.L.S. N N N L optional Leastsq /
lsqrsolve
Non-linear Y N N L Y optim
Non-linear N N N s fminsearch
Objective Single
Objective
Multi
Objective
Use
Continuous/
Differentiable
Fminsearch
Non Smooth optim
Discrete Optim_sa,
optim_ga
Optim_moga
Optim_nsga2
![Page 17: SCILAB WORKSHOP...approximation for pour Hessian π β’ Οπ pour control convergence Caracteristics β’ High memory need to store π: O(nΒ²) (O(n) in case of limited method)](https://reader035.vdocuments.mx/reader035/viewer/2022071603/613ecf65c500cf75ab36208b/html5/thumbnails/17.jpg)
17www.esi-group.com
Copyright Β© ESI Group, 2017. All rights reserved.
Linear problemminπ₯
π π₯ = πππ₯
Linear constraints
β’ EqualitiesπΆ1ππ₯ = π1
β’ InequalitiesπΆ2ππ₯ β₯ π2
Linear optimization
![Page 18: SCILAB WORKSHOP...approximation for pour Hessian π β’ Οπ pour control convergence Caracteristics β’ High memory need to store π: O(nΒ²) (O(n) in case of limited method)](https://reader035.vdocuments.mx/reader035/viewer/2022071603/613ecf65c500cf75ab36208b/html5/thumbnails/18.jpg)
18www.esi-group.com
Copyright Β© ESI Group, 2017. All rights reserved.
Interior Points Method
Β« Affine Scaling Β»
Solve suite of optimization problems onellipsoid.
Ellipsoids size are decreasing at eachiteration by a given scale factor (between 0and 1).
Caracteristic:
β’ Polynomial time O(ππ)
β’ Direct search
β’ (In)Equalities and bounds constraints
Linear optimization - karmarkar
![Page 19: SCILAB WORKSHOP...approximation for pour Hessian π β’ Οπ pour control convergence Caracteristics β’ High memory need to store π: O(nΒ²) (O(n) in case of limited method)](https://reader035.vdocuments.mx/reader035/viewer/2022071603/613ecf65c500cf75ab36208b/html5/thumbnails/19.jpg)
19www.esi-group.com
Copyright Β© ESI Group, 2017. All rights reserved.
Quadratic problem
minxπ π₯ =
1
2π₯πππ₯ + πππ₯
With Q la Β« hessian Β» and c Β« gradient Β»
of a virtual energy function
Linear constraints
β’ EqualitiesπΆ1ππ₯ = π1
β’ InequalitiesπΆ2ππ₯ β₯ π2
Quadratic optimization
Concave shape, convex problem
![Page 20: SCILAB WORKSHOP...approximation for pour Hessian π β’ Οπ pour control convergence Caracteristics β’ High memory need to store π: O(nΒ²) (O(n) in case of limited method)](https://reader035.vdocuments.mx/reader035/viewer/2022071603/613ecf65c500cf75ab36208b/html5/thumbnails/20.jpg)
20www.esi-group.com
Copyright Β© ESI Group, 2017. All rights reserved.
Quadratic optimization - qpsolve
Goldfarb-Idnani solver
Β« Feasible active set method Β»
Find non-constrained global optimum,then add non-respected constraints oneafter the other (activation)
Caracteristics :
β’ Strictly convex problems
β’ (In)Equalities and bounds constraints
β’ Polynomial time O(ππ)
β’ Direct search
![Page 21: SCILAB WORKSHOP...approximation for pour Hessian π β’ Οπ pour control convergence Caracteristics β’ High memory need to store π: O(nΒ²) (O(n) in case of limited method)](https://reader035.vdocuments.mx/reader035/viewer/2022071603/613ecf65c500cf75ab36208b/html5/thumbnails/21.jpg)
21www.esi-group.com
Copyright Β© ESI Group, 2017. All rights reserved.
Least squares problem
minπ₯
π π₯ 2
π: πΌπ π β πΌπ π
π number of unknonw, π nb of points
π non necessarily linear or differentiable
Linear regression (QUADRATIC)
Minimize distance between measurements (π = 2) and an unknown line (π = 1)
1
2ππ₯ β π 2 =
1
2(π₯πππππ₯ β 2π₯ππππ + πππ)
Non Linear Least Squares
![Page 22: SCILAB WORKSHOP...approximation for pour Hessian π β’ Οπ pour control convergence Caracteristics β’ High memory need to store π: O(nΒ²) (O(n) in case of limited method)](https://reader035.vdocuments.mx/reader035/viewer/2022071603/613ecf65c500cf75ab36208b/html5/thumbnails/22.jpg)
22www.esi-group.com
Copyright Β© ESI Group, 2017. All rights reserved.
Non Linear Least Squares - lsqrsolve
Levenberg-Marquardt algorithm
To find a new direction:
β’ Gradient method (green) to provide steepestdescent direction. Efficient when far from theoptimal position.
β’ Gauss-Newton (red), approximate the problem aslocally quadratic and solve it to find new position.Efficient when close to the optimal position.
Caracteristics
β’ Polynomial time
![Page 23: SCILAB WORKSHOP...approximation for pour Hessian π β’ Οπ pour control convergence Caracteristics β’ High memory need to store π: O(nΒ²) (O(n) in case of limited method)](https://reader035.vdocuments.mx/reader035/viewer/2022071603/613ecf65c500cf75ab36208b/html5/thumbnails/23.jpg)
23www.esi-group.com
Copyright Β© ESI Group, 2017. All rights reserved.
Non Linear optimization - fminsearch
Nelder-Mead Simplex
Simplex manipulation
Caracteristics
β’ Non constrained
β’ High complexityBetter suited for small problem
β’ Direct search
3D Simplex
![Page 24: SCILAB WORKSHOP...approximation for pour Hessian π β’ Οπ pour control convergence Caracteristics β’ High memory need to store π: O(nΒ²) (O(n) in case of limited method)](https://reader035.vdocuments.mx/reader035/viewer/2022071603/613ecf65c500cf75ab36208b/html5/thumbnails/24.jpg)
24www.esi-group.com
Copyright Β© ESI Group, 2017. All rights reserved.
Quasi-Newton method
Quasi-Newton statementπ₯π+1 β π₯π = π΅π π π₯π+1 β π π₯π
β π₯π+1 β π₯π β Οππ΅ππ(π₯π)
β’ Broyden-Fletcher-Goldfarbapproximation for pour Hessian π΅π
β’ Οπ pour control convergence
Caracteristics
β’ High memory need to store π΅π : O(nΒ²)(O(n) in case of limited method)
β’ Gradient info must be provided
β’ Polinomial time O(nΒ²) but not accurate
Non Linear optimization - optim
Nota
β’ Fsolve (alternative)
β’ Datafit / leastsq (built upon optim)
![Page 25: SCILAB WORKSHOP...approximation for pour Hessian π β’ Οπ pour control convergence Caracteristics β’ High memory need to store π: O(nΒ²) (O(n) in case of limited method)](https://reader035.vdocuments.mx/reader035/viewer/2022071603/613ecf65c500cf75ab36208b/html5/thumbnails/25.jpg)
25www.esi-group.com
Copyright Β© ESI Group, 2017. All rights reserved.
GA Theory
- Natural selection -
Probabilistics and non deterministics transitons:
Selection, Cross-over et Mutation
Caracteristics
β’ Bounds constraints
β’ Non Linear, Non Convex
Remarks
optim_ga /optim_moga: Single/Multi objective
pareto_filter: Non-dominated sorting
Genetic Algorithms
![Page 26: SCILAB WORKSHOP...approximation for pour Hessian π β’ Οπ pour control convergence Caracteristics β’ High memory need to store π: O(nΒ²) (O(n) in case of limited method)](https://reader035.vdocuments.mx/reader035/viewer/2022071603/613ecf65c500cf75ab36208b/html5/thumbnails/26.jpg)
26www.esi-group.com
Copyright Β© ESI Group, 2017. All rights reserved.
Simulated Annealing
Simulated Annealing optim_sa
Empirical method based on metallurgicalprocess
Alternate slow cooling cycles with heatingcycles (annealing) in order to minimize materialinternal energy (strongest configuration)
Metropolis-Hastings Algorithm
Starting at a given state, we modify systemtowards another state. Either it get better(energy decrease) or it get worse.
Drawbacks:
β’ Empirical set-up
β’ Bounds constraints only
Benefits:
β’ Global optimum
β’ Discrete optimization