Lecture 0 0-1

ECE733

Nonlinear Optimization for Electrical Engineers

Dr. Mohamed Bakr,

905 525 9140 x24079

mbakr@mail.ece.mcmaster.ca

Lecture 0 0-2

Info About Myself

B.Sc. in Electronics and Communication Engineering, Cairo

University, Cairo, Egypt with Distinction (honors), 1992

M.Sc. in Engineering Mathematics (Optimization), Cairo

University, 1996

Ph.D. in Computer Aided Design (CAD) of Microwave Circuits,

McMaster University, 2000

P.Eng., Ontario, 2003

Full professor, 2013

Author/CoAuthor of over 200 journal and conference papers,

one book, two book chapters, and two patents

Lecture 0 0-3

Info About Myself (Contd)

Research Areas: Optimization methods, computer-aided design

and modeling of microwave circuits, neural networks

applications, computational electromagnetics, and nanophotonics

Awards/Scholarships:

TRIO Student Internship in OSA, inc. 1997

Ontario Graduate Scholarship (OGS) 1998-2000,

NSERC PostDoctoral Fellowship 2000-2001,

Premiers Research Excellence Award (PREA) 2003-2009

McMaster Tenure 2007

Sabbatical Leave with RIM (2008-2009)

NSERC Accelerator Supplement Award (DAS), 2011

Supervisor/Co-supervisor to a number of graduate students

Lecture 0 0-4

Teaching Experience

Teaching Assistant in Engineering Mathematics (Cairo

University), 1992-1996

Teaching Assistant in Electrical Engineering (McMaster

University) 1996-1999

Assistant Professor in the Department of Electrical and

Computer Engineering, McMaster University 2002-2007:

ECE 750 Advanced Engineering Electromagnetics

ECE 2EI4 Electronic Devices and Circuits

ECE 3TP4 Signals and Systems

ECE 757 Numerical Techniques in Electromagnetics

ECE 2EI5 Electronic Devices and Circuits

ECE 3FI4 Theory and Applications in Electromagnetics

Lecture 0 0-5

Teaching Experience (Contd)

ECE 2FH3 Electromagnetics I

ECE 2CI5 Introduction To Electrical Engineering

ECE 3FK4 Electromagnetics II

ECE 4OI6 Engineering Design

ECE 718 Nonlinear Optimization

Developer of a number of coursewares for several courses

Lecture 0 0-6

Course Overview

1-Introduction To Vector Analysis and Optimization

Introductory mathematical tools

Historical background

Jargon of optimization problems and their classifications

Lecture 0 0-7

Course Overview (Contd)

2-Classical Optimization Approaches

Single-variable methods

Multi-variable methods

The KKT conditions for equality and inequality constraints

Lecture 0 0-8

Course Overview (Contd)

3-One Dimensional Search Techniques

Why one-dimensional search is so important?

Derivative-free methods

Gradient and Hessian based methods

Lecture 0 0-9

Course Overview (Contd)

4-Unconstrained Optimization

Derivative-free approaches

Gradient-based techniques

Second-order methods

Lecture 0 0-10

Course Overview (Contd)

5-Constrained Optimization

Quadratic programming

Sequential quadratic programming

Penalty methods

Gradient projection methods

Methods of feasible direction

Lecture 0 0-11

Course Overview (Contd)

6. Global Optimization Methods

Old population New population

Simulated annealing

Genetic algorithms

Particle swarm optimization

Lecture 0 0-12

Course Overview (Contd)

7. Space Mapping Optimization

Aggressive space mapping

Trust region space mapping

Implicit space mapping

Surrogate-based space mapping

fine

model

space

mapping

responses

surrogate

input

mapping

design

parameters responses

coarse

model

space

mappingspace

mapping

output

mapping

implicit

mapping

Lecture 0 0-13

Course Overview (Contd)

8. Adjoint Sensitivities and Their Applications

Using only at most one extra simulation, the sensitivities of the

response with respect to all design parameters are obtained

This makes gradient-based optimization far more efficient

Adjoint

Simulation x R R Original

Simulation

x

Lecture 0 0-14

Course Overview (Contd)

Text: Engineering Optimization Theory and Practice, Singiresu

S. Rao, Third Edition

or

Text: Nonlinear Optimization in Electrical Engineering with

Applications in MATLAB, Mohamed H. Bakr, IET Press, 2013

CLASSES: TBD

Course Webpage:

http://www.ece.mcmaster.ca/faculty/bakr/

ECE733/ECE733_Main_2014.htm

4 Matlab assignments and one final project are required

Lecture 0 0-15

Detailed Course Outline

Date Lecture Description

0 Course Outline

1 Introduction: Historical Background, statement of

optimization problem

2 Introduction: Classifications of Optimization problems

3 Classical Optimization Methods: single variable

optimization, unconstrained multivariate optimization

4 Equality Constraints: Solution by Direct substitution,

Method of constrained variation

5 Equality Constriants: Method of Lagrange multipliers

6 Inequality constraints: Kuhn-Tucker Conditions,

Constraint qualification

7

One Dimensional Search: why one dimensional search?,

Search with Fixed Step Size, Search with Accelerated Step

size

8

One Dimensional Search: Interval halving Method,

Fibonacci Method, Golden Section Search

Lecture 0 0-16

9 One Dimensional Search: Interpolation Methods, Newton

Method

10 One Dimensional Search: Quasi-Newton Method, Secant

Method, Practical Consideration

11 Unconstrained Nonlinear Optimization: Introduction and

basic concepts

12 Direct Search Methods: Random Walks, Grid Search,

Univariate Method, Simplex Method

13 Conjugate Gradient Methods: Powells Method, Conjugate Directions

14 Indirect Methods: Steepest Descent, Conjugate Gradients

15 2nd Order Methods: Newton Method, Marquardt Method,

and Quasi Newton Methods

16 2nd Order Methods (Contd): The DFP formula, the BFGS formula, summary

17 Constrained Nonlinear Optimization: Introduction and

basic concepts

18 Some Constrained Optimization Methods: Zoutendijks method of feasible directions

19 Constrained Optimization (Contd): Rosens Gradient

Lecture 0 0-17

projection Method, sequential quadratic programming

20 Constrained Optimization (Contd): Penalty Methods

21 Global Optimization Techniques: Genetic Algorithms

22 Global Optimization Techniques (Contd): Simulated

annealing

23 Global Optimization Techniques(Contd): Particle Swarm Optimization

24

Space Mapping Optimization and Modelling: Basic

Concepts, classical Space Mapping, Aggressive Space

Mapping

25 Space Mapping (Contd): surrogate-based optimization,

Output Space Mapping

26 Adjoint Variable Methods: The Frequency Domain Case

27 Adjoint Variable Methods: The Dynamic Case

28 Areas of Research in Optimization

Lecture 0 0-18

General Comments

Lecture is divided into two parts each for about 1.0 Hr to

1.25 Hr. We will have a break in the middle

We will not focus on theorem proving. We will give a proof

as long as it is concise and useful

Engineering Applications will be given as much as possible

We will write all our optimization code. Ready functions in

packages will only be used for comparison

Material will be posted on the course webpage the day

before. Copy only examples not in the slides.