8/12/2019 cvxchap1

1/26

Convex Optimization

Chapter 1 Introduction

8/12/2019 cvxchap1

2/26

What, Why and How

What is convex optimization

Why study convex optimization

How to study convex optimization

8/12/2019 cvxchap1

3/26

8/12/2019 cvxchap1

4/26

Mathematical Optimization

Convex Optimization

Least-squares LP

Nonlinear Optimization

8/12/2019 cvxchap1

5/26

Mathematical Optimization

8/12/2019 cvxchap1

6/26

Convex Optimization

8/12/2019 cvxchap1

7/26

Least-squares

8/12/2019 cvxchap1

8/26

Analytical Solution of Least-squares

f0( x ) = j j A x bj j2

2= ( A x b)> ( A x b)

x = ( A>A ) 1A> b

@f0( x )

@x

= 2A> ( A x b) = 0

8/12/2019 cvxchap1

9/26

Linear Programming (LP)

8/12/2019 cvxchap1

10/26

Why Study Convex Optimization?

8/12/2019 cvxchap1

11/26

Mathematical Optimization

Convex Optimization

Least-squares LP

Solving Optimization Problems

Nonlinear Optimization

8/12/2019 cvxchap1

12/26

Analytical solution

Good algorithms and softwareHigh accuracy and high reliabilityTime complexity:

Mathematical Optimization

Convex Optimization

Least-squares LP

Nonlinear Optimization

knC 2

A mature technology!

8/12/2019 cvxchap1

13/26

No analytical solution

Algorithms and softwareReliable and efficientTime complexity:

Mathematical Optimization

Convex Optimization

Least-squares LP

Nonlinear Optimization

mnC 2

Also a mature technology!

8/12/2019 cvxchap1

14/26

8/12/2019 cvxchap1

15/26

Mathematical Optimization

Convex Optimization

Nonlinear Optimization

Far from a technology! (something to avoid)

Least-squares LP

Sadly, no effective methods to solveOnly approaches with some compromiseLocal optimization: more art than technology

Global optimization: greatly compromised efficiencyHelp from convex optimization

1) Initialization 2) Heuristics 3) Bounds

8/12/2019 cvxchap1

16/26

Why Study Convex Optimization

If not,

-- Section 1.3.2, p8, Convex Optimization

there is little chance you can solve it.

8/12/2019 cvxchap1

17/26

How to Study Convex Optimization?

8/12/2019 cvxchap1

18/26

Two Directions

As potential users of convex optimization

As researchers developing convexprogramming algorithms

8/12/2019 cvxchap1

19/26

Recognizing least-squares problems

Straightforward: verify

the objective to be a quadratic function

the quadratic form is positive semidefinite

Standard techniques increase flexibility Weighted least-squares

Regularized least-squares

8/12/2019 cvxchap1

20/26

Recognizing LP problems

Example: Sum of residuals approximation

Chebyshev or minimax approximation

t = maxij a>

ix b

ij

t i = j r ij

8/12/2019 cvxchap1

21/26

Recognizing Convex Optimization

Problems

8/12/2019 cvxchap1

22/26

An Example

8/12/2019 cvxchap1

23/26

8/12/2019 cvxchap1

24/26

8/12/2019 cvxchap1

25/26

8f j1; j

2; ; j

1 0g

P10k = 1

pj

k

12

Pmj = 1

pj

Adding linear constraints?????C10m

8/12/2019 cvxchap1

26/26

Summary

From the book, we expect to learn

To recognize convex optimization problems

To formulate convex optimization problems

To (know what can) solve them!