c&o 355 mathematical programming fall 2010 lecture 1 n. harvey texpoint fonts used in emf. read...
TRANSCRIPT
![Page 2: C&O 355 Mathematical Programming Fall 2010 Lecture 1 N. Harvey TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AA A A](https://reader036.vdocuments.mx/reader036/viewer/2022070407/56649e255503460f94b146d5/html5/thumbnails/2.jpg)
What is Optimization?
• Basic Idea: From a certain set of objects, choose the best one.
• Example: Getting married– “objects” = { all possible spouses }– “best one”… this is very hard to make precise
![Page 3: C&O 355 Mathematical Programming Fall 2010 Lecture 1 N. Harvey TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AA A A](https://reader036.vdocuments.mx/reader036/viewer/2022070407/56649e255503460f94b146d5/html5/thumbnails/3.jpg)
What is Mathematical Optimization?• Additional Idea: Describe objects by vectors in Rn
• Example:
• Each possible spouse described by a vector in R4
• Best could mean “highest IQ” or “fewest spouses”, etc.
Age 32 30 67 93
Prev Spouses 1 1 0 9
IQ 152 200 109 122
Eye Color 1 2 1 3
(Using the Encoding: 1 = Brown, 2 = Blue, 3 = Green)
![Page 4: C&O 355 Mathematical Programming Fall 2010 Lecture 1 N. Harvey TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AA A A](https://reader036.vdocuments.mx/reader036/viewer/2022070407/56649e255503460f94b146d5/html5/thumbnails/4.jpg)
What is Mathematical Optimization?• Additional Idea: Describe objects by vectors in Rn
• Example:
• We can formalize the “highest IQ spouse” problem as:
Age 32 30 67 93
Prev Spouses 1 1 0 9
IQ 152 200 109 122
Eye Color 1 2 1 3
![Page 5: C&O 355 Mathematical Programming Fall 2010 Lecture 1 N. Harvey TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AA A A](https://reader036.vdocuments.mx/reader036/viewer/2022070407/56649e255503460f94b146d5/html5/thumbnails/5.jpg)
A Continuous Problem• Often there are infinitely many objects from
which we want to find the best one.• Example: Allocating assets– x1 = Dollar value of my Walmart shares
– x2 = Dollar value of my US treasury bonds
Here f : R2 ! R is some function that measures “risk”
![Page 6: C&O 355 Mathematical Programming Fall 2010 Lecture 1 N. Harvey TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AA A A](https://reader036.vdocuments.mx/reader036/viewer/2022070407/56649e255503460f94b146d5/html5/thumbnails/6.jpg)
• Definition: A mathematical program is an optimization problem that looks like this:
where x2Rn, f : Rn ! R, gi : Rn ! R, bi 2 R
objective function
constraints
could be ¸ or · or =
• Definitions:• A point satisfying all constraints is called a feasible solution.• A feasible point x achieving the minimum value
of the objective function is called an optimal solution.
![Page 7: C&O 355 Mathematical Programming Fall 2010 Lecture 1 N. Harvey TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AA A A](https://reader036.vdocuments.mx/reader036/viewer/2022070407/56649e255503460f94b146d5/html5/thumbnails/7.jpg)
• Restrictions:– For arbitrary f and gi’s, this is totally hopeless.
– We assume f and gi’s are “nice”.
• Definition: A mathematical program is an optimization problem that looks like this:
where x2Rn, f : Rn ! R, gi : Rn ! R, bi 2 R
objective function
constraints
could be ¸ or · or =
• Goals:– Find an optimal solution, by some algorithm.
Typically no closed form-expression.– Find “optimality conditions”: necessary and sufficient
conditions for x to be optimal.
![Page 8: C&O 355 Mathematical Programming Fall 2010 Lecture 1 N. Harvey TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AA A A](https://reader036.vdocuments.mx/reader036/viewer/2022070407/56649e255503460f94b146d5/html5/thumbnails/8.jpg)
Simple Example: Linear Equations• Consider the system of linear equations Ax = b• Can write this as:
where ai is the i’th row of A.• This is a very simple mathematical program:
where f(x) = 0, and gi(x) = ai ¢ x.
(the objective function is irrelevant / trivial)
![Page 9: C&O 355 Mathematical Programming Fall 2010 Lecture 1 N. Harvey TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AA A A](https://reader036.vdocuments.mx/reader036/viewer/2022070407/56649e255503460f94b146d5/html5/thumbnails/9.jpg)
x2
• Develop a rigorous and useful theory for mathematical programs.
• Geometric View:
• Algorithms:Gaussian Elimination,Conjugate Gradient…
x3
Purpose of CO 355
x1
x2
a1Tx = b1
a2Tx = b2
Linear Systems
Intersection of Hyperplanes
x3
x1
g1(x) ¸ b1
Mathematical Programs
Intersection of More General Sets
g2(x) · b2
ObjectiveFunction
Simplex Method, Ellipsoid Method,Interior Point Methods…
SolutionsOptimalSolution
![Page 10: C&O 355 Mathematical Programming Fall 2010 Lecture 1 N. Harvey TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AA A A](https://reader036.vdocuments.mx/reader036/viewer/2022070407/56649e255503460f94b146d5/html5/thumbnails/10.jpg)
Why care?
OperationsResearch
MathProgramming
• O.R.: applying mathematical modeling and analysis to business decision making.
![Page 11: C&O 355 Mathematical Programming Fall 2010 Lecture 1 N. Harvey TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AA A A](https://reader036.vdocuments.mx/reader036/viewer/2022070407/56649e255503460f94b146d5/html5/thumbnails/11.jpg)
Why care?
OperationsResearch
• Math Programming & O.R. were jointly developed during World War II.– Invention of Linear Programming
• Dantzig almost won Nobel Prize in EconomicsGeorge Dantzig1914-2005
Economics
MathProgramming
![Page 12: C&O 355 Mathematical Programming Fall 2010 Lecture 1 N. Harvey TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AA A A](https://reader036.vdocuments.mx/reader036/viewer/2022070407/56649e255503460f94b146d5/html5/thumbnails/12.jpg)
Why care?
OperationsResearch
Game Theory
Economics
MathProgramming
• At same time: the birth of game theory– Zero sum games
(closely connected to linear programming)
![Page 13: C&O 355 Mathematical Programming Fall 2010 Lecture 1 N. Harvey TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AA A A](https://reader036.vdocuments.mx/reader036/viewer/2022070407/56649e255503460f94b146d5/html5/thumbnails/13.jpg)
Why care?
OperationsResearch
Game Theory
Economics
ConvexGeometry
• ~1900: theory of polytopes (closely connected to linear programming)
H. MinkowskiG. Farkas
FunctionalAnalysisMath
Programming
![Page 14: C&O 355 Mathematical Programming Fall 2010 Lecture 1 N. Harvey TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AA A A](https://reader036.vdocuments.mx/reader036/viewer/2022070407/56649e255503460f94b146d5/html5/thumbnails/14.jpg)
Why care?
OperationsResearch
Game Theory
Economics
FunctionalAnalysis
ConvexGeometry
Combinatorics
MathProgramming
• Many theorems in combinatorics canbe proven or understood better using mathematical programming
![Page 15: C&O 355 Mathematical Programming Fall 2010 Lecture 1 N. Harvey TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AA A A](https://reader036.vdocuments.mx/reader036/viewer/2022070407/56649e255503460f94b146d5/html5/thumbnails/15.jpg)
Why care?
OperationsResearch
Game Theory
Economics
FunctionalAnalysis
MachineLearning
ConvexGeometry
Combinatorics
MathProgramming
• Machine Learning= Statistics + Optimization
![Page 16: C&O 355 Mathematical Programming Fall 2010 Lecture 1 N. Harvey TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AA A A](https://reader036.vdocuments.mx/reader036/viewer/2022070407/56649e255503460f94b146d5/html5/thumbnails/16.jpg)
Why care?
OperationsResearch
Game Theory
Economics
FunctionalAnalysis
MachineLearning
ConvexGeometry
ComputerScience
Combinatorics
MathProgramming
• Many powerful theoretical algorithms are based on mathematical programming.(max flow, max cut, sparsest cut, traveling salesman, …)
![Page 17: C&O 355 Mathematical Programming Fall 2010 Lecture 1 N. Harvey TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AA A A](https://reader036.vdocuments.mx/reader036/viewer/2022070407/56649e255503460f94b146d5/html5/thumbnails/17.jpg)
Why care?
OperationsResearch
Game Theory
Economics
FunctionalAnalysis
MachineLearning
ConvexGeometry
ComputerScience
Combinatorics
MathProgramming
• Even deep complexity theory results are based on math programming!(Yao’s minimax principle, the “hard-core lemma”, QIP = PSPACE, …)
![Page 18: C&O 355 Mathematical Programming Fall 2010 Lecture 1 N. Harvey TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AA A A](https://reader036.vdocuments.mx/reader036/viewer/2022070407/56649e255503460f94b146d5/html5/thumbnails/18.jpg)
Types of Mathematical Programs• Linear Program (LP):
• Rewrite as:
f(x) is a linear function
gi(x) is a linear function
c is vector defining f
A is matrix defining gi’s
![Page 19: C&O 355 Mathematical Programming Fall 2010 Lecture 1 N. Harvey TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AA A A](https://reader036.vdocuments.mx/reader036/viewer/2022070407/56649e255503460f94b146d5/html5/thumbnails/19.jpg)
Types of Mathematical Programs• Linear Program (LP):
• Convex Program:f : Rn ! R is a convex function
gi : Rn ! R is a convex function
Recall: Assume f is twice-differentiable. Thenf is convex , Hessian is positive semi-definite
could be ¸ or · or =
![Page 20: C&O 355 Mathematical Programming Fall 2010 Lecture 1 N. Harvey TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AA A A](https://reader036.vdocuments.mx/reader036/viewer/2022070407/56649e255503460f94b146d5/html5/thumbnails/20.jpg)
Types of Mathematical Programs• Linear Program (LP):
• Convex Program:
• Semidefinite Program (SDP):Special type of convex program where x is not a vector but actually a positive semi-definite matrix.
• Integer Linear Program (IP):Only finitely many solutions!But harder to solve than LPs.
could be ¸ or · or =
![Page 21: C&O 355 Mathematical Programming Fall 2010 Lecture 1 N. Harvey TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AA A A](https://reader036.vdocuments.mx/reader036/viewer/2022070407/56649e255503460f94b146d5/html5/thumbnails/21.jpg)
x1
x2 x2 - x1 · 1
x1 + 6x2 · 15
4x1 - x2 · 10
(1,1)
(0,0)
Feasible region
Constraint
Optimalpoint
2D Example(Matousek-Gartner, Ch 1)
x1¸0
x2¸0
Unique optimal solution exists
(3,2)