linear programming and simplex algorithm reference: numerical recipe sec. 10.8
TRANSCRIPT
2
Content
• Linear programming problems• Simplex algorithm• QSOpt solver• LP Applications
– Collision detection– Melodic similarity– Underdetermined linear systems
3
The Diet Problem
• Dietician preparing a diet consisting of two foods, A and B.
Cost Protein Fat Carbohydrate
A 0.60 20 12 30
B 0.40 30 6 15
Minimum requirement: protein 60g; fat 24g; carbohydrate 30gLooking for minimum cost diet
4
Linear Programming
0,
301530
246 12
603020 ..
40.060.0min
21
21
21
21
21
xx
where
xx
xx
xxts
xxC
0)(
0)(
)(min
xg
xh
xfsubject to
Cp. NLP
5
The ProblemMaximize
Subject to
N: dimension of xM: number of constraints (M = m1+m2+m3)
N: dimension of xM: number of constraints (M = m1+m2+m3)
6
Theory of Linear Optimization
Terminology: Feasible vector Feasible basic vector Optimal feasible vector
7
Theory (cont)
• Feasible region is convex and bounded by hyperplanes
• Feasible vectors that satisfy N of the original constraints as equalities are termed feasible basic vectors.
• Optimal occur at boundary (gradient vector of objective function is always nonzero)
• Combinatorial problem: determining which N constraints (out of the N+M constraints) would be satisfied by the optimal feasible vector
8
Simplex Algorithm (Dantzig 1948)
• A series of combinations is tried to increase the objective function
• Number of iterations less than O(max(M,N))• Worst case complexity is exponential (in number
of variables); yet has polynomial smoothed complexity (and works well in practice)
9
The Canonical ProblemMaximize
Subject to
N: dimension of xM: number of constraints (M = m1+m2+m3)
N: dimension of xM: number of constraints (M = m1+m2+m3)
I.II.
III.
11
QSopt [ref]
• Contains a GUI and a callable library
12
LP Format Example
0,
301530
246 12
603020 ..
40.060.0min
21
21
21
21
21
xx
where
xx
xx
xxts
xxC Problem smallExampleMinimize obj: 0.60x1 + 0.40x2Subject to c1: 12x1 + 6x2 >= 24 30x1 + 15x2 >= 30
Bounds 0 <= x1 0 <= x2End
Somehow x1 >= 0x2 >= 0
Doesn’t work!? BNF for QSOpt scripts
16
Duality
• If a linear program has a finite optimal solution then so does the dual. Furthermore, the optimal values of the two programs are the same.
• If either problem has an unbounded optimal solution, then the other problem has no feasible solutions.
17
Diet Problem
0,
301530
246 12
603020 ..
40.060.0min
21
21
21
21
21
xx
where
xx
xx
xxts
xxC
0,,,,
301530
246 12
603020 ..
40.060.0max
32121
321
221
121
21
sssxx
where
sxx
sxx
sxxts
xxC
19
The Problem
Under-determined system (infinite number of solutions)Find the solution with minimum magnitude (i.e., closest to origin)
x
bAx
min
20
Example
Approach 1: find complete solution
npc
pn
xxx
xcx
0
0
2
,
1
1
1
0110
2011
(-2,0,0) subtract its projection along (1,-1,1)
32
32
34
32
32
32
0
0
2
0
0
2
111
111
111
3
1
0
0
2
111
111
111
3
1,
1
1
1
3
1 Tqqq
0
2
32
21
xx
xx
924d(1,-1,1)
(-2,0,0)
21
Linear Programming Solution
0
wheremin,
ix
x
bAx
a
• Most simplex algorithm assume nonnegative variables
• Set
Rxzy
zyx
iii
iii
0 and 0 where
22
Example
0,0,0
3,2,1,
3,2,1,
0
2
:subject to
3min
3322
2211
ii
ii
ii
zy
iyz
iyz
yzyz
yzyz
iii yzx 0
2
0
0
0
0
0
0
3min
3322
2211
33
22
11
33
22
11
zyzy
zyzy
zy
zy
zy
zy
zy
zy
0
2
32
21
xx
xx
23
Solved by QSopt
The real minimum (-4/3, -2/3, 2/3)The LP constraint is not exactly ||x||2(but ||x||)
3d
This problem can also be solved by SVD (singular value decomposition)
24
LP in CD (narrow phase, ref)
• A pair of convex objects: each facet represented by the plane inequality aix+biy+ciz di
• If two sets of inequality (from two convex objects) define a feasible region, then a collision is detected
• The type of optimization (min|max) and the objective function used is irrelevant.
Transportation Distance as a Measure to Melodic Similarity
Ref: Typke et al. 2003
28
Earth Mover Distance
ijjiij
j
jjjjj
n
i
iiiii
m
dyxf
uU
RuRyuyb
bbB
wW
RwRxwxa
aaA
distance over the to from weight of flow:
}0{,,,
setpoint weighted,,
}0{,,,
setpoint weighted,,
2
1
2
1
30
EMD for Melodic Similarity
• Ground distance: Euclidean distance
• Scale time coordinate (so that they are comparable)
• Transposition: transpose one of the melodies so that the weighted average pitch is equal (not optimum but acceptable)
22jijiij ppttd
31
HW: Verify planar CD using LP
Warning: Non-negative assumption
(for some solvers)!!
Warning: Non-negative assumption
(for some solvers)!!
32
Verify planar CD using LP.
1
2
3
4
5
6 11[I] 11 .3
3[II] 3 .2
3[II] 3 .1
3
2
1
syxyx
sxx
syy
0II][0 .6
10I][ 10 .5
5II][ 5 .4
5
5
4
syxyx
sxx
syy
35
QSopt [ref]
• Contains a GUI and a callable library
36
LP Format Example
0,
301530
246 12
603020 ..
40.060.0min
21
21
21
21
21
xx
where
xx
xx
xxts
xxC Problem smallExampleMinimize obj: 0.60x1 + 0.40x2Subject to c1: 12x1 + 6x2 >= 24 30x1 + 15x2 >= 30
Bounds 0 <= x1 0 <= x2End
[weird] Somehow x1 >= 0x2 >= 0
Doesn’t work!?
41
Accessing a Solution
• value: objective function
• x: solution vector (columns)
• pi: dual variables• slack: slack variables• rc: reduced cost
• In our applications, we don’t really care about pi, slack, and rc.
• Set them to NULL
42
01.10.5
4.13.21.3
cmat
# of nonzeros in jth columnLocation of start entry
0 0 0
1 1
cmatind
QS_MAXor
QS_MIN