cse 101, winter 2018 design and analysis of...
TRANSCRIPT
CSE 101, Winter 2018
Design and Analysis of Algorithms
Lecture 13: Linear Programming
Class URL: http://vlsicad.ucsd.edu/courses/cse101-w18/
Worked Example: Final Gf
S B
D
T1
11
S
D
E
T
1
1
1
C1
S B
D
T1
11
S
A
B
D
E
T1
8
2
5
1
1
1
2
2
C3
4
1
Final Gf
E
2
2
1
1
#2
#3
#4
S
A D
T2
22
#1
S
A
B
D
E
T3
10
2
6
1
5
1
2
2
C4
Initial Gf
Lecture 12
Ford-Fulkerson Algorithm – Summary• Initialize f = 0• REPEAT:
– Construct the residual graph Gf
– Find a path P from S to T in Gf
– If there is no such path, HALT– cp = minimum cf-capacity edge on path P– Increase f by cp units along path P
• Flow always increases the algorithm terminates• But, if capacities are B-bit integers, can take up to
|E| 2B iterations in worst case
S
A
B
T106
1
106
106
106
The Max-Flow Min-Cut Theorem• Ford-Fulkerson constructively proves that the maximum
flow equals the minimum cut• Define an (L,R) cut in the original network G as follows:
– L = nodes reachable from S in final residual graph Gf
– R = rest of nodes = V – L Note that T cannot be in L T must be in R
• Consider edges between L and R in original network G– Edges e going from L R : must have flow = capacity (by def. of L, R)– Edges e’ going from R L : must have flow = 0 (by def. of L, R)
The flow across this cut = LR edges e c(e) = the capacity of the cut
Since any flow is ≤ any cut, we have max flow = min cut
RL S T
e
e’
Three DP Examples
(Lecture 11 Tablet, Slides 44-46)
Longest Palindromic Subsequence (DPV 6.7)
• Subproblem definition
– Goal
• Recurrence
https://www.geeksforgeeks.org/print-longest-palindromic-subsequence/
Min-Edit “Palindromization”• Given a string, find an efficient algorithm that
computes the minimum number of letters that must be inserted to make the string into a palindrome.
loyal loayaol (2 insertions)aaaabbb bbbaaaabbb (3 insertions)
• Subproblem
– Goal
• Recurrence
Longest Common Substring (DPV 6.8)
• Subproblem definition
– Goal
• Recurrence
Linear Programming (LP)
• Tool for optimal allocation of scarce resources– Optimizations subject to “compatibility constraints”
• Powerful and general problem-solving method– Shortest paths, maximum flows, min-cost flows, MST, matching, 2-
person games, …
• Significance and Practice– Among most important scientific advances of 20th century– Dominates industrial practice
• Delta Airlines: $100M/year benefit from use of LP
– Commercial solvers (CPLEX, COIN, OSL), modeling languages (AMPL)– General tool for attacking intractable (NP-hard) optimization problems
LP Example: Production of Bowls vs. Mugs
Labor Clay RevenuePRODUCT (hr/unit) (lb/unit) ($/unit)Bowl 1 4 40Mug 2 3 50
Constraints: There are 40 hours of labor and 120 pounds of clay available each day
Decision variables: x1 = number of bowls to producex2 = number of mugs to produce
RESOURCE COSTS REVENUE
• Note 1: x1, x2 must be non-negative• Note 2: x1, x2 can be fractional (non-integer)
LP Example: Production of Bowls vs. Mugs
Labor Clay RevenuePRODUCT (hr/unit) (lb/unit) ($/unit)Bowl 1 4 40Mug 2 3 50
Constraints: There are 40 hours of labor and 120 pounds of clay available each day
Decision variables: x1 = number of bowls to producex2 = number of mugs to produce
RESOURCE COSTS REVENUE
Maximize Z = $40 x1 + 50 x2
Subject to x1 + 2x2 40 hr (labor constraint)4x1 + 3x2 120 lb(clay constraint)
x1 , x2 0
Example solution: x1 = 24 bowls x2 = 8 mugsRevenue = $1,360
Geometric Interpretation
4 x1 + 3 x2 120 lb
x1 + 2 x2 40 hr
Feasible Region= Area common toboth constraints
50 –
40 –
30 –
20 –
10 –
0 – |10
|60
|50
|20
|30
|40 x1
x2
x1 = #bowlsx2 = #mugs
Feasible Region Has Extreme Points
x1 = 24 bowlsx2 =8 mugsZ = $1,360 x1 = 30 bowls
x2 =0 mugsZ = $1,200
x1 = 0 bowlsx2 =20 mugsZ = $1,000
A
BC|
20|
30|
40|
10 x1
x2
40 –
30 –
20 –
10 –
0 –
• The feasible region is an intersection of half-spacesthat arise from the constraints
– Vertices of the feasible region = where 2 constraints are tight– Vertices are like “corners”
Extreme Point = Solution of Simultaneous Equations
x1 + 2x2 = 404x1 + 3x2 = 1204x1 + 8x2 = 160
-4x1 - 3x2 = -1205x2 = 40x2 = 8
x1 + 2(8) = 40x1 = 24
4 x1 + 3 x2 120 lb
x1 + 2 x2 40 hr
40 –
30 –
20 –
10 –
0 – |10
|20
|30
|40
x1
x2
Z = $50(24) + $50(8) = $1,360
248
Geometry: Convex, Concave Annotation added (see also Slide 18)
The closed line segment with endpoints a and b consists of all points a + (1 - ) b, 0 1. These points are convex combinations of a and b.
Geometry: Convex, Concave, Extreme• Inequalities induce halfspaces with respect to hyperplanes• Bounded feasible region: convex polygon or polytope• Feasible region = convex set: If a, b feasible, so is (a+b)/2• Extreme point: Feasible solution x that cannot be written as
(a+b)/2 for two distinct feasible solutions a and b
Functions: Convex, Concave, and Linear
• Function f is convex if f[x1 + (1-)x2] ≤ f(x1) + (1-)f(x2)– f defined over a convex domain– If f is convex, then a local minimum is a global minimum
• Function f is concave if –f is convex– If f is concave, then a local minimum can occur only at an extreme
point of the domain of f
• KEY OBSERVATION: A linear function is both convex and concave (!!!)– Local minima occur only at extreme points (from concavity)– Any local minimum is a global minimum (from convexity)– To find a global minimum, only need to look at extreme points
0 1
Geometry: Convex Sets, Convex Functions• Inequalities induce halfspaces with respect to hyperplanes• Bounded feasible region: convex polygon or polytope• Feasible region = convex set: If a, b feasible, so is (a+b)/2• Extreme point: Feasible solution x that cannot be written as
(a+b)/2 for two distinct feasible solutions a and b
• Function f is convex if f[x1 + (1-)x2] ≤ f(x1) + (1-)f(x2)– f defined over a convex domain– If f is convex, then a local minimum is a global minimum
0 1
4x1 + 3x2 120 lb
x1 + 2x2 40 hr
40 –
30 –
20 –
10 –
0 –
B
|10
|20
|30
|40 x1
x2
C
A
New Z = 70x1 + 20x2New Optimum:x1 = 30 bowlsx2 =0 mugsZ = $2,100
Geometric Interpretation of Objective Function
• A global optimum has maximum projection on (or, maximum inner product with) the objective function
“Level set” of objective function value
Minimization Formulation
CHEMICAL CONTRIBUTIONBrand Nitrogen (lb/bag) Phosphate (lb/bag)
Gro-plus 2 4Crop-fast 4 3
Minimize Z = $6x1 + $3x2
subject to2x1 + 4x2 16 lb of nitrogen4x1 + 3x2 24 lb of phosphate
x1, x2 0
14 –
12 –
10 –
8 –
6 –
4 –
2 –
0 – |2
|4
|6
|8
|10
|12
|14 x1
x2
C
Geometric Interpretation
x1 = 0 bags of Gro-plusx2 = 8 bags of Crop-fastZ = $24
Z = 6x1 + 3x2
• Possible scenarios: (1) Optimal solution can be a vertex of the feasible region, or (2) optimal solution can be unbounded, or (3) there is no feasible region at all
A
B
Simplex Method• Invented by George Dantzig in 1947
– Developed shortly after WWII in response to logistics challenges– Used for 1948 Berlin airlift
• Objective function is linear, and feasible region is convex can use a greedy strategy (called “hill-climbing” in textbook)
– Start at an extreme point– “Pivot” from one extreme point to a neighboring one
• NEVER WORSEN THE OBJECTIVE FUNCTION IN DOING SO !!!
– Stop when the current extreme point is a local optimum
• Which (improving) neighbor?• How to avoid degeneracy / stalling (= new set of
“tight constraints” but same extreme point)• Cycling • Known implementations can take exponential in
worst-case, but in practice see linear #pivots
Network Flow Can Be Solved as a LP
• Optimization variables: fSA, fBA, etc.• Capacity constraints: 0 ≤ fSA ≤ 3, 0 ≤ fET ≤ 2, etc.• Conservation constraints: fSA + fBA = fAD, etc.• Non-negative flow constraints: fIJ 0 for all I,J
• LP: maximize F = fSA + fSB subject to constraints• Observation: “Max Flow reduces to LP”• Observation: “Matching reduces to Max Flow” “Matching reduces to LP”
S
A
B
D
ET
3
1
1
1
1
1
2
2
5
What is the maximum flow?
Matching Reduces to Max FlowWhat is the size of the maximum matching between Workers and Jobs?W1
W2
W4
W3
J1
J2
J4
J3
“boils down to”,“can be solved using”Slide 3,
Lec 12 Tablet
Matching Reduces to Max Flow
S T
What is the size of the maximum flow from S to T? (All edge capacities = 1; red-colored edges used by max flow.)
W1
W2
W4
W3
J1
J2
J4
J3
What is the size of the maximum matching between Workers and Jobs?
System of Distinct Representatives
LP Example: Fitting a Line
• General form of a line: ax + by = c• Find a, b, c such that maxi |axi + byi – c| is minimized
How do we write this down as an LP?
Problem: Given points (x1,y1), …, (xm,ym), find the best-fit line.
“Linear Regression Modeling”
LP Example: Fitting a Line
• General form of a line: ax + by = c• Find a, b, c such that maxi |axi + byi – c| is minimized• Trick: Introduce a new variable e = max deviation from line• LP: minimize e
s.t. axi + byi – c ≤ e for all i = 1, …, maxi + byi – c -e for all i = 1, …, m
• E.g., with m = 7 points, we get an LP with 14 constraints
Problem: Given points (x1,y1), …, (xm,ym), find the best-fit line.
“Linear Regression Modeling”
LP: “Dummy Fill” in Chip Design• Manufacturing of metal wires in a chip requires planarization of each layer
before the next layer is fabricated• Varying pattern density of wires causes unevenness in planarization result• Chip layouts have “density uniformity rules”, e.g., “In any 200um x 200um
window of the layout of each wiring layer, the metal density must be between 40% and 60%.”
• Designers satisfy these rules by adding “dummy fill” into the chip layout
0 Amount of Dummy Fill “Slack”
= unused slack= dummy fill= orig layout
= fill “slack”
= fill “slack”
LP: “Dummy Fill” in Chip Design• Manufacturing of metal wires in a chip requires planarization of each layer
before the next layer is fabricated• Varying pattern density of wires causes unevenness in planarization result• Chip layouts have “density uniformity rules”, e.g., “In any 200um x 200um
window of the layout of each wiring layer, the metal density must be between 40% and 60%.”
• Designers satisfy these rules by adding “dummy fill” into the chip layout
• Problem: Given a layout of wires and a set of “windows” to verify, insert the dummy fill to minimize the maximum difference of window densities. w/r
Overlapping windows
w
n
tile
+ = fill slack= fill= orig layout
LP for Dummy Fill
• Same trick of introducing a new variable, M• M is lower-bounded by the difference of densities between any
two windows pushing down M means more uniform density• Minimize M s.t.
M |dens(Wi) – dens(Wj)| for all windows i,j// force minimum variation of post-fill window densities
dens(W) = density(orig layout + fill) in all tiles of W// density of a window = original + added fill
0 ≤ fillij ≤ slackij for all tiles Tij
// cannot put more fill than will fit (i.e., more than “slack”)// or negative amount of fill into any window
w/r
Overlapping windows
w
n
tile