EML 4550 - Spring’08
EML4552 - Engineering Design Systems II(Senior Design Project)
Optimization Theory andOptimum DesignLinear Programming
Hyman: Chapter 10
EML 4550 - Spring’08
Optimization
Minimize (Maximize) an Objective Function of certain Variables subject to Constraints
0),...,(
...
0),...,(
0),...,(
),...,(min
21
212
211
21
nm
n
n
n
xxxg
xxxg
xxxg
xxxf
EML 4550 - Spring’08
Linear Programming
Maximize a Linear Objective Function of i positive variables subject to j Linear Constraints
0
,...,2,1
),...,(max
1
121
i
n
ijiij
n
iiin
x
mj
rxa
xkxxxU
EML 4550 - Spring’08
Linear Programming
Almost all linear optimization problems can be reduced to the canonical (standard) form Minimization, change sign on objective
function Variables that can be negative, define:
Constraint reversal, ‘slack’ and ‘surplus’ variables
0,
ii
iii
xx
xxx
EML 4550 - Spring’08
L.P.: Geometric Interpretation
0
0
)4(....................5
)3..(..........4
)2(..........42
)1(..........93
2),(max
2
1
2
21
12
21
2121
x
x
x
xx
xx
xx
xxxxU
EML 4550 - Spring’08
L.P.: Geometric Interpretation
x1
x2(1)
(2)
(3)
(4)
U= 0 2 4 6.5
EML 4550 - Spring’08
L.P.: The “Simplex” Algorithm
Standard method to solve linear programming optimization problems
Geometric interpretation of linear programming problems (simple example)
Based on principle that optimum will be ON a constraint (typically a “corner”)
EML 4550 - Spring’08
Simplex Algorithm: Example
0,,
4204
46023
4302
523max
321
21
31
321
321
xxx
xx
xx
xxx
xxxU
EML 4550 - Spring’08
Simplex Algorithm: Example
VariablesSlackxxx
xxxxxx
xxx
xxx
xxxx
xxxU
_"",,
0,,,,,
4204
46023
4302
523max
654
654321
621
531
4321
321
Add x4 to make the Inequality an equality
EML 4550 - Spring’08
Simplex Algorithm: Example
The Simplex “Algorithm” consists of a number of steps carried out in ‘cycles’ until an optimum is reached
“Geometrically”, the Simplex method explores the objective function by ‘walking’ along the constraints in the ‘steepest’ direction until it reaches the optimum
Based on: Unique solution Optimum on a constraint
EML 4550 - Spring’08
Simplex Algorithm: Example
Cycle 1 Select a starting point (a ‘corner’) In this example a corner is set by 3 variables (the
other 3 can be solved from the constraints)
Step 1: Select the 3 ‘basis’ variables such that they appear in only one constraint and have coefficient of unity (x4, x5, x6 in this case). Assign a value of 0 to the ‘non-basis’ variables.
Step 2:Determine coordinates of initial ‘corner’ (x1=0, x2=0, x3=0, x4=430, x5=460, x6=420)
EML 4550 - Spring’08
Simplex Algorithm: Example
Step 3: Calculate objective function at the corner point (x1=0, x2=0, x3=0, x4=430, x5=460, x6=420), U=0
Step 4: Move from corner to an ‘adjacent’ corner along the ‘edges’ represented by the constraints. Vary the non-basis variables to move along the edges. Move along the edge which will increase “U” the most. To do this, express the objective function in terms of the non-basis (U=3x1+2x2+5x3, already, if not, solve the constraints to do so)
EML 4550 - Spring’08
Simplex Algorithm: Example
Step 5: From the present ‘corner’ (x1=0, x2=0, x3=0) move along the variable that will produce the largest increase in U (highest coefficient), in this case x3. We re-formulate the basis by introducing x3 and deleting from the basis one of the variables.
Step 6: We want to increase x3 by as much as possible without violating any constraint (i.e., move to the next ‘corner’). To do this, reformulate the constraints for the ‘incoming’ variable to the basis
EML 4550 - Spring’08
Simplex Algorithm: Example
Step 6 (cont.): In this case for x1=x2=0 and the incoming x3, the constraints are:
Step 7: Determine how much x3 can increase before driving any variable negative. Limits are: (a) 430, (b) 230, (c) therefore, x3 can be increased up to 230 and x5 leaves the basis
).......(..........420
)........(4602
)(..........430
6
53
43
cx
bxx
axx
EML 4550 - Spring’08
Simplex Algorithm: Example
Step 8: x3, x4, and x6 are the new basis, set the non-basis variables to zero, x1=0, x2=0, x5=0
Prepare to start the next cycle (repeat procedure)
EML 4550 - Spring’08
Simplex Algorithm: Example
Cycle 2Step 1: The new basis was chosen as the last
step of the previous cycle (x3, x4, x6).Step 2:Determine coordinates of new ‘corner’
by re-writing constraints:
(x1=0, x2=0, x3=230, x4=200, x5=0, x6=420)
).......(..........420
)......(..........4602
)(..........430
6
3
43
cx
bx
axx
EML 4550 - Spring’08
Simplex Algorithm: Example
Step 3: Calculate objective function at the corner point (x1=0, x2=0, x3=230, x4=200, x5=0, x6=420), U=3(0)+2(0)+5(230)=1150
Step 4: Prepare to choose new basis. Re-write U in terms of non-basis variables (using the constraints to eliminate the basis variables): 521 )2/5(2)2/9(1150 xxxU
EML 4550 - Spring’08
Simplex Algorithm: Example
Step 5: Move away from present corner along the edge that will increase U the most. From the objective function, only an increase in x2 will yield improvement (all other coefficients are negative). Introduce x2 to the basis.
Step 6: Re-write constraints in terms of new basis variable:
).(..........4204
).........(..........4602
)...(4302
62
3
432
cxx
bx
axxx
EML 4550 - Spring’08
Simplex Algorithm: Example
Step 7: Determine how much x2 can increase before driving any variable negative. Limits are: (a) 100 (for x3=230),
(b) , (c) 105 therefore, x2 can be increased up to 100 and x4 leaves the basis
Step 8: x2, x3, and x6 are the new basis, set the non-basis variables to zero, x1=0, x4=0, x5=0
Prepare to start the next cycle (repeat procedure)
EML 4550 - Spring’08
Simplex Algorithm: Example
Cycle 3Repeat the same steps now with x2, x3, x6
as the basis. After steps 2 and 3 the solution is x1=0, x2=100, x3=230, x4=0, x5=0, x6=20, U=1350 (further improvement). At step 4, however, all coefficients on the objective function, U, are negative, so no further improvements are possible, we are done! We have found the optimum (x1=0, x2=100, x3=230, U=1350).
EML 4550 - Spring’08
EML 4550 - Spring’08
Simplex Algorithm
Found the optimum with exploration of just 3 ‘corners’
Efficiency increases tremendously with the complexity of the problem (number of variables and constraints)
“New” L.P. algorithms may be even more efficient in extremely large problems