tutorial on lagrangean decomposition: theory and...

1

Tutorial on Lagrangean Decomposition: Theory and Applications

Ignacio Grossmann and Bora TarhanCenter for Advanced Process Decision-making

Department of Chemical EngineeringCarnegie Mellon University

Pittsburgh, PA 15213

2

Decomposable MILP Problems

A

D1

D3

D2

Complicating Constraints

max

1,..{ , 1,.. , 0}

T

i i i

i i

c xst Ax b

D x d i nx X x x i n x

== =

∈ = = ≥

x1 x2 x3

Lagrangean decomposition

complicatingconstraints

D1

D3

D2

Complicating Variables

A

x1 x2 x3y

1,..

max

1,..0, 0, 1,..

T Ti i

i n

i i i

i

a y c x

st Ay D x d i ny x i n

=

+

+ = =

≥ ≥ =

∑

Benders decomposition

complicatingconstraints

Note: can reformulate by defining1i iy y +=

and apply Lagrangean decompositionComplicating constraints

3

About Lagrange

Joseph Louis Lagrange January 25, 1736 – April 10, 1813

Born in Turin, Italy: Italian parents (French ancestors father side)Born: Giuseppe Lodovico Lagrangia

1766: Lagrange succeeded Euler as director of mathematics at the Prussian Academy of Sciences in Berlin

1794: Became the first professor of analysis at theopening of École Polytechnique

1808: Napoleon named him to the Legion of Honour and made hima Count of the Empire in 1808

4

Lagrangean or Lagrangian Decomposition?

Google hits:– “Lagrangian” returned 2,190,000 hits.– “Lagrangean” returned 104,000 hits.

We will spell it as “Langrangean”

Google hits:– “Lagrangian decomposition” returned 671,00 hits.– “Lagrangean decomposition” returned 1,580,000 hits.

Google hits:– “Lagrangian relaxation” returned 133,000 hits.– “Lagrangean relaxation” returned 275,000 hits.

5

Major references on Lagrangean Relaxation/Lagrangean Decomposition

Geoffrion, A.M., “Lagrangian relaxation and its uses in integer programming, “Mathematical Programming Study 2 (1974) 82

Fisher, M.L. The Lagrangian Relaxation Method for solving integer programming problems, Management Science (1981) 27, 1.

Guignard, M.; Kim, S., “Lagrangean decomposition: A model yielding strongerLagrangean bounds,” Mathematical Programming (1987) 39, 215.

Fisher, M.L., “An applications oriented guide to Lagrangian Relaxation,”Interfaces (1985) 15, 10.

Bertsekas, D. Nonlinear Programming, 2nd ed.; Athena Scientific: Belmont, MA, 1999.Guignard, M., “Lagrangean relaxation: A short course,” Belgian Journal of OR:

Special Issue Francoro (1995) 35, 3.Guignard, “Lagrangean Relaxation,” Top, 11, 151-228 (2003)Frangioni, A.“About Lagrangian Methods in Integer Optimization,” Annals of

Operations Research (2005)139, 163

6

Gupta, A.; Maranas, C. D., “A Hierarchical Lagrangean Relaxation Procedure for Solving Midterm Planning Problems,” I&EC. Res.

(1999) 38, 1937

Chemical Engineering Papers

Van den Heever, S.A., I.E. Grossmann, S. Vasantharajan and K. Edwards, "A Lagrangean Decomposition Heuristic for the Design and Planning of Offshore Hydrocarbon Field Infrastructures with Complex Economic Objectives," I&EC Res. 40, 2857-2875(2001).

Maravelias, C.T. and I.E. Grossmann, “Simultaneous Planning for New Product Developmentand Batch Manufacturing Facilities,” I&EC Res.

40, 6147-6164 (2001).

Jackson, J. and I. E. Grossmann, "A Temporal Decomposition Scheme for Nonlinear Multisite Production Planning and Distribution Models," I&EC Res., 42, 3045-3055 (2003).

Goel, V., I. E. Grossmann, A.S. El-Bakry and E.L. Mulkay, “A Novel Branch and Bound Algorithm for Optimal Development of Gas Fields under Uncertainty in Reserves,” Computers and Chemical Engineering, 30, 1076-1092 (2006).

Tarhan, B. and I.E. Grossmann, “A Multistage Stochastic Programming Approach with Strategies for Uncertainty Reduction in the Synthesis of Process Networks with Uncertain Yields,” Computers and Chemical Engineering

32, 766-788 (2008).

Karuppiah, R.and Grossmann, I. E. “A Lagrangean based Branch-and-Cut Algorithm for Global Optimization of Nonconvex Mixed-Integer Nonlinear Programs with Decomposable Structures”, J. Global Opt.,

41 (2008) 163

Terrazas-Moreno, S., A. Flores-Tlacuahuac, I.E. Grossmann, “Lagrangean heuristic for the scheduling and control of polymerization reactors,” AIChE J., 54, 63-182 (2008).

You, F. and I.E. Grossmann, “Mixed-Integer Nonlinear Programming Models and Algorithms for Large-Scale Supply Chain Design with Stochastic Inventory Management,” submitted (2008)

7

Lagrangean Relaxation (Fisher, 1985)

MILP optimization problems can often be modeled as problems with

complicating constraints.

The complicating constraints are added to the objective function (i.e.

dualized) with a penalty term (Lagrangean multiplier) proportional to the

amount of violation of the dualized constraints.

The Lagrangean problem is easier to solve (eg. can be decomposed) than the

original problem and provides an upper bound to a maximization problem.

8

Lagrangean Relaxation

nZxeDxbAxcxZ

+∈

≤≤

= max

bAx ≤Assume that is complicating constraint

n

LR

ZxeDx

AxbucxuZ

+∈

≤

−+= )(max)(

0where u Lagrange multipliers≥

(IP)

Assume integers onlyEasily extended cont. vars.

9


0≥uwhere

( )LRZ u Z≥

nZxeDxbAxcxZ

+∈

≤≤

= max

n

LR

ZxeDx

AxbucxuZ

+∈

≤

−+= )(max)(

bAx ≤ZuZ LR ≥)( 0)( ≥− Axb

0≥u

This is a relaxation of original problem because:

i) removing the constraint relaxes the original feasible space,

ii) always holds as in the original space since

and Lagrange multiplier is always .

Lagrangean Relaxation Yields Upper Bound

Complicating Constraint

⇒

10


n

LR

ZxeDx

AxbucxuZ

+∈

≤

−+= )(max)(Relaxed problem:

min ( )0

D LRZ Z uu

=≥

Lagrangean dual:

)( 1uZLR

)( 2uZLR

)( 3uZLR

Z

nZxeDxbAxcxZ

+∈

≤≤

= maxOriginal problem:

DZdualgap

11

Graphical Interpretation

{ }n

xuD

ZxeDx

AxbucxZ

+

≥≥

∈

≤

−+= )(maxmin00

min ( )0

D LRZ Z uu

=≥

n

LR

ZxeDx

AxbucxuZ

+∈

≤

−+= )(max)(Relaxed problem:

Lagrangean dual:

Combine Relaxed and Lagrangean Dual Problems:

12


{ }n

xuD

ZxeDx

AxbucxZ

+

≥≥

∈

≤

−+= )(maxmin00

{ }nZxeDxx +∈≤ ,

{ }nZxbxAx +∈≤ ,

Optimization of Lagrange multipliers (dual) can be interpreted as optimizing

the primal objective function on the intersection of the convex hull of non-

complicating

constraints set and the LP relaxation of the

relaxed

constraints set .

0),(

max

≥∈≤∈

≤=′

+

xZxeDxConvx

bAxcxZ

n

D

Nice Proof Frangioni

(2005)

13


{ }eDxx ≤

Conv{ }nZxeDxx +∈≤ ,

{ }bAxx ≤

cx

ZLP

ZD

Z

0),(

max

≥∈≤∈

≤=′

+

xZxeDxConvx

bAxcxZ

n

D

dualgap

14

Lagrangean relaxation yields a bound at least as tight as LP relaxation

{ }eDxx ≤

Conv { }nZxeDxx +∈≤ ,

{ }bAxx ≤

cx

ZLP

ZD

Z

( ) ( )D LR LPZ P Z Z u Z≤ ≤ ≤

Theorem

15

Lagrangean Decomposition (Guignard & Kim, 1987)

Lagrangean Decomposition is a special case of Lagrangean Relaxation.

Define variables for each set of constrain, add constraints equating different variables

(new complicating constraints) to the objective function with some penalty terms.

nZxeDxbAxcxZ

+∈

≤≤

= max

n

n

Zy

Zxyx

eDybAxcxZ

+

+

∈

∈

=≤≤

=′ max

New complicating constraints n

n

LD

Zy

ZxeDybAx

xyvcxvZ

+

+

∈

∈

≤≤

−+= )(max)(

Dualize x = y

16

Lagrangean Decomposition

n

n

LD

Zy

ZxeDybAx

xyvcxvZ

+

+

∈

∈

≤≤

−+= )(max)(

n

LD

ZxbAx

xvcvZ

+∈

≤−= )(max)(1

n

LD

ZyeDy

vyvZ

+∈

≤= max)(2

Subproblem 1 Subproblem 2

( ))()(min 210vZvZZ LDLDvLD +=

≥

Lagrangeandual

17

Notes

Lagrangean decomposition is different from other possible relaxations

because every constraint in the original problem appears in one of the

subproblems.

Subproblem 1Subproblem 2

Graphically: The optimization of Lagrangean multipliers can be interpreted as

optimizing the primal objective function on the intersection of the convex hulls of

constraint sets.

18

Z

Graphical Interpretation?

Subproblem 1

{ }eDxx ≤ { }bAxx ≤

cx

ZLP

ZLR

Conv{ }nZxeDxx +∈≤ ,

Conv{ }nZxbxAx +∈≤ ,

ZLD

Subproblem 2

Note: ZLR , ZLD refer to dual solutions

19

Theorem

The bound predicted by “Lagrangean decomposition” is at least as tight as

the one provided by “Lagrangean relaxation” (Guignard and Kim, 1987)

For a maximization problem

LPLRLD ZZZPZ ≤≤≤)(

Solution of Dual ProblemZLR

or ZLD

u or ν

minimum

Piecewise linear

=>

Non-differentiable

20

1,...max{ ( ) , } max { ( )}k k

x k Kcx u b Ax Dx d x X cx u b Ax

=+ − ≤ ∈ = + −

Assuming Dx ≤ d is a bounded polyhedron (polytope) with extreme points

1,2...kx k K= , then

How to iterate on multipliers u?

0 01,..min max{ ( )} min{ ( ), 1,.. }k k k k

u uk Kcx u b Ax cx u b Ax k Kη η

≥ ≥=+ − = ≥ + − =

=> Dual problem

Cutting plane approach

1

min

. . ( ), 1,..

0,

k kns t cx u b Ax k K

u R

η

η

η

≥ + − =

≥ ∈

Kn

= no. extreme pointsiteration n

subgradient

Note: xk generated from max{cx + uk(b-Ax) subproblems

21

Update formula for multipliers (Fisher, 1985)

21 ( )( ) /

[0,2]

k k LB k k kk LD

k

u u Z Z b Ax b Ax

where

α

α

+ = + − − −

∈

Subgradient ( )k ks b Ax= −

Steepest ascent search 1k k ku u sμ+ = +

Subgradient Optimization Approach

Note: Can also use bundle methods for nondifferentiable optimizationLemarechal, Nemirovski, Nesterov

(1995)

22

Solution of Langrangean Decomposition

2. Perform branch and bound searchwhere LP relaxation is replaced byLagrangean relaxation/decomposition to a) Obtain tighter bound b) Decompose MILP

Typically in Stochastic ProgrammingCaroe and Schultz (1999)Goel and Grossmann (2006)Tarhan and Grossmann (2008)

Select MaxI, ε, ak

Set UB = +∞, LB= - ∞Solve (RP’) to find v0

| ZLD - ZLB |<ε?or k=MaxI?

Solve (P) with fixed binaries or use heuristics: Obtain ZLB

Solve (P1) and (P2):Obtain ZLD

k = k+1

Update uk

For k = 1..K

Return ZLB

&Current Solution

YES

NO

1. Iterative search in multilpliers of dual

Notes: Heuristic due to dual gapObtaining Lower Bound might be tricky

Remarks1. Methods can be extended to NLP, MINLP2. Size of dual gap depends greatly on

how problems are decomposed3. From experience gap often decreases with

problem size.

Upper Bound

Lower Bound

23

Design and Planning of Offshore Oil Infrastructures

Design decisions:WP and pipeline selectionsWP type (allows connection or not)Capacities of PP, WPs and pipelines

Planning decisions:Investment timing for WPs, pipelinesProduction profile in each time period

Objective: Maximize NPV

Complicating factor: Complex economic objectives

Productionplatform

(PP)

ShoreWell

platforms (WP)

Pipelines

Van den Heever

(2001)

24

Model elements (Multiperiod MINLP)ObjectiveMaximize Net Present Value

Reservoir and surface constraints for all t, e.g.Oil production vs. deliverabilityDeliverability vs. reservoir and surface pressure (nonlinear)Reservoir pressure vs. cumulative production (nonlinear)Mass balancesPressure balances

Logical conditions for all t, e.g.Each WP only invested in onceEach WP connected to another WP or PP

Economic calculations for all t, e.g.Sales revenueCapital costOperating costTaxesTariffsRoyalties

Complex economics

Van den Heever &Grossmann (2000)

25

WP1

WP2

WP3(Q1,3,t ; Pout

1,t)

(QP1,3,t ; Pout,P

1,t)

(Q2,3,t ; Pout2,t)

(QP2,3,t ; Pout,P

2,t)

Qwp,3,t = QPwp,3,t

Poutwp,t = Pout,P

wp,t

(wp = 1,2)

Solution method: Lagrangean DecompositionMotivation: Complex economics not linked between WPs!

Linking variables - Flows, Pressures

New objective = Old objective +Σwp,t [ λQwp,t (QP

wp,3,t - Qwp,3,t ) + λPwp,t (Pout,P

wp,t - Poutwp,t ) ]

Model decomposes - one model each WP

Lagrangean multipliers

Equationsto dualize

26

Example: 16 WPs, 15 time periods

Shore

AB

C

I

F

G

H

D

E

J

P

M

N

KL

O

Full space yields no solution in > 5 days

27

Example: ResultsConstraints

Continuous vars.Binary vars.

Solution time (CPU h.)NPV ($ mil.)

12696

1219

855275912.3

$ 95 million (8.5 %) increase in NPV compared to simple economics!

1.9% gapafter 20 iterations

28

Example: Solution

Shore

AB

C

F

Years 1 to 5

Year 4

29

Example 2: Solution

Shore

AB

C

I

F

H

D

E

J

Years 6 to 10

Year 9

Year 8 Year 8

Year 7

Year 6

30

Example: Solution

Shore

AB

C

I

F

H

D

E

J

Years 11 to 15

G

Year 12

M

N

P

Year 11

Year 13

Year 12

31

Increased importance of New Product DevelopmentTesting of Pharmaceutical and Agrochemical productsDesign and modification batch plants

MOTIVATION:

Discovery

TESTINGPhase I Phase II Phase III FDA Approval

ProcessDevelopment

PlantDesign

Build New Plant orExpand/Modify Existing

Production

Develop a new integration model and solution method forSimultaneous Optimization of Planning and Design Manufacturing

GOAL:

1. Which products to test and how to test them2. In what plants to invest and when3. Production profiles

DECISIONS:

NEW PRODUCT DEVELOPMENTMaravelias, Grossmann (2001)

32

Objective: Find the schedule that maximizes the NPVDifficulty: Probability failing tests

TRADE-OFF: Time to market vs cost

Testing in New Product DevelopmentObjective: Find the optimal investment strategy Difficulty: Risk associated with investment

Design of Process Network

DESIGN ALTERNATIVES:Retrofit existing plants

Expand existing plants

Build new plants

P1

P2

S1S2

P3RAA

RBB

P1

P2

S1S2

P3RAA

RBB

P1

P2

S1S2

P3RAA

RBB

P1

P2

S1S2

P3RAA

RBB

P1

P2

S1S2

P3RAA

RBB

P1

P2

S1S2

P3RAA

RBB P1

P2

S1S2

P3

RAA

RBB

RCC

P1

P2

S1S2

P3

RAA

RBB

RCC

TRADE-OFF: Early/large investment vs risk

TRADE-OFFS

Probability p :Cost c :

Groundwater Studies

$200,000 0.7

Acute Toxicology$700,000

0.4

Formulation Chemistry

R

$100,0000.9

Three Tasks to Schedule

Expected Value

0 1 2 3 4 5 6 7 8 9 10

23

1

Expected Value

0 1 2 3 4 5 6 7

12

3

$1,000,000 $200,000 + (0.7) $100,000 + (0.7) (0.9) $700,000 = $711,000

ii

Three tasks to schedule

33

SCHEDULING

DESIGN &PRODUCTIONPLANNING

LINKINGCONSTRAINTS

PROPOSED MILP MODELMax NPV = Income – CTEST – CINVEST – CPURCH – COPER

CINVEST = ∑p∑t αNpt yNPpt + ∑i∑t (αEit yEit + βit QEit ) Income = ∑m Pm{∑j ∑l ∑t γjlt SSjltm}

sk + dk zj – sk’ – U (wj - ykk’) ≤ 0 ∀j∈JP, ∀k, k’∈K(j) ykk’ = zj, yk’k = 0 ∀j∈JP, ∀k, k’∈K(j), ∀(k,k’)∈A sk + dk wj ≤ Tj ∀j∈JP, ∀k∈K(j) ykk’ + yk’k ≤ wj ∀j∈JP, ∀k, k’∈K(j)| k<k’ ∑q∈(QT(k)∩QC(r)) kqx = Nkr (wj – xk) ∀j∈JP, ∀k∈K(j), ∀r∈R

kqx + qkx 'ˆ – y kk’ – y k’k ≤ 1 ∀q,∀k∈Κ(q), ∀k’∈(K(q)\KK(k))|k<k’

yEitQELit ≤ QEit ≤ yEitQEU

it ∀i∈I, ∀t∈T Qit = Qit-1 + QEit ∀i∈I, ∀t∈T yEit ≤ ∑t’≤ t yNPpt’ ∀p∈PN, ∀i∈I(p), ∀t∈T ∑l PPjltm + ∑i∈Oj(j) Wijtm = ∑l SSjltm + ∑i∈Ij(j) Wijtm ∀j∈J,∀t∈T,∀m∈M Wijtm = ∑s∈PS(i) μijs ρis θistm ∀i∈I,∀ j∈J(i,s),∀t∈T,∀m∈M ∑s∈PS(i) θistm ≤ Qit HPit ∀i∈I,∀t∈T,∀m∈M

∑t zjt = wj ∀j∈JP Tj = ∑t Tj

t ∀j∈JP HTt-1 zjt ≤ Tj

t ≤ HTt zjt ∀j∈JP,∀t∈T Sjltm ≤ djlt

U fmj Ht ∑t’≤ t zjt’ ∀j∈JP, ∀l∈L, ∀t∈T, ∀m∈M Sjltm ≤ (HTt - Tj

t) djltU fmj ∀j∈JP, ∀l∈L, ∀t∈T, ∀m∈M

34

SOLUTION METHOD

Constraint matrix special structure that can be exploited

x1,y1 x3 x2,y2

Testing &Linking

Design/Planning

Use Lagrangean Decomposition to get two independent problems• (Fisher, 1985; Guignard & Kim, 1987)

Solve two decoupled problems separately (computationally cheaper)Combine independent solutions to obtain solution of integrated problem

x1, x2, x3 : Vectors of continuous variablesy1, y2 : Vectors of discrete variables

(I)

(II)

35

Formulation Decomposed MILP

Scheduling Subproblem (P1) Max NPV1 = c1x1 + λx31 s.t. A1x1 + A31x31 + B1y1 ≤ b1 (I) Design/Planning Subproblem (P2) Max NPV2 = c2x2 + c3x32 + dy2 – λx32

s.t. A2x2 + A32x32 + B2y2 ≤ b2 (II) λ: Lagrange Multipliers

-λ(x32-x31)

(P) Max NPV = c1x1 + c2x2 + c3x3 + dy2 s.t. A1x1 + A31x3 + B1y1 ≤ b1 (I) A2x2 + A32x3 + B2y2 ≤ b2 (II) (P’) Max NPV = c1x1 + c2x2 + c3x3 + dy2 s.t. A1x1 + A31x31 + B1y1 ≤ b1 (I) A2x2 + A32x32 + B2y2 ≤ b2 (II) x31=x32 (III)

36

EXAMPLE: Process Network

A1

B1

C1

D1

E1

F1

D2

E2

F2

A4

B4

C4

D4

F4

B

C

D

EF

C3

B3

A3

E4

AS1S2

S3S4

S5S6

S7

S8

S9

S10S11

S12

S13

S14S15

D3

E3

F3

S16S17

S18

S19

S20S21

P1 P2 P3 P4

A1

B1

C1

D1

E1

F1

D2

E2

F2

A4

B4

C4

D4

F4

B

C

D

EF

C3

B3

A3

E4

AS1S2

S3S4

S5S6

S7

S8

S9

S10S11

S12

S13

S14S15

D3

E3

F3

S16S17

S18

S19

S20S21

P1 P2 P3 P4

Existing Proteins: A, B, D, EPotentially New Proteins: C, F

Prices of Chemicals ($103/kg) Raw Material Price Product Price A1 40 A 300 B1 50 B 350 C1 60 C 550 D1 50 D 450 E1 60 E 400 F1 70 F 700

Process Design Data Process Conversion Capacity

(kg/month) Fixed ($103)

Variable ($103⋅month/kg)

Operating ($103/kg)

P1 0.77 10 4,500 1,100 18 P2 0.83 6 4,000 800 14 P3 0.71 8 3,500 1,000 16 P4 0.91 8 5,000 1,200 14

Demand forecasts (kg/month) Product 1st yr 2nd yr 3rd yr 4th yr 5th yr 6th yr A 3 3 3 3 3 3 B 3 3 3 4 4 4 C 4 4 4 4 4 4 D 5 5 5 5 5 5 E 4 4 2 2 0 0 F 6 6 6 6 6 6

37

EXAMPLE: Testing Tasks

Product C Product F

Test Cost ($104)

Cost of out-sourcing ($104)

Duration (months)

Prob/lity of success

ResourceReq/ment

11 160 320 3 1 Lab1 12 1130 2260 1 0.87 Lab2 13 10 20 1 0.91 Lab3 14 130 260 3 1 Lab4 15 530 1060 2 1 Lab1 16 90 180 1 1 Lab2 17 117 234 3 1 Lab3 18 400 800 3 1 Lab4 19 570 1140 5 1 Lab3 20 230 460 3 1 Lab4

Test Cost ($104)

Cost of out-sourcing ($104)

Duration (months)

Prob/lity of success

ResourceReq/ment

1 80 160 5 1 Lab1 2 80 160 4 1 Lab2 3 50 100 4 1 Lab3 4 10 20 1 0.84 Lab4 5 490 980 3 0.98 Lab1 6 111 222 6 1 Lab2 7 60 120 1 0.95 Lab3 8 1740 3480 7 1 Lab4 9 620 1240 9 1 Lab1

10 10 20 1 1 Lab2

23

4

567

8

910

1 20

1112

13

141516

17

181923

4

5

6

7

8

9

10

123

4

567

8 9

10

1 20

11

1213

141516

17

1819 20

11

12

13

141516

17

1819

Schedule 1 Schedule 2 Schedule 1 Schedule 2

38

Solution Scheduling Tests & Design Batch Plant

Lab1 1 15 11 5 9Lab2 16 12 2 6 10Lab3 19 7 17 13Lab4 20 18 4 14 8

Outsourced 30 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 Time

Tf = 10 months (months)Tc = 22 months

TESTING: Gantt Charts for Resources

C

F

Time(months)

DESIGN: Expansions of Processes

0 6 12 18

Existing Capacity Expansion

P1 P2 P3 P4P1 P2 P3 P4P1 P2 P3 P4

HEURISTIC: Min Completion Time ⇒ Tc = 10, TF = 17, NPV = $7.62 billion

PRPOSED MILP MODEL: Tc = 10, TF = 22, NPV = $8.12 billion (+7%)

39

COMPUTATIONAL RESULTS

Example 1 Example 2 Example 3Binary Variables 236 354 612Continuous Variables 9,372 7,456 32,184Constraints 8,255 6,825 30,903Full Space Method Optimal Solution 9,518 94,986 135,024 CPU Time (sec) 8.9 57.2 836.6Decomposition Heuristic Best Solution 9,518 94,986 134,834 Major iterations 5 3 6 CPU Time (sec) 20.5 37.6 252.4 Relative Duality Gap 3.96% 0.10% 0.77%

0

5

10

15

20

0 2 4 6 8 10Iteration No

Dua

lity

Gap

(%)

Ex 1

Ex 2

Ex 3

40

Multisite Distribution Network

Objective: Develop model and effective solution strategy for large-scale multiperiod planning with Nonlinear Process Models

SITE A

SITE BSITE F

SITE D

SITE C

North America

Latin America

Europe

Africa/MidEast

SITE E

SITE G

Jackson, Grossmann,Wassick, Hoffman (2002)

41

Multisite Distribution Model

•Develop Multisite Model to determine:1)What products to manufacture in each site2)What sites will supply the products for each market3)Production and inventory plan for each site

Objective: Maximize Net Present Value

•Challenges/Optimization Bottlenecks: Large-Scale NLP–Interconnections between time periods & sites/markets

Apply Lagrangean Decompostion Method

42

Spatial Decomposition

SITE S Market M

( )MPRS

MPRS

MPRS

MPRS

MPRS

MPRS

MPRS

SALESPROD

PRODPCostSALESSCostPROFIT,,,

,,,,

**max

−+

−=

λ

( )( )MPR

SMPR

SMPR

SMPR

S

MPRS

PRODPRODPCost

PRODfS

,,,,

,

max

0:SCONSTRAINT SITE

λ+−

≤ ( )( )MPR

SMPR

SMPR

SMPR

S

MPRS

SALESSALESSCost

SALESfM

,,,,

,

max

0:SCONSTRAINT Market

λ−

≤

Site SUBPROBLEM for all S (NLP) Market SUBPROBLEM for all M (LP)

43

Temporal Decomposition

PRtSINV ,

SITE S

Market M

PRtSINV 1, +

PRtSINV 1, −

SITE S

Market M

•Decompose at each time period

•Duplicate variables for Inventories for each time period

•Apply Langrangean Decomposition Algorithm

44

Multisite Distribution Model - Spatial

# Time Periods(months)

Variables/Constraints

Optimal SolutionProfit (million-$)

Full Space Solution Time(CPU sec)

Lagrangean Solution Time(CPU sec)

% Within Full Optimal Solution

2 3345 / 2848 164 52 10 10%

4 6689 / 5698 326 478 127 11%

6 10033 /8548 497 1605 279 9%

8 13377/11398 666 2350 550 9%

•3 Multi-Plant Sites, 3 Geographic Markets•Solved with GAMS/Conopt2

45

Multisite Distribution Model - Temporal

• 3 Multi-Plant Sites, 3 Geographic Markets• Solved with GAMS/Conopt2

# Time Periods(months)

Variables/Constraints

Optimal SolutionProfit (million-$)

Full Space Solution Time(CPU sec)

Lagrangean Solution Time(CPU sec)

% Within Full Optimal Solution

3 5230 / 5005 116.05 395 97 2.2

6 9973 / 8551 236.53 2013 138 2.3

12 19945 /17101 474.18 10254 278 2.2

Temporal much smaller gap!

Reason: material balances not violated at each time period

46

SuperstructureCertain Fields

• FieldsUncertain Fields

• Well Platforms (WP) • Connecting pipelines• Production Platforms (PP)

Stochastic Optimization for Gasfield Planning

Productionplatform

(PP)

ShoreWellplatforms (WP)

Pipelines

Field

Goel

, Grossmann, El-Amr, Mulckay

(2006)

Decisions•Investment

–Platforms: Which and When –Pipelines: Which and When –Platform Capacities

•Operational–Production profile: each field

Time HorizonDiscretized into time periods: 1 year each

SizeDeliverability

47

Representation of Uncertainty

• Discrete probability distribution– Size and Initial Deliverability of

each uncertain field

L M H

Size of a field

Probability

0.17

0.66

0.17

L = Low

M = Medium

H = High

Del

iver

abili

ty

Cumulative Production

H

M

L

L HM

•Scenario–Possible combination of sizes and

initial deliverabilities of different uncertain fields

–Each scenario has a given probability

•Objective–Maximize Expected NPV–Expected NPV (ENPV) = ∑s ps NPVs

48

Decision Dependent Scenario Trees

Scenario tree – Not unique: Depends on timing of investment at uncertain fields– Central to defining a Stochastic Programming Model

Invest in F in year 1

H

Invest in F

Size of F: M L

Assumption: Uncertainty in a field resolved as soon as WP installed at field

Invest in F in year 5

HSize of F: L

Invest in F

M

49

∑ ∑ ∑+++s t uf

stufuft

stt

stt

stt

s bcycdcqcpMax )]([ ,,4321∑ ∑ ∑+++s t uf

stufuft

stt

stt

stt

s bcycdcqcpMax )]([ ,,4321

)',,()]([ ,1)',(

', sstbZ suf

t

ssDuf

sst ∀¬∧⇔ ∧

=∈ ττ

)',(

)',(

)',(

'11

'11

'1,1,

ssdd

ssyy

ssbb

ss

ss

suf

suf

∀=

∀=

∀=

)',,( sst∀

}1,0{,

}1,0{,

0,

0,

'1,1,

)dim('11

'11

'

',

∈

∈

≥

≥

¬

++

++

++

stuf

stuf

yst

st

st

st

st

st

sst

bb

yy

dd

qq

Z

}1,0{,

}1,0{,

0,

0,

'1,1,

)dim('11

'11

'

',

∈

∈

≥

≥

¬

++

++

++

stuf

stuf

yst

st

st

st

st

st

sst

bb

yy

dd

qq

Z

ufbb

yy

dd

qq

Z

stuf

stuf

st

st

st

st

st

st

sst

∀=

=

=

=

++

++

++

'1,1,

'11

'11

'

',

∨

),,(0),...,,,,...,,,,( 21,2,1,, stufyyybbbdq st

ssstuf

suf

suf

st

sttufr ∀≤

Non-Anticipativity

Constraints

LinearInvestment, Operation

Constraints

Maximize Expected NPV

qts: Cont. Operation vars.

dts: Cont. Investment vars.

bsuf,t: 0-1 Investment vars., uf

yst: Other 0-1 investment vars.

Zts,s’: Boolean variables

uf: uncertain fields, s’: scenariot: time period

),(1

),(0),...,,(

),(0),...,,(

),(

,

21

21

sufb

stdddh

stqqqg

staqA

t

stuf

st

sst

st

sst

sst

s

∀≤

∀≤

∀≤

∀≤

∑

Novel Stochastic Programming Model (SPM)

Generalized Disjunctive Program

50

),(1

),(0),...,,(

),(0),...,,(

),(

,

21

21

sufb

stdddh

stqqqg

staqA

t

stuf

st

sst

st

sst

sst

s

∀≤

∀≤

∀≤

∀≤

∑

Novel Stochastic Programming Model (SPM)

Generalized Disjunctive Program

∑ ∑ ∑+++s t uf

stufuft

stt

stt

stt

s bcycdcqcpMax )]([ ,,4321∑ ∑ ∑+++s t uf

stufuft

stt

stt

stt

s bcycdcqcpMax )]([ ,,4321

)',,()]([ ,1)',(

', sstbZ suf

t

ssDuf

sst ∀¬∧⇔ ∧

=∈ ττ

)',(

)',(

)',(

'11

'11

'1,1,

ssdd

ssyy

ssbb

ss

ss

suf

suf

∀=

∀=

∀=

)',,( sst∀

}1,0{,}1,0{,

0,0,

'1,1,

)dim('11

'11

'

',

∈

∈

≥

≥

¬

++

++

++

stuf

stuf

yst

st

st

st

st

st

sst

bbyydd

qqZ

}1,0{,}1,0{,

0,0,

'1,1,

)dim('11

'11

'

',

∈

∈

≥

≥

¬

++

++

++

stuf

stuf

yst

st

st

st

st

st

sst

bbyydd

qqZ

ufbbyydd

qqZ

stuf

stuf

st

st

st

st

st

st

sst

∀=

=

=

=

++

++

++

'1,1,

'11

'11

'

',

∨

),,(0),...,,,,...,,,,( 21,2,1,, stufyyybbbdq st

ssstuf

suf

suf

st

sttufr ∀≤

Non-Anticipativity

Constraints

LinearInvestment, Operation

Constraints

Maximize Expected NPV

qts: Cont. Operation vars.

dts: Cont. Investment vars.

bsuf,t: 0-1 Investment vars., uf

yst: Other 0 -1 investment vars.

Zts,s’: Boolean variables

uf: uncertain fields, s’: scenariot: time period

),(1

),(0),...,,(

),(0),...,,(

),(

,

21

21

sufb

stdddh

stqqqg

staqA

t

stuf

st

sst

st

sst

sst

s

∀≤

∀≤

∀≤

∀≤

∑

51

Branch and Bound Algorithm

Major steps (Maximization problem) at every node• Upper bound: Lagrangean dual• Lower bound: Feasible solutions• Branching

52

Formulation of Lagrangean dual

Relaxation• Relax disjunctions, logic

constraints• Penalty for equality

constraints

53

Relaxation• Relax disjunctions, logic

constraints• Penalty for equality

constraints

Formulation of Lagrangean dual

Lagrangean Dual: Tightest upper bound

Lagrangean Relaxation: Upper bound for any λ

Model decomposes: one MILP, each scenario!!!

54

15 time periodsUncertainty sizes of fields B and C: 9 scenarios

B

C

PP

Shore

A

D

Example

Size MILP Model:16,281 0-1, 125,956 cont var, 386,597 const

$65.770 Million Stochastic vs Deterministic solution: $61.215 Million

Stochastic programming solution

55

Branch and Bound Tree

9 nodes, 1683 CPU sec (1% optimality gap)

← Optimum

353 CPUsec Root Node

56

Concluding Remarks

1. Lagrangean relaxation/decomposition is well established method for solvinglarge-scale MILPs, MINLPs, NLPs

2. Non-obvious part is how to apply it to effectively decompose a problem

Some guidelines:a) Use Lagragean decomposition, not Lagrangean relaxationa) Avoid if possible relaxing “critical” constraints (eg mass balances)b) Try to decompose so that effect of relaxed equations is not too large

tutorial on lagrangean decomposition: theory and...

Documents