Column Generation

ENGG 6070 Instructor: Shawki M Areibi Students: Limin Ma, Hao Qin

What is Column Generation ?

Column generation is a technique that is widely used to solve large scale integer programs

Prototype of Linear Programs of Large-scale Systems

z* = Max z = c1 x1 + c2 x2 + … +cn xn ,

Subject to:

ai1x1 + ai2 x2 + … + ainxn = bi ( i = 1, 2, …, m) (1)

xn 0 ( j = 1, 2, …, n)

where n is very large , may be more than millions.

Example : Cutting stock problem

A paper company must produce various sizes of its paper products from its raw rolls to meet customer demand.

Example : Cutting stock problem

If customers need bi rolls of sizes Li , how the company meet the demand using the fewest number of rolls ?

Example : Cutting stock problem

A key obstacle : In general, there are exponentially many patterns xn to cut raw rolls:x1

x2

xn

Independent subsystem Primal block angular Dual block angular

Bordered angular Block triangular Staircase

Figure 1

Structure of Large-Scale Problems

The Decomposition AlgorithmDecompose the large scale system in many subsystem for computation. For Independent subsystem , it can be written as:

Minimize

subject to:

= bi (i = 1, 2, …,t)

= bi (i = t+1, t+2, …,u)

= bi (i = u+1, u+2, …,m)

xj >=0 (j = 1, 2, …, n)

j

n

sjjj

s

rjjj

r

jj xcxcxc

111

j

r

jij xa

1

j

s

rjij xa

1

j

n

sjij xa

1

Why Decomposition ?• Provide significant computational savings, since the computations for linear programs are quite sensitive to m, the number of constraints, in practice proportionally to m3. So it can save 1/m2 times.

• Each of the independent subproblem can be treated separately. Data can be gathered, analyzed and stored separately.

• Because of the large number of variables and columns, directly solution by the simplex method may be inappropriate. Simply generating all the coefficients data aij usually prohibits this approach.

• Column generation extends the technique introduced in the decomposition algorithm, of using the simplex method (SM), but generating the coefficient data only as needed. Consequently, new data could be generated by solving this linear program with an appropriate objective function.

Why Column generation ?

The Approach of Column Generation

As in decomposition, assume a priori that certain variables, say xJ+1, xJ+2, …, xn are nonbasic and restrict their

values to zero. The problem becomes :

zJ = Max c1x1 + c2x2 + … + cJxJ , (1)

subject to: ai1x1 + ai2 x2 + … + aiJxJ = bi ( i = 1, 2, …, m) (2) xj ≥ 0 ( j = 1, 2, …, J)

The original problem (1) is called a master problem (MP), whereas problem (2) is called the restricted master problem(RMP).

Suppose that the RMP has been solved by SM and that J1,

J2, …, J

m are the optimal shadow prices. It is optimal if the simplex optimality condition holds, that is, if 0

1 ij

m

i

Jijj acc

the original problem has been solved without specifying all of the aij data or solving the full MP.

01

ij

m

i

Jijj acc

then the simplex method, when applied to MP, would introduce variable xs into the basis. The new RMP can be solved by the simplex method and the entire procedure can be repeated.

If

This procedure avoids solving the full MP; instead it alternately solves a RMP . It is referred to as a subprobelm.

The method is specified in flow-chart in Figure 2. Its efficiency is predicted upon:

• Obtaining an optimal solution before many columns have been added to RMP.

• Being able to solve the subproblem effectively.

The Significance of Column Generation

Solve RMP

j

J

jj

J xcMaxz

1

Subject to:

jj

J

jij bxa

1

(i = 1, 2, …,m)

xj ≥ 0 ( j = 1, 2, …, J) Let πJ

1, πJ2, …, πJ

m be the optimal shadow prices.

Solve the subproblem

is

m

i

Jis

m

iij

Jij

nj

J acacv Max

111

vJ≤ 0

Yes

Terminate The last restricted master solution is optimal with xJ+1 = xJ+2 = … = xJ+n = 0

Increase J to J+1 and add variable xs as the (J+1)st variable to the RMP.

No

Figure 2. Flow-chart form of column generation

The column generation method might generate anon-integer solution. For applications (with largedemands), rounding to nearest integers producessatisfactory answers. If an exact optimal isneeded, one can apply the B&B technique.

Remark :

Type I : Airline crew scheduling (ACS)

Type II : The cutting stock (CS) problem

Two Major Types of Column Generation

Type I column generation

• Uses an auxiliary model (AM) to identify an “attractive” set of columns, defining a RMP that optimizes over these explicitly defined columns.

• The RMP accepts these columns and does not interact further with the AM.

Two Types of Column Generation

Type I : Airline crew scheduling (ACS)

• The objective is to minimize total crew cost. Airline schedules give the type of aircraft and its departure and arrival times for each flight segment, a flight from one city to another.

• A crew works a “rotation,” which services a series of segments and may last several days. Each rotation must satisfy a number of constraints imposed by official agencies, union rules and airline restrictions.

• The cost of rotation includes lodging and other travel expenses, flight time and deadheading

Defining aij =1 if rotation j ( jJ ) serves segment i (i I ), 0 otherwise and employing decision variables x j =1 if rotation j is prescribed, 0 otherwise ( j J ), Problem ACS may be formulated as:

Min ZACS =

(4.1)

s.t. i I

(4.2) x j {0, 1} j J (4.3)

1

jJj

ij xa

JJj

j xc

The algorithm of ACS

Challenge :

there may be millions of feasible rotations for a commercial airline so that even generating them may require a prohibitive amount of run time.

Solution :

1. Invokes an AM to generate a set of feasible rotations. Each generated column aj ( j J ) consists of elements aij =1 (i Ij ) for each segment i served by rotation j , and aij =0 (i Ij ).2. Solves the set partitioning problem with the columns defined explicitly by the AM.

Experience Result :

• It is typically not possible to assure that a restricted set of rotations includes a subset that comprises an optimal solution, so the overall approach is a heuristic.

• Nevertheless, experience has shown that it is possible to prescribe good solutions if the set partitioning problem can be solved effectively for a large number of generated rotations.

Type II column generation

Type II column generation has two classic applications:

i) The cutting stock (CS) problem;

ii) The vehicle routing problem;

The Cutting Stock (CS) Problem.

A company stocks rolls of sheet metal in standard lengths Lm (m M). Assume that the company has an unlimited number of each standard length and that will cut these standard lengths into shorter lengths to satisfy customers orders. Customers require bi pieces of length i (i I ), where I is the set of cut lengths ordered. Note that order I can be satisfied as long as li Lm for some m. Standard length Lm costs cm(m M) and the total cost of filling orders includes the costs of all standard lengths cut to fill the orders. The cutting stock problem is to fill customer orders at minimum total cost.

Let aij denote the number of pieces of length i (i I ) in cutting pattern j ; and c j , the cost of the standard roll used in pattern j . Thus, the index j implies m as well as I j , the subset of cut lengths included in the cutting pattern. To be feasible, cutting pattern j must observe the limitation imposed by the standard length Lm(m M):

mijIi

i Lal

The Cutting Stock (CS) Problem.

The algorithmDecision variables x j =number of cutting patterns of type j prescribed ( j J ) to present Problem CS as formulated: Min (5.1)

s.t. iI (5.2)

x j 0, Integer j J. (5.3)

jJj

jRPM xcz

ijJj

ij bxa

The algorithm

Linear relaxation for integer constraints:

The requirement of the cutting stock problem is known to be tight. This heuristic is not guaranteed to prescribe the optimal integer solution. To guarantee such a solution, the linear relaxation may be used to obtain a lower bound at each node in a branch-and-bound (B&B) search tree.

The primary difficulty is that the number of cutting patterns, |J |, may be extremely large, so that solving the LP problem with all |J | columns could take a prohibitive amount of time.

The algorithm

Instead of listing all |J | columns explicitly and pricing each out to identify the column that should enter solution at each simplex iteration, the RMP generates a limited set of columns and must be optimized over these columns to obtain dual variables that are used to define an dual objective function .

Defining wi =dual variable associated with constraint i of type (5) (i . I ) and cm =cost of standard length Lm(m . M) and employing decision variables yi =number of length i in a cutting pattern j .

The algorithm

It can be formulated as:

Min (6.1) s.t. (6.2)

yi 0, Integer i I (6.3)

If Z*POP 0 (m M), the current RMP solution is

optimal. Otherwise, if Z*POP 0, the dual solution

generates an improving column, which is assigned a new column number j ( j j +1), an objective function coefficient c j = cm and coefficient in each row i (i I )aij = y*i (i I ).

iIi

imPOPm ywcz

*

miIi

i Lyl

The dual solution

• The primary limitation of the Type I method is that the AM generates columns without interacting with the RMP. It normally has to generating a large set of columns with the hope of including the subset that comprises the optimal solution

• Type II methods, which do involve interaction between the RMP and SP(s), allow columns to be identified and introduced as needed

Comparison between Type I and Type II

Other Problem Can be Solved by Column Generation :

• Generalized assignment problem

• The vehicle routing problem

• Single machine scheduling

• Assembly system design with tool changes.

• Problems that can be solved by the Dantzig-Wolfe

decomposition.

Technical considerations

i) Formulation issues

a) RMP formulation

The goal is to formulate a decomposition that will allow the solution to the RMP to serve as a tight bound on the value of the optimal integer solution to facilitate the B&B search: (1) They may provide tighter bounds than more compact formulations can; (2) They may avoid symmetry that requires B&B to expend considerable run time to enumerate a large number of equivalent solutions; (3) They are amenable to decomposition, perhaps separating into a number of independent SPs that may be solved easily; and (4) They may provide the only formulation possible.

b). SP formulation

SP(s) also play a crucial role through their structure, symmetry, complexity and whether of not they exhibit the Integrality Property. The integer variables in an IP typically become decision variables in one or more SPs. Formulations that have the block diagonal structure are attractive because they result in small, independent SPs that are typically more effectively solved.

ii) Algorithm design issues

a) Branching strategiesBranching must not destroy SP structure and should result in siblings that represent balanced sets of solutions. Balancing is important because it can be expected to result in a tighter bound at each sibling node, facilitating solution.

b) Subproblem solution strategiesSP solution strategies select the column that will enter the RMP and a method to solve each SP. One strategy would be to solve all SPs and select the best improving column to enter the RMP.

iii) Implementation issues

a) Pool management

1. Determine an initial BFS for the RMP2. Initialize the column pool to be empty3. Optimize the RMP over columns that are known explicitly4. Delete nonbasic generated columns in the RMP that have positive reduced costs5. Select a subset of remaining pool columns with negative reduced costs, add them to RMP and go to step (3)

6. Eliminate any remaining columns from the pool7. Use a heuristic to solve the SP(s), If improving columns are generated, add them to the pool and go to step (5)8. If no columns are generated in step (7), use an optimizing algorithm to solve the SP(s) If improving columns are generated, add them to the pool and go to step (5)9. Stop

b) Degeneracy

Degeneracy in the RMP may be problematic for several reasons. • First, a number of pivots ( and corresponding SP solutions to generate entering columns) may be required before the search can progress to a better, adjacent extreme point. • Second, special steps may be required to avoid a large number of pivots (called stalling) and the worst-case, cycling. • Thirdly, if the primal RMP has a degenerate optimum, the associated dual has alternative optimal solutions and the “best” set of optimal dual variables must be identified so that they can be passed into the SPs and the primal basic feasible solution that satisfies the LP optimality criterion may be identified.

Several issues must be resolved by future research to enhance existing capabilities.

• First, as in any LP problem, degeneracy causes inefficiency by requiring a number of pivot operations to advance beyond a degenerate extreme point and may lead to stalling and cycling.

• Second, SP solution strategies may require a large number of SPs to be solved for each column that is entered into the solution of the RMP.

Conclusions and recommendations for future research

Thank you !