1

Nuno Abreu, Zafeiris Kokkinogenis, Behdad Bozorg, Muhammad Ajmal

TABU SEARCH

2

OUTLINE Introduction to TS Parameters of TS Basic concepts of TS TS vs. other Meta-heuristics Memory in TS Use of Memory in TS Tabus Aspiration criteria Stop condition TS basic algorithm Search Process – 1 Search Process – 2 Flowchart of a Standard TS Algorithm Example Pros and Cons

3

INTRODUCTION TO TS TS is an iterative procedure designed for the solution of

optimization problems Invented by Glover (1986) Uses a neighborhood search procedure to iteratively move

from a solution x to a solution x* in the neighborhood of x Uses memory structures so that the algorithm does not visit a

given solution repeatedly Tabu Search Benefits

Cycle avoidance which also saves time Guide search to promising regions of the search space

4

PARAMETERS OF TABU SEARCH

Space search procedure

Neighborhood structure

Short-term memory: Tabu list

Types of moves

Addition of a Tabu move

Maximum size of Tabu list

Aspiration conditions

Stopping rule

5

BASIC CONCEPTS OF TABU SEARCH Saves information according to the exploration process It will be used to limit the moves through the neighborhood Structure of the neighborhood of the solutions varies from

iteration to iteration Infeasible solutions can be accepted and evaluated to escape

local minimum. To prevent from cycling, recent moves are forbidden. A tabu list records forbidden moves, which are referred to as

tabu moves A tabu move can be accepted using aspiration criteria. Allows exploitation of good solution and exploration of

unvisited region of the search space

6

TS VS OTHER META-HEURISTICS

Traditional descent methods cannot allow non-improving moves, TS can.

SA and GA does not have memory, TS has. SA uses randomness to escape local minimum, TS uses

forbidden moves. TS claims that a bad strategic choice can yield more

information than a random choice.

7

MEMORY IN TS

TS uses mainly two types of memory:

Short-term memory Recent solutions Structure were tabu moves are stored Avoids cycling

Long-term memory (frequency-based) Number of iterations that “solution components” have been

present in the current solution

8

USE OF MEMORY IN TS

Use of memory leads to learning

Prevent the search from repeating moves

Explore the unvisited area of the solution space

By using memory to avoid certain moves, TS can be seen

as global optimizer rather than local.

9

TABUS

Tabus are one of the distinctive elements of TS when compared to LS

Prevent cycling when moving away from local minimum through non-improving moves

Stored in the short-term memory – Tabu list

Tabu tenure is the number of iteration the move is tabu

Tabu list can be of fixed-length or dynamically varying

10

ASPIRATION

If a move which is tabu can lead to better solution, the tabu status should be overruled

This will be performed using aspiration level conditions

Aspiration criteria: accepting an improving solution even if generated by a tabu move

A tabu move becomes admissible if it yields a solution that is better than an aspiration value

A tabu move becomes admissible if the direction of the search (improving or non-improving) does not change

11

BASIC TABU SEARCH ALGORITHM

Step 1: Choose an initial solution i in S. Set i*=i and k=0.

Step 2: Set k=k+1 and generate a subset V of solutions in N(i,k) such that the Tabu conditions are not violated or the aspiration conditions hold.

Step 3: Choose a best j in V and set i=j.

Step 4: If f(i) < f(i*) then set i* = i.

Step 5: Update Tabu and aspiration conditions.

Step 6: If a stopping condition is met then stop. Else go to Step 2.

12

STOPPING CONDITIONS

N(i, K+1) = 0 (no feasible solution in the neighborhood of solution i)    

     K is larger than the maximum number of iterations allowed

The number of iterations since the last improvement of i* is larger than a specified number

Evidence can be given than an optimum solution has been obtained

13

SEARCH PROCESS - 1

Intensification:  searches solutions similar to the current solution wants to intensively explore known promising areas of the search space creates solutions using the more attractive components of the best

solutions in memory (recency memory) Another technique consists in changing the neighborhood’s structure by

allowing different moves

Diversification:  Examines unvisited regions of the search space Generates different solutions

Using rarely components present in the current solution Biasing the evaluation of a move by modifying the objective function adding

a term related to component frequencies

Intensification and diversification phases alternate during the search

14

SEARCH PROCESS - 2

Allowing infeasible solution Constraints defining the search space can lead the process to

mediocre solutions Induces diversification By dropping some constraints (relaxation) a larger space can be

explored Penalize objective for the violation A well known technique: Strategic oscillation

Surrogate objective Evaluates neighbors using a simpler function than the objective in

order to spot promising candidates. (Intensification) Auxiliary objective

Objective function can’t drive the search to more interesting areas Orient the search by measuring desirable attributes of solutions

15

Generate initial solution and

initialize memory structures

Construct modified

neighborhood

Select best neighbor

Execute specializes procedures

Update best solution

Update memory structures

More iteration?

Stop

NoYes

Heuristic procedure

Tabu restrictions

Candidate lists Aspiration

criteriaElite solutions

Modified choice rules for

diversification or

intensification

Restarting Strategic oscillation

Short and long term memory

FLOWCHART OF A STANDARD TABU SEARCH ALGORITHM

16

EXAMPLE

• Minimum spanning tree problem with constraints.• Objective: Connects all nodes with minimum costs

A

B

D

C E

20 30

15 40

10 5

25

A

B

D

C E

20 30

15 40

10 5

25

Costs

An optimal solution without considering constraints

Constraints 1: Link AD can be included only if link DE also is included. (penalty:100)Constraints 2: At most one of the three links – AD, CD, and AB – can be included.(Penalty of 100 if selected two of the three, 200 if all three are selected.)

17

EXAMPLE

New cost = 75 (iteration 2) ( local optimum)

A

B

D

C E

20 30

15 40

10 5

25Delete Add

Iteration 1Cost=50+200 (constraint penalties)

Add Delete Cost

BEBEBE

CEACAB

75+200=27570+200=27060+100=160

CDCD

ADAC

60+100=16065+300=365

DEDEDE

CEACAD

85+100=18580+100=180

75+0=75

Constraints 1: Link AD can be included only if link DE also is included. (penalty:100)Constraints 2: At most one of the three links – AD, CD, and AB – can be included.(Penalty of 100 if selected two of the three, 200 if all three are selected.)

18

EXAMPLE

* A tabu move will be considered only if it would result in a better solution than the best trial solution found previously (Aspiration Condition)

Iteration 3 new cost = 85 Escape local optimum

A

B

D

C E

20 30

15 40

10 5

25Tabu

DeleteAdd

Tabu list: DEIteration 2 Cost=75

Add Delete Cost

ADADAD

DE*CEAC

Tabu move85+100=18580+100=180

BEBEBE

CEACAB

100+0=10095+0=9585+0=85

CDCD

DE*CE

60+100=16095+100=195

Constraints 1: Link AD can be included only if link DE also is included. (penalty:100)Constraints 2: At most one of the three links – AD, CD, and AB – can be included.(Penalty of 100 if selected two of the three, 200 if all three are selected.)

19

Add25

EXAMPLE

* A tabu move will be considered only if it would result in a better solution than the best trial solution found previously (Aspiration Condition)

Iteration 4 new cost = 70 Override tabu status

A

B

D

C E

20 30

15 40

10 5

Tabu

Tabu

Delete

Tabu list: DE & BEIteration 3 Cost=85

Add Delete Cost

ABABAB

BE*CEAC

Tabu move100+0=100

95+0=95

ADADAD

DE*CEAC

60+100=16095+0=9590+0=90

CDCD

DE*CE

70+0=70 105+0=105

Constraints 1: Link AD can be included only if link DE also is included. (penalty:100)Constraints 2: At most one of the three links – AD, CD, and AB – can be included.(Penalty of 100 if selected two of the three, 200 if all three are selected.)

20

EXAMPLE

Optimal Solution Cost = 70

Additional iterations only find inferior solutions

A

B

D

C E

20 30

15 40

10 5

25

21

PROS AND CONS Pros:

Allows non-improving solution to be accepted in order to escape from a local optimum

The use of Tabu list For larger and more difficult problems tabu search can beat

other approaches

Cons: Too many parameters to be determined Number of iterations could be very large Global optimum may not be found, depends on parameter

settings Too complex

22

REFERENCES

Glover, F., Kelly, J. P., and Laguna, M. 1995. Genetic Algorithms and Tabu Search: Hybrids for Optimization. Computers and Operations Research. Vol. 22, No. 1, pp. 111 – 134.

Glover, F. and Laguna, M. 1997. Tabu Search. Norwell, MA: Kluwer Academic Publishers.

Hanafi, S. 2001. On the Convergence of Tabu Search. Journal of Heuristics. Vol. 7, pp. 47 – 58.

Hertz, A., Taillard, E. and Werra, D. A Tutorial on Tabu Search. Accessed on April 14, 2005: http://www.cs.colostate.edu/~whitley/CS640/hertz92tutorial.pdf

Gendreau, M. 2002. An Introduction To Tabu Search, Centre de Recherche sur les Transports et Département d´informatique et de Recherche opérationnelle, Université de Montréal.

Hillier, F.S. and Lieberman, G.J. 2005. Introduction to Operations Research. New York, NY: McGraw-Hill. 8th Ed.

23

MULTI-DIMENSIONAL PACKING BY TABU SEARCH

WRITTEN BY ANDREA LODI

24

OUTLINE

Introduction Tabu Search Basic Approach – 1 Basic Approach – 2 The TS framework for multi-dimensional bin packing Search methods Computational test - Dataset Computational test Sample of results Conclusions

25

INTRODUCTION

The paper deals with multi-dimensional cutting and packing algorithms

Assumes that the item will be packed with fixed orientation (no rotation)

Lodi, Martello and Vigo present an effective BP Tabu Search framework

The main characteristic is the adoption of a search scheme and a neighborhood independent of the packing problem to be solved

26

TABU SEARCH

The goal is empty a specified target bin

Given a current solution, the moves modify it by changing the packing of a set S of items

The target bin is the one minimizing a filling function φ(.) φ(.) measures the easiness of emptying the bin

The idea is to favor target bins packing a small area and a relatively large number of items

27

BASIC APPROACH - 1Select target bin

Include one item, j, from the target bin, and contents of k other bins in a set S

Execute an greedy-type heuristic A on S

The size of the neighborhood k is automatically updated

If the items of S are packed into k (or less) bins, item j is removed from the target bin. Otherwise, S is changed by selecting a different set of k bins, or a different item from the target bin

Then, a new item is selected, a new set S is defined and a new move is performed

The execution is halted as soon as a proven optimal solution is found, or a time limit is reached

28

BASIC APPROACH - 2If the algorithm gets stuck The target bin is not emptied, the neighborhood is enlarged by increasing

the value of up to a prefixed upper limit

Tabu list stores,for each forbidden move, the sum of the filling function values of the k+1 involved bins

Variable Neighborhood Search strategy Small values of k = Small neighborhoods, fast to explore Accepting moves dealing with increased k = Enlarged neighborhood,

more chances to improve solution By changing the size of the neighborhood, the algorithm plays

“Intensification” and “Diversification”

The execution is halted as soon as a proven optimal solution is found, or a time limit is reached

29

THE TS FRAMEWORK FOR MULTI-DIMENSIONAL BIN PACKING

30

SEARCH METHODS

 Intensification procedure explores the neighborhood of the current solution (Inner loop) Two types of diversification are defined: 

“soft” diversification - select as target bin the one having the second smallest filling function value “hard” diversification - re-pack into separate bins the items currently packed in the z/2 bins ( z being the number of bins in the current solution) with smallest filling function value

The tabu list stores, for each forbidden move: the sum of the filling function values the last moves

31

COMPUTATIONAL TESTS - DATASET

The benchmark consists of 500 random instances with n = {20,40,60,80,100}

  Ten different classes of instances were used

wi (width), hi (height) generated from uniform distribution of varying intervals

32

COMPUTATIONAL TESTS

Measures the improvement brought by TS when comparing to three greedy-type heuristics:

Hybrid Best-Fit algorithm (HBP) Knapsack Packing (KP)  Alternate Directions (AD)

Results show that the TS allows a good improvement in the quality of the solution obtained by the greedy-type heuristics

33

SAMPLE OF RESULTS

34

CONCLUSIONS

The main idea of the framework: Isolate the information concerning the problem Let a greedy-type heuristic take care of the structure and construct

feasible solutions

Tabu Search is then used to drive the search through the solution space by:

re-combining the packed items, exploring a neighborhood by alternating between “intensification”

and “diversification”

This results in a very general algorithm for multidimensional bin packing