tabusearch example minimum spanning tree

Tabu Search

UW Spring 2005 INDE 516Project 2

Lei Li, HongRui Liu, Roberto Lu

Li, Liu, Lu UW Spring 2005 INDE 516 2

Introduction• Tabu – socially or culturally proscribed: forbidden to be

used, mentioned, or approached because of social or cultural rather than legal prohibitions. (http://encarta.msn.com/dictionary_1861698691/taboo.html)

• Glover, F. 1986. Future Paths for Integer Programming and Links to Artificial Intelligence. Computers and Operations Research. Vol. 13, pp. 533-549.

• Hansen, P. 1986. The Steepest Ascent Mildest Descent Heuristic for Combinatorial Programming. Congress on Numerical Methods in Combinatorial Optimization, Capri, Italy.

http://encarta.msn.com/dictionary_1861698691/taboo.html


Overview of Tabu Search

• Tabu search is based on introducing flexible memory structures in conjunction with strategic restrictions and aspiration levels as a means for exploiting search spaces [1].

• Meta-heuristic that guides a local heuristic search procedure to explore the solution space beyond local optimum by use of a Tabu list.


Overview of Tabu Search• Originated from surrogate constraint methods and

cutting plane approaches [2]

• Used to solve combinatorial (finite solution set) optimization problems

• A dynamic neighborhood search method

• Use of a flexible memory to restrict the next solution choice to some subset of neighborhood of current solution


Tabu Search Strategy

• 3 main strategies [7]:– Forbidding strategy: control what enters the

tabu list– Freeing strategy: control what exits the tabu

list and when– Short-term strategy: manage interplay

between the forbidding strategy and freeing strategy to select trial solutions


Parameters of Tabu Search [5]

• Local search procedure• Neighborhood structure• Aspiration conditions• Form of tabu moves• Addition of a tabu move• Maximum size of tabu list• Stopping rule


• A chief way to exploit memory in tabu search is to classify a subset of the moves in a neighborhood as forbidden (or tabu) [1].

• A neighborhood is constructed to identify adjacent solutions that can be reached from current solution [8].

• The classification depends on the history of the search, and particularly on the recency or frequency that certain move or solution components, called attributes, have participated in generating past solutions [1].

• A tabu list records forbidden moves, which are referred to as tabu moves [5].

• Tabu restrictions are subject to an important exception. When a tabu move has a sufficiently attractive evaluation where it would result in a solution better than any visited so far, then its tabu classification may be overridden. A condition that allows such an override to occur is called an aspiration criterion[1].

Basic Ingredients of Tabu Search


Basic Tabu Search Algorithm [4]

• Step 1: Choose an initial solution i in S. Set i* = i and k=0.

• Step 2: Set k=k+1 and generate a subset V* of solution in N(i,k) such that either one of the Tabu conditions is violated or at least one of the aspiration conditions holds.

• Step 3: Choose a best j in V* and set i=j.

• Step 4: If f(i) < f(i*) then set i* = i.

• Step 5: Update Tabu and aspiration conditions.

• Step 6: If a stopping condition is met then stop. Else go to Step 2.


Tabu Search Stopping ConditionsSome immediate stopping conditions could be the following [4]:

1. N(i, K+1) = 0. (no feasible solution in the neighborhood of solution i)2. K is larger than the maximum number of iterations allowed.3. The number of iterations since the last improvement of i* is larger

than a specified number.4. Evidence can be given than an optimum solution has been

obtained.

Hillier and Lieberman [5] outlined the tabu search stopping criterion by, for example, using a fixed number of iterations, a fixed amount of CPU time, or a fixed number of consecutive iterations without an improvement in the best objective function value. Also stop at any iteration where there are no feasible moves into the local neighborhood of the current trial solution.


Flowchart of a Standard Tabu Search Algorithm [7]

Initial solution (i in S)

Create a candidate list of solutions Evaluate solutions

Choose the best admissible solution

Stopping conditions satisfied ?

Update Tabu & AspirationConditions

Final solution

No

Yes


Minimum spanning Tree• Given a connected, undirected graph, a

spanning tree of that graph is a subgraph which is a tree and connects all the vertices together.

• A single graph can have many different spanning trees. We can also assign a weight to each edge, which is a number representing how unfavorable it is, and use this to assign a weight to a spanning tree by computing the sum of the weights of the edges in that spanning tree. A minimum spanning tree or minimum weight spanning tree is then a spanning tree with weight less than or equal to the weight of every other spanning tree..

http://en.wikipedia.org/wiki/Connected_graph

http://en.wikipedia.org/wiki/Undirected_graph

http://en.wikipedia.org/wiki/Spanning_tree_%28mathematics%29

http://en.wikipedia.org/wiki/Subgraph

http://en.wikipedia.org/wiki/Tree_graph

http://en.wikipedia.org/wiki/Vertex_%28graph_theory%29


• One example would be a cable TV company laying cable to a new neighborhood. If it is constrained to bury the cable only along certain paths, then there would be a graph representing which points are connected by those paths.

• Some of those paths might be more expensive, because they are longer, or require the cable to be buried deeper; these paths would be represented by edges with larger weights.

• A spanning tree for that graph would be a subset of those paths that has no cycles but still connects to every house. There might be several spanning trees possible. A minimum spanning tree would be one with the lowest total cost.


Example [5]

– Minimum spanning tree problem with constraints.– Objective: Connects all nodes with minimum costs

A

B

D

C E

20 30

15 40

10 5

25

A

B

D

C E

20 30

15 40

10 5

25

Costs

An optimal solution without considering constraints

Constraints 1: Link AD can be included only if link DE also is included. (penalty:100)Constraints 2: At most one of the three links – AD, CD, and AB – can be included.(Penalty of 100 if selected two of the three, 200 if all three are selected.)


Example

New cost = 75 (iteration 2) ( local optimum)

A

B

D

C E

20 30

15 40

10 5

25Delete Add

Iteration 1Cost=50+200 (constraint penalties)

Add Delete Cost

BEBEBE

CEACAB

75+200=27570+200=27060+100=160

CDCD

ADAC

60+100=16065+300=365

DEDEDE

CEACAD

85+100=18580+100=180

75+0=75



Example

* A tabu move will be considered only if it would result in a better solution than the best trial solution found previously (Aspiration Condition)

Iteration 3 new cost = 85 Escape local optimum

A

B

D

C E

20 30

15 40

10 5

25Tabu

DeleteAdd

Tabu list: DEIteration 2 Cost=75

Add Delete Cost

ADADAD

DE*CEAC

Tabu move85+100=18580+100=180

BEBEBE

CEACAB

100+0=10095+0=9585+0=85

CDCD

DE*CE

60+100=16095+100=195



Add25

Example

* A tabu move will be considered only if it would result in a better solution than the best trial solution found previously (Aspiration Condition)

Iteration 4 new cost = 70 Override tabu status

A

B

D

C E

20 30

15 40

10 5

Tabu

Tabu

Delete

Tabu list: DE & BEIteration 3 Cost=85

Add Delete Cost

ABABAB

BE*CEAC

Tabu move100+0=10095+0=95

ADADAD

DE*CEAC

60+100=16095+0=9590+0=90

CDCD

DE*CE

70+0=70 105+0=105



Example

Optimal SolutionCost = 70Additional iterations only find inferior solutions

A

B

D

C E

20 30

15 40

10 5

25


Pros and Cons• Pros:

– Allows non-improving solution to be accepted in order to escape from a local optimum

– The use of Tabu list– Can be applied to both discrete and continuous solution spaces– For larger and more difficult problems (scheduling, quadratic

assignment and vehicle routing), tabu search obtains solutions that rival and often surpass the best solutions previously found by other approaches [1].

• Cons:– Too many parameters to be determined– Number of iterations could be very large– Global optimum may not be found, depends on parameter

settings


Advanced Topics

• Intensification: penalize solutions far from the current solution

• Diversification: penalize solutions close to the current solution


Some Convergence Results

• Memory Tabu Search converges to the global optimum with probability one if randomly generated vectors (x) follows Gaussian or uniform distribution [6].

• Convergent Tabu Search converges to the global optimum with probability one [3].


References[1] Glover, F., Kelly, J. P., and Laguna, M. 1995. Genetic Algorithms and Tabu Search:

Hybrids for Optimization. Computers and Operations Research. Vol. 22, No. 1, pp. 111 – 134.

[2] Glover, F. and Laguna, M. 1997. Tabu Search. Norwell, MA: Kluwer Academic Publishers.

[3] Hanafi, S. 2001. On the Convergence of Tabu Search. Journal of Heuristics. Vol. 7, pp. 47 – 58.

[4] Hertz, A., Taillard, E. and Werra, D. A Tutorial on Tabu Search. Accessed on April 14, 2005: http://www.cs.colostate.edu/~whitley/CS640/hertz92tutorial.pdf

[5] Hillier, F.S. and Lieberman, G.J. 2005. Introduction to Operations Research. New York, NY: McGraw-Hill. 8th Ed.

[6] Ji, M. and Tang, H. 2004. Global Optimizations and Tabu Search Based on Mamory. Applied Mathematics and Computation. Vol. 159, pp. 449 – 457.

[7] Pham, D.T. and Karaboga, D. 2000. Intelligent Optimisation Techniques – Genetic Algorithms, Tabu Search, Simulated Annealing and Neural Networks. London: Springer-Verlag.

[8] Reeves, C.R. 1993. Modern Heuristic Techniques for Combinatorial Problems. John Wiley & Sons, Inc.

tabusearch example minimum spanning tree

Documents