Metaheuristic Optimizationvia Memory and Evolution

Tabu Search and Scatter Search

OPERATIONS RESEARCH/COMPUTER SCIENCEINTERFACES SERIES

Series EditorsProfessor Ramesh Sharda Prof. Dr. Stefan VoBOklahoma State University Universitat Hamburg

Other published titles in the series:

Greenberg I A Computer-Assisted Analysis System for Mathematical Programming Models andSolutions: A User's Guide for ANALYZE

Greenberg / Modeling by Object-Driven Linear Elemental Relations: A Users Guide for MODLERBrown & Scherer / Intelligent Scheduling SystemsNash & Sofer / The Impact of Emerging Technologies on Computer Science & Operations

ResearchBarth / Logic-Based 0-1 Constraint ProgrammingJones / Visualization and OptimizationBarr, Helgason & Kennington / Interfaces in Computer Science & Operations Research:

Advances in Metaheuristics, Optimization, & Stochastic Modeling TechnologiesEllacott, Mason & Anderson / Mathematics of Neural Networks: Models, Algorithms &

ApplicationsWoodruff IAdvances in Computational & Stochastic Optimization, Logic Programming, and

Heuristic SearchKlein / Scheduling of Resource-Constrained ProjectsBierwirth /Adaptive Search and the Management of Logistics SystemsLaguna & Gonzalez-Velarde / Computing Tools for Modeling, Optimization and SimulationStilman / Linguistic Geometry: From Search to ConstructionSakawa / Genetic Algorithms and Fuzzy Multiobjective OptimizationRibeiro & Hansen / Essays and Surveys in MetaheuristicsHolsapple, Jacob & Rao / Business Modelling: Multidisciplinary Approaches — Economics,

Operational and Information Systems PerspectivesSleezer, Wentling & CuddHuman Resource Development And Information Technology: Making

Global ConnectionsVoB & Woodruff / Optimization Software Class LibrariesUpadhyaya et al / Mobile Computing: Implementing Pervasive Information and Communications

TechnologiesReeves & Rowe / Genetic Algorithms—Principles and Perspectives: A Guide to GA TheoryBhargava & Ye / Computational Modeling And Problem Solving In The Networked World:

Interfaces in Computer Science & Operations ResearchWoodruff /Network Interdiction And Stochastic Integer ProgrammingAnandalingam & Raghavan / Telecommunications Network Design And ManagementLaguna & Marti / Scatter Search: Methodology And Implementations In CGosavi/ Simulation-Based Optimization: Parametric Optimization Techniques and Reinforcement

LearningKoutSOukis & Mitra / Decision Modelling And Information Systems: The Information Value ChainMilano / Constraint And Integer Programming: Toward a Unified MethodologyWilson & Nuzzolo / Schedule-Based Dynamic Transit Modeling: Theory and ApplicationsGolden, Raghavan & Wasil / The Next Wave In Computing, Optimization, And Decision

Technologies

Metaheuristic Optimization viaMemory and Evolution

Tabu Search and Scatter Search

edited byCesar Rego and Bahrain Alidaee

Kluwer Academic PublishersBoston/Dordrecht/London

Distributors for North, Central and South America:Kluwer Academic Publishers101 Philip DriveAssinippi ParkNorwell, Massachusetts 02061 USATelephone (781) 871-6600Fax (781) 871-9045E-Mail: kluwer@wkap.com

Distributors for all other countries:Kluwer Academic Publishers GroupPost Office Box 3223300 AH Dordrecht, THE NETHERLANDSTelephone 31 786 576 000Fax 31 786 576 254E-mail: services@wkap.nl

Electronic Services <http://www.wkap.nl>

Library of Congress Cataloging-in-Publication Data

A CLP. Catalogue record for this book is available from the Library of Congress.

Rego & Alidaee/ METAHEURISTIC OPTIMIZATION VIA MEMORY AND EVOLUTION:Tabu Search and Scatter Search

ISBN 1-4020-8134-0ISBN 0-387-23667-8 (e-book)

Copyright © 2005 by Kluwer Academic Publishers.

All rights reserved. No part of this work may be reproduced, stored in a retrievalsystem, or transmitted in any form or by any means, electronic, mechanical,photocopying, microfilming, recording, or otherwise, without the written permissionfrom the Publisher, with the exception of any material supplied specifically for thepurpose of being entered and executed on a computer system, for exclusive use bythe purchaser of the work.

Permission for books published in Europe: permissions@wkap.nl

Permissions for books published in the United States of America: permissions@wkap.com

Printed on acid-free paper.

Printed in the United States of America.

CONTENTS

Foreword vii

Part I: ADVANCES FOR NEW MODEL AND SOLUTION APPROACHES

1. A Scatter Search Tutorial for Graph-Based Permutation Problems 1Cesar Rego and Pedro Ledo

2. A Multistart Scatter Search Heuristic for Smooth NLP and 25MINLP ProblemsZsolt Ugray, Leon Lasdon, John C. Plummer, Fred Glover, Jim Kellyand Rafael Marti

3. Scatter Search Methods for the Covering Tour Problem 59Roberto Baldacci, Marco A. Boschetti, Vittorio Maniezzo and MarcoZamboni

4. Solution of the Sonet Ring Assignment Problem withCapacity Constraints 93Roberto Aringhieri and Mauro DelVAmico

Part II: ADVANCES FOR SOLVING CLASSICAL PROBLEMS

5. A Very Fast Tabu Search Algorithm for Job Shop Problem 117Jozef Grabowski and Mieczyslaw Wodecki

6. Tabu Search Heuristics for the Vehicle Routing Problem 145Jean-Frangois Cordeau and Gilbert Laporte

7. Some New Ideas in TS for Job Shop Scheduling 165Eugeniusz Nowicki and Czeslaw Smutnicki

8. A Tabu Search Heuristic for the Uncapacitated Facility LocationProblem 191Minghe Sun

9. Adaptive Memory Search Guidance for Satisfiability Problems 213Arne L0kketangen and Fred Glover

Part III: EXPERIMENTAL EVALUATIONS

10. Lessons from Applying and Experimenting with Scatter Search 229Manuel Laguna and Vinicius A. Armentano

11. Tabu Search for Mixed-Integer Programming 247Jodo Pedro Pedroso

12. Scatter Search vs. Genetic Algorithms: An Experimental Evaluationwith Permutation Problems 263Rafael Marti, Manuel Laguna and Vicente Campos

Part IV: REVIEW OF RECENT DEVELOPMENTS

13. Parallel Computation, Co-operation, Tabu Search 283Teodor Gabriel Crainic

14. Using Group Theory to Construct and Characterize MetaheuristicSearch Neighborhoods 303Bruce W. Colletti and J. Wesley Barnes

15. Logistics Management: An Opportunity for Metaheuristics 329Helena R. Lourenco

PartV: NEW PROCEDURAL DESIGNS

16. On the Integration of Metaheuristic Strategies in ConstraintProgramming 357Mauro DelVAmico and Andrea Lodi

17. General Purpose Metrics for Solution Variety 373David L Woodruff

18. Controlled Pool Maintenance for Metaheuristics 387Peter Greistorfer and Stefan Vop

19. Adaptive Memory Projection Methods for Integer Programming 425Fred Glover

20. RAMP: A New Metaheuristic Framework for CombinatorialOptimization 441Cesar Rego

Index 461

Foreword

WHAT'S ALL THE COMMOTION ABOUTSCATTER SEARCH AND TABU SEARCHANYWAY?

Fred GloverLeads School of Business, University of Colorado, Boulder, CO 80309-0419, USA,fred.glover@colorado. edu

Given an opportunity to dig into a good book, such as the one youpresently hold in your hands, little can be more of a nuisance than to read apreface. Let me acknowledge at once that I certainly won't be offended if youdon't read this one. (After all, I'll never know whether you did or not!) Asmy grandmother used to say, a preface to a book is like a speech before abanquet: it only delays enjoying what follows.

If you've bothered to read this far, at the least I owe you the courtesy ofletting you know what to expect by reading farther. First, I'll quickly describesome features of the book that I'm delaying you from enjoying. Then I'll seeI can offend as many of our mutual friends as possible (including you, to besure) by suggesting that some popular conceptions of search strategies maybe leading us down the wrong track. If at this point you haven't yet shreddedthe preface in the firm resolve to read the remainder of the book withoutfurther distraction, you will come to a proposal for an experiment to put mycontentions to the test, and thereby to expose their underlying sense ornonsense. Finally, I'll wrap up by commenting on where we are headed in thegrand scheme of things, as a way to remind you once again of what you'vebeen missing by pausing to scan these preliminary pages.

What the book offers.

Tabu search and scatter search have come a long way from their origins,and the papers in this volume disclose some of the significant recentdevelopments that are fueling advances in these areas. New contributions aredocumented for solving a varied collection of challenging problems, boththose representing classical optimization models that have attracted the

attention of researchers for many years, and those representing quite recentmodels, that belong to forefront of modern optimization concerns. Thevolume additionally includes papers devoted to experimentation, exposinglessons learned from evaluating the performance of past and recent proposals.Still another section focuses on identifying currently established andemerging roles for tabu search and scatter search in realms such as paralleloptimization and constraint programming, thereby affording a crossfertilization with other disciplines. And not least, the volume contains acollection of papers offering innovative proposals for new research directions.

This work represents the first time that tabu search and scatter search havebeen examined together in the same volume. Such a side-by-side examinationis long overdue, particularly in view of the fact that the two approaches grewout of common beginnings, and share overlapping principles andperspectives. Today tabu search is typically applied from the perspective ofadaptive memory concepts and strategies that it has introduced into themetaheuristic literature, while scatter search is primarily pursued within thecontext of an evolutionary approach, emphasizing processes concerned withgenerating and managing a population of solutions. Yet tabu search hasalways likewise embraced strategies for managing populations of solutions,notably within the setting of two of its basic types of intensificationprocesses, the first concerned with saving and re-visiting elite solutions toexplore their vicinity more thoroughly and the second concerned withanalyzing these solutions to isolate critical features, such as variables thatstrongly and frequently receive particular values, as a basis for generatingother solutions sharing these features.

In a complementary manner, scatter search has made use of adaptivememory, chiefly by virtue of incorporating improvement processes which,curiously enough, have only become recognized as relevant in otherevolutionary approaches in relatively recent years. In the setting of thesecontrasting evolutionary procedures, the slowly dawning appreciation of thevalue of such improvement procedures has spawned the appearance ofmethods that currently go by the names of "hybrid" and "memetic"procedures. Characteristically, however, by its association with tabu search,scatter search is more often disposed to make use of improvement processesthat embody adaptive memory, whereas other evolutionary approaches oftenstill lag in bridging the gap to exploit such strategies.

Curious as it may seem, an eminent body of traditional wisdom inclines usto see little of interest in designs anchored to adaptive memory and associatedstrategies for exploiting it. The TS focus on adaptive memory admittedly

Foreword ix

poses challenges that many researchers may prefer to sidestep. But, grantingsome liberty to speculation, these challenges may also help to define thefrontiers of research into problem-solving processes that seek to incorporateabilities analogous to our own. Perhaps the human impulse to avoidacknowledging that our methods are still somewhat primitive steers us awayfrom ideas that expose the yawning chasms of our ignorance. In any case, theblueprints drawn up for methods that do not venture far into the realms ofmemory and learning seem to capture the devotion of many searchpractitioners. From this standpoint, the papers of this volume represent amore daring collection than those found in many treatments of metaheuristics,by confronting important issues connected with the use of adaptive memoryin search and seeking to draw inferences about experiments that may shedlight on such issues.

Randomization: For Better or Worse1

Apart from the issue of adaptive memory, a topic that conspicuouslyseparates tabu search and scatter search from other approaches concerns theapplication of random choice. It is impossible to look very far in themetaheuristic literature without becoming aware that it is engaged in a loveaffair with randomization. Our "scientific" reports of experiments with naturereflect our fascination with the role of chance. When apparently chaoticfluctuations are brought under control by random perturbations, we seizeupon the random element as the key, while downplaying the importance ofattendant restrictions on the setting in which randomization operates. Thediligently concealed message is that under appropriate controls, perturbationis effective for creating outcomes of desired forms. If the system andattendant controls are sufficiently constrained, perturbation works even whenrandom, but a significant leap is required to conclude that randomization ispreferred to intelligent design. (Instead of accentuating differences betweenworkable and unworkable kinds of perturbation, in our quest to mold theuniverse to match our mystique we often portray the source of usefuloutcomes to be randomization in itself.)2

1 This section and the next draw on portions of the reference F. Glover (2003) "Tabu Search -Uncharted Domains," University of Colorado, Boulder.

The attraction of randomization is particularly evident in the literature of evolutionarymethods, but the theme that randomization is an essential underpinning of modern searchmethods is also echoed throughout many other segments of the metaheuristic literature.Perhaps this orientation relates to our natural disposition to take comfort in the notion thatall can be left up to chance and everything will turn out for the best in the end. Or perhapsrandomization has a seductive charm akin to the appeal of a certain kind of romantic

The orientation behind tabu search and scatter search evidently contrastswith this perspective. Many of the fundamental TS and SS strategies do notrequire randomization for their execution. While tabu search and scattersearch principles do not exclude recourse to randomization, neither do theyembrace it as essential. Occasionally such implementations are found whererandomization is given a highly significant role, a fact that is consonant withthe notion that a probing literature should entertain exceptions. However, as arule, variants of TS and SS that incorporate randomized elements operate in acontext where variation is tightly controlled by strategy.

There is of course a setting where randomization makes good sense. In agame against a shrewd opponent, it can be desirable to make sure one'sbehavior exhibits no demonstrable pattern, in order to avoid the chance thatthe pattern will be detected and turned to the adversary's advantage. In thiscase, recourse to randomization is a prudent policy. On the other hand, if weare entitled to presume that our search spaces are not possessed of a deviousintelligence bent on thwarting our moves, random behavior fulfills nopurpose comparable to its role in game playing. Instead, within the context ofsearch, randomization seems more likely to be a means of outwitting theplayer who uses it. A policy of embarking on a series of random steps wouldseem one of the best ways of confounding the goal of detecting andexploiting structure.

We may grant that systematic search runs the risk of systematic blundering,something we are all susceptible to. But with appropriate monitoring,systematic search also affords an opportunity to isolate and rectify ourblunders, and therefore to carry out more effective explorations. (Of course, ifthe search terrain itself is inherently random, no amount of systematizationwill do any good. The presence or absence of patterned design makes nodifference when all is reduced to blind luck.)3

encounter - which seems somehow more captivating when marked by an element ofcapriciousness.

Even the statement that structured search has little value for a problem whose character is"essentially random" deserves qualification. A sorting problem may involve entirelyrandom data, but may still be handled advantageously by a method that is highlysystematic. (The "P vs. NP" distinction is relevant, but does not remove the need foradditional qualification.)

Foreword xi

An Experiment with Randomization.

I am tempted to pose a research challenge, as a way of testing the relevanceof randomization in search. If a policy of random choice produces a usefulsequence of responses, then we may presume that a different randomlygenerated sequence will likewise prove useful. On the other hand, not allsequences are apt to be equally good if the total number of moves is limited -a relevant consideration if it is important to reach good outcomes before theknell of doomsday, or at least before next summer's vacation. When timematters, we may anticipate that not all randomly generated sequences willprovide the same quality of performance.

This observation prompts the following experiment. To determine therelative utility of randomization, we might examine the outcomes ofexecuting some number, say k, of randomly guided searches (each of limitedduration) with the goal of finding a value of k such that at least one of theunderlying choice sequences will lead to a solution of specified quality. Weseek a value for our parameter that is not too large and that is applicable to alarge portion of problems from a given class.

Now the evident question comes to mind. If we replace the k randomlygenerated choice sequences with k systematically generated sequences, canwe obtain comparable or better results? Or can the replacement allow us toproduce such results for a smaller value of k? In accordance with the theme ofdiversification strategies used in TS and SS, the systematically generatedsequences may be equipped to incorporate feedback, allowing a newsequence may be influenced by the outcomes of preceding sequences. (Thereis an issue of time invested in the systematic approach, so that the value of kmust be adjusted to account for this.)

The outcome of such an experiment of course depends on our skills indesigning systematic choice sequences, and also depends on the types ofproblems we confront. It must be acknowledged that the experiment I amadvocating has been implicit in a variety of TS and SS implementations of thepast. But there may be advantages to bringing the relevant factors into moreexplicit focus. Perhaps this will help us to better understand what is essentialand what is superfluous when we speak of "intelligent strategy," and toidentify better instances of such strategy over time.

Where Are We Headed?

The papers of this book provide a series of landmarks along the way as weinvestigate and seek to better understand the elements of tabu search andscatter search that account for their successes in an astonishingly varied rangeof applications. The contributions of the chapters are diverse in scope, and arenot uniform in the degree that they plumb or take advantage of fundamentalprinciples underlying TS and SS. Collectively, however, they offer a usefulglimpse of issues that deserve to be set in sharper perspective, and that moveus farther along the way toward dealing with problems whose size andcomplexity pose key challenges to the optimization methods of tomorrow.

Foreword xiii

Acknowledgment and Disclaimer

This material is based upon work supported by the National ScienceFoundation under Grant No. 0118305. Any opinions, findings, andconclusions or recommendations expressed in this material are thoseof the author(s) and do not necessarily reflect the views of the NationalScience Foundation.