scatter search: methodology and...

58
Scatter Search: Scatter Search: Methodology and Methodology and Applications Applications Manuel Laguna Manuel Laguna University of Colorado University of Colorado Rafael Rafael Martí Martí University of Valencia University of Valencia

Upload: others

Post on 23-Jul-2020

9 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Scatter Search: Methodology and Applicationsyalma.fime.uanl.mx/~roger/work/teaching/mecbs5122/4... · Scatter Search Methodology. 4 Metaheuristic A metaheuristic refers to a master

Scatter Search:Scatter Search:Methodology and Methodology and

ApplicationsApplicationsManuel LagunaManuel Laguna

University of ColoradoUniversity of Colorado

Rafael Rafael MartíMartíUniversity of ValenciaUniversity of Valencia

Page 2: Scatter Search: Methodology and Applicationsyalma.fime.uanl.mx/~roger/work/teaching/mecbs5122/4... · Scatter Search Methodology. 4 Metaheuristic A metaheuristic refers to a master

22

Based on …Based on …

Scatter Search: Methodology and Scatter Search: Methodology and Implementations in CImplementations in CLaguna, M. and R. Laguna, M. and R. MartíMartíKluwer Academic Publishers, Boston, 2003.Kluwer Academic Publishers, Boston, 2003.

Page 3: Scatter Search: Methodology and Applicationsyalma.fime.uanl.mx/~roger/work/teaching/mecbs5122/4... · Scatter Search Methodology. 4 Metaheuristic A metaheuristic refers to a master

Scatter SearchScatter Search

MethodologyMethodology

Page 4: Scatter Search: Methodology and Applicationsyalma.fime.uanl.mx/~roger/work/teaching/mecbs5122/4... · Scatter Search Methodology. 4 Metaheuristic A metaheuristic refers to a master

44

MetaheuristicMetaheuristic

A metaheuristic refers to a master strategy A metaheuristic refers to a master strategy that guides and modifies other heuristics that guides and modifies other heuristics to produce solutions beyond those that are to produce solutions beyond those that are normally generated in a quest for local normally generated in a quest for local optimality. optimality.

A metaheuristic is a procedure that has A metaheuristic is a procedure that has the ability to escape local optimalitythe ability to escape local optimality

Page 5: Scatter Search: Methodology and Applicationsyalma.fime.uanl.mx/~roger/work/teaching/mecbs5122/4... · Scatter Search Methodology. 4 Metaheuristic A metaheuristic refers to a master

55

Typical Search TrajectoryTypical Search Trajectory

0

10

20

30

40

50

60

70

80

90

100

1 4 7 10 13 16 19 22 25 28 31 34 37 40

Iteration

Obj

ectiv

e Fu

nctio

n

ValueBest Value

Page 6: Scatter Search: Methodology and Applicationsyalma.fime.uanl.mx/~roger/work/teaching/mecbs5122/4... · Scatter Search Methodology. 4 Metaheuristic A metaheuristic refers to a master

66

Metaheuristic ClassificationMetaheuristic Classification

x/y/zx/y/z ClassificationClassificationx = A (adaptive memory) or M (x = A (adaptive memory) or M (memorylessmemoryless))y = N (systematic neighborhood search) or S (random y = N (systematic neighborhood search) or S (random sampling)sampling)z = 1 (one current solution) or P (population of z = 1 (one current solution) or P (population of solutions)solutions)

Some ClassificationsSome ClassificationsTabuTabu search (A/N/1)search (A/N/1)Genetic Algorithms (M/S/P)Genetic Algorithms (M/S/P)Scatter Search (M/N/P)Scatter Search (M/N/P)

Page 7: Scatter Search: Methodology and Applicationsyalma.fime.uanl.mx/~roger/work/teaching/mecbs5122/4... · Scatter Search Methodology. 4 Metaheuristic A metaheuristic refers to a master

77

Scatter SearchScatter Search

P

Diversification GenerationMethod

Repeat until |P| = PSize

Subset GenerationMethod

ImprovementMethod

Solution CombinationMethod

ImprovementMethod

Stop if no morenew solutions

Reference SetUpdate Method

RefSet

Page 8: Scatter Search: Methodology and Applicationsyalma.fime.uanl.mx/~roger/work/teaching/mecbs5122/4... · Scatter Search Methodology. 4 Metaheuristic A metaheuristic refers to a master

88

Repeat until |P| = PSize

Scatter Search with RebuildingScatter Search with RebuildingP

Diversification GenerationMethod

Subset GenerationMethod

ImprovementMethod

Solution CombinationMethod

ImprovementMethod

No more new solutions

Reference SetUpdate Method

RefSet

Diversification GenerationMethod

ImprovementMethod

Stop if MaxIterreached

Page 9: Scatter Search: Methodology and Applicationsyalma.fime.uanl.mx/~roger/work/teaching/mecbs5122/4... · Scatter Search Methodology. 4 Metaheuristic A metaheuristic refers to a master

99

TutorialTutorial

Unconstrained Nonlinear Optimization Unconstrained Nonlinear Optimization ProblemProblem

( ) ( ) ( ) ( )( ) ( )( ) ( )( )

4,,1for 1010toSubject

118.19111.10

1901100Minimize

422

42

2

23

2234

21

2212

K=≤≤−

−−+−+−

+−+−+−+−

ix

xxxx

xxxxxx

i

Page 10: Scatter Search: Methodology and Applicationsyalma.fime.uanl.mx/~roger/work/teaching/mecbs5122/4... · Scatter Search Methodology. 4 Metaheuristic A metaheuristic refers to a master

1010

Diversification Generation Diversification Generation MethodMethod

Subrange 1 Subrange 2 Subrange 3 Subrange 4

-10 -5 0 +5 +10

Probability of selecting a subrange is proportional to a frequency count

Page 11: Scatter Search: Methodology and Applicationsyalma.fime.uanl.mx/~roger/work/teaching/mecbs5122/4... · Scatter Search Methodology. 4 Metaheuristic A metaheuristic refers to a master

1111

Diverse SolutionsDiverse Solutions

x1

1.11-9.587.428.83-6.231.640.2-3.09-6.08-1.97

x2

0.85-6.57-1.71-8.457.483.34-3.646.620.678.13

x3

9.48-8.819.284.526-8.32-5.3-2.33-6.48-5.63

x2

-6.35-2.275.923.187.8-8.66-7.03-3.121.488.02

f(x)835546.21542078.9901878.0775470.7171450.5546349.8114023.87469.1279099.954537.2

Solution123456789

10

Page 12: Scatter Search: Methodology and Applicationsyalma.fime.uanl.mx/~roger/work/teaching/mecbs5122/4... · Scatter Search Methodology. 4 Metaheuristic A metaheuristic refers to a master

1212

Improvement MethodImprovement MethodSolution x1 x2 x3 x2 f(x)

1 2.5 5.4 2.59 5.67 1002.72 -0.52 -0.5 0.35 -0.14 138.53 -2.6 5.9 4.23 10 7653.74 0.49 0.53 2.47 5.89 213.75 -3.04 9.45 1.14 0.41 720.16 -1.4 2.46 0.37 -3.94 1646.77 -0.36 -0.31 0.8 1.14 57.18 -1.63 2.51 0.73 0.56 21.59 -0.8 0.69 -1.16 1.5 11.2

10 -2.47 5.32 -2.92 8.15 1416.7

Nelder and Mead (1965)

Page 13: Scatter Search: Methodology and Applicationsyalma.fime.uanl.mx/~roger/work/teaching/mecbs5122/4... · Scatter Search Methodology. 4 Metaheuristic A metaheuristic refers to a master

1313

Reference Set Update MethodReference Set Update Method(Initial (Initial RefSetRefSet))

Objective functionvalue to measurequality

b1 high-qualitysolutions

Max-min criterion andEuclidean distances tomeasure diversity

b2 diversesolutions

RefSet of size b

Page 14: Scatter Search: Methodology and Applicationsyalma.fime.uanl.mx/~roger/work/teaching/mecbs5122/4... · Scatter Search Methodology. 4 Metaheuristic A metaheuristic refers to a master

1414

Initial Initial RefSetRefSetSolution

number in P x1 x2 x3 x4 f(x) 35 -0.0444 0.0424 1.3577 1.8047 2.1 46 1.133 1.2739 -0.6999 0.5087 3.5 34 -0.0075 0.0438 1.4783 2.2693 3.5 49 1.1803 1.4606 -0.344 0.2669 5.2 38 1.0323 0.9719 -0.8251 0.695 5.3

Solution x1 x2 x3 x4 f(x) 37 -3.4331 10 1.0756 0.3657 1104.1 30 3.8599 10 -4.0468 10 9332.4 45 -4.4942 10 3.0653 10 13706.1 83 -0.2414 -6.5307 -0.9449 -9.4168 17134.8 16 6.1626 10 0.1003 0.1103 78973.2

High-Quality Solutions

Diverse Solutions

Page 15: Scatter Search: Methodology and Applicationsyalma.fime.uanl.mx/~roger/work/teaching/mecbs5122/4... · Scatter Search Methodology. 4 Metaheuristic A metaheuristic refers to a master

1515

Subset Generation MethodSubset Generation Method

All pairs of reference solutions that include All pairs of reference solutions that include at least one new solutionat least one new solution

The method generates (b2The method generates (b2--b)/2 pairs from b)/2 pairs from the initial the initial RefSetRefSet

Page 16: Scatter Search: Methodology and Applicationsyalma.fime.uanl.mx/~roger/work/teaching/mecbs5122/4... · Scatter Search Methodology. 4 Metaheuristic A metaheuristic refers to a master

1616

Combination MethodCombination Method

1 2 3 4 5 6 7 8 9 10 11 12 13

1

2

3

4

5

6

7

8

9

10

x2 = (8,4)

x1 = (5,7)

x3 = x1 - r(x2 - x1)x4 = x1 + r(x2 - x1)x5 = x2 + r(x2 - x1)

x4 = (6.5,5.5)r = 1/2

x5 = (11,1)r = 1

x3 = (9,7)r = 2/3

y

x

Page 17: Scatter Search: Methodology and Applicationsyalma.fime.uanl.mx/~roger/work/teaching/mecbs5122/4... · Scatter Search Methodology. 4 Metaheuristic A metaheuristic refers to a master

1717

Alternative Combination MethodAlternative Combination Method

x3

1 2 3 4 5 6 7 8 9 10 11 12 13

1

2

3

4

5

6

7

8

9

10

x2 = (8,4)

x1 = (5,7)

x3 = x1 - r(x2 - x1)x4 = x1 + r(x2 - x1)x5 = x2 + r(x2 - x1)

y

x

x4

x5

Page 18: Scatter Search: Methodology and Applicationsyalma.fime.uanl.mx/~roger/work/teaching/mecbs5122/4... · Scatter Search Methodology. 4 Metaheuristic A metaheuristic refers to a master

1818

Reference Set Update MethodReference Set Update Method

Updated RefSet

Worst

Best1

b

2...

Quality 1

b

2...

Worst

Best

New trialsolution

RefSet of size b

Page 19: Scatter Search: Methodology and Applicationsyalma.fime.uanl.mx/~roger/work/teaching/mecbs5122/4... · Scatter Search Methodology. 4 Metaheuristic A metaheuristic refers to a master

1919

Static UpdateStatic UpdatePool of new trial solutions

RefSet of size b

Worst

BestQuality 1

b

2...

Updated RefSet = Best b from RefSet ∪ Pool

Page 20: Scatter Search: Methodology and Applicationsyalma.fime.uanl.mx/~roger/work/teaching/mecbs5122/4... · Scatter Search Methodology. 4 Metaheuristic A metaheuristic refers to a master

2020

RefSetRefSet after Updateafter Updatexx11 xx22 xx33 xx44

ff((xx))

1.13831.1383 1.29651.2965 0.83060.8306 0.7150.715 0.140.14

0.70160.7016 0.52970.5297 1.20781.2078 1.46331.4633 0.360.36

0.52690.5269 0.2870.287 1.26451.2645 1.60771.6077 0.590.59

1.19631.1963 1.39681.3968 0.68010.6801 0.4460.446 0.620.62

0.33260.3326 0.10310.1031 1.36321.3632 1.83111.8311 0.990.99

0.33680.3368 0.10990.1099 1.38181.3818 1.93891.9389 1.021.02

0.31270.3127 0.09490.0949 1.35121.3512 1.85891.8589 1.031.03

0.75920.7592 0.5230.523 1.31391.3139 1.71951.7195 1.181.18

0.20040.2004 0.03440.0344 1.40371.4037 1.94381.9438 1.241.24

1.38921.3892 1.93051.9305 0.12520.1252 --0.01520.0152 1.451.45

Page 21: Scatter Search: Methodology and Applicationsyalma.fime.uanl.mx/~roger/work/teaching/mecbs5122/4... · Scatter Search Methodology. 4 Metaheuristic A metaheuristic refers to a master

2121

TutorialTutorial

00--1 Knapsack Problem1 Knapsack Problem

Maximize 10x1 + 14x2 + 9x3 + 8x4 + 7x5 + 5x6 + 9x7 + 3x8

S.t. 7x1 + 12x2 + 8x3 + 9x4 + 8x5 + 6x6 + 11x7 + 5x8 < 100

xi = { 0, 1} for i = 1, …, 8

Page 22: Scatter Search: Methodology and Applicationsyalma.fime.uanl.mx/~roger/work/teaching/mecbs5122/4... · Scatter Search Methodology. 4 Metaheuristic A metaheuristic refers to a master

2222

Additional StrategiesAdditional Strategies

Reference SetReference SetRebuildingRebuildingMultiMulti--tiertier

Subset GenerationSubset GenerationSubsets of size > 2Subsets of size > 2

Combination MethodCombination MethodVariable number of solutionsVariable number of solutions

Page 23: Scatter Search: Methodology and Applicationsyalma.fime.uanl.mx/~roger/work/teaching/mecbs5122/4... · Scatter Search Methodology. 4 Metaheuristic A metaheuristic refers to a master

2323

RebuildingRebuildingRefSet Rebuilt RefSet

b1

b2

DiversificationGeneration Method

Reference SetUpdate Method

Page 24: Scatter Search: Methodology and Applicationsyalma.fime.uanl.mx/~roger/work/teaching/mecbs5122/4... · Scatter Search Methodology. 4 Metaheuristic A metaheuristic refers to a master

2424

22--Tier Tier RefSetRefSet

RefSet

b1

b2

Solution CombinationMethod

ImprovementMethod

Try here first

If it fails, thentry here

Page 25: Scatter Search: Methodology and Applicationsyalma.fime.uanl.mx/~roger/work/teaching/mecbs5122/4... · Scatter Search Methodology. 4 Metaheuristic A metaheuristic refers to a master

2525

33--Tier Tier RefSetRefSet

RefSet

b1

b2

Solution CombinationMethod

ImprovementMethod

Try here first

If it fails, thentry here

b3

Try departingsolution here

Page 26: Scatter Search: Methodology and Applicationsyalma.fime.uanl.mx/~roger/work/teaching/mecbs5122/4... · Scatter Search Methodology. 4 Metaheuristic A metaheuristic refers to a master

2626

Subset GenerationSubset GenerationSubset Type 1:Subset Type 1: all 2all 2--element subsets.element subsets.Subset Type 2:Subset Type 2: 33--element subsets derived element subsets derived from the 2from the 2--element subsets by augmenting element subsets by augmenting each 2each 2--element subset to include the best element subset to include the best solution not in this subset.solution not in this subset.Subset Type 3:Subset Type 3: 44--element subsets derived element subsets derived from the 3from the 3--element subsets by augmenting element subsets by augmenting each 3each 3--element subset to include the best element subset to include the best solutions not in this subset.solutions not in this subset.Subset Type 4:Subset Type 4: the subsets consisting of the the subsets consisting of the best best ii elements, for elements, for ii = 5 to b. = 5 to b.

Page 27: Scatter Search: Methodology and Applicationsyalma.fime.uanl.mx/~roger/work/teaching/mecbs5122/4... · Scatter Search Methodology. 4 Metaheuristic A metaheuristic refers to a master

2727

Subsets of Size > 2Subsets of Size > 2

Type 1 Type 2Type 3 Type 4

LOLIB

Random0%10%20%30%40%50%60%70%80%90%

Page 28: Scatter Search: Methodology and Applicationsyalma.fime.uanl.mx/~roger/work/teaching/mecbs5122/4... · Scatter Search Methodology. 4 Metaheuristic A metaheuristic refers to a master

2828

Variable Number of SolutionsVariable Number of SolutionsQuality

Worst

Best1

b

2... Generate 5 solutions

Generate 3 solutions

Generate 1 solution

RefSet of size b

Page 29: Scatter Search: Methodology and Applicationsyalma.fime.uanl.mx/~roger/work/teaching/mecbs5122/4... · Scatter Search Methodology. 4 Metaheuristic A metaheuristic refers to a master

2929

Hybrid ApproachesHybrid Approaches

Use of MemoryUse of MemoryTabuTabu Search mechanisms for intensification Search mechanisms for intensification and diversificationand diversification

GRASP ConstructionsGRASP ConstructionsCombination MethodsCombination Methods

GA OperatorsGA OperatorsPath Path RelinkingRelinking

Page 30: Scatter Search: Methodology and Applicationsyalma.fime.uanl.mx/~roger/work/teaching/mecbs5122/4... · Scatter Search Methodology. 4 Metaheuristic A metaheuristic refers to a master

3030

MultiobjectiveMultiobjective Scatter SearchScatter Search

This is a fruitful research areaThis is a fruitful research area

Many Many multiobjectivemultiobjective evolutionary evolutionary approaches exist (approaches exist (CoelloCoello, et al. 2002), et al. 2002)

SS can use similar techniques developed SS can use similar techniques developed for MOEA (for MOEA (multiobjectivemultiobjective evolutionary evolutionary approchesapproches))

Page 31: Scatter Search: Methodology and Applicationsyalma.fime.uanl.mx/~roger/work/teaching/mecbs5122/4... · Scatter Search Methodology. 4 Metaheuristic A metaheuristic refers to a master

3131

MultiobjectiveMultiobjective EA TechniquesEA Techniques

Independent SamplingIndependent SamplingSearch on Search on ff((xx)) = = ∑∑wwii ffii((xx))Change weights and rerunChange weights and rerun

Criterion SelectionCriterion SelectionDivide reference set into Divide reference set into kk subsetssubsetsAdmission to Admission to iithth subset is according to subset is according to ffii((xx))

Page 32: Scatter Search: Methodology and Applicationsyalma.fime.uanl.mx/~roger/work/teaching/mecbs5122/4... · Scatter Search Methodology. 4 Metaheuristic A metaheuristic refers to a master

Scatter SearchScatter SearchApplications and ImplementationsApplications and Implementations

Page 33: Scatter Search: Methodology and Applicationsyalma.fime.uanl.mx/~roger/work/teaching/mecbs5122/4... · Scatter Search Methodology. 4 Metaheuristic A metaheuristic refers to a master

3333

ScatterScatter Search Search ElementsElements

Diversification Generator MethodDiversification Generator MethodImprovement MethodImprovement MethodReference Set Reference Set

InitializationInitializationUpdateUpdateRebuildRebuild

Subset Generation Method Subset Generation Method Solution Combination MethodSolution Combination Method

Problem Independent

Page 34: Scatter Search: Methodology and Applicationsyalma.fime.uanl.mx/~roger/work/teaching/mecbs5122/4... · Scatter Search Methodology. 4 Metaheuristic A metaheuristic refers to a master

3434

C C CodeCode ObjectiveObjective

Illustrate the methods and strategiesIllustrate the methods and strategiesUseUse itit in different applicationsin different applicationsExtend the previous version:Extend the previous version:

http://http://www.uv.es/~rmarti/sscode.htmlwww.uv.es/~rmarti/sscode.htmlModularityModularityVersatilityVersatility

Non expertsNon experts (use as (use as itit isis withwith smallsmall changeschanges))Experts (can Experts (can easilyeasily introduce introduce importantimportant modificationsmodifications))

AnsiAnsi C (different platforms and systems)C (different platforms and systems)

Page 35: Scatter Search: Methodology and Applicationsyalma.fime.uanl.mx/~roger/work/teaching/mecbs5122/4... · Scatter Search Methodology. 4 Metaheuristic A metaheuristic refers to a master

3535

Basic Basic DesignDesign

P

Diversification GenerationMethod

Repeat until |P| = PSize

Subset GenerationMethod

ImprovementMethod

Solution CombinationMethod

ImprovementMethod

Stop if no morenew solutions

Reference SetUpdate Method

RefSet

Page 36: Scatter Search: Methodology and Applicationsyalma.fime.uanl.mx/~roger/work/teaching/mecbs5122/4... · Scatter Search Methodology. 4 Metaheuristic A metaheuristic refers to a master

3636

RefSet Data RefSet Data StructureStructuretypedeftypedef structstruct REFSETREFSET

{{ intint b;b; /* Size/* Size */*/

doubledouble **sol;**sol; /* Solutions/* Solutions */*/

doubledouble **ObjValObjVal;; //* Objective value of solutions* Objective value of solutions */*/

intint *order;*order; /* Order of solutions *//* Order of solutions */

intint **iteriter;; /* Entering iteration *//* Entering iteration */

intint NewSolutionsNewSolutions;; /* =1 if new element has been added *//* =1 if new element has been added */

} } REFSETREFSET;;

bsol

ObjValorderiter

NewSolutions

RefSet nvar

Page 37: Scatter Search: Methodology and Applicationsyalma.fime.uanl.mx/~roger/work/teaching/mecbs5122/4... · Scatter Search Methodology. 4 Metaheuristic A metaheuristic refers to a master

3737

MainMain FunctionFunctionintint main(void)main(void){{

SS *SS *pbpb;; /* Pointer to problem data/* Pointer to problem data **//intint nvarnvar = 10;= 10; /* Number of variables/* Number of variables */*/intint b = 10;b = 10; /* Size of reference set/* Size of reference set **//intint PSizePSize = 100;= 100; /* Size of P/* Size of P */*/

pbpb = = SSProblem_DefinitionSSProblem_Definition(nvar,b,PSize(nvar,b,PSize););

/* Insert here the Scatter Search code *//* Insert here the Scatter Search code */

SSFree_DataStructuresSSFree_DataStructures(pb(pb););return 0;return 0;

}}

Page 38: Scatter Search: Methodology and Applicationsyalma.fime.uanl.mx/~roger/work/teaching/mecbs5122/4... · Scatter Search Methodology. 4 Metaheuristic A metaheuristic refers to a master

3838

Basic Basic DesignDesign

pbpb = = SSProblem_Definition(nvar,b,PSizeSSProblem_Definition(nvar,b,PSize););

SSCreate_PSSCreate_P(pb(pb););

SSCreate_RefSetSSCreate_RefSet(pb(pb););

WhileWhile((pbpb-->>rsrs-->>NewSolutionsNewSolutions))

SSUpdate_RefSetSSUpdate_RefSet(pb(pb););

SSBestSol(pb,sol,&valueSSBestSol(pb,sol,&value););

SSFree_DataStructures(pbSSFree_DataStructures(pb););

Page 39: Scatter Search: Methodology and Applicationsyalma.fime.uanl.mx/~roger/work/teaching/mecbs5122/4... · Scatter Search Methodology. 4 Metaheuristic A metaheuristic refers to a master

3939

AdvancedAdvanced DesignDesignpbpb = = SSProblem_Definition(nvar,b,PSizeSSProblem_Definition(nvar,b,PSize););SSCreate_PSSCreate_P(pb(pb););SSCreate_RefSetSSCreate_RefSet(pb(pb););

for(Iterfor(Iter=1; =1; IterIter<<100100; ; IterIter++)++){{

if(pbif(pb-->>rsrs-->>NewSolutionsNewSolutions))SSUpdate_RefSetSSUpdate_RefSet(pb(pb););

elseelseSSRebuild_RefSetSSRebuild_RefSet(pb(pb););

}}SSBestSol(pb,sol,&valueSSBestSol(pb,sol,&value););SSFree_DataStructures(pbSSFree_DataStructures(pb););

Page 40: Scatter Search: Methodology and Applicationsyalma.fime.uanl.mx/~roger/work/teaching/mecbs5122/4... · Scatter Search Methodology. 4 Metaheuristic A metaheuristic refers to a master

4040

SSCreate_PSSCreate_Pwhile(currentPSizewhile(currentPSize <=<= pbpb-->p>p-->>PSizePSize)){{SSGenerate_Sol(pb,solSSGenerate_Sol(pb,sol););obj_valobj_val = sol_value(sol);= sol_value(sol);SSImprove_solutionSSImprove_solution(pb,sol,&obj_val(pb,sol,&obj_val););

/* Check whether sol is a new solution *//* Check whether sol is a new solution */j=1;equal=0;j=1;equal=0;while(j<while(j<currentPSizecurrentPSize && !equal)&& !equal)

equal=equal=SSEqualSolSSEqualSol(sol,pb(sol,pb-->p>p-->>sol[j++],pbsol[j++],pb-->>nvarnvar););

/* Add improved solution to P *//* Add improved solution to P */if(!equal){if(!equal){

for(j=1;j<=for(j=1;j<=pbpb-->>nvar;jnvar;j++)++)pbpb-->p>p-->>sol[currentPSize][jsol[currentPSize][j]=sol[j];]=sol[j];

pbpb-->p>p-->>ObjVal[currentPSizeObjVal[currentPSize++] = ++] = obj_valobj_val;;}}

}}

Page 41: Scatter Search: Methodology and Applicationsyalma.fime.uanl.mx/~roger/work/teaching/mecbs5122/4... · Scatter Search Methodology. 4 Metaheuristic A metaheuristic refers to a master

4141

SSCreate_RefSetSSCreate_RefSet/* Order solutions in P *//* Order solutions in P */p_order = p_order = SSOrderSSOrder(pb(pb-->p>p-->>ObjValObjVal, , pbpb-->p>p-->>PSizePSize, , pbpb-->Opt);>Opt);b1 = b1 = pbpb-->>rsrs-->b / 2;>b / 2;

/* Add the /* Add the firstfirst ((bestbest) b1) b1 solutions in P to RefSet */solutions in P to RefSet */

/* Compute minimum distances from P to RefSet *//* Compute minimum distances from P to RefSet */for(i=1;i<=for(i=1;i<=pbpb-->p>p-->>PSize;iPSize;i++)++)

min_dist[i]= min_dist[i]= SSDist_RefSetSSDist_RefSet(pb,b1,pb(pb,b1,pb-->p>p-->sol[i]);>sol[i]);

/* Add b/* Add b--b1 diverse solutions to RefSet */b1 diverse solutions to RefSet */for(i=b1+1;i<=for(i=b1+1;i<=pbpb-->>rsrs-->b;i++)>b;i++) {{

a=a=SSMax_dist_index(pb,min_distSSMax_dist_index(pb,min_dist););/* Copy sol a from P to /* Copy sol a from P to RefsetRefset */*/SSUpdate_distancesSSUpdate_distances(pb,min_dist,i(pb,min_dist,i););

}}/* Compute the order in RefSet: /* Compute the order in RefSet: orderorder */*/pbpb-->>rsrs-->>NewSolutionsNewSolutions == 1;1; pbpb-->>CurrentIterCurrentIter = 1;= 1;

Page 42: Scatter Search: Methodology and Applicationsyalma.fime.uanl.mx/~roger/work/teaching/mecbs5122/4... · Scatter Search Methodology. 4 Metaheuristic A metaheuristic refers to a master

4242

SSUpdate_RefSetSSUpdate_RefSet

pbpb-->>rsrs-->>NewSolutionsNewSolutions=0;=0;

SSCombine_RefSet(pbSSCombine_RefSet(pb););

pbpb-->>CurrentIterCurrentIter++;++;

for(a=1;a<=for(a=1;a<=pbpb-->pool_size;a++)>pool_size;a++)

{{

value=value=sol_value(pbsol_value(pb-->pool[a]);>pool[a]);

SSImprove_solution(pb,pbSSImprove_solution(pb,pb-->pool[a],&value);>pool[a],&value);

SSTryAdd_RefSet(pb,pbSSTryAdd_RefSet(pb,pb-->pool[a],value);>pool[a],value);

}}

pbpb-->>pool_sizepool_size=0;=0;

Page 43: Scatter Search: Methodology and Applicationsyalma.fime.uanl.mx/~roger/work/teaching/mecbs5122/4... · Scatter Search Methodology. 4 Metaheuristic A metaheuristic refers to a master

4343

AdvancedAdvanced DesignsDesigns

ReferenceReference SetSet UpdateUpdateDynamicDynamic / / StaticStatic2 2 TierTier / 3 / 3 TierTier

Subset Subset GenerationGenerationUse of Use of MemoryMemory

ExplicitExplicit MemoryMemoryAttributiveAttributive MemoryMemory

PathPath RelinkingRelinking

Page 44: Scatter Search: Methodology and Applicationsyalma.fime.uanl.mx/~roger/work/teaching/mecbs5122/4... · Scatter Search Methodology. 4 Metaheuristic A metaheuristic refers to a master

AnAn ExampleExampleTheThe Linear Ordering ProblemLinear Ordering Problem

Given a matrix of weights E = {eij}mxm, the LOP consists of finding a permutation p of the columns (and rows) in order to maximize the sum of the weights in the upper triangle

ApplicationsTriangulation for Input-Output Economic Tables.Aggregation of individual preferencesClassifications in Sports

C p eE p pj i

m

i

m

i j( ) =

= +=

∑∑11

1

Maximize

Page 45: Scatter Search: Methodology and Applicationsyalma.fime.uanl.mx/~roger/work/teaching/mecbs5122/4... · Scatter Search Methodology. 4 Metaheuristic A metaheuristic refers to a master

An An InstanceInstance

1234

1 2 3 4

3412

3 4 1 20 9 8 32 0 11 45 3 0 122 3 4 0

⎢⎢⎢⎢

⎥⎥⎥⎥

0 12 5 34 0 2 68 3 0 911 4 2 0

⎢⎢⎢⎢

⎥⎥⎥⎥

p=(1,2,3,4)

cE(p)=12+5+3+2+6+9=37

p*=(3,4,1,2)

cE(p*)=9+8+3+11+4+12=47

Page 46: Scatter Search: Methodology and Applicationsyalma.fime.uanl.mx/~roger/work/teaching/mecbs5122/4... · Scatter Search Methodology. 4 Metaheuristic A metaheuristic refers to a master

4646

SS SS forfor thethe LOPLOP

Complete the “generic” codeComplete the “generic” code

Design specific methods for the “problemDesign specific methods for the “problem--dependent” elementsdependent” elements

Diversification Generator MethodDiversification Generator MethodImprovement MethodImprovement MethodCombination MethodCombination Method

Page 47: Scatter Search: Methodology and Applicationsyalma.fime.uanl.mx/~roger/work/teaching/mecbs5122/4... · Scatter Search Methodology. 4 Metaheuristic A metaheuristic refers to a master

4747

Diversification GeneratorDiversification GeneratorUse of Use of problemproblem structurestructure toto createcreate methodsmethods in in orderorder totoachieveachieve a a goodgood balance balance betweenbetween qualityquality andand diversitydiversity..QualityQuality

DeterministicDeterministic constructiveconstructive methodmethodDiversityDiversity

RandomRandom GeneratorGeneratorSystematicSystematic GeneratorsGenerators (Glover, 1998)(Glover, 1998)

GRASP constructions. GRASP constructions. The method randomly selects from a short list of the most The method randomly selects from a short list of the most attractive sectors. attractive sectors.

Use of Use of MemoryMemoryMModifyingodifying a measure of attractiveness proposed by Becker with a measure of attractiveness proposed by Becker with a frequencya frequency--based memory measure that discourages sectors based memory measure that discourages sectors from occupying positions that they have frequently occupiedfrom occupying positions that they have frequently occupied

Page 48: Scatter Search: Methodology and Applicationsyalma.fime.uanl.mx/~roger/work/teaching/mecbs5122/4... · Scatter Search Methodology. 4 Metaheuristic A metaheuristic refers to a master

4848

Diversity vs. QualityDiversity vs. Quality

Compare several diversification generatorsCreate a set of 100 solutions with each one

0.0

0.2

0.4

0.6

0.8

1.0

1.2

1.4

1.6

DG05 DG04 DG02 DG01 DG09 DG08 DG06 DG03 DG07 DG10

Procedure

∆C

∆d

∆C+∆d

∆d = Standardized Diversity∆C = Standardized Quality

Page 49: Scatter Search: Methodology and Applicationsyalma.fime.uanl.mx/~roger/work/teaching/mecbs5122/4... · Scatter Search Methodology. 4 Metaheuristic A metaheuristic refers to a master

4949

ImprovementImprovement MethodMethod

INSERT_MOVE (INSERT_MOVE (pjpj, i) consist of deleting , i) consist of deleting pjpj from from its current position j to be inserted in position i its current position j to be inserted in position i Apply a first strategyApply a first strategy

scans the list of sectors in search for the first sector scans the list of sectors in search for the first sector whose movement results in an improvement whose movement results in an improvement

E p( ) = − −−

⎜⎜⎜⎜⎜⎜⎜⎜⎜

⎟⎟⎟⎟⎟⎟⎟⎟⎟

0 12 5 3 1 8 36 0 3 6 4 4 28 5 0 5 7 0 32 7 2 0 3 6 08 0 3 1 0 4 19 1 6 2 13 0 42 9 4 5 8 1 0

MoveValue = CE(p’) - CE(p)

CE (p’) = 78 + (1 - 4) + (6 - 0) + (2 - 6) + (13 - 4) = 78 + 8 = 86

Page 50: Scatter Search: Methodology and Applicationsyalma.fime.uanl.mx/~roger/work/teaching/mecbs5122/4... · Scatter Search Methodology. 4 Metaheuristic A metaheuristic refers to a master

5050

Solution Combination MethodSolution Combination MethodThe method scans (from left to right) each reference The method scans (from left to right) each reference permutation.permutation.

Each reference permutation votes for its first element that is sEach reference permutation votes for its first element that is still till not included in the combined permutation (“incipient element”). not included in the combined permutation (“incipient element”). The voting determines the next element to enter the first still The voting determines the next element to enter the first still unassigned position of the combined permutation.unassigned position of the combined permutation.The vote of a given reference solution is weighted according to The vote of a given reference solution is weighted according to the incipient element’s position.the incipient element’s position.

Incipient element(3,1,4,2,5) votes for 4 Solution under construction:(1,4,3,5,2) votes for 4 (3,1,2,4,_ )(2,1,3,5,4) votes for 5

Page 51: Scatter Search: Methodology and Applicationsyalma.fime.uanl.mx/~roger/work/teaching/mecbs5122/4... · Scatter Search Methodology. 4 Metaheuristic A metaheuristic refers to a master

5151

ExperimentsExperiments withwith LOLIB LOLIB

49 Input-Output Economic Tables

GDGD CKCK CK10CK10 TSTS SSSS

OOptimaptimadevdeviationiation

0.15%0.15% 0.15%0.15% 0.02%0.02% 0.04%0.04% 0.01%0.01%

NNumber of umber of optimaoptima

1111 1111 2727 3333 4242

RRun time un time (sec(secondsonds))

0.010.01 0.100.10 1.061.06 0.490.49 2.352.35

Page 52: Scatter Search: Methodology and Applicationsyalma.fime.uanl.mx/~roger/work/teaching/mecbs5122/4... · Scatter Search Methodology. 4 Metaheuristic A metaheuristic refers to a master

5252

Another ExampleAnother ExampleA commercial SS implementationA commercial SS implementation

OptQuest Callable Library (by OptTek)As other context-independent methodsseparates the method and the evaluation.

OptimizationProcedure

Input

Output

SystemEvaluator

Page 53: Scatter Search: Methodology and Applicationsyalma.fime.uanl.mx/~roger/work/teaching/mecbs5122/4... · Scatter Search Methodology. 4 Metaheuristic A metaheuristic refers to a master

5353

OptQuestOptQuest basedbased ApplicationsApplications

Solution Generator

Solution EvaluatorUser-writtenApplication

OptQuestCallable Library

SystemEvaluator

Page 54: Scatter Search: Methodology and Applicationsyalma.fime.uanl.mx/~roger/work/teaching/mecbs5122/4... · Scatter Search Methodology. 4 Metaheuristic A metaheuristic refers to a master

5454

FeasibilityFeasibility andand EvaluationEvaluation

User Implementation

ConstraintMapping

Complex SystemEvaluator

PenaltyFunction

x x*F(x*)

G(x*)P(x*)

Returns toOptQuest

The OptQuest enginegenerates a new solution

Page 55: Scatter Search: Methodology and Applicationsyalma.fime.uanl.mx/~roger/work/teaching/mecbs5122/4... · Scatter Search Methodology. 4 Metaheuristic A metaheuristic refers to a master

5555

ComparisonComparison withwith GenocopGenocopTestTest onon 28 28 hardhard nonlinearnonlinear instancesinstances

1.0E+03

1.0E+04

1.0E+05

1.0E+06

1.0E+07

1.0E+08

1.0E+09

1.0E+10

1.0E+11

1.0E+12

1.0E+13

0 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000

Evaluations

Ave

rage

obj

ectiv

e fu

nctio

n va

lue

(Log

arith

mic

scal

e)

GenocopOCL

1.0E+03

1.0E+04

1.0E+05

1.0E+06

1.0E+07

1.0E+08

1.0E+09

1.0E+10

1.0E+11

1.0E+12

1.0E+13

0 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000

Evaluations

Ave

rage

obj

ectiv

e fu

nctio

n va

lue

(Log

arith

mic

scal

e)

GenocopOCL

Page 56: Scatter Search: Methodology and Applicationsyalma.fime.uanl.mx/~roger/work/teaching/mecbs5122/4... · Scatter Search Methodology. 4 Metaheuristic A metaheuristic refers to a master

5656

ConclusionsConclusions

The development of metaheuristics usually The development of metaheuristics usually entails a fair amount of experimentation entails a fair amount of experimentation (“skill comes from practice”).(“skill comes from practice”).Code objectives:Code objectives:

Quick StartQuick StartBenchmark Benchmark Advanced DesignsAdvanced Designs

Scatter Search provides a flexible Scatter Search provides a flexible “framework” to develop solving methods“framework” to develop solving methods..

Page 57: Scatter Search: Methodology and Applicationsyalma.fime.uanl.mx/~roger/work/teaching/mecbs5122/4... · Scatter Search Methodology. 4 Metaheuristic A metaheuristic refers to a master

5757

CallCall forfor PapersPapers

European Journal of Operational ResearchEuropean Journal of Operational Research

Feature Issue on Feature Issue on SCATTER SEARCH METHODSSCATTER SEARCH METHODS

FOR OPTIMIZATIONFOR OPTIMIZATION

Deadline for submissions: June 30, 2003Deadline for submissions: June 30, 2003http://http://www.uv.es/~rmarti/ejor.htmlwww.uv.es/~rmarti/ejor.html

Page 58: Scatter Search: Methodology and Applicationsyalma.fime.uanl.mx/~roger/work/teaching/mecbs5122/4... · Scatter Search Methodology. 4 Metaheuristic A metaheuristic refers to a master

5858

QuestionsQuestions