Hidden (Tabu) Secrets of Successful Evolutionary Search Methods

Peng-Yeng YinNational Chi Nan University, TaiwanFred GloverUniversity of Colorado, Boulder, CO, USA

2007 IEEE Congress on Evolutionary Computation

P.Y. Yin & Fred Glover 2

New Directions for Evolutionary Algorithms

• We are witnessing a radical departure from the early 1990s

• Original theme for Solution Combination: restrict attention to a handful of genetically inspired “crossover” operations. (Basic simple approaches suffice.)

• Today emphasize: – New solution codings

– New forms of crossover

– Exploit problem-specific knowledge

P.Y. Yin & Fred Glover 3

Limitations Still Exist!

Regarding Solution Combinations and Their Use

We still often focus on narrow metaphors for combining solutions

• Advantages of the metaphors: – Stimulate researchers to create methods based on the

metaphors

– Catch the attention of wide audiences

(Easy to describe. Great sources of publicity!)

• Disadvantages of the metaphors: – Create a biased perspective about preferred forms of

methods

– Cause valuable alternative strategies to be overlooked

P.Y. Yin & Fred Glover 4

Remedy – An Orientation Shift

• View Solution Combinations as a process of Path Relinking

• Employ frameworks of Adaptive Structured Combinations

• Exploit the neighborhood concept in generating combinations

• Integrate with adaptive memory

P.Y. Yin & Fred Glover 5

Structured Combinations for Populations of Vectors

• Property 1 (Representation Property). Each vector represents a set of votes for particular decisions

• Property 2 (Trial Solution Property). The set of votes translates into a trial solution by a well-defined process

• Property 3 (Update Property). If a decision is made according to the votes of a given vector, thus producing a residual problem requiring fewer decisions, a well-defined rule exists to update all vectors relative to the residual problem so that Properties 1 and 2 continue to hold

(Details and illustrations in F. Glover (1994) “Tabu search for nonlinear and parametric optimization (with links to genetic algorithms),” Discrete Applied Mathematics 49, 221-235.)

P.Y. Yin & Fred Glover 6

Path Relinking

XA

B

C

Using A and B as guiding solutions

Using B and C as guiding solutions

Using C as a guiding solution

X is the initial solution

• Valuable information can be contained in trajectories from bad solutions to good solutions (or from good solutions to other good solutions)

• Path relinking selects moves that introduce attributes contained in the guiding solution(s) into the initial solution. Basically only one attribute of the initial solution is modified at each time a move is made.

• Instead of interchanging information between solutions in a wholesale fashion, path relinking is a stepwise approach considering neighborhood structure and adaptive memory strategies that may generate solutions not accessible by EAs

P.Y. Yin & Fred Glover 7

Scatter Search

• Useful information about the form or location of the global solution is typically contained in a sufficiently diverse collection of elite solutions

Repeat until |P| = PSizeP

Diversification GenerationMethod

Subset GenerationMethod

ImprovementMethod

Solution Combination Method

ImprovementMethod

No more new solutions

Reference SetUpdate Method

RefSet

Diversification GenerationMethod

ImprovementMethod

Stop if MaxIterreached

P.Y. Yin & Fred Glover 8

Neighborhood Construction

• Candidate list strategy– Effectively construct a neighborhood structure to

save computational time or even to construct a more complex (or compound) neighborhood that is not accessible using simple moves. It accommodates variable neighborhood search (VNS) method Ejection chain Filter and fan

P.Y. Yin & Fred Glover 9

Ejection Chain

• Construct neighborhood by combining successive interindependent (component) moves to form a single compound move.

• Tunnel infeasible region by successive ejection moves and transform to a feasible solution by a trial move.

• This form of compound neighborhood structures are usually not accessible by feasibility-preserving search methods.

t tip

rs2s1

rootsubrootsubroot

t tip

rs2s1

rootsubrootsubroot

Ejection move Trial move

Traveling salesman problem

P.Y. Yin & Fred Glover 10

Filter and Fan

• The neighborhood tree is explored breadth first and is restricted by a maximum number of levels L. Each level is governed by the filter strategy that selects a subset of moves induced by the fan strategy

• Filter & Fan is usually applied after a local heuristic has been executed to explore a larger neighborhood in order to overcome local optimality

• The two search strategies are alternated when a new local optimum is found until the Filter & Fan fails to improve the current best solution

Protein folding problem

P.Y. Yin & Fred Glover 11

Strategic Oscillation

• In many cases the elite (or globally optimal) solution lies on the feasibility boundary or the search method would stop at a critical level

• Strategic oscillation drives the search toward or away from an oscillation boundary. The approach proceeds for a specified depth beyond the boundary, and turns around. The boundary again is approached and crossed from the opposite direction.

• The oscillatory behavior is established by generating modified evaluations and rules of movement, depending on the regional locality and trajectory direction

• Choose oscillation pattern and change rate to achieve an effective interplay between intensification and diversification

Oscillation boundary

Depth

Iterations

P.Y. Yin & Fred Glover 12

Adaptive Memory Programming

• Adaptive memory programming (Tabu Search) constitutes of adaptive memory and a set of responsive strategies

Adaptive memory: purposefully comparing previous states or transactions to those currently contemplated Recency, frequency, influence, quality

Responsive strategies: take advantage of adaptive memory to exploit good solution features while explore new promising regions Tabu restriction, aspiration criteria

Intensification, diversification

P.Y. Yin & Fred Glover 13

Adaptive Memory Programming

• Adaptive memory programming (Tabu Search) Tabu restriction: In each move iteration, the best mov

e (evaluated in aspects of quality, influence, frequency) in the neighborhood is selected unless it is tabu. Tabu tenure (static/dynamic/reactive) is determined according to the adaptive memory

Aspiration criteria: The tabu restriction can be overruled if the corresponding move meets the aspiration criteria such that the search can be guided to a promising region along the course

Intensification/diversification strategies: applying incentives/penalties to induce attributes of good solutions

path relinking, strategic oscillation, multi-start

P.Y. Yin & Fred Glover 14

Create Hybrid – Integrate EA and TS

Solution combination Path relinking exploiting neighborhood structure and adaptive memory – directly address problem context without metaphor restrictions – stepwise and systematic, reducing the probability of missing valuable information on the course

Solution improvement Adaptive memory strategies can be directly applied to improve the combined

solution – accommodate implicit variable neighborhood search that prevents from local optimality

Population distribution control Tabu search has provided a wealth of diversity control strategies – tabu restrictions, altering move evaluations, strategic oscillation

Constraint handling Strategic oscillation repeatedly drives the search across the feasibility boundar

y – non-monotonically change the mixes of feasible and infeasible solutions

P.Y. Yin & Fred Glover 15

For Additional Background

Using Google search:  

“Path Relinking” returns about 28,000 web pages.

“Scatter Search” returns about 50,000 web pages.

 

The first references encountered on Google give a good background for basic understanding.

(Incidental remark: “Tabu Search” returns about 530,000 web pages.)

P.Y. Yin & Fred Glover 16

• Portfolio Management• Supply Chain Applications• Strategic and Operational Planning• Financial Planning• Manufacturing Process Flow • Resource-Constrained Scheduling• Network Planning• Routing & Distribution• Data Mining• Biotechnology• Health Care

OptTek Customized Simulation Optimization Applications

P.Y. Yin & Fred Glover 17

MetaheuristicOptimizer

SimulationModel

Input parameters Objective function value

Metaheuristic – Based Simulation Optimization

P.Y. Yin & Fred Glover 18

• Function to be Optimized• Highly Nonlinear• Nondifferentiable• Discrete or Continuous or Mixed

• Function Evaluations• Complex• Extremely Computation Intensive• One second to One Day per Evaluation!

The Optimization Challenge

P.Y. Yin & Fred Glover 19

• Evolutionary Scatter Search• Advanced Tabu Search• Linear & Mixed Integer Programming• Pattern Classification & Curve Fitting

• Neural Networks

• Support Vector Machines & Trees

• SAT Data Mining

OptQuest® Components

P.Y. Yin & Fred Glover 20

Example 5 Problem 14 Best solution = -8695.012285

-4397.23 Risk Pop 10-4576.85 Risk Pop 20

-4272.22 Risk Pop 50

-4765.34 Risk Pop 100

-8543.49 OptQuest Pop 20

-8695.01 OCL Boundary=.7

-9000

-8000

-7000

-6000

-5000

-4000

-3000

0 200 400 600 800 1000 1200 1400 1600 1800 2000

Simulations

Ob

ject

ive Efficiency is Critical!

OptQuest® vs. RiskOptimizer

P.Y. Yin & Fred Glover 21

• Given a set of opportunities and limited resources…

• …determine the best set of projects that maximizes performance

Problem

P.Y. Yin & Fred Glover 22

• Constraints: • Budget • Resource Availability • Scheduling and Sequencing of Projects • Project Dependencies, etc.

• Objectives:• Maximize Net Present Value (NPV)• Maximize Internal Rate of Return (IRR)• Maximize Business-Case Value (BCV)

Portfolio Selection Problem

P.Y. Yin & Fred Glover 23

• 5 Projects:• Tight Gas Play Scenario (TGP)• Oil – Water Flood Prospect (OWF)• Dependent Layer Gas Play Scenario (DL)• Oil – Offshore Prospect (OOP)• Oil – Horizontal Well Prospect (OHW)

• Ten year models that incorporate multiple types of uncertainty

• Evaluation Time: 1s / Scenario

Application Example

P.Y. Yin & Fred Glover 24

Determine project participation levels [0,1] that

• Maximize E(NPV)

• Keep < 10,000 M$ (Risk Control)

• All projects start in year 1

Frequency Chart

M$

Mean = $37,393.13.000

.007

.014

.021

.028

0

7

14

21

28

$15,382.13 $27,100.03 $38,817.92 $50,535.82 $62,253.71

1,000 Trials 16 Outliers

Forecast: NPV

Base Case

TGP = 0.4, OWF = 0.4, DL = 0.8, OHW = 1.0

E(NPV) = 37,393 =9,501

Base Case

P.Y. Yin & Fred Glover 25

Determine project participation levels [0,1] AND

starting times for each project that• Maximize E(NPV) • Keep < 10,000 M$ (Risk Control)• Projects may start in year 1, 2, or 3

Frequency Chart

M$

Mean = $47,455.10.000

.007

.014

.020

.027

0

6.75

13.5

20.25

27

$25,668.28 $37,721.53 $49,774.78 $61,828.04 $73,881.29

1,000 Trials 8 Outliers

Forecast: NPV

TGP1 = 0.6, DL1=0.4, OHW3=0.2

E(NPV) = 47,455 =9,513 10th Pc.=36,096

Deferment Case

Deferment Case

P.Y. Yin & Fred Glover 26

Determine project participation levels AND

starting times for each project that

• Maximize P(NPV > 47,455 M$)

• Keep 10th Percentile of NPV > 36,096 M$

• Projects may start in year 1, 2, or 3

Frequency Chart

M$

Mean = $83,971.65.000

.008

.016

.024

.032

0

8

16

24

32

$43,258.81 $65,476.45 $87,694.09 $109,911.73 $132,129.38

1,000 Trials 13 Outliers

Forecast: NPV

TGP1 = 1.0, OWF1=1.0, DL1=1.0, OHW3=0.2

E(NPV) = 83,972 =18,522 P(NPV > 47,455) = 0.99 10th Pc.=43,359

Probability of Success Case

Probability of Success Case

P.Y. Yin & Fred Glover 27

• Cash Flow Control• Capital Expenditure Control• Reserve Replacement Goals• Production Goals• Finding Costs Control• Dry Hole Expectations Control• Reserve Goals• Net Profit Goals

Extensions…

P.Y. Yin & Fred Glover 28

Treatment

Patient Arrival

Emergency Room (ER)

Approach= optimize current process, redesign process and re-optimize.

Objective = minimize expected total asset

cost while ensuring a reasonable average

patient cycle time

Release

Admit

Joseph DeFee, CACI, Inc.

Hospital Emergency Room Process

P.Y. Yin & Fred Glover 29

• Nurses• Physicians• Patient Care Technicians (PCTs)• Administrative Clerks• Emergency Rooms (ER)

ER Resources

P.Y. Yin & Fred Glover 30

• Minimize E[Total Asset Cost]• Subject to:

– E[Cycle Time] for Level 1 Patients < 2.4 hours

– Number of Nurses between 1 and 7

– Number of Physicians between 1 and 3

– Number of PCTs between 1 and 4

– Number of Clerks between 1 and 4

– Number of ER between 1 and 20

Problem

P.Y. Yin & Fred Glover 31

• Set up OptQuest to run for 100 iterations and 5 runs per iteration

• Each run simulates 5 days of ER operation

• Results:– Best solution found in 6 minutes

– E[TAC] = $ 25.2K (31% improvement)

– E[CT] for P1 = 2.17 hours

Solution

P.Y. Yin & Fred Glover 32

Possible to improve E[CT] for P1 even further?

Arrive at ER

Transfer toroom

Receivetreatment

Fill outregistration OK? Released

AdmittedInto

Hospital

Y

N

Current Process

Arrive at ER

Transfer toroom

Receivetreatment

Fill outregistration

OK? Released

AdmittedInto

Hospital

Y

N

Redesigned Process

Process Redesign

P.Y. Yin & Fred Glover 33

• Set up OptQuest to run for 100 iterations and 5 runs per iteration

• Each run simulates 5 days of ER operation

• Results:– Best solution found in 8 minutes

– E[TAC] = $ 24.6K (new best, 3.4% improvement)

– E[CT] for P1 = 1.94 hours (12% improvement)

Solution of the Redesigned Process

P.Y. Yin & Fred Glover 34

• Simulation Optimization with OptQuest is able to find high-quality solutions in reasonable time and re-optimizes the redesigned model

• These applications are only a fraction of the ways that metaheuristics and simulation are used in optimization involving non-linearity and uncertainty

• Over 60,000 user licenses of the system have been sold (each licensed user might have multiple kinds of problems)

Simulation Optimization with Metaheuristic

P.Y. Yin & Fred Glover 35

• The genetic metaphors served by most EAs today have caused valuable alternative strategies from human problem solving to be overlooked

• The human brain and its higher level processes already exist. The adaptive memory strategies directly take advantage of human intelligence that creates another dimension to adapt to problem solving environment

Conclusions

P.Y. Yin & Fred Glover 36

• The successes by integrating scatter search and its path relinking extensions with tabu search disclose potential advantages for evolutionary algorithms that incorporate adaptive memory

• It’s time for a new generation of evolutionary algorithms – Cyber-Evolutionary Algorithms – that are evolutionary processes based on neighborhood structures, adaptive memory, and responsive strategies

Conclusions