djamila ouelhadj, simon martin, patrick beullens and ender Özcan logistics and management...

*Djamila Ouelhadj, *Simon Martin, **Patrick Beullens and ***Ender Özcan

*Logistics and Management Mathematics Group, Department of Mathematics University of Portsmouth

**School of Mathematics - School of Management,University of Southampton

***Automated Scheduling, optimisAtion And Planning (ASAP)School of Computer Science University of Nottingham

Cooperative search and hyper-heuristics in combinatorial optimisation

Outline of the talk

Introduction and aims

Hyper-heuristics

Agent based framework for cooperative search

Case Studies: Permutation flow shop, nurse rostering

Computational results

Conclusions

Future work

Motivations

• Meta-heuristics have been successfully used to solve a wide range of optimisation problems.

• Meta-heuristics developed for a specific problem domain cannot necessarily achieve the same performance on the same instances or instances from another problem domain.

• This frequently requires probably parameter tuning and/or design of new neighbourhood operators for the new problem domain.

Motivations

Which heuristic or meta-heuristic is best for my problem?

There is almost no guidance available in choosing the best meta-heuristic for solving a problem in hand?

Motivations

To build a generic framework where cooperating agents can use different heuristics and meta-heuristics to solve complex OR problems and increase the level of generality.

Motivations

The key ideas of the generic framework:

• Automated selection of (meta) heuristics: Propose a hyper-heuristic framework to automatically find the best combination of heuristics/meta-heuristics and parameters that best solve a particular problem.

• Use cooperative search to combine the strengths of different meta-heuristics to balance intensification and diversification and direct the search towards promising regions of the search space.

7

Hyper-heuristics

The aim of hyper-heuristics is to develop general domain-independent search methodologies that are capable of performing well-enough, soon enough, and cheap-enough across a wide-range of optimisation problems.

Hyper-heuristics

Research work on hyper-heuristics was dated back to the 1960’s (Fisher and Thompson, 1963).

The term hyper-heuristic has only been introduced recently (Cowling et al., 2000).

8

Hyper-heuristics

Research work on hyper-heuristics was dated back to the 1960’s (Fisher and Thompson, 1963).

The term hyper-heuristic has only been introduced recently (Cowling et al., 2000).

Hyper-heuristics

• A hyper-heuristic is a high-level heuristic which, when given a particular problem instance and a number of low level heuristics, selects and applies an appropriate low level heuristic at each decision point (Cowling et al., 2000; Soubeiga, 2003; Burke et al., 2003).

• Low level heuristics are simple local search operators, domain dependent heuristics, or meta-heuristics.

Properties

• Take advantage of strengths and avoid weaknesses of each low level heuristic.

• No problem specific knowledge is required.

• Problem independent.

• Goal: increase the level of generality and reusability.

11

Hyper-heuristic framework

12

Domain Barrier

Hyper-heuristic level

h1 h2 hk……

Low level heuristics

Non-domain data flow

Non-domain data flow

Heuristic Selection

Acceptance Criteria

(Cowling et al., 2000)

Hyper-heuristic framework

13

Hyper-heuristic

Selection strategy Acceptance criteria

Random (R)

Choice function (CF)

Greedy (GR)

Tabu Search based (TS)

Simulated Annealing (SA)

All Moves (AM)

Improving Only (IO)

Monte Carlo (MC)

Great Deluge (GD)

14

Random hyper-heuristics

• Simple Random (SR)

Do

Select a low level heuristic uniformly at random

and apply it once.

Until stopping condition is met

NUS October 2008

15


• Random Descent (RD)

Do

Select a low level heuristic uniformly at

random and apply it until no further improvement is possible.


NUS October 2008

16

• Random Permutation Heuristic (RP)

Create a random permutation of all low level heuristics.

Do

Select the next low level heuristic in the sequence

and apply it once.



NUS October 2008

17

• Random Permutation Descent (RPD)

Create a random permutation of all low level heuristics

Do

Select the next low level heuristic in the sequence and

apply it in a steepest descent fashion.



NUS October 2008

18

Do

Apply all the low level heuristics and select the heuristic

providing the best improvement.


Greedy hyper-heuristic

NUS October 2008

• Different heuristics have different strengths and weaknesses.

• Cooperation allows the strengths of heuristics to compensate for the weaknesses of others.

• Undertake a novel investigation into the role of asynchronous cooperation in a hyper-heuristic framework.

19

Cooperative hyper-heuristicsCooperative hyper-heuristics

Cooperative search

Cooperative optimisation consists of a search performed by agents that exchange information about states, models, entire sub-problems, solutions or other search space characteristics (Blum and Roli, 2003).

Parallelism v Agents

Parallelism in combinatorial optimisation is based on speed-up.

An Agent-based approach is focussed on cooperation.

Agent-based cooperation

What are agents?

Wooldridge and Jennings (1995)“ An agent is a computer system that is situated in some environment and that is capable of autonomous action in the environment in order to meet its design objectives”.

1. Woodridge, M and Jennings, N.R (1995) Intelligent Agents: Theory and Practice. The Knowledge Engineering review, 10(2), 115-152

The agent-based framework

meta-heuristic agents

JADE Platform

Java Agent Development Framework (JADE)

Is an Open Source platform for peer-to-peer agent-based applications.

It provides all the infrastructure necessary to develop and runagent-based applications.

1,3 4, …2,7 9, ....

Simulated Annealing

Send

Receive

Config file

Read in parameters

Create new Solutions

Generate patterns

Good patternsAnd solutions

Meta-heuristicsMeta-

heuristicsMeta-heuristics

Neighbourhood search

Good patternsAnd solutions

Store goodpatterns

Structure of an agent

Intensification and diversification

Meta-heuristics

Search Space

Heuristic SearchIntensification




Diversification

.

Cooperation protocol

Agents cooperate by looking for good patterns that are the constituent parts of a good solution.

• Use of reinforcement learning to score good patterns.

• Good patterns are shared amongst the agents which will be used to generate new solutions.

.

Cooperation protocol

Any combinatorial optimisation problem that involves manipulatinga non repeating list of size n can be split into a list of n pairs.

Take the permutation where n = 10: 2,4,7,6,5,8,9,0,1,3. The following n pairs can be generated:

(2,4)(4,7)(7,6)(6,5)(5,8)(8,9)(9,0)(0,1)(1,3)(3,2).

Reward/penalty values are:

Selection function: Roulette wheel

29

Reinforcement learning

0,1

0,1, f

fkh

nkk

jj w

wp

,1

Flexibility using ontologies

• The system is able to solve problems in different domains by using ontologies.

• Ontologies are generic conceptualisations of combinatorial optimisation models.

• The heuristics and messaging use the same generic representation of the model.

• Agents cooperate by communication through ontologies.

Ontologies

• JADE has good support for them.

• Ontologies are parsed into XML. In this way they can communicate to other agents across the web if required.

Ontologies

Flexibility: To test if the system is flexible enough to work on different problem domains.

To test if agents cooperating produced better results than the equivalent meta-heuristic heuristic combinations running as stand alone programmes.

Scalability: To see if more agents cooperating produced better results than fewer agents.

Testing and results

Permutation Flow Shop Problem (PFSP)

Nurse Rostering (NR)

Case studies

Meta-heuristic agents:

Tabu search, Simulated annealing, Variable neighbourhood search

Testing and results

For the experiments, we have considered the followingscenarios:

1 agent in stand alone mode.

5 agents, 1 launcher and 4 meta-heuristic agents.



Testing and results

PFSP

Process n jobs on m machines.

Jobs are processed in the same sequence on all machines.

Operations are not preemptable and set-up times of operations are included in the processing time.

Objective function: minimisation of the makespan.

Bench mark problems: Taillard (1990, 1993) 120 instances of 12 different sizes, n × m: 20x5, 20x10, 20x20, 50x5, 50x10, 50x20, 100x5, 100x10, 100x20, 200x10, 200x20, 500x20.

Results

The Nurse Rostering problem

There are different staffing needs on different days and shiftsStaff work in shiftsHealthcare institutions work around the clock: need for day and night shiftsThe correct staff mix for each wardMany different employment contracts: Part-time, full time, etc.

Nurse rostering problem consists of the assignment of shifts to nurses subject to several constraints such as workload, legal and contractual restrictions, personal preferences, etc. (Burke et al., 2004)

The Nurse Rostering problem

Hard constraints:

Over cover and under cover is not permitted, a nurse may not work more than one of the same shift type on the same day, a shift which requires a certain skill can only be assigned to a nurse that has that skill, etc.

Soft constraints:

Time related constraints, rest times, weekend shifts, etc.

Fairness in Nurse Rostering

Traditionally, solutions are evaluated using a weighted sum of soft constraint violations.

How can we guarantee fairness?

The standard objective function

Models of fairness

Let C be the set of constraints. wc is weight associated with a given constraint

and N the number of nurses

Models of fairness

New Fairness objective functions

MinMax = minimise the worst nurse violation

MinDev = minimise the sum of deviations from the average

MinError = minimise the differences between the best and the worst rosters

Fairness evaluation

Fairness evaluation using Jain’s index function (Jain et al., 1984; Muhlenthaler and Wanka, 2012)

Its values range from the worse case 1/N to 1 where the roster is completely fair

Jain’s index

Instance Nr/nurses Nr/shifts Planning period

Emergency 27 27 28 days

Geriatrics 21 9 28 days

Psychiatry 19 14 31 days

Reception 19 19 42 days

Testing and results

Instances from two Belgium hospitals.

For the experiments, we have considered the followingscenarios:

1 meta-heuristic agent in stand alone mode.


Testing and results

Instance MinWS MinMax MinDev MinError

Emergency_i 0.5638 0.9983 0.9991 0.9074

Geriatric_i 0.4451 0.9784 0.9861 0.6023

Psychiatry_i 0.6069 0.9995 0.9995 0.9054

Reception_i 0.6273 0.8808 0.8515 0.7191

Emergency_d 0.6295 0.9973 0.9982 0.8574

Geriatric_d 0.6013 0.9980 0.9986 0.9491

Psychiatry_d 0.4446 0.9986 0.9988 0.6882

Reception_d 0.3321 0.9895 0.9824 0.8052

Results

Table : The average Jains fairness index over 20 runs of a given fairness-based objective function for each benchmark instance, where the bold entries indicate the best one for a given instance.

Results

Conclusions

Agent-based framework for cooperative search.

Cooperation using pattern matching and reinforcement learning.

Cooperating agents produced better results than the equivalent meta-heuristic and heuristic combinations running as stand alone basis.

Use of ontologies to rise the level of generality of the agent-based framework.

The framework will be available in the near future on the University of Portsmouth website for public use.

Future Work

New problem domains: We are currently working on implementing VRP instances.

New meta-heuristics and heuristics.

Thank you

*djamila ouelhadj, *simon martin, **patrick beullens and ***ender Özcan *logistics and management...

Documents

djamila ouelhadj, simon martin, patrick beullens and ender Özcan logistics and management...