the automatic design of multi-objective ant colony...

The Automatic Design of Multi-Objective

Ant Colony Optimization Algorithms

Manuel Lopez-Ibanez and Thomas Stutzle

IRIDIA – Technical Report Series

Technical Report No.

TR/IRIDIA/2011-003

February 2011

IRIDIA – Technical Report SeriesISSN 1781-3794

Published by:

IRIDIA, Institut de Recherches Interdisciplinaires

et de Developpements en Intelligence Artificielle

Universite Libre de BruxellesAv F. D. Roosevelt 50, CP 194/61050 Bruxelles, Belgium

Technical report number TR/IRIDIA/2011-003

The information provided is the sole responsibility of the authors and does not neces-sarily reflect the opinion of the members of IRIDIA. The authors take full responsibilityfor any copyright breaches that may result from publication of this paper in the IRIDIA– Technical Report Series. IRIDIA is not responsible for any use that might be madeof data appearing in this publication.

The Automatic Design of Multi-Objective Ant Colony

Optimization Algorithms

Manuel Lopez-Ibanez Thomas [email protected] [email protected]

IRIDIA, CoDE, Universite Libre de Bruxelles, Brussels, Belgium

February 9, 2011

Abstract

Multi-objective optimization problems are problems with several, typically conflictingcriteria for evaluating solutions. Without any a priori preference information, the Paretooptimality principle establishes a partial order among solutions, and the output of the al-gorithm becomes a set of nondominated solutions rather than a single one. Various antcolony optimization (ACO) algorithms have been proposed in recent years for solving suchproblems. These multi-objective ACO (MOACO) algorithms exhibit different design choicesfor dealing with the particularities of the multi-objective context.

This paper proposes a classification of algorithmic components that suffices to describemost MOACO algorithms proposed so far. This classification also shows that existingMOACO algorithms often share equivalent design choices but they are described in differ-ent terms. Moreover, this classification is synthesised into a flexible algorithmic framework,from which not only existing MOACO algorithms may be instantiated, but also combina-tions of components that were never studied in the literature. In this sense, this paper goesbeyond proposing a new MOACO algorithm, but it rather introduces a family of MOACOalgorithms. Using this framework, this paper empirically compares existing MOACO algo-rithms, and possible variants, without the bias introduced by different implementations ordifferent underlying ACO algorithms.

The flexibility of the proposed MOACO framework facilitates the application of auto-matic algorithm configuration techniques. The experimental results presented in this papershow that the automatically configured MOACO framework outperforms the MOACO al-gorithms that inspired the framework itself. To the best of our knowledge, this paper is alsoamong the first to apply automatic algorithm configuration techniques to multi-objectivealgorithms.

Keywords: Multiobjective Optimization, Ant Colony Optimization, Travelling SalesmanProblem, Automatic Configuration

1 Introduction

Ant colony optimization (ACO) [11] is a metaheuristic inspired by the pheromone trail lay-ing and following behavior of some real ant species. ACO was originally designed for solvingsingle-objective combinatorial optimization problems. Due to notable results on these problems,ACO algorithms were soon extended to tackle problems with more complex features [29], and,in particular, multiple objective functions. Two recent reviews [2, 14] compile more than ten

1

mailto:[email protected]

mailto:[email protected]

different extensions of ACO algorithms for tackling multi-objective combinatorial optimizationproblems (MCOPs), where the quality of candidate solutions is evaluated with respect to variousconflicting objectives. The majority of these multi-objective ACO (MOACO) algorithms focuson problems in terms of Pareto optimality, that is, they do not make a priori assumptions aboutthe decision maker’s preferences. Moreover, most of these MOACO algorithms were proposedfor bi-objective optimization problems, and, hence, these will be the focus of this paper.

While there is a substantial amount of research on evolutionary computation and local searchalgorithms for solving multi-objective optimization problems, there are relatively few works onMOACO algorithms. Most articles propose only one specific design of a MOACO algorithm [4, 9];rare are studies that compare a few design alternatives [1, 15, 20]. A first review of existingMOACO algorithms identified, among the ones proposed in the literature, the best MOACOalgorithm adapted to a particular problem in a traditional “horse-race” analysis [14]. The algo-rithms tested in that review differ substantially with respect to the underlying ACO algorithm,e.g., some of them are based on the classical Ant System, whereas others build upon the typicallybetter performing MAX -MIN (MMAS) [27] and Ant Colony System (ACS) [10]. Therefore,it is difficult to conclude anything about the specific design of each MOACO algorithm. Amore recent review [2] categorizes existing MOACO algorithms into several classes without anyempirical comparison.

This paper integrates and extends our recent work on MOACO algorithms [17, 18, 19], whichhas advanced in several directions. As a first step, we present an algorithmic framework thatsynthesizes many of the design choices proposed in the literature and that offers new possibilitiesnever proposed before. This framework allows us to instantiate existing MOACO algorithmsand, more importantly, to combine ideas from diverse MOACO algorithms to produce new ones.A first insight from the use of this framework is that existing MOACO algorithms share morealgorithmic components than the respective publications may suggest. A second contribution isthat we can devise more powerful combinations of algorithmic components and, at the same time,we can assess the influence of each design choice on performance. In addition, the flexibility ofthis framework facilitates the application of automatic techniques for configuring multi-objectivealgorithms. Most previous works that aim to produce high performing algorithms by combin-ing a flexible software framework and an automatic configuration tool have dealt so far onlywith single-objective optimization problems [16]. We proposed [18] integrating the hypervolumeindicator into the iterated F-race tool (I/F-Race) [3] in order to automatically configure ourproposed MOACO framework for tackling the bi-objective traveling salesman problem (bTSP).The integration of unary quality indicators, such as hypervolume and epsilon measures, intoI/F-Race allows us to automatically configure multi-objective algorithms with many continuous,categorical and conditional parameters for an optimization problem, given a set of represen-tative instances of this problem. Independently, Wessing et al. [30] have presented results forconfiguring a multi-objective evolutionary algorithm using the hypervolume indicator. However,they considered only a single parameter, the variation operator, and configured the algorithmon a single instance, thus their approach is prone to overtuning [5]. In this paper, we showthat, by automatically configuring a flexible framework that synthesizes design concepts fromdifferent MOACO algorithms, we are able to find new designs of MOACO algorithms for thebTSP that outperform the MOACO algorithms found in the literature. Our results demonstratethe effectiveness of this approach for the design of stochastic local search (SLS) algorithms formulti-objective optimization problems in terms of Pareto-optimality.

We present a general MOACO framework for bi-objective combinatorial optimization prob-lems in Section 3. The components of this framework correspond to different alternative choicesin the design of a MOACO algorithm. Next, we describe in Section 4 how the existing frame-work is able to replicate the design of most MOACO algorithms proposed in the literature. In

2

other words, those MOACO algorithms may be instantiated by properly configuring the MOACOframework. In Section 5, we propose a method to automatically configure the MOACO frame-work for a particular bi-objective problem. The problem chosen is the bTSP, and we compare theresults to the best MOACO configuration proposed in the literature. In particular, we perform amore comprehensive automatic configuration and comparison than the preliminary one describedin our previous work [18]. Finally, in addition to the proposed configuration of the frameworkfor the bTSP, we extract some general conclusions from this work and describe steps to exploitour proposals in Section 7.

2 Preliminaries

2.1 Bi-objective Combinatorial Optimization

We propose here a framework based on ACO for tackling bi-objective combinatorial optimiza-tion problems (bCOPs) in terms of Pareto optimality. In such bCOPs, candidate solutions are

evaluated according to an objective function vector ~f = (f1, f2). Without a priori assumptionson the preferences of the decision maker, the goal is to determine a set of feasible solutions that“minimizes” the objective function vector ~f . Let ~u and ~v be vectors in R2. We say that ~u domi-nates ~v (~u ≺ ~v) iff ~u 6= ~v and ui ≤ vi, i = 1, 2. Furthermore, ~u and ~v are nondominated iff ~u ⊀ ~vand ~v ⊀ ~u. To simplify the notation, we also say that a feasible solution s dominates anothersolution s′ iff ~f(s) ≺ ~f(s′). A solution s is a Pareto optimum iff no other feasible solution s′

exists such that ~f(s′) ≺ ~f(s). The goal in bCOPs then typically is to determine the set of allPareto-optimal solutions. However, this task is often computationally intractable, and, hence, itis usually preferable to approximate the Pareto set as well as possible in a given amount of time.Such an approximation is always a set of solutions that are mutually nondominated.

2.2 The Bi-objective Traveling Salesman Problem (bTSP)

The single-objective TSP is one of the most important combinatorial optimization problemsand a typical case study in the development of ACO algorithms. In the TSP, we are given acomplete graph G = (V,E) with n = |V | nodes {v1, . . . , vn}, a set of edges E that fully connectsthe nodes, and a cost c(vi, vj) associated to each edge. The goal is to find a Hamiltonian tourp = (p1, . . . , pn) that minimizes the total cost:

minimize f(p) = c (vp1 , vpn) +n−1∑i=1

c(vpi , vpi+1

). (1)

The bi-objective TSP (bTSP) is the direct extension of the above formulation. For each edge,a vector of costs is given with two components c1(vi, vj) and c2(vi, vj), i 6= j, which correspondto the cost of the first and the second objective, respectively. We consider here the symmetricbTSP, where it additionally holds that cq(vi, vj) = cq(vj , vi), i 6= j, q = 1, 2. We assumeno a priori knowledge of the preferences of the decision maker. Therefore, the goal is to findthe set of Hamiltonian tours that minimizes, in the sense of Pareto optimality, the vector ofobjective functions ~f = (f1, f2). The bTSP is often used as a benchmark problem for testingmulti-objective combinatorial optimization algorithms [12, 14, 17, 22, 24].

2.3 Ant Colony Optimization (ACO)

The ACO metaheuristic [11] describes a number of algorithms inspired by the swarm behavior ofreal ants species. In ACO algorithms, a number of artificial ants construct candidate solutions

3

Algorithm 1 The ACO metaheuristic for single-objective combinatorial problems.

Procedure ACO MetaheuristicrepeatConstructSolutions()OptionalActions() // e.g. local searchUpdatePheromones()

until stopping criteria metOutput: best solution found

to a problem. The decisions of the ants are biased by a common numerical information called(artificial) pheromone. This pheromone is updated according to the quality of the solutionsconstructed by the ants in order to bias the construction of new solutions in subsequent iterations.In particular, an ant constructs a candidate solution to a problem by iteratively adding solutioncomponents to its partial solution in a stochastic fashion. In the case of the TSP, the probabilitythat an ant k chooses to visit city j after city i is given by:

pkij =[τij ]

α · [ηij ]β∑l∈Nk

i[τil]

α · [ηil]βif j ∈ N k

i , (2)

where τij is the pheromone associated to adding the edge (i, j) to the current partial tour, ηilis a static greedy measure of the “goodness” of edge (i, j) called heuristic information, and N k

denotes the set of feasible choices available for ant k given its current partial solution.After a number of ants have constructed a solution each, one or more of these solutions are

used to update the pheromone information in such a way as to bias future choices towards highquality solutions. It is common in ACO algorithms to perform other actions, such as local search,to further improve solutions before updating the pheromone information. The pseudo-code ofthe ACO metaheuristic is shown in Algorithm 1.

3 A Configurable MOACO Framework

There are a number of design questions when extending ACO algorithms to bCOPs. First, insingle-objective ACO, the pheromone information is related to the objective function, that is, thecomponents of higher quality solutions receive more pheromone. In a multi-objective context,the objective function is multi-dimensional and not scalar, and there is only a partial orderamong solutions. Moreover, in some problems, it could make sense to have different solutioncomponents for each objective, and, hence, associate different pheromone matrices to them [15].For example, in applications of ACO to multi-objective scheduling problems, one pheromonematrix may represent jobs×jobs relationships, and the other position×jobs relationships. In thiscase, both matrices could be updated with the same pheromone amount because their meaningis different. In other problems, such as the bTSP, both pheromone matrices represent the samesolution components (an edge), and, hence, they would need to be updated either using differentsolutions or using different pheromone amounts [1, 9]. Alternatively, it could also make sense touse a single pheromone matrix [4]. If multiple pheromone or heuristic matrices are used, theyare typically aggregated during the solution construction by means of weights [9, 15]. However,there are strong differences among MOACO algorithms in the way this aggregation is done andhow many weights are used. A further design question is which ants are selected for depositingpheromone. Existing MOACO algorithms select some (or all) nondominated solutions [15], orthey select the best solutions with respect to the objective associated to the pheromone matrix

4

that is updated [1, 9]. Finally, several MOACO algorithms make use of multiple colonies ofants [1, 15].

We take a component-wise view of the design of MOACO algorithms, which allows us toabstract the particular design choices taken in earlier works. We have identified several algo-rithmic components that correspond to the design alternatives described above. We divide thesecomponents in three main groups: (i) those related to the definition of pheromones and the con-struction of solutions; (ii) those related to the update of the pheromones; (iii) and those relatedto the use of multiple colonies. We examine these components in more detail in the following.We will discuss the connection between these components and existing MOACO algorithms inSection 4.

Our proposed MOACO framework does not specify many details, such as how pheromonevalues are initialized, updated or evaporated. It neither specifies whether the pheromone infor-mation is updated using the best-so-far or the iteration-best solutions. These details are relegatedto the underlying ACO algorithm, which we assume that it is an efficient implementation, for theproblem at hand, of any modern ACO algorithm such as MAX -MIN (MMAS) [27] or AntColony System (ACS) [10]. This approach allows us to focus on the multi-objective componentsof MOACO algorithms.

3.1 MOACO algorithmic components for solution construction

One of the most crucial design decisions when applying an ACO algorithm to a problem is thedefinition of pheromone (and heuristic) information. In the multi-objective context, this problemis exacerbated by the fact that there are several ways to evaluate a single solution, and also, bythe fact that multiple solutions may be (Pareto-)optimal. This design question is answered bythe following algorithmic components.

Single/Multiple Pheromone/Heuristic Information. We may have one (single) or sev-eral (multiple) matrices for either the pheromone information τ or, if applicable, the heuristicinformation η. In the case of multiple heuristic matrices, each matrix is associated to a differ-ent objective, such that η1 and η2 correspond to the heuristic information of each objective,respectively. For some problems, it may not be possible to define the heuristic information foreach objective independently, so we will be forced to use a single heuristic matrix. Similarly, inthe case of multiple pheromone matrices, τ1 and τ2 would be associated to each objective; theprocedure that selects the solutions used for updating each matrix (see next subsection) shouldenforce this distinction somehow.

During solution construction, whenever we have multiple matrices, we will need to aggregatethem into a single matrix. The aggregation method is defined by the component Aggregation,which is described below. It is conceivable to use a different aggregation method for pheromoneand heuristic information; however, we do not explore this possibility here.

Aggregation. The values from multiple pheromone (or heuristic) matrices need to be aggre-gated into a single pheromone (or heuristic) value. We have identified three alternatives in theliterature:

Weighted sum, e.g., τij = (1− λ)τ1ij + λτ2ij and ηij = (1− λ)η1ij + λη2ij .

Weighted product, e.g., τij = (τ1ij)(1−λ) · (τ2ij)λ and ηij = (η1ij)

(1−λ) · (η2ij)λ .

Random. At each construction step, given a uniform random number U(0, 1), an ant selectsthe first of the two matrices if U(0, 1) < 1− λ, otherwise it selects the other matrix.

5

In the three aggregation methods described above, there is a weight λ that biases the aggre-gation towards one objective or the other. The set of weights Λ is defined by the componentsNweights and NextWeight.

Nweights and NextWeight. The set of weights is defined within the interval [0, 1] as

Λ ={λi = 1− (i− 1)/(Nweights − 1), i = 1, . . . , Nweights

},

where Nweights = |Λ| is a parameter of the framework. Component NextWeight determines whichparticular weight is used by an ant at a certain iteration. The options tested for NextWeight areeither that all ants use the same weight at a certain iteration (one-weight-per-iteration), or thatall weights are used at each iteration (all-weights-per-iteration). In the case of one-weight-per-iteration, the weight used in successive iterations follows an ordered sequence of the elements ofΛ, and the order is reversed when the last weight in the sequence is reached. In the case of all-weights-per-iteration, when the number of ants Na is larger than the number of weights Nweights,several ants will use the same weight. An obvious speed-up is to compute the aggregation of thepheromone matrices only once per weight per iteration. This describes the behavior for a singlecolony; the multi-colony case will be discussed in Section 3.3.

3.2 MOACO algorithmic components for pheromone update

Given a set Aupd of candidate solutions for updating the pheromone information, the Pheromone-Update component decides which solutions update the pheromone information and how. TheNupd parameter determines how many solutions are used to update each pheromone matrix. Weconsider the following alternatives for the PheromoneUpdate component:

Nondominated solutions. The solutions used for updating the pheromone information arethe nondominated solutions in Aupd. When there are more nondominated solutions thanNupd, we apply the truncation mechanism of SPEA2 [31] to select only Nupd solutions.In principle, it is possible to combine this nondominated PheromoneUpdate method andmultiple pheromone matrices, simply using the same solutions to update both matrices [15]as long as the update is different in each pheromone matrix.

Best-of-objective. This mechanism first selects from Aupd the Nupd best solutions with respectto each objective. In the case of multiple pheromone matrices, each pheromone matrix isupdated using the Nupd solutions associated to the corresponding objective. Otherwise,the 2 ·Nupd solutions update the single pheromone matrix.

Best-of-objective-per-weight. For each weight λ and each objective, there is a list of theNupd best solutions for that objective generated using λ. In the particular case of λ = 0,we keep only the list for the first objective, and we do the same for λ = 1 and the secondobjective. When using multiple pheromone matrices, each matrix is updated using onlysolutions from lists associated to the same objective. In the single pheromone matrixcase, both lists are used for the update. Therefore, in the particular case of two weights,this method is equivalent to best-of-objective. This best-of-objective-per-weight method isused by the existing mACO-1 and mACO-2 algorithms [1], and, hence, we include it forcompleteness. However, it is not clear how this approach should be extended to multiplecolonies, since solutions may be exchanged among colonies with different weights.

The above methods do not describe whether Aupd is composed of the best-so-far or iteration-best solutions, since this is determined by the particular underlying ACO algorithm.

6

3.3 MOACO algorithmic components for multiple colonies

Many MOACO algorithms propose the use of multiple ant colonies, using various definitionsof what a colony is. One popular approach is to define two colonies, each of them having itsown pheromone information. These colonies may exchange solutions that are used to updatethe pheromone information. Some ants, which sometimes are said to belong to “extra” colonies,aggregate the pheromone information of the two other colonies [1, 8]. This approach is actuallyequivalent to using multiple pheromone information, which are aggregated by means of differentweights, and it does not seem appropriate to say that ants belong to different colonies simply be-cause they use different weights. By comparison, the multi-colony architecture of Iredi et al. [15],defines a colony as a group of ants associated to a particular pheromone information, such thatants from each colony construct solutions according only to the pheromone information of theircolony. In the case of multiple pheromone information, each colony has two pheromone matrices,one for each objective. When a set of weights is used, each colony has its own set of weightsΛc. This multi-colony architecture allows us to generalize all previous components. Moreover,colonies cooperate by using a common archive of nondominated solutions for detecting dominatedones. Further cooperation is enforced by exchanging solutions for updating the pheromone in-formation. This setting is controlled by the MultiColonyUpdate component described below. Thenumber of colonies is given by component N col, and the components described below only havean effect when N col > 1.

MultiColonyWeights. In the case of multiple colonies, we create a set of weights Λc of sizeNweights for each colony c. These sets are constructed by following the two alternatives proposedby Iredi et al. [15]: disjoint and overlapping intervals. In both cases, we first generate thenecessary number of equally distributed weights in the interval [0, 1]. Then, for disjoint intervals,this set is partitioned into equal disjoint sub-intervals per colony, e.g., weights within [0, 0.5) andwithin (0.5, 1] for each of two colonies. In the case of overlapping intervals, the sub-intervalsoverlap by 50%, e.g., [0, 0.75) and (0.25, 1] for two colonies, and hence, Λc and Λc+1 share 50%of their weights.

MultiColonyUpdate. In the case of multiple colonies, the solutions generated by all coloniesin the current iteration are stored in a common archive Aiter, so that all colonies contribute todetect and remove dominated solutions. After this step, the remaining nondominated solutionsare assigned back to each colony for updating the pheromone information. The basic method,called origin, assigns each solution from Aiter to its original colony. To enforce more cooperation,colonies may exchange solutions. One method of exchange, called update by region, divides Aiter

in equal parts among the colonies in such a way that each colony roughly corresponds to oneregion of the objective space. Both settings, origin and region, were originally proposed by Irediet al. [15].

3.4 The MOACO framework

Table 1 summarizes the algorithmic components defined above and their domains. Some settingsare only significant for certain values of other settings. We propose Algorithm 2 as a way tointegrate these components into a flexible MOACO framework for bi-objective optimization. Itfollows the basic outline of the ACO metaheuristic. First, the algorithm initializes the pheromoneinformation and the set of weights of each colony, whereas heuristic information is initialized onlyonce, since it is shared by the different colonies. It also initializes the best-so-far archive Abf,the archive of all nondominated solutions ever found. At each iteration, each colony constructs

7

Table 1: Algorithmic components of the proposed MOACO framework.

Component Domain Description

[τ ] { single, multiple } Definition of pheromone matrices

[η] { single, multiple } Definition of heuristic matrices

Aggregation

weighted sum,

weighted product,

random

How weights are used to aggregate different matrices

Nweights N+ Number of weights (per colony)

NextWeight

{one weight per iteration (1wpi),

all weights per iteration (awpi)How weights are used at each iteration

PheromoneUpdate

nondominated solutions (ND),

best-of-objective (BO),

best-of-objective-per-weight (BOW)

Which solutions are selected for updating the pheromone matrices

Nupd N+ Number of solutions that update each [τ ] matrix

N col N+ Number of colonies

MultiColonyWeights { disjoint, overlapping } How weights are partitioned among colonies

MultiColonyUpdate { origin, region } How solutions are assigned to colonies for update

8

Na solutions according to its own pheromone information, which may be possibly aggregated bya weight λ from its own set of weights. Depending on the setting of NextWeight, λ may be thesame weight or a different one for each ant in the colony. In single-objective ACO algorithms,solutions are often improved by means of local search. Therefore, we have included an optionalWeightedLocalSearch procedure that improves a solution by applying local search to a weightedsum of the objective functions, using the same weight as used for constructing the solution.The new solution is added to the archive of the current iteration shared by all colonies Aiter.Once all ants from all colonies have finished constructing solutions, the best-so-far archive Abf

is updated with the nondominated solutions found in the current iteration Aiter. The updateof the pheromones consists of two phases. First, MultiColonyUpdate distributes the archive ofsolutions for update (Aupd) among the colonies. Second, procedure PheromoneUpdate decideshow the solutions assigned to each colony update the pheromone information of the colony. TheMOACO algorithm continues until a certain number of iterations or a time limit is reached.

4 The Design of MOACO Algorithms

Our previous work [19] has examined the design of several MOACO algorithms from the litera-ture. This study has informed the design of the MOACO framework described in the previoussection. In this section, we explain how certain configurations of the framework replicate thedesign of these MOACO algorithms. The aim here is not to faithfully reproduce the origi-nal algorithms, but to identify which particular configuration of algorithmic components in theframework corresponds to the algorithmic design choices made in each case. In particular, wedo not pay attention here to the underlying ACO algorithm, pheromone initialization or updateamounts, heuristic information, or problem-dependent speed-ups. Table 2 summarizes the con-figuration of the MOACO framework that instantiates the MOACO algorithms reviewed in thissection.

4.1 MOAQ

Although the original design of Multiple Objective Ant-Q (MOAQ) [23] was for problems witha given preference order of the objectives (lexicographical optimization), Garcıa-Martınez et al.[14] generalize MOAQ to bi-objective problems in terms of Pareto optimality. Their proposaluses a single pheromone matrix and multiple heuristic matrices, one for each objective. For thepheromone update, they use all nondominated solutions. Garcıa-Martınez et al. [14] divide theants into two groups, and each group only uses the heuristic information corresponding to oneobjective. The equivalent in our framework is to use the weights Λ = {0, 1} for aggregatingthe two heuristic matrices, such that half of the ants use each weight. Such a set of weightseffectively means that no actual aggregation takes place, but instead half of the ants use oneheuristic matrix and the other half use the other one.

4.2 BicriterionAnt

Iredi et al. [15] proposed a MOACO algorithm, henceforth called BicriterionAnt, with multiplepheromone and heuristic matrices, aggregated by weighted product. Each ant k uses a differ-ent weight λk for aggregating the pheromone matrices; thus, there are as many weights as ants(Nweights = Na). The authors of BicriterionAnt suggest to update the pheromone matrices usingthe nondominated solutions found in the current iteration. Moreover, they update both phero-mone matrices with the same amount. Such an update does not result in identical pheromonematrices because, in the problem tackled by them, the pheromone matrices represent different

9

Algorithm 2 MOACO framework

1: Abf := ∅2: InitializeHeuristicInformation()3: for each colony c ∈ {1, . . . , N col} do4: InitializePheromoneInformation()5: Λc := MultiColonyWeights()6: end for7: iter := 08: while not stopping criteria met do9: Aiter := ∅

10: for each colony c ∈ {1, . . . , N col} do11: for each ant k ∈ {1, . . . , Na} do12: λ := NextWeight(Λc, k, iter)

13: τ :=

{Aggregation(λ, {τ1c , τ2c }) if multiple [τ ]

τc if single [τ ]

14: η :=

{Aggregation(λ, {η1, η2}) if multiple [η]

η if single [η]

15: s := ConstructSolution(τ, η)16: s := WeightedLocalSearch(s, λ) // Optional17: Aiter := RemoveDominated(Aiter ∪ {s})18: end for19: end for20: Abf := RemoveDominated(Abf ∪Aiter)21: for each colony c ∈ {1, . . . , N col} do22: Aupd

c := MultiColonyUpdate(Aupd)23: PheromoneUpdate(Aupd

c , Nupd)24: end for25: iter := iter + 126: end while27: Output: RemoveDominated(Abf)

10

Table 2: Taxonomy of MOACO algorithms as instantiations of the proposed framework (Na is the number of ants).

Algorithm [τ ] [η] Aggregation Nweights PheromoneUpdate Nupd

MOAQ [14, 23] 1 2 – 2 (Λ = {0, 1}) nondominated solutions ∞BicriterionAnt [15] 2 2 weighted product Na nondominated solutions ∞MACS [4] 1 2 weighted product Na nondominated solutions ∞COMPETants [8] 2 2 weighted sum 3 (Λ = {0, 0.5, 1}) best-of-objective anyP-ACO [9] 2 1, 2 weighted sum Na best-of-objective 2

mACO-1 [1] 2 2

{random (τ)

weighted sum (η)3 (Λ = {0, 0.5, 1}) best-of-objective-per-weight 1

mACO-2 [1] 2 2 weighted sum 3 (Λ = {0, 0.5, 1}) best-of-objective-per-weight 1mACO-3 [1] 1 1 – – nondominated solutions ∞mACO-4 [1] 2 1 random (τ) 1 (Λ = {0.5}) best-of-objective 1

11

solution components as explained on page 4 of this article. When applying BicriterionAnt tothe bTSP, Garcıa-Martınez et al. [14] used the objective function value of each objective toupdate the corresponding matrix ∆τk = 1/fk(sa). We do the same in our framework for thecombination of multiple pheromone matrices and nondominated pheromone update.

In addition to the above algorithm, Iredi et al. [15] proposed the use of multiple colonies, anddefine a colony as a group of ants that construct solutions according to their own pheromoneinformation. In this sense, a multi-colony BicriterionAnt algorithm is similar to a multi-startapproach with some particularities. First, different colonies specialize in different regions of thePareto frontier by using different sets of weights to aggregate pheromone information. Second,colonies cooperate to detect dominated solutions by keeping solutions in a common archive.Third, colonies may also cooperate by exchanging solutions. As explained in Section 3.3, thismatches the multi-colony approach adopted in our MOACO framework.

4.3 Multiple Ant Colony System

Multiple Ant Colony System (MACS) [4] uses one heuristic matrix for each objective and asingle pheromone matrix. The heuristic matrices are aggregated by weighted product, and eachant uses a different weight. In addition, pheromone information is updated with nondominatedsolutions. MACS can be seen as a variant of MOAQ, as defined by Garcıa-Martınez et al. [14],that uses more than two weights to aggregate the heuristic information. On the other hand,the only difference between MACS and single-colony BicriterionAnt is the number of pheromonematrices. As a result, it is straightforward to define a multi-colony MACS, as we do in ourMOACO framework.

4.4 COMPETants

COMPETants [8] is presented as a multi-colony approach, with one colony for each objective.Each colony has one pheromone and heuristic matrix. Each colony constructs solutions inde-pendently, except for a number of ants (called “spies”), which aggregate the two pheromonematrices by weighted sum (with λ = 0.5) using either the first or the second heuristic matrix,thus creating two solutions. Finally, a number of ants from each colony are used to update thepheromone matrix of each colony.

We can formulate COMPETants in the MOACO framework by using two pheromone andheuristic matrices, which are aggregated by weighted sum and three weights Λ = {0, 0.5, 1}.Thus, COMPETants is a single-colony approach in our framework, and it is straightforwardto define variants of this formulation with an arbitrary number of colonies. For the sake ofsimplicity, in our MOACO framework, the total number of ants is equally divided by the numberof weights, and the number of ants per weight does not change during the run, as in the originalproposal; nevertheless, this feature could be added to the framework as an additional component.Moreover, our previous work [19] showed that, at least in the bi-objective TSP, never aggregatingthe heuristic information, as in the original COMPETants, leads to poor quality in the middleof the Pareto front. Hence, we do not handle specially the heuristic information for ants usingλ = 0.5; heuristic matrices are aggregated in the same way as pheromone matrices. Finally,each pheromone matrix is updated with the Nupd best solutions for the corresponding objective,which is the best-of-objective setting for PheromoneUpdate in the MOACO framework.

4.5 Pareto Ant Colony Optimization

Pareto Ant Colony Optimization (P-ACO) [9] uses multiple pheromone matrices, one for eachobjective, aggregated by means of a weighted sum. Like in BicriterionAnt and MACS, a dif-

12

ferent weight is associated to each ant. Moreover, pheromone matrices are updated with thebest and second-best solution for each objective, which is basically the same update methodused by COMPETants, and in our framework it corresponds to the best-of-objective setting forPheromoneUpdate and Nupd = 2. In the problem being solved in the original paper [9], therewas no clear definition of heuristic information for each objective, and, hence, the authors useda single heuristic matrix. However, in later publications [25], P-ACO uses multiple heuristic ma-trices, one for each objective, which are aggregated in the same way as the pheromone matrices.In our earlier work [19], we show that there are important differences between using one or twoheuristic matrices in P-ACO for the bi-objective TSP. In particular, we observe that a singleheuristic matrix leads to a very narrow Pareto front and, hence, we use here multiple heuristicmatrices. Nonetheless, our MOACO framework can replicate both variants.

4.6 mACO Variant 1 (mACO-1)

Alaya et al. [1] proposed four alternatives for the design of a MOACO algorithm. The first variant,mACO-1, is described as using multiple-colonies: one colony per objective, and an extra colonythat builds solutions by aggregating the pheromone matrices of the other two colonies followinga random aggregation. Each colony uses the heuristic information of its corresponding objective,whereas the extra colony aggregates the heuristic matrices. In our framework, this proposal isformulated as a single colony with multiple pheromone and heuristic matrices aggregated usingthree weights Λ = {0, 0.5, 1}. In the original mACO-1, the pheromone information of each colonyis updated with the best solution generated by the colony for the corresponding objective of thesame colony. Moreover, the algorithm keeps a best solution generated by the extra colony foreach objective, and uses them to update the corresponding pheromone matrix of the other twocolonies. Since each colony in the original corresponds to a different weight λ in our formulation,this update method corresponds in our framework to best-of-objective-per-weight with Nupd = 1.


The second variant (mACO-2) by Alaya et al. [1] only differs from mACO-1 in the pheromoneaggregation. In mACO-2, pheromone matrices are aggregated by summing the pheromone matri-ces of each objective. When the underlying ACO algorithm is scale invariant [6], like AS, MMASand ACS, this is equivalent to a weight λ = 0.5. Our previous work on the bi-objective TSPsuggests that there is not a significant different in quality between mACO-1 and mACO-2 [19].


The third variant proposed by Alaya et al. [1] uses a single pheromone matrix, which is up-dated by using all nondominated solutions (either from the iteration archive or the best-so-fararchive). Alaya et al. [1] emphasize that every pheromone value associated to a solution compo-nent is updated at most once, despite how many solutions contain it. This is in contrast withother algorithms, such as MOAQ, MACS, and BicriterionAnt, that use such a nondominatedPheromoneUpdate. However, we did not find any advantage in this special requirement, thereforeour framework does not include it for the sake of simplicity. The heuristic information is alsoa single matrix. In problems where there is heuristic information available for each objective,these are aggregated prior the run into a single heuristic matrix.

13


In the last variant proposed by Alaya et al. [1], there is one pheromone matrix per objective,and these are always aggregated in the same random manner as for mACO-1, which in practicecorresponds to a random aggregation with a single weight λ = 0.5. However, there is a singleheuristic matrix, as in mACO-3. Finally, each pheromone matrix is updated with the bestsolution for each objective, which is the definition of the best-of-objective PheromoneUpdate usedin our framework.

4.10 New Design Alternatives

From the review in this section, we can say that, despite the number of different MOACOalgorithms proposed in the literature, there is a much larger number of unexplored combinationsof their algorithmic components. In particular, multi-colony variants in the sense of Iredi etal. [15] can be defined for all the algorithms reviewed in this section. The random aggregationintroduced by Alaya et al. [1] has only been tested with λ = 0.5 so far. Moreover, all thealgorithms reviewed above use all weights available in each iteration (all-weights-per-iteration).In our earlier work for the bi-objective QAP [20], we proposed that all ants use the same weightin one iteration, and the next weight in the sequence in the next iteration (one-weight-per-iteration). Therefore, we can easily construct new variants of most algorithms in Table 2. Suchnovel variants can be considered new MOACO algorithms. However, we expect that many ofthem would not lead to any significant breakthrough. On the other hand, the particular problembeing solved may influence the choice of the best design. In any case, it would be a major effortto test them one by one in order to find the best design. Instead, we propose to automaticallyfind the best design for a particular problem by automatic (offline) configuration of the MOACOframework.

There are several automatic methods for offline configuration of single-objective optimizationalgorithms. In an earlier work, we have extended Iterated F-Race (I/F-Race) [7] to the multi-objective case by using unary quality measures [18]. In the next section, we apply this approachto the framework described in this paper and carry out a more detailed analysis of the results.

5 Automatic Configuration of MOACO framework

Instead of the classical “horse-race” between fully instantiated MOACO algorithms that wouldentail a great deal of human effort, we exploit automated algorithm configuration tools to obtainvery high-performing algorithmic variants. The goal of automated (offline) configuration is tofind the best parameter settings of an algorithm to solve unseen instances of a problem, givena set of training instances representative of the same problem. We are inspired by the workof KhudaBukhsh et al. [16], who combined ideas from many different algorithms for SAT intoa highly configurable SAT solver, and then used automatic configuration tools to find the bestconfiguration for specific classes of SAT instances. Instead of configuring a problem-specificsolver for particular instance classes, we aim to configure a problem-independent metaheuristicfor a specific problem. In other words, our goal is to find a good instantiation of a metaheuristicfrom a large space of potential designs. Moreover, we extend this idea for the first time tothe multi-objective context. We show in this section that the automatic configuration of ourflexible MOACO framework allow us to find better MOACO algorithms for the bTSP than thoseavailable in the literature. We also study different configuration strategies, and examine theresults of several independent automatic configuration runs.

14

The automatic configuration method used here is a new implementation of I/F-Race [7].which is a state-of-the-art automatic configuration method able to deal with continuous, cat-egorical and conditional parameters. I/F-Race alternates between generating new candidateconfigurations and performing races to discard the worst-performing ones. Within a race, can-didate configurations are run on one instance at a time. I/F-Race uses the Friedman test todiscard configurations whenever there is sufficient statistical evidence that they perform worsethan the rest. When only a small number of configurations remain in the race, the race stops. Anew race starts with the best configurations previously found and with new candidate configura-tions generated from the best configurations using a simple probabilistic model. The automaticconfiguration process stops after reaching a given maximum budget (number of runs or timelimit).

The current version of I/F-Race was designed for single-objective optimization problems, and,hence, it requires an evaluation criterion that assigns a single value to each run of a configuration.We apply I/F-Race to the multi-objective context by means of unary quality measures, whichassign a single quality value to a nondominated set. In particular, we test two unary measuresas the evaluation criterion of I/F-Race, namely, the hypervolume1 and the (additive) epsilonmeasure [32]. The hypervolume is the volume of the objective space weakly dominated by a non-dominated set and bounded by a reference point that is strictly dominated by all Pareto-optimalobjective vectors. The larger the hypervolume, the better is the corresponding nondominatedset. The additive epsilon measure provides the minimum value that must be subtracted from allobjectives of a nondominated set so that it weakly dominates a reference set. This reference setis usually the nondominated set of all known solutions. A smaller epsilon measure is preferable.

As training instances, we generated 36 bTSP random uniform Euclidean instances for eachof n = {100, 200, 300} nodes (108 instances in total). We use a different set of 12 (test) instanceswhen comparing algorithms. These test instances are taken from Luis Paquete’s webpage athttp://eden.dei.uc.pt/~paquete/tsp.2 Each experiment, that is, each run of the MOACOframework on each instance, is stopped after 300 · (n/100)2 CPU-seconds.

As the underlying ACO algorithm, we useMMAS as defined for the TSP [27]. In particular,we use the default settings described in Table 3, we set ∆τ = 1 for the amount of pheromonedeposited by an ant, and we do not use candidate lists for the solution construction. Followingprevious work [28], we also incorporate the pseudo-random action choice rule of ACS [10], whichallows for a greedier solution construction. Parameter q0 controls the greediness of the pseudo-random action choice rule. A value of q0 = 0 disables it and reverts back to the traditionalMMAS. The MOACO framework is implemented in C, and the underlying ACO algorithmis derived from ACOTSP [26]. The code is compiled with gcc, version 4.4. All experimentsreported in the following are carried out on a single core of Intel Xeon E5410 CPUs, runningat 2.33 Ghz with 6MB of cache size under Cluster Rocks Linux version 4.2.1/CentOS 4. Theimplementation is sequential and experiments run on a single core.

We divide the configuration effort in three stages:

• First, we study whether it is possible to automatically find a novel MOACO design. Hence,we configure only the multi-objective components of the MOACO framework, and we com-pare the results with the algorithms described in the literature.

• Second, we assess how much improvement may be achieved by configuring the parametersettings of the underlying ACO algorithm. We do so by automatically configuring the ACOalgorithm settings of one of the configurations found in the previous stage.

1The hypervolume is calculated using the algorithm proposed by Fonseca et al. [13].2We use instances euclidAB100, euclidAB300, euclidAB500, euclidCD100, euclidCD300, euclidCD500,

euclidEF100, euclidEF300, kroAB100, kroAB200, kroAD100, and kroBC100

15

http://eden.dei.uc.pt/~paquete/tsp

Table 3: Default parameter settings of the underlying ACO algorithm (MMAS).

Parameter Value

Na 24 · (n/100)ρ 0.02 (n < 300), 0.05 (n ≥ 300)q0 0α 1β 2

• Finally, we ask whether this two-stage configuration approach is better than configuringall components and parameters at once using a configuration effort equivalent to the twoprevious stages.

5.1 Configuration of multi-objective components

In the first experiment, our goal is to find a new, hopefully better design of a MOACO algorithmfor the bTSP. We search for this design in an automatic fashion by configuring the multi-objectivecomponents of the MOACO framework, while keeping fixed the underlying ACO algorithm pa-rameter settings. The configuration domain of the multi-objective components is described inTable 4, whereas the fixed ACO algorithm settings are given in Table 3. The configurationbudget is set to 1000 runs of the MOACO framework. We perform five independent repetitionsof the configuration process using the hypervolume as the evaluation criterion, and another fiverepetitions using the unary epsilon measure.

To keep the paper short, the particular configurations found are given in the supplementarymaterial (Appendix A, Tables 6 and 7, for the hypervolume and epsilon measures respectively).All ten configurations found use multiple heuristic matrices, large number of colonies, updateby region and aggregation by weighted product. Most configurations use multiple pheromonematrices updated by the best-of-objective (BO), and one-weight-per-iteration (1wpi).

We apply these ten configurations to the test instances, and perform 15 independent runs ofeach configuration with different random seeds. For comparison, we also evaluate the MOACOalgorithms from the literature described in Section 4. We evaluate the quality of the nondomi-nated sets found in these test runs by means of both the hypervolume and epsilon measures. Thefull analysis is available as supplementary material (Appendix A, Figs. 4 and 5). The results leadto the same conclusions independently of the quality measure used in the analysis. Moreover,there were no significant differences, according to the sign test, between configurations foundwhen using the hypervolume as the evaluation criterion in I/F-Race and those found when usingthe epsilon measure. Hence, we discuss here only the results with respect to the hypervolumemeasure.

Figure 1 displays, for each instance, one boxplot per configuration (y-axis) summarizing thehypervolume value (x-axis) obtained by the 15 independent runs. The boxplots are often reducedto a single point because the variance is extremely small across runs. The results obtained bythe MOACO algorithms from the literature are consistent with previous works [14, 19], withthe multi-colony variant of BicriterionAnt being the best. Nonetheless, the improvement of theconfigurations obtained automatically, MOACO (1,2,3,4,5), over the MOACO algorithms fromthe literature is significantly large and consistent. In other words, our method is able to find anew design that surpasses the current state-of-the-art in the MOACO literature. Not surprisingly,this new design is similar to the multi-colony variant of BicriterionAnt. Yet, it includes significantdifferences, such as the use of best-of-objective update, a small value of Nupd, more than one

16

Table 4: Domain of algorithmic components of the proposed MOACO framework used for theautomatic configuration.

Component Domain Constraint

[τ ] { single, multiple }[η] { single, multiple }

Aggregation

weighted sum,

weighted product,

random

only if multiple τ or η

Nweights {2, 3, Na/3, Na/2, Na} (per colony)

NextWeight

{one weight per iteration,

all weights per iteration

PheromoneUpdate

nondominated solutions,

best-of-objective,

best-of-objective-per-weight∗

∗ only with N col = 1

Nupd {1, 2, 5, 10}

N col {1, 2, 3, 5, 10}

MultiColonyWeights { disjoint, overlapping } only if N col > 1

MultiColonyUpdate { origin, region } only if N col > 1

17

Table 5: Domain of the parameter settings of the underlying ACO algorithm (MMAS) used forautomatic configuration.

Component Domain

af {1, 2, . . . , 25} where Na = 6 · af · (n/100)

ρ [0.01, 0.99]

q0 [0.25, 0.99] or q0 = 0

α [0, 5]

β [0, 5]

ant per weight, and one-weight-per-iteration (1wpi).

5.2 Configuration of the underlying ACO algorithm

In the next experiment, we arbitrarily choose one of the configurations found when configuringthe multi-objective components of the MOACO framework,3 and we carry out an automatic con-figuration of the settings of the underlying ACO algorithm. The domains of the ACO algorithmsettings are described in Table 5. Instead of configuring directly the number of ants per colonyNa, we configure a surrogate parameter af that defines the number of ants in dependence of theinstance size n, that is, Na = 6 · af · (n/100). We multiply this factor by six in order to be ableto define Nweights as either Na/3 or Na/2 in later experiments.

We perform five independent repetitions of the configuration process using the hypervol-ume as the evaluation criterion, and each repetition is stopped after 1000 runs of the MOACOframework. For comparison, we also configure in this way the ACO algorithm settings of themulti-colony version of BicriterionAnt, which was found to be the best MOACO algorithm fromthe literature in the previous section. The resulting configurations of parameter settings aregiven as supplementary material (Appendix A, Tables 8 and 10, for the MOACO framework andBicriterionAnt respectively. Tables 9 and 11 display the results when configuring with respectto the epsilon measure).

One may ask whether it would not be better to configure both the multi-objective componentsand the ACO parameter settings of the MOACO framework at once, instead of consecutively asdone above. To answer this question, we carry out the automatic configuration of all parametersof the MOACO framework at once, with the domains described in Tables 4 and 5, and with abudget of 2000 experiments, that is, equivalent to the combined effort spent in the previous two-stage automatic configuration process. The resulting configurations are given as supplementarymaterial (Appendix A, Tables 12 and 13, when using the hypervolume and epsilon measures asthe evaluation criterion, respectively).

For the comparison of the configurations found in the various automatic configuration proce-dures, we perform 15 independent runs of each configuration on the test instances. We evaluatethe results using the hypervolume measure, and the corresponding boxplots are shown in Fig. 2.4

The plots show that automatically configuring the ACO algorithm settings of BicriterionAntgreatly improves its performance, and for instances with 500 cities, it surpasses the quality ofthe MOACO configuration with default ACO algorithm settings. Nonetheless, by automati-

3Concretely, run 5 of Table 6 in the supplementary material (Appendix A).4Similar figures that include results with respect to the epsilon measure are available as supplementary material

(Appendix A, Figs. 8 and 9).

18

Hypervolume

MOACO (5)MOACO (4)MOACO (3)MOACO (2)MOACO (1)

mACO−4mACO−3mACO−2mACO−1

PACOCOMPETants

MACSBicriterionAnt (3 col)BicriterionAnt (1 col)

MOAQ

0.5 0.6 0.7 0.8 0.9 1.0 1.1

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

euclidAB100.tsp

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

euclidCD100.tsp

0.5 0.6 0.7 0.8 0.9 1.0 1.1

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

euclidEF100.tsp

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●●●●●

euclidAB300.tspMOACO (5)MOACO (4)MOACO (3)MOACO (2)MOACO (1)


PACOCOMPETants


MOAQ

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

euclidCD300.tsp

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●●●

●●

●●

euclidEF300.tsp

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

euclidAB500.tsp

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●●●

euclidCD500.tspMOACO (5)MOACO (4)MOACO (3)MOACO (2)MOACO (1)


PACOCOMPETants


MOAQ

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

kroAB100.tsp

0.5 0.6 0.7 0.8 0.9 1.0 1.1

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

kroAD100.tsp

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

kroBC100.tsp

0.5 0.6 0.7 0.8 0.9 1.0 1.1

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●●

●

●●●

kroAB200.tsp

Figure 1: Boxplot of the hypervolume (larger is better) of 15 repetitions of each configurationof the MOACO framework on the test instances. The configurations from the literature areMOAQ, BicriterionAnt with 1 and 3 colonies, COMPETants, PACO, MACS, and the four vari-ants of mACO (Section 4). MOACO (1,2,3,4,5) denote configurations found by five independentautomatic configuration runs.

19

cally configuring the ACO algorithm settings of the MOACO framework, the improvement inhypervolume is even more remarkable, clearly outperforming the best configurations of Bicriteri-onAnt. The plot does not show a clear difference between automatically configuring the MOACOframework using the two-stage approach or by configuring all parameters at once. However, con-figurations obtained by a two-stage approach obtain a lower hypervolume, i.e., they are worse,than those fully configured in 80% of the runs. The sign-test confirms that this difference isstatistically significant.

6 MOACO framework + local search (MOACO+ls)

The results presented above conclusively show that automatic configuration leads to better de-signs of MOACO algorithms than those present in the literature. Although a large amount ofwork on MOACO algorithms do not consider the usage of local search [1, 14], it is known that, onthe bTSP, MOACO algorithms with local search by far outperform those without local search [17].Therefore, we repeat the automatic configuration hybridizing the MOACO framework and localsearch.

6.1 Experimental Setup

The local search used here is an iterative improvement algorithm based on the 2-exchange neigh-borhood (2-opt). Local search is applied to each solution constructed by an ant. The bTSP isconverted to a single-objective TSP by means of a weighted sum aggregation of the two distancematrices. If an ant uses a weight vector to construct a solution, the subsequent local search willuse the same weight vector. Otherwise, if ants do not use weights for the solution construction,a weight is still assigned to each ant following the settings described in Section 3 but they areonly used by the local search. The local search also exploits standard speed-up techniques forthe TSP, e.g. an ordered list of candidate edges of size 20 is computed for each weight vector.The ants use a different candidate list of size 20 for construction, which is obtained by sortingedges according to dominance ranking [21].

The remaining parameters of the MOACO framework are equal to the previous settingswithout local search (Tables 3, 4, and 5) except for: the default value of ρ is 0.2; the number ofants does not depend on the instance size and it is now calculated as Na = 6 · af ; and the timelimit of each run is now 4 · (n/100)2.

As training instances, we generated 10 bTSP random uniform Euclidean instances for eachof n = {500, 600, 700, 800, 1000} nodes (60 instances in total). The comparison and evaluationof the configurations obtained use a different set of 15 test instances, 3 instances of each size n.

6.2 Automatic configuration of MOACO+ls

We carry out four separate automatic configuration setups of the MOACO+ls framework for thebTSP.

1. We configure the multi-objective components of the MOACO framework and use defaultsettings for the underlying ACO algorithm.

2. We configure the underlying ACO algorithm settings and use one of the MOACO designsfound in the previous step.

3. We configure all settings of the MOACO framework at once using two times the budget ofexperiments as for the previous two steps. The aim of this setup is to investigate whether

20

Hypervolume

MOACO−full (5)MOACO−full (4)MOACO−full (3)MOACO−full (2)MOACO−full (1)

MOACO−aco (5)MOACO−aco (4)MOACO−aco (3)MOACO−aco (2)MOACO−aco (1)

MOACO (5)BicriterionAnt−aco (5)BicriterionAnt−aco (4)BicriterionAnt−aco (3)BicriterionAnt−aco (2)BicriterionAnt−aco (1)

BicriterionAnt (3 col)

0.85 0.95 1.05

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

euclidAB100.tsp

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

euclidCD100.tsp

0.85 0.95 1.05

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●●●

●●

euclidEF100.tsp

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●●

●

●

●

●

●

euclidAB300.tspMOACO−full (5)MOACO−full (4)MOACO−full (3)MOACO−full (2)MOACO−full (1)




●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●●

●

euclidCD300.tsp

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

euclidEF300.tsp

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

euclidAB500.tsp

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●●●●

euclidCD500.tspMOACO−full (5)MOACO−full (4)MOACO−full (3)MOACO−full (2)MOACO−full (1)




●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●●

kroAB100.tsp

0.85 0.95 1.05

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

kroAD100.tsp

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

kroBC100.tsp

0.85 0.95 1.05

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

kroAB200.tsp

Figure 2: Boxplot of the hypervolume (larger is better) of 15 repetitions of each configuration ofthe MOACO framework on the test instances. BicriterionAnt (3 col) denotes BicriterionAnt with3 colonies and default ACO algorithm settings; BicriterionAnt-aco (1,2,3,4,5) denote the sameconfiguration but with automatically configured ACO settings (five independent configurationruns); MOACO (5) denotes the configuration obtained when configuring the multi-objective com-ponents with default ACO settings; MOACO-aco (1,2,3,4,5) denote the same configuration butwith automatically configured ACO settings (five independent configuration runs); and MOACO-full (1,2,3,4,5) denote five configurations obtained by automatically configuring all parametersettings.

21

a two-stage configuration approach is more or less effective than configuring all parametersat once.

4. We configure the ACO algorithm settings of BicriterionAnt+ls for comparison with theother results.

we perform five repetitions of each automatic configuration setup, and the budget of eachrepetition is set to 1000 experiments (2000 for configuring all settings at once). The configurationtool uses the hypervolume measure as the evaluation criteria. For the sake of conciseness, theparticular configurations found are reported in the supplementary pages (Appendix A, Tables 14,15, 16, 17). It is enough to say here that the configurations show a larger variability of settings,explained by the fact that the local search compensates for effect of other parameters.

The configurations found automatically are evaluated on the test instances by performing 15independent runs of each configuration, and computing the hypervolume measure of the resultingnondominated sets. Figure 3 summarizes with boxplots the hypervolume values of each config-uration on each instance. The results follow the conclusions outlined in the previous section.First, the performance of BicriterionAnt is greatly improved by configuring its ACO algorithmsettings. Second, despite this improvement, the automatically configured BicriterionAnt does notoutperform an automatically-found design of the MOACO framework with default ACO algo-rithm settings. Moreover, automatically configuring the ACO algorithm settings of such a gooddesign leads to further improved quality, clearly outperforming the best results of BicriterionAnt.Finally, although the boxplots do not visually show a clear advantage of the two-stage configu-ration process versus fully configuring the MOACO+ls framework at once, a sign-test indicatesthat there is a significantly larger probability, about 84%, of obtaining a larger hypervolume witha configuration obtained by the latter method. This is consistent with the conclusions reachedwhen automatically configuring the MOACO framework without local search.

These results clearly indicate that we have found new designs of MOACO algorithms thatoutperform the previous state of the art in the MOACO literature for the bTSP. More impor-tantly, we have found these designs in a semi-automatic fashion, where most of the effort has beenspent on the definition of the alternative design choices, leaving the task of finding the correctdesign to automatic (and unbiased) tools. The fact that some design choices never appear insome configurations gives a clear indication that they are not contributing to the performanceof the algorithm. On the other hand, the variability of designs indicates that there are severalalternative designs that produce similar quality of results.

7 Conclusion

Instead of proposing a new MOACO algorithm that is slightly different from those found in theliterature, we have proposed a novel approach to automatically design MOACO algorithms forspecific multi-objective optimization problems. We have synthesized the existing knowledge fromthe MOACO algorithms found in the literature into a flexible, configurable framework, and wehave explored the vast space of potential algorithmic designs in an automatic fashion by usingoffline configuration tools.

In this paper we have shown that such an approach has both scientific and practical ad-vantages. From a scientific standpoint, the process of synthesis involved in constructing theframework has led to identify many more commonalities in existing MOACO algorithms thanpreviously thought. In some cases, some algorithms were found to be specializations of otheralgorithms, despite the fact that both were independently proposed in the literature. In addi-tion, the integration of design choices from diverse algorithms within the same framework leads

22

Hypervolume




BicriterionAnt−aco (5)BicriterionAnt−aco (4)BicriterionAnt−aco (3)BicriterionAnt−aco (2)BicriterionAnt−aco (1)


1.125 1.135

●●●●

●●●●●●

●●●●●

●●●●●

●

●●

●●

●

600−67

●●●●●●●●●●

●●●●●

●●●●●

●

●

●●●

●

●

●

●

●

●●●

700−67

1.125 1.135

●●●●●●●●●●

●●●●●

●●●●●

●

●

●

●

800−67

●●●●●●●●●●

●●●●●

●●●●●

●

●

●

900−67

1.125 1.135

●●●●●●●●●●

●●●●●●●●●●

●

●●

●

●●

1000−67MOACO−full (5)MOACO−full (4)MOACO−full (3)MOACO−full (2)MOACO−full (1)





●●●●●●●●●●

●●●●●

●●●●●

●

●

●

●

●

●

●

600−78

●●●●

●●●●●●

●●●●●

●●●●●

●

●

●

●

●

●

●

●●

700−78

●●●●●●●●●●

●●●●●●●●●●

●

●

●

●

●●●

800−78

●●●●●●●●●●

●●●●●●●●●●

●

●

●

●

●

900−78

●●●●●●●●●●

●●●●●●●●●●

●

●

●●●●

●

●●

1000−78MOACO−full (5)MOACO−full (4)MOACO−full (3)MOACO−full (2)MOACO−full (1)





●●●●●●●●●●

●●●●●

●●●●●

●

●●

●

●

●

●

●●

●

●

●

600−89

1.125 1.135

●●●●●●●●●●

●●●●●

●●●●●

●

●●

●●

●

●

●

●

700−89

●●●●●●●●●●

●●●●●

●●●●●

●

●●

● ●

●●

●

●

●

●●

800−89

1.125 1.135

●●●●●●●●●●

●●●●●●●●●●

●

●

●

●●

●

900−89

●●●●●●●●●●

●●●●●●●●●●

●

●

●●

●●

●

1000−89

Figure 3: Boxplot of the hypervolume (larger is better) of 15 repetitions of each configuration ofthe MOACO+ls framework on the test instances. BicriterionAnt (3 col) denotes BicriterionAntwith 3 colonies and default ACO algorithm settings; BicriterionAnt-aco (1,2,3,4,5) denote thesame configuration but with automatically configured ACO settings (five independent configura-tion runs); MOACO (1,2,3,4,5) denote five configurations obtained by automatically configuringthe multi-objective components with default ACO settings; MOACO-aco (1,2,3,4,5) denote theconfiguration MOACO (5) but with automatically configured ACO settings (five independent con-figuration runs); and MOACO-full (1,2,3,4,5) denote five configurations obtained by automaticallyconfiguring all parameter settings.

23

to a large number of potential new designs, and each of them can be considered a new MOACOalgorithm by itself.

From a practical point of view, testing all possible designs on different problems would be amajor effort. The use of automatic configuration tools greatly simplifies this task by identifyingwhich design choices produce the best results for a particular problem. Our experimental resultshave shown that such an automatically configured design outperforms the MOACO algorithmsfound in the literature, even after the settings of the latter have been properly configured (whichis almost never the case in the MOACO literature). In this sense, we conclude that we havedesigned a new state-of-the-art MOACO algorithm for the bTSP.

There are several research questions that we have left open in this paper. First, furtheranalysis of the best configurations obtained should shed some light on the causes behind thegood performance of some design choices. Second, there are open questions about the best wayto automatically configure multi-objective optimization algorithms. Our experiments have notfound any significant difference between configuring using the hypervolume or the epsilon mea-sure, however, further research is needed to confirm whether this generalizes to other problemsor algorithms. Third, we have found that there is a small advantage in configuring all settingsat once rather than configuring first the “design” of a MOACO algorithm and then its ACOsettings. This may be due to interactions between the different components. More research isneeded to confirm whether this result generalizes to other configuration scenarios.

In the future, we will apply the same technique to other bi-objective problems already tackledby MOACO algorithms. We make available our implementation of the MOACO framework forthe bTSP,5 and we welcome extensions of the proposed framework, either by applying it to newproblems or by extending its components to include recent advances in MOACO algorithms. Weexpect that, properly configured, the MOACO framework will outperform any other MOACOalgorithm, or at worst, match their results. However, the ideas presented here can be extendedto other multi-objective metaheuristics, such as multi-objective evolutionary algorithms, wherevarious design choices exist without a clear configuration of choices regarded as the best.

Acknowledgments.

This work was supported by the META-X project, an Action de Recherche Concertee fundedby the Scientific Research Directorate of the French Community of Belgium. Thomas Stutzleacknowledges support from the Belgian F.R.S.-FNRS, of which he is a Research Associate. Theauthors also acknowledge support from the FRFC project “Methodes de recherche hybrides pourla resolution de problemes complexes”.

References

[1] I. Alaya, C. Solnon, and K. Ghedira. Ant colony optimization for multi-objective optimiza-tion problems. In 19th IEEE International Conference on Tools with Artificial Intelligence(ICTAI 2007), volume 1, pages 450–457. IEEE Computer Society Press, Los Alamitos, CA,2007.

[2] D. Angus and C. Woodward. Multiple objective ant colony optimization. Swarm Intelligence,3(1):69–85, 2009.

[3] P. Balaprakash, M. Birattari, and T. Stutzle. Improvement strategies for the F-race al-gorithm: Sampling design and iterative refinement. In T. Bartz-Beielstein, M. J. Blesa,

5Source code is available at http://iridia.ulb.ac.be/~manuel/moaco

24

http://iridia.ulb.ac.be/~manuel/moaco

C. Blum, B. Naujoks, A. Roli, G. Rudolph, and M. Sampels, editors, Hybrid Metaheuristics,volume 4771 of Lecture Notes in Computer Science, pages 108–122. Springer, Heidelberg,Germany, 2007.

[4] B. Baran and M. Schaerer. A multiobjective ant colony system for vehicle routing problemwith time windows. In Proceedings of the Twentyfirst IASTED International Conference onApplied Informatics, pages 97–102, Insbruck, Austria, 2003.

[5] M. Birattari. Tuning Metaheuristics: A Machine Learning Perspective, volume 197 of Stud-ies in Computational Intelligence. Springer, Berlin / Heidelberg, 2009.

[6] M. Birattari, P. Pellegrini, and M. Dorigo. On the invariance of ant colony optimization.IEEE Transactions on Evolutionary Computation, 11(6):732–742, 2007.

[7] M. Birattari, Z. Yuan, P. Balaprakash, and T. Stutzle. F-race and iterated F-race: Anoverview. In T. Bartz-Beielstein, M. Chiarandini, L. Paquete, and M. Preuss, editors,Experimental Methods for the Analysis of Optimization Algorithms, pages 311–336. Springer,Berlin, Germany, 2010.

[8] K. F. Doerner, R. F. Hartl, and M. Reimann. Are CompetAnts more competent for problemsolving? The case of a multiple objective transportation problem. Central European Journalfor Operations Research and Economics, 11(2):115–141, 2003.

[9] K. F. Doerner, W. J. Gutjahr, R. F. Hartl, C. Strauss, and C. Stummer. Pareto ant colonyoptimization: A metaheuristic approach to multiobjective portfolio selection. Annals ofOperations Research, 131:79–99, 2004.

[10] M. Dorigo and L. M. Gambardella. Ant Colony System: A cooperative learning approachto the traveling salesman problem. IEEE Transactions on Evolutionary Computation, 1(1):53–66, 1997.

[11] M. Dorigo and T. Stutzle. Ant Colony Optimization. MIT Press, Cambridge, MA, 2004.

[12] M. Ehrgott and X. Gandibleux. Approximative solution methods for combinatorial multi-criteria optimization. TOP, 12(1):1–88, 2004.

[13] C. M. Fonseca, L. Paquete, and M. Lopez-Ibanez. An improved dimension-sweep algo-rithm for the hypervolume indicator. In Proceedings of the 2006 Congress on EvolutionaryComputation (CEC 2006), pages 1157–1163. IEEE Press, Piscataway, NJ, July 2006.

[14] C. Garcıa-Martınez, O. Cordon, and F. Herrera. A taxonomy and an empirical analysisof multiple objective ant colony optimization algorithms for the bi-criteria TSP. EuropeanJournal of Operational Research, 180(1):116–148, 2007.

[15] S. Iredi, D. Merkle, and M. Middendorf. Bi-criterion optimization with multi colony antalgorithms. In E. Zitzler, K. Deb, L. Thiele, C. A. Coello, and D. Corne, editors, Evolution-ary Multi-criterion Optimization (EMO 2001), volume 1993 of Lecture Notes in ComputerScience, pages 359–372. Springer, Heidelberg, Germany, 2001.

[16] A. R. KhudaBukhsh, L. Xu, H. H. Hoos, and K. Leyton-Brown. SATenstein: Automaticallybuilding local search SAT solvers from components. In Proceedings of the Twenty-FirstInternational Joint Conference on Artificial Intelligence (IJCAI-09), pages 517–524, 2009.

25

[17] M. Lopez-Ibanez and T. Stutzle. An analysis of algorithmic components for multiobjectiveant colony optimization: A case study on the biobjective TSP. In P. Collet et al., editors,Artificial Evolution, volume 5975 of Lecture Notes in Computer Science, pages 134–145.Springer, Heidelberg, Germany, 2010.

[18] M. Lopez-Ibanez and T. Stutzle. Automatic configuration of multi-objective ACO algo-rithms. In M. Dorigo et al., editors, Ant Colony Optimization and Swarm Intelligence, 7thInternational Conference, ANTS 2010, volume 6234 of Lecture Notes in Computer Science,pages 95–106. Springer, Heidelberg, Germany, 2010.

[19] M. Lopez-Ibanez and T. Stutzle. The impact of design choices of multi-objective ant colonyoptimization algorithms on performance: An experimental study on the biobjective TSP. InM. Pelikan and J. Branke, editors, Proceedings of the Genetic and Evolutionary ComputationConference, GECCO 2010, pages 71–78. ACM press, New York, NY, 2010.

[20] M. Lopez-Ibanez, L. Paquete, and T. Stutzle. On the design of ACO for the biobjectivequadratic assignment problem. In M. Dorigo et al., editors, Ant Colony Optimization andSwarm Intelligence, 4th International Workshop, ANTS 2004, volume 3172 of Lecture Notesin Computer Science, pages 214–225. Springer, Heidelberg, Germany, 2004.

[21] T. Lust and A. Jaszkiewicz. Speed-up techniques for solving large-scale biobjective TSP.Computers & Operations Research, 37(3):521–533, 2010.

[22] T. Lust and J. Teghem. Two-phase Pareto local search for the biobjective traveling salesmanproblem. Journal of Heuristics, 16(3):475–510, 2010.

[23] C. E. Mariano and E. Morales. MOAQ: An Ant-Q algorithm for multiple objective optimiza-tion problems. In W. Banzhaf et al., editors, Proceedings of the Genetic and EvolutionaryComputation Conference, GECCO 1999, pages 894–901. Morgan Kaufmann Publishers, SanFrancisco, CA, 1999.

[24] L. Paquete and T. Stutzle. Design and analysis of stochastic local search for the multiob-jective traveling salesman problem. Computers & Operations Research, 36(9):2619–2631,2009.

[25] M. Schilde, K. F. Doerner, R. F. Hartl, and G. Kiechle. Metaheuristics for the bi-objectiveorienteering problem. Swarm Intelligence, 3(3):179–201, 2009.

[26] T. Stutzle. ACOTSP: A software package of various ant colony optimization algo-rithms applied to the symmetric traveling salesman problem, 2002. URL http://www.

aco-metaheuristic.org/aco-code/.

[27] T. Stutzle and H. H. Hoos. MAX -MIN . Future Generation Computer Systems, 16(8):889–914, 2000.

[28] T. Stutzle and H. H. Hoos. MAX -MIN and local search for combinatorial optimizationproblems. In S. Voss, S. Martello, I. Osman, and C. Roucairol, editors, Meta-Heuristics:Advances and Trends in Local Search Paradigms for Optimization, pages 137–154. KluwerAcademic Publishers, Dordrecht, The Netherlands, 1999.

[29] T. Stutzle, M. Lopez-Ibanez, and M. Dorigo. A concise overview of applications of ant colonyoptimization. In Encyclopedia of Operations Research and Management Science. John Wiley& Sons, New York, NY, 2011.

26

http://www.aco-metaheuristic.org/aco-code/

http://www.aco-metaheuristic.org/aco-code/

[30] S. Wessing, N. Beume, G. Rudolph, and B. Naujoks. Parameter tuning boosts performanceof variation operators in multiobjective optimization. In R. Schaefer, C. Cotta, J. Kolodziej,and G. Rudolph, editors, Parallel Problem Solving from Nature, PPSN XI, volume 6238 ofLecture Notes in Computer Science, pages 728–737. Springer, Heidelberg, Germany, 2010.

[31] E. Zitzler, M. Laumanns, and L. Thiele. SPEA2: Improving the strength Pareto evolutionaryalgorithm for multiobjective optimization. In K. Giannakoglou et al., editors, EvolutionaryMethods for Design, Optimisation and Control, pages 95–100. CIMNE, Barcelona, Spain,2002.

[32] E. Zitzler, L. Thiele, M. Laumanns, C. M. Fonseca, and V. Grunert da Fonseca. Performanceassessment of multiobjective optimizers: an analysis and review. IEEE Transactions onEvolutionary Computation, 7(2):117–132, 2003.

27

A Supplementary material

A.1 Configuring the multi-objective components of the MOACOframework

Table 6: Configurations obtained by five independent tuner runs, when configuring the multi-objective components of the proposed MOACO framework according to the hypervolume.

Component Run 1 Run 2 Run 3 Run 4 Run 5

[τ ] single multiple multiple multiple multiple[η] multiple multiple multiple multiple multiple

Aggregation w. product w. product w. product w. product w. productNweights Na/3 3 Na/2 Na/3 2

NextWeight 1wpi 1wpi 1wpi 1wpi 1wpiPheromoneUpdate ND BO BO BO BO

Nupd 1 2 2 2 1N col 10 5 10 10 10

MultiColonyWeights disjoint overlap disjoint overlap overlapMultiColonyUpdate region region region region region

Table 7: Configurations obtained by five independent tuner runs, when configuring the multi-objective components of the proposed MOACO framework according to the epsilon measure.


[τ ] multiple single multiple multiple multiple[η] multiple multiple multiple multiple multiple

Aggregation w. product w. product w. product w. product w. productNweights 2 Na/3 Na/2 Na Na

NextWeight awpi 1wpi 1wpi 1wpi 1wpiPheromoneUpdate BO ND BO BO BO

Nupd 1 1 2 2 2N col 10 10 5 10 10

MultiColonyWeights overlap overlap overlap overlap overlapMultiColonyUpdate region region region region region

28

Hypervolume

MOACO−eps (5)MOACO−eps (4)MOACO−eps (3)MOACO−eps (2)MOACO−eps (1)

MOACO−hv (5)MOACO−hv (4)MOACO−hv (3)MOACO−hv (2)MOACO−hv (1)


PACOCOMPETants


MOAQ

0.5 0.6 0.7 0.8 0.9 1.0 1.1

●●

●●●●●●

●●

●●

●●●

●●

●●

●

●

●

euclidAB100.tsp

●●

●●●●●●●●

●●

●●

●●

●●

●●

●

●

●

●●

●

●

euclidCD100.tsp

0.5 0.6 0.7 0.8 0.9 1.0 1.1

●●

●●●●●●●●

●●

●●

●●

●●

●●

●

●●

●

●

●

●

●

●●

●

euclidEF100.tsp

●●

●●●●●●

●●

●●

●●

●●

●●

●●

●●

●

●●●

●

●●

●

●●

●

●●●●●

euclidAB300.tspMOACO−eps (5)MOACO−eps (4)MOACO−eps (3)MOACO−eps (2)MOACO−eps (1)



PACOCOMPETants


MOAQ

●●

●●●●●●

●●

●●

●●

●●

●●

●●

●

●

●

●

●

euclidCD300.tsp

●●

●●●●●●

●●

●●

●●

●●

●●

●●

●

●●

●●●●●

●●

●●

euclidEF300.tsp

●●

●●●●●●

●●

●●

●●

●●

●●●

●

●

●

●

●●

●

euclidAB500.tsp

●●

●●●●●●

●●

●●

●●

●●

●●●

●

●●

●●

●

●●

●

●●●

euclidCD500.tspMOACO−eps (5)MOACO−eps (4)MOACO−eps (3)MOACO−eps (2)MOACO−eps (1)



PACOCOMPETants


MOAQ

●●

●●●●●●●●

●●

●●●

●●

●●

●

●

●

●

●●

●

●

●●

kroAB100.tsp

0.5 0.6 0.7 0.8 0.9 1.0 1.1

●●

●●●●●●●●

●●

●●●

●●

●●

●

●

●

●

●●●

●

●

kroAD100.tsp

●●●●●●●●●●

●●

●●●

●●

●●

●

●

●

●●

●●

●

●

●●

●

kroBC100.tsp

0.5 0.6 0.7 0.8 0.9 1.0 1.1

●●●●●●●●●●

●●

●●

●●

●●●

●

●

●

●●

●

●

●●

●●

●●

●

●●●

kroAB200.tsp

Figure 4: Boxplot of the hypervolume (larger is better) of 15 repetitions of each configuration ofthe MOACO framework on the test instances. The configurations from the literature are MOAQ,BicriterionAnt with 1 and 3 colonies, COMPETants, PACO, MACS, and the four variants ofmACO (Section 4). MOACO-hv (1,2,3,4,5) and MOACO-eps (1,2,3,4,5) denote configurationsfound by five independent automatic configuration runs using hypervolume and epsilon measure,respectively, as evaluation criterion.

29

Additive epsilon measure

MOACO−eps (5)MOACO−eps (4)MOACO−eps (3)MOACO−eps (2)MOACO−eps (1)



PACOCOMPETants


MOAQ

0.1 0.2 0.3 0.4 0.5 0.6

●●●●●●●●●●

●●

●●●

●●

●●

●

●

●

●●

●

●

●

●

euclidAB100.tsp

●●●

●●●●●●●

●●

●●●

●●

●●

●

●

●

euclidCD100.tsp

0.1 0.2 0.3 0.4 0.5 0.6

●●●●●●●●●●

●●

●●●

●●

●●

●

●●●●

●●

●●

●

euclidEF100.tsp

●●●●●●●●●●

●●

●●

●●

●●●

●

●●

●●

●

●

●

●●

euclidAB300.tspMOACO−eps (5)MOACO−eps (4)MOACO−eps (3)MOACO−eps (2)MOACO−eps (1)



PACOCOMPETants


MOAQ

●●●●●●●●●●

●●

●●

●●

●●●

●

●●

●

●●

●●

euclidCD300.tsp

●●●●●●●●●●

●●

●●

●●

●●●

●

●●●

●●●

●

●

euclidEF300.tsp

●●●●●●●●●●

●●●

●●

●●

●●

●

●

●

●●●●●

●●

●

euclidAB500.tsp

●●●●●●●●●●

●●●

●●

●●

●●

●

●

●●

●

euclidCD500.tspMOACO−eps (5)MOACO−eps (4)MOACO−eps (3)MOACO−eps (2)MOACO−eps (1)



PACOCOMPETants


MOAQ

●●●●●●●●●●

●●●

●●

●●

●●

●

●

●

●●●

kroAB100.tsp

0.1 0.2 0.3 0.4 0.5 0.6

●●●●●●●●●●

●●

●●●

●●

●●

●

●●●

●●●

●●

●●

kroAD100.tsp

●●●●●●●●●●

●●

●●●

●●

●●

●

●●

●

●

●

●

●●●

●●

kroBC100.tsp

0.1 0.2 0.3 0.4 0.5 0.6

●●●●●●●●●●

●●●

●●

●●

●●

●

●

●●●

●

●

●●

●

●

●●

kroAB200.tsp

Figure 5: Boxplot of the epsilon measure (lower is better) of 15 repetitions of each configuration ofthe MOACO framework on the test instances. The configurations from the literature are MOAQ,BicriterionAnt with 1 and 3 colonies, COMPETants, PACO, MACS, and the four variants ofmACO (Section 4). MOACO-hv (1,2,3,4,5) and MOACO-eps (1,2,3,4,5) denote configurationsfound by five independent automatic configuration runs using hypervolume and epsilon measure,respectively, as evaluation criterion.

30

A.2 Configuring the ACO parameter settings of the MOACOframework

Table 8: Configurations obtained by configuring the ACO parameter settings of the MOACOframework according to the hypervolume.


af 18 19 19 11 12ρ 0.54 0.53 0.56 0.92 0.67q0 0.69 0.71 0.77 0.00 0.91α 1.59 1.71 1.46 1.97 0.96β 2.44 4.14 3.64 4.92 2.21

Table 9: Configurations obtained by configuring the ACO parameter settings of the MOACOframework according to the epsilon measure.


af 24 23 16 23 13ρ 0.82 0.52 0.39 0.46 0.93q0 0.89 0.54 0.88 0.65 0.70α 1.04 1.98 1.63 1.22 1.56β 2.79 4.46 3.57 2.65 3.95

Table 10: Configurations obtained by configuring the ACO parameter settings of BicriterionAnt(3 colonies) according to the hypervolume.


af 18 10 14 4 8ρ 0.98 0.99 0.82 0.87 0.94q0 0.99 0.92 0.91 0.98 0.94α 3.68 2.38 1.75 4.93 2.7β 3.1 2.94 4.95 2.23 2.26

Table 11: Configurations obtained by configuring the ACO parameter settings of BicriterionAnt(3 colonies) according to the epsilon measure


af 6 10 10 8 11ρ 0.96 0.91 0.79 0.68 0.95q0 0.89 0.95 0.92 0.9 0.93α 4.4 1.72 2.16 1.65 1.25β 3.64 4.92 4.53 4.82 3.97

31

Hypervolume

MOACO−aco−eps (5)MOACO−aco−eps (4)MOACO−aco−eps (3)MOACO−aco−eps (2)MOACO−aco−eps (1)

MOACO−aco−hv (5)MOACO−aco−hv (4)MOACO−aco−hv (3)MOACO−aco−hv (2)MOACO−aco−hv (1)

MOACO−hv (5)BicriterionAnt−aco−hv (5)BicriterionAnt−aco−hv (4)BicriterionAnt−aco−hv (3)BicriterionAnt−aco−hv (2)BicriterionAnt−aco−hv (1)


0.85 0.95 1.05

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

euclidAB100.tsp

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●●

●

●

●

euclidCD100.tsp

0.85 0.95 1.05

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●●

euclidEF100.tsp

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

euclidAB300.tspMOACO−aco−eps (5)MOACO−aco−eps (4)MOACO−aco−eps (3)MOACO−aco−eps (2)MOACO−aco−eps (1)




●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●●

●

●●

●

●●●

●

euclidCD300.tsp

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

euclidEF300.tsp

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

euclidAB500.tsp

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●●●●

euclidCD500.tspMOACO−aco−eps (5)MOACO−aco−eps (4)MOACO−aco−eps (3)MOACO−aco−eps (2)MOACO−aco−eps (1)




●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

kroAB100.tsp

0.85 0.95 1.05

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

kroAD100.tsp

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

kroBC100.tsp

0.85 0.95 1.05

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

kroAB200.tsp

Figure 6: Boxplot of the hypervolume (larger is better) of 15 repetitions of each configurationof the MOACO framework on the test instances. BicriterionAnt (3 col) denotes BicriterionAntwith 3 colonies and default ACO algorithm settings; BicriterionAnt-aco-hv (1,2,3,4,5) denote thesame configuration but with automatically configured ACO settings (five independent configura-tion runs); MOACO-hv (5) denotes the fifth configuration obtained when configuring the multi-objective components with default ACO algorithm settings; MOACO-aco-{hv,eps} (1,2,3,4,5)denote the same configuration but with automatically configured ACO settings (five indepen-dent configuration runs). The suffixes -hv and -eps denote that the configuration tool used,respectively, the hypervolume and the epsilon measure as the evaluation criterion.

32






0.00 0.05 0.10 0.15 0.20 0.25

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●●

●

●

●

●●●

●

euclidAB100.tsp

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

euclidCD100.tsp

0.00 0.05 0.10 0.15 0.20 0.25

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

● ●●

●

●

euclidEF100.tsp

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

euclidAB300.tspMOACO−aco−eps (5)MOACO−aco−eps (4)MOACO−aco−eps (3)MOACO−aco−eps (2)MOACO−aco−eps (1)




●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

euclidCD300.tsp

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

euclidEF300.tsp

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●●

●

euclidAB500.tsp

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

euclidCD500.tspMOACO−aco−eps (5)MOACO−aco−eps (4)MOACO−aco−eps (3)MOACO−aco−eps (2)MOACO−aco−eps (1)




●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●●

●●●

●

kroAB100.tsp

0.00 0.05 0.10 0.15 0.20 0.25

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●● ●

kroAD100.tsp

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

kroBC100.tsp

0.00 0.05 0.10 0.15 0.20 0.25

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

kroAB200.tsp

Figure 7: Boxplot of the epsilon measure (lower is better) of 15 repetitions of each configurationof the MOACO framework on the test instances. BicriterionAnt (3 col) denotes BicriterionAntwith 3 colonies and default ACO algorithm settings; BicriterionAnt-aco-hv (1,2,3,4,5) denote thesame configuration but with automatically configured ACO settings (five independent configura-tion runs); MOACO-hv (5) denotes the fifth configuration obtained when configuring the multi-objective components with default ACO algorithm settings; MOACO-aco-{hv,eps} (1,2,3,4,5)denote the same configuration but with automatically configured ACO settings (five indepen-dent configuration runs). The suffixes -hv and -eps denote that the configuration tool used,respectively, the hypervolume and the epsilon measure as the evaluation criterion.

33

Table 12: Configurations obtained by configuring all parameters of the proposed MOACO frame-work according to the hypervolume.


[τ ] single multiple single multiple single[η] multiple multiple multiple multiple multiple

Aggregation product product product product productNweights Na Na 2 2 2

NextWeight 1wpi 1wpi 1wpi 1wpi 1wpiPheromoneUpdate ND ND ND BO ND

Nupd 1 1 1 5 1N col 10 10 10 10 10

MultiColonyWeights disjoint overlapping overlapping overlapping overlappingMultiColonyUpdate region region region region region

a 20 10 10 20 17ρ 0.54 0.39 0.3 0.88 0.4q0 0.00 0.86 0.85 0.28 0.92α 3.14 2.07 3.62 2.24 1.43β 4.73 0.56 1.73 4.1 0.26

Table 13: Configurations obtained by configuring all parameters of the proposed MOACO frame-work according to the epsilon measure.


[τ ] multiple single multiple single single[η] multiple multiple multiple multiple multiple

Aggregation product product product product productNweights 2 Na Na Na Na/3

NextWeight awpi 1wpi 1wpi 1wpi 1wpiPheromoneUpdate BO ND ND ND ND

Nupd 2 1 1 1 1N col 10 10 5 10 10

MultiColonyWeights overlapping disjoint disjoint overlapping disjointMultiColonyUpdate region region region region region

a 20 12 10 15 14ρ 0.27 0.28 0.32 0.43 0.36q0 0.00 0.94 0.89 0.95 0.65α 2.78 3.23 2.33 4.95 4.99β 5 0.65 0.9 1.31 2.37

34

Hypervolume




MOACO−hv (5)

1.00 1.04 1.08 1.12

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

euclidAB100.tsp

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●●

●

●

euclidCD100.tsp

1.00 1.04 1.08 1.12

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●●●

euclidEF100.tsp

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●●

●●

●




MOACO−hv (5)

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●●

●

●●

●

euclidCD300.tsp

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

euclidEF300.tsp

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

euclidAB500.tsp

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●●




MOACO−hv (5)

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

● ●

●

kroAB100.tsp

1.00 1.04 1.08 1.12

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

kroAD100.tsp

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

kroBC100.tsp

1.00 1.04 1.08 1.12

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

kroAB200.tsp

Figure 8: Boxplot of the hypervolume (larger is better) of 15 repetitions of each configurationof the MOACO framework on the test instances. MOACO-hv (5) denotes the fifth configurationobtained when configuring the multi-objective components with default ACO settings; MOACO-aco-{hv,eps} (1,2,3,4,5) denote the same configuration but with automatically configured ACOsettings (five independent configuration runs); and MOACO-full (1,2,3,4,5) denote five configu-rations obtained by automatically configuring all parameter settings. The suffixes -hv and -epsdenote that the configuration tool used, respectively, the hypervolume and the epsilon measureas the evaluation criterion.

35





MOACO−hv (5)

0.02 0.06 0.10

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●●

●

euclidAB100.tsp

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

euclidCD100.tsp

0.02 0.06 0.10

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

● ●●

euclidEF100.tsp

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●




MOACO−hv (5)

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

euclidCD300.tsp

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

euclidEF300.tsp

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

euclidAB500.tsp

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●●● ●

●

●

●




MOACO−hv (5)

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

kroAB100.tsp

0.02 0.06 0.10

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

kroAD100.tsp

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●●●

●

kroBC100.tsp

0.02 0.06 0.10

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●● ●

kroAB200.tsp

Figure 9: Boxplot of the epsilon measure (lower is better) of 15 repetitions of each configurationof the MOACO framework on the test instances. MOACO-hv (5) denotes the fifth configurationobtained when configuring the multi-objective components with default ACO settings; MOACO-aco-{hv,eps} (1,2,3,4,5) denote the same configuration but with automatically configured ACOsettings (five independent configuration runs); and MOACO-full (1,2,3,4,5) denote five configu-rations obtained by automatically configuring all parameter settings. The suffixes -hv and -epsdenote that the configuration tool used, respectively, the hypervolume and the epsilon measureas the evaluation criterion.

36

A.3 Configuring MOACO+ls

Table 14: Configurations obtained by configuring the multi-objective components of the proposedMOACO+ls framework according to the hypervolume.


[τ ] multiple multiple multiple single multiple[η] multiple multiple multiple single multiple

Aggregation product product product — productNweights Na/2 Na/2 Na Na Na

NextWeight 1wpi 1wpi 1wpi 1wpi 1wpiPheromoneUpdate BO BO BO ND BO

Nupd 1 1 2 1 5N col 10 10 10 10 10

MultiColonyWeights disjoint disjoint disjoint disjoint disjointMultiColonyUpdate origin origin origin origin origin

Table 15: Configurations obtained by configuring the ACO parameter settings of the MOACO+lsframework according to the hypervolume.


af 20 18 21 23 24ρ 0.42 0.67 0.69 0.9 0.79q0 0.00 0.00 0.00 0.55 0.39α 4.84 3.61 4.92 4.91 3.86β 0.18 0.5 1.58 0.28 1.35

37

Table 16: Configurations obtained by configuring all parameters of the proposed MOACO+lsframework according to the hypervolume.


[τ ] single multiple multiple multiple multiple[η] multiple single multiple multiple single

Aggregation product product sum sum sumNweights Na Na/2 Na/3 Na/3 Na/3

NextWeight 1wpi 1wpi 1wpi 1wpi 1wpiPheromoneUpdate ND BO ND BO BO

Nupd 10 10 10 10 10N col 10 10 10 10 10

MultiColonyWeights overlapping overlapping overlapping overlapping overlappingMultiColonyUpdate origin origin origin origin region

af 23 24 24 22 24ρ 0.76 0.72 0.4 0.62 0.51q0 0.9 0.00 0.45 0.33 0.00α 2.92 4.52 3.68 4.92 4.73β 0.01 0.12 0.92 0.34 0.78

Table 17: Configurations obtained by configuring the ACO parameter settings ofBicriterionAnt+ls (3 colonies) according to the hypervolume.


af 14 25 17 20 23ρ 0.72 0.52 0.37 0.98 0.7q0 0.61 0.00 0.35 0.45 0.53α 3.84 4.95 4.35 4.22 3.68β 0.15 0.21 0.38 0.11 0.04

38

the automatic design of multi-objective ant colony...

Documents