an investigation into the suitability of the ant colony
TRANSCRIPT
An Investigation into the Suitability of the Ant Colony OptimizationHeuristic in Noisy and Dynamic Environments
by
Andrew Schoenrock100326415
Honours Project (COMP 4905)Submitted in fulfilment of therequirements for the degree of
Bachelor of Computer Science, Honours
at
Carleton UniversityOttawa, OntarioDecember, 2006
Supervised by Dr. Tony White
Contents
1 Abstract 5
2 Acknowledgements 7
3 Introduction 8
3.1 Inspiration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
3.2 NP-Hard Problems . . . . . . . . . . . . . . . . . . . . . . . . 9
3.3 Travelling Salesman problem . . . . . . . . . . . . . . . . . . . 9
3.4 Extensions to TSP . . . . . . . . . . . . . . . . . . . . . . . . 11
4 ACO Heuristic 13
4.1 Ant System . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
4.2 Ant Colony System . . . . . . . . . . . . . . . . . . . . . . . . 14
4.2.1 Transition Rule . . . . . . . . . . . . . . . . . . . . . . 14
4.2.2 Pheromone Trail Update Rule . . . . . . . . . . . . . . 15
4.2.3 Local Pheromone Updates . . . . . . . . . . . . . . . . 15
4.2.4 Pheromone Decay . . . . . . . . . . . . . . . . . . . . . 16
4.2.5 Candidate Lists . . . . . . . . . . . . . . . . . . . . . . 16
4.2.6 ASC Pseudocode . . . . . . . . . . . . . . . . . . . . . 17
5 Problem Descriptions and Algorithm
Modifications 18
5.1 Problem 1: Noisy Environment . . . . . . . . . . . . . . . . . 18
1
5.1.1 Adding Noise . . . . . . . . . . . . . . . . . . . . . . . 19
5.1.2 Adapting to Noise . . . . . . . . . . . . . . . . . . . . 20
5.2 Problem 2: Dynamic Environment . . . . . . . . . . . . . . . 22
5.2.1 Pheromone Lower Bound . . . . . . . . . . . . . . . . . 24
5.2.2 Local Pheromone Smoothing . . . . . . . . . . . . . . . 24
6 Test Results 26
6.1 Problem 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
6.2 Problem 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
7 Discussion 43
7.1 Problem 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
7.2 Problem 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
8 Looking Forward 47
9 References 49
List of Tables
1 Paramaters Used in Tests . . . . . . . . . . . . . . . . . . . . 26
2 51 City Test Problem Used . . . . . . . . . . . . . . . . . . . . 27
3 Problem 1 Test Results on Low to High Noise Values . . . . . 28
4 Problem 1 Test Results on Extremely High Noise Values . . . 32
5 Problem 2 Test Results for 5 Moving Cities . . . . . . . . . . . 37
2
6 Problem 2 Test Results for 15 Moving Cities . . . . . . . . . . 37
7 Problem 2 Test Results for 25 Moving Cities . . . . . . . . . . 38
8 Problem 2 Test Results for 35 Moving Cities . . . . . . . . . . 38
9 Problem 2 Test Results for 45 Moving Cities . . . . . . . . . . 39
10 Problem 2 Overall Test Results . . . . . . . . . . . . . . . . . 39
List of Figures
1 6 City Travelling Salesman Problem Example . . . . . . . . . 11
2 6 City TSP with 3 Moving Cities at Time t . . . . . . . . . . 22
3 6 City TSP with 3 Moving Cities at Time t + n . . . . . . . . 23
4 Best Tour Lengths Found at Different Noise Levels . . . . . . 29
5 Average Tour Lengths Found at Different Noise Levels . . . . 30
6 Standard Deviation of Tour Lengths Found at Different Noise
Levels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
7 Best Tour Lengths Found at Extremely High Noise Levels . . 33
8 Average Tour Lengths Found at Extremely High Noise Levels 34
9 Standard Deviation of Tour Lengths Found at Extremely High
Noise Levels . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
10 Average Difference of Solution Lengths with Different Num-
bers of Moving Cities . . . . . . . . . . . . . . . . . . . . . . . 40
11 Standard Deviation of Solution Lengths with Different Num-
bers of Moving Cities . . . . . . . . . . . . . . . . . . . . . . . 41
3
12 Solution Quality with Different Numbers of Moving Cities . . 42
4
1 Abstract
This paper will discuss the ability that the Ant Colony Optimization heuris-
tic has to adapt to different environmental disturbances. Ant Colony Opti-
mization is a popular swarm intelligence heuristic which maps problems onto
graphs where an optimal solution of the problem is represented as a short
Hamiltonian path through said graph. Two separate problems will be looked
at. The first problem deals with a noisy environment where the length of
the edges of the graph are subjected to normally distributed noise every time
they are polled by the algorithm. The second problem deals with dynamic
environments where a random subset of the vertices of the graph are con-
stantly moving. With a thorough examination of these problems and the
measures taken to counteract them, more will be known about the overall
abilities of the ACO heuristic. The changes to the original algorithm were
done in a very efficient manner, which allows the heuristic to overcome these
disturbances without significantly increasing the running time.
The paper will begin with an introduction to the field of swarm intelli-
gence, Ant Colony Optimization and the the Travelling Salesman Problem. A
detailed look at the original Ant Colony Optimization is then examined. The
two problems are then looked at in detail and general algorithm modifica-
tions are discussed. In depth testing results are then given for each problem,
followed by a discussion of their implications. The paper concludes with a
5
look at how these ideas can be extended for further study.
6
2 Acknowledgements
I would like to thanks Dr. White for supervising this project and his continual
guidance and suggestions. I would also like to thank my friends and family
for their support.
7
3 Introduction
3.1 Inspiration
Swarm intelligence has been a very popular and important research area
in computer science in the last 15 years. These artificial intelligence tech-
niques are based on using collective behaviour in a decentralized manner to
accomplish a desired task. These systems are typically made up of simple
agents who can communicate with other agents either directly or indirectly
through modifications of their environment. These agents then act on this
information which helps the task at hand get accomplished. This method of
indirect communication to guide a group of simple agents is called stigmergy
and is vital to these heuristics. Many swarm intelligence ideas were inspired
from nature, be it the way birds search for their prey, or the way that ants
manoeuvre themselves in an organized manner. This paper will be looking
at a swarm intelligence heuristic known as Ant Colony Optimization. This
heuristic was inspired by the way in which real ants actually organize them-
selves to find short paths between destinations. The way it works is quite
simple, but it has impressive results. At first the ants seem to travel in a
random fashion, not really paying attention to their surroundings or fellow
ants. As these ants move around, they release a chemical called pheromone,
which other ants are attracted to. Over time, popular paths are carved out
by many ants travelling through the same areas, and other, worse, paths are
8
forgotten as the older pheromone evaporates. The notion of indirect com-
munication between ants, through the use of pheromone, allows them to find
short paths between their initial and destination points quite quickly. This
behaviour is the basis for the Ant Colony Optimization heuristic.
3.2 NP-Hard Problems
Swarm Intelligence heuristics, in particular ACO, are a popular choice when
dealing with NP-Hard problems. When dealing with these kind of problems,
one generally does not expect to achieve the optimal solution, rather reach
a solution that meets certain requirements. Swarm intelligence heuristics
generally offer a good compromise between computation time and solution
quality. These heuristics can give a very good solution in a fraction of the
time of solving a NP-Hard problem completely. Specifically, the Ant Colony
Optimization heuristic has been used to find solutions for popular NP-Hard
problems like the set-packing problem [6], wireless network routing [9], com-
binatorial optimization [7], assembly line balancing [8] and many, many more.
3.3 Travelling Salesman problem
The Travelling Salesman problem is a very popular problem in computer
science. Essentially this problem consists of a salesman who has to travel
to a number of different cities and is interested in finding the shortest route
between all of these cities. Since this problem is in the class of NP-Hard prob-
9
lems, there is no efficient, polynomial time algorithm to solve it. There are
numerous ways to find acceptable solutions to this, one being Ant Colony Op-
timization. This is the problem which this paper will focus on. The travelling
salesman problem is a convenient problem to use Ant Colony Optimization
on because it is intuitively easy to understand and also easy to implement.
Since the ACO heuristic solves problems by finding short paths through a
corresponding graph, when dealing with the travelling salesman problem the
cities will just become nodes in a graph, with their edges representing the dis-
tances between cities. This paper will look at the undirected version of this
problem where the distance between cities A and B is equal to the distance
between cities B and A. Below is an example of the Travelling Salesman
Problem with 6 cities. All possible paths are shown, with the optimal path
highlighted.
10
Figure 1: 6 City Travelling Salesman Problem Example
3.4 Extensions to TSP
It is already widely known that the ACO heuristic works very well at find-
ing very short, if not the shortest, paths in many instances of the travelling
salesman problem. This paper will deal with two extensions on this problem.
The first instance to be looked at is when we are dealing with the travel-
ling salesman problem in a noisy environment. This means that whenever
an ant queries the environment as to how far city x is away from the city
it is currently at, it will receive a noisy value. The noisy response will be
11
normally distributed around the actual value, with the standard deviation
being a runtime parameter. The second problem will focus on an instance of
the TSP where a random subset of cities move in a constant, random direc-
tion. The number of cities to be moved will be a runtime parameter. Both
of these problems will be discussed as far as implementation and original
algorithm modifications in further sections. The aim of the modifications
are to let the algorithm adapt to the new, noisy or dynamic environments
without them being expensive in terms of computation time or memory use.
These extensions, although maybe not both relevant to the travelling sales-
man problem, both occur in the real world and can effect potential problems
that the ACO heuristic could be used to solve. These tests will be either
expanding the bounds of what the ACO heuristic can solve or will show the
heuristic’s weakness in certain inconsistent environments. Inconsistent envi-
ronments are a part of life, and as such we should know how this well used
heuristic will perform in these types of situations.
12
4 ACO Heuristic
This paper will be using the Ant Colony System implementation of the ACO
heuristic [3]. This implementation is similar to the original Ant System
implementation [3] with a few improvements. This code was built off of a
basic Ant Colony System obtained through private communication with Dr.
Tony White.
4.1 Ant System
The AS implementation worked is the following way. First, the problem
must be transformed into a graph problem where an optimal solution can be
mapped onto a short path over the graph which visits each node exactly once
and ends up where it began (a Hamiltonian cycle). The Ant System then
builds its solutions gradually in the following manner. During each iteration,
each ant in the colony builds a tour throughout the graph, visiting each node
exactly once. For each ant, the transition from city a to b depends on the
following things:
1. If city b has already been visited. Each ant has a memory of which
cities it has already visited on this tour and makes sure not to revisit
previously visited cities.
2. The visibility of city b. This is calculated by taking the inverse of the
distance between a and b.
13
3. The amount of pheromone on the edge between a and b. The pheromone
trails are updated on each tour by each ant and represent how desir-
able a path is. This is where the indirect communication is used to
guide ants along previously known, good solutions through the use of
pheromone.
The way in which each city is chosen and how the pheromone levels are
update will be discussed in the next section, as they have been altered.
4.2 Ant Colony System
The Ant Colony System generally works in the same manner as the Ant
System, with a few minor alterations. These new special features will be
discussed in the following subsections.
4.2.1 Transition Rule
The transition rule is defined as follows. An ant k on city i chooses the city
j to move to following the rule:
j =
argmaxuεJk
i{[τiu(t)] · [ηiu]
β} if q ≤ q0;
J if q > q0,
where q is a random number uniformly distributed over [0,1], q0 is a tunable
runtime parameter where 0 ≤ q0 ≤ 1, τiu(t) is the amount of pheromone on
edge iu on iteration t, ηiu is the visibility of u from i, β is a tunable runtime
14
parameter, and JεJki is a city randomly selected, according to the following
formula:
pki =
[τiJ(t)] · [ηiJ ]β∑lεJk
i[τil(t)] · [ηil]β
4.2.2 Pheromone Trail Update Rule
The global pheromone updating formula only applies to the best tour found.
It is applied, on iteration t in the following manner:
τij(t)← (1− p) · τij(t) + p ·∆τij(t)
Where each ij is an edge of the best tour found thus far, p is the tunable
runtime parameter that governs the decay of the pheromone, and
∆τij(t) = 1/L,
where L is the length of the best tour found thus far.
4.2.3 Local Pheromone Updates
The local pheromone updates take place as each ant is performing a tour.
When an ant travels over edge ij the following rule is applied:
τij(t)← (1− p) · τij(t) + p · τ0,
15
where τ0 is the initial amount of pheromone on each edge prior to the heuristic
running. This has the effect of decreasing the pheromone on all travelled
paths, which encourages exploration of other paths, which in turn could lead
to better solutions.
4.2.4 Pheromone Decay
Pheromone decay happens after every iteration. It is done in the following
manner.
τij(t) = (1− p) · τij(t) + p · τ0
4.2.5 Candidate Lists
The final new aspect of the ACS is the use of candidate lists. These lists
help the heuristic run faster and allow it to scale to larger problems easily.
A candidate list is a list of cities who’s visibility are the highest out of all
other cities. When performing a tour, an ant will first exhaust the candidate
list of a specific city before moving on to other cities. This means we no
longer have to keep track of the distances between all cities, only between
cities which exist in each other’s candidate list.
16
4.2.6 ASC Pseudocode
1 for every edge (i,j)2 do3 τij(0) = τ04 CalculateDistance(i, j)5 end for6 for k = 1 to numAnts7 do8 Place ant k on a randomly chosen city9 end for
10 Let T+ be the shortest tour found and L+ be its length11 /* Main Loop */12 for t = 1 to numIterations13 do14 for k = 1 to numAnts15 do16 build Tour T k by applying numCities− 1 times
the following steps:17 if there exists at least one city j ε candidate list18 then19 choose the next city j, jεJk
i from thecities in the candidate list according to the transition rule
20 else21 choose the closest jεJk
i
22 end if23 After each transition any k applies the local pheromone update24 end for25 for k = 1 to numAnts26 do27 Compute Lk(t) of T k(t) produced by ant k28 end for29 if an improved tour is found30 then update T+ and L+
31 end if32 for every edge (i, j)εT+
33 do34 update pheromone trails35 end if36 for every edge (i, j)37 do38 decay pheromone trails39 end if40 end for41 output T+
17
5 Problem Descriptions and Algorithm
Modifications
As mentioned in previous sections, the problem we will look at in this paper
will be the travelling salesman problem. The travelling salesman problem is
a perfect problem to test new improvements of the ACO heuristic on because
the problem or any extensions to the problem can be easily understood in
terms of implementation, goals and results. In the following two subsections
the specific problems that this paper will deal with will be explained.
5.1 Problem 1: Noisy Environment
This problem deals with a situation where the environment in which the ants
live in is noisy. It is important to understand that in the real world we do not
always encounter static environments where no noise exists whatsoever, and
it is important to be able to deal with these situations when we encounter
them. In this case, every time an ant polls the system for information about
how far a certain city is from the city they are currently at they receive a
noisy response. The noise is normally distributed about the actual distance,
making it more realistic situation then a simple uniform distribution. The
standard deviation of the noise is a tunable runtime parameter. This allowed
the testing of low noise levels (a small standard deviation) or very high
levels of noise (large standard deviation). The following two subsections will
18
describe how the noise was added to the environment and then how this noise
was adapted to to allow for the heuristic to continue functioning.
5.1.1 Adding Noise
The method for inserting noise into this algorithm was done in the follow-
ing manner. It changed the original function used to calculate the distance
between two cities, which is used in the original ACS algorithm on line 4 of
the pseudocode in section 4.1.6, by taking a randomly generated Gaussian
number (which is a normally distributed random number with mean 0 and
standard deviation 1) multiplying it by the distribution defined by the user,
then adding this to the actual distance which results in a randomly gener-
ated, normally distributed number about the real distance value. The pseudo
code for the original method and the altered method is shown below.
Original code:
Distance(City1, City2)
return√
(City1.x− City2.x)2 + (City1.y − City2.y)2)
Modified code:
Distance(City1, City2)
value←√
(City1.x− City2.x)2 + (City1.y − City2.y)2)
value← value + RandomGaussian() ∗NOISE SD
return value
19
Where NOISE SD is the tunable runtime argument for the standard devi-
ation of the noise.
5.1.2 Adapting to Noise
The original heuristic only needed to calculate the distance between each
city once at the beginning of the computation and these values were stored
in an array to be accessed later, which is done on line 4 of the pseudocode.
Since these initial values can no longer be trusted to accurately describe the
distance between two cities, we must rebuild the distance array after each
iteration, and instead of just putting a the noisy value into the distance array,
we successively average these numbers to hopefully close in on the correct
value. This is done in the following way:
Array DistanceArray
Array RunningSumArray
UpdateDistances(iteration)
for i← 1 to numCities
for j ← 1 to numCities
RunningSumArray[i][j]← RunningSumArray[i][j] + Distance(i, j)
DistanceArray[i][j]← RunningSumArray[i][j]/iteration
20
Where iteration is the current tour number that the heuristic has made.
This code would be added to the original ACS code on line 40 to ensure these
values are updated once every iteration.
This algorithm basically builds a better idea of all of the distances between
each city by averaging the noisy values received when polling the system over
time, meaning after a few hundred iterations it should have a very close es-
timation of the distance. This means that before the algorithm really has a
chance to begin to solve the TSP problem, it should already have a pretty
good estimation of each of the distances.
This modification is also relatively light weight. It only requires an ad-
ditional array to store the running sums of sample distances, which is O(n)
and computationally this modification requires the recalculating of all of the
distances after each iteration, which runs in O(n2), which is no more then
each iteration originally took. These two facts make this modification an easy
to implement and inexpensive in terms or memory space and computation
time.
21
5.2 Problem 2: Dynamic Environment
In this problem we are dealing with an instance of the TSP where a randomly
chosen subset of cities are constantly moving in a randomly chosen direction.
Below is an illustrative example of a 6 city problem at two different times,
showing the movement on the cities.
Figure 2: 6 City TSP with 3 Moving Cities at Time t
22
Figure 3: 6 City TSP with 3 Moving Cities at Time t + n
To deal with the dynamic nature of this problem pheromone smoothing
was employed to act as a counter measure and to keep the heuristic func-
tioning. The pheromone smoothing simply changes the ratio of the amount
of pheromone on each edge adjacent to a specific city, while preserving the
previous relative ordering. This type of approached has been used before in
other dynamic ACO environments. One instance was on a problem where
edges suddenly dropped off of the graph [2] and another was where ”traffic
jams” occurred, causing path lengths to increase and decrease in cost over
time [5]. The dynamic nature of the environments in these two problems were
23
overcome relatively well by using simple pheromone smoothing. The method
used here will be called local smoothing, where only the edges connected to
the moving cities are changed. This should allow paths previously deemed
sub-optimal to be used again, and insures that it is possible to deviate from
paths that are currently very good in case they become sub-optimal. The
modifications done to the original ACS heuristic are described below.
5.2.1 Pheromone Lower Bound
The first modification of the ACS heuristic was the use of a lower bound
on the amount of pheromone allowed to exist on an edge. This was done
so that no edges are essentially ”lost” and never used again due to a lack
of pheromone. This was needed because the fundamental structure of the
underlying graph is always changing where good solutions can become poor
quite quickly and vice-versa. The pheromone update rule, which is used in
multiple places in the heuristic, was changed to the following:
τij(t) =
(1− p) · τij(t) + p · τ0 if ((1− p) · τij(t) + p · τ0) > τ0;
τ0 otherwise,
5.2.2 Local Pheromone Smoothing
A smoothing technique was used to change the amount of pheromone levels on
specific edges, while leaving them in the same order relative to their neighbour
edges (ie. in the case of local smoothing about city i, if τij > τik was true
24
prior to smoothing, it will remain true after smoothing.) This technique
just ensures that certain paths do not become exceedingly popular and other
paths do not become forgotten. This is needed because the the shortest
path throughout the graph can change fundamentally at any moment. The
formula [5] to smooth an edge ij is given below.
τij ← τ0 · (1 + log(τij/τ0))
Applying this formula will only cause pheromone levels close to τ0 to move
slightly closer towards τ0 and larger pheromone levels to move more close to
τ0. Local smoothing (the applying of that formula to edges leaving a particu-
lar city) will be applied to the moving cities only. This does not need to hap-
pen every iteration, as the tour lengths do not increase dramatically from one
iteration to the next. The moving cities will have local pheromone smooth-
ing applied to them after every 10 iterations to ensure that the pheromone
levels stay in check, and is inserted at the bottom of the main loop in the
pseudocode in section 4.2.6, on line 40.
25
6 Test Results
The following values were used for the tunable parameters of the ACS imple-
mentation of the ACO heuristic. They were found to yield very good results
in static ACS trials.
Variable Valueβ 8
p (evaporation rate) 0.2q0 0.9
cl (candidate list size) 5τ0 10−7
number of ants 25number of iterations 2000
Table 1: Paramaters Used in Tests
Both problems were run on the following 51-city problem [1]. The optimal
tour has a length of 426.
26
City# Location City# Location City# Location City# Location1 37,52 14 12,42 27 30,48 40 5,62 49,49 15 36,16 28 43,67 41 10,173 52,64 16 52,41 29 58,48 42 21,104 20,26 17 27,23 30 58,27 43 5,645 40,30 18 17,33 31 37,69 44 30,156 21,47 19 13,13 32 38,46 45 39,107 17,63 20 57,58 33 46,10 46 32,398 31,62 21 62,42 34 61,33 47 25,329 52,33 22 42,57 35 62,63 48 25,5510 51,21 23 16,57 36 63,69 49 48,2811 42,41 24 8,52 37 32,22 50 56,3712 31,32 25 7,38 38 45,35 51 30,4013 5 25 26 27,68 39 59,15
Table 2: 51 City Test Problem Used
6.1 Problem 1
To test how well the noise filtering modifications worked it was tested with
a wide range of amounts of noise. The standard deviation of the noise is
the tunable parameter in this case. Increasing this parameter would cause a
wider range a values further away from the actual values to be given from the
algorithm, which can be seen as a ”noisier” environment then with a lower
noise standard deviation. Of course an environment with a noise standard
deviation of zero would an environment free of noise. The following table
describes tests run on various environments spanning a wide range of noise
amounts. Each trial consists of 25 distinct tests run at a specific noise level.
The best tour and average tour length, along with the standard deviation of
27
each trial are given.
Noise Best Tour Found Average Tour Found Standard Deviation of Tours0.0 430 448.88 6.59893.0 440 448.2 4.25216.0 436 448.04 5.42949.0 437 445.68 5.489712.0 436 445.8 4.939615.0 427 443.76 6.784018.0 439 447.28 3.954921.0 429 443.92 6.770124.0 431 444.32 5.815227.0 432 445.28 6.684430.0 435 447.36 6.565933.0 430 446.44 7.787636.0 430 445.68 8.365339.0 436 451.52 7.705242.0 438 448.16 6.460245.0 430 451.32 10.3989
Table 3: Problem 1 Test Results on Low to High Noise Values
Below the results of best tour lengths, average tour lengths and the stan-
dard deviations are given in a graph to illustrate the effect that the amount
of noise has.
28
Figure 4: Best Tour Lengths Found at Different Noise Levels
29
Figure 5: Average Tour Lengths Found at Different Noise Levels
30
Figure 6: Standard Deviation of Tour Lengths Found at Different NoiseLevels
Although an environment with a noise standard deviation of 45 could be
seen as very noisy, further tests were done to examine how well the noise
cancelling modifications adapted to extremely noisy environments. The data
about these tests are given in the following table.
31
Noise Best Tour Found Average Tour Found Standard Deviation50.0 437 452.04 7.8713100.0 442 462.64 11.8216150.0 453 480.4 11.9833200.0 479 503.88 13.2524250.0 514 532.68 12.7427300.0 531 561.0 18.1769
Table 4: Problem 1 Test Results on Extremely High Noise Values
Below the results of best tour lengths, average tour lengths and the stan-
dard deviations in these extremely noisy environments are illustrated in the
following three graphs.
32
Figure 7: Best Tour Lengths Found at Extremely High Noise Levels
33
Figure 8: Average Tour Lengths Found at Extremely High Noise Levels
34
Figure 9: Standard Deviation of Tour Lengths Found at Extremely HighNoise Levels
35
6.2 Problem 2
This problem is significantly harder to test due to the fact that nothing will
be known about the state of the underlying graph after any given test. This
is due to the fact that for each test a randomly selected subset of cities are
chosen to move, and the direction they move in is also randomly chosen. The
direction can be any multiple of 45◦. The only controllable parameter was the
number of cities to move. To test how well the modified algorithms worked,
it was run for 2000 iterations, and then the underlying graph was exported
and a new test with the original ACS heuristic was done. This would give us
a basis for comparison. Below are the results for 5 tests, of 10 trials each, all
with different amounts of moving cities. Also given is the difference between
the path lengths given as a percentage of how much better of a solution the
standard ACS gave on the same graph as the dynamic ACS ended on.
36
ACO on Dynamic Environment ACO on Static Environment % Difference1893 1878 0.82410 2396 0.61950 1935 0.81906 1893 0.71699 1685 0.82063 2059 0.22223 2215 0.41416 1416 0.01836 1820 0.91871 1862 0.5
Table 5: Problem 2 Test Results for 5 Moving Cities
ACO on Dynamic Environment ACO on Static Environment % Difference3423 3395 0.82790 2773 0.62520 2512 0.32489 2482 0.32494 2485 0.42418 2347 2.92450 2432 0.73042 3013 1.03206 3201 0.22493 2477 0.6
Table 6: Problem 2 Test Results for 15 Moving Cities
37
ACO on Dynamic Environment ACO on Static Environment % Difference3833 3830 0.12598 2596 0.13039 3001 1.23544 3511 0.94046 4025 0.52974 2941 1.13517 3479 1.03342 3332 0.23872 3824 1.23672 3646 0.7
Table 7: Problem 2 Test Results for 25 Moving Cities
ACO on Dynamic Environment ACO on Static Environment % Difference4504 4489 0.35220 5204 0.33930 3920 0.33962 3950 0.33423 3397 0.84243 4215 0.74763 4704 1.24807 3739 1.43365 3360 0.13656 3643 0.4
Table 8: Problem 2 Test Results for 35 Moving Cities
38
ACO on Dynamic Environment ACO on Static Environment % Difference4542 4537 0.14649 4559 1.94736 4387 7.45724 5245 8.44268 4216 1.25031 4928 2.04487 4449 0.83968 3957 0.34474 4432 0.95732 5706 0.4
Table 9: Problem 2 Test Results for 45 Moving Cities
Below is a table giving the statistical information for all of the tests done
for problem 2, followed by graphs visually displaying these results. Solution
Quality is defined as 100% - the average difference between the dynamic ACO
solutions and the static ACO solutions.
# of Moving Cities Average Difference Standard Deviation Solution Quality5 0.57 0.29 99.4315 0.78 0.79 99.2225 0.7 0.45 99.335 0.58 0,43 99.4245 2.34 3.01 97.66
Table 10: Problem 2 Overall Test Results
39
Figure 10: Average Difference of Solution Lengths with Different Numbersof Moving Cities
40
Figure 11: Standard Deviation of Solution Lengths with Different Numbersof Moving Cities
41
Figure 12: Solution Quality with Different Numbers of Moving Cities
42
7 Discussion
7.1 Problem 1
As we can see the noise cancellation worked very well across most levels of
noise and we saw a slight increase of path length when dealing with extreme
levels of noise. This can be attributed to the fact that the modifications,
through averaging, can reach a point where the estimation of the distances
between cities is very close to the actual value. Even after as little as 25 it-
erations, the heuristic will already have a pretty good idea about the actual
distances between the cities, and this estimation will only get better over
time. This approach takes advantage of the fact that the nature of the ACO
heuristic is to build solutions over many iterations through the graph, and it
can estimate the distance between the cities well enough before the heuris-
tic begins to converge upon a solution. This implies that the pheromone
updates during the early stages of the solution building process are either
reasonably accurate or that previous updates done on incorrect estimations
can be corrected through the accurate application of pheromone on good
edges and the decay of pheromone on older, suboptimal edges. The algo-
rithm seems to breakdown slightly when running in environments of extreme
noise which lead to a slight drop in solution quality. From this we can see
that the algorithm has a harder time to successfully estimate the distances,
meaning that it must prematurely chooses a suboptimal path based on inac-
43
curate pheromone trails. This type of noise would be very hard to deal with
in any situation without a more intrusive noise cancellation scheme, and con-
sidering how good the solutions the algorithm still gives at this point, we can
conclude that it performs extremely well in a wide range of noise conditions.
There are a few other benefits of implementing the noise cancellation
in this manner. First of all, it is efficient in both computation time and
memory space. This makes the running time of both the standard ACS and
the modified ACS almost indistinguishable, which is always a desirable factor.
Secondly, this method of noise cancellation does not need to know how much
noise exists in the environment, or even if it is noisy at all. Since no prior
knowledge of the noise is needed these modifications can be implemented
in instances where noise could exist or where it is known to exist but the
intensity of the noise is not known.
7.2 Problem 2
When it came to testing this problem we could not rely on any previous
benchmarks obtained, the only way to evaluate how well the dynamic ACS
worked was to run it for a given number of iterations, in this case 2000, letting
the underlying map randomly grow. This map was then exported and the
static ACS heuristic was ran on it. The two solutions were then compared.
As it turns out, the use of pheromone smoothing around the moving cities
44
worked very well and let the heuristic adapt to the dynamic aspects of the
problem. When looking at both the average solution found and the standard
deviation of these tests, it is obvious that the solutions obtained were quite
good. In all cases very reasonable solutions were reached, especially when
there were less then 35 moving cities where the static ACS heuristic reported
solutions less then 1% better then the dynamic ACS. When more and more
cities are moving the heuristic has a harder time, but still gives very reason-
able solutions with the worst solution found was 8.7% worse then the static
ACS. Its very important to realize that the dynamic ACS not only gave good
results, but also tracked them as time went on meaning that it could always
provide a very good path at any time, while running the static ACS on a
given map at a given time will not give very relevant information about the
map when the computation is completed due to the fact the environment is
always evolving.
This heuristic performed very well for one simple reason, it allowed paths
that were previously travelled and found not to be too desirable to be revis-
ited in hopes of finding a new, better solution in the ever-changing environ-
ment. When dealing with a smaller amount of moving cities, the algorithm
correctly established the link between the moving cities and the static cities,
allowing the pheromne levels on the dynamic edges to change as their quality
in the final solution changed. Even when almost all the cities were moving,
45
the algorithm still produced very respectable results. The use of a lower
bound on the pheromone levels was also very important. Simply making the
lowest possible pheromone level on any given edge be the initial pheromone
level, which in this case was 10−7, made it impossible for any path to be com-
pletely forgotten and the pheromone smoothing ensured that paths would not
become too popular. The combination of these two rather simple modifica-
tions allowed the dynamic ACS to perform so well.
The new modifications to the dynamic ACS were also very efficient. The
modifications only moved the cities and updating the distances, and updating
the pheromone levels. Both of these operations run in O(n · c), where c is the
number of moving cities. The only added memory needs was keeping track
of the moving cities, which was only O(c). Both the running time and space
complexity of the modifications are practically insignificant with respect the
the overall running time and space complexity of the ACO heuristic in gen-
eral.
46
8 Looking Forward
Since the field of swarm intelligence has been evolving so quickly, it is always
a good idea to think of the future. Both of these modifications seem to work
very well, but there could be room for a slight improvement. This could be
obtained through the fine tuning of all of the runtime parameters. Some of
these parameters, such as pheromone evaporation rate or initial pheromone
levels, could potentially have a positive effect in either of the two problems.
Noisy environments are quite common in the real world, and as such they
effect a lot of different problems. Although the first problem seems to be
pretty self contained, these noise cancellation techniques could be tested in
conjunction with other environmental disturbances, like the dynamic envi-
ronments introduced in the second problem for instance, to try to realize
more of the ACO heuristic’s potential.
An interesting extension to the second problem would be to contain all
cities within a certain area, and if a city hit the boundary it would be redi-
rected back to the center of the map. This would take away the possibility
of ever reaching a stable state because, after a while, the current problem
reaches a state where the tour lengths keep increasing but the structure of
these tours remain constant. Another interesting extension would be to in-
vestigate different wandering paths for the cities other then just straight lines,
47
or the ability for the cities to change speed and direction.
All of these extension to the base problems discussed in this paper would
be very relevant by testing the boundaries of that the ACO heuristic can
handle.
48
References
[1] Tsplib. http://www.iwr.uni-heidelberg.de/groups/comopt/software/TSPLIB95/.
[2] Daniel Angus and Tim Hendtlass. Ant colony optimisation applied to a
dynamically changing problem. In Developments in Applied Artificial In-
telligence: Proceedings of the 15th international conference on Industrial
and engineering applications of artificial intelligence and expert systems,
IEA/AIE 2002, pages 618–627, London, UK, 2002. Springer-Verlag.
[3] Eric Bonabeau, Marco Dorigo, and Guy Theraulaz. Swarm Intelligence:
From Natural to Artificial Systems. Oxford, New York, USA, 1999.
[4] Marco Dorigo and Thoms Stutzle. Ant Colony Optimization. MIT Press,
Cambridge, Massachusetts, USA, 2004.
[5] Casper Joost Eyckelhof and Marko Snoek. Ant systems for a dynamic
tsp. In Ant Algorithms: Third International Workshop, ANTS 2002,
pages 88–99, London, UK, 2002. Springer-Verlag.
[6] Xavier Gandibleux, Xavier Delorme, and Vincent T’Kindt. An ant colony
optimisation algorithm for the set packing problem. In Ant Colony Opti-
mization and Swarm Intelligence: Fourth International Workshop, ANTS
2004, pages 49–60, Berlin, Heidelberg, Germany, 2004. Springer-Verlag.
[7] Walter J. Gutjahr. S-aco: An ant-based approach to combinatorial op-
timization under uncertainty. In Ant Colony Optimization and Swarm
49
Intelligence: Fourth International Workshop, ANTS 2004, pages 238–
249, Berlin, Heidelberg, Germany, 2004. Springer-Verlag.
[8] Joaquin and Jordi Pereira. Ant algorithms for assembly line balancing.
In Ant Algorithms: Third International Workshop, ANTS 2002, pages
65–75, London, UK, 2002. Springer-Verlag.
[9] Ying Zhang, Lukas D. Kuhn, and Markus P.J. Fromherz. Improvements
on ant routing for sensor networks. In Ant Colony Optimization and
Swarm Intelligence: Fourth International Workshop, ANTS 2004, pages
154–165, Berlin, Heidelberg, Germany, 2004. Springer-Verlag.
50