Natural Computation CO637

Swarm Intelligence III:Ant Colony Optimisation applications

Comparing PSO and ACO

Alex Freitas

Recap - basic ideas of Ant Colony Optimisation (ACO) algorithms

• Each path followed by an ant represents a candidate solution to the target problem– An ant constructs its path (candidate solution) incrementally,

typically by adding one component at a time to the solution

• The amount of pheromone that an ant deposits on its path is proportional to the quality of the corresponding candidate solution

• When an ant has to choose between alternative paths, the larger the amount of pheromone on a path, the larger the probability of the ant choosing that path

Recap - main elements in the design of an ACO algorithm

• An appropriate representation of the problem– components that an ant will use to incrementally construct a solution

• A problem-dependent heuristic function () that measures the quality of each component that can be added to a solution

• A rule for updating the amount of pheromone () associated with each component in a path followed by an ant– Pheromone increases in proportion to quality of the path (solution)

• A probabilistic transition rule: the probability of choosing a component i is proportional to the product i i 1 1

2 2

1

2

?

component 1

component 2

Applying Ant Colony Optimisation (ACO)to Data Network Routing: AntNet

• In a data network each data packet can follow a different route– E.g., the Internet

• Problem: to find the fastest (minimum cost) path between two nodes, using routing tables at each node of the network

• This is a difficult problem because:– Traffic load varies (potentially a lot) with time, in unpredictable ways

– Network topology also varies with time (e.g. links break down)

– The problem involves distributed processing – no central coordinator

• All these characteristics suggest that ant colony algorithms are a suitable type of algorithm for this problem

AntNet’s main data structure

A pheromone matrix T associated with each node i of the network

Each element ijd of T indicates the desirability for an ant in node i and with destination node d to move to node j

i,1,1 . . . i,1,n

i,N(i),1 . . . i,N(i),n

destination nodes

. . . . . . . . .

Neighbour nodes

Structure of transition matrix at node i:

N(i) is the number of neighbours of node i

n is the total number of nodes in the network

The AntNet Algorithm for Data Network Routing (1)

• At regular intervals, artificial ants are launched from network nodes towards destination nodes

• Two kinds of ants, forward and backward ants:

• Forward ant searches for fastest path from its source to its destination node– A forward ant shares the same queues as data packets, collecting

information about the time taken to travel to each node visited in its path– At each node i, each ant having destination d selects the node j to move to

with probability Pijd proportional to: ijd + ij , where: ijd = pheromone (capture current and past status of the network) ij = instantaneous state of the queue in the link from node i to node j N(i)

ij = 1 – qij / ( qik ) , qij = number of bits waiting to be sent from i to j k=1

= parameter specifying the weight of heuristic w.r.t. pheromone

The AntNet Algorithm for Data Network Routing (2)

• Once it reaches its destination, a forward ant is transformed into a backward ant, which goes back to the source along the same path (but in the opposite direction)– When moving backwards, the information stored in the ant (collected in the

forward phase) is used to update the pheromone matrix, as a function of the goodness of the path that they followed in the forward phase

– A backward ant does not share the same link queues as data packets; it uses higher-priority queues, to speed up the updating of pheromone matrix

1 2 3

4 5 6

– At each node i in the forward ant’s path before d, pheromone ijd (suggesting to choose neighbour j when destination is d) is incremented by an amount inversely proportional to the forward ant’s trip time from i to d

E.g., forward ant 16, backward ant 61

The AntNet Algorithm for Data Network Routing (3)

• AntNet has been extensively evaluated in experiments/ simulations with models of real networks, e.g.:– National Science Foundation network (USA), NSFnet

– Japanese NTT company backbone, NTTnet

• AntNet was compared with the best-known routing algorithms

• Overall the performance of AntNet was very competitive with the performance of those routing algorithms

• See sections 6.3 and 6.4 of the ACO book by Dorigo & Stutzle for details

An application of ACO in automated customer support (1)

• An approach to provide customer support on the web consists of a set of FAQs (Frequently Asked Questions), with the corresponding answer

• Manual maintenance of FAQs is expensive and slow in many large-scale applications

• Alternative: providing a dynamic database of FAQs, using some intelligent method to automatically update the FAQs and their relationships

• We will study a system – called RightNow Web– that does that in a way loosely inspired by ant colony principles

An Application of ACO in automated customer support (2)

• Human visitors to the site are the ant colony members

• Assumptions about the behaviour of human visitors:– Each user is accessing the web site to find an answer to a specific

question in a FAQ – each FAQ will also be referred to as an Answer

– Each user follows a directed (rather than random) approach to find Answers: i.e. each user chooses at each step the FAQ/Answer which seems to be the best Answer to her/his question

• Basic idea: to increment the usefulness score for an Answer visited by a user, loosely analogous to dropping more pheromone in a part of a path in ant colony

An Application of ACO in automated customer support (3)

• Knowledge base initially contains a set of FAQs, or Answers, for the most predictable questions of users

• Users view a table of Answers, for instance:• Subject Score requirements for retirement 1,801

calculating your pension 1,769 pension for your dependents 1,752

. . .

• List of Answers is kept in decreasing order of score, and it is dynamically updated– The FAQs (Answers) are continuously changing– Users’ interest change with time

An Application of ACO in automated customer support (4)

• “Score” is proportional to the usefulness of the Answer to the user and to the the number of user queries solved by an Answer

• Explicit and Implicit methods to compute an answer’s score:

• Explicit: users rates the usefulness of an answer, after reading it– More reliable than implicit rating, but…

– Less than 10% of users bother to rate answers

• Implicit: based on users’ navigation through the “network” of answers – the more users read an Answer, the more its score is increased– Less reliable than explicit rating, has a smaller weight, but…

– Guaranteed participation of every user, without bothering the user

An Application of ACO in automated customer support (5)

• Clicking on an Answer, the user can move to a related answer:

Subject: calculating your pensionAnswer: for each year of service you get 1% of your final salary as your pension, up to the limit of 40% of your final salaryRelated Answers: pension for your dependents

requirements for retirement

• As users click on related Answers, they navigate through the “network” of FAQs (Answers) – users act as “ants” in an ACO

A1 A4

A3

A2

Human users are “foraging” for best answer

Move from answer A1 to answer A2:increasing “pheromone” (score) in the link

An Application of ACO in automated customer support (6)

• According to a paper describing the RightNow Web system:

– its users experienced between 10% and 99.5% reduction in customer support load

– In some notable cases customers reported an estimated savings of more than $1.2 million per month due to reduced phone support volume

Summary about ACO applications

• We studied 3 types of applications of Ant Colony algorithms:

• Travelling Salesman Problem – original and most well-known application of ACO

• Data network routing: ants explore the network to find less overloaded routes; shorter routes more pheromone

• Automated customer support: human visitors to a website are considered the “ants”; more visited FAQs more pheromone

• In all these applications, the ants act in a completely distributed manner (no central coordinator), and the dynamic nature of the environment is clear in the data network routing and automated customer support applications

Comparing PSO and ACO (1)

• Both are population-based, stochastic, approximate algorithms – their “global search” hopefully finds a near-optimal solution

• ACO uses, in addition to its generic adaptation strategy (increasing pheromone for the best solutions), problem-dependent information as a local heuristic; unlike conventional PSO, which uses only its generic adaptation strategy (moving particles towards its best neighbours’ and personal best positions)

• An ant incrementally constructs a solution; unlike particles in PSO

• Both are very generic heuristic methods, but in practice:– PSO is mainly used in problems with continuous variables, e.g. numerical function

optimization– ACO is mainly used in problems with discrete or nominal variables, e.g.

combinatorial optimization problems

• Main drawback of ACO for coping with continuous variables: the pheromone matrix is a discrete data structure

– Usually requires continuous variables to be discretized

Comparing PSO and ACO (2)

• Conventional PSO’s equations for velocity and position updating are based on computing differences between the numerical coordinates of different particles in the search space

• This approach tends not to be appropriate for some problems (in particular combinatorial optimization ones), where what needs to be optimized is a combination of “nominal” (non-numeric) values– E.g., in the case of the data network routing problem, the goal is to find

the optimal sequence of nodes n1, n2, ….

• A more effective application of PSO to nominal data requires a new approach for updating velocity and position which makes sense for the problem being solved

References

M. Dorigo and T. Stutzle. Ant Colony Optimization. MIT Press, 2004.

N. Monmarche, M. Slimane, G. Venturini. AntClass: discovery of clusters in numeric data by an hybridization of an ant colony with the Kmeans algorithm. Internal Report No. 213, E3i. Laboratoire d’Informatique. Universite de Tours, France. Jan. 1999. (URL: citeseer.ist.psu.edu/559169.html)

D. Warner, J.N. Richter, S.D. Durbin and B. Banerjee. Mining user session data to facilitate user interaction with a customer service knowledge base in RightNow Web. Proc. 7th ACM SIGKDD Int. Conf. on Knowledge Discovery and Data Mining (KDD-2000), pp. 467-472. ACM Press, 2001.