Bioinspired ComputingLecture 2

Collective Behaviour and

Swarm Intelligence

Nearly all slides …

• By Seth Bullock (2005 lectures)

Overview…

This Time:• command & control vs. self-

organisation• the sophisticated abilities of

insect colonies• stigmergy• algorithms inspired by insect

intelligence– Foraging

– Clustering & Sorting

– Building

– Swarming & Flocking

Next Time:• insect-like social robots

– sorting & clustering– cooperative transport

• Applications– sport– education– entertainment

Amazonian Categorisation

• Imagine you are faced with the task of organising a huge number of Amazonian species. There are hundreds of ways in which these species might be divided up: colour, size, etc.

• Your boss tells you that your categorisation will have to match that of his customers over a very long time scale, so it must be flexible, because not only will customers change, but species may also change (colour, size, prevalence, etc.).• One approach to this knowledge management problem is to interview customers, devise a general-purpose, explicit categorisation scheme, pay someone to keep it up to date, and hope customers and species change little and slowly.• amazon.com solve an analogous problem like this:“Customers who bought this book, also bought…”

Collective problem−solving

• "Problem−solving can occur at a level above a collection of idealized agents, without "intentional solving" on the part of the individual.”

– N.L. Johnson, Collective Problem Solving, LANL tech−report, 1998.

• In other words, the individual agents do not know they are solving a problem, but their collective interaction so solves the problem.

• Emergent functionality

Amazonian Self-Organisation

Why is this automatic categorisation approach successful?

• no need to interview customers• no need to discover explicit categories before using them• adapts to changing trends automatically as they happen

In contrast, “command and control” approaches suffer from the problem that explicit, hand-designed categorisations…• may be hard to discover through customer interrogation• will require constant updating and may still be out of date• may sometimes require a radical overhaul

In the next few lectures, we will learn that simple, self-organising systems such as amazon.com’s often enjoy advantages over their command and control cousins…

Advantages

These systems tend to involve many partially independent entities working together to solve a problem without a central executive. Each entity may be unaware of many or all of its colleagues, attending to only its local environment.

Such systems are:• parallel systems of simple

agents• robust to noise & damage• dynamic• flexible• self-organising

• adaptive• possibly complex• hard to build?• hard to understand?• fast, cheap, out of control?• intelligent?

Ants, Termites, Bees, etc.

Some of the most impressive natural self-organising systems are to be found in the world of colonial insects…

…with no central planning and very little communication – complicated, coordinated, goal-directed behaviour often seems to arise spontaneously from the interactions of many simple insects.

Such species…• forage for food, dividing colonial resources effectively • construct complex hives, nests, etc.• efficiently & dynamically divide labour amongst the colony• sort and cluster different objects (eggs, corpses, etc.)• cooperate in moving objects, defeating enemies, etc. that

would be impossible for a single individual to deal with

E.g.

Ant Algorithms and Bee Bots

Recently software engineers and roboticists have begun to exploit our understanding of social insect behaviour to design new kinds of algorithm and new kinds of robot.

• These systems idealise insect behaviour in much the same way that ANNs idealise the behaviour of neurons (topic 3)• Researchers pick and choose aspects of natural systems in the hope that the artificial systems they inspire will share some of their desirable properties.• Of course, ants and bees are not designed to solve the problems of today’s software engineers or roboticists.• A piece of software or a robot will not perform well just because it behaves like an ant colony – the trick is to find aspects of insect behaviour which can be profitably exploited.

Foraging for the Shortest Route

A particularly striking result from ant experimentation concerns the ability of a colony to discover the shortest routes to the resources it requires:

• One pheromone trail is now stronger than the other, directing the ants to the food via the shorter route.

nest

food

As ants forage they deposit a trail of slowly evaporating pheromone.

Those that reach the food first return before the others.

• It is not just ants that need to find optimal pathways.• Traffic on telecommunications systems, the internet, roads, rail, and sea would all benefit from the reduction in congestion that efficient routing algorithms could provide.

Ant-Inspired Routing

Consider an in-car system that suggests the best route to take for any journey from A to B across a road network.

In this way, routes via congested nodes are gradually weakened, prompting ants to take alternative paths.

An “ant” traverses the network following the strongest pheromone trails from A to B.At the end of the journey the ant lays down pheromone along the path taken, leaving less pheromone at nodes that were congested.

Since many ants traverse the network constantly and their pheromone evaporates gradually, the system automatically adapts to the current load on the road network.

A

B

How Good Is It?

• In their book Swarm Intelligence, Eric Bonabeau, Marco Dorigo & Guy Théraulaz claim that “work on ant-based routing is only beginning … but in all tested situations it appears that … ant-based routing with agents patrolling the network outperforms all other routing algorithms”.

• France Télécom and BT are developing algorithms for their systems, but the application of ant-based routing is potentially much wider – e.g., routing internet traffic.

• Like amazon.com, these algorithms rely on constant user traffic to build an up-to-date picture of what is going on (whether it be trends in book shopping, jams on the Otley Road, or congestion at telecom hubs). The power of these algorithms is their simplicity and their ability to direct traffic and build this picture simultaneously.

Sorting, Clustering & BuildingMany species of ant cluster corpses into cemeteries, gradually piling them up together. Brood sorting is also observed, with larger larvae lying further from the brood centre. In addition, some species are able to construct walls, arches and other architectural structures.

These behaviours are yet to be fully understood, but have all been modelled as the result of simple probabilistic rules:

• Clustering relies on two rules concerning the ant’s local environment: (i) items are more likely to be picked up when they

differ from those around them, and (ii) items are more likely to be put down amongst

similar items.• Wall building, etc., is slightly more complex, relying on

chemical templates to direct what are essentially the same basic processes.

Clusterring and sorting

Deneubourg (1998):• Memory•I N items, time T: f = N/T

Agent based systemStarting form a homogeneous distribution

Ants for Catering

Imagine you’re want to seat many guests. It’s best if you group guests that know each other together. But how?

• First draw a graph that represents which of your guests know each other. Then apply an algorithm inspired by ant clustering: scatter the nodes of the graph and a load of ants on a page; let an ant pick up a node, i, if it is surrounded by nodes to which i is not connected; let an ant drop i if it is surrounded by graph neighbours of i; let the ants wander about at random picking up and dropping nodes.

Slowly, clusters of acquainted guests will form on the page.

This graph-partitioning technique has applications in chip design (where connected components must be placed close together on a chip) and load balancing on parallel processor machines.

Partitioning a Graph

Here we see a random graph being partitioned by ants…

After the ant algorithm of Kuntz, Layzell & Snyers (1997) has been at work, a few clear clusters have emerged.Cluster members are more connected to each other than to members of other clusters.

This technique can be used to efficiently load the processors of a parallel machine – minimising the amount of communication required.

Ants for Architecture How can insects in a colony coordinate their behaviour in order to build highly complex architectures? Ants and termites don’t appear to have blueprints in their heads: they seem to follow simple rules in an almost random manner.

If the blueprint isn’t in the insects’ heads, it may be in their environment: ants appear to use their own previous work to stimulate their behaviour – the building of arches, towers, etc., appears to be governed by the structures themselves.

“The worker does not direct his work, but is guided by it”

Stigmergy

Stigmergy is a slippery concept. At its root is the ability of agents to influence each other and their future selves by altering their environment – often seemingly unknowingly.

Some examples:• pedestrians crossing a park make paths in the grass – the

most popular will guide future walkers and be reinforced• amazon.com customers buy books – their purchases change

the descriptions of the books, guiding future customers• cells divide & differentiate during morphogenesis

according to chemical gradients that they themselves influenceIn these examples, the behaviour of individual walkers,

shoppers, cells, etc., is shaped by their environment, which in turn was shaped by their own prior activity.

?

Artificial Architects

Bonabeau and Théraulaz have demonstrated these ideas in action through simple artificial paper wasp architects.

Wasps wander at random over a 3-d grid of cells and follow a simple set of microrules that govern building behaviour.Depending on the contents of the 26 cells that surround the wasp, it can deposit one of two types of brick, or leave the cell alone – thus wasps are reactive agents with no memory.

B&T show that starting from a single brick, swarms of these wasps following simple sets of microrules can construct complex structures that resemble natural paper wasp nests – layered, cellular combs with internal cavities and a surrounding envelope.

Could sets of these microrules be evolved to build human habitation or useful artefacts?

- cf work on urban planning by Georg Vrachliotis, ETH Zurich

Termite Construction Simulation (D Ladley, Leeds)

www.comp.leeds.ac.uk/danl/cconstruct.html

3 kinds of termite:- trail-laying

- Follow/lay pheromone- Nursing

- Move towards/away queen up to fixed distance

- BuildingTo place block:Pheromone level must lie within a

certain bound and one of:1. Material above or below

(allows vertical stacks)2. Site shares face with horizontal

loc containing material and satisfying (1)(allows stack horiz extension)

1. 1 face of the site must neighbour 3 horizontal locations that each contain material(allows gradual construction of

elevated horiz surfaces)

Queen: green, ground red, yellow: deposited building material.

1. formation of "pillar-like" structures:simulates reality

2. construction of walls between the pillars

Building the queen chamber

Swarms for Data Mining

We have already seen how swarms of insects are able to cluster similar objects together. Software engineers have exploited similar ideas to cluster database records, discovering trends in, say, financial information (which customers are likely to default on their loan), or health data (which patients are most likely to develop heart disease).

James MacGill has developed an approach to spatial data mining inspired by insect swarming.Consider a map of the UK with every case of CJD marked.Eyeballing the map will reveal the existence of clusters.But most of these are probably just over the major UK cities, that’s where most of the people live, after all…We need to spot anomalous clusters – where there are more cases than you would expect

Swarms for Data Mining

Imagine a flock of agents flying across the CJD data set…

Each agent is aware of nearby agents and of the data that passes under it, constantly comparing the number of CJD cases in its vicinity to the local population size.

• When it finds an anomalous “hotspot” it gets excited

• When it finds no interesting data it gets bored

• When it finds no data at all it dies

Agents that are excited slow down, agents that are bored speed up and agents that are dead don’t move at all.

In addition, agents are attracted to nearby excited agents and repelled by any nearby deceased agents.

How does such a system behave?

Self-Organising Data Foragers

The swarm adaptively scans the data set focusing on the interesting parts and ignoring the boring areas.

• The swarm quickly isolates parts of the map with no cases of CJD, littering them with dead agents which dissuade further exploration of those areas.

• Areas where CJD cases are in line with local population density are boring and are quickly passed over.

• Areas where the number of CJD cases in abnormally high or low attract more and more agents of different sizes, scanning the area at different scales and resolutions.

In comparison with exhaustively scanning the map at many different spatial scales, the flocking approach is faster, but perhaps slightly less reliable – MacGill suggests a hybrid approach combing both methods.

Issues in Insect Algorithms

The development of swarm intelligence is only just beginning. Many open questions exist…

• how can we design individual agents such that en masse, they are able to achieve a desired swarm-level behaviour?– should they be complex or simple?– should they differ from one another?– should they be reactive or non-reactive?– should they learn? How?– should they communicate? How?– should they utilise stigmergy? How?

• should we worry that swarms are not predictable/reliable?

As yet we have few answers – what would a theory of swarm intelligent systems look like?

Recap…

Last Time:• command & control vs. self-

organisation• the sophisticated abilities of

insect colonies• stigmergy• algorithms inspired by insect

intelligence– Foraging

– Clustering & Sorting

– Building

– Swarming & Flocking

This Time:• insect-like social robots

– sorting & clustering

– cooperative transport

• Applications– sport

– education

– entertainment

Insect-like Social RobotsSo far, we have considered software algorithms inspired by the various behaviours of social insects.• Researchers are also implementing insect-like robot control systems for a range of tasks. These include single robots inspired by crickets & cockroaches, and multi-robot systems.• Before reviewing some of the collective behaviours that have been achieved with multiple robot systems, we will consider the question:• Why take a multi-robot approach?

single complex robot working aloneversus

many simple robots working together

Multi-Robot Advantages

There are many reasons to consider building multiple simple robots rather than a single complex solution:

• robustness – a simple machine may be less likely to fail than a complex one

• redundancy – many robots carrying out the same task ensures that the system can tolerate the failure of a few

• cost-effectiveness – many simple robots may be built for the price of a single complex machine

• simplicity – we often may not know how to build a single robot solution

• cooperation – the task may simply be impossible to carry out by an individual

Sorting and Clustering in Reality

We have already seen the application of insect-inspired clustering and sorting algorithms in software. Hardware implementations of similar ideas have shown how little intelligence each agent really needs…

At UWE Owen Holland, Chris Melhuish and colleagues, have been developing collective robot systems for some time.In experiments with robots the size of footballs, they have shown that individuals with very crude sensors, no memory or sense of spatial location, and no communication, can…

• segregate two kinds of object• sort them into a central cluster of object A surrounded by

a ring of object B – annular sorting resembling the brood sorting of ant species Leptothorax unifasciatus.

The Basic Sorting Task

Several mobile robots equipped with special grippers patrol an arena littered with pucks of two kinds – red and blue.

arena

robot with gripper

pucks

The robots can detect the gripper pushing against anything heavier than a puck, a gripped puck’s colour, and whether an obstacle is directly ahead…Three rules achieve simple clustering:1. if (gripper pushed & obstacle) then

turn at random2. else if (gripper pushed) then

reverse (dropping puck) and turn at random

3. else go forward

How Does It Work?

Notice that the robots know nothing about the location of a cluster or even the local density of pucks.

The robots succeed at the task because…• they pick up loose pucks without noticing • when they have a puck, they tend to deposit it next to

another puck (pushing a puck into another triggers rule 2)• when they don’t, they tend to only pick up pucks which

are on their own (pucks in groups trigger rule 2)

Interestingly, as in some experiments with ants, the robots often clustered the pucks at the edge of the arena.Through artificially varying the probability that rule 1 would fail to fire, the researchers found that they could increase or decrease this edge-seeking tendency.

Annular Sorting

In an attempt to speed up the clustering, the robots’ rule set was altered slightly to treat blue and red pucks differently:

The new robots treat the red pucks as normal, but drag blue pucks back a small distance before letting go of them.These robots did not cluster pucks any faster, but succeeded in roughly arranging the blue pucks in a doughnut-shaped ring around a cluster of red pucks – in a similar fashion to the brood sorting Leptothorax species…

1. if (gripper pushed & obstacle) then turn at random2. else if (gripper pushed) then

if (blue puck) then pull puck backwards endifreverse (dropping puck) and turn at random

3. else go forward

Collective Stigmergic Behaviour

These robot experiments show that what appears to be organised collective behaviour can be achieved by very simple robots following very simple rules:

• the robots made use of properties of their physical environment to help them coordinate their behaviour

• they exploited the stigmergic effects of each other’s behaviour to achieve their joint task

• the global-level effects of changes to the rules of individual robots was very hard to predict

Through this kind of simple robot research it may be possible to gain an understanding of…

• natural collective behaviour in insect colonies, etc.• the nature of stigmergy itself

Natural Collaborative Transport

Social ants often transport large items to their nests in teams, achieving feats that would be beyond a single ant.

In contrast, getting robots to work together as a team on a joint project, as we have just seen, is a tricky task.

Bonabeau, Dorigo and Theraulaz list some of the interesting questions that this insect behaviour suggests:

• Do individuals behave differently when they are in a team?• How are enough ants recruited when help is needed?• How do ants co-operate and co-ordinate their behaviour?• How do they handle deadlocks, obstacles, conflicts, etc.?

Studying the behaviour of collective behaviour in groups of robots is helping to answer some of these questions...

Building Robot Collaboration

In addition to helping us understand natural insect behaviour, achieving effective robot collaborative transport techniques would represent a significant feat of engineering.

Some engineering considerations:• How can interference between team-mates be reduced?• What is the optimal team size for a particular task?• How should team-mates accommodate each other?• How complicated must conflict resolution mechanisms be?• How should team-mates communicate with each other?

Observation of co-operative transport in ants tends to suggest that an efficient solution could be engineered using a group of robots lacking explicit means of communication, or any centralised control.

Ant-Like Robot Box Pushing

Co-operative box pushing is considered a bench-mark test for teams of multiple robots. Many researchers have taken an approach involving planning and explicit centralized mechanisms for conflict resolution. In contrast, Kube and Zhang have developed an ant-inspired system…

The box-pushing task is as follows:A lit box, which is too heavy for a single robot to push, must be moved from some random location in the middle of an arena to the arena’s edge.

The sensors on Kube & Zhang’s circular, wheeled robots can detect i) the box, ii) other robots, and iii) nearby obstacles.Each robot is identical and can carry out five different behaviours: GOAL, AVOID, FOLLOW, SLOW, & FIND.

www.cs.ualberta.ca/~kube/research.html

A Subsumption Architecture

Kube & Zhang use a subsumption architecture:

slow

followrobot?

goalgoal?

avoidobstacle?

find wheels

Each behaviour is triggered by a certain stimulus and suppresses those below it in the architecture.

The robot wanders until it detects either another robot, which it follows, or the goal, which it approaches.

Encountering an obstacle at any time will trigger a simple avoidance behaviour – turning away from the obstacle.

This controller enables robots to converge on the box. How do they co-operate in pushing it to the wall?

Co-operative Box Pushing

The robots described so far will succeed in locating the box and pushing it to the wall, but they are far from efficient.

Like ants, the robots suffer from periods of stagnation in which the box makes no significant progress.Kube and Zhang’s approach to solving this problem is inspired by ant behaviours: realignment and repositioning.If the box has not moved for a while a robot will realign itself by rotating slightly.Realignment alters the forces acting on the box while ensuring that it is still pushed in the same general direction. If realignment proves fruitless, a robot will reposition itself by backing away and re-approaching the box.This allows other robots to act on the box in its absence and changes its influence on the box.

Box Pushers in Action…

In this movie, six of Kube and Zhang’s robots attempt to push a lit box to the side of the arena.

As described, periods of stagnation cause robots to realign, or back off & reposition themselves.

Sport - FootballWe have all encountered tasks that demand more sophisticated coordination. A good example is football.

RoboCup, a football tournament for teams of robots, has been running for a few years now and acts as an informal way to assess the state-of-the-art in collective robotics.

- www.robocup.org

Many teams are controlled by traditional AI systems: modelling, planning and communicating in order to co-ordinate players on the joint task of scoring goals.Although no insects play anything like football, could a swarm intelligence approach succeed at RoboCup?This decentralised, minimally intelligent approach is clearly lacking in sophisticated reasoning, planning, etc. But could it make up for this through increased speed, robustness, fault tolerance, opportunism, etc?

Education – Star Logohttp://education.mit.edu/starlogo/

The literature on research into swarm systems and collective behaviour in general suggests that designing these systems is very hard. They behave in counter-intuitive ways. Understanding them requires new ideas and lateral thinking.

Are swarm systems intrinsically hard to understand, or have we just had too little experience of building and using them?Mitchel Resnick, in his book Turtles, Termites & Traffic Jams, sets out to explore whether exposure to these systems can provide young children with insights into how they behave.His StarLogo is a programming language designed to help users to build systems of distributed, interacting agents.Could exposure to systems like StarLogo help to create a new generation of thinkers who have been “liberated from the confines of the centralised mindset”?

Example – Levels of Description

One theme running through this module is levels of description – at one level a system may comprise many simple interacting individuals, while at a higher level it is a single organised whole, exhibiting complex coordinated behaviour.

The high-school students working with Resnick encountered this idea while building a StarLogo traffic jam simulator:

While the cars were always moving forward, the jam sometimes seemed to be moving backwards.

Some students coped well with this phenomenon – “[The traffic jam is] like a blob, which can move either backward or forward regardless of how the cars are moving”Others (e.g., MIT researchers) had difficulty – “A loop [in a StarLogo program] does the same thing every time. So the whole thing should be repeating itself.”

Entertainment - Animation

One perhaps surprisingly successful application of swarm systems has been in the domain of computer graphics.

Rather than animating a flock or herd or squadron or team or crowd by hand, swarm systems are able to generate complex, life-like sequences of behaviour automatically.The behaviour of each member of the group is governed by a control system which determines its movements (and perhaps the noises or gestures that it makes), taking into account the positions and movements of those around it.Systems inspired by insects or simple animals, will be simple to build and fast to simulate – perhaps fast enough to allow the generation of animation sequences in real-time.

Behavioural Animation in Action

Perhaps the most famous example of this technique is Craig Reynolds’ Boids – simple artificial birds which flock in a natural, life-like manner despite being controlled by a tiny set of very simple rules…

http://www.red3d.com/cwr/boids/

Boids• Basic flocking model consists of 3 simple steering

behaviours which describe how an individual boid manoeuvres based on the positions and velocities its nearby flockmates:

Separation: steer to avoid crowding local flockmates

Alignment: steer towards the average heading of local flockmates

Cohesion: steer to move toward the average position of local flockmates