Asynchronous Spatial Evolutionary Games: Spatial Patterns,Diversity and Chaos

D. Newth Member, IEEE and D. Cornforth

Abstract— Over the past 50 years, much attention has beengiven to the Prisoner’s Dilemma as a metaphor for problemssurrounding the evolution and maintenance of cooperative andaltruistic behavior. The bulk of this work has dealt with thesuccessfulness and robustness of various strategies. Nowakand May [1], considered an alternative approach to studyingevolutionary games. They assumed that players were distributedacross a two-dimensional lattice, and interactions betweenplayers occurred locally, rather than at long range, as in thewell mixed situation. The resulting spatial evolutionary gamesdisplay dynamics not seen in there well-mixed counterparts. Anassumption underlying much of the work on spatial evolution-ary games is that the state of all players is updated in unison orin synchrony. Using the framework outline in [1], we examinethe effect of various asynchronous updating schemes on thedynamics of spatial evolutionary games. There are potentialimplications for the dynamics of a wide variety of spatiallyextended systems in physics, biology and chemistry.

I. INTRODUCTION

Since its conception by Maynard Smith and Price, evo-lutionary game theory [2] has proven itself to be a usefultool in the study of phenotypic evolution in situations wherethe fitness of an individual is dependant upon the frequencyof a particular trait within a population. Evolutionary gametheory has been successfully applied to a wide range ofbiological phenomena including: animal contests [2], [3],[4], sex allocation [5], dispersal in a uniform environment[6], plant growth and reproduction [7], and the evolution ofcooperation [8], [9].

Typically, the evolution of cooperation and altruistic be-havior is studied through the game Prisoner’s Dilemma. In itscanonical form, the Prisoner’s Dilemma is a game consistingof two players, each of whom can elect to adopt one of twostrategies: to cooperate (i.e. play C); or to defect (i.e. playD), in any given encounter. Should both players choose toplay C, then they both receive the payoff R; if they choosedifferent strategies, then the player choosing D receives thehighest payoff T , while the player choosing C receives thepayoff S; should both players play D, then they both receivethe payoff P . For a game to be considered a Prisoner’sDilemma, the payoffs must satisfy the following conditions:(1) defection is always worth more than cooperation T > R

and P > S; (2) mutual cooperation is better than mutual

David Newth is with the CSIRO Centre for Complex Systems Science.CSIRO Marine and Atmospheric Research. The Commonwealth Scientificand Industrial Research Organisation. G.P.O. Box 284. Canberra. ACT.2601. Australia. (phone: +61 2 62421744; fax: +61 2 62 421677; email:david.newth@csiro.au).

David Cornforth is with the University of New South Wales, at theAustralian Defence Force Academy, Canberra ACT 2600 Australia. (phone:+61 2 62688956; fax: +61 2 62688581; email: d.cornforth@adfa.edu.au).

defection R > P ; and (3) alternating doesn’t pay as 2R >

(T + S). From this, it is immediately clear that in a gameconsisting of a single round, an individual should always playD, regardless of their opponent’s choice. But in a sequence ofencounters, repeatedly playing D leaves both players worseoff, as mutual cooperation pays more than mutual defection(R > P ). Following the pioneering work of Axelrod [8],[9], many studies have sought to understand what strategiesevolve under various conditions and constraints [4], [10],[11].

In this paper, we shall consider evolutionary games froma somewhat different perspective to that which is commonlyadopted. This approach was first proposed by Nowak andMay [1], and involves studying evolutionary games, wherethe players occupy regions distributed across some spatialdomain. Interactions between players are constrained to bebetween nearest neighbors, rather than long distance inter-actions as per the classical mean-field approach. A playeroccupying a particular region changes his strategy, if hisneighbors are more successful. This represents the evolu-tionary scenario where more successful phenotypes replaceless successful ones. This derivation of the game is knownas the Spatial Prisoner’s Dilemma. An assumption underlingmuch of the work on Spatial Prisoner’s Dilemma is thatall players update their strategy in unison, or in synchronywith some global clock. In many biological contexts, this isan unrealistic assumption, as a clock that causes all systemelements to update their state synchronously seldom exists.While seasonal effects and other external clocks can synchro-nize metabolic and reproductive cycles in some organisms,in many social settings individuals act at different and oftenuncorrelated time scales, making decisions on informationthat may be imperfect or delayed [12]. In a social context forexample, two players may be just concluding an encounter,while two other players may be just initiating an encounter.However, the result of the first encounter may influence thestrategies adopted by the individuals in the second encounter.Such scenarios are referred to as asynchronous updating.

Fig. 1 illustrates the contrast between the synchronous andasynchronous version of the Spatial Prisoner’s Dilemma. Inboth simulations a 99×99 square lattice with fixed boundaryconditions was used. Both simulations were seeded with thesame initial condition of a single defector surrounded bya sea of cooperators. Fig. 1 (left) shows the state of thesystem after 217 time steps, where each player is updatedin synchrony. As the system evolves, symmetry is main-tained, and the system consists of a polymorphic populationcontaining both cooperators and defectors. By contrast, Fig.

2463

1-4244-1340-0/07$25.00 c©2007 IEEE

1 (right) illustrates, the state of the system after 217 timesteps when players are updated asynchronously. Under thisupdating scheme, the population is dominated by defectors.

Fig. 1. Synchronous and asynchronous updating of players in the spatialPrisoners Dilemma. Both simulations were seeded with a single defectorsurrounded by a sea of cooperators and b = 1.9 (see Section III formore details). The color coding is outlined in Section IV The snapshotof the system was taken at time step 217. In the synchronous case (left)[1] the system is made up of a cooperator-defector polymorphic populationthat persists indefinitely. This configuration also generates complex spatialpatterns. In the asynchronous case (right), the population collapses anddefection is the only stable strategy

Synchronous and asynchronous updating are only two ofa wide range of possible updating schemes. In previouswork [13], we have shown that random asynchronous andordered asynchronous updating schemes can generate com-plex behaviors not seen in their synchronous counterparts.The main focus of this paper is to explore the effect ofalternative updating schemes on the evolutionary dynamicsof spatial Prisoner’s Dilemma. In the following section, webriefly review a number of updating schemes observed inbiological systems. Following on from this, we provide aformal definition of spatial evolutionary games and detailhow each of the updating schemes is implemented, alongwith the measures used to analyze the behavior of the system.In section IV, we describe the simulation experiments andresults, and finally in sections V we provide a discussion ofthe findings and some concluding comments.

II. ASYNCHRONOUS PROCESS IN BIOLOGICAL SYSTEMS

Biological systems provide abundant evidence that agents(players) update their state in accordance with one of a widearray of updating schemes. In this section we will provide anumber of examples, drawn from biology and ecology, wherethe dynamics of the system are driven by different updatingschemes. We have already seen the affect of synchronousand asynchronous updating on artificial systems; however,many natural systems fall somewhere between these twoschemes. Often these alternative updating schemes containsome degree of structure or ordering and can even demon-strate the ability to synchronize over time. We collectivelyrefer to these semi-structured updating schemes, as orderedasynchronous processes.

A. Examples from Biology of Various Updating Schemes

1) Social networks: Social networks are groups of peopleinteracting via social contacts. The “states” of people include

their opinions and beliefs. Interactions and changes in statemay take place synchronously (e.g., the influence of massmedia) or asynchronously [14]. We can identify two typesof random asynchronous updating in social networks. Thefirst type occurs when people meet by chance and subsequentinteractions cause them to re-evaluate their opinions. This isindependent random sampling, that is, an individual is chosenand updated at random with replacement. So the probabilityof a state update is independent of the number of previousstate updates. The second type occurs during an election. Inthis case, once a person has voted, they are not allowed tovote again until the next election. In this scenario, the orderof update is random, but each individual is updated once andonce only during each round. At each stage, the individualto be updated is chosen by random without replacement.Another example of the second type of updating scheme isvaccination, where the order of people being vaccinated isessentially determined at random, but each person is onlyvaccinated once for a particular disease, thus the dynamic ofan outbreak is altered.

2) Neural activity: The behavior of interconnected neu-rons in the brain leads to global patterns of behavior acrossthe whole brain. This activity does not exhibit stationarypatterns, but periodic, quasi-periodic and chaotic patterns[15]. There is no known mechanism such as a global clockin the brain, yet neurons exhibit synchronized behavior fora time, suggesting a mechanism of asynchronous updatingleading to periods of synchronous updating.

3) Forest succession: The competition between differentspecies within a forest ecosystem, coupled with catastrophicevents such as forest fires, leads to a complex system ofinteractions. Succession between different community classes(such as a transition from rainforest to open savanna) requiresvastly different time scales to complete [16]. A fire-inducedtransition from woodland to grassland may be virtuallyinstantaneous, whereas a transition to mature rainforest mighttake thousands of years to complete. Recognition of theasynchronous nature of forest succession led to the adoptionof the semi-Markov model in this context [17]. Although anordered-asynchronous updating is implicit in these models,forest succession has not been widely recognized as belong-ing to this category of processes.

4) Coordination of resource sharing: Within a large pop-ulation, a sub-population may act in synchrony. Separationof flowering times in eucalypts is a good example of thisbehavior. This separation of time scales (where one sub-species will flower followed by another and so on) is amechanism for maintaining genetic “identity”, and limithybridization [18]. In the late 1950’s Hutchinson [19] notedthat the reproductive cycles of sub-species of beetles wereordered in lock-step, so as to avoid direct competition foravailable resources.

5) Synchronization of ovulation: The synchronization ofreproductive cycles via pheromone signalling is one of thebest examples of a self-synchronizing processes. Initiallyindividuals within a population have a well defined repro-

2464 2007 IEEE Congress on Evolutionary Computation (CEC 2007)

ductive cycle. Pheromone signals passed between membersof the population “nudge” the phase of the individualsreproductive cycle until they fall into phase. This is believedto be the major driver in ovarian synchrony, and has beenreported in several mammalian species including: humans[20]; chimpanzees [21]; golden lion tamarins, Leontideusrosalia [22]; and golden hamsters [23].

B. Updating Schemes

Here we will consider a total of six specific update pat-terns, including two random asynchronous updating schemesand three ordered asynchronous updating schemes. Theseare:

1) Synchronous updating: Here all agents are updated inparallel at each time step.

2) Random asynchronous updating with replacement up-dating: At each time step, an agent is chosen at random withreplacement.

3) Random asynchronous updating without replacementupdating: At each time step, all agents are updated, but ina random order, and each agent is only updated once.

4) Random asynchronous updating with a fixed orderupdating: At each time step an agent is chosen accordingto a fixed random order, which was decided upon during theinitialization of the model. All players are updated once.

5) Clocked updating: Clocked updating scheme is anexample of an ordered asynchronous process. Each agenthas an internal clock (oscillator). Initially all clocks are setto a random initial phase. During each time-step, the clockphase is advanced by some fraction. When the clock reachesor passes a particular time it expires, and the agent updates.If all clocks were initialized to the same initial phase, thesystem would behave synchronously. This scheme modelssituations such as the lock-step behavior of the flowering ofEucalypts sub-species.

6) Self-synchronizing updating: This updating scheme issimilar to the clocked scheme, but incorporates local syn-chrony. Each clock can tune itself to fall into phase with otherclocks within the system. This update scheme is analogousto synchronization of reproductive cycles.

III. SPATIAL GAMES

In this section we shall give a general outline of thespatial evolutionary game studied here. We consider a gameconsisting of a set of finite strategies S and a finite numberof players. Each player I adopts a strategy in S. Let E(i, j)be the payoff to an individual adopting strategy i ∈ S againstan opponent adopting strategy j ∈ S. Each player I occupiesa site on a regular lattice A —which represents a landscape,and each site on the lattice represents a territory within thelandscape. In this context A is a regular 2-dimensional lattice.Every site on the lattice has a set of neighbors denoted byN(I). For this study, N(I) is chosen to be I’s eight nearestneighbors or the Moore neighborhood. A fixed boundarycondition was used for all experiments presented here.

A spatial evolutionary game is defined by an associationat time t of a strategy σt(I) ∈ S to each cell I ∈ A, with a

rule that determines the strategy occupying I at t + 1. Thedynamical process is defined as follows:

1) the total st(I) for player I time t is defined as thesum of the payoffs resulting from playing all theneighboring cells. That is:

st(I) =∑

J∈N(I)

E(σt(I), σt(J)). (1)

2) Using these scores, we can associate a strategy witheach player I at time t + 1. For any player I ∈ A,let i ∈ S be the strategy associated at generation t

with the cell J ∈ N(I) which has the maximum scoresmax

t (I).3) We then set σt+1(I) = i. More specifically, the

strategy occupying site I at t+1 is the strategy withinN(I) that receives the highest payoff.

We follow Nowak and May [1] in the selection of thepayoff values. Specifically, R = 1, T = b (with b > 1),S = P = 0. That is, mutual cooperation scores 1, mutualdefection scores 0, and double crossing your opponent paysb. The parameter b characterizes the advantage of defectorsover cooperators, and as such forms a control parameterthrough which dynamical behavior of the system can bestudied. In their study Nowak and May [1], found the most“interesting” regime occurred when 1.8 ≤ b < 2 (see fig. 1left). Huberman and Glance [12], showed that in the randomasynchronous case, this region was a point attractor (See fig.1 right).

A. Updating schemes

Within the framework defined above, it is possible to de-fine alternative updating schemes. In our alternative updatingschemes we consider a time step to be completed when n

updates have occurred. We define n to equal the number ofplayers within the model. We refer to the update of a singleplayer as a micro-time-step, and the update of all n playersas a macro-time-step.

1) Synchronous updating: This is the behavior describedabove. In short, at each time step, every player’s score isevaluated and state updated in parallel.

2) Random asynchronous updating with replacement:Under this updating scheme, each macro-time-step is dividedinto n micro-time-steps. At each micro-time-step a player isselected at random from the population, and updated. As aconsequence, as each player is updated, they “awake” to seea slightly different world from that of the players updatedbefore and after them.

3) Random asynchronous updating without replacement:Under this updating scheme, for each micro-time step, aplayer is chosen at random and updated. Unlike the randomasynchronous updating with replacement scheme, once aplayer is updated it cannot be updated again.

4) Random asynchronous updating with a fixed order:This updating scheme is identical to the random asyn-chronous updating without replacement scheme; however,players are updated in a fixed random order throughout theentire simulation [24].

2007 IEEE Congress on Evolutionary Computation (CEC 2007) 2465

5) Clocked updating: The Clock updating scheme [25]assigns a clock or an oscillator to each player. Initially eachclock is setto a random starting position along its period.During each micro-time-step, the oscillator moves along itsperiod by some fixed amount. When the oscillator reachesthe top of its period, the player awakens and updates its state.The process is repeated until n updates are completed.

6) Self-Synchronizing updating: This scheme is similar tothe clocked updating scheme, except the clock of each playeris influenced by the clocks of other players. Here, clocks aremodeled as a Kuramoto style self-synchronizing oscillator[26]:

θi = ωi +∑

j

Γij(θi − θj), (2)

where θi is the phase, ωi is the natural frequency of oscillatori. Updating is identical to the clocked scheme, however eachplayer can influence the clocks of all other players throughthe phase difference function Γ(·). Here Γ(θi − θj) is afunction of the phase difference between the two clocks.This model is used when there is total coupling betweenall oscillators. For our model we modified this and use:

θi(t+1) = θi(t) + ωi + β(θ − θi), (3)

where θ is the mean phase in the system and is given by

θ = arctan

(∑i sin(θi)∑i cos(θi)

). (4)

B. Measures

To compare the dynamical properties of the various updat-ing schemes, we employ the use of three measures. Theseare: (1) the Fraction of cooperators (h); (2) Neighborhoodentropy (H); and (3) Lyapunov exponent (λ).

1) Fraction of cooperators: This statistic characterizesthe number of cooperators (and subsequently the numberof defectors) making up the population [1]. The fraction ofcooperators h is calculated by:

h =Cn

L2(5)

where Cn is the number of lattice sites occupied by acooperator, and L is the side length of the lattice.

2) Neighborhood entropy: The second measure used hereis entropy of the neighborhood configurations. This is theShannon-Weaver information content [27]:

H =∑

i

pi log2 pi (6)

where pi is the frequency of the occurrence of the ith

neighborhood configuration. The statistic shows the diversityand thus the complexity of the structures formed upon thelattice.

0.0

0.2

0.4

0.6

0.8

1.0

h

SynchronousAsynchronous

0.0

1.0

2.0

3.0

4.0

5.0

6.0

H

100

101

102

103

104

Time Step

0.0

0.1

0.2

0.3

0.4

0.5

λ

(a)

(b)

(c)

Fig. 2. Dynamics of the Spatial Prisoner’s Dilemma for the first 10,000time steps for the initial conditions outlined in Fig. 1. (a) The frequency ofcooperators. In the synchronous model this oscillates around h = 0.32; bycontrast, cooperators become extinct by 200 generations in the random asyn-chronous model, and h = 0. (b) Entropy of the neighborhood configurations.For the synchronous model, H ≈ 5.1, while in the random asynchronousversion, H ≈ 1.18, as the model converges to the point attractor where onlydefectors exist. (c) Lyapunov exponent. The synchronous model λ ≈ 0.41,meaning the small perturbation at t = 0 forces the two models to divergequickly. In the random asynchronous version, λ ≈ 0, indicating that theperturbation dies out and the models converge to the same state.

3) Lyapunov exponent: To measure the sensitivity ofthe model to initial conditions, we calculate the Lyapunovexponent λ for each updating scheme. To do this we maintaina second model that is identical to the first, with the state at asingle site changed. Both models are updated in the same wayas they evolve. At each time step, the number of differencesin the two models Nd is calculated. The Lyapunov exponentis then calculated as:

λ =Nd

L2t(7)

where L is the side length of the lattice, and t is the numberof iterations since initialization. The Lyapunov exponentcharacterize the divergence or instability within the systemdue to a small perturbation.

Combined, these three measures tell us about the dynami-cal properties of the various updating schemes. Fig. 2 showshow each of these values changes over time for the two casesshown in fig. 1.

IV. EXPERIMENTS AND RESULTS

As an initial analysis of the dynamics of the updatingschemes, fig. 3 illustrates typical asymptotic patterns for eachof the updating schemes and various b values, on a 99 × 99square lattice with fixed boundary conditions. The modelswere seeded with a random starting configuration containingapproximately 50% cooperators and 50% defectors. Thecolor coding is a s follows: blue represents a player whoon the current and previous time step adopted the strategyC; green is a player who adopted C following a D on the

2466 2007 IEEE Congress on Evolutionary Computation (CEC 2007)

previous time step; yellow a player who adopted strategy D

but played C on the previous time step; and red is a playerwho played D on the current and previous time step. For b =1.1, all updating schemes converged to a static configuration,with defectors surrounded by cooperators. When b = 1.3 andb = 1.5, the synchronous and self-synchronizing schemescreated spatial patterns with stable lines of defectors, andsmall sub-populations that “flip” between cooperation anddefection (blinkers). For these parameter settings, the otherschemes produced a network of defector lines, that contin-ually pulled themselves apart and reassembled in a slightlydifferent configuration. When b = 1.7 all updating schemesconverged to almost static configurations of defector lines,with a few blinkers. When b = 1.9, both synchronousand self-synchronizing schemes generated rich dynamicalbehavior creating complex spatial patterns, while the clockedand random asynchronous schemes both lead to a pointattractor where defection was the only long term viablestrategy. Finally, for b = 2.1, all the updating schemes leadto a point attractor when the only surviving strategy wasdefection.

The asymptotic pictures shown in fig. 3, show that thealternative updating schemes are capable to creating an arrayof interesting behaviors. To examine these behaviors further,100 runs were made of each the updating schemes on a 200×200 regular lattice with fixed boundary conditions, for 1 ≤b ≤ 2. The asymptotic values for h, H , and λ were recordedand averaged over the 100 runs. Figs 4–6, show the resultsof these experiments.

First, we examine the fraction of the population made upof cooperators (h). Fig. 4 shows the fraction of cooperators h

for each of the updating schemes. The most notable differ-ence between the synchronous and asynchronous schemes,is that the synchronous updating scheme supports morecooperators than the other updating schemes. The great-est difference between the synchronous and asynchronousschemes occurs when b > 1.8. In this parameter range,all the asynchronous schemes, with the exception of theself-synchronizing scheme, converge to population composedentirely of defectors (h = 0). For the synchronous schemethe population displays rich dynamical behavior as describedby [1] and shown in fig. 1 (left), here h ≈ 0.3. Under theself-synchronizing scheme, local pockets of agents synchro-nize their behavior and sustain a small sub-population ofcooperators with h ≈ 0.1.

The neighborhood entropy H , describes the diversity ofneighborhood configurations of cooperators and defectorsupon the lattice. Fig. 5 show the neighborhood entropy foreach updating schemes. As suggested by Nowak and May[1], the most interesting behavior in the synchronous caseoccurs between 1.8 ≤ b < 2. The self-synchronizing schemealso displays high H value for this region. However, all theasynchronous schemes display high neighborhood entropyfor 1.28 < b ≤ 1.3 and 1.5 ≤ b < 1.6. The reason for this,is that, in these parameter ranges, certain combinations ofneighborhoods are likely to create and destroy structures,

1.0

1.0

1.2

1.2

1.4

1.4

1.6

1.6

1.8

1.8

2.0

2.0

b

0.0

0.0

0.2

0.4

0.2

0.6

0.4

0.8

0.6

1.0

0.8

h

1.0

h

1.0

1.0

1.2 1.4

1.2

1.6

1.4

1.8

1.6

2.0

1.8 2.0

b

0.0

0.2

0.0

0.4

0.6

0.2

0.8

0.4

1.0

0.6

0.8

1.0

1.0 1.2 1.4 1.6 1.8 2.00.0

0.2

0.4

0.6

0.8

1.0

h

1.0 1.2 1.4 1.6 1.8 2.00.0

0.2

0.4

0.6

0.8

1.0

(a) (b)

(c) (d)

(e) (f)

Fig. 4. Fraction of cooperations occupying the lattice for each of theupdating schemes. (a) Synchronous updating; (b) Random asynchronousupdating with replacement; (c) Random asynchronous updating withoutreplacement; (d) Random asynchronous updating with a fixed order; (e)Clocked updating; (f) Self-synchronizing updating.

resulting in rich behavior. We now move on to see howsensitive the updating schemes are to initial conditions.

The fraction of cooperators, and neighborhood entropydescribe the features of the evolving population (their com-position and diversity of structure). The Lyapunov exponent(λ), describes how sensitive the model is to initial conditions,by measuring how fast a small disturbance grows through theentire population. The synchronous updating scheme is mostsensitive to initial conditions when 1.8 < b < 2. However,for all other values of b the model is quite robust to initialconditions. By contrast, all the other updating schemes arequite sensitive to initial conditions. In particular there aretwo regions, where the asynchronous updating schemes aremost sensitive to initial conditions: 1.28 < b ≤ 1.3 and1.5 ≤ b < 1.6. These correlate to the regions that displayhigh neighborhood entropy. Somewhat surprisingly, the self-synchronizing scheme does not display sensitivity to initialconditions like the synchronizing scheme. The reason forthis, is the model is initially behaves as a random asyn-chronous updating scheme; as a result, small perturbationsdie out. Over time the agents become synchronized, andthe system takes on the dynamical properties seen in thesynchronous updating scheme. The time required for theagents to synchronize is grater than the time required forthe perturbation to die out, so the self-synchronizing schemeis highly robust to initial conditions for b > 1.8.

2007 IEEE Congress on Evolutionary Computation (CEC 2007) 2467

(d)

(e)

(f)

b=1.3 b=1.5 b=1.7 b=1.9 b=2.1b=1.1

(a)

(c)

(b)

Fig. 3. Typical asymptotic patterns for each of the updating schemes, for a range of b values. (a) Synchronous updating; (b) Random asynchronousupdating with replacement; (c) Random asynchronous updating without replacement; (d) Random asynchronous updating with a fixed order; (e) Clockedupdating; (f) Self-synchronizing updating.

It is interesting to note the relationship between b valuesand complexity classes of cellular automata as proposed byWolfram [28]. Under this classification scheme, the behaviorof a cellular automata can be classified as belonging to one offour classes. Class I behavior, is very simple behavior, wherealmost all initial conditions lead to exactly the same uniformfinal state. Class II behavior, displays many different possiblefinal states, but all of them consist just of a certain set ofsimple structures that either remain the same forever or repeatevery few time steps. Class III behavior is more complex, andseems in many respects random, although coherent structuresappear repeatedly. Class IV behavior involves a mixture oforder and randomness to produce complex patterns. Cellularautomata that can produce Class IV behavior, generally cre-ate a variety of simple local structures, that interact with eachother to produce more complicated structures [29]. TableI provides a coarse-grained classification of the behaviorcreated by each updating scheme for various b values.

On further examination of each updating scheme, we

found that all the random asynchronous updating schemesand clocked updating scheme, are able to create “gliders”that randomly walk across the lattice for 1.13 ≤ b < 1.14.Fig. 7 (left) illustrates these gliders. However due to thefixed boundary conditions, as the gliders are lost the systemconverges to a fixed state containing only cooperators (seefig. 7 right). For this narrow range of b values, the SpatialPrisoner’s Dilemma can be compared to Conway’s Game ofLife. Under the synchronous updating scheme, these glidersare blinkers. More work is needed to examine if these glidersare capable of creating new gliders and ultimately morecomplex behavior.

V. DISCUSSION

The Prisoners Dilemma in an interesting metaphor for thebiological problem of how cooperative behavior may emergeand persist. Many of the previous studies are confined to in-dividuals or groups of individuals who have memories of pastevents; expectations of future encounters and can formulate

2468 2007 IEEE Congress on Evolutionary Computation (CEC 2007)

TABLE I

CLASSIFICATION OF THE COMPLEXITY CREATED BY EACH UPDATING SCHEMES FOR VARIOUS RANGES OF b VALUES. (A) SYNCHRONOUS UPDATING;

(B) RANDOM ASYNCHRONOUS UPDATING WITH REPLACEMENT; (C) RANDOM ASYNCHRONOUS UPDATING WITHOUT REPLACEMENT; (D) RANDOM

ASYNCHRONOUS UPDATING WITH A FIXED ORDER; (E) CLOCKED UPDATING; AND (E) SELF-SYNCHRONIZING. † FIX POINTS ONLY. ‡ FIXED POINTS

AND SHORT CYCLES.

Scheme 1 ≤ b < 1.25 1.25 ≤ b ≤ 1.28 1.28 < b ≤ 1.3 1.3 < b < 1.5 1.5 ≤ b < 1.6 1.6 ≤ b < 1.8 1.8 ≤ b < 2

(a) Class II† Class II‡ Class II‡ Class II‡ Class II‡ Class II‡ Class IV

(b) Class II† Class II† Class III Class II‡ Class III Class II‡ Class I

(c) Class II† Class II† Class III Class II‡ Class III Class II‡ Class I

(d) Class II† Class II† Class III Class II‡ Class III Class II‡ Class I

(e) Class II† Class II† Class III Class II‡ Class III Class II‡ Class I & III

(f) Class II† Class II† Class III Class III Class III Class II‡ & III Class I & IV

1.0

1.0

1.2

1.2

1.4

1.4

1.6

1.6

1.8

1.8

2.0

2.0

b

0.0

0.0

1.0

1.0

2.0

2.0

3.0

3.0

4.0

4.0

5.0

5.0

6.0

6.0

7.0

7.0

HH

1.0

1.0

1.2

1.2

1.4

1.4

1.6

1.6

1.8

1.8

2.0

2.0

b

0.0

0.0

1.0

2.0

1.0

3.0

2.0

4.0

3.0

5.0

4.0

6.0

5.0

7.0

6.0

7.0

1.0 1.2 1.4 1.6 1.8 2.00.0

1.0

2.0

3.0

4.0

5.0

6.0

7.0

H

1.0 1.2 1.4 1.6 1.8 2.00.0

1.0

2.0

3.0

4.0

5.0

6.0

7.0

(a) (b)

(c) (d)

(e) (f)

Fig. 5. Neighborhood entropy for each of the updating schemes. (a)Synchronous updating; (b) Random asynchronous updating with replace-ment; (c) Random asynchronous updating without replacement; (d) Randomasynchronous updating with a fixed order; (e) Clocked updating; (f) Self-synchronizing updating.

complex strategies such as Tit-for-tat [8]. By contrast, theapproach taken here was to study the emergence of coopera-tion amongst simple organisms with limited memories, whoformulate very simple strategies. Specifically we examinedthe effect of asynchrony in the spatial version of Prisoner’sDilemma. Spatial Prisoner’s Dilemma is analogous to apopulation distributed across a landscape with interactionsbetween players occurring locally rather than long distance.The inclusion of alternative updating schemes means thatplayers make decisions about future interactions on delayedand imperfect information. When players are updated in these

1.0

1.0

1.2

1.2

1.4

1.4

1.6

1.6

1.8

1.8

2.0

2.0

b

10

10

−4.0

10

−4.0

10

−3.0

10

−3.0

10

−2.0

10

−2.0

10

−1.0

10

−1.0

10

0.0

λ

0.0

λ

1.0 1.2

1.0

1.4 1.6

1.2

1.8

1.4

2.0

1.6 1.8 2.0

b

10−4.0

10

10

−3.0

−4.0

10

10

−2.0

−3.0

10

10

−1.0

−2.0

10

10

0.0

−1.0

100.0

1.0 1.2 1.4 1.6 1.8 2.010

−4.0

10−3.0

10−2.0

10−1.0

100.0

λ

1.0 1.2 1.4 1.6 1.8 2.010

−4.0

10−3.0

10−2.0

10−1.0

100.0

(a) (b)

(c) (d)

(f)(e)

Fig. 6. Lyapunov exponent for each of the updating schemes. (a)Synchronous updating; (b) Random asynchronous updating with replace-ment; (c) Random asynchronous updating without replacement; (d) Randomasynchronous updating with a fixed order; (e) Clocked updating; (f) Self-synchronizing updating.

alternative ways, it doesn’t always follow that cooperationwill emerge and persist, or even that the system will displayrich dynamical behavior.

In our experiments, we found that the inclusion of variousasynchronous interactions between players lead to simu-lations that produced complicated dynamical features, notseen in the synchronous counterparts. The Spatial Prisoner’sDilemma is essentially a 2D cellular automata with a rulespace of 225. This rule space creates a rich “zoo” of dy-namical objects (such as rotators, gliders, blinkers etc.) Thealternative updating schemes, presented here, alterer the rule

2007 IEEE Congress on Evolutionary Computation (CEC 2007) 2469

Fig. 7. Random walking gliders. The random asynchronous updatingschemes and the clocked scheme are capable of creating gliders that ran-domly walk across the world (left). However the fixed boundary conditions,lead to a fixed point (right) where the world is dominated by cooperators.As a result, for these boundary conditions, this range of b values, theseupdating schemes display Class II complexity

space by limiting the transitions that can/cannot occur. As aresult new structures not observed in the synchronous version(e.g. the random walking glider, which in the synchronouscase is a blinker). Also we found that the degree of coop-eration achieved was strongly dependant upon the updatingscheme and the reward of double-crossing your opponent.Population that were able to support both cooperators anddefectors in the synchronous world, were unable to do so inany of the alternative schemes examined here.

These results show that while computers experimentsprovide a versatile approach for studying complex systems,an understanding of their subtle characteristics is required inorder to reach valid conclusions about the real world. Thestudy by Nowak and May [1] is only one of many studieswhere the outcomes of synchronous computer models havebeen invoked to provided insights into the workings of livingsystems.

In the real world, purely synchronous or asynchronousupdating appears to be rare in system composed of manyinteracting elements. Many natural processes seem to liesomewhere between these two extremes. We refer to the pro-cesses that fill this gap as ordered asynchronous processes.As our results show, the exact manner of updating can havea profound effect on overall system behavior and structure.One implication of this is the realization that a majority ofnatural systems are able to function without external synchro-nization. Another is that when building models of naturalsystems, it is important to consider the updating schemeused. While the motivation for this work was to examinehow alternative updating schemes change the dynamics ofSpatial Prisoner’s Dilemma, the results here are also relevantto the dynamics of a spatially extended systems such as Isingsystems, spatially distributed predatory systems [30], [31],and even models of pre-biotic evolution [32].

REFERENCES

[1] M. A. Nowak and R. M. May, “Evolutionary games and spatial chaos,”Nature. vol. 359. pp. 826–829. 1992.

[2] J. Maynard Smith and G. R. Price, “The logic of animal confilict,”Nature. vol. 246. pp. 15–18. 1973.

[3] J. Maynard Smith, “The theory of games and teh evolution fo animalconflicts,” Journal of Theoretical Biology. vol. 47. pp. 209–221. 1974.

[4] J. Maynard Smith, Evolution and the theory of games. CambridgeUniversity Press. 1982.

[5] E. L. Charnov, D. W. Gotshall and J. G. Robinson, “Sex ratio: adaptiveresponse to population fluctuations in pandalid shrimp. Scicence. vol.200. pp. 204–206. 1978.

[6] W. D. Hamilton and R. M. May, “Dispersal in stable habitats,” Nature.vol. 269. pp. 578–581. 1977.

[7] M. Mirmirani and G. Oster, “Competition, kin selection and evolu-tionary stable strategies,” Thoretical Population Biology. vol. 13. pp.36–40. 1979.

[8] R. Axelrod, The evolution of cooperation. Basic Books: New York.1984.

[9] R. Axelrod and W. D. Hamilton, “The evolution of cooperation,”Science. vol. 211. pp. 1390–1396. 1981.

[10] M. R. Frean, “The prisoner’s dilemma without synchrony,” Proceed-ings of the Royal Society, Series B, vol. 257. pp. 75–79. 1994.

[11] M. A. Nowak and K. Sigmund, “The alternating prisoners dilemma,”Journal of Theoretical Biology. vol. 168. pp. 219-226. 1994.

[12] B. A. Huberman and N. S. Glance, “Evolutionary games and computersimulations,” Proceedings of the National Academy of Science. vol. 90.pp 7716–7718. 1993.

[13] D. Cornforth, D. G. Green, and D. Newth, “Ordered asynchronousprocesses in multi-agent systems,” Physica D. vol. 204. pp. 70–82.2005.

[14] R. Stocker, D. G. Green and D. Newth,“Consensus and Cohesion inSimulated Social Networks,” The Journal of Artificial Societies andSocial Simulation (JASSS), vol. 4. 2001.

[15] W. Freeman, “Tutorial on neurobiology: from single neurons to brainchaos,” International Journal of Bifurcation and Chaos. vol. 2. pp.451–482. 1992.

[16] I. R. Noble and R. O. Slatyer, “The use of vital attributes to pre-dict successional changes in plant communities subject to recurrentdisturbances,” Vegetation. vol. 43. pp. 5–21. 1980.

[17] R. A. Howard, Dynamic Probabilistic Systems: Semi-Markov andDecision Processes, Wiley:New York. 1971.

[18] M. R. Keatley, I. L. Hudson, I. L. and T. D. Fletcher, “Long-termflower synchrony of box-ironbark eucalypts,” Australian Journal ofBotany, vol. 52, pp. 47–54. 2004.

[19] G. E. Hutchinson, “Homage to santa rosalia or why are there so manykinds of animals?” The American Naturalist. vol. XCIII. pp 145–158.1959.

[20] M. K. McClintock, “Menstrual synchrony and suppression,” Nature,vol. 229. 244–245. 1971.

[21] J. Wallis, “Synchrony of estrous swelling in captive group-livingchimpanzees (Pan troglodytes),” International Journal of Primatology,vol. 6, 335–350. 1985.

[22] J. A. French and J. A. Stribley, “Synchronization of ovarian cycleswithin and between social groups in golden lion tamarins (Leontop-ithecus rosalia),” American Journal of Primatololgy. vol. 12. pp. 469–478. 1987.

[23] G. Handelmann, R. Ravizza, and W. J. Ray “Social dominancedetermines estrus entrainment among female hamsters,” Hormones andBehavior, vol. 14. pp. 107–115. 1985.

[24] Y. Kanada, “The Effects of Randomness in Asynchronous 1D CellularAutomata,” Proceedings of Artificial Life IV. 1994.

[25] S. H. Low and D. E. Lapsley, “Optimization Flow Control, Part I: BasicAlgorithm and Convergence,” IEEE/ACM Transactions on Networking.1999.

[26] S. H. Strogatz, “From Kuramoto to Crawford: exploring the onset ofsynchronization in populations of coupled oscillators,” Physica D, vol.143. pp. 1–20. 2000.

[27] C. E. Shannon and W. Weaver The mathematical theory of communi-cation. University of Illinois Press: Illinois. 1963.

[28] S. Wolfram, “Cellular automata as models of complexity,” Nature. vol.311. pp. 419–424. 1984.

[29] S. Wolfram, A new kind of science. Wolfram Media: Illinois. 2002.[30] M. Frean and E. R. Abrahams. “Rock–paper–scissors and the survival

of the weakest,” Proceedings of the Royal Society of London. SeriesB, vol 268. pp. 1323–1327. 2001.

[31] C. R. Johnson and I. Seinen. “Selection for restraint in competitiveability in spatial competition systems,” Proceedings of the RoyalSociety of London. Series B, vol. 269. pp. 655–663. 2002.

[32] M. C. Boerlijst and P. Hogeweg, “Spiral wave structure in pre-bioticevolution: hypercycles stable against parasites,” Physica D, vol. 48.pp. 17–28. 1991.

2470 2007 IEEE Congress on Evolutionary Computation (CEC 2007)