Phase Transitions and Cellular Automata

Blake Johnson

Paper Group A1CS 297: Complex Systems

Introduction

Computation, Dynamics and the Phase Transition, by J.Avnet, Santa Fe Institute. 2000.

An introduction to the relationship between Cellular Automata, computation, and dynamical systems, that reviews and explicates previous work, primarily by Crutchfield, Wolfram, and Langton.

Cellular Automata

Cellular Automata are "Systems of Finite Automata," i.e. Deterministic Finite Automata (DFAs) arranged in a lattice structure. The input tp each DFA is the collective state of itself and some group of nearby cells considered its neighborhood, N.

Each cell contains the same DFA as the others and the same neighborhood template.

"Uniform" CAs are most commonly studied, where the state transition functions are the same for all automatons. (But non-Uniform CAs have been shown to be able to solve some problems that Uniform CAs cannot.)

The authors believe CAs to have properties relevant to the studies of computation and dynamical systems.

The Transition Function A DFA consists of a set of states Q and rules that

transition between them based on an input. The authors are interested in what properties of the

ruleset correspond to CAs which are most likely to be capable of performing useful computations.

The number of possible rule sets for a DFA is |outputs||inputs|. (where |inputs| is the number of possible inputs times the number of states)

For example, for a DFA with 2 states and 3 possible input values, there are 2(3*2) = 64 possible rule sets. Here are the first few:

s0(0)=s0, s0(1)=s0, s0(2)=s0, s1(0)=s0, s1(1)=s0, s1(2)=s0 s0(0)=s0, s0(1)=s0, s0(2)=s0, s1(0)=s0, s1(1)=s0, s1(2)=s1 s0(0)=s0, s0(1)=s0, s0(2)=s0, s1(0)=s0, s1(1)=s1, s1(2)=s0 s0(0)=s0, s0(1)=s0, s0(2)=s0, s1(0)=s0, s1(1)=s1, s1(2)=s1 s0(0)=s0, s0(1)=s0, s0(2)=s0, s1(0)=s1, s1(1)=s0, s1(2)=s0 …

The Transition Function

In a Cellular Automaton, the input is the state of a neighborhood of |N| cells. The number of possible inputs is therefore |Q||N|.

Thus, the rule space – the number of possible rule sets, in a CA is

|outputs||inputs| = |Q||Q|^|N|. That’s a lot of possible rule sets. For the game of

live, we have 22^9 = approximately 10153.

How Can We Analyze 10153 Possible Rulesets? We would like to partition, or organize, the

space of rulesets in some structured way, so that we can calculate some parameter, or attribute, of each rule set which might correspond, in some rough way, with those rulesets that produce CAs capable of performing computation.

Such a parameter should be designed so that rulesets with similar parameter values have similar properties.

Introducting λ (lambda) Christopher Langton proposed the parameter λ

as an important property of rulesets. Pick one state in your CA to be considered the quiescent, or inactive state, sq.

Let there be n randomly selected transitions to sq in the ruleset. Let all other transitions be selected uniformly and randomly to states other than sq.

Define λ as (|Q||N|-n)/|Q||N|. Essentially, this means that λ is the proportion of all transitions that lead to the quiescent state sq.

Introducting λ (lambda)

Some interesting values of λ: λ = 0 means that all transitions are to sq. This

is the most homogeneous scenario. λ = 1 means that no transitions are to sq. λ = 1.0 – 1/|Q| means that all states are

equally represented in the rule set. (I’ll elaborate on the next page). This is the most heterogeneous scenario. For the Game of Life, where each cell has two states, λ = 1.0 – 1/|Q| = 0.5

Derivation of λ = 1.0 – 1/|Q| There are |Q| states, so if we want each state to

be equally represented, 1/|Q| of all transition rules should result in each state.

There are |Q||N| rules in each ruleset, so n, the number of rules resulting in sq, must be

n = 1/|Q| * |Q||N| = |Q||N|-1

So, λ = (|Q||N|-n)/|Q||N| = (|Q||N| - |Q||N|-1|)/|Q||N|

= 1 – |Q||N|-1|/|Q||N| = 1 – 1/|Q| The other transitions are equally distributed

because we required they be chosen randomly and uniformly.

Searching with λ

λ “blurs” the space of rulesets, washing out small differences, and is used as a “low-resolution” survey to identify interesting areas of the rule space.

The authors search the rulespace by stepping through the range of λ in discrete steps and examining the behavior of the CA system to find a relationship between λ and behavior.

Additional Restrictions Before beginning the search, however, the authors

add a few more restrictions on rulesets in the hope of obtaining more meaningful results: Quiescence – When all cells in the neighborhood are

in the quiescent state sq, the rule should map to sq. Strong queiescence – When all cells in neighborhood

are in a given state si, the rule should map to si. (Note that many systems like the Game of Life do not follow this restriction)

Spatial Isotropy – All planar rotations of a neighborhood will map to the same state.

I know we all want to get on with it and see what the search found. But first, a bit of a digression to try and explain what we are actually searching for…

Brief Detour: CAs and Dynamical Systems

Dynamical Systems are complex systems with variables that change over time and for which we cannot find formal mathematical solutions.

Rather than attempting to find exact solutions or statistical approximations, Dynamical Systems researchers try to analyze, categorize, and describe the geometric and topological structure of solutions.

Phase Space

The study of a dynamical system frequently involves the analysis of the system’s “phase space,” the space of all possible values that the system’s variables can take on.

The state points taken over time form a trajectory of the system.

It is this trajectory which is analyzed for its geometric and topologic properties.

Behavior of Dynamical Systems

There have been three common behaviors identified in many dynamical systems: (1) Fixed Point Attractive – the system’s behavior

stabilizes to a single point in state space. The system is said to tend towards a “fixed attractor”.

(2) Periodic Attractive – the system’s behavior stabilizes to a closed, repeating path through state space. It is said to tend towards a periodic attractor.

Behavior of Dynamical Systems

(3) Chaotic Attractive –Behavior never really stabilizes, but it does seem to track to a “bounded manifold” (a multidimensional surface in phase space) with a complex structure. Similar start states do NOT produce similar long term trajectories. Said to tend towards a “strange attractor”.

Dynamical Systems and CAs How can we relate this back to Cellular Auomata? Wolfram (1984) identified four main types of CAs

Class I: From almost any initial state, the CA degrades to a homogeneous state in a finite amount of time.

Class II: The CA evolves into simple periodic structures. (the crystalline growth the Prof. Simha mentioned).

Class III: The CA tends towards aperiodic patterns. After many stems, the systems become statistically indistinguishable (maximal disorder).

Class IV: The CA produces stable, periodic and propagating structures which can last for a long time. Final states with any cycle length can be obtained with the right initial structure. A great deal of local order. Such CAs exhibit “very long transient lengths, having no direct analogue in the field of dynamical systems”.

Wolfram’s Classes of CAs

Transient Behavior

We can examine a dynamical system’s behavior by putting it in a “non-typical” state and watching the resulting behavior as it moves towards its “attractor”.

The time period between the initial state and the settled state is known as the “transient behavior”.

The authors are interested in how long the transient period is and what its relationship is to the size of the system (for a CA, the number of cells).

Transient Behavior

For class I, II, and III systems, there is little relationship between transient length and system size.

For class IV systems (and for dynamical systems which are “in-between” periodic and chaotic), the transient length seems to show a strong dependence on the size of the system. Sometimes the transient period can become more-or-less infinite.

Brief Detour 2: CAs and Computation

Christopher Langton: “Cellular Automata can be viewed as (1) computers themselves or as (2) logical universes within which computers may be embedded.”

In case (1), the starting configuration of the CA is the data to be processed and the state transition function represents the algorithm being computed on that data.

In case (2), the starting configuration is a computer, and input data, and algorithm, and the transition function is the “physics” under which the computer operates.

It has been shown that some CAs (such as the game of Life) are capable of performing any algorithm, i.e. they are universal general-purpose computers.

CAs and Computation It is believed that a CA must have certain properties to

be capable of universal general computation: It must support the “storage” of information – local

regions of state information which can be preserved for long periods of time.

It must support the “transmission” of information – the ability for small regions of state information to propagate over long distances.

The stored/transmitted information must be able to interact with/modify each other.

There seems to be a relationship between Wolfram’s Class IV CAs and computational capability.

Finally – Searching the λ Space

Langton and Avnet each performed a search using circular one dimensional CAs with 128 cells, with a neighborhood size of 5 (the cell being updated and its two neighbors on either side). The CAs were randomly chosen for specific λ values.

It really was an experiment, similar to experiments in biology or chemistry, where initial parameters are altered and the outcome is observed.

Results λ = 0.0 – After the first step, the entire array is in sq

λ = 0.15 – Decay to sq takes 4 to 5 steps λ = 0.2 – Permanent periodic structures can be produced,

transient time is 7-10 steps λ = 0.25 – Cells can get stuck in a single (non-sq) state λ = 0.4 – Periodic structures exist with periods up to 40 steps,

transient time is up to 60 steps before system collapses to isolated areas of periodic activity

λ = 0.45 – Transient length nearly 1000 steps – near balance between collapsing and expanding activity – propagating structures with up to 15000 steps are possible

λ = 0.5 – Transient Time 12000 steps – Long transients leading to sudden break-down

λ = 0.55 – “Transient” activity is now the long-term behavior, system now tends towards chaotic

λ = 0.65 – System becomes chaotic within 10 steps λ = 0.75 – Transient time is decreasing. System becomes

chaotic in about 1 step (quick decay to a strange attractor).

Sample Results

Analysis

The authors categorize the behavior into four regimes:

λ0.0 1.0

Fixed Regime

Periodic Regime

Transition Regime

Chaotic Regime

0.5

Analysis

The authors note that they observed all 4 classes of CA behavior by varying λ, which they suggest means λ is a good choice of parameter.

The transition regime tends to support propagating structures (like the “gliders” from the Game of Life). Langton observed that propagating structures that collide with static structures can produce a new structure that propagates in the opposite direction.

This sort of interaction would seem to allow for the storage/transmission/modification of information needed for a CA to perform general purpose computation.

λ versus Complexity

Phase Transition

In some experiments, the transition regime seems to be just a sharp dividing line between periodic and chaotic regimes.

Other times, it is more of a smooth transition range – a phase transition between “degrading to periodic” and “degrading to chaotic” behavior.

Analogies We can make an analogy between the behavior of CAs and

fundamental attributes of computation. When λ is near the critical value, either chaotic or periodic

outcomes are possible. The transient times are very long (effectively infinite, the

authors claim), making the question of which outcome will occur essentially undecidable.

This is reminiscent of the undecideability of the Halting Problem in computer theory. Some turing machines can be determined to either halt or not halt, but for some systems the problem is undecidable.

Could it be that these problems are near some sort of phase transition?

CAs explain Computation Langton goes so far as to suggest that many aspects of

general computational theory can be explained by phase transitions: Computability classes (computable, not computable) can be

explained by the transition between ordered/disordered regimes.

Undecidability is explained by transient times approaching infinity at the phase transition.

Complexity classses (polynomial, exponential, etc), can be explained by the increase in transient length at the phase transition.

Universal computation itself is explained as the “susceptability” of the system near the phase transition. This means, roughly, the responsiveness of the system to small changes in state. (Does this mean that λ(a rock) = 0.0, λ(the ocean) = 1.0, and λ (a computer) = pc (the critical point of phase transition)??)

Wow, Neat! …. so what’s next? The authors believe that this is a profoundly new way

of understanding computing that will have many applications: If phase transitions and information processing are

closely related, then maybe we can explain confusing aspects of dynamical systems through the theory of computation.

There is an analogy between the phases of matter (solid/fluid) and the phases of CAs (static/chaotic) and the phase transitions between them. Could it be that the phases of matter are just another dynamical system with a phase transition? Then it could be possible to reproduce the world of matter in the world of computers.

The Authors Conclude

The phase transition manifested in the CA state space gives us a new tool to explore and explain the concept of computation.

Additionally, it may help explain the complex behavior of dynamical systems near phase transitions.

Case closed.

NOT SO FAST!!!

M.Mitchell, J.P.Crutchfield, and P.T.Hraber. Dynamics, computation, and the ``edge of chaos'': A re-examination. In G. Cowan, D. Pines, and D. Melzner (editors), Complexity: Metaphors, Models, and Reality. Reading, MA: Addison-Wesley, 1994.

Another View

The authors of this paper believe that Langton and the other proponents of the λ parameter have applied questionable assumptions: First, they question whether λ, and the rule tables it is

derived from, are actually the drivers of dynamical behavior. In dynamical systems theory, it is assumed that functions on the equations of motion (the equivalent of transition rules) are inadequate to describe the behavior of the system.

Another View

Second, they question the assumption that λ, and the other parameters Langton analyzed, like transient time and entropy, actually converge. They say this is not always true.

Third, and most strongly, they wonder why even assume that the statistics Langton describes are really the only measures of complexity and computational potential?

The authors support their argument by critical analysis of another CA experiment performed by Packard (1984).

Packard’s Experiment

Packard began with a rule set called GKL for a one dimensional CA with a neighborhood size of 3.

When using the GKL rule, an input state with less than half ones would trend towards a final state of all zeroes. When used with an input state of more than half ones, it would trend towards a final state of all ones.

The algorithm is not correct in all cases, but it is very good (about 98% correct).

Packard’s Experiment

The problem is not actually as easy to solve as it sounds for a 2 state, 3 neighbor CA. Essentially, it corresponds to the recognition of a non-regular language, and it requires transmission of information across long distances.

Packard used a genetic algorithm to try and improve the rule.

Digression Can this problem (determining majority 1’s or

majority 0’s) ever by solved perfectly by a CA? The papers don’t really address this question. Thought 1: A CA is a set of DFAs put together, which

means it is still just a big finite automaton. It can never really distinguish an infinite, non-regular language.

Thought 2: True, but with this kind of DFA we add many more states as the input size grows. We know that some CAs can perform any algorithm, so maybe the real question is can a 1-dimensional, 3-neighbor CA solve the problem, and if so, can it be done with only as many cells as there are input bits? (If not, how much additional “memory” is required?)

Packard’s Prep

Before performing his experiment, Packard wanted to find the “critical λ points” for this kind of CA (the points of phase transition).

He did this by statistically analyzing the behavior of γ , the “difference-pattern spreading rate”, a measure of chaotic behavior, as a function of λ.

The Critical Points

Packard identified critical points at about λ=0.25 and λ=0.8

The Genetic Algorithm

Packard wanted to show that the best rulesets for solving the problem would be found near the phase transition points, and also that genetic algorithms would evolve algorithms towards those points.

The genetic algorithm started with 100 randomized CAs, then picked the 50 that were best at solving the problem, discarded the rest, and generated fifty new ones by mutating the first 50. Each repetition of that process is a generation.

Packard’s Result

Sure enough, in the end, the algorithms evolved primarily towards the critical points.

Packard’s Result

The results seemed to support Packard’s hypothesis that the best algorithms for solving the problem would be at or near the phase transition points.

The Authors’ Experiment The authors attempted to replicate Packard’s

experiment with minor differences. Their results, however, were quite different:

The Authors Explain The authors have several explanations for why their results

differed from Packard’s. First Explanation: the authors also had two “towers” that the

algorithms tended to converge to, like Packard, but they were much closer to λ=0.5 than Packard’s results, and they were not near the critical points. The authors believe this is because an optimal solution to the “more zeroes or more ones” problem MUST be close to λ=0.5. (the initial GKL rule has λ=0.5)

They do not provide a proof in this paper, but they state intuitively that, for any CA with λ below 0.5, there must be some rules which decrease the number of ones in a neighborhood. Thus, there must be some initial conditions of the CA, with majority 1’s, for which the CA will decrease the number of 1’s rather than increasing it, which is the desired behavior. As λ gets further away from 0.5, the number of such configurations must increase, reducing the effectiveness of the algorithm. The reverse applies for majority 0.

The Authors Explain

Also, the authors say that the “combinatorial drift” force in genetic algorithms, where random actions of mutation tend to push the algorithms towards λ=0.5.

The authors explain the dip at λ=0.5 as a weakness of the genetic algorithm, which seemed to pick up and amplify one of two strategies (expanding clusters of 1’s or expanding clusters of 0’s) rather than using both of them as the GKL rule does.

The Authors Explain

The authors do not claim to know exactly why Packard’s results came about the way they did, but they suspect that there was some additional randomization or other aspect of Packard’s procedure that has not become public.

The Implications The authors conclude that λ may not be a good indicator of

computational ability. First, they say that Packard’s experiment was the only strong link between λ and computational capability (aside from the fact that the game of life is capable of universal computation, and it has λ at the critical point), and they call his result into question.

Since they believe they can show that a good solution to the problem requires λ be close to 0.5, they say there is no reason to believe there is any generic relationship between λ, the critical points, and the computational capability of a CA.

The authors propose to find new ways of analyzing the computational potential of a CA, that do not rely on “rough”, statistical parameters like λ.

Conclusion

Between the two papers, the question of the relationship between dynamic systems, phase transitions, CAs, and computation is uncertain, but intriguing.

Fortunately, group A2 will sort this all out.