Gary William Flake

The Computational Beauty of Nature

Computer Explorations of Fractals, Chaos,Complex Systems, and Adaptation

A Bradford BookThe MIT PressCambridge, MassachusettsLondon, England

Contents

Preface xiiiHow to Read This Book xivDealing with Difficult Subjects xvPersonal Motivation xviAcknowledgments xvi

1 Introduction 11.1 Simplicity and Complexity 21.2 The Convergence of the Sciences1.3 The Silicon Laboratory 6

I Computation 9

2 Number Systems and Infinity 112.1 Introduction to Number Properties 122.2 Counting Numbers 142.3 Rational Numbers 152.4 Irrational Numbers 162.5 Further Reading 22

3 Computability and Incomputability 233.1 Godelization 253.2 Models of Computation 263.3 Lisp and Stutter 303.4 Equivalence and Time Complexity 363.5 Universal Computation and Decision Problems 403.6 Incomputability 423.7 Number Sets Revisited 453.8 Further Reading 48

viii Contents

4 Postscript: Computation 514.1 Godel's Incompleteness Result 524.2 Incompleteness versus Incomputability 534.3 Discrete versus Continuous 554.4 Incomputability versus Computability 564.5 Further Reading 57

II Fractals 59

5 Self-Similarity and Fractal Geometry 615.1 The Cantor Set 625.2 The Koch Curve 655.3 The Peano Curve 665.4 Fractional Dimensions 675.5 Random Fractals in Nature and Brownian Motion 715.6 Further Exploration 755.7 Further Reading 76

6 L-Systems and Fractal Growth 776.1 Production Systems 786.2 Turtle Graphics 806.3 Further Exploration 816.4 Further Reading 92

7 Afflne Transformation Fractals 937.1 A Review of Linear Algebra 947.2 Composing Affine Linear Operations 967.3 The Multiple Reduction Copy Machine Algorithm 987.4 Iterated Functional Systems 1037.5 Further Exploration 1057.6 Further Reading 106

8 The Mandelbrot Set and Julia Sets 1118.1 Iterative Dynamical Systems 1128.2 Complex Numbers 1128.3 The Mandelbrot Set 1148.4 The M-Set and Computability 1188.5 The M-Set as the Master Julia Set 1208.6 Other Mysteries of the M-Set 1258.7 Further Exploration 125

Contents

8.8 Further Reading 127

Postscript: Fractals 1299.1 Algorithmic Regularity as Simplicity 1309.2 Stochastic Irregularity as Simplicity 1329.3 Effective Complexity 1349.4 Further Reading 136

III Chaos 137

10 Nonlinear Dynamics in Simple Maps 13910.1 The Logistic Map 14110.2 Stability and Instability 14410.3 Bifurcations and Universality 14810.4 Prediction, Layered Pastry, and Information Loss 15010.5 The Shadowing Lemma 15310.6 Characteristics of Chaos 15410.7 Further Exploration 15610.8 Further Reading 158

11 Strange Attractors 15911.1 The Henon Attractor 16011.2 A Brief Introduction to Calculus 16511.3 The Lorenz Attractor 16811.4 The Mackey-Glass System 17311.5 Further Exploration 17611.6 Further Reading 180

12 Producer-Consumer Dynamics 18112.1 Producer-Consumer Interactions 18212.2 Predator-Prey Systems 18312.3 Generalized Lotka-Volterra Systems 18612.4 Individual-Based Ecology 18712.5 Unifying Themes 19712.6 Further Exploration 19812.7 Further Reading 201

13 Controlling Chaos 20313.1 Taylor Expansions 20413.2 Vector Calculus 20513.3 Inner and Outer Vector Product 207

Contents

13.4 Eigenvectors, Eigenvalues, and Basis 20913.5 OGY Control 211

- 13.6 Controlling the Henon Map 21513.7 Further Exploration 21813.8 Further Reading 219

14 Postscript: Chaos 22114.1 Chaos and Randomness 22214.2 Randomness and Incomputability 22414.3 Incomputability and Chaos 22614.4 Further Reading 227

IV Complex Systems 229

15 Cellular Automata 23115.1 One-Dimensional CA 23215.2 Wolfram's CA Classification 23615.3 Langton's Lambda Parameter 24215.4 Conway's Game of Life 24515.5 Natural CA-like Phenomena 25115.6 Further Exploration 25515.7 Further Reading 258

16 Autonomous Agents and Self-Organization 26116.1 Termites 26216.2 Virtual Ants 26416.3 Flocks, Herds, and Schools 27016.4 Unifying Themes 27516.5 Further Exploration 27616.6 Further Reading 278

17 Competition and Cooperation 28117.1 Game Theory and Zero-Sum Games 28217.2 Nonzero-Sum Games and Dilemmas 28817.3 Iterated Prisoner's Dilemma 29317.4 Stable Strategies and Other Considerations 29517.5 Ecological and Spatial Worlds 29717.6 Final Thoughts 30317.7 Further Exploration 30317.8 Further Reading 304

Contents

18 Natural and Analog Computation 30718.1 Artificial Neural Networks, 30918.2 Associative Memory and Hebbian Learning 31218.3 Recalling Letters 31618.4 Hopfield" Networks and Cost Optimization 31818.5 Unifying Themes 32418.6 Further Exploration 32518.7 Further Reading 326

19 Postscript: Complex Systems 32719.1 Phase Transitions in Networks 32819.2 Phase Transitions in Computation 33219.3 Phase Transitions and Criticality 33419.4 Further Reading 336

V Adaptation 337

20 Genetics and Evolution 33920.1 Biological Adaptation 34020.2 Heredity as Motivation for Simulated Evolution 34220.3 Details of a Genetic Algorithm 34320.4 A Sampling of GA Encodings 34820.5 Schemata and Implicit Parallelism 35320.6 Other Evolutionary Inspirations 35520.7 Unifying Themes 35620.8 Further Exploration 35820.9 Further Reading 360

21 Classifier Systems 36121.1 Feedback and Control 36321.2 Production, Expert, and Classifier Systems 36421.3 The Zeroth Level Classifier System 37021.4 Experiments with ZCS 37321.5 Further Exploration 37921.6 Further Reading 380

22 Neural Networks and Learning 38322.1 Pattern Classification and the Perceptron 38522.2 Linear Inseparability 39022.3 Multilayer Perceptrons 392

Contents

22.4 Backpropagation 39322.5 Function Approximation, 39822.6 Internal Representations 40422.7 Other Applications 40922.8 Unifying Themes 41022.9 Further Exploration 41122.10 Further Reading 413

23 Postscript: Adaptation 41523.1 Models and Search Methods 41623.2 Search Methods and Environments 41923.3 Environments and Models 42223.4 Adaptation and Computation 42323.5 Further Reading 424

Epilogue 425

24 Duality and Dichotomy 42724.1 Web of Connections 42824.2 Interfaces to Hierarchies 429

24.3 Limitations on Knowledge 431

Source Code Notes 435

Glossary 443

Bibliography 469

Index 483