Practical Numerical Algorithms for Chaotic Systems

Thomas S. Parker Leon o. Chua

Practical Numerical Algorithms for Chaotic Systems

With 152 Illustrations

Springer-Verlag New York Berlin Heidelberg London Paris Tokyo Hong Kong

Thomas S. Parker Hewlett Packard Santa Rosa, CA 95401, USA

Leon O. Chua Department of Electrical Engineering

and Computer Sciences University of California, Berkeley, CA 94720, USA

Front cover: The photograph on the cover shows a strange attractor internationally known as the "double scroll from Chua's circuit."

Library of Congress Cataloging-in-Publication Data Parker, Thomas S.

Practical numerical algorithms for chaotic systems 1 Thomas S. Parker, Leon O. Chua.

p. cm.

ISBN-J3: 978-1-4612-8121-4 001: 10.1007/978-1-4612-3486-9

e-ISBN-J3: 978-1-4612-3486-9

1. Chaotic behavior in systems. 2. Algorithms. 3. Nonlinear theories. I. Chua, Leon 0., 1936- . II. Title.

QI72.5.C45P37 1989 003-dc20 89- 11307

Printed on acid-free paper.

© 1989 by Springer-Verlag New York Inc.

Softcover reprint of the hardcover 1 st edition 1989

All rights reserved. This work may not be translated or copied in whole or in part without the written permission of the publisher (Springer-Verlag, 175 Fifth Avenue, New York, NY 10010, USA), except for brief excerpts in connection with reviews or scholarly analysis. Use in connection with any form of information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed is forbidden. The use of general descriptive names, trade names, trademarks, etc. in this publication, even if the former are not especially identified, is not to be taken as a sign that such names, as understood by the Trade Marks and Merchandise Marks Act, may accordingly be used freely by anyone.

Camera-ready copy prepared by the authors using LATEX

9 8 7 6 5 4 3 2 1

To Don and Betty for their love and support over the years.

T.S. Parker

To Diana.

L.O. Chua

Preface

One of the basic tenets of science is that deterministic systems are completely predictable-given the initial condition and the equations describing a system, the behavior of the system can be predicted for all time.1 The discovery of chaotic systems has eliminated this viewpoint. Simply put, a chaotic system is a deterministic system that exhibits random behavior.

Though identified as a robust phenomenon only twenty years ago, chaos has almost certainly been encountered by scientists and engi­neers many times during the last century only to be dismissed as physical noise. Chaos is such a wide-spread phenomenon that it has now been reported in virtually every scientific discipline: astronomy, biology, biophysics, chemistry, engineering, geology, mathematics, medicine, meteorology, plasmas, physics, and even the social sci­ences.

It is no coincidence that during the same two decades in which chaos has grown into an independent field of research, computers have permeated society. It is, in fact, the wide availability of inex­pensive computing power that has spurred much of the research in chaotic dynamics. The reason is simple: the computer can calculate a solution of a nonlinear system. This is no small feat. Unlike lin­ear systems, where closed-form solutions can be written in terms of the system's eigenvalues and eigenvectors, few nonlinear systems and virtually no chaotic systems possess closed-form solutions.

The computer allows nonlinear systems to be studied in a way that was undreamt of by the pioneers of nonlinear dynamics. One can perform numerical "experiments" quickly and easily. Parameters can be changed, system equations modified, and solutions displayed-all at the touch of a key. Before the advent of low-cost computing, the

lThere are random events in nature, but science has traditionally used prob­abilistic models to describe them.

ability to perform such simulations was restricted to researchers with access to a large computing facility. Today, anyone with a personal computer can simulate a nonlinear system.

Simulations are a powerful tool for gaining intuition about non­linear systems and for exploring the exciting terrain of chaotic dy­namics, but they do have their limitations. Computers have finite precision and inevitably generate errors when evaluating floating­point expressions. Furthermore, computers are discrete-time in na­ture and there are unavoidable errors when they are used to simulate continuous-time systems. Finally, a simulation is of little or no help in proving theoretical results; even if the result of a simulation were completely accurate, it is just one solution of one system from one initial condition using one set of parameter values.

The moral is: though simulations are a useful tool, simulation data must be interpreted carefully, checked against intuition and theory, and used only for purposes for which it is suited.

The book The purpose of this book is to present robust, reliable algorithms for simulating nonlinear dynamics, with an emphasis on chaotic behavior. In addition, we present the theoretical underpin­nings of the algorithms. This allows the reader to understand in which situations an algorithm is applicable and how to interpret its results.

Mathematics is used throughout the book to describe the solu­tions and properties of dynamical systems, but we rarely state the­orems and even less often prove them. The few proofs that are in­cluded in the book are relegated to the appendices. No prior knowl­edge of nonlinear systems is assumed though we do assume that the reader is familiar with the basic concepts from linear systems­equilibrium points, eigenvalues, eigenvectors, sinusoidal steady state, stability. Much of the language of dynamical systems theory is couched in the terms of differential topology, so we have included an appendix that reviews the concepts of diffeomorphisms, manifolds, and transversality.

We explain the basic theory behind chaotic systems and present algorithms for simulating and characterizing chaotic systems. Thus, most of the chapters contain two sections, the first section introduces and explains some aspect of dynamical systems theory and the second section presents algorithms that implement the ideas from the first section.

viii

Pseudo-code It has been our experience that a verbal description of an algorithm often makes complete sense until one actually sits down to implement it. To alleviate this problem, we give detailed pseudo-code for the algorithms.

There are a few points to remember while reading the pseudo­code. Local variables are not explicitly declared. Thus, any variable that is not global and not an argument to the function in which it occurs is a local variable defined only in the function in which it occurs. The type of a variable should be clear from its usage and the context. Vectors are indicated by brackets. For example, x[] is an array and a[][] is a matrix. The ith row of a[][] is a[ i][] and the jth column is a[] [j].

We have also included a chapter devoted to programming tech­niques and style in which we share the experience we have gained from writing simulation programs.

INSITE Most of the algorithms presented in this book are avail­able in the software package INSITE.2 INSITE is a collection of inter­active, graphically-oriented programs for the simulation and investi­gation of nonlinear systems, with an emphasis on chaotic systems. INSITE is written in C and runs under the UNIX and PC-DOS op­erating systems. For information on ordering INSITE, please write

INSITE P.O. Box 9662 Berkeley, CA 94709-9662.

Acknowledgments We would like to thank Peter Kennedy for his thorough and thoughtful reading of .the manuscript. The clear, concise portions of this book owe much to him. We also deeply ap­preciate the comments and suggestions on the manuscript provided by Dr. Alistair Mees. Thanks are gratefully extended to Dr. Greg­ory Bernstein for his comments on Chapter 4 and to Dr. Kenneth Kundert for his comments on Chapter 5. Special thanks are due Dr. Stephen Boyd for suggesting that the book be written.

We would also like to thank Dr. Klaus Kiihlert, Dr. Rene Lozi, Professor Gui-nian Lin, and OrIa Feely for their careful reading of the final manuscript. We gratefully acknowledge the Office of Naval Re-

2Interactive Nonlinear Systems Investigative Toolkit for Everyone

ix

search and the National Science Foundation, without whose research support this book would not have been possible.

T. Parker would like to thank Bill's Bed and Breakfast for pro­viding accommodations during the final phases of writing. He would also like to thank Bruce Donecker and David Sharrit of Hewlett­Packard for their understanding and patience while he was working on this project. Finally, he would like to thank Nicholas, Ginger, and Luther for their constant support and understanding.

The book was typeset by T. Parker using the J.j\'IEX document preparation system written by Leslie Lamport and Donald Knuth.

x

Contents

1 Steady-State Solutions 1 1.1 Systems............................. 1

1.1.1 Autonomous continuous-time dynamical systems .. 1 1.1.2 1.1.3

1.1.4

Non-autonomous continuous-time dynamical systems Relationship between autonomous and non-autono-mous systems . . . . . . . . . . . . . . . . . . . . . . Useful facts regarding continuous-time dynamical sys-tems ......... .

1.1.5 Discrete-time systems 1.2 Limit sets . . . . . . . . .

1.2.1 1.2.2 1.2.3 1.2.4

Equilibrium points .. Periodic solutions ... Quasi-periodic solutions Chaos ..... .

1.2.5 Predictive power 1.3 Summary ....... .

2 Poincare Maps 2.1 Definitions.

2

2

3 4 4 6

8 13 18 24 28

31 31

2.1.1 The Poincare map for non-autonomous systems. 31 2.1.2 The Poincare map for autonomous systems 32

2.2 Limit Sets . . . . . . . . . 35 2.2.1 Equilibrium points . . . 35 2.2.2 Periodic solutions. . . . 36 2.2.3 2.2.4

Quasi-periodic solutions Chaos ....... .

2.3 Higher-order Poincare maps . . 2.4 Algorithms .......... .

2.4.1 Choosing the hyperplane E 2.4.2 Locating hyperplane crossings.

2.5 Summary . . . . . . . . . . . . . . . .

37

40

43 47 47 48

55

3 Stability 57 3.1 Eigenvalues . . . . . . . . . . . . 59 3.2 Characteristic multipliers . . . . 61

3.2.1 Characteristic multipliers 61 3.2.2 Characteristic multipliers and the

variational equation ...... 64 3.2.3 Characteristic multipliers and

equilibrium points 66 3.3 Lyapunov exponents . . . . . . . . . . 66

3.3.1 Definition............ 67 3.3.2 Lyapunov exponents of an equilibrium point. 67 3.3.3 Lyapunov numbers of a fixed point . . . . . . 68 3.3.4 Perturbation subspaces ............ 69 3.3.5 Lyapunov exponents of non-chaotic limit sets 70 3.3.6 Lyapunov exponents of chaotic attractors 71

3.4 Algorithms .................. 71 3.4.1 Eigenvalues at an equilibrium point 71 3.4.2 Characteristic multipliers 73 3.4.3 Lyapunov exponents 73

3.5 Summary . . . . . . . . . . . . . 81

4 Integration 83 4.1 Types ....... 83 4.2 Integration error .. 92

4.2.1 Local errors . 92 4.2.2 Global errors 95 4.2.3 Numerical stability. 97

4.3 Stiff equations. . . . . . . . 99 4.4 Practical considerations . . 104

4.4.1 Variable step-size and order 105 4.4.2 Output points ...... 106 4.4.3 Solving implicit equations . 107 4.4.4 Error considerations . . . . 109 4.4.5 Integrating chaotic systems 110 4.4.6 Start-up costs. 112

4.5 Summary . . . . . . . . . . . . . . 112

5 Locating Limit Sets 115 5.1 Introduction............. 115

5.1.1 Brute-force approach. . . . 115 5.1.2 Newton-Raphson approach 116

5.2 Equilibrium points 116 5.3 Fixed points. . . . 117 5.4 Closed orbits . . . 118 5.5 Periodic solutions. 118

5.5.1 The non-autonomous case 119

xii

5.5.2 The autonomous case 5.6 Two-periodic solutions .

5.6.1 Finite differences 5.6.2 Spectral balance

5.7 Chaotic solutions 5.8 Summary

6 Manifolds

7

6.1 Definitions and theory ..... . 6.1.1 Continuous-time systems 6.1.2 Discrete-time systems ..

6.2 Algorithms ........... . 6.2.1 Continuous-time systems 6.2.2 Discrete-time systems

6.3 Summary . . . . . . . . . . .

Dimension 7.1 Dimension.

7.1.1 Definitions 7.1.2 Algorithms

7.2 Reconstruction 7.3 Summary .....

8 Bifurcation Diagrams 8.1 Definitions .... . 8.2 Algorithms .... .

8.3

8.2.1 Brute force . 8.2.2 Continuation Summary.

9 Programming 9.1 The user interface ............. .

9.1.1 The dynamical system interface .. . 9.1.2 The program initialization interface 9.1.3 The interactive interface.

9.2 Languages ....... . 9.2.1 Modular design ..... .

9.3 Library definitions . . . . . . . . 9.3.1 RKF-Runge-Kutta-Fehlberg integration 9.3.2 PARSE-input parsing routines 9.3.3 BIN FILE-binary data files . 9.3.4 GRAF-graphics

xiii

120 131 131 134 136 137

139 139 139 145 150

150 154 165

167 167 168 179 191

197

201 201

210 210 222 234

237 237 238 242

243 247 249 252 253

257 260 261

10 Phase Portraits 10.1 Trajectories . . . . . . . . . . . . . . . . . . .

10.1.1 Selection of initial conditions 10.1.2 Calculating the trajectory 10.1.3 Arrowheads ...... . 10.1.4 Drawing the vector field

10.2 Limit sets . . . . . . . . . 10.2.1 Equilibrium points . 10.2.2 Limit cycles . 10.2.3 Index ...

269 270 270 272 274 276 277 278 279 280

10.3 Basins . . . . . . . . 281 10.3.1 Definitions . 281 10.3.2 Examples . . . . . . . . . 284 10.3.3 Calculating boundaries of basins of attraction. 291

10.4 Programming tips ....... 296 10.4.1 Consistency checking. . 296 10.4.2 History files . . 297

10.5 Summary . . . . . . . . . . . . 299

A The Newton-Raphson Algorithm 301

B The Variational Equation 305

C Differential Topology 307 C.1 Differential topology 307 C.2 Structural stability 310

D The Poincare Map 315

E One Lyapunov Exponent Vanishes 321

F Cantor Sets 323

G List of Symbols 329

Bibliography 333

Index 341

xiv