Computational Methods for Nonlinear Systems

!

Cornell Physics 7682 Fall 2015

Chris Myers c.myers@cornell.edu

Physical Sciences Building 424

Computational Methods for Nonlinear Systems

• Graduate computational science laboratory course developed by Myers, Sethna & Mueller, starting in 2004 - developed originally to support interdisciplinary

IGERT program on Nonlinear Systems - class work focused on implementation of

computer programs from papers, hints and skeletal code

Computational Methods for Nonlinear Systems

• Hands-on introduction to computational science & scientific computing: modeling, algorithms, data structures, programming, simulation techniques, data analysis, etc. - a broad variety of scientific themes: complex networks,

biomechanics, pattern formation, chemical kinetics, statistical mechanics, molecular dynamics, chaos, random matrix theory, constraint satisfaction, infectious diseases, etc.

- a broad variety of numerical methods: ODEs, PDEs, graph theory, Monte Carlo, data analysis, fitting, root finding, etc.

- many exercises incorporated into Sethna’s textbook, “Statistical Mechanics: Entropy, Order Parameters, and Complexity”

- complementary to Physics 4480 / 7680 (“Computational Physics with Numerical Recipes”) which focuses more specifically on algorithms and numerical analysis

Course structure

• Primary focus is working on course modules in class

• Periodic lectures on science and/or computation to complement work on course modules

• Self-paced and autonomous: work on those modules you are most interested in, or propose other topics of interest you would like to explore in more detail

• Can work in groups if desired, although be sure to take turns driving

• Students can use PECF workstations in Rockefeller B3 (computing.physics.cornell.edu), or bring own laptop for use in class

Course structure

• No homework assignments — no graded assignments; reading background material outside of class is useful

• Attendance, engagement and participation are paramount

• Ask questions if you are confused, need help debugging, or want to discuss material

• Students will be expected to deliver at least one short lecture (~15 minutes) on a subject of interest, and to participate in other initiatives that might unfold (e.g., “Bring your problems to class day”)

• Opportunities to bring relevant projects of interest to class (e.g., related to research), or to work with me to implement some new modules for the course - random graphs, forest fires, flocking, parameter estimation,

sloppiness, zombies, mosh pits, etc.

Course structure

• Recommended to take S/U

• If a letter grade is needed for a particular program, we will conduct oral examination / review of work done at end of semester (can be based in part on extra project)

• Flexible within reason: can accommodate absences due to occasional conflicts (e.g., special lab meeting, travel to a conference); can make up missed time outside of class - talk to instructor if conflicts arise

Programming

• all programming done in Python

• 3rd party Python libraries for numerics, graphics, visualization, etc.

• graphical tools provided to give feedback, help debug, etc.

• why Python? - interpreted: rapid program development and interactive access - built-in data structures & high-level syntax - rich standard library and 3rd party libraries (for science,

visualization, databases, internet programming, etc.) - supports procedural programming, object-oriented programming,

functional programming, scripting, large systems - free software (monetarily and intellectually) - good job skill

Goals for semester

• Yours? - get exposed to a broad set of interesting scientific and

computational problems and methods - learn how to integrate a variety of tools to solve problems - learn how to conduct research that integrates computation with

theory, data, analysis, etc.

• Mine - teach you all of the above, and learn from all of you - convey how to think computationally: abstraction, interface &

information - add new course modules (suggestions are welcome!) - new features/modules to be rolled out over the course of the

semester (and you are welcome to help if you are interested in such things)

- continue to improve the course website and workflow

Scientific topics

Networks

Dynamics & Dynamical Systems

Biological dynamics & patterns

Statistical mechanics

etc.

Course modules

Small-world networks

Percolation

Pendulum

Walker

Invariant measure

Chaos & Lyapunov

Fractal dimensions

Period doubling

Cardiac dynamics

Random matrix theory

Stochastic cells

Repressilator

NP-completeness

Ising model

Molecular dynamics

Infectious diseases

Small-world networks

• Science - small world networks (Watts & Strogatz) - “six degrees of separation” - shortest path lengths in randomly wired

graphs

• Computing - data structure for undirected graphs

‣ good introduction to built-in Python containers (lists and dictionaries)

- object-oriented encapsulation of complex data structures

- graph traversal algorithms (breadth-first search) for shortest path and betweenness

- simple graph visualization software supports debugging and provides quick feedback

Percolation

• Science - statistical mechanics of percolation

‣ connected clusters in randomly wired graphs (e.g., bond percolation on a lattice)

‣ universality of phase transitions

• Computing - reuse of objects with generic interface

(reuse of UndirectedGraph class from small-world network module)

- graph traversal algorithms (breadth-first search) for cluster finding

- scaling collapses

Walker

• Science - Simple model of bipedal walker (Ruina and

coworkers)

‣ double pendulum with impulse for heelstrike

‣ single pendulum as warmup - period-doubling bifurcations in physical

system

• Computing - integration of ODEs

‣ finite differences, time-step dependence, integration schemes (stability, fidelity, accuracy)

‣ calling 3rd party numerical libraries - change of integration variables for event

detection (heelstrike) - tracking unstable periodic orbit - visualization tools for animation

Maps & dynamical systems

• Science - bifurcations and chaos in iterated maps - period doubling in logistic map - density (invariant measure) in chaotic

regime - Lyapunov exponents: divergence of

nearby trajectories - fractal dimensions of attractors - renormalization group of logistic map &

universality of period doubling route to chaos

• Computing - iterating maps - root-finding - fitting

Cardiac dynamics

• Science - pattern formation in excitable medium - FitzHugh-Nagumo model (type of

reaction-diffusion equation) - spiral defects forming in electrical

pulsing can lead to cardiac arrythmias - model extensions to simulate dead

tissue, cardiac chambers, etc.

• Computing - numerical solution of PDEs

‣ finite-differences, operator stencils - nullcline analysis for single cell (root-

finding) - simple animation tool allows interactive

steering of simulation (needed to provide targeted electrical pulses and “defribrillator” shock)

Cell dynamics: stochastic cells and Repressilator

• Science - Repressilator (Elowitz & Leibler): genetic

oscillator from realm of synthetic biology

‣ oscillatory mRNA/protein dynamics from mutually repressing proteins

- telegraph noise and shotgun noise in stochastic systems

• Computing - chemical kinetics and reaction networks

‣ Petri nets

‣ synthesis of aggregate kinetic equations from network

- Monte Carlo algorithms (Gillespie) - stochastic vs. deterministic descriptions

• Additional projects exploring other gene regulatory networks (switches, feed-forward loops, etc.)

NP-completeness & constraint satisfaction

• Science - NP-complete problems - phase transitions in 3SAT and

parametric complexity - integer partitioning problem

• Computing - algorithms for NP-complete

problems - backtracking, recursion

Random matrix theory

• Science - eigenvalue spacings in random

matrices - developed originally to describe

energy level spacings in quantum systems

• Computing - random matrix generation - eigenvalue computation (numpy)

Molecular dynamics

• Science - thermodynamics: emergence of

effective properties from molecular chaos

• Computing - design of large software systems - geometric data structures - integration of ODEs - thermodynamics

‣ pressure

‣ pair distribution functions

Ising model

• Science - phase transitions in simple

model of magnetic system

‣ nucleation

‣ self-similarity

‣ fluctuation-dissipation

• Computing - Monte Carlo algorithms

‣ heat bath

‣ Metropolis

‣ cluster flipping

Randomness: random walks & Gumbel distributions

• Science - ensembles and distributions

‣ averages

‣ extremal behavior

‣ emergent symmetries

• Computing - random number generation - statistical analysis of ensembles

The Python for Science Ecosystem

• SciPy and NumPy [www.scipy.org] - NumPy: provides (in a compiled library) high-level array syntax,

linear algebra, random number generation, Fourier transforms - Python interfaces to well-tested compiled numerical routines,

interfaced to work with NumPy arrays - provides routines for integration of functions and differential

equations, root-finding, minimization, etc.

• matplotlib (a.k.a. pylab) [matplotlib.org] - provides 2D (x-y) plotting, histograms, etc. (modeled after

MATLAB plotting)

• ipython [ipython.org] - enhancement of the standard python interpreter: command

completion & history, “magic” functions, etc. - notebook mode for integrated programs, documentation, results

The Python for Science Ecosystem (continued)

• Python standard library: pickle, time, os, collections, multiprocessing, string, url lib, xml, json, etc.

• Python Imaging Library (PIL, Pillow): image processing and analysis

• scikit-learn: machine learning

• NetworkX: graph theory and network analysis

• sympy: symbolic mathematics

• Pandas: data analysis

• mpmath: multiple precision mathematics

• h5py: HDF5 interface (btw, also interface to MATLAB .mat files)

• SWIG, Cython, weave, theano, …: tools for integrating compiled code with interpreted Python code

• etc.

Course mechanics & information

• Course web page - www.physics.cornell.edu/~myers/teaching/ComputationalMethods

• Module web pages - www.physics.cornell.edu/~myers/teaching/ComputationalMethods/

ComputerExercises - linked from “Computer Exercises” on course web page

• Navigation bar on course web page - News & Updates: information about new modules, bug fixes,

special projects, etc. - Lecture Notes: copies of in-class lecture slides - Python Resources: tutorials, documentation, guides, etc.

Course mechanics: working with modules

• For a given module: - possibly some background reading (e.g., journal paper) - text describing an exercise (pdf or on web page) — read this

first! - a “Hints” file (e.g., SmallWorldHints.py) ‣ Hints files have skeletal Python code with comments

(docstrings) ‣ Hints files have minimal Python code body consisting of

keyword pass

‣ Copy “Hints” file to new file (e.g., SmallWorld.py) and then fill in code body to implement desired functionality

‣ %run Python code file within IPython notebook to test code, run analyses, make plots, document work, etc.

Python installations

• Encourage use of Anaconda Scientific Python distribution (unless you are already a seasoned Python user with your own setup) !

• If using your own laptop: • Anaconda distribution (free): download & install Python 3.4

- https://store.continuum.io/cshop/anaconda/ !

• Physics Educational Computing Facility (PECF) - python, ipython, scipy, etc. all installed - add ~myers/anaconda3/bin to beginning of shell path - let me know if packages are missing or broken !

• Text editors have auxiliary python editing mode: syntax highlighting, indenting, etc. (e.g., python-mode in emacs)

Development Environments, Workflows & Interpreters

• python interpreter (“python”) - bare bones interpreter of Python code - useful to run programs in background/batch once fully developed

• ipython interpreter (“ipython” or “ipython qtconsole”) - enhanced Python interpreter with help, autocompletion, editing,

etc. - extra functionality (“magics”) outside of Python language - magics prepended by %

- %who : list of what variables, functions, etc. defined in namespace

- %timeit : provide timing information of specific command

• ipython notebooks (“ipython notebook”) - graphical interface integrating code, documentation, analyses,

plots, etc. (a la Mathematica notebooks)

Some new twists this year

• Python 3 - core numerical code for all the modules has been updated - some of the visualization tools will need to be updated

and/or replaced (anyone interested in helping?)

• Encouraging use of Anaconda python v. 3.4 by everyone

• IPython notebooks to organize and summarize analyses and results