pit 05/01/06 physics of information technology norm margolus crystalline computation

PIT 05/01/06

Physics of Information Technology

Norm Margolus

Crystalline Computation

PIT 05/01/06

Spatial Computers» Architectures and algorithms for

large-scale spatial computation

Emulating Physics» Finite-state, locality, invertibility,

and conservation laws

Physical Worlds» Incorporating comp-universality at

small and large scales


see: http://people.csail.mit.edu/nhm/cc.pdf

http://people.csail.mit.edu/nhm/cc.pdf

PIT 05/01/06

Spatial Computers

• Finite-state computations on lattices are interesting

• Crystalline computers will one day have highest performance

• Today’s ordinary computers are terrible at these computations

PIT 05/01/06

CAMs 1 to 6• Tom Toffoli and I

wanted to see CA-TV• Stimulated by new

rev rules• Rule = a few TTL

chips!• Later, rule = SRAM

LUT• LGA’s needed more

“serious” machine!

PIT 05/01/06

Lattice example

//* Four direction lattice gas *//

2D-square-lattice {512 512}

create-bit-planes {N S E W}

define-step

shift-planes {(N,0,-1) (S,0,1)

(E,1,0) (W,-1,0)}

for-each-site

if {N==S && E==W}

{xchng(N,E); xchng(S,W)}

endif

end-site

end-step

PIT 05/01/06

Lattice example//* Four direction lattice gas *//

2D-square-lattice {512 512}

create-bit-planes {N S E W}

define-step

shift-planes {(N,0,-1) (S,0,1)

(E,1,0) (W,-1,0)}

for-each-site

if {N==S && E==W}

{xchng(N,E); xchng(S,W)}

endif

end-site

end-step

QuickTime™ and aCinepak decompressor

are needed to see this picture.

PIT 05/01/06

Lattice Gas Machines

Better for modeling:

• Flexible neighborhoods

• Incorporate physics

Better for hardware:

• Easier communication

• Flexible parallelismThe lattice gas programming model: alternate steps of movement and interaction.

PIT 05/01/06

CAM-8 node

• maps directly onto programming model (bit fields)• 16-bit lattice sites (use internal dimension to get more)• each node handles a chunk of a larger space

PIT 05/01/06

PIT 05/01/06



CAM-8: discrete hydrodynamics

Six direction LGA flow past a half-cylinder. System is 2Kx1K.

PIT 05/01/06

Crystallization

Crystallization using irreversible forces (Jeff Yepez, AFOSR)



PIT 05/01/06

CAM-8: EM scattering

PIT 05/01/06

CAM-8: materials simulation

512x512x64 spin system, 6up/sec w rendering 256x256x256 reaction-diffusion (Kapral)

PIT 05/01/06

QuickTime™ and aPNG decompressor


CAM-8: logic simulation

• Physical sim

• Logical sim

• Wavefront sim

• Microprocessor at right runs at 1KHz

A simple logic simulation in agate-array-like lattice dynamics.

Wavefront simulation

(R. Agin)

PIT 05/01/06

CAM-8: porous flow• Used MRI data from a

rock (Fontainebleau sandstone)

• Simulation of flow of oil through wet sandstone agrees with experiment within 10%

• 3D LGA’s used for chemical reactions, other complex fluids64x64x64 simulation (.5mm/side)

PIT 05/01/06

CAM-8: image processing• Local dynamics can be used to

find and mark features• Bit maps can be “continuously”

rotated using 3 shears

(Toffoli)

PIT 05/01/06

CAM-8 node (1988 technology)

• 25 Million 16-bit site-updates/sec

• 8 node prototype• arbitrary shifts of

all bit fields at no extra cost

• limited by mem bandwidth

• still 100x PC!

6MB/sec x 16 DRAMs + 16 ASICs

25 MHz 64Kx16

PIT 05/01/06

FPGA node design (1997)

• 100 x faster per DRAM chip

• No custom hardware

• Needed fast FPGA interface to DRAM 600MB/sec

1 RDRAM + 1 FPGA

150 MHz 64Kx16 pipelined

FPGARDRAM

PIT 05/01/06

Embedded DRAM (2000)

2K

2K

2K bits / 40 ns

4 Mbit Blockof DRAM:

40K

2K

40K bits / 40 ns = 1 Tbit/sec per chip!

20 Blocks of DRAM(80 Mbits total):

Each chip would be 1000x faster than 128-DRAM CAM-8!

PIT 05/01/06

SPACERAM:SPACERAM: A general purpose crystalline lattice computer

tyx

tyxtyxtyx

tyxtyxtyx

tyxtyxtyx

tyxtyxtyxtyx

f

fff

fff

fff

ffff

,,

,2,,2,,,2

,,2,1,1,1,1

,1,1,1,1,1,

,1,,,1,,11,,

τσνμκιηζεδγβα

+

+++

+++

+++

++=

−+−

+−−+−

−+++−

+−++

• Large scale prob with spatial regularity• 1 Tbit/sec ≈ 1010 32-bit mult-accum /

sec (sustained) for finite difference• Many image processing and rendering

algorithms have spatial regularity• Brute-force physical simulations

(complex MD, fields, etc.)• Bit-mapped 3D games and virtual

reality (simpler and more direct)• Logic simulation (physical, wavefront)• Pyramid, multigrid and multiresolution

computations

PIT 05/01/06

SPACERAM• The Task: Large-scale brute force

computations with a crystalline-lattice structure

• The Opportunity: Exploit row-at-a-time access in DRAM to achieve speed and size.

• The Challenge: Dealing efficiently with memory granularity, commun, processing

• The Trick: Data movement by layout and addressing

• The Result: Essentially perfect use of memory and communications bandwidth, without buffering

http://people.csail.mit.edu/nhm/isca.pdf

. 18µm CMOS, 10 Mbytes DRAM, 1 Tbit/sec to mem, 215 mm2, 20W

PIT 05/01/06

Lattice computation model

1D example: the rows are bit-fields and columns sites

Shift bit-fields periodically and uniformly (may be large)

Process sites independently and identically (SIMD)

A

B

A

B

PIT 05/01/06

Mapping the lattice into memory

A

B

A

B

A

B

A

B

PIT 05/01/06


CAM-8 approach: group adjacent into memory words

Shifts can be long but re-aligning messy chaining

Can process word-wide (CAM-8 did site-at-a-time)

A

B

A

B

PIT 05/01/06


CAM-8: Adjacent bits of bit-field are stored together

=

+

+

+

PIT 05/01/06


SPACERAM: bits stored as evenly spread skip samples.

=

+

+

+

PIT 05/01/06

The SPACERAM chip

module #00

module #01

module #02

module #03

module #19

PE0 PE63

……

…

…

…

…

PIT 05/01/06

Hardware Evolution• Virtual Processor

» Space-efficient parallelism» Static routing (systolic)

• Lattice Gas Machines» Simpler & more general» Ideal for physical modeling» Virtual SIMD programming

• General Lattice Computer » Perfect memory embedding» Finer-grained virtual-SIMD

• Computing Crystals » Architecture in software» Easier to design/build/test

PIT 05/01/06










PIT 05/01/06

Why emulate physics?

• Comp must adapt to microscopic physics

• Comp models may help us understand nature

• Rich dynamics

• Start with locality: Cellular Automata

PIT 05/01/06

Conway’s “Game of Life”

If total = 2: center unchanged

If total = 3: center becomes 1

Else: center becomes 0

In each 3x3 neighborhood, count the ones, not including the center:

256x256 region of a larger grid.Glider gun inserted near middle.



PIT 05/01/06

Conway’s “Game of Life”• Captures physical locality and

finite-state

But,

• Not reversible (doesn’t map well onto microscopic physics)

• No conservation laws (nothing like momentum or energy)

• No interesting large-scale behavior256x256 region of a larger grid.

About 1500 steps later.

PIT 05/01/06

Reversibility & other conservations

• Reversibility is conservation of information

• Why does exact conservation seem hard?

The same information is visibleat multiple positions

For rev, one nth of the neighborinfo must be left at the center

PIT 05/01/06

• With traditional CA’s, conservations are a non-local property of the dynamics.

• Simplest solution: redefine CA’s so that conservation is a manifestly local property

• CA regular computation in space & time

» Regular in space: repeated structure

» Regular in time: repeated sequence of steps

Adding conservations

PIT 05/01/06

Diffusion rule

a b

c d

c a

d b

a b

c d

b d

a c

Even steps: rotate cw or ccw

Odd steps: rotate cw or ccw

orb d

a c

orc a

d b

We “randomly” choose to rotate blocks90-degrees cw or ccw (we actually use afixed sequence of choices for each spot).

Use 2x2 blockings. Use solidblocks on even time steps, usedotted blocks on odd steps.

PIT 05/01/06



Diffusion rule

a b

c d

c a

d b

a b

c d

b d

a c

Even steps: rotate cw or ccw

Odd steps: rotate cw or ccw

orb d

a c

orc a

d b

We “randomly” choose to rotate blocks90-degrees cw or ccw (we actually use afixed sequence of choices for each spot).

PIT 05/01/06

TM Gas rule

a b

c d

c a

d b

a b

c d

b d

a c

Even steps: rotate cw

Odd steps: rotate ccw

Except: 2 ones on diag, nc


PIT 05/01/06

TM Gas rule

a b

c d

c a

d b

a b

c d

b d

a c




Even step: update solid blocks.

PIT 05/01/06

TM Gas rule

a b

c d

c a

d b

a b

c d

b d

a c




Odd step: update dotted blocks

PIT 05/01/06

TM Gas rule

a b

c d

c a

d b

a b

c d

b d

a c




Even step: update solid blocks

PIT 05/01/06

TM Gas rule

a b

c d

c a

d b

a b

c d

b d

a c





PIT 05/01/06

TM Gas rule

a b

c d

c a

d b

a b

c d

b d

a c




Even step: update solid blocks

PIT 05/01/06

TM Gas rule

a b

c d

c a

d b

a b

c d

b d

a c





PIT 05/01/06



TM Gas rule

a b

c d

c a

d b

a b

c d

b d

a c




PIT 05/01/06





TM Gas rule

PIT 05/01/06

• Half the time: HPP gas rule everywhere:

• Half the time: HPP gas rule outside of blue region, ID rule inside (no change).

Lattice gas refraction

a b

c d

d c

b a

Except: two ones on diag flip

Even & odd: swap along diags



PIT 05/01/06

2D ILGA for EM Fields

• Integer lattice gas version of TLM

• Like HPP, but with 4-bit integer particle counts for each direction

• Conserve E and H fields, momentum and energy

http://people.csail.mit.edu/nhm/EM2D.pdf

(IEEE MICROWAVE AND GUIDED WAVE LETTERS, VOL. 9, NO. 1, JAN 1999)

PIT 05/01/06

3D LGA for EM Fields

• Based on TDFD model of K. S. Yee and expanded TLM

• Unit cell is 8 sites, two of which aren’t used

• Six components of E and H occupy separate sites

http://people.csail.mit.edu/nhm/EMLGA.pdf (J Comp Phys 151, 816–835,1999)

PIT 05/01/06



Lattice gas hydrodynamics

Six direction LGA flow past a half-cylinder, with vortex shedding. System is 2Kx1K.

PIT 05/01/06

Dynamical Ising rule

Even steps: update gold sublattice

Odd steps: update silver sublattice

Gold/silver checkerboard

A spin is flipped if exactly 2 of its 4neighbors are parallel to it. After theflip, exactly 2 neighbors are still parallel.

We divide the space into twosublattices, updating the goldon even steps, silver on odd.

PIT 05/01/06







PIT 05/01/06





Homework Problem #1:• Show that the waves running

along the boundary obey the wave equation exactly

Hint:• The wave equation’s solutions

consist of a superposition of right- and left-going waves

PIT 05/01/06

Bennett’s 1D rule



Gold/silver 1D lattice


At each site in a 1D space, we put2 bits of state. We’ll call one the“gold” bit and one the “silver” bit.We update the gold bits on evensteps, and the silver on odd steps.

PIT 05/01/06



Bennett’s 1D rule




PIT 05/01/06

Bennett’s 1D rule




Homework Problem #2:• Simulate Bennett’s 1D rule and

measure the average width of the 5 smallest “gliders” and their speed.

• How is speed related to width?

PIT 05/01/06

3D Ising with heat bath

If the heat bath is initially much cooler than thespin system, then domains grow as the spins cool.



PIT 05/01/06

2D “Same” rule



A spin is flipped if all 4 of its neighborsare the same. Otherwise it is leftunchanged.


We divide the space into twosublattices, updating the goldon even steps, silver on odd.

PIT 05/01/06



2D “Same” rule



A spin is flipped if all 4 of its neighborsare the same. Otherwise it is leftunchanged.

PIT 05/01/06

QuickTime™ and aMotion JPEG B decompressor




3D “Same” rule

PIT 05/01/06

xx

Reversible aggregation rule

xh

xxh xg



When a gas particle diffuses next to exactlyone crystal particle, it crystallizes and emitsa heat particle. The reverse also happens.

We update the gold sublattice,then let gas and heat diffuse,then update silver and diffuse.

g

for more info, see cond-mat/9810258


PIT 05/01/06



Reversible aggregation rule

for more info, see cond-mat/9810258

xx xh

xxh xg



When a gas particle diffuses next to exactlyone crystal particle, it crystallizes and emitsa heat particle. The reverse also happens.

g

PIT 05/01/06

Adding forces irreversibly

3D momentum conserving crystallization.

becomes:

Particles six sites apart alongthe lattice attract each other.

PIT 05/01/06



Adding forces irreversibly


PIT 05/01/06

Conservations summary

1. the data are rearranged without any interaction, or

2. the data are partitioned into disjoint groups of bits that change as a unit. Data that affect more than one such group don’t change.

To make conservations manifest, we employ a sequence of steps, in each of which either

Conservations allow computations to map efficiently onto microscopic physics, and also allow them to have interesting macroscopic behavior.

b c da c d ab

xx xhg

PIT 05/01/06

Conservations

b c da c d ab

xx xhg

Homework Problem #3:• Make up a CA rule that

is reversible and has an exact conservation.

PIT 05/01/06










PIT 05/01/06

Review: Why emulate physics?

• Comp must adapt to microscopic physics

• Comp models may help us understand nature

• Rich dynamics

• Started with locality (Cellular Automata).

PIT 05/01/06

Review: Conway’s “Life”• Captures physical locality and finite-state

But,

• Not reversible (doesn’t map well onto microscopic physics)

• No conservation laws (nothing like momentum or energy)

• No interesting large-scale behavior

Observation:

• It’s hard to create (or discover) conservations in conventional CA’s.

256x256 region of a larger grid.Activity has mostly died off.

PIT 05/01/06

Physical Worlds

Some regular spatial systems:

1. Programmable gate arrays at the atomic scale

2. Fundamental finite-state models of physics

3. Rich “toy universes”

• All of these systems must be computation universal

PIT 05/01/06

Computation Universality If you can build basic logic elements and connect them together, then you can construct any logic function -- your system can do anything that any other digital system can do!

• It doesn’t take much.

• Can construct CA that support logic.

• Can discover logic in existing CAs (eg. Life)

• Universal CA can simulate any other

Logic circuit in gate-array-like CA

PIT 05/01/06



Computation Universality If you can build basic logic elements and connect them together, then you can construct any logic function -- your system can do anything that any other digital system can do.

• It doesn’t take much.

• Can construct CA that support logic.

• Can discover logic in existing CAs (eg. Life)

• Universal CA can simulate any other

Logic circuit in gate-array-like CA

PIT 05/01/06

What’s wrong with Life?

Glider guns in Conway’s “Game of Life” CA. Streams of gliders can be used as signals in Life logic circuits.

• One can build signals, wires, and logic out of patterns of bits in the Life CA

PIT 05/01/06



What’s wrong with Life?

• One can build signals, wires, and logic out of patterns of bits in the Life CA

• Life is short!• Life is microscopic• Can we do better with

a more physical CA? Life on a 2Kx2K space, run from arandom initial pattern. All activitydies out after about 16,000 steps.

PIT 05/01/06

Billiard Ball Logic

• Simple reversible logic gates can be universal

• Turn continuous model into digital at discrete times!

• (A,B) AND(A,B) isn’t reversible by itself

• Can do better than just throw away extra outputs

• Need to also show that you can compose gates

Fredkin’s reversibleBilliard Ball Logic Gate

PIT 05/01/06

Billiard Ball Logic

Fixed mirrors allow signals to be routed around.

Mirrors allow signals to cross without interaction.

PIT 05/01/06

A BBM CA rule

2x2 blockings.The solid blocksare used at eventime steps, thedotted blocks atodd steps.

A BBMCA collision:

BBMCA rule.Single one goesto opposite corner,2 ones on diagonalgo to other diag, noother cases change.

PIT 05/01/06

The “Critters” rule


This rule is appliedboth to the even andthe odd blockings.

We show all cases:each rotation of a caseon the left maps to thecorresponding rotationof the case on the right.

Note that the number ofones in one step equals the number of zeros inthe next step.

PIT 05/01/06



The “Critters” rule

This rule is appliedboth to the even andthe odd blockings.

We show all cases:each rotation of a caseon the left maps to thecorresponding rotationof the case on the right.

Note that the number ofones in one step equals the number of zeros inthe next step.Reversible “Critters” rule, started from

a low-entropy initial state (2Kx2K).

PIT 05/01/06

“Critters” is universal

A BBMCA collision:

Critters “glider” collision:

PIT 05/01/06

UCA with momentum conservation

Hard sphere collision

• Hard-sphere collision conserves momentum

• Can’t make simple CA out of this that does

• Problem: finite impact parameter required

• Suggestion: find a new physical model!

PIT 05/01/06


Hard sphere collision Soft sphere collision

PIT 05/01/06


Can shrink balls to points! Soft sphere collision

PIT 05/01/06


SSM rule: rotations also act like this. All other cases remain unchanged.This is a Lattice Gas: movement and interaction steps alternate. Can shrink balls to points!

PIT 05/01/06


SSM rule: rotations also act like this. All other cases remain unchanged.This is a Lattice Gas: movement and interaction steps alternate. Add mirrors at lattice

points to guide balls.

PIT 05/01/06


Add mirrors at lattice points to guide balls. SSM rule with mirrors

PIT 05/01/06


Add mirrors at lattice points to guide balls.

Mirrors allow signals to cross without interacting.

PIT 05/01/06

SSM collisions on other lattices

Triangular lattice 3D Cubic lattice

PIT 05/01/06

Macroscopic universalityWith exact microscopic control of every bit, the SSM model lets us compute reversibly and withmomentum conservation, but

• an interesting world should have macroscopic complexity!• Relativistic invariance would allow large-scale structures to

move: laws of physics same in motion• This would allow a robust Darwinian evolution• Requires us to reconcile forces and conservations with

invertibility and universality.

PIT 05/01/06

Relativistic conservation Non-relativistically, mass

and energy are conserved separately

Simple lattice gasses that conserve only m and mv are more like rel than non-rel systems!

€

E∑ = ′ E ∑E i

r v i∑ = ′ E i ′

r v i∑

€

12 miv

2∑ = 12 ′ m i∑ ′ v 2

mi∑ = ′ m i∑mi

r v i∑ = ′ m i ′

r v i∑

€

(sincer p = γm

r v = γmc 2 ×

r v /c 2)

Non-relativistic:

Relativistic:

(energy)

(mass)

(mom)

(energy)

(mom)

PIT 05/01/06

Can we add macroscopic forces?

3D momentum conserving crystallization.

becomes:

Particles six sites apart alongthe lattice attract each other.

PIT 05/01/06



Can we add macroscopic forces?


PIT 05/01/06

Universality• Universality is a low threshold that separates triviality

from arbitrary complexity• More of the richness of physical dynamics can be

captured by adding physical properties:» Reversible systems last longer, and have a realistic

thermodynamics.» Reversibility plus conservations leads to robust “gliders” and

interesting macroscopic properties & symmetries.

• We know how to reconcile universality with reversibility and relativistic conservations

PIT 05/01/06

• Use CA hardware to try to do computation better

• Use CA models to try to understand the world better

• CA’s are new worlds that we can explore and study

Conclusions



pit 05/01/06 physics of information technology norm margolus crystalline computation

Documents