an accelerated strategic computing initiative (asci) academic strategic alliances program (asap)...

An Accelerated Strategic Computing Initiative (ASCI) Academic Strategic Alliances Program (ASAP) Center

at The University of Chicago

The Center for Astrophysical Thermonuclear Flashes

An Overview of the ASCI/Alliances Program at Chicago:

Year 3 Site Review

Robert Rosner, DirectorOctober 30, 2000

Chicago, IL(http://flash.uchicago.edu)

The ASCI/Alliances Center for Astrophysical Thermonuclear FlashesThe University of Chicago

What is the FLASH Problem?

To simulate matter accumulation on the surface of compact stars, nuclear ignition of the accreted (and possibly underlying stellar) material, and the subsequent evolution of the star’s interior, surface, and exterior• X-ray bursts (on neutron star surfaces)• Novae (on white dwarf surfaces)• Type Ia supernovae (in white dwarf interiors)

Credit: NASA/STScI


Observations of Astrophysical Flashes

Credit: NASA/Rossi XTE

Credit: NASA/STScICredit: NASA/STScI

Type Ia SN: SN 1994D

Nova: Nova Cygni 1992

X-ray burst: GS 1826-24


Schematics of the astrophysical problems

Type Ia supernova

X-ray burst (neutron star)Nova (white dwarf)

White dwarf

White dwarf/neutron star

White dwarf: R ~ REarth

(C, O) Neutron star: R ~ RChicago

(Neutrons)

H, He


Why are these interesting problems - Part 1

Defense programsParadigm for attacking physics problem without ability to

experimentTraining of the next generation of computational

scientistsFLASH is the only Center which includes MHD

AstrophysicsEndpoints of accretionNucleosynthesis of elementsCosmology

Physics Interface instabilities, mixingFundamentals of combustion


Why are these interesting problems - Part 2

Computational Science Integration of complex physicsExploration of grids, hydro algorithms, ...Practical application of OO-based programming

paradigm Sociology

Mixing astrophysicists, physicists, applied mathematicians, computer scientists …


What are the FLASH Center’s Goals?

To build a new-generation simulation code for computing astrophysical thermonuclear flashes involving compact stars Incorporate advances from CS, applied math,

physics modular, reusable code

Verify and validate code and its components

To produce great scienceAstrophysics and physicsComputational sciences [CS, Applied math, Math]


The Challenges

Extreme dynamic ranges of spatial and temporal phenomena AMR, subgrid modeling, front tracking

Large dynamic range of flow velocities Accurate computation of very low and high Mach number flows

Physical processes can be highly localized and non-uniform Load balancing

Physical processes can be non-local Example: Elliptic solvers on AMR grid

Rapidly evolving computing environment Creation of programming and computing environment

Astrophysical validation is extremely indirect Development of laboratory proxy validation experiments


What have we accomplished to date?

We are on track: we have a newly architected code Production version: Flash-1.61 OO-based data structures introduced: Flash-2 (in test) We have started on all three astrophysics problems

We responded to past Site Review suggestions Management changes Code architecture

We have obtained exciting science results

(Y. Young 2000)

(M. ZIngale 2000)

(N. Vladimirova 2000)

(F. Timmes 2000) (A. Calder 2000)


What you’ll hear about today

Talks

Flash Code: B. Fryxell, P. Ricker, A. Siegel

Astrophysics: J. Truran

Basic Science/Validation: T. Dupont

Computer Science: E. Lusk

Posters

Astro, CS, Math, Physics

Demos


What’s New in Year 3: Part 1

Re-organization Code group and Computational Physics combined Code architecture team formed within Code group: Code architect! Management team tightened

T. Dupont (Basic Sci/Validation), B. Fryxell (Code), E. Lusk (CS), R. Rosner (director), R. Stevens (ex officio), J. Truran (Astro)

Weekly management team meeting

Focus on re-engineering Flash Re-engineered OO-based architecture: Flash-1.6 Current optimized production version: Flash-1.61

Includes improved version of Paramesh New features incorporated in Flash-2.0 (now in active test)

OO-based data structures and data-sharing: Paves the way for extensibility of Flash

Python-based user interface, setup, automatic test facility, tools, …


What’s New in Year 3: Part 2

Focus on basic science Cellular detonations Interface mixing: R-T/R-M, wave breaking Flame propagation/modeling MHD Rad. transfer Implicit conduction Self-consistent gravity

Focus on astrophysics X-ray bursts Novae Type Ia supernovae


How was this done: Our strategy

Initial instantiation of Flash Code: Flash 1.0 Legacy hydro solver (“Prometheus”/F77) New adaptive mesh package (“Paramesh”/F90) New physics packages (nuclear reaction networks, EOS/F90)

“Classic” modular designAllowed first-cut astrophysics and validation calculationsGuided refinement of code architecture

New architecture code OO-based code architecture/framework

Optimized for productionNew module interface designAllows continued use as applications codeGradual transition to/introduction of object-oriented programming

Physics modules Automatic code verification/test facility New data sharing scheme Revised user interface

“Crude, but useful” Dick Watson (1999)

Flash-2.0

Flash-1.61


How was this done: Our Center’s structure

AstrophysicsJ. Truran

ComputerScience

R. Lusk

Basic Science& Validation

T. Dupont

FLASH CodeB. Fryxell

Chan, Clark,Curfman-McInnes,

Foster, Freitag,Flaherty, Gomez,Gropp, Hudson,

Loy, PapkaScott, Shephard,

Smith, Stevens, Tufo

Calder, Linde,Mignone,

Olson, Ricker,Timmes, Tufo, Weirs, Zingale

Arnett, Brown,Calder,Dursi, Fryxell, Lamb,

Mignone, Peng,Ricker, Timmes, Zingale

Caceres, Ricker,Reilly,

Vladimirova,K. Young

Alexakis, Constantin,Curtis, Draganescu,Kadanoff, Litwin, Liu,Medved, Oberman,

Ruchayski, Vladimirova, Y. Young

LANL, LLNL, MPIUCSC,

SUNY, UA, NWUCollaborators

Model problems(R-T, …)

Code Architecture,grids/AMR

Sub-grid turbulence,combustion,front tracking

LANL, LLNL,SNL, PPPL

PSU,LANL, LLNL,

SNL

Cattaneo, Constantin, Dupont, Dursi, Fryxell, Malagoli,Oberman, Rosner, Vladimirova, Weirs, Young

Curfman-McInnes, Flaherty, Freitag, Fryxell, Malagoli,Olson, Ricker, Scott, Shephard, Siegel, Tufo

Alexakis, Calder, Cattaneo, Constantin, Dupont, Flaherty, Fryxell, Kadanoff, Litwin, Niemeyer,Oberman, Olson, Rosner, Scott, Shephard, Tufo, Young

Clark, Chan, Freitag, Fryxell, Gomez, Gropp, Lusk, Olson,Siegel, Smith, Timmes, Tufo, Zingale

Performance/tuning,scaling

Astrophysics modules,Astrophysics

simulations andvalidation

Tool kits, Libraries,Visualization

Validation experiments,Fundamental physics

PhysicsModules

Architectureteam

A. Siegel

LANL, LLNL,SNL

GSFC

WorkingGroups

Ad hoc groups

Development,Maintenance,

& Testing

Calder, Dursi,Olson, Ricker,Timmes, Tufo,

Zingale


How was this done: Effective interactions

Flash CodeGroup

Astrophysics Group

Basic Science/Validation Group

Computer Science Group

Production CodeAstrophys. modules

Production Code

Validation, models

Code

Vis., Tools, Architect.

Monday 2:00pm Astro meeting

Monday 3:00pm Code/CS meeting

Wednesday 12:30pm Comp. Sci seminar

Wednesday 10:00am FlamesWednesday 11:00am MHD


Responses to Year 2 Site Review: Part 1

Concerns regarding progress on Flash code Decision regarding Flash architecture

Re-architected Flash exists! See talks by B. Fryxell/P. Ricker/A. Siegel “Insertion plan”

Plan for module insertion in place and being exercised; see talk by A. Siegel

Concerns regarding physics modules Nuclear reactions

See talk by J. Truran Radiative transport

See poster by A. Calder


Responses to Year 2 Site Review: Part 2

Concerns regarding Center management and staffing Management structure tightened Code architect

A. Siegel (Flash-2 on) P. Ricker (Basic Flash architecture)

Code group leader B. Fryxell

Executive director Center consensus: position not needed


Training the next generation ...

Name Field Present position

Graduates:Y. Liu fluid dynamics (Math) postdoc/SUNY Stony Brook, winter 1999-00R. Loy algorithms (CS) staff member/ANL, fall 1999M. Zingale astro/theory (A&A) postdoc /A&A, fall 2000Y.N. Young fluid dynamics (A&A) postdoc at Northwestern/Applied Math, fall 2000S. Wunsch fluid dynamics (Physics) postdoc, A. Kerstein/Sandia/Livermore, fall 1998S. Zhan astro/theory (A&A) financial company, spring 2000

Current:A. Alexaksis MHD student/physicsJ. Biello fluid dynamics (A&A) postdoc at RPI/Applied Math, fall 2000A. Caceres comp. physics student/physicsJ. Curtis fluid dynamics student/physicsA. Draganescu algorithms student/mathematicsJ. Dursi astro/theory student/A&A Sandia/summer 2000E. Gomez parallel tools (CS) assoc. professor at UC San Bernardino, fall

2000A. Mignone MHD student/A&AA. Oberman combustion student/mathematics Sandia/summer 2000M. Papka visualization student/CS & staff member/ANLF. Peng comp. Astro student/A&AO. Ruchayskiy fluid dynamics student/physics


The Future of Flash: The larger picture

1. Follow the FLASH Center plan Continue Flash code development: push extensibility Pursue Flash development for the 3 astrophysics problems Pursue Astrophysics, Basic Science/Validation, and

Computer Science activities as planned

2. Up the ante …go beyond our earlier ambitions Include new physics features not previously planned on

Two-fluid hydro/MHD N-body

Attack FLASH-related astrophysics problems not in original FLASH Center plan

Accretion flows and disks Stellar evolution (including star formation)

Explore new ideas from CS and computational science Exploratory/collaborative computing


A final comment and summary

Project oversight by DOE has been productive First-rate scientists appointed as project managers Project oriented toward helping Centers succeed

Annual external site review Annual DOE-only site visit Constant interactions on science

We have succeeded in getting where we wanted to be, and in some cases, are further along than anticipated New architecture in place, and producing many science results All astrophysics problems have been started, with astro. results Center structure has evolved to function effectively

The challenges Effective validation and model building is very hard … Available computational resources no longer match our needs


… and that leads us to

QUESTIONS AND DISCUSSION

an accelerated strategic computing initiative (asci) academic strategic alliances program (asap)...

Documents

q training

ascialliances center

physics q modular

reusable code q

nonlocal q example

great science q astrophysics

temporal phenomena q

qdefense programs q