csar legacy

CSAR Legacy

Michael T. HeathFebruary 21, 2007

©2007 Board of Trustees of the University of Illinois

Center for Simulation of Advanced Rockets

University of Illinois at Urbana-Champaign©

2


University of Illinois at Urbana-Champaign©2007 Board of Trustees of the University of Illinois


Detailed, whole-system simulation of solid propellant rockets under normal and abnormal operating conditions

Accurate models of physical components Subscale simulations of materials and

accident scenarios Software framework to facilitate

component integration Computational infrastructure to support

large-scale simulations Research collaborations with government

laboratories and rocket industry

Overarching Goal: Simulation of solid propellant rockets from first principles

STS Discovery

3



Solid Rocket BoosterIgniter

Case

Core flow

Propellant

Nozzle

Joint & slot

Ammonium perchlorate (oxidizer)

Polymeric binder and aluminum particles (fuel)

200 m

Micrograph of solid propellant

4



Challenges in Rocket Simulation

Full 3-D modeling essential to capture physics Strong, nonlinear coupling among components Complex, dynamically changing geometry Extremely diverse spatial and temporal scales Complex material properties and physical

processes

Enormous computational capacity required for high-resolution simulation of full burn

Scalability to 1000s of processors essential

CSAR Legacy

Rocstar code Major simulations Fundamental research Impact

CampusState and localNational and international

Human resources Lessons learned

QuickTime™ and aGIF decompressor

are needed to see this picture.

Aft Joint Slot

ForwardJoint Slot

Titan IV Turbulence

6



Rocstar Code Suite Nation’s only 3-D integrated simulation capability for solid rocket

motors How did we accomplish this?

Fully-coupled RSRM— fluid temperature and

propellant stressprofiles

Adequate financial resourcesLong time horizonCritical mass of talent across

disciplinesAvailability of massively parallel

platforms

Now in position to solve problems of national interest, and already doing so

Five-segment RSRM for lunar program

7



Rocstar Code Suite

Scalable to thousands of processors Applicable to many other problems

Fluid-structure interactionReactive, multiphase flows

ExamplesPyrovalveHelicopter bladesVolcanoesTall buildings

NASABooster Separation Motor

8



Major Simulations“Science with the Code”

Full RSRM Titan propellant slumping Turbulence around

flexible inhibitor Aluminum impingement

on nozzle

Al particles impacting nozzle

Flow field near slumping

propellant

Verification, Validation, and Uncertainty Quantification

Verification Convergence studies Comparison with analytical solutions Method of manufactured solutions Comparison between codes

e.g., do Rocflo and Rocflu give similar results for same problem?

Code coverage Coupled codes — verify component

codes first Validation

RSRM (ATK/Thiokol & NASA) Titan IV (Lockheed Martin) 15 and 70 lb BATES (AFRL) Lab scale rocket (NAWC) Attitude Control Motor (Aerojet) Shock tube with thin steel panel

New sampling techniques for UQ

10



Navy “StarAft” Motor

Full burnout Tactical motor

11



Titan IV Motor

Heavy-lift booster Accident scenario Detailed vorticity

studies

12



NASA RSRM

Late stage in full burn (~100 s)

Flexible inhibitor 3-D vorticity

study

NASA RSRM Burnout simulation

Red is burning propellant; blue is at insulated surface

Viewing Propellant/fluid interface Inhibited surfaces

Unique capabilities High rate “time zooming” Significant burnout — never seen in

industry Case constraints Propellant walkback Inhibitor regression Dynamically changing topology

Initially inhibited

surface

14



Fundamental Research inFluids and Combustion

Multiphase flowEquilibrium Eulerian method

for fine particlesLagrangian superparticles

Turbulence modelingOptimal LES

Time zooming Propellant morphology

Parallel packing code

Propellant modeling3-D simulations with Rocfire

15



Fundamental Research inStructures and Materials

Constitutive and damage modeling

Heterogeneous propellants and HEMetallic components

Crack propagationBurning, pressure driven

Multiscale materials modeling Molecular-level modeling of

material interfaces Space-time discontinuous

Galerkin methods

16



Fundamental Research inComputer Science & Applied Math

Parallel programming environmentsCharm++, AMPI, ParFUM

Parallel I/OPanda

Parallel performance monitoring/modelingRocprof

MeshingMesh smoothing, repair, and

remeshingMesh quality metricsHybrid geometric/topological

partitioner Visualization

Rocketeer

17



Fundamental Research inIntegration Methodology

Object-oriented integration framework Flexible parallel orchestration Stable component coupling and time-stepping

Accurate and conservative data transfer based on common refinement

Stable and efficient explicit surface propagation

Fluid DynamicsSolid MechanicsCombustion

Mesh GenerationCAD ModelingPreprocessingIntegrationInterface

Orchestration Solution TransferSurface PropagationMesh ModificationProfilingData FormatsParallel I/OVisualization

Computational Scienceand Engineering

EducationProgram

13 departments

130 faculty associates

130 grad students

10 graduate fellows

ResearchProgram

Computational Science &

Engineering Option


Center for Simulation of

Advanced Rockets

Center forProcess

Simulation and Design

DOE funded$4-5 million per year 10 year program20 faculty35 graduate students5 undergrads20 professional staff

NSF funded$6 million over 6 years

12 faculty13 students & postdocs

Midwest Structural SciencesCenter

AFRL funded$5 million over 5 years

10 faculty15 students & postdocs

19



Campus Impact CSAR put CSE “on the map” CSAR followed by two additional centers hosted by CSE

Center for Process Simulation and DesignMidwest Structural Sciences Center

CSE seen as “center of centerness” on campus, advising other groups seeking or establishing centers

CSE taken as model in developing other new interdisciplinary programs (e.g., bioinformatics)

CSE’s Turing cluster Established value of large-scale computing facilities

for local usersFinancial commitment from campus administration

20



Turing G5 Xserve Cluster

1536 compute processors Dual 2GHz G5 processors

per node 4GB RAM per node Myrinet interconnection

network

6 Xserve head nodes 49TB Xserve RAID

512 and 256 node partitions

23 racks (15 and 8 racks, respectively)

Mac OS 10.4 Server

21



State and Local Impact

Economic development through industrial collaborations

Boeing, Caterpillar

Visibility of simulation promoted through various venues

Recurring appearances in media (TV, newspapers)

RSRM

inhibitor vorticity

Flexible inhibitor

22



National and International Impact Publications (~900 journal articles and proceedings) Conference sessions Industrial collaborations

ATK ThiokolAerojetBoeingCaterpillar

Other government agency funded researchUSAFNASA

Technology transfer to and from NNSA labs Participation in Integrated Product Team for USAF

solid propulsion program

23



Pedagogical Impact

QuickTime™ and aTIFF (Uncompressed) decompressor


QuickTime™ and aTIFF (Uncompressed) decompressor


1st edition, 1997 2nd edition, 2002

24



Human Resources

Supported 160 graduate students, about 40 of whom have gone on to DOE labs and many others to positions in government or industry

Recruited and developed 50 research staff members, many of whom have gone on to positions at DOE labs, industry, and academia

Contributed to faculty career development (promotions, awards, etc.), counter to conventional wisdom about collaboration

25



CSAR Students and Staffin DOE/NNSA Labs

Boyana Norris, ANL Michael Parks, SNL Jason Petti, SNL Ali Pinar, LBNL Jason Quenneville, LANL Christopher Siefert, SNL Donald Siegel, SNL Christopher Tomkins, LANL Michael Tonks, LANL Jason Weber, BBL Bradley Wescott, LANL Amanda White, LANL Steven Wojtkiewicz, SNL Jin Yao, LLNL Jack Yoh, LLNL

Former CSAR Staff Now at DOE Labs (5)

Michael Ham, ORNL (LANL) Dennis Parsons, LLNL James Quirk, LANL Mark Short, LANL Jeff Vetter, LLNL (ORNL)

Former CSAR Students Now at DOE Labs (34)

Tyler Alumbaugh, LLNL Balakumar Balasubramanium, LANL Michael Bange, LANL Benjamin Chorpening, SNL Nathan Crane, SNL Zhiqun Deng, PNNL Erik Draeger, LLNL Michelle Duesterhaus, SNL Jeff Grossman, LBNL (LLNL) Arne Gullerud, SNL Thomas Hafenrichter, SNL Rebecca Hartman-Baker, ORNL Jason Hales, SNL Jonghyun Lee, ANL Vanessa Lopez, LBNL Xiaosong Ma, ORNL Greg Mackey, SNL Burkhard Militzer, LLNL Jeffrey Murphy, SNL-L

Gray now in other employment.

26



What Worked Well?

Single, over-arching applicationMaintain focus, avoid mission creep

Pre-existing, interdisciplinary framework independent of individual academic departments

Tight integration of CS with application disciplines

Professional staff in addition to faculty and students

Wealth of thesis-sized research issues for graduate students

Pipeline of students from Illinois to lab employment

27



What Could Have Worked Better?

Collaborations with NNSA labs were numerous, but could have been tighter and more comprehensive

Suggestions for improvementCommon application focus between labs and

universityDeeper commitment to joint activities, such as long-

term visits and joint hiring (e.g., postdocs)Two-way software exchanges

29



Prof. Michael T. Heath, Director


University of Illinois at Urbana-Champaign

2270 Digital Computer Laboratory

1304 West Springfield Avenue

Urbana, IL 61801 USA

[email protected]

http://www.csar.uiuc.edu

telephone: 217-333-6268

fax: 217-333-1910

csar legacy

Technology

slumping propellant

rocket simulation

propellant blue

micrograph of solid

highresolution simulation

process simulation

propellant stress profiles

d modeling essential