The Education of

Computational Scientists

Robert Voigt

Krell Institute

“… computational science has been described as essential to advances throughout society and

deemed one of the most important technical fields of the 21st century.”

https://www.nitrd.gov/pitac/reports/20050609_computational/computational.pdf

Outline

• History

• Formal coursework and programs

• Exposure to multidisciplinary research

• Putting it all together: A successful model

Brief History

When did Computational Science

“Begin”

• Scientific Computing (SC) or Computational Science

and Engineering (CSE)

• 1950s: Von Neumann’s work leading to

– MANIAC

– ORACLE

and their use in solving engineering problems

• Clearly SC/CSE but not recognized as such until

1982 Lax Report

(Report of the Panel on Large-Scale Computing in Science and Engineering,

http://www.pnl.gov/scales/docs/lax_report1982.pdf)

From a presentation by Jack Worlton, LANL, August 5, 1985

Early View of Scientific Computing

“During its spectacular rise, the computational has joined

the theoretical and the experimental branches of

science, and is rapidly approaching its two older sisters

in importance and intellectual respectability”, Lax, 1986.

Evolution to Computational Science

Evolution of Computational

Science

O. Yas¸ar, K. S. Rajasethupathy, R. E. Tuzun, R. A. McCoy, and J. Harkin, A new

perspective on computational science education, Comput. Sci. Engrg., 5 (2000)

Computational Science as a

Discipline

Computer

Science

Applied

Discipline

CSE

Mathematics

Yas¸ar, et al. 2000

Petzold et al. 2001

So What is CSE

• Many definitions in the literature

• Key components

– Education

• Mathematics

• Computer science

• Science or Engineering discipline

– Exposure to multidisciplinary research

– Integration of knowledge & methodologies

• More than Computational X

CSE: A dynamic field of its own

• New areas of science and engineering

– From CFD to bioinformatics to social science

– Predictive Science

• Explosion of data

– analysis of data-centric applications

– data-driven scientific discovery and data

enabled uncertainty quantification

– analysis of experiments

• Rapidly changing computer architectures

Formal PhD Programs

• Survey (2012):

http://icl.cs.utk.edu/survey/summary/

• Types of graduate programs:

– A degree in CSE: 41, 19 PhD

– A minor in CSE: 7

– A certificate in CSE: 6

– A track in CSE 2

CSE Course Work

• Most of the 19 have course requirements

involving at least a subset of

– Computer Science

– Applied Mathematics

– Science or Engineering

• What is missing is immersion in

multidisciplinary research

• Many reports cite this as critical to the

development of computational scientists

Immersion in Multidisciplinary

Research: Early Examples• Department of Energy Research Facilities

– Extensive postdoc opportunities

– Seldom combines application, math and CS

• Institute for Computer Applications in

Science and Engineering (ICASE)

– Set inside NASA Langley Research Center

but independent

– Research combining application, math and

CS

– Postdoc opportunities and some

undergraduates

More Recent DOE Examples

• Characteristics

– Partnerships among DOE Labs and Academia

– Applications, math and CS

• ASCR Scientific Discovery through

Advanced Computing (2000)

– Software advances to support applications

• Co-Design (2011)

– Application characteristics influence hardware

and software design

No focus on student development

Academic Example

• Academic Strategic Alliance Program

– Initiated in 1997 by DOE/NNSA Office of

Advanced Simulation and Computing

– Focus on modeling and simulation of

multidisciplinary, multiscale applications

• Predictive Science Academic Alliance

Program (2008)

– Added Verification/Validation & Uncertainty

Quantification research to support prediction

PSAAP II Program

PSAAP II Characteristics

• Funded in 2014 for 5 years at $4M and $2M per

year

• Added specific CS research to support evolution

to useable Exascale computing demonstrated in

the context of their application

• Co-located students immersed in application,

math, CS environment

• Requires internship at NNSA National

Laboratory by all students

• http://www.sandia.gov/psaap/

Stanford UniversityPredictive Simulations of Particle-laden

Turbulence in a Radiation Environment

Use particles to provide higher energy absorption

and transfer rates

• Multidisciplinary– Turbulence, DNS with particles

– Particle transport

– Radiative transport

– Eulerian PDE solver for turbulent flow

– Lagrangian method/particle tracking for solid phase

– Ray tracing/discrete ordinates for radiative transport

• CS research – Domain-Specific Languages (DSL)

• High-level library/framework to dynamically compile

optimized code

• Multiple DSLs for multiphysics coupling

– Containment Domains for resiliency

Exascale Simulation of Plasma-coupled CombustionPredict ignition threshold for jet in crossflow via

spark-discharge and dielectric-barrier-discharge

(DBD) plasmas

• Multidisciplinary– Turbulent mixing and combustion

– Plasma heating, ionization, and transport

(Poisson solver)

– Chemical kinetics and transport

– Prediction of onset of ignition of jet

– Scalable asynchronous numerical algorithms

• CS research – Suite of interoperable tools

– Overdecomposition for locality, latency, load

balancing

– Source-to-source transformations for architecture

opt. while supporting programmer productivity

University of Illinois

Plasma-ignited fuel jet in crossflow

University of Utah

Oxygen-fired “Clean Coal” Boiler for High

Efficiency Electric Power Generation with

Carbon Capture

Predict heat flux for an as yet to be built system

• Multidisciplinary– Turbulence with particle transport

– Radiative transport via rays or discrete ordinates

– Combustion of particles and turbulent gas

– Heterogeneous chemistry of reacting particles

– Higher order numerical methods

• CS research– In-situ analysis and visualization

– DSL for solution of PDE systems

•embedded in C++

•allows for gradual adoption of features, continuous evolution

of large legacy codebases

50-mm coal

particles

High efficiency advanced ultra-supercritical (AUSC) oxy-coal tangentially-fired power boiler

Texas A&M University

Exascale Radiation Transport for High

Energy Density Physics

Validation of photon transport algorithms using

neutrons as surrogates

• Multidisciplinary

– Neutron transport

– Neutron scattering cross-sections

– Adaptive parallel multigrid algorithms

– Scalable iterative methods: Adaptation in time,

space, and angle

• CS research– Multilevel libraries for architecture independence

(like DSLs)

– Methodology for automated selection of optimum

algorithms

– Replication of task graphs for fault tolerance

– Performance models, machine models

Neutron group fluxes

in 1m x 1m x 2m

graphite block, with

pulsed source

Possible 3D

streaming paths in

neutron transport

experiments

University of FloridaCompressible Multiphase

Turbulence in Explosive-driven

Particle-laden Flows

Multiscale models for shock/particle interaction

• Multidiciplinary– Turbulence with particle interaction

– DNS and LES

– Hybrid spectral-WENO schemes

– High-order discontinuous Galerkin methods

– Heterogeneous space-time discretizations• Perform computation only when and where needed

• CS research

– Field-programmable gate array (FPGA)

emulation at device, node, and system-level

– Behavioral Emulation Objects representing

software/hardware for performance prediction

Explosive dispersal of 114-mm Al particles

University of Notre Dame

Shock Wave-processing of Advanced Reactive Materials

Predict conditions for synthesis of cubic boron

nitride (c-BN) by reverse Taylor impact exp

• Multidisciplinary– Shocks in heterogeneous reactive materials– Solid-solid phase transitions, plastic flow, shear band formation, grain sliding, debonding– Micro/meso/macro scale bridging– Simulation of shock-induced synthesis of Ni/Al

composite

• CS research– Asynchronous parallel computing model

– Dynamic adaptive control of computing and resource management

• Light-weight user threads

• “Move the work to the data” when advantageous

• Light-weight semantically rich synchronization

mechanisms like dataflowStress in a Ni/Al composite

Training and Education

• Opportunities for training via national

programs

– NSF and DOE Supercomputer Centers

– NSF Extreme Science and Engineering

Discovery Environment

– NSF Research Traineeship Program

– DOE National Laboratories

• Informal education in CSE that fills in

missing gaps in academic programs

DOE Computational Science

Graduate Fellowship (CSGF)

• Administered by the Krell Institute

https://www.krellinst.org

– With support from the DOE

– Guidance from Steering Committee

• Undergraduates & 1st year graduates

eligible

• Students complete fellowship application

on line focused on a science/engineering

application

Program Support

• Stipends ($36,000/year for 4 years)

• Full tuition and fees

• Professional development support

• $5,000 first year and $1,000 each renewed

year

• Laptop/conference travel/society dues …

• Practicum support (living expenses and travel)

• Annual program review

• Fellows “own” the fellowship

CSGF

• Includes three major components

– Formal education

– Focus on an application

– Immersion in DOE Lab via a 12 week practicum

• Application requires essays & courses

• Complex, thorough review of applications &

renewal process

• Continual monitoring of progress

Formal Education

Program of Study (POS)• Proposed POS must contain

– Year of courses in the chosen application

– Year of courses in math/statistics

– Year of courses in computer science

• Courses must be from traditional

department for the discipline

• POS must contain exposure to HPC

• POS is continually reviewed by Steering

Committee

Practicum

• Each fellow required to spend 12 weeks at

a DOE facility

• Fellow submits a proposal reviewed by

Steering Committee

• Immersed in a multidisciplinary research

environment

• Mentored by active researcher

• Intended as a broadening experience

Some Statistics

• CSGF entering its 26th year

• Approximately 400 alums

• Annual awards range from 5 to 23

– Driven by budget and federal policy

– Approximately $10M annual budget

• Highly competitive

– 456 applications for ~ 20 awards this year

• Large research universities dominate

Where do Alums End UP

Krell has current employment data for 314 alumni grouped as follows:

– Academia 101

– Industry 105

– DOE Labs 47

– Other Gov’t. 28

– Grad. Student 25

– Other 4

– Nonprofit 4

– Unknown 15

Academia31%

DOE Labs14%

Graduate Student

8%

Industry32%

Other1%

Other Government

8%

Unknown5%

Non-Profit1%

Alumni by Employment Category

0

2

4

6

8

10

12

14

16

18

1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016

Incoming Fellow Classes by Field Area

Biology and Bioengineering Computer Science and Math Engineering Physical Science

$-

$2,000,000

$4,000,000

$6,000,000

$8,000,000

$10,000,000

$12,000,000

$14,000,000

FY2004 FY2005 FY2006 FY2007 FY2008 FY2009 FY2010 FY2011 FY2012 FY2013 FY2014 FY2015 FY2016

DOE CSGF Funding (FY2004-FY2016)

ASCR NNSA Total

16 15 1917

18 16

21 18

11

21

10

23

27

Funding Challenge

What’s Missing in CSGF

• Present program focus on application is

barrier for Math/CS majors

• Add enabling technologies track

– Focus on HPC

• Exascale algorithms

• Data analytics

• Machine Learning

– Still require exposure to applications but not

the focus

Summary

• CSE education requires exposure to

– Applications

– Mathematics

– Computer Science

– Immersion in multidisciplinary team research

• Examples exist

• CSGF is getting it right!