for more information ... http://www.tops-scidac.org

unscalable

scalable

Problem Size (increasing with number of processors)

Tim

e to

So

luti

on

200

150

50

0

100

10 100 10001

Convergence rate nearly independent of discretization parameters

Multilevel schemes for linear and nonlinear problems Newton-like schemes for quadratic convergence of nonlinear problems

Convergence rate as independent as possible of physical parameters

Continuation schemes Asymptotics-induced, operator-split preconditioning

Keyword, key challenge: “Optimal”

Scalable SolversIn support of SciDAC fusion, astophysical, combustion, and other simulations, TOPS is creating a new generation of solvers for PDE field problems.

Many DOE mission-critical systems are modeled by PDEs Finite-dimensional models of PDEs must be large for

accuracy Qualitative insight is not enough (Hamming notwithstanding) Simulations must resolve policy controversies

Advances in algorithms are at least as important as advances in hardware, in supporting simulation

Easily demonstrated for PDEs in the period 1945–2000 Continuous problems provide exploitable hierarchy of

approximation models, creating hope for “optimal” algorithms

Software lags both hardware and algorithms

Data Layout

structured composite block-struc unstruc CSR

Linear Solvers

GMG, ... FAC, ... Hybrid, ... AMGe, ... ILU, ...

Linear System Interfaces

PETSc codeUser code

ApplicationInitialization

FunctionEvaluation

JacobianEvaluation

Post-Processing

PC KSP

Main Routine

Linear Solvers (SLES)

Nonlinear Solvers (SNES)

Timestepping Solvers (TS)

ADIC code

Interoperability

TOPS brings together and will make interoperable some of the most popular solver software toolkits in the DOE, such as Hypre, PETSc, and SUNDIALS. TOPS solvers will also interoperate with APDEC and TSTT codes.

Multiple interfaces

TOPS’s conceptual interfaces (from Hypre, below) allow users to access its multilevel solvers from data structures close to the applications. TOPS’s interface to automatic differentiation tools (through PETSc, below right) provides rapid nonlinear solution and optimization, all matrix-free.

0

100

200

300

400

500

600

700

3 12 27 48 75

ASM-GMRESAMG-FMGRES

0

10

20

30

40

50

60

3 12 27 48 75

ASM-GMRESAMG-FMGRESAMG inner

size, procs

size, procs

iters

time

Algebraic multigrid (AMG) above, shows perfect iteration scaling, above, in contrast to additive Schwarz (ASM), but still needs performance work to achieve temporal scaling, below, on CEMM fusion code, M3D

PDE solver software is strategic to SciDAC Applications

PERC, CCA

TSTTAPDEC

TOPS

SS

SDM

How SciDAC apps engage TOPS solvers Directly (now)

Apps code sets up own discretization, possibly built on a grid made up of distributed objects from PETSc (like M3D)

Apps code calls a TOPS solver, possibly with explicit matrix elements, or in a Jacobian-free mode

Through APDEC or TSTT (coming in 2003) Apps code calls on Chombo, Overture, Trellis, etc., to express its PDEs with

an automatically adapted discretization

Through componentization (coming later) Apps code, discretization frameworks, TOPS solvers are all peer components

interacting in a Common Component Architecture framework

Benefits to apps With solver, get stability analysis and sensitivity analysis functionality

Early TOPS partnersTOPS has many application partners, including the Center for Extended Magnetohydrodynamic Modeling (CEMM, left), the Center for Magnetic Reconnection Studies (CMRS, below left), and the Terascale Supernovae Initiative (TSI, below right). For CEMM, TOPS’s scalable linear solvers power linear solvers inside an operator-split time integration of tokamak dynamics. For CMRS, TOPS has developed a fully implicit nonlinear capability, permitting accurate implicit time stepping that exceeds the Courant stability limit for an explicit method. For TSI, TOPS is extending TSI’s 1D operator-split solvers to 2D and 3D operator-split and nonlinearly implicit, both.