ad2016 - the 7th international conference on algorithmic ... · ad-suite - a test suite for ... the...

AD2016 - The 7th International Conference onAlgorithmic DifferentiationProgramme and Abstracts

Conference Chairs:Patrick Farrell (University of Oxford, UK)

Shaun Forth (Cranfield University, UK)Andreas Griewank (Yachay Tech, Ecuador)

Paul Hovland (Argonne National Laboratory, USA)Jens-Dominik Muller (Queen Mary University of London, UK)

Monday 12th - Thursday 15th September 2016Christ Church Oxford, UK

Invited Speakers

I Paul I. Barton - Computationally Relevant GeneralizedDerivatives: Theory, Evaluation and Applications

I Jacques du Toit - Applying AD in the Finance Industry, andan Operator Overloading Tool for “Handwritten” Adjoints

I Patrick Farrell - Automatic parallel adjoints of finite elementsimulations with dolfin-adjoint

I Daniel Goldberg - Automatic differentiation of current icesheet models

I Siegfried M. Rump - Reliable Computing and AD inMatlab/Octave

I Andrea Walther - Covering resilience: A recent developmentfor binomial checkpointing

Programme CommitteeChristian Bischof TU Darmstadt, GermanyMartin Bucker Friedrich-Schiller-Universitat Jena, GermanyLuca Capriotti Credit Suisse, UKBruce Christianson University of Hertfordshire, UKBoris Diskin National Institute of Aerospace, USANicolas R Gauger TU Kaiserslautern, GermanyRalf Giering FastOpt, GermanyLaurent Hascoet INRIA, FrancePatrick Heimbach University of Texas at Austin, USASri Hari Krishna Narayanan Argonne National Laboratory, USAUwe Naumann RWTH Aachen University, GermanyBoyana Norris University of Oregon, USAValerie Pascual INRIA, FranceEric T Phipps Sandia National Laboratories, USAOlivier Pironneau Pierre and Marie Curie University, FranceDetlef Stammer University of Hamburg, GermanyTrond Steihaug University of Bergen, NorwayEmmanuel Tadjouddine University of Leicester, UKJean Utke Allstate, USAAndrea Walther Universitat Paderborn, GermanyHaibin Zhang Beijing University of Technology, China

Acknowledgements

The Chairs of AD2016 wish to thank the following organisationsfor their financial support to AD2016:

I SIAM/NSF for travel grants to invited speaker Paul I. Barton,two early career researchers and two students

I SIAM United Kingdom and Republic of Ireland section forsponsoring a student prize

I Numerical Algorithms Group Ltd. for their support of invitedspeaker Jacques du Toit

I Taylor & Francis for post-conference publication of selectedpapers arising from AD2016 in their journal OptimizationMethods and Software.

Acknowledgements (ctd.)

The Chairs also wish to express their thanks to:

I the members of the Programme Committee for their help inchoosing the venue, selecting the Invited Speakers andrefereeing the extended abstracts

I Joanna Malton, Conference and Events Administrator atChrist Church Oxford, and her team for their professionalorganisation of the conference.

Monday 12th September 201608:30 Registration and Opening

Piecewise linear and non-smooth I - Chair Andrea Walther08:50 Kamil Khan Branch-locking AD techniques for composite nonsmooth functions and nonsmooth im-

plicit functions09:10 Richard Hasenfelder An Algorithmic Piecewise Linearization based Method for Nonsmooth Dynamical Sys-

tems09:30 Tom Streubel Newton-like Methods for Solving Piecewise Smooth Systems of Equations Based on

Successive Piecewise Tangent or Secant Linearization09:50 Break

Invited Presentation - Chair Andreas Griewank10:20 Siegfried M. Rump Reliable Computing and AD in Matlab/Octave

Piecewise linear and non-smooth II - Chair Andreas Griewank11:20 Sabrina Fiege A Method for the Minimization of Piecewise Smooth Objective Functions11:40 Koichi Kubota Simple implementation and examples for piecewise linearization with abs-normal form12:00 Poster Presentations I - Chair Shaun Forth

Laurent Hascoet Piecewise Linear AD via Source TransformationManuel Radons Additions to an AD-Based Linearization FrameworkJean Utke A Nonsmoooth Optimization Case StudyBenjamin Jurgelucks Parallel Algorithmic Differentiation in OpenModelica

12:20 Lunch in the Great HallInvited Presentation - Chair Paul Hovland

13:50 Paul I. Barton Computationally Relevant Generalized Derivatives: Theory, Evaluation and ApplicationsParallelisation and Benchmarking I - Chair Paul Hovland

14:50 Markus Towara SIMPLE Adjoint Message Passing15:10 Daniel Walther Transforming OpenMP Programs by Automatic Differentiation: Beyond Preserving the

Parallelization Specified in the Original Program15:30 Break

Parallelisation and Benchmarking II - Chair Patrick Farrell16:00 Jan Huckelheim Exact replication of OpenMP-parallelism in reverse-mode AD for loops with symmetric

memory access16:20 Mahesh Narayana-

murthiAD-Suite - A Test Suite for Algorithmic Differentiation

16:40 Filip Srajer A Benchmark of Selected Algorithmic Differentiation Tools on Some Problems in Ma-chine Learning and Computer Vision

17:00 Shaun Forth A Matlab implementation of the Minpack-2 Test Problem Collection17:20 Close

Tuesday 13th September 2016And now for something completely ... - Chair Chris Bischof

08:30 Bradley Bell The Newton Step Method for Algorithmic Differentiation with Implicit Functions08:50 Toshio Fukushima Algorithmic Computation of Finite Difference09:10 Barak A Pearlmutter Tricks from Deep Learning09:30 Paul Hovland A Forward Mode Version of the CENA Method for Roundoff Error Estimation09:50 Break

Invited Presentation - Chair Laurent Hascoet10:20 Daniel Goldberg Automatic differentiation of current ice sheet models

Iterative Systems - Chair Laurent Hascoet11:20 Bruce Christianson Differentiating through Conjugate Gradient11:40 Siamak Akbarzadeh Consistent treatment of incompletely converged iterative linear solvers in reverse-mode

AD12:00 Poster Presentations II - Chair Jens-Dominik Muller

Jonathan Huser KKT-based integration of differential-algebraic equations with optimality criteriaJohannes Lotz Mixed Integer Programming for Call Tree TraversalBarak A. Pearlmutter Evolving the Incremental Lambda Calculus Into a Model of Foward AD

12:20 Lunch in the Great HallAD for Other Languages and Interoperability I - Chair Jean Utke

13:50 Kshitij Kulshreshtha Interfacing source transformation AD with operator overloading libraries14:10 Dominic Jones Block Scope Differentiation14:30 Valerie Pascual Mixed-language Automatic Differentiation14:50 Laurent Hascoet Interfacing OpenAD and Tapenade15:10 Jarrett Revels Forward-Mode Automatic Differentiation in Julia15:30 Break

AD for Other Languages and Interoperability II - Chair Valerie Pascual16:00 Jeffrey Mark Siskind Efficient Implementation of a Higher-Order Language with Built-In AD16:20 Feng Qiang On efficient Hessian computation using the edge pushing algorithm in Julia16:40 Atilim Gunes Baydin DiffSharp: An AD Library for .NET Languages17:00 Kshitij Kulshreshtha Wrappers for ADOL-C in scripting languages using SWIG17:20 Close

Wednesday 14th September 2016Exploiting Structure I - Chair Bruce Christianson

08:30 Alex Pothen Enhancing sparsity of Hermite polynomial expansions with Automatic Differentiation08:50 Uwe Naumann Adjoint Code Design Patterns09:10 Jeffrey Mark Siskind Binomial Checkpointing for Arbitrary Programs with No User Annotation09:30 Viktor Mosenkis On Lower Bounds for the Optimal Jacobian Accumulation Problem on linearized DAGs09:50 Break

Invited Presentation - Chair Andreas Griewank10:20 Andrea Walther Covering resilience: A recent development for binomial checkpointing

Exploiting Structure II - Chair Andreas Griewank11:20 Martin Bucker First-order Derivatives of Associated Legendre Functions11:40 Mohammad Ali Ros-

tamiBidirectional Partitioning is no Better than Unidirectional Partitioning when Computingthe Diagonal Entries of a Sparse Jacobian Matrix

12:00 Poster Presentations III- Chair Jens-Dominik MullerAlexander Huck A Case Study of ADOL-C and CoDiPack applied to the Ice Sheet System ModelGuillaume Sal Vibrato and Automatic Differentiation for High Order Derivatives and Sensitivities of

Financial OptionsGilberto Perez An Approach to Computing Discrete Adjoints for MPI-Parallelized Models Applied to

the Ice Sheet System Model12:20 Lunch in the Great Hall

Invited Presentation - Chair Uwe Naumann13:50 Jacques du Toit Applying AD in the Finance Industry, and an Operator Overloading Tool for “Handwrit-

ten” AdjointsExploiting Structure III- Chair Uwe Naumann

14:50 John Pryce How AD Can Help Solve Differential-Algebraic Equations15:10 Alexander Huck Operator Overloading Compatibility for AD - A Case Study of Scientific C++ Codes15:30 Break + Wednesday Posters

C++ - Chair Martin Bucker16:20 Eric Phipps Operator Overloading-based Automatic Differentiation of C++ Codes on Emerging

Manycore Architectures16:40 Max Sagebaum Expression templates for primal value taping in the reverse mode of AD17:00 Laurent Hascoet Source-transformation Differentiation of a C++-like Ice Sheet model17:20 Close18:15 Drinks in the Masters Garden19:00 Conference Dinner in the Great Hall

Thursday 15th September 2016Applications - Chair Ralf Giering

08:30 Mladen Banovic Automatic Differentiation of a CAD System applied in an Adjoint CFD Method08:50 Lisa Kusch Semi-automatic transition from simulation to one-shot optimization with equality con-

straints09:10 Marc Schwalbach Developments of a Discrete Adjoint Structural Solver for Shape and Composite Material

Optimization09:30 Benjamin Jurgelucks Optimisation of triple-ring-electrodes on piezoceramic transducers using algorithmic dif-

ferentiation09:50 Break

Invited Presentation - Chair Paul Hovland10:20 Patrick Farrell Automatic parallel adjoints of finite element simulations with dolfin-adjoint

Higher Order Derivatives I- Chair Paul Hovland11:20 Jens Gustedt Arbogast - Higher order AD for special functions with Modular C11:40 Mu Wang Performance Evaluation of Automatic Differentiation Algorithms for Hessian Computa-

tion12:00 Poster Presentations IV- - Chair Jens-Dominik Muller

Viktor Mosenkis Adjoint NAG LibraryArindam Sen Discrete adjoint OpenFoam for surface and volume sensitivitiesOlivier Mullier Validated Computation of the Local Truncation Error of Runge-Kutta Methods with

Automatic Differentiation12:20 Lunch in the Great Hall

Higher Order Derivatives II - Chair Andrea Walther13:50 Richard Neidinger Comparing High-Order Multivariate AD Methods14:10 Shahadat Hossain Pattern Graph for Sparse Hessian Matrix Determination14:30 Mu Wang Evaluating High Order Derivative Tensors in Reverse Mode of Automatic Differentiation14:50 Isabelle Charpentier A full higher-order AD continuation and bifurcation framework15:10 Closing Remarks15:30 End of Conference

Monday - Poster Session 1

Monday - Poster Session 1Piecewise Linear AD via Source Transformation

Torsten Bosse, Sri Hari Krishna Narayanan and LaurentHascoet


Piecewise Linear ADvia Source Transformation

Torsten Bosse Krishna Narayanan Laurent Hascoet

ANL/ANL/INRIA

AD2016 Poster Session

Bosse/Narayanan/Hascoet (ANL/ANL/INRIA) Piecewise Linear AD Posters AD2016 1 / 2

0

-1

0.2

0.4

-0.5

F(x

)

1

0.6

x1

Original function

0.8

0 0.5

1

x2

00.5

-0.51 -1

-0.2

-1

0

0.2

-0.5

∆F

(x;∆

x)

1

0.4

∆x1-x

1

Piecewise linear model at x=(0.5,0.5)

0.6

0 0.5

0.8

∆x1-x

2

00.5

-0.51 -1

Nonsmooth issues: need notpresenting, monday morning did it!

Andreas &al ”Abs Normal Form”

Torsten+Krishna+myself: ANF canbe built with ST-AD quite easily(yet another ANF explanation ?)

Just replace ˙ABS with ahand-written 10-liner !

. . . and it works !⇐ example with OpenAD

Bosse/Narayanan/Hascoet (ANL/ANL/INRIA) Piecewise Linear AD Posters AD2016 2 / 2

Monday - Poster Session 1Additions to an AD-Based Linearization

Framework

Manuel Radons


Perturbation-stable Surjectivity of Piecewise

Affine Functions

Manuel Radons

Institute for Applied Mathematics, Humboldt Universitat zu Berlin, [email protected]

AD2016 - 7th International Conference on AlgorithmicDifferentiation

Monday 12th - Thursday 15th September 2016; Christ Church Oxford, UK.

12.9.2016

Manuel Radons (HUB) Perturbation-stable Surjectivity 12.9.2016 1 / 3

Generalities

What did we investigate?Piecewise affine functions

F : Rn −→ Rn

with finite corresponding subdivision P(F ) of the domain.

What did we find out?Sepcific: New criteria for sujectivity and perturbation-stablesurjectivity.

Specific: Sharpness of bounds.

General: Behaviour of F largely determined by behaviour onunbounded sets of P(F ).

Manuel Radons (HUB) Perturbation-stable Surjectivity 12.9.2016 2 / 3

Specifics

Pointed special caseSufficient condition for perturbation-stable surjectivity: F iscoherently orientated on polyhedra of P(F ).

General topological resultsAnalogon to mapping degree on compact manifolds isconstructed.

F with nonzero degree are stably surjective.

F with zero degre may be surjective, but not necessarily stablysurjective.

Bonus: Short proofs for classical resultsInjective⇒ Coherently Oriented⇒ Surjective.

Coherently oriented F are branched coverings.Manuel Radons (HUB) Perturbation-stable Surjectivity 12.9.2016 3 / 3

Monday - Poster Session 1A Nonsmoooth Optimization Case Study

Jean Utke, Sabrina Fiege, Andrea Walther and KshitijKulshreshtha


Monday - Poster Session 1Parallel Algorithmic Differentiation in

OpenModelica

Bernhard Bachmann, Willi Braun, Martin Flehmig, RudigerFranke, Leo Gall, Benjamin Jurgelucks, Kshitij Kulshreshtha,

Kilian Link, Ulf Markwardt, Vitalij Ruge, Andrea Walther, andNiklas Worschech


Tuesday - Poster Session 2

KKT-based integration of differential-algebraicequations with optimality criteria

Tuesday - Poster Session 2Jonathan Huser Jens Deusseny and Uwe Naumann


STCEDAE with Optimality Criteria (DAEO)

I Problem formulation, case study of atmospheric aerosol particle

x = f(t,x,y,p), t ∈ (t0, tf ], x(t0,p) = x0(p)

y ∈ argminy∈Rnyw(t,x, y,p)

s.t. 0 = gi(t,x, y,p), i = 1, . . . , ng

0 ≤ hi(t,x, y,p), i = 1, . . . , nh

I Integration tool chain, some numerical challenges are discussed

,

STCE, RWTH Aachen University, Integration of DAEO, AD 2016 1

Tuesday - Poster Session 2Mixed Integer Programming for Call Tree

Traversal

Johannes Lotz, Uwe Naumann and Sumit Mitra


STCE

Mixed Integer Programming for Call Tree Reversal

Johannes Lotz, Uwe NaumannSoftware and Tools for Computational EngineeringRWTH Aachen University

Sumit MitraChemical EngineeringCarnegie Mellon University

September 13, 2016

STCEOne Slide: MIP for CTR

,

AD2016: J. Lotz, Mixed Integer Programming for Call Tree Reversal 2/2

Tuesday - Poster Session 2Evolving the Incremental Lambda Calculus Into a

Model of Foward AD

Robert Kelly, Barak A. Pearlmutter and Jeffrey Mark Siskind


Why?

Evolving the Incremental λ Calculusinto a Model of Forward AD

Robert Kelly1 Barak A. Pearlmutter1 Jeffrey Mark Siskind2

1Maynooth University 2Purdue University

D (λ x . f (g x)) = (λ x x′ . D f (g x) (D g x x′))

D ( f ◦ g) x = D f (g x) ◦D g x

How?

Incrementalλ-Calculus

PowerSeries

DualNumbers

BundledILC

Higher-OrderForward AD

ForwardAD

∆R = R[ε]

uncu

rry

uncu

rry

∆R = R[ε]/ε2

∆R = R[ε] ∆R = R[ε]/ε2

uncu

rry

Wednesday - Poster Session 3

Wednesday - Poster Session 3A Case Study of ADOL-C and CoDiPack applied

to the Ice Sheet System Model

Tim Albring, Christian Bischof, Nicolas R. Gauger, AlexanderGraf, Alexander Huck, Benjamin Jurgelucks, Max Sagebaum

and Jean Utke


14th September 2016

Study of ADOL-C and CoDiPack applied to theIce Sheet System Model Alexander Hück, Christian Bischof, Tim Albring, Nicolas R. Gauger, Max Sagebaum, Benjamin Jurgelucks

The Ice Sheet System Model (ISSM) is an ice flow modeling software

developed by NASA/JPL employs ADOL-C for, e.g., sensitivity studies

Thus, ISSM provides a validated AD implementation We explore the AD tool CoDiPack

Performance improvements Feature completeness

14th September 2016

Study of ADOL-C and CoDiPack applied to the Ice Sheet System Model

Wednesday - Poster Session 3Vibrato and Automatic Differentiation for HighOrder Derivatives and Sensitivities of Financial

Options

Gilles Pages, Olivier Pironneau and Guillaume Sal


Wednesday - Poster Session 3An Approach to Computing Discrete Adjoints forMPI-Parallelized Models Applied to the Ice Sheet

System Model

Gilberto Perez, Eric Larour, Jean Utke, Anton Bovin andMathieu Morlighem


A D 2 0 1 6

AD2016

An Approach to Computing Discrete Adjoints for MPI-Parallelized Models Applied to the Ice Sheet System Model

Gilberto PÉREZ Eric Larour, Jean Utke, Anton Bovin, Mathieu Morlighem

September 2016

A D 2 0 1 6 J P L / U C I R V I N E

My Poster

Automatic Differentiation or Algorithmic Differentiation(AD) provides the ability to calculate derivatives without truncation and approximation of finite difference alternatives and is much less error prone and time consuming than manually programming their computations. Take the following generic function example with n independent variables and m dependent variables:

With AD, developers need only worry about implementing the function itself, while still having access to any derivative. Otherwise, any modification to the implementation of the function f would require equivalent changes to the implementation of the derivatives. For example, any of the following would be readily available:

● Gradient: 𝛻 f● Jacobian: J f● Directional Derivative: 𝛻 f · u

Access to these derivatives is incredibly useful. Some of these uses include:

● Sensitivity Analysis ● Uncertainty Quantification ● Optimization Methods● Data Assimilation

Forward VS Reverse

Obviously, computing these derivatives comes at a computational cost. The cost is dependent on how the derivatives are computed. In general, there are two major approaches:

● Forward Mode ● Reverse Mode

The forward mode is relatively simple and involves computing all the derivatives that one is interested in simultaneously. This is equivalent to applying the chain rule all the way through the computation. Unfortunately, this results in a large computational penalty proportional to the number of derivatives required. For example, if we were interested in the gradient of f, then we would need to compute n partial derivatives. Consequently, we have a computational complexity of:

O(O(f(n)) · n)

On the other hand, the reverse mode involves delaying computing derivatives. Instead of computing the derivatives as we go, intermediate results are stored( known as the trace) until the very end. Finally, the derivatives of interest can be computed by applying the chain rule in reverse. The result is a computation that is impacted by the number of dependent variables instead:

O(O(f(n)) · m)

This result is particularly powerful when, like in our case, we only have one dependent variable.

Source to Source VS Operator Overloading

AD tools come in two flavors:

● Source to Source● Operator Overloading

They both offer the same functionality, but take two very different approaches. Source to source works by taking the source code, producing an intermediate representation (IR), modifying the IR so as to calculate derivatives of interested variables, then convert the IR back to the source language. The resulting code can then be compiled normally.

Operator Overloading works by overloading basic operations like addition with functions that compute the result along with the derivative.

Unfortunately, there is no source to source tool available for C++. As such, the choice of an Operator Overloading tool like ADOL-C was obvious to ISSM developers [2].



AbstractAbstract

Gilberto L. Pérez1, Eric Y. Larour2, Jean Utke3, Anton Bovin4, Mathieu Morlighem1

Background: Ice ModelingBackground: Ice Modeling

AD ParallelismAD Parallelism BenchmarkingBenchmarkingWithin the framework of sea-level rise projections, there is a strong need for hindcast validation of the evolution of ice sheets in a way that tightly matches observational records. However, the computational cost for making hindcast reconstructions possible are severe and rely on the evaluation of the adjoint state of transient ice-flow models. Here, we look at the computation of adjoints in the context of the Ice Sheet System Model(ISSM), written in C++ and designed for parallel execution with MPI. We concentrate on the use of operator overloading coupled with the AdjoinableMPI library to achieve the adjoint computation of ISSM. We present a comprehensive approach to:

1) Carry out type changing through ISSM, hence facilitating operator overloading, 2) Bind to external solvers such as MUMPS and GSL-LU3) Handle MPI-based parallelism to scale the capability.

We demonstrate the success of the approach by computing sensitivities of hindcast metrics such as the misfit to observed records of surface altimetry on the North-East Greenland Ice Stream. We also provide metrics for the scalability of the approach, and the expected performance. This approach has the potential of enabling a new generation of hindcast-validated projections that make full use of the wealth of datasets currently being collected, or already in Greenland and Antarctica.

Background: ADBackground: AD

f :ℝn→ℝm

The two ice sheets of the world in Greenland and Antarctica contain 99% of the world’s fresh water. Scientists project that if all the ice contained in these ice sheets were to melt, sea levels would rise by 6 and 20 meters, respectively. Furthermore, approximately 40% of the world’s population live within 100Km of a coast. Consequently, it is imperative that we can project how the ice sheets will change in the coming decades. It is exactly this goal that ISSM aims for.

However, modeling ice sheets involves complicated physics such as ice calving, basal drag at the bed and grounding line migration. Moreover, the behavior of ice is inexorably coupled with the behavior of the oceans and atmosphere.

Image displaying some of the many processes involved in modeling ice sheets

Fortunately, the system of partial differential equations governing ice flow are fairly well understood. These governing equations are known as Stokes Flow [5]:

However, Stokes flow equations are often over kill. For this reason, ISSM also makes use of increasingly approximate equations:

● Higher Order Approximation● Shallow Shelf Approximation● Shallow Ice Approximation

Nevertheless, solving these systems is inherently onerous. Consequently, ISSM opts for approximating solutions to an arbitrary degree of precision by utilizing the Finite Element Method(FEM) [5].

∂∂ x

(2μ∂ v x

∂ x)+ ∂

∂ y(μ

∂ v x

∂ y+μ

∂ v y

∂ x)+ ∂

∂ z(μ

∂ v x

∂ z+μ

∂ v z

∂ x)−

∂ρ

∂ x=0

∂∂ x

(μ∂ vx

∂ y+μ

∂ v y

∂ x)+ ∂

∂ y(2μ

∂ v y

∂ y)+ ∂

∂ z(μ

∂ v y

∂ z+μ

∂ v z

∂ y)−

∂ρ

∂ y=0

∂∂ x

(μ∂ vx

∂ z+μ

∂ v z

∂ x)+ ∂

∂ y(μ

∂ v y

∂ z+μ

∂ vz

∂ y)+ ∂

∂ z(2μ

∂ v z

∂ z)−

∂ρ

∂ z=0

∂ v x

∂ x+

∂ v y

∂ y+

∂ v z

∂ z=0

Processor clock speeds have essentially stalled over the last decade. Instead, improvements in computational horse power have mean increased number of cores per CPU. Moreover, most of the best super computers in the world are computational clusters. Consequently, it is imperative that any model or simulation take advantage of parallelism, which comes in two basic forms:

● Shared Memory Model: PTHREADS● Message Passing Model: Message Passing Interface(MPI)

The shared memory model works best in single machine scenarios like workstations. Otherwise, it is quite involved to deploy on a cluster or grid. For this reason, ISSM uses MPI implementations like MPICH. However, this presents the problem of how to generate the trace efficiently. The four possibilities are as follows:

AdjoinableMPI

Development of AdjoinableMPI began in 2012. It is designed to provide an AD tool-independent implementation for adjoining MPI-parallelized C, C++ and Fortran models. The wrapper interfaces distinguish themselves from the original MPI by the prefix AMPI and a few additional parameters where needed to enable the adjoint functionality. AMPI provides also additional types and predefined symbols [1]. The following is a short code example:

int ISSM_MPI_Reduce(void* sendbuf, void* recvbuf, int count, ISSM_MPI_Datatype datatype, ISSM_MPI_Op op , int root,

ISSM_MPI_Comm comm) {int rc = 0;

#ifdef _HAVE_MPI_#ifdef _HAVE_AMPI_

Rc = AMPI_Reduce(sendbuf, recvbuf, count, datatype, op, root, comm);# else

Rc = MPI_Reduce(sendbuf, recvbuf, count, datatype, op , root, comm);#endif#else#ifdef _HAVE_ADOLC_

if(datatype == ISSM_MPI_DOUBLE) {IssmDouble* activeSendBuf = (IssmDouble*) sendbuf;IssmDouble* activeRecvBuf = (IssmDouble*) recvbuf;

for(int i=0; i < count; ++i) activeRecvBuf[i] = activeSendBuf[i];}else

#endifmemcpy(recvbuf, sendbuf, sizeHelper(datatype)* count);

#endifreturn rc ;

}

The end result is code that is both parallel and differentiable; a powerful combination [6].

We developed a new adjoint capability in ISSM, based on the ADOL-C framework and the AdjoinableMPI library, which is to our knowledge, the first time this type of approach has been systematically applied to a software framework of this size and complexity. Despite the difficulties encountered rewriting the software, the overloaded approach is totally transparent to the user, which is critical given the size of the larger Cryosphere Science community that is not familiar with the adjoint work, and for which classic approaches such as source-to-source transformation have proven to be overly cumbersome. The flexibility of this approach allows for quick turn-around in developing adjoint models of new parameterizations which are not easily hand derived. This is a massive advantage in that it opens this approach to the wider community. This could prove paramount if we are to hindcast validate projections of sea-level rise. Further work is required to bring in additional observations such as gravity sensors, or radar stratigraphy observations, which will involve development of new cost functions, and scalability in 3D. Though this is complex in that it requires integrated resiliency and adjoint checkpointing schemes for long running transient modeling scenarios, our approach has proven flexible, and should lead to a brand new set of data assimilation capabilities that have already been available to other Earth Science communities for a long time. Indeed, by allowing temporal data assimilation for a large number of sensors and models, ISSM paves the way for wider integration between the modeling and observational Cryosphere community.

Binding to External SolversBinding to External Solvers

ConclusionConclusion

ReferencesReferences

AD MPI

1 No No No derivatives and model runs sequentially

2 No Yes No derivatives and model runs in parallel

3 Yes No Derivatives, but everything runs sequentially

4 Yes Yes Derivatives and everything runs sequentially

Models built around Finite Element Method inherently require solving linear systems. The process involves populating so-called stiffness matrices and load vectors (related to boundary conditions) and then solving the system. Considering the system and accompanying solver:

This needs an adjoint counterpart, since this is a part of the code that needs to be differentiated. However, solvers are specifically tuned to work fast with specific data types. As such, we would like to avoid applying AD to the solvers themselves. Fortunately, we can reuse the same solvers to calculate the adjoint [6]:

A s=rs=S (A , r)

t=S(AT , s)

The benefits of AD are great, but if the performance does not scale well, then it loses its value. As such, benchmarking is necessary. The bottleneck for ISSM, without AD, is always the linear system solvers. Consequently, we will compare performance against the following solvers:

● GNU Scientific Library(GSL): Provides sequential solver [3]● Multi-Frontral Massively Parallel Sparse Direct Solver(MUMPS) [4]

GSL is easy to use, but not very efficient. As such, the AD overhead compares well:

Plots over max mesh distance. Approx. run-time portions of execution with adjoint computation(left). Overhead factors(right).

Plots over max mesh distance. Approx. run-time portions of execution with adjoint computation(left). Overhead factors the trace creation and interpretation (right)

Clearly, the use of a specialized solver like MUMPS makes the adjoint computation blow up in terms of proportion of run-time. However, we note that it does level off and will likely scale [6].

[1] AdjoinableMPI: ://trac.mcs.anl.gov/projects/AdjoinableMPI/wiki.

[2] ADOL-C: http://www.coin-or.org/projects/ADOL-C.xml.

[3] Amestoy, P. R., Duff, I. S., Koster, J., and L’Excellent, J.-Y.: A Fully Asynchronous Multifrontal Solver Using Distributed Dynamic Scheduling, SIAM J. Matrix Anal. Appl., 23, 15–41, 2001.

[4] Galassi, M. e. a.: GNU Scientific Library Reference Manual, Network Theory Ltd, 3rd ed edn., http://www.gnu.org/software/gsl/manual/gsl-ref.ps.gz, 2009.

[5] Larour, E., Seroussi, H., Morlighem, M., and Rignot, E.: Continental scale, high order, high spatial resolution, ice sheet modeling using the Ice Sheet System Model (ISSM), J. Geophys. Res., 117, 1–20, doi:10.1029/2011JF002140, 2012.

[6] Larour, E., Utke, J., Bovin, A., Morlighem, M., and Perez, G.: An Approach to Computing Discrete Adjoints for MPI-Parallelized Models Applied to the Ice Sheet System Model, Geosci. Model Dev. Discuss., doi:10.5194/gmd-2016-99, in review, 2016.

1 University of California, Irvine, 2 Jet Propulsion Laboratory, 3 Allstate Insurance Company, 4 The University of Chicago

G. Pérez — An Approach to Computing Discrete Adjoints for MPI-Parallelized Models Applied to the Ice Sheet System Model 1/1

Thursday - Poster Session 4

Thursday - Poster Session 4Adjoint NAG Library

Johannes Lotz, Klaus Leppkes, Viktor Mosenkis and UweNaumann


STCE

The NAG AD Library

Adjoint version of the NAG Library routines

Viktor Mosenkis, The Numerical Algorithm Group (NAG)Klaus Leppkes, Johannes Lotz and Uwe NaumannSoftware and Tools for Computational Engineering, RWTH Aachen University

September 14, 2016

STCEThe AD NAG Library

NAG AD Library

dco/c++ Tape

dco/fortran

dco/c++

overloaded routines

by name InterfaceCode using

source transformation

C++ Code using dco/c++

Fortran Code using dco/fortran

,

AD2016: The NAG AD Library 2/2

Thursday - Poster Session 4Discrete adjoint OpenFoam for surface and

volume sensitivities

Arindam Sen, Markus Towara and Uwe Naumann


Discrete Adjoint OpenFOAM for Surface andVolume sensitivities

Arindam Sen, Markus Towara, Uwe Naumann

LuFG Informatik 12: Software and Tools for Computational EngineeringRWTH Aachen

Thursday - Poster Session 4Validated Computation of the Local TruncationError of Runge-Kutta Methods with Automatic

Differentiation

Olivier Mullier, Alexandre Chapoutot and Julien Alexandre DitSandretto


Validated Computation of the Local TruncationError of Runge-Kutta Methods with Automatic

Differentiation

Olivier Mullier1 Alexandre ChapoutotJulien Alexandre Dit Sandretto

ENSTA ParisTech

[email protected]

ContextNew method for the validated solution of initial value problems.

MethodAutomatic computation of Taylor components to validateRunge-Kutta methods.

Bounding the Local Truncation Error (difference between truesolution and numerical solution)

Butcher theory : elementary differentialsLink with rooted trees and Taylor components of an univariatefunction.

Resultsreliable computing,time complexity improvement compared to previous method,make possible to consider large state space dimension andlarge order Runge-Kutta methods.

O. Mullier, A. Chapoutot, J. Alexandre Dit Sandretto Computation of the LTE of RK Methods with AD

SIAM UK-EI Prize - Best Student Presentation

I Awarded to those studying for, or presenting work from, theirPhD (or lower qualification study, or something like that).

I Panel:I Andrea WaltherI Paul HovlandI Bruce Christianson

I And the winner is . . .

Collected Presentations of AD2016

I Martin Bucker has agreed to host on www.autodiff.org

I Can all presenters (oral and poster) happy to do so email pdfpresentations to Shaun.

I Please use naming convention

[family name] [first significant word in title].pdf

eg, Shaun Forth presenting A Matlab implementation of theMinpack-2 Test Problem Collection would use

Forth_Matlab.pdf

I Please remove any photos etc for which you do not havecopyright.

Post-Conference Publication

I Andreas Griewank will edit a special edition of OptimizationMethods and Software

I 30 December 2016 - submission of papers to OptimizationMethods and Software

I Open to invited, oral and poster presenters at AD2016 butsubmissions must be in line with journal Aims - seehttp://www.tandfonline.com/toc/goms20/current

I Usual refereeing policy will apply.

http://www.tandfonline.com/toc/goms20/current

Euro AD Workshop - Spring 2016

Hosted by Laurent Hascoet

INRIA, Sophia Antipolis, France

Spring 2017

AD2019 or AD2020The 8th International Conference on

Algorithmic Differentiation

Hosted by - To be Confirmed (not Shaun or Patrick)

West USA, East USA, Europe (not UK due to Brexit), or ...

Summer 2019 or Summer 2020 or, ....

ad2016 - the 7th international conference on algorithmic ... · ad-suite - a test suite for ... the...

Documents