what is esmf and what does it mean to adopt it? 3 rd esmf community meeting cecelia deluca...
TRANSCRIPT
What is ESMF and what does it mean to adopt it?
3rd ESMF Community Meeting
Cecelia DeLuca [email protected]
Nancy Collins [email protected]
Jon Wolfe [email protected]
ClimateData Assimilation
Weather GMAO Seasonal Forecast
NCAR/LANL CCSM
NCEP Forecast
GFDL FMS Suite
MITgcm
NASA GMAO Analysis
15 July 2004
Outline
• What is ESMF?• ESMF users• A closer look at ESMF• Adoption strategies
– A couple suggestions for how to get started
15 July 2004
ESMF Architecture
1. ESMF provides an environment for assembling geophysical components into applications, with support for ensembles and hierarchies.
2. ESMF provides a toolkit that components use to
i. increase interoperability
ii. improve performance portability
iii. abstract common services
ESMF InfrastructureData Classes: Bundle, Field, Grid, Array
Utility Classes: Clock, LogErr, DELayout, Machine
ESMF SuperstructureAppDriver
Component Classes: GridComp, CplComp, State
User Code
15 July 2004
Hierarchies and Ensembles
assim
sea iceocean
landatm
physics dycore
assim_atm
atmland
Seasonal Forecast
coupler
ESMF encourages applications to be assembled hierarchically
Coupling interfaces are standard at each layer
Components can be used in different contexts
assim_atm
Ensemble Forecast
assim_atm assim_atm
ESMF supports ensembles with multiple instances of components running sequentially (and soon, concurrently)
15 July 2004
ESMF Data Classes
Model data is contained in a hierarchy of multi-use classes. The user can reference a Fortran array to an Array or Field, or retrieve a Fortran array out of an Array or Field.
• Array – holds a Fortran array (with other info, such as halo size)
• Field – holds an Array, an associated Grid, and metadata• Bundle – collection of Fields on the same Grid• State – contains States, Bundles, Fields, and/or Arrays• Component – associated with an Import and Export State
15 July 2004
ESMF Utilities
• Communication libraries• Regridding library (parallelized, on-line SCRIP)• Message logging • Calendar management – clocks, alarms, calendars• Configuration (replaces namelists)• IO – currently netCDF• Performance profilng (not implemented yet)
15 July 2004
Outline
• What is ESMF?• ESMF users• A closer look at ESMF• Adoption strategies
– A couple suggestions for how to get started
15 July 2004
What kind of user are you?
Individual/Small Group Large Group/Institution
User
Provider
Support routines, utilities, communication code,
leverage code developed elsewhere
Full framework,inter-platform,
inter-institution solutions
Provide components to others without large overhead
Package code to run many places under many systems,
support, documentation
15 July 2004
What do you get from ESMF?• Different running configurations
– Standalone– Coupled to data only (dead components)– Coupled to live components– Strategy for structuring complicated applications
• Different communication strategies• Different data decompositions• Utilities
15 July 2004
What do you get (cont.)
• Standard component interfaces• Coupler-building utilities
15 July 2004
Outline
• What is ESMF?• ESMF users• A closer look at ESMF• Adoption strategies
– A couple suggestions for how to get started
15 July 2004
ESMF Superstructure Classes• Gridded component
– Models, data assimilation systems - “real code”
• Coupler component– Data transformations between Gridded
Components
• State - Data sent between Components• AppDriver – Generic driver
15 July 2004
ESMF Components
• “Your code goes here”• Framework specifies interface, user supplies
code which is to be called• Initialize/Run/Finalize subroutine paradigm• Can create nested subcomponents
15 July 2004
ESMF Gridded Components
• Does the computational work• Depending on how the current code is
structured, may be possible to wrap without structural changes
• Or might use ESMF conversion for excuse to make structural changes!
15 July 2004
ESMF Gridded Components (cont)• States for import/export• Sequential mode (concurrent coming soon)
– All import data available at start– All export data ready at end
• Registration routine (SetServices) to tell the framework what routines you’re supplying
15 July 2004
ESMF Coupler Components
• States for import/export• Does the transformation needed to take data
from one Component and make it fit the requirements of another Component
• Not automatic - needs to be customized for each new configuration
• Expected to be thin, however - making use of the transformation routines in ESMF
15 July 2004
ESMF States
• Description/reference to other ESMF data objects
• All data passed between Components is in the form of States and States only
• Data is referenced so does not need to be duplicated
• Can be Bundles, Fields, Arrays, States, or name-placeholders
15 July 2004
ESMF Infrastructure Data Classes
• Grids – Coordinates and grid decomposition• Arrays – Hold data• Fields – include Grid, Array, and metadata• Bundles – Fields packed together for
convenience, data locality, latency reduction
15 July 2004
ESMF DataMap Classes
These classes give the user a systematic way of expressing interleaving and memory layout, also hierarchically (partially implemented)
• ArrayDataMap – relation of array to decomposition and grid, row / column major order, complex type interleave
• FieldDataMap – interleave of vector components • BundleDataMap – interleave of Fields in a Bundle
15 July 2004
ESMF TimeMgr
• Clocks - timesteps, time intervals • Calendars - many different varieties• Alarms - periodic, fixed time• Can be used independent of other parts of
ESMF
15 July 2004
ESMF Parallelism
• VM - hardware and OS abstraction• DELayout - data decomposition• Grids (ESMF_GridDistribute)
15 July 2004
Virtual Machine (VM)
• VM handles resource allocation• Elements are Persistent Execution Threads or PETs• PETs reflect the physical computer, and are one-to-one
with Posix threads or MPI processes• Parent Components assign PETs to child Components• The VM communications layer does simpleMPI-like
communications between PETs (alternative communication mechanisms are layered underneath)
15 July 2004
DELayout• Handles decomposition• Elements are Decomposition Elements, or DEs (decomposition
that’s 2 pieces in x by 4 pieces in y is a 2 by 4 DELayout)• DELayout maps DEs to PETs, can have more than one DE per PET
(for cache blocking, user-managed OpenMP threading) • Simple connectivity or more complex connectivity, with weights
between DEs - users specify dimensions where greater connection speed is needed
• Array, Field, and Bundle methods perform inter-DE communications
15 July 2004
ESMF Communications
• Halo– Updates edge data for consistency between partitions
• Redistribution – No interpolation, only changes how the data is
decomposed• Regrid
– Based on SCRIP– Methods include bilinear, conservative, higher-order
conservative• Bundle, Field, Array-level interfaces
15 July 2004
ESMF Quickstart
• Directory with the shell of an application• 2 Gridded Components• 1 Coupler Component• 1 Top level Gridded Component• 1 AppDriver main program
15 July 2004
ESMF Quickstart Application
AppDriver (generic main program)
Top level Gridded Component
GridComp 1 GridComp 2CplComp
15 July 2004
Outline
• What is ESMF?• ESMF users• A closer look at ESMF• Adoption strategies
– A couple suggestions for how to get started
15 July 2004
Adoption Strategies: Top Down• Wrap existing application as ESMF
Component• Wrap existing data with Fields and put them
into States• Can run as a Component at this point• Integrate Field level communication• Use Time Manager, Config, LogErr
15 July 2004
Adoption Strategies: Bottom Up• Use ESMF utilities as needed• Incorporate the Time Manager, Config• Use Grids, Arrays for internal data• Use VM, DELayout routines for data decomposition• Wrap data structures as Fields• Use data communication routines like Halo, Regrid
15 July 2004
How can you learn more?
• Web site• Examples• Documentation• Interoperability experiments• Users forum
15 July 2004
What do you need to get started?• Supported build platforms include SGI, IBM,
Compaq, many Linuxes• C++ (gcc ok) and Fortran 90/95 compiler• MPI (or mpiuni substitute lib)• Understand how to run a job
15 July 2004
Where do you go from here?• Download ESMF 2.0 and compile it
– or find someone who already has it on your platform of choice!
• Run the demo• Run the quickstart application• Look at the examples• Start writing code!