the prism infrastructure for earth system...

The PRISM infrastructure

The PRISM infrastructureThe PRISM infrastructurefor Earth system modelingfor Earth system modeling

Eric Guilyardi (IPSL/CNRS and CGAM/Reading)and the PRISM Team

• Background and drivers• PRISM achievements (so far)• PRISM Support Initiative• Need for meta-data standards• How WMP can help

WCRP/WMP meeting, Exeter Oct 5-7 2005


Why aWhy a common software infrastructure ?common software infrastructure ?

• Earth system modelling expertise widely distributed

Geographically

Thematically


Why aWhy a common software infrastructure ?common software infrastructure ?

• Earth system modelling expertise widely distributed– Scientific motivation: facilitate sharing of scientific

expertise, of models and of model output (model intercomparisons are a revolution !)

– Technical motivation: the technical challenges arelarge compared with available effort

• Need to keep scientific diversity while increasingefficiency – scientific and technical

• Need for concerted effort to promote community-wide standards (WMP !)


« Share Earth System Modelling software infrastructure across community »

To:• share development, maintenance and support• aid performance on a variety of platforms• standardize model software environment• ease use of different climate model components

PRISM conceptPRISM concept


Expected benefitsExpected benefits

• high performance ESM software, developed by dedicatedIT experts, available to institutes/teams at low cost:

- helps scientists to focus on science- helps key scientific diversity (survival of smallers groups)

• Easier to assemble ESMs based on community models

• shared infrastructure = increased scientific exchanges (AR4!)

• computer manufacturers inclined to contribute:- efficiency (porting, optimisation) on variety of platforms- next generation platforms optimized for ESM needs- easier procurements and benchmarking- reduced computing costs


Software structure of an Earth System ModelSoftware structure of an Earth System Model

Running environment

Coupling+I/O infrastructure

Scientific core

Supporting software

Share(i.e. f90)


Hardware

Fortran Compiler

Earth System model(Science + support +

environment)

The long term viewThe long term view

Today

Modeller

IT expert

Standard support Library (incl. Env.)

Hardware

Fortran Compiler

Earth System model(Science)

Tomorrow

Climate science work

Towards standard ESM support library(ies)


1. a standard coupler and I/O software, OASIS3 (CERFACS) and OASIS42. a standard compiling environment (SCE) at the scripting level3. a standard running environment (SRE) at the scripting level4. a Graphical User Interface (GUI) to the SCE (PrepIFS, ECMWF)5. a GUI to the SRE for monitoring the coupled model run (SMS, ECMWF)6. standard diagnostic and visualisation tools

PRISM achievements (so far):

• Adaptation of 15+ community Earth System componentmodels (GCMs) and demonstration coupled configurations

• A well coordinated network of experts

• Community buy-in and trust-building

• Software environment (the tool box):


• providing ES models/CPU to the community

• dealing with physical interfaces for componentmodel swaping (maybe should be tackled by WMP?)

PRISM is notPRISM is not


The PRISM shellsThe PRISM shells

Standard Running EnvironmentStandard Running Environment

Standard Compile Environ.Standard Compile Environ.

PSMILe (coupling and I/O)PSMILe (coupling and I/O)

Historic I/O Historic I/O

Outer shellsOuter shells

Inner shellInner shell

Scientific coreScientific core


Adapting Earth System Components to PRISMAdapting Earth System Components to PRISM

SRESRESCESCE

UserUserInterfaceInterface

Levels of Levels of adaptation adaptation

++ PSMILe + PMIODPSMILe + PMIOD

PRISM Model Interface LibraryPRISM Model Interface LibraryPotential Model IO DescriptionPotential Model IO Description


Configuration management and deploymentConfiguration management and deployment

Binary executablesBinary executables

SCESCE

UserUserInterfaceInterface

SRESRE

Transf.

DriverDriver

disks


InternetInternet

PRISM GUI remote PRISM GUI remote functionalityfunctionality

User RepositoriesRepositories

Config.Config.

Inst

rum

ente

d In

stru

men

ted

site

ssi

tes

Transf.

DriverDriver

Architecture A

Transf.

DriverDriver

Architect. B

Transf.

DriverDriver

Architect. C

DeployDeploy

PrepIFS/SMSWeb services


ATM SMIOC V1 : from OCE, T1V2: to OCE, T2V3 : to LAND

user

LAND SMIOC V3 : from ATMV4 : from fileV4

user

OCE SMIOCV1 : to ATM, T1V2 : from ATM, T2

user

Driver

TOCE

ATM

LANDfileV4Definition Phase

OCE PMIODV1: out, metadata V1V2: in, metadata V2

ATM PMIODV1: in, metadata V1V2: out, metadata V2V3: out, metadata V3

LAND PMIODV3: in, metadata V3V4: in, metadata V4

LAND AD

ATM AD

OCE AD

V2

V1

V1

V3

V4

Deployment Phase

V2user

Composition Phase

user

SCC ATM:...OCE:...LAND:...

user

OASIS 4


Meta-data for describing the coupling+I/O

PMIOD: Potential Model Input and Output Description

configure

SMIOC: Specific Model Input and Output Configuration

SCC: Specific Coupling Configuration

deploy

(XML files)


The PRISM Support Initiative (PSI)The PRISM Support Initiative (PSI)• 2001-2004: initiation via a EU FP5 project:

22 partners, 5 Meur (~80 py)

• Since January 2005: the PSI to coordinate andprovide effort to fulfil:

1. Software maintenance and service support,2. Support for adaptation of models to PRISM,3. Development against current requirements of adapted

models,4. Development against future requirements5. Coordination with community

more on new web site: http://prism.enes.org


The PRISM partners

1.

Associate Partners

CGAM CRAY MPI NEC SGI SMHI SUN

Partners Contribute significant effort, form the PSI Steering Board

CERFACS CNRS M&D ECMWF Met Office NEC-CCRLE


The PRISM Team• Management coordination (E. Guilyardi, IPSL/CNRS)

• Technical coordination (S. Valcke, CERFACS)

• PRISM User Group (Lead: R. Budich, MPI-M)

• PRISM software technical developments• Coupler (Lead: S. Valcke, CERFACS)• SCE/SRE (Lead: S. Legutke, M&D)• GUI (Lead: N. Wedi, ECMWF)• Versioning control / CM (Lead: M. Carter, Met Office )• Data management issues (Lead: M. Lautenschalger, M&D)

~ 9 py/y


Community buy-inCommunity buy-in• Growing !

– Workshops and seminars– PRISM Community Meeting 2005 (Toulouse, Nov 16-17)– Community programmes taking PRISM onboard

(ENSEMBLES, COSMOS, MERSEA, GEMS,…)

• To go further:– User Survey underway:

help define priorities for next 3 years– coordinate and promote meta-data standards


CollaborationsCollaborations Active collaborations:

• ESMF (supporting software, PMIOD, MOM4)• FLUME (PRISM software)• GENIE (PRISM software)• PCMDI (visualisation, PMIOD)• CF group (CF names)• NERC (BADC & CGAM) (meta-data, PMIOD)

Coupling infrastructure

Scientific core

Supporting software

Running environment

PRISM has organised Europe for community-wideconvergence on basic standards in ES modelling


Need for meta-data standardsNeed for meta-data standards

1. Model meta-data (basic meta-data, needed for modelintercomparison)

– outputs (CF +CMOR-like library, file names,…)– model description– grids– more …

2. Coupling+I/O infrastructure meta-data (required forcomponent swaping*)

– component coupling+I/O description (PMIOD,...)– more...

* but you alsoneed the physicalinterface !!

The community is ready for it !

Longer term development


How can WMP helpHow can WMP help

Organise a « ESM meta-data standards definition »sub-group of WMP (inputs to data community-ies)

Act as a forum to coordinate organisation andstandardisation of ESM meta-data

ESMF, PRISM and other groups already identified anumber of tasks towards this: what support can beexpected from WCRP/WMP ?

The PRISM infrastructurePRISM Final Project Meeting De Bilt, October 7-8, 2004

Thank you !

The PRISM infrastructureOcean model

1- Rainfall + int. energy2- Snowfall + int. energy3- Incoming solar radiat.4- Solar zenith angle5- Fraction of diffuse solar radiation6- Downward infrared radiation7- Sensitivity of atmos temp. & humidity to surf. fluxes

1*- Sensible heat flux2*- Surf. emissivity3*- Albedo, direct4*- Albedo, diffuse5*- Surf. radiative temp.6*- Evaporation + int. energy [+ Qlat]7*- Wind stress8- Subgrid fractions

1- Surface pressure2-4 Air temperature, humidity and wind5- Wind module6- Height of these 4 variables

1*- ρCd2*- ρCe3*- ρCh

1x- Non solar heat flux2x- Solar radiation3x- Fresh water flux4x- Salt flux5x- Wind stress6x- U^37x- Mass of snow and ice8- Subgrid fractions

1*- Surf. Temp2*- Surf. Roughness3*- Displacement height4x- Surface velocity

1- Continental runoff + internal Energy

1-2 Temp./Salinity at sea-ice base3- Highest level temperature (SST)4- Surf. radiative temp.5- Surface ocean current6- Sea surface salinity7- Surface height8- Absorbed solar radiation (in first oceanic layer)

Iceberg parameters

1 2

3

4 5

67

8

Ocean surface module

Surface layer turbulence

Sea ice model wave model

+34

1

2

Atmosphere model

Land surface model

Note on subgrid fractiondependance:<>x- Sea Ice categories (incl. open ocean)<>*- Sea Ice or Land Surf. categories


1- Rainfall + int. energy2- Snowfall + int. energy3- Incoming solar radiat.4- Solar zenith angle5- Fraction of diffuse solar radiation6- Downward infrared radiation7- Sensitivity of atmos temp. & humidity to surf. fluxes

1*- Sensible heat flux2*- Surf. emissivity3*- Albedo, direct4*- Albedo, diffuse5*- Surf. radiative temp.6*- Evaporation + int. energy [+ Qlat]7*- Wind stress8- Subgrid fractions

1- Surface pressure2-4 Air temperature, humidity and wind5- Wind module6- Height of these 4 variables

1*- ρCd2*- ρCe3*- ρCh

1*- Surf. Temp2*- Surf. Roughness3*- Displacement height4x- Surface velocity

1 2

3

4 5

Ocean surface module

Surface layer turbulence

Sea ice model wave model

+34

1

2

Atmosphere model

Land surface model

1- SST2- Sea ice extent3- Sea ice thickness

Data from previous run [ unused ]1

1-2 Temp./Salinity at sea-ice base3- Sea surface temperature4- Surf. radiative temp.5- Surface ocean current6- Sea surface salinity7- Surface height8- Absorbed solar radiation (in first oceanic layer)

7

Data from previous run


Time sequence

Atm

SLT

OSM

Oce

7

Oce t-1

5

3

1

4

+3Compute

Fluxes

2

6

Oce t

Sea Ice t

Atm t (implicit)

The PRISM framework


Outline

•The PRISM project

•PRISM first coupler: Oasis3 (08/2003)

•PRISM final coupler (12/2004)


The PRISM project

• PRISM: PRogram for Integrated Earth System Modelling• A European project, started December 2001, funded

for 3 years by the European Commission (4.8 M€)• Coordinators:

• Guy Brasseur (MPI, Hamburg)• Gerbrand Komen (KNMI, Amsterdam)

• PRISM Director: Reinhard Budich (MPI)


=> 22 partners: leading climate research institutes and computer vendors

• MPG-IMET, Germany• KNMI, Netherlands• MPI-MAD,• Met-Office, UK• UREADMY, UK• IPSL, France• Météo-France, France• CERFACS, France• DMI, Denmark• SHMI, Sweden

• NERSC, Norway• ETH Zurich, Switzerland• ING, Italy• MPI-BGC, Germany• PIK, Germany• ECMWF, Europe• UCL-ASTR, Belgium• NEC Europe• FECIT/Fujitsu• SGI Europe• SUN Europe

The PRISM project partners


Provide software infrastructure to easily assemble Earth system coupled models based on

existing state-of-art European components models launch/monitor complex/ensembles Earth system simulations

Undertake a pilot infrastructure project toward, on the longerterm, the establishment of a European Climate and Earth SystemModeling Supercomputer Facility (ENES).

The PRISM project goals

Help climate modellers spend more time on science:


Define and promote technical and scientific standards for ESM:

Technical standards:• Compiling, running, post-processing environment• Graphical User Interface • Coupler and I/O software• Data and grid format• Coding and quality

Scientific standards:• Physical interfaces• Global Earth System parameters

Interaction with other groups (ESMF, ESG/NOMADS, CF, RPN?,...)

The PRISM project goals


Atmosphere:Météo-France (ARPEGE), MPG-IMET (ECHAM), IPSL (LMDZ),MetOffice (Unified Model),UREADMY, INGV

AtmosphericChemistry:MPG-IMET, UREADMY, IPSL,MetOffice, Météo-France,KNMI

Land Surface:IPSL (Orchidée),MetOffice, MPG-IMET,UREADMY, Météo-France(ISBA)

Sea Ice:NERSC, UCL-ASTR,MetOffice, IPSL, MPG-IMET

OceanBiogeochemistry:MPI-BGC, IPSL,MPG-IMET, MetOffice

Ocean:UREADMY, MetOffice(FOAM), MPI-M (HOPE),IPSL (OPA/ORCA)

Regional Climate:SHMI, DMI, MetOffice

Coupler:CERFACS, NEC,

CCRLE, FECIT, SGI,MPI-MAD

The PRISM project climate components


OASIS: Ocean Atmosphere Sea Ice Soil

The PRISM project first coupler: Oasis3

•developed since 1991 in CERFACS to couple existing GCMs developed independently at the time:

•Models at relatively low resolution (~10000-20000 pts)•Small number of 2D coupling fields (~10)•Low coupling frequency (~once/day)flexibility was very important, efficiency not so much!

•performs:synchronisation of the component modelscoupling fields exchange and interpolationI/O actions

•new Oasis3 now available !



Oasis3 configuration:•In text file namcouple read by Oasis3 at the beginning of the run, e.g.

•total run time•number and names of component models•number and names of coupling fields; for each field:

•coupling and/or I/O status,•coupling or I/O period•transformations/interpolations• …

•Component model grid (longitudes, latitudes, masks, mesh surfaces, meshcorner locations) must be available in binary or NetCDF files.



Oasis3 communication:•New PRISM System model interface (PSMILe)based on MPI1 or MPI2 message passing

•Parallel communication between parallel models andOasis3 interpolation process

A

A

A

B

B

B

A

A

A

file

A

A

A

O

O

O

O

Oasis3

•Direct communication between models with samegrid and partitioning•I/O functionality (automatic switch between coupledand forced mode)

•Modularity: at each model time step, exchange isperformed or not depending on user’s specificationsin namcouple.

• Automatic time integration depending on user’sspecification



•Oasis3 communication; PSMILe API: :•Initialization:

call prism_init_comp(…)

•Retrieval of component model local communicatorcall prism_get_localcomm (…)

•Coupling or I/O field declarations (name, type, shape, local partition, …)call prism_def_var(field_idx, …)

•End of definitioncall prism_enddef(…)

•In model time stepping loop, coupling or I/O field exchangecall prism_put(field_id1, time, field_array1, ierror),call prism_get(field_id2, time, field_array2, ierror)=> Automatic averaging/accumulation, coupling exchange, and/or I/O depending ontime argument and user’s specifications in namcouple

•Termination:call prism_terminate(…)


•Oasis3 interpolations/transformations


A

A

A

O

O

O

O

Oasis3Oasis3=> performed by separate sequential process

=> on 2D scalar fields only• Interfacing with RPN Fast Scalar INTerpolator package

• nearest-neighbour, bilinear, bicubic for regular Lat-Lon grids

• Interfacing with SCRIP1.4 library (Los Alamos Software Release LACC 98-45):• nearest-neighbour, 1st and 2nd order conservative remapping for all grids• bilinear and bicubic interpolation for «logically-rectangular» grids

• Bilinear and bicubic interpolation for reduced atmospheric grids• Other spatial transformations: flux correction, merging, etc.• General algebraic operations


The PRISM project final coupler

•Higher resolution, parallel and scalable models

•Higher coupling frequencies desirable

•Higher number of models and (3D) coupling fields

Need to optimise and parallelise the coupler

The final PRISM coupler will be composed of:• a Driver• a Transformer• a new PRISM System Model Interface Library



Final coupler configuration (XML files):•The user chooses the models through the GUI.• Each component model comes with:

• an Application Description (AD)• a Potential Model Input and Output Description (PMIOD).

•The user configures his particular coupled run through the GUI :• total run time, etc.• for each field described in the PMIOD:

•coupling or I/O status,•coupling or I/O period•transformations/interpolations, etc.

•Based on the user’s choice, the GUI produces:• the Specific Coupling Configuration (SCC) file• a Specific Model Input and Output Configuration (SMIOC) for each model

•At run-time the Driver reads and distributes SCC and SMIOCs information•The PSMILes and Transformer act accordingly to the user’s specifications.


ATM SMIOC V1 : from OCE, T1V2: to OCE, T2V3 : to LAND

user

LAND SMIOC V3 : from ATMV4 : from fileV4

user

OCE SMIOCV1 : to ATM, T1V2 : from ATM, T2

user

Driver

TOCE

ATM

LANDfileV4Definition Phase

OCE PMIODV1: out, metadata V1V2: in, metadata V2

ATM PMIODV1: in, metadata V1V2: out, metadata V2V3: out, metadata V3

LAND PMIODV3: in, metadata V3V4: in, metadata V4

LAND AD

ATM AD

OCE AD

V2

V1

V1

V3

V4

Deployment Phase

V2user

Composition Phase

user

SCC ATM:...OCE:...LAND:...

user



Final coupler communication:•More elaborate PSMILe based on MPI1 or MPI2•Modularity as for Oasis3: at each model time step,exchange is performed or not depending on user’sspecifications in namcouple.•As for Oasis3, automatic time integration dependingon user’s specification• As for Oasis3, I/O functionality (automatic switchbetween coupled and forced mode)•Parallel communication: as for Oasis3 +repartitioning.

OB

OB

OB

C

C

C

O1

O1

C

C

•Parallel calculation of interpolation weights andaddresses in the source PSMILe•Extraction of useful part of source field only.



Final coupler communication; PSMILe API: :As for Oasis3 PSMILe +•Definition of grid (1D, 2D, 3D)

call prism_def_grid(…)call prism_set_corners(…)call prism_set_mask(…)

•Definition of grid for vector and bundle fieldscall prism_set_vector(…)Call prism_set_subgrid(…)

•Coupling or I/O field declarations support vector, bundles, 1D, 2D and3D fields•Extraction of SCC and SMIOC information:

call prism_get_persist(…)


•Final coupler interpolations/transformations => as for Oasis3 +

•Support of vector fields

•Support of 3D fields

•More flexibility for field combination/merging, etc.



Conclusions

•Oasis3 now available•Final coupler prototype due 11/2003•Final coupler due 12/2004

… and after PRISM ?•Follow-on project re-submitted at next EU-call in2004 (CAPRI rejected)International interaction essential in all cases!Formal collaboration?

http://www.enes.orghttp//www.cerfacs.fr/PRISM/prism.html

[email protected]

the prism infrastructure for earth system...

Documents