![Page 1: A Software Developing Environment for Earth System Modeling](https://reader034.vdocuments.mx/reader034/viewer/2022042723/6267d5c338b16c5a947d3af1/html5/thumbnails/1.jpg)
1
A Software Developing Environment for Earth System Modeling
Depei QianBeihang University
CScADS Workshop, Snowbird, UtahJune 27, 2012
![Page 2: A Software Developing Environment for Earth System Modeling](https://reader034.vdocuments.mx/reader034/viewer/2022042723/6267d5c338b16c5a947d3af1/html5/thumbnails/2.jpg)
2
Outline
Motivation
Purpose and Significance
Research Contents
Technology Roadmap
IDE design & development functions
ESM Module Encapsulation
Runtime Support
Deployment
![Page 3: A Software Developing Environment for Earth System Modeling](https://reader034.vdocuments.mx/reader034/viewer/2022042723/6267d5c338b16c5a947d3af1/html5/thumbnails/3.jpg)
3
CCSM/CESM♦ Coupled climate models used to simulate the Earth's
climate system♦ Includes four separate models: the atmosphere, oceans,
land surface and sea ice♦ Support the research of global climate change in the past,
present and future
Motivation
![Page 4: A Software Developing Environment for Earth System Modeling](https://reader034.vdocuments.mx/reader034/viewer/2022042723/6267d5c338b16c5a947d3af1/html5/thumbnails/4.jpg)
4
Primitive program development approaches
Various model development tools
Program development coupled with machine hardware / software
details
Graphical IDE is in urgent need
1. All-in-one platform, comprehensive tool library and the module / template
library, to improve development efficiency
2. Decoupled with software/hardware details
to allow researchers to focus on model development
3. Customized development environment to provide
lifecycle support in developing the earth system
models
Lack of integrated HPC software development environment for earth
system modeling
Motivation (cont‘d)
![Page 5: A Software Developing Environment for Earth System Modeling](https://reader034.vdocuments.mx/reader034/viewer/2022042723/6267d5c338b16c5a947d3af1/html5/thumbnails/5.jpg)
5
Outline
Motivation
Purpose and Significance
Research Contents
Technology Roadmap
IDE design & development functions
ESM Module Encapsulation
Runtime Support
Deployment
![Page 6: A Software Developing Environment for Earth System Modeling](https://reader034.vdocuments.mx/reader034/viewer/2022042723/6267d5c338b16c5a947d3af1/html5/thumbnails/6.jpg)
6
Purpose and significance
Computing ResourcesAbstraction
Abstraction of SW/HW details
Satisfy model developers' specific needs
Reduce the cost of switching between tools and improve efficiency
Support the complete lifecycle of earth system modeling
Scalability
All-In-One Platform
Lifecycle Management
Focus on support to the development of earth system models
Build an integrated development platform for research on global climate change
![Page 7: A Software Developing Environment for Earth System Modeling](https://reader034.vdocuments.mx/reader034/viewer/2022042723/6267d5c338b16c5a947d3af1/html5/thumbnails/7.jpg)
7
Outline
Motivation
Purpose and Significance
Research Contents
Technology Roadmap
IDE design & development functions
ESM Module Encapsulation
Runtime Support
Deployment
![Page 8: A Software Developing Environment for Earth System Modeling](https://reader034.vdocuments.mx/reader034/viewer/2022042723/6267d5c338b16c5a947d3af1/html5/thumbnails/8.jpg)
8
Research content
Architecture of the all-in-one IDE
Demo Application
developed by using IDE
Tool library for developing
earth system models
Integrating development
tools by plug-ins
Template-based graphical
development library
![Page 9: A Software Developing Environment for Earth System Modeling](https://reader034.vdocuments.mx/reader034/viewer/2022042723/6267d5c338b16c5a947d3af1/html5/thumbnails/9.jpg)
9
Outline
Motivation
Purpose and Significance
Research Contents
Technology Roadmap
IDE design & development functions
ESM Module Encapsulation
Runtime Support
Deployment
![Page 10: A Software Developing Environment for Earth System Modeling](https://reader034.vdocuments.mx/reader034/viewer/2022042723/6267d5c338b16c5a947d3af1/html5/thumbnails/10.jpg)
10
Technology roadmap
Reusable code blocks for model development
Portability, scalability and feasibility of platform
Graphical drag&drop -based program coding
Key Technologies
Module / Template Library Tool Library IDE interface
research outcomes
Integrated Development Platform
Plug-in Integration & Graphical interface support
Remote code organization and management
Remote compilation, debugging and execution
Parallel support, e.g, MPI & JASMIN
Description of heterogeneous HPC resources Integrating existing tools
![Page 11: A Software Developing Environment for Earth System Modeling](https://reader034.vdocuments.mx/reader034/viewer/2022042723/6267d5c338b16c5a947d3af1/html5/thumbnails/11.jpg)
11
Based on the open source Eclipse: mature community, abundant references, plenty of free plug-ins, wide range of languages
Overall strategy
PTP
RDT
RSE
EMFPhotranCode assist
Parallel support
Remote Support
Remote client
Graphic modeling code generation
Technology roadmap
![Page 12: A Software Developing Environment for Earth System Modeling](https://reader034.vdocuments.mx/reader034/viewer/2022042723/6267d5c338b16c5a947d3af1/html5/thumbnails/12.jpg)
12
Outline
Motivation
Purpose and Significance
Research Contents
Technology Roadmap
IDE design & development functions
ESM Module Encapsulation
Runtime Support
Deployment
![Page 13: A Software Developing Environment for Earth System Modeling](https://reader034.vdocuments.mx/reader034/viewer/2022042723/6267d5c338b16c5a947d3af1/html5/thumbnails/13.jpg)
13
IDE design & development functions Works have been done:
♦ IDE overall design♦ Fortran code assistant♦ Source code version control♦ Remote online multi-collaboration♦ Integration of other useful tools
![Page 14: A Software Developing Environment for Earth System Modeling](https://reader034.vdocuments.mx/reader034/viewer/2022042723/6267d5c338b16c5a947d3af1/html5/thumbnails/14.jpg)
14
IDE overall design
Platform Runtime
Perspective
Editor
VersionControl
Workbench
ESM IDE
JavaDevelopment
Tools(JDT)
Model Data
Extraction
Visualization
MPMDProgram
Diagnosis
Plug-inDevelopmentEnvironment
(PDE)
IDE Platform
Debug
![Page 15: A Software Developing Environment for Earth System Modeling](https://reader034.vdocuments.mx/reader034/viewer/2022042723/6267d5c338b16c5a947d3af1/html5/thumbnails/15.jpg)
15
ESM Perspective
Fortran Subroutine
OutlineHighlighted & File Layout
VI Edit
Fortran code assistant
![Page 16: A Software Developing Environment for Earth System Modeling](https://reader034.vdocuments.mx/reader034/viewer/2022042723/6267d5c338b16c5a947d3af1/html5/thumbnails/16.jpg)
16
Source code version control
Version Control & Change History
![Page 17: A Software Developing Environment for Earth System Modeling](https://reader034.vdocuments.mx/reader034/viewer/2022042723/6267d5c338b16c5a947d3af1/html5/thumbnails/17.jpg)
17
Remote Online Multi-Collaboration
Remote online multi-collaboration
![Page 18: A Software Developing Environment for Earth System Modeling](https://reader034.vdocuments.mx/reader034/viewer/2022042723/6267d5c338b16c5a947d3af1/html5/thumbnails/18.jpg)
18
Terminal
FTP Tools
Integrated tools
![Page 19: A Software Developing Environment for Earth System Modeling](https://reader034.vdocuments.mx/reader034/viewer/2022042723/6267d5c338b16c5a947d3af1/html5/thumbnails/19.jpg)
19
Remote function invocation Remote editing, compilation, debugging and execution
File Services
Indexing Service
Compiling Service
Launch Service
Debugging Service
Remote Develop
Computing
Source Codeedit
search/index/n
avigator
build
executable
run
debug
Local Machine
Remote Machine
![Page 20: A Software Developing Environment for Earth System Modeling](https://reader034.vdocuments.mx/reader034/viewer/2022042723/6267d5c338b16c5a947d3af1/html5/thumbnails/20.jpg)
20
Outline
Motivation
Purpose and Significance
Research Contents
Technology Roadmap
IDE design & development functions
ESM Module Encapsulation
Runtime Support
Deployment
![Page 21: A Software Developing Environment for Earth System Modeling](https://reader034.vdocuments.mx/reader034/viewer/2022042723/6267d5c338b16c5a947d3af1/html5/thumbnails/21.jpg)
21
Encapsulation of reusable code blocks for model development
Radiation Module
Time integration module
Boundary layer module
Coupler module
Solver module
…… module unit
High-performanceReusable
Modular Encapsulation Specification
model module libraryEnrichmentOptimization
ESM module encapsulation
![Page 22: A Software Developing Environment for Earth System Modeling](https://reader034.vdocuments.mx/reader034/viewer/2022042723/6267d5c338b16c5a947d3af1/html5/thumbnails/22.jpg)
22
Decomposition of POP (Parallel Ocean Program) into modules
Program decomposition
![Page 23: A Software Developing Environment for Earth System Modeling](https://reader034.vdocuments.mx/reader034/viewer/2022042723/6267d5c338b16c5a947d3af1/html5/thumbnails/23.jpg)
23
Parallel computing resources allocation for POP components
Resource allocation
![Page 24: A Software Developing Environment for Earth System Modeling](https://reader034.vdocuments.mx/reader034/viewer/2022042723/6267d5c338b16c5a947d3af1/html5/thumbnails/24.jpg)
24
Dynamic process organization based on grid partitionDynamic process organization
![Page 25: A Software Developing Environment for Earth System Modeling](https://reader034.vdocuments.mx/reader034/viewer/2022042723/6267d5c338b16c5a947d3af1/html5/thumbnails/25.jpg)
25
Functional validation and performance evaluationValidation and performance evaluation
Number of Porcesses POP (s) CPOP (s) K.E
DiagnosisTracer Diagnosis
1 7482 7856 √ √
2 4042 4243 √ √
4 1981 2083 √ √
8 1191 1250 √ √
16 1082 1130 √ √
32 1238 1242 √ √
64 1257 1251 √ √
128 1534 1506 √ √
256 2582 2453 √ √
![Page 26: A Software Developing Environment for Earth System Modeling](https://reader034.vdocuments.mx/reader034/viewer/2022042723/6267d5c338b16c5a947d3af1/html5/thumbnails/26.jpg)
26
Code generation
Graphic Drag&Drop Editor
Module Repository
Code Auto-Generation
Module Property
![Page 27: A Software Developing Environment for Earth System Modeling](https://reader034.vdocuments.mx/reader034/viewer/2022042723/6267d5c338b16c5a947d3af1/html5/thumbnails/27.jpg)
27
Component specification Specification for component encapsulation♦ Templates to guide component development♦ Each component is associated with a xml-based
description file before importing to the component library
![Page 28: A Software Developing Environment for Earth System Modeling](https://reader034.vdocuments.mx/reader034/viewer/2022042723/6267d5c338b16c5a947d3af1/html5/thumbnails/28.jpg)
28
Component specification (cont’d)
Module Descriptive Specification
![Page 29: A Software Developing Environment for Earth System Modeling](https://reader034.vdocuments.mx/reader034/viewer/2022042723/6267d5c338b16c5a947d3af1/html5/thumbnails/29.jpg)
29
Code generation (cont’d)Drag-Drop based component usage♦ Automatically locate and intercept the information
about the component based on CDL, and then fill in the panel of attributes about this component
♦ Component substitution can be done easily through selection
![Page 30: A Software Developing Environment for Earth System Modeling](https://reader034.vdocuments.mx/reader034/viewer/2022042723/6267d5c338b16c5a947d3af1/html5/thumbnails/30.jpg)
30
Code generation (cont’d)
ComponentSkeletonCode
![Page 31: A Software Developing Environment for Earth System Modeling](https://reader034.vdocuments.mx/reader034/viewer/2022042723/6267d5c338b16c5a947d3af1/html5/thumbnails/31.jpg)
31
Primary auto-verification for module validityModule auto-verification
Coupler not Match
![Page 32: A Software Developing Environment for Earth System Modeling](https://reader034.vdocuments.mx/reader034/viewer/2022042723/6267d5c338b16c5a947d3af1/html5/thumbnails/32.jpg)
32
Model components substitutionModel component encapsulation allows the
components developed by different groups or communities to be inter-changeable, reducing duplicated programming efforts♦ Component substitution is demonstrated between POP and
MOM♦ MOM
a three-dimensional ocean circulation model designed primarily by the Geophysical Fluid Dynamics Laboratory (GFDL) of the National Oceanic and Atmospheric Administration
♦ POP developed and supported by Los Alamos National Laboratory (LANL)
![Page 33: A Software Developing Environment for Earth System Modeling](https://reader034.vdocuments.mx/reader034/viewer/2022042723/6267d5c338b16c5a947d3af1/html5/thumbnails/33.jpg)
33
FMS
• Components are executed sequentially• All component resides in flat• Single coupler• Data is transferred through shared
public variables
Comp Comp Comp CompCoupler
ESMF
Application Driver
Comp3.1
Comp2Comp1
Comp3.2.2Comp3.2.1
Comp3
Comp3.2 coupler
• Child components are executed simultaneously• Invocation among components is nested• Multiple couplers• Data is transferred through data structure called state
POP vs MOM Framework Comparison♦ POP is based on ESMF, while MOM is based on FMS
![Page 34: A Software Developing Environment for Earth System Modeling](https://reader034.vdocuments.mx/reader034/viewer/2022042723/6267d5c338b16c5a947d3af1/html5/thumbnails/34.jpg)
34
MOM code structure analysis
![Page 35: A Software Developing Environment for Earth System Modeling](https://reader034.vdocuments.mx/reader034/viewer/2022042723/6267d5c338b16c5a947d3af1/html5/thumbnails/35.jpg)
35
MOM code structure analysis Analyzing the code structure of MOM♦ MOM component interface
init() #initialize and configurerun()
♦ update() #update data, time step, status and so on ♦ restart() #reload configuration and restart
final(): #release the resouces
The above interface would make sure that the components derived from MOM, which is based on FMS specification, are compatible with components derived from POP which is based on ESMF, so that the components are substitutable
![Page 36: A Software Developing Environment for Earth System Modeling](https://reader034.vdocuments.mx/reader034/viewer/2022042723/6267d5c338b16c5a947d3af1/html5/thumbnails/36.jpg)
36
Outline
Motivation
Purpose and Significance
Research Contents
Technology Roadmap
IDE design & development functions
ESM Module Encapsulation
Runtime Support
Deployment
![Page 37: A Software Developing Environment for Earth System Modeling](https://reader034.vdocuments.mx/reader034/viewer/2022042723/6267d5c338b16c5a947d3af1/html5/thumbnails/37.jpg)
37
Runtime support Work have been done:
♦ Seamless integration with Slurm batch system with an easy-to-use management portal
♦ Support ESM program execution monitoring and trace analysis
![Page 38: A Software Developing Environment for Earth System Modeling](https://reader034.vdocuments.mx/reader034/viewer/2022042723/6267d5c338b16c5a947d3af1/html5/thumbnails/38.jpg)
38
Integration with the batch system Integration with Condor to provide persistent execution
environment The jobs submitted to the IDE are delegated to underlying
slurm batch system Extend condor to talk to slurm Submit, query and management
![Page 39: A Software Developing Environment for Earth System Modeling](https://reader034.vdocuments.mx/reader034/viewer/2022042723/6267d5c338b16c5a947d3af1/html5/thumbnails/39.jpg)
39
Resource Monitor
Job Query
Job Submit
Program execution monitoring
![Page 40: A Software Developing Environment for Earth System Modeling](https://reader034.vdocuments.mx/reader034/viewer/2022042723/6267d5c338b16c5a947d3af1/html5/thumbnails/40.jpg)
40
Trace analysis based on STAT
![Page 41: A Software Developing Environment for Earth System Modeling](https://reader034.vdocuments.mx/reader034/viewer/2022042723/6267d5c338b16c5a947d3af1/html5/thumbnails/41.jpg)
41
Trace analysis based on STAT (cont’d)
![Page 42: A Software Developing Environment for Earth System Modeling](https://reader034.vdocuments.mx/reader034/viewer/2022042723/6267d5c338b16c5a947d3af1/html5/thumbnails/42.jpg)
42
Outline
Motivation
Purpose and Significance
Research Contents
Technology Roadmap
IDE design & development functions
ESM Module Encapsulation
Runtime Support
Deployment
![Page 43: A Software Developing Environment for Earth System Modeling](https://reader034.vdocuments.mx/reader034/viewer/2022042723/6267d5c338b16c5a947d3af1/html5/thumbnails/43.jpg)
43
Deployment Alpha version released (March 2011) Five versions (v0.2, v0.4, v0.6, v1.0, v2.0)released so far
Deployed in Beijing Normal University (since March 2011)♦ Pilot usage of four versions(v0.4, v0.6, v1.0, v2.0 )♦ Used in BNU_ESM development♦ Latest version is v2.0
Deployed in Institute of Atmospheric Physics, Chinese Academy of Sciences (since March 2011)♦ Latest version is v2.0
Deployed in National Climate Center (since June 2011)♦ Latest version is v2.0
IDE and model encapsulation approach adopted by the coupler development project (since July 2011)
![Page 44: A Software Developing Environment for Earth System Modeling](https://reader034.vdocuments.mx/reader034/viewer/2022042723/6267d5c338b16c5a947d3af1/html5/thumbnails/44.jpg)
44
Thank You!