infso-ri-031688 enabling grids for e-science es applications in egeeii – m. petitdidier –11...

INFSO-RI-031688

Enabling Grids for E-sciencE

www.eu-egee.org

ES applications in EGEEII – M. Petitdidier –11 February 2008

Earth Science Application Overview in EGEE Infrasctructure

M. Petitdidier (IPSL/CETP), [email protected]

In collaboration with EGEE and DEGREE EU-project

SSA- IST 2005-034619 ES applications in EGEEII –– M. Petitdidier –11 February 2007

Topography & Motion

Land cover & vegetation

Marine SST, SSH& colour

Currents, bathymetry & ice

Planet Earth : a complex system

Stratospheric Ozone

Industrial Emissions

Atmospheric profiles

10 y displacement of Etna 1992-01

GOME total ozone assimilation

ES applications in EGEEII – M. Petitdidier – 11 February 2008


INFSO-RI-031688

Acute Questions for Earth Science

• Forecasting of meteorological events– Extreme events: storm, tornado, hurricane, earthquake, tsunami…– Water management: precipitations, flood, aquifer..– Pollution,,…

=> To provide real time information: real time data access, data assimilation and modelling

• Long term prediction : Climate Change In October 2007, Albert Arnold (Al) Gore Jr. and the IPCC (Intergovernmental Panel

on Climate Change): Nobel Peace Prize 2007 "for their efforts to build up and disseminate greater knowledge about man-made climate change, and to lay the foundations for the measures that are needed to counteract such change”.

• Climatology i.e. trend of parameter variations like temperature, precipitations….– Polar ozone hole– Impact on agriculture, weather…

=> Long series of multiple data sets, Intense processing, • Discoveries and Dissemination of the knowledge

– How it works, Why…

These are Questions for Operational and Science organisations



INFSO-RI-031688

Earth Science Challenges

To answer the questions, needs to address the following situations:

• Global, regional, local applications– Alternative use of the data at different time and spatial resolutions

• Large historical distributed archives– Long term data archives to be exploited for climatology

• Near real-time access to data and processing– For processing, value adding and dissemination– For now-casting and alert

• Models to provide long term trends and forecast– Processing-intensive, data-intensive and complex applications

• Integrate different data sources– Standardisation, Data policy, …– Data fusion, data assimilation, data mining, modelling …

• Link data to technical information and scientific resultsData are at the centre of ES applications



INFSO-RI-031688

GRID Technology

• GRID infrastructure– Considered a comfortable “open platform” for handling computing resources,

data, tools… – Impressive number of shared resources– A collaborative possible platform among teams and/or countries– Secure and restricted access to resources, data, tools

• Use of Grid: application requirements of the proposed application, data, numerical approach, type of computing resources, languages of code

• EGEE well adapted to:– Sharing data and algorithm– Processing and/or production of large data sets of data– Complex workflow– Statistical approach– Other approaches: MPI, Modelling, simulation

Grid opens new fields of investigation in Earth Science



INFSO-RI-031688

Earth Science Applications in EGEEII

ESA, UTV(IT), ESA, UTV(IT), KNMI(NL), IPSL(FR)- KNMI(NL), IPSL(FR)- Production and Production and validation of 7 years of validation of 7 years of Ozone profiles from Ozone profiles from GOMEGOME

Rapid Earthquake Rapid Earthquake analysis analysis (mechanism and (mechanism and epicenter) epicenter) 50- 100CPUs 50- 100CPUs IPGP(FR)IPGP(FR)

Modelling seawater Modelling seawater intrusion in costal intrusion in costal aquifer (SWIMED) aquifer (SWIMED) CRS4(IT),INAT(TU),CRS4(IT),INAT(TU),Univ.Neuchâtel(CH)-Univ.Neuchâtel(CH)-

Geocluster for Geocluster for Academy and Academy and industry CGG(FR)-industry CGG(FR)-

Flood of a Danube river-Flood of a Danube river-Cascade of models Cascade of models (meteorology,hydraulic ,(meteorology,hydraulic ,hydrodynamic….) hydrodynamic….) UISAV(SK)-UISAV(SK)-

Specfem3D: Specfem3D: Seismic Seismic application. application. Benchmark for Benchmark for MPI (2 to 2000 MPI (2 to 2000 CPUs) (IPGP,FR)CPUs) (IPGP,FR)

DKRZ(DE)- Data access DKRZ(DE)- Data access studies, climate impacts on studies, climate impacts on agricultureagriculture

Data mining Data mining Meteorology & Meteorology & Space Weather Space Weather (GCRAS, RU)(GCRAS, RU)

Air Pollution Air Pollution model- BAS(BG)model- BAS(BG)Mars atmosphere CETP(

FR):



INFSO-RI-031688

Sharing data and algorithm

• GOME/ERS – Production of ozone profiles and validation by Lidar data– ESA, UTV, KNMI, IPSL– 2 papers JQSRT

• GEOCLUSTER – seismic plateform developed by CGGVeritas– Available for academic end-users

• ELMER – available for ESR VO partners

• CODESA3D - Hydrology



INFSO-RI-031688

GOME

ESA(IT) – KNMI(NL)Processing of raw GOMEdata to ozone profiles.

2 alternative algorithms~28000 profiles/day

IPSL(FR)Validate some of the

GOME ozone profiles (~106/y)Coincident in space and time

with Ground-Based measurements

Visualization & Analyze

EGEEenvironment

Level 2

(example of 1 day total O3)Level 1

Raw satellite data from the GOME instrument(~75 GB - ~5000 orbits/y)

Meta Database server PosgreSQL

– geospatial search



INFSO-RI-031688

GOME

• 7 years of data, 14,5 orbits/day• Algorithms:

– Neural network, NNO, (ESA, UTV) using IDL - 2 versions– Inversion Algorithm (KNMI) –data, O3 climatology, ECMWF..

• Lidar data (NSDC)– 7 stations maximum (IPSL)

• Number of filed: 70000 for both 2 versions of NNO• Common development

– Metadata base on a server with security and restricted access– Query by Geolocalisation in time and space of orbits passing over a

lidar site by using PosgreSQL)

• Results– Unique case of validation of a whole satellite data set with all the data

taken by other instruments– Once the application was ported the validation of another algorithm or

version is very fast



INFSO-RI-031688

Geocluster – CGG Veritas

• Seismic platform software developed by CGG (400 modules) ported on EGEE for research (R&D Industry + Academy) including tools le for data processing, simulation and visualisation

• License server for authorized users• Available for academic french users of Geocluster

– last Version available no need to download– Processing at a larger scale by using Grid resources



INFSO-RI-031688

ELMER - CSC

• Physical models of fluid dynamics, structural mechanics, electromagnetics, heat transfer and acoustics.

• Available on Grid to ESR VO• http://www.csc.fi/elmer

Pictures describe the flow of the glacier inside volcano Uskovsky, Kamchatka, Russia

See the poster



INFSO-RI-031688

Processing and/or production of large data sets

• GOME• Geoscope:

– Noise analysis of Geoscope database 25 years of seismic data

• Evolution of pesticides into the soil– BRGM –European project Footprint– On going



INFSO-RI-031688

Geoscope

•25 years of data•28 seismological stations and data center•. Data access: query by mail, and made available on the geoscope ftp server•A daily data file for all the stations SEED (~200MB)•Job download the data (wget) and compute the noise DSP (~1h CPU).

IPGP:E. Stutzmann, G. Patau, IPSL/EGEE: D. Weissenbach

Code Fortran , octave (~Matlab), shell and perl scripts

Impact on the design of other seismic data server

See the poster



INFSO-RI-031688

Footprint (http://www.eu-footprint.eu)

• European project coordinator BRGM (Bureau deRecherches Géologiques et Minières)

• Risk evaluation of contamination of water resources by pesticides at different time and spatial scales.

• Weather scenarios (data base by Meteo France), soil scenarios and 100 pesticides

12 millions de run 1-2h each 10 TBytes Interest to use EGEE:

24/24 et 7 days/7 Possibility to run simultaneously hundreds of jobs



INFSO-RI-031688

Complex Workflow

• Forecasting of river flood– Development of a platform on EGEE by IISAS (SK)– Collaboration IISAS in the framework of INTAS (CNES, Ukraine

and Russia) to be used for Ukrainian rivers– Collaboration IISAS with l’Ecole des Mines d’Alès for French

rivers– Interest of European project CYCLOPS (Cyber-infrastructure for

CiviL protection Operative Procedures)



INFSO-RI-031688

Forecasting of flood

• Danube river

• Data :météorology, river network rivières, landscape

• Meteorology model ALADIN (MPI-parallel), MM5 (MPI-parallel)

• Hydrology HSPF (sequential-parametric), NLC (sequential-parametric),

• Hydraulic. DaveF (MPI-parallel), FESWMS (MPI-parallel)

• output: weather, précipitations, hydrography, water level and flood speed forecast

• Cascade of jobs managed by dynamical workflow

Data sources

Hydrological simulation

Hydraulic simulation

Portal

Meteorological simulation

Cascade de simulations

L. Hluchy, Viet Tran, M. Ciglan …. (II-SAS, Bratislava Slovaquie)



INFSO-RI-031688

Statistical approach

• Intrusion of seawater into coastal aquifer– CRS4 (Italie), Univ. Neuchâtel (Suisse), INAT (Tunisie), France– Large interest by mediterranean countries –SWIMED,

EUMedGrid– Monte Carlo method to get probability map with good accuracy

• Radiation Transfer – OPGC/LaMP, France– Simulation of the back-scattered signal of the CALIOP

experiment aboard the satellite CALIPSO using Monte Carlo approach.

– high-resolution vertical profiles of aerosols and clouds



INFSO-RI-031688

Hydrology

Management of water resources in Mediterranean area (SWIMED): G. Lecca (CRS4 Italy), P. Renard (Unine, CH), J.

Kerrou (INAT, Tunisia), R. Ababou (IMFT, Fr)

CODESA-3D: Density-dependent 3D coupled groundwater flow and transport simulations Data requirement: Geology, Topography, Meteorology, Water extraction by the farmer, Aquifer properties (Soil maps, Land use)GRID Impacts:Enhancing the data sharing among the field geologists, modelers, and water managersAllowing the water managers in Tunisia to investigate the potential impacts of management decision through a remote use of the grid : web interface - EUMEDGrid



INFSO-RI-031688

OTHER Approaches

• Control and optimisation of execution time of a bunch of simultaneous jobs– Earthquakes

• Near –Real time job– Flood -Cyclops– RISICO - Cyclops

• parametric Jobs– Geomorphologiy

• Chain of jobs– Polar Ozone– Jobs longs (>24h)

• MPI– SPECFEM3D– 3DSEM_UNSTRUCT

• Meteorological Modelling– MM5, WRF, RAMS– Danish Pollution model



INFSO-RI-031688

Earthquake Characteristics

Fast Determination of mechanisms of important earthquakes (IPGP: E. Clévédé, G. Patau; IPSL: D. Weissenbach)

Application to run on alertCollect data of 30 seismic stations from GEOSCOPE worldwide networkSelect stations and dataDefine a spatial 3D grid +time based on the assumed earthquake locationRun for each grid point or group of grid points a job => ~ 50-100jobs Results obtained ~6hr after the

earthquake data arrival

Important for emergency action and other related researches

All major earthquakes so treated: 21/24 in 2006 => catalogue



INFSO-RI-031688

Geomorphology

Examples : erosion of the mountains, dynamic of dunes ...

Algorithm : 3D cellular automaton for geomorphological research very simple transition rules between cells of different states (transport, deposition ...).

Objective : understanding landscapes formation and evolution

IPGP: C. Narteaux and O. Rozier



INFSO-RI-031688

Modelling study of polar stratospheric ozone loss (IPSL)

Advectiontime step: 1 h

interpolationvertical scheme

every 6 h

ECMWF Winds (2.5°/2.5°)

Diabatic heating rates calculation

chemical scheme

REPROBUS

(2°/2°)

MIMOSA-CHIM

resolution:(1 pt/ °)

regular grid advected grid

S. Godin-Beekmann, J. Gazeaux A. Pazmiño, C. Boonne, O.P. Tripathi, D.Weissenbach

•.Long Job

•Complex workflow



INFSO-RI-031688

SpecFEM3D ([email protected])

• Resolution of regional scale seismic wave propagation problems to model wave propagation at high frequencies and for complex geological structures, with use of the spectral-element method.

• Application first written by D. Komatitsch (Université de Pau), used in more than 75 laboratories in the world,especially for the study of earthquakes.

• Very scalable application using F90 + MPI needing NFS mounted homes and requiring the successive launch of two mpirun on the SAME nodes, allocated in the SAME order.

• Project: to be distributed on EGEEII to authorized users



INFSO-RI-031688

Meteorological modelling

MM5, WRF or Rams• Very common software in Earth Science

• Large variety of applications:• Dependent on the input data for initialization and added modules

• weather forecasting, climate, time evolution of phenomena, sensitivity test

• Rams already ported

• MM5, WRF on going

MPI based Skiron/Eta weather forecasting system for short time periods

Limited Area Weather and Environmental Forecasting on Grid Platform (LAWEF-GRID)



INFSO-RI-031688

Final remarks

• In EGEEIII improvement and/or implementation of functionalities more adapted to Earth Science– License server– Data management– Workflow– Portal

• New tools needed to use the whole Grid potential – Due to Change in scale of computing power– Need of Exploration of huge data sets– Creation of Platform integrating web services, computing power, information

systems….• New conceptual approach of Earth Science

– Role of Scientist§ Interactive collaboration -> less duplication of development and/or

adaptation§ More time for new ideas, new research§ Confidentiality of the research

– Application development ( access to several large data sets, more CPUs…)

Dissemination and Exploitation of GRids in Earth sciencE

SSA- IST 2005-034619 ES applications –AAAS’07 – M. Petitdidier –February 2007

• Strategic objectives– Bridge the ES and GRID communities throughout Europe – Ensure that ES requirements are satisfied in next Grid generation– Ensure the integration of emerging technologies for managing ES

knowledgeThe DEGREE team:The DEGREE team:

IISAS, IISAS, Slovakia Slovakia (Coordinator)(Coordinator)

CNRSCNRS, France , France KNMI, KNMI, The Netherlands The Netherlands UNINEUNINE, Switzerland, SwitzerlandCRS4CRS4, Italy, ItalySCAI, SCAI, Germany Germany GCRASGCRAS, Russia, RussiaESA-ESRIN, ESA-ESRIN, ItalyItalyCGGCGG, France, FranceDutch Space, Dutch Space, The NetherlandsThe Netherlands

Project Vision

Build a bridge linking the ES and Grid communitiesBuild a bridge linking the ES and Grid communities

http://www.eu-degree.eu



INFSO-RI-031688

Earth Science Expectations

Pushing frontiers of scientific discovery by exploiting advanced computational methods.

EGU (European Geophysical Union), Vienna, Austria – 13-18 April. 2008

EGEE booth & 2 Grid sessions ESSI8 and ESSI9

infso-ri-031688 enabling grids for e-science es applications in egeeii – m. petitdidier –11...

Documents

data assimilation

data policy

data mining

data fusion

modelling link data

scientific results data

real time data access

escience infsori