page 1 jordi farres models for scientific exploitation of eo data esrin, 11-12 oct 2012 a web of eo...
TRANSCRIPT
Page 1
Jordi Farres
Models for scientific exploitation of EO Data
ESRIN, 11-12 Oct 2012
A Web of EO Processing Services
ESA UNCLASSIED - For Offical Use 11/10/2012
Page 2
Agenda
1. Introduction
2.A case study - GPOD
3.A model of scientific exploitation – Earth Web
4.Conclusions
ESA UNCLASSIED - For Offical Use 11/10/2012
Page 3
Intro: GSCB meeting [ESRIN, 6/6/2012]
• PI work more frequently on long-term data series (M.Albani)
• Sentinel data policy: “data available at no fee within the financial limits for operations costs” (J. Aschbacher)
• SAR Data provision limited by “data quota based on technical constraints” (H.Laur)
• ESA processing on-demand (GPOD) is employed at 10% in 2009, peaks to 100% in 2012 (J.Farres)
ESA UNCLASSIED - For Offical Use 11/10/2012
The policy may meet expectations but not its implementation High pressure for better dissemination and processing services
Page 4
Intro: Cloud Computing & Large ICT providers
• The Science Cloud – Helix Nebula
• SuperSites Exploitation Platform
• GAIA on Amazon
• NASA uses Amazon's cloud computing in Mars landing mission
ESA UNCLASSIED - For Offical Use 11/10/2012
HPC becomes an utility Opportunities to team with large ICT providers
Page 5
What is the preferred exploitation model?
An Earth Box
1. Drop all data on large Dropbox
2. Place all data in the cloud
3. Users / service autonomously develop: Applications, Services and Research
An Earth Web
1. Expose data on an exploitation platform
2. Federate services from different EO providers
3. Users develop the value added services on a common framework
ESA UNCLASSIED - For Offical Use 11/10/2012
Page 6
Intro: An Earth Web
• A facility
• for EO dissemination and processing services
• geographically distributed
• in a federated domain
• largely scalable
• with reliable QoS
ESA UNCLASSIED - For Offical Use 11/10/2012
Page 7
Agenda
1. Introduction
2.A case study - GPOD
3.A model of scientific exploitation – Earth Web
4.Conclusions
ESA UNCLASSIED - For Offical Use 11/10/2012
Page 8
A case study: G-POD
Service ladder ESAESRIN
ESAUK-PAC
ESAESAC*
Cloud
6- Portal – User and Info mgt
√
5- Service Mgt Deployment
√
4- Processing Services >50 3
3- Data Mgt Discovery & Catalogue
√ √
2- Data Resources ASAR, RA, GOME,
GOMOS, MIPAS, ...
(A)ATSR, MERIS
SMOS SAR subset
1- ICT Resources 2 Clusters 1 Cluster 1 Cluster IaaS
ESA UNCLASSIED - For Offical Use 11/10/2012
Page 9
A case study: G-POD
Service ladder ESAESRIN
ESAUK-PAC
ESAESAC*
Cloud
6- Portal – User and Info mgt
5- Service Mgt Deployment
4- Processing Services
3- Data Mgt Discovery & Catalogue
2- Data Resources
1- ICT Resources
ESA UNCLASSIED - For Offical Use 11/10/2012
PORTAL
CATALOGUE CATALOGUE
DATA DATA DATA
COMPUTING COMPUTING COMPUTING
Page 10
A case study: G-POD
Service ladder ESAESRIN
ESAUK-PAC
ESAESAC*
Cloud
6- Portal – User and Info mgt
5- Service Mgt Deployment
4- Processing Services
3- Data Mgt Discovery & Catalogue
2- Data Resources
1- ICT Resources
ESA UNCLASSIED - For Offical Use 11/10/2012
PORTAL
PROCESSOR
COMPUTING COMPUTING COMPUTING
PROCESSOR
Page 11
A case study: G-POD / SSEGrid
Service ladder ESAESRIN
ESAUK-PAC
ESAESAC
Cloud VITO
6- Portal – User and Info mgt
√
5- Service Mgt Deployment
√ √
4- Processing Services >50 3 1
3- Data Mgt Discovery & Catalogue
√ √ √
2- Data Resources ASAR, RA, GOME,
GOMOS, MIPAS, ...
(A)ATSR, MERIS
SMOS SAR subset
SPOTVeg
1- ICT Resources 2 Clusters 1 Cluster 1 Cluster IaaS √
ESA UNCLASSIED - For Offical Use 11/10/2012
Page 12
Agenda
1. Introduction
2.A case study - GPOD
3.A model of scientific exploitation – Earth Web
4.Conclusions
ESA UNCLASSIED - For Offical Use 11/10/2012
Page 13
An Earth Web Model – Service ladder
Service ladder Partner ALocation 1
Partner ALocation 2
Partner B Partner C
6- Portal – User and Info mgt
√
5- Service Mgt Deployment
√
4- Processing Services √ √
3- Data Mgt Discovery & Catalogue
√ √ √
2- Data Resources √ √ √
1- ICT Resources √ √ √ √
ESA UNCLASSIED - For Offical Use 11/10/2012
Cloud Provider
EO ResourcesProvider
EO ServiceProvider
EO Community
Mgt
Page 14
An Earth Web Model – Interoperability
Service ladder Technologies Standards HMA-AWG
6- Portal – User and Info mgt
SSO SAML Partial (Identity mgt)
5- Service Mgt Deployment
PFVirtual Machines
Web Services
IPF I/FOVF
WPS-T
NO
4- Processing Services Web Services WPS Partial
3- Data Mgt Discovery & Catalogue
Web Services CSW YES
2- Data Resources https
Web Services
MetadataFile Formats
WCS
YES
1- ICT Resources IaaSGrid brokers
OCCI, EC2 NO
ESA UNCLASSIED - For Offical Use 11/10/2012
Page 15
An Earth Web Model – Challenges
ESA UNCLASSIED - For Offical Use 11/10/2012
1. Execute processing X on data Y over server Z (portability)
2. Quantify user authorization in relation processing resources
3. Profile users in relation to QoS
4. Ability to quote “budget” for processing X on data Y
5. Accounting à la cloud for ICT resources
6. Harmonized authorization and accounting
7. …
Page 16
Conclusions
1. An “Earth Web” model is better suited for scientific exploitation than a simple “Earth Box”
2. Facilities/Institutions can federate their own resources/services and get access to the federated pool.
3. Facilities can profile different service layers: ICT, Data, Data Mgt, Processing Services, …
4. BUT Interoperability / Standardization for Processing Services is NOT there
ESA UNCLASSIED - For Offical Use 11/10/2012