![Page 1: 1 Improving Data Mobility & Management for International Climate Science, Michael.Schick@eumetsat.int & Lothar.Wolf@eumetsat.int Challenges in the provision](https://reader038.vdocuments.mx/reader038/viewer/2022110319/56649c725503460f94924858/html5/thumbnails/1.jpg)
1Improving Data Mobility & Management for International Climate Science, [email protected] & [email protected]
Challenges in the provision of large data sets to multiple user communities
14-16 July 2014, Boulder CO
MONITORING WEATHER AND CLIMATE FROM SPACE
![Page 2: 1 Improving Data Mobility & Management for International Climate Science, Michael.Schick@eumetsat.int & Lothar.Wolf@eumetsat.int Challenges in the provision](https://reader038.vdocuments.mx/reader038/viewer/2022110319/56649c725503460f94924858/html5/thumbnails/2.jpg)
2Improving Data Mobility & Management for International Climate Science, [email protected] & [email protected]
Mission Statement
We are the European Organisation for the Exploitation of Meteorological Satellites (EUMETSAT).
EUMETSAT is an intergovernmental organisation and was founded in 1986. Our purpose is to supply weather and climate-related satellite data, images and products
• 24 hours a day, • 365 days a year
to the National Meteorological Services of our Member and Cooperating States in Europe, and other users worldwide.
www.eumetsat.int
![Page 3: 1 Improving Data Mobility & Management for International Climate Science, Michael.Schick@eumetsat.int & Lothar.Wolf@eumetsat.int Challenges in the provision](https://reader038.vdocuments.mx/reader038/viewer/2022110319/56649c725503460f94924858/html5/thumbnails/3.jpg)
3Improving Data Mobility & Management for International Climate Science, [email protected] & [email protected]
EUMETSAT Headquarters (Darmstadt, Germany)
![Page 4: 1 Improving Data Mobility & Management for International Climate Science, Michael.Schick@eumetsat.int & Lothar.Wolf@eumetsat.int Challenges in the provision](https://reader038.vdocuments.mx/reader038/viewer/2022110319/56649c725503460f94924858/html5/thumbnails/4.jpg)
4Improving Data Mobility & Management for International Climate Science, [email protected] & [email protected]
EUMETSAT Data Services - High Level Overview
Users
Discover, search, register, order and subscribe
Subscribe to services
Discover/search products
Register to messages
Distributed search, remote order
Order/ subscription products
Subscribe to services
GEONETCast Catalogue
EUMETCast/ GEONETCast
Unified Meteorological
Archive(U-MARF)
EUMETSAT Partner Agencies/
GMES, WMO
User Notifcations(UNS)
EUMETCast / GEONETCast
Other Dissemination
Services
Registration
Discovery (Product Navigator)
Order Followup
Access to applications & data
EUMETSAT EO Portal
![Page 5: 1 Improving Data Mobility & Management for International Climate Science, Michael.Schick@eumetsat.int & Lothar.Wolf@eumetsat.int Challenges in the provision](https://reader038.vdocuments.mx/reader038/viewer/2022110319/56649c725503460f94924858/html5/thumbnails/5.jpg)
5Improving Data Mobility & Management for International Climate Science, [email protected] & [email protected]
EUMETSAT Data Services - Data Flows
EUMETSAT GROUND INFRASTRUCTURE
![Page 6: 1 Improving Data Mobility & Management for International Climate Science, Michael.Schick@eumetsat.int & Lothar.Wolf@eumetsat.int Challenges in the provision](https://reader038.vdocuments.mx/reader038/viewer/2022110319/56649c725503460f94924858/html5/thumbnails/6.jpg)
6Improving Data Mobility & Management for International Climate Science, [email protected] & [email protected]
Single Access Point
One stop shop
Data discovery
User registration (>10.000)
Subscription for data services: Near-real time data
Online Data Access
Long Term Archive
Additional Image Gallery (WMS/ WCS)
User Notifications
Operational Real-Time Services Status Indicator
EUMETSAT Earth Observation Portal
![Page 7: 1 Improving Data Mobility & Management for International Climate Science, Michael.Schick@eumetsat.int & Lothar.Wolf@eumetsat.int Challenges in the provision](https://reader038.vdocuments.mx/reader038/viewer/2022110319/56649c725503460f94924858/html5/thumbnails/7.jpg)
7Improving Data Mobility & Management for International Climate Science, [email protected] & [email protected]
EUMETCast Dissemination Near Real Time data delivery to users in 3 continents via Satellite
Users (2/2014)
4
3
2-10
11-50
51-200
201-500
1 User
500+
0 User
3,280 users worldwide,4168 stations5 new users per month
EUMETCast AmericasSES 6, C-BAND
EUMETCast AfricaEUTELSAT 5 WEST A, C-BAND
EUMETCast EuropeEUTELSAT 9 A, KU-BAND
![Page 8: 1 Improving Data Mobility & Management for International Climate Science, Michael.Schick@eumetsat.int & Lothar.Wolf@eumetsat.int Challenges in the provision](https://reader038.vdocuments.mx/reader038/viewer/2022110319/56649c725503460f94924858/html5/thumbnails/8.jpg)
8Improving Data Mobility & Management for International Climate Science, [email protected] & [email protected]
EUMETCast Dissemination:Integrating terrestrial dissemination
![Page 9: 1 Improving Data Mobility & Management for International Climate Science, Michael.Schick@eumetsat.int & Lothar.Wolf@eumetsat.int Challenges in the provision](https://reader038.vdocuments.mx/reader038/viewer/2022110319/56649c725503460f94924858/html5/thumbnails/9.jpg)
9Improving Data Mobility & Management for International Climate Science, [email protected] & [email protected]
MULTICAST ACCROSS GÉANT and NREN INFRASTRUCTURE
GEANT/NRENs
EUMETCast Platform (MMDS)
EUMETCast User Stations
Reference Stations
Multicast ...Multicast
Multicast
Exploring the multicast usage on GÉANT infrastructure in Europe and their world wide connectivity;
2011 Proof of concept on terrestrial multicast on GÉANT test bed
2012 3 month feasibility study on terrestrial multicast via GÉANT
2013 Multicast test to KMA Korea
2014 Multicast test to NOAA (Boulder)
2014 Start of 2 year prototype emulating operational scenarios of multicast across Europe and our overseas partners using GÉANT, NREN, I2 and Teint infrastructure.
![Page 10: 1 Improving Data Mobility & Management for International Climate Science, Michael.Schick@eumetsat.int & Lothar.Wolf@eumetsat.int Challenges in the provision](https://reader038.vdocuments.mx/reader038/viewer/2022110319/56649c725503460f94924858/html5/thumbnails/10.jpg)
10Improving Data Mobility & Management for International Climate Science, [email protected] & [email protected]
EUMETCast Terrestrial real-time monitoring
- Multiple Multicast channels- SSM mode- 20-30Mbit/s- FEC ¾- Closed loop monitoring based on successful file receipt- Packet/channel based ack/nack retransmission possible but not yet enabled
Acknowledgements: N-WAVE&NOAA Alex Hsia, David Hartzell
![Page 11: 1 Improving Data Mobility & Management for International Climate Science, Michael.Schick@eumetsat.int & Lothar.Wolf@eumetsat.int Challenges in the provision](https://reader038.vdocuments.mx/reader038/viewer/2022110319/56649c725503460f94924858/html5/thumbnails/11.jpg)
11Improving Data Mobility & Management for International Climate Science, [email protected] & [email protected]
EUMETCast Dissemination:Low cost, off the shelf Receiving Station
DVB PCI Card
EKU
AntennaLaptop/ PC
SoftwareDVB USB
DVB Router
Typical total cost of a station < 2000 Euros
![Page 12: 1 Improving Data Mobility & Management for International Climate Science, Michael.Schick@eumetsat.int & Lothar.Wolf@eumetsat.int Challenges in the provision](https://reader038.vdocuments.mx/reader038/viewer/2022110319/56649c725503460f94924858/html5/thumbnails/12.jpg)
12Improving Data Mobility & Management for International Climate Science, [email protected] & [email protected]
EUMETSAT Long Term Archive
Archived data starting 1981 All EUMETSAT Satellite data
including reprocessed data > 1 Petabytes in the archive
for MTP, MSG, Metop Retrieval/Ingestion: 10 to 1 Long-Term Data Preservation
in practice since 1995; Disaster resilient - 3rd Copy Access via EUMETSAT EO
Portal
![Page 13: 1 Improving Data Mobility & Management for International Climate Science, Michael.Schick@eumetsat.int & Lothar.Wolf@eumetsat.int Challenges in the provision](https://reader038.vdocuments.mx/reader038/viewer/2022110319/56649c725503460f94924858/html5/thumbnails/13.jpg)
13Improving Data Mobility & Management for International Climate Science, [email protected] & [email protected]
What we are working on…
![Page 14: 1 Improving Data Mobility & Management for International Climate Science, Michael.Schick@eumetsat.int & Lothar.Wolf@eumetsat.int Challenges in the provision](https://reader038.vdocuments.mx/reader038/viewer/2022110319/56649c725503460f94924858/html5/thumbnails/14.jpg)
14Improving Data Mobility & Management for International Climate Science, [email protected] & [email protected]
Outlook – Enhance Future abilities in terms of
• Scalability• Flexibility,• Responsiveness to emerging needs,• Resource efficiency• While aiming for:
• Better sharing of resources,• Avoidance of data islands,• Reducing costs
• Fit the organisational structure
![Page 15: 1 Improving Data Mobility & Management for International Climate Science, Michael.Schick@eumetsat.int & Lothar.Wolf@eumetsat.int Challenges in the provision](https://reader038.vdocuments.mx/reader038/viewer/2022110319/56649c725503460f94924858/html5/thumbnails/15.jpg)
EUM/TSS/VWG/14/762641, 7 July 201415
Many aspects to consider => Challenge
Viewing Angle
![Page 16: 1 Improving Data Mobility & Management for International Climate Science, Michael.Schick@eumetsat.int & Lothar.Wolf@eumetsat.int Challenges in the provision](https://reader038.vdocuments.mx/reader038/viewer/2022110319/56649c725503460f94924858/html5/thumbnails/16.jpg)
16Improving Data Mobility & Management for International Climate Science, [email protected] & [email protected]
Outlook - Product delivery and management
• Generation of user tailored products• Efficient for write once, read many times• Suitable products for user needs and
dissemination infrastructure (bandwidth)• Avoid/ minimise processing where possible• Efficient for slicing
• Usage of Format evolutions• Secure the raw data (golden copy)• Initially smaller products which increase over
time• Reprocessing to re-build formats
![Page 17: 1 Improving Data Mobility & Management for International Climate Science, Michael.Schick@eumetsat.int & Lothar.Wolf@eumetsat.int Challenges in the provision](https://reader038.vdocuments.mx/reader038/viewer/2022110319/56649c725503460f94924858/html5/thumbnails/17.jpg)
17Improving Data Mobility & Management for International Climate Science, [email protected] & [email protected]
Outlook - Bringing data to Users
• Adaptive and flexible services to deliver data to users/ communities (e.g. EUMETCast)
• Community driven formats optimised for dissemination mean
• Serving large user groups from different communities
• Managing data policies in accordance with data provider requirements via a data policy and legal framework support by the related technical mechanisms
![Page 18: 1 Improving Data Mobility & Management for International Climate Science, Michael.Schick@eumetsat.int & Lothar.Wolf@eumetsat.int Challenges in the provision](https://reader038.vdocuments.mx/reader038/viewer/2022110319/56649c725503460f94924858/html5/thumbnails/18.jpg)
18Improving Data Mobility & Management for International Climate Science, [email protected] & [email protected]
Outlook - Bringing Users to data
Standardisation activities in relation to Discovery, Access and Retrieval (DAR) of data
Strong focus on interoperability and ease of user experience
One stop shop experience for discovering services and data
Facilitating the Interoperability of portals by exploiting standards (OGC, INSPIRE)
Enhance Long Term Archive, Online Data Access
Outreach activities Exploiting capabilities to execute user algorithms on data
Post-Processing, Hosted Processing
![Page 19: 1 Improving Data Mobility & Management for International Climate Science, Michael.Schick@eumetsat.int & Lothar.Wolf@eumetsat.int Challenges in the provision](https://reader038.vdocuments.mx/reader038/viewer/2022110319/56649c725503460f94924858/html5/thumbnails/19.jpg)
19Improving Data Mobility & Management for International Climate Science, [email protected] & [email protected]
Outlook - Information to Users
• Delivering Information to users rather than data
• Processing where the data is located
• Service Interfaces instead Facility interfaces
• Application of (private) cloud principles
![Page 20: 1 Improving Data Mobility & Management for International Climate Science, Michael.Schick@eumetsat.int & Lothar.Wolf@eumetsat.int Challenges in the provision](https://reader038.vdocuments.mx/reader038/viewer/2022110319/56649c725503460f94924858/html5/thumbnails/20.jpg)
20Improving Data Mobility & Management for International Climate Science, [email protected] & [email protected]
Thank you for your attention