big data earth observation · 28-11-17 1 florin Șerban, anca liana costea, codrina ilie...
TRANSCRIPT
28-11-17 1www.bdva.eu
Florin Șerban, Anca Liana Costea, Codrina IlieTERRASIGNA
on behalf of TF7 / SG5
Big Data Earth Observation
28-11-17 2www.bdva.eu
Main objective: linking the Earth Observation community and ICT community, in an effective dialog on tackling the big data from space related issues.
Earth Observation white paper
BDVA Earth Observation working group
28-11-17 3www.bdva.eu
BDVA Earth Observation groupGroupleader:
28-11-17 4www.bdva.eu
1. CONTEXT & OPPORTUNITIES2. ECONOMICS & BUSINESS OF EO3. EO DATA APPLICATION AREAS4. EUROPEAN JOINT EFFORTS ON BIG DATA5. BIG DATA IN SPACE – TECHNOLOGIES6. BIG DATA – INFRASTRUCTURE7. EO BIG DATA – SOCIAL DIMENSION8. BIG DATA – EDUCATION9. CONCLUSIONSANNEX I – EU DATA POLICYANNEX II - EARTH OBSERVATION DATA APPLICATION DOMAINS
a. AGRICULTUREb. FORESTRYc. SAFETY / SECURITY (INCLUDING DISASTER MONITORING / RISK ASSESSMENT/ INSURANCE
RISK ASSESSMENT)d. ENERGY - OIL AND GASe. MARINE AND MARITIME (INCLUDING COASTAL ZONE MANAGEMENT/ FISHERIES AND
AQUACULTURE)f. PUBLIC HEALTHg. URBAN DEVELOPMENT (INCLUDING SMART CITIES AND CULTURAL HERITAGE)
ANNEX III - COPERNICUS CORE SERVICESANNEX IV - ESA THEMATIC EXPLOITATION PLATFORMSANNEX V – EC AND ESA EO BIG DATA PROJECTS
BDVA Earth Observation White Paper
28-11-17 5www.bdva.eu
1. Context and Opportunities
4TB per day of Earth Observation open data!
Unprecedented access to a wealth of data that allows analysis and understanding of Earth as never before in human history
Copernicus Market Report 2016evaluated downstream market EUR 2.8 billion for 2015 and predicted EUR 5.3 billion for 2020
28-11-17 6www.bdva.eu
2. Economics and business of EOSector Revenues
for EO in 2015
Revenues for EO in
2020
Comments
Agriculture 71M€ 221 M€ Focus on precision farming onlyForestry 36 M€ 41M€Renewable Energy 22 M€ 44 M€ Solar, wind, hydro, biomassUrban development 45 M€ 99 M€ Urban monitoring
Ocean monitoring 103 M€ 116 M€Oil & Gas 73 M€ -- Consistent estimates for 2020 are not
provided
1. The EO community must implement the appropriate business models to transformthe EO activities into a sustainable, self-supporting business addressing thedemand of the users in the various economic sectors in the most efficient andcost-effective way.
2. The EO and ICT technologies together must manage to make available suitablesolutions to address the Big Data challenges of exploiting EO data.
28-11-17 7www.bdva.eu
3. EO DATA APPLICATION AREASAgricultureForestrySafety / Security / Disaster Monitoring / Risk Assessment/ Insurance Energy / Oil and Gas/ Resource ManagementOcean Monitoring: Marine and Maritime / coastal zone management/ Fisheries and AquaculturePublic HealthUrban Development/ Smart cities/ Cultural HeritageTransport - Navigation
Services / products using EO data (+ standards)Typical clients for EO services and applicationsEO companies: business models Collaboration between domain experts and data analysts Interdisciplinary collaboration Awareness / stimulating the use of EO dataConclusions: EO Big data related needs, barriers, challenges
28-11-17 8www.bdva.eu
4. European Joint Efforts on Big Data
28-11-17 9www.bdva.eu
4. European Joint Efforts on Big Data
H2020 2014, 2014-2015 and 2016-2017 –LEIT SPACEAdditional topics for EO - Societal Challenges (1+2)
Societal Challenge for climate action, environment, resource efficiency and raw materialsBlue Growth (Ocean observation technologies) Sustainable Food Security (smart farming),Competitiveness of the European Space Sector (COMPET)
28-11-17 10www.bdva.eu
4. European Joint Efforts on Big Data
Collaborative Ground SegmentThematic Exploitation PlatformsRegional Exploitation PlatformsData Access and Information Services
28-11-17 11www.bdva.eu
5. EO Big Data in Space - TechnologiesData Vizualization
Visual Data Mining3D Visualisation
Data AnalyticsMachine and Deep Learning
Data ProcessingComplex event processing
Data ManagementSatellite Image Time Series Content Based Image Retrieval
The3DvisualizationofaLandsat7ETM+imagebasedonspectraldescriptors,notingfivesemantic clusters
SampleofimmersivevisualizationofaRS
imagedataset
28-11-17 12www.bdva.eu
6. EO Big Data - InfrastructurePublic Infrastructure
NationalTHEIA Land Data Center (France)Climate, Environment and Monitoring from Space (UK)Etc.
InternationalGeohazard Exploitation Platform (GEP) integrated within EGI infrastructure
Private InfrastructureGoogle Earth Engine Amazon Web ServiceEO Cloud - Earth Observation Innovative Platform Testbed Poland
Hybrid Infrastructure Projected: The European Open Science Cloud
28-11-17 13www.bdva.eu
Weather and climate forecasts Monitoring our water and food resources Ensuring healthy, productive oceans Vulnerable ecosystems monitoring Contributing to human health
7. EO Big Data – Social Dimension
EO technologies should support policymakers in reaching the Sustainable Development Goals agenda of the United Nation.
28-11-17 14www.bdva.eu
8. EO Big Data Education
Key Challenge for Europe: to ensure the availability of highly skilled people
European Data Science Academy Eit Digital Master School
What is a Data Scientist? – the answer given within the H2020 Edison projectBig Data Value Ecosystem: activities with the objective of developing skills, education, and Centers of Excellence around Big Data (H2020)
28-11-17 15www.bdva.eu
Outside Europe
FIELDS program – by the University of California, Riverside (UCR) and NASA’s Jet Propulsion Laboratory - to train the next generation of scientists and engineers in large-scale data analysis and visualization.Initiative Building a Big Data Analytics Workforce in iSchools –University Penn State develops:
Geospatial Big Data Analytics and Geographic EducationGeospatial Big Data Visualization with Technology
8. EO Big Data Education
28-11-17 16www.bdva.eu
9. Conclusions – Challenges
Challenge-1: Generic framework for multi-level Big Data processing
standardization of further specialized data fusion, analytics and adaptive modelling with uncertainty control servicesestimating the propagation of uncertainty across the multiple Big Data sources at pre-processing and post-processing levels
28-11-17 17www.bdva.eu
9. Conclusions – ChallengesChallenge-2: Technical priorities
1. Data Management 2. Data Analytics 3. Data Advanced Visualisation 4. Data Standardization 5. Data ProcessingScalability Quality of data Multiple axis/parameters Ontologies Robustness
Adequate and affordable infrastructure tostore and process data.
Generating reliable high-quality andaffordable information for fieldmanagement (especially in complexurban systems).
Implementation of 3dimensionalvisualization services to improve theunderstanding of spatial relationshipsbetween image texture and topography,allowing land use features to be observednot only from the normal vertical view, butalso at different scales, and differentorientations and perspectives. As example,using this method, the range of sprawl andgrowth of cities can be identified anddetermined.
The lack of some standards, particularly tofacilitate data analysis at multiscale levels, (e.g.TileMatrixSet).
Data Fusion of multisource data.
Enterprise-class data integration that isdynamic - meeting current and futureperformance requirements - andextendable - partitioning for fast and easyscalability.
Evidence-based knowledge isrequired for the management of keypublic health issues.Detailed travel survey data to captureindividual activity-based behavior isrequired for Human Mobility PatternsInferred from Mobile Phone Data.
Access to a business intelligence (BI) andanalytics data in eye-catching and easy-to-understand formats.
Advanced instruments for territorialplanning/management are required. Data set hasto be accurate, objective, reliable, comprehensiveand always up-to-date, following an integration ofmultispectral (and hyperspectral) data fromvarious satellites.
Develop automatic or semi-automatedprocessing techniques of very highresolution imagery to successfully provideproducts that cover extended areas inshort time.
Cloud computing platforms to providevirtual clusters and elastic auto-scaling ofresources.
The lack of trained people onInnovative Data Analysis (SpatialAnalysis and Statistics)
Definition of advanced and integrated datavisualization solutions allowing new ways toview EO and non-EO data, throughinteractive visuals such as bubble charts,word clouds and dynamic geospatial maps.
Implementation of new standards for Big Data(EO, non-EO and fused products) metadata andmanagement.
For multi-temporal monitoring of changeson the land surface implement integratedservices with: Image differencing,vegetation index differencing, principalcomponent analysis, direct multi-dateunsupervised classification, post-classification change differencing and acombination of image enhancement andpost-classification comparison. Developspectral mixture analysis, artificial neuralnetworks for change detectionapplications. Developed a curvelet-basedchange detection algorithm for radarimages.
Designing isolation approach to use adata processing design in any hardwareconfiguration without needing to redesignand retune.
Use of multi-temporal imageryapproach - combined spring andsummer images.
Implementation of customized, innovative,networked and highly affordablevisualization solutions (e.g. for Smart CityPlatforms - indoor and outdoor areas).
Developing interoperable data infrastructurethrough extensible governance and metadatalifecycle framework.
Developing systems for a stable and fastreal-time data handling that can interpretand provide results in real-time.
28-11-17 18www.bdva.eu
9. Conclusions – Challenges
Challenge-3: Data Analysts and Knowledge Experts
Challenge-4: Data driven servicesThe main challenges of this kind of services are heterogeneity, lack of structure, error-handling, privacy, timeliness, interoperability and effective dissemination at all stages of the analysis workflow. These challenges are common across the different business models and it is not possible to find a single solution applicable to all of them.
28-11-17 19www.bdva.eu
8. ConclusionsSocial: The development of a fruitful and efficient ecosystem of Big Datain EO would have a great social impact in critical areas such as Weatherand climate forecasts, management of water resources, ensuring healthyand productive oceans, keeping vulnerable ecosystems safe, humanhealth and much more.
Economic: Business opportunities must be promoted to capitalizecurrent EO data. Public End-Users adoption needs to be encouraged bymarket-oriented policies (i.e. governance that promote privatedownstream services). In the near future, the innovation should focus onEO data feeds in other domains than the ‘traditional’ ones and oningesting other available data (than the ones already ‘traditionally’integrated) into EO services / applications.
Technical: Big EO Data collection, dissemination, processing and addedvalue service delivery for End-Users represent existing technologicalchallenges.
28-11-17 20www.bdva.eu
Thank you!Florin Șerban: [email protected] Costea: [email protected] Ilie: [email protected] & BDVA TF7 / SG5