a walk-through on machine learning techniques for sentinel
TRANSCRIPT
![Page 1: A Walk-Through on Machine Learning Techniques for Sentinel](https://reader036.vdocuments.mx/reader036/viewer/2022062222/62a45cc4bbc82c3916535a0b/html5/thumbnails/1.jpg)
Issue/Revision: 0.0
Reference: Status: ESA UNCLASSIFIED - For Official Use
A Walk-Through on Machine Learning Techniques for Sentinel Big Data Fusion
Sara Aparício
Φ-Week - 12th November 2018
ESA ESRIN, EOP-Φ
@_SaraAparicio_
![Page 2: A Walk-Through on Machine Learning Techniques for Sentinel](https://reader036.vdocuments.mx/reader036/viewer/2022062222/62a45cc4bbc82c3916535a0b/html5/thumbnails/2.jpg)
ESA/PB-EO/173/RoomDoc(2018)20 | Slide 2ESA UNCLASSIFIED - For Official Use
The world’s most comprehensive suite of dedicated EO missions.
20 Tb daily (2017)
![Page 3: A Walk-Through on Machine Learning Techniques for Sentinel](https://reader036.vdocuments.mx/reader036/viewer/2022062222/62a45cc4bbc82c3916535a0b/html5/thumbnails/3.jpg)
Major trends
a) The advent of cloud computing & increase computing power,b) proliferation of open-access satellite data streams, c) growing use of machine-learning algorithms
Supervised LearningDevelop predictive model based on
both input and output data
Classification
Regression
Unsupervised LearningGroup and interpret data based only
on input dataClustering
Artificial Intelligence
Machine Learning
Deep Learning
![Page 4: A Walk-Through on Machine Learning Techniques for Sentinel](https://reader036.vdocuments.mx/reader036/viewer/2022062222/62a45cc4bbc82c3916535a0b/html5/thumbnails/4.jpg)
VV
Sentinel data fusion with Machine Learning Techniques
Classification
Machine Learning #1Machine Learning #2Machine Learning #3Machine Learning #4Machine Learning #5Machine Learning #6
Classify
VH
ClassificationClassification
ClassificationClassificationTime
Multisensor/multitemporal dataBest technique?
Best band selection?
VVstdDevVV/VHVV
VH
VVstdDevVV/VH
B4 – B3 – B2
B1 – B5 – B6 – B7 B8A – B9 – B10 –
B11B4 – B3 – B2
B1 – B5 – B6 – B7 B8A – B9 – B10 –
B11NDVI
NDWI
![Page 5: A Walk-Through on Machine Learning Techniques for Sentinel](https://reader036.vdocuments.mx/reader036/viewer/2022062222/62a45cc4bbc82c3916535a0b/html5/thumbnails/5.jpg)
ESA/PB-EO/173/RoomDoc(2018)20 | Slide 5ESA UNCLASSIFIED - For Official Use
Motivation I – Merging different datatypes
A promising direction of machine learning in Earth Observation is its pairing with data fusion
Sentinel-2
Sensitive to soil properties
Day & night capabilityEasy interpretation due to true colour image
Gives information on spectral signaturesGives information on soil features
![Page 6: A Walk-Through on Machine Learning Techniques for Sentinel](https://reader036.vdocuments.mx/reader036/viewer/2022062222/62a45cc4bbc82c3916535a0b/html5/thumbnails/6.jpg)
Motivation 2 – AI4EO the white paper
This document addresses elements of what needs to be done at European level, some specifically by ESA, to harness the full potential of artificial intelligence to exploit earth observation data and the data supply chain and vice versa.
It captures the recommendations of a community-led AI4EO workshop held at ESA/ESRIN on 27 Mar 2018 (c.f. participant list in Annex 1). The aim of the workshop was to informally assess progress in the development and application of AI techniques to the world of EO and to explore the potential value of a concerted Research and Innovation (R&I) effort on this topic at European level. This document is meant to be a dynamic report capturing the evolving needs of the community. It will be reviewed and updated in a follow-on workshop at ESA/ESRIN on November 14th, 2018…….(….)
Classification/ Recognition- updating land-cover maps
Detection- large scale in automatic basis- use SAR in machine learning
Data Fusion- Merging diverse EO data
![Page 7: A Walk-Through on Machine Learning Techniques for Sentinel](https://reader036.vdocuments.mx/reader036/viewer/2022062222/62a45cc4bbc82c3916535a0b/html5/thumbnails/7.jpg)
Objectives/Goal
1. To develop a pixel-based classification, reproducible, scalable with a machine learning-based approach of large-area mapping/land cover of high resolution (10m) based on a multi-sensor & multi-temporal approach;
2. To evaluate the additive value of open-access satellite optical and radar variables, processed using cloud computing, to a topographic baseline model.
3. To explore/understanding/address efficiency of Google Earth Engine to effectively execute big data workflows using machine learning techniques on Google Earth Engine (and accuracy) for multi-temporal land use mapping.
![Page 8: A Walk-Through on Machine Learning Techniques for Sentinel](https://reader036.vdocuments.mx/reader036/viewer/2022062222/62a45cc4bbc82c3916535a0b/html5/thumbnails/8.jpg)
Goole Earth Engine
Code Editor ConsoleScript manager
Map Output Panel
LANDSAT DATA
MODIS DATAPROBA-V
ElevationClimate
Topography
![Page 9: A Walk-Through on Machine Learning Techniques for Sentinel](https://reader036.vdocuments.mx/reader036/viewer/2022062222/62a45cc4bbc82c3916535a0b/html5/thumbnails/9.jpg)
METHODOLOGY
![Page 10: A Walk-Through on Machine Learning Techniques for Sentinel](https://reader036.vdocuments.mx/reader036/viewer/2022062222/62a45cc4bbc82c3916535a0b/html5/thumbnails/10.jpg)
(Sentinel-2 Natural Color)
Main reasons for selection:1) Ground truth data available 2) Relatively plain area > since ground range
products (GRD) were already terrain corrected3) A good variety of land cover types to access >
agriculture, forest, water and urban areas.
West Wieljopolska
Case study
Landcover classesAgriculture, Coniferous forest, Mixed Forest, Grassland, Bare soil, Wetland, Urban fabric, Water body.
![Page 11: A Walk-Through on Machine Learning Techniques for Sentinel](https://reader036.vdocuments.mx/reader036/viewer/2022062222/62a45cc4bbc82c3916535a0b/html5/thumbnails/11.jpg)
Pre-processed on EE
Workflow – data processing & preparation
Sentinel-1
Apply orbit fileBorder noise removal
Thermal noise removalRadiometric calibration
Terrain corrected
Filtering data
Filter data:
IWBoth orbits
Further processing& data preparation
Speckle Filter Lee 3x3Band ratios creation
Mean, std Dev calc.
Sentinel-2
Time selectionClip to ROI
Filtering data
Filter data:Cloudy Pixel %<30
Time selectionClip to ROI
Data processing
Cloud masked
Cirrus masked
NDVI, NDWI
Stacking
Image
![Page 12: A Walk-Through on Machine Learning Techniques for Sentinel](https://reader036.vdocuments.mx/reader036/viewer/2022062222/62a45cc4bbc82c3916535a0b/html5/thumbnails/12.jpg)
Workflow – input dataImage
Training Data Validation Data
Sample regions
(..…)
ML #1 ML #5ML #4ML #2 ML #3 ML #6
![Page 13: A Walk-Through on Machine Learning Techniques for Sentinel](https://reader036.vdocuments.mx/reader036/viewer/2022062222/62a45cc4bbc82c3916535a0b/html5/thumbnails/13.jpg)
Trained
Workflow – classification
CART NBPERRF SVM GMO
Classification &Regression trees Random Forest Support Vector
MachinesPerceptron Naïve Bayes GMO Maximum
Entropy
Training Data Validation Data
Image
Output Output Output Output Output Output
Test model performances:- Overall Accuracy
- Kappa- Producers’acc.
- Users’s acc.Output
Classification:
![Page 14: A Walk-Through on Machine Learning Techniques for Sentinel](https://reader036.vdocuments.mx/reader036/viewer/2022062222/62a45cc4bbc82c3916535a0b/html5/thumbnails/14.jpg)
RESULTS OVERVIEW
![Page 15: A Walk-Through on Machine Learning Techniques for Sentinel](https://reader036.vdocuments.mx/reader036/viewer/2022062222/62a45cc4bbc82c3916535a0b/html5/thumbnails/15.jpg)
Machine Learning performances – a first overview!
![Page 16: A Walk-Through on Machine Learning Techniques for Sentinel](https://reader036.vdocuments.mx/reader036/viewer/2022062222/62a45cc4bbc82c3916535a0b/html5/thumbnails/16.jpg)
Classifier - CARTBands – VVμ
Classifier - CARTBands – VVμ +VHμ
Classifier - CARTBands – VVμ +VHμ + VVσ
Classifier - CARTBands – VV μ (lee)
![Page 17: A Walk-Through on Machine Learning Techniques for Sentinel](https://reader036.vdocuments.mx/reader036/viewer/2022062222/62a45cc4bbc82c3916535a0b/html5/thumbnails/17.jpg)
A closer look
Classifier - CARTBands - NDVI NDWI
Classifier - CARTBands – B4 B8 B11
![Page 18: A Walk-Through on Machine Learning Techniques for Sentinel](https://reader036.vdocuments.mx/reader036/viewer/2022062222/62a45cc4bbc82c3916535a0b/html5/thumbnails/18.jpg)
Classifier - RFBands - NDVI NDWI
Classifier - RFBands – B4 B8 B11
Classifier - RFBands – All except…
Classifier - RFBands – All S2 bands
Classifier - RFBands – S1 + S2 bands
![Page 19: A Walk-Through on Machine Learning Techniques for Sentinel](https://reader036.vdocuments.mx/reader036/viewer/2022062222/62a45cc4bbc82c3916535a0b/html5/thumbnails/19.jpg)
Behaviour to data inputAveraged from best performing models
![Page 20: A Walk-Through on Machine Learning Techniques for Sentinel](https://reader036.vdocuments.mx/reader036/viewer/2022062222/62a45cc4bbc82c3916535a0b/html5/thumbnails/20.jpg)
Improving model performances…Changing temporal coverage- updating land-cover maps
Changing training dataIn size or coverage
Trying new band selectionDifferent combinations of S1+S2
![Page 21: A Walk-Through on Machine Learning Techniques for Sentinel](https://reader036.vdocuments.mx/reader036/viewer/2022062222/62a45cc4bbc82c3916535a0b/html5/thumbnails/21.jpg)
Best performing models with S1+S2 variations
![Page 22: A Walk-Through on Machine Learning Techniques for Sentinel](https://reader036.vdocuments.mx/reader036/viewer/2022062222/62a45cc4bbc82c3916535a0b/html5/thumbnails/22.jpg)
Other accuracy parameters for best band & models
Random Forest
Support Vector Machines
CART
Random Forest (Tuned)
![Page 23: A Walk-Through on Machine Learning Techniques for Sentinel](https://reader036.vdocuments.mx/reader036/viewer/2022062222/62a45cc4bbc82c3916535a0b/html5/thumbnails/23.jpg)
Further accuracy metrics…Reference
Classified Water body Wetland Urban Fabric
Agriculture Coniferous Forest
Mixed Forest
Grassland Bare soil
PA UA
Water body 193 0 0 0 0 0 0 0 1.000 1.00
Wetland 0 29 0 1 0 0 0 00.967 0.94
Urban Fabric 0 0 239 2 0 0 0 00.992 1.00
Agriculture 0 1 0 428 0 1 0 00.995 0.99
Coniferous Forest
0 0 0 0 154 2 0 00.987 0.99
Mixed Forest 0 0 0 0 2 157 0 00.987 0.98
Grassland 0 1 0 1 0 1 60 00.952 1.00
Bare soil 0 0 0 0 0 0 0 5 1.000 1.00
Kappa0.9881
Overall Accuracy0.9906
![Page 24: A Walk-Through on Machine Learning Techniques for Sentinel](https://reader036.vdocuments.mx/reader036/viewer/2022062222/62a45cc4bbc82c3916535a0b/html5/thumbnails/24.jpg)
Ground Truth/Reference VS Final model
![Page 25: A Walk-Through on Machine Learning Techniques for Sentinel](https://reader036.vdocuments.mx/reader036/viewer/2022062222/62a45cc4bbc82c3916535a0b/html5/thumbnails/25.jpg)
Line of trees
Wetland & Grassland
Crop transition
Road construction
Isolated strucutre
![Page 26: A Walk-Through on Machine Learning Techniques for Sentinel](https://reader036.vdocuments.mx/reader036/viewer/2022062222/62a45cc4bbc82c3916535a0b/html5/thumbnails/26.jpg)
Limitations…bear in mind that….
So…. Is this then the best model?
• Training data (plays a huge role) affecting:Models performancesSmaller size – the more data input…not always is the bestBigger size – more difference in ‘additive power’ but models behaved + similarly
• Tuning hyperparameters of ML are done manually Some models were inserted by default & data was not normalized
![Page 27: A Walk-Through on Machine Learning Techniques for Sentinel](https://reader036.vdocuments.mx/reader036/viewer/2022062222/62a45cc4bbc82c3916535a0b/html5/thumbnails/27.jpg)
ESA/PB-EO/173/RoomDoc(2018)20 | Slide 27ESA UNCLASSIFIED - For Official Use
What potential…?
![Page 28: A Walk-Through on Machine Learning Techniques for Sentinel](https://reader036.vdocuments.mx/reader036/viewer/2022062222/62a45cc4bbc82c3916535a0b/html5/thumbnails/28.jpg)
Example of straightforward applications…
• Analysis of temporal land use and land cover change…
Attention!This graph moves -
![Page 29: A Walk-Through on Machine Learning Techniques for Sentinel](https://reader036.vdocuments.mx/reader036/viewer/2022062222/62a45cc4bbc82c3916535a0b/html5/thumbnails/29.jpg)
Water bodies mapping – Random Forest
Larger scale + less classes
Mapping water bodies
![Page 30: A Walk-Through on Machine Learning Techniques for Sentinel](https://reader036.vdocuments.mx/reader036/viewer/2022062222/62a45cc4bbc82c3916535a0b/html5/thumbnails/30.jpg)
Larger scale + less classes
Mapping forest cover
![Page 31: A Walk-Through on Machine Learning Techniques for Sentinel](https://reader036.vdocuments.mx/reader036/viewer/2022062222/62a45cc4bbc82c3916535a0b/html5/thumbnails/31.jpg)
Large scale + data input
Elevation(DEM STRM)
+
+
![Page 32: A Walk-Through on Machine Learning Techniques for Sentinel](https://reader036.vdocuments.mx/reader036/viewer/2022062222/62a45cc4bbc82c3916535a0b/html5/thumbnails/32.jpg)
Large scale + data input + classes
S1 +S2 = + accurate + frequent +higher resolution
CORIN LAND COVER (100m)
![Page 33: A Walk-Through on Machine Learning Techniques for Sentinel](https://reader036.vdocuments.mx/reader036/viewer/2022062222/62a45cc4bbc82c3916535a0b/html5/thumbnails/33.jpg)
Overall Conclusions
Google Earth Engine• GEE offers powerful capabilities in handling large volumes of remote sensing imagery• GEE contains state-of-art machine learning algorithms achieving high accuracies and
excellent tool for rapidly prototype AI applications.• A big limitation is the need of manually tune the machine learning algorithmsAdded value of fusing Sentinel data• The integration of texture and spectral information for pixel-based classification improves
classification accuracy (S2 outperforms S1 alone but together detect finer structures).• Data: the more, the merrier!Large scale mapping – further work?• Results can be used to calculate/estimate use cover and land change dynamics• Normalize data and test more combinations!
![Page 34: A Walk-Through on Machine Learning Techniques for Sentinel](https://reader036.vdocuments.mx/reader036/viewer/2022062222/62a45cc4bbc82c3916535a0b/html5/thumbnails/34.jpg)
Thank you for your attention!