evaluation of different cloud computing resources … · cloud-computing platform, provided access...
TRANSCRIPT
![Page 1: EVALUATION OF DIFFERENT CLOUD COMPUTING RESOURCES … · cloud-computing platform, provided access via web portal, downloadable current trend software companies provided processing](https://reader034.vdocuments.mx/reader034/viewer/2022042913/5f4b2b7309a9b07336662b03/html5/thumbnails/1.jpg)
E VA L U AT I O N O F D I F F E R E N T C L O U D C O M P U T I N G R E S O U R C E S F O R L A R G E S C A L E S AT E L L I T E D ATA P R O C E S S I N G
P O T E N T I A L A N D L I M I TAT I O N
D R . T U O N G - T H U Y V U
![Page 2: EVALUATION OF DIFFERENT CLOUD COMPUTING RESOURCES … · cloud-computing platform, provided access via web portal, downloadable current trend software companies provided processing](https://reader034.vdocuments.mx/reader034/viewer/2022042913/5f4b2b7309a9b07336662b03/html5/thumbnails/2.jpg)
B I G D ATA
E A R T H O B S E R VAT I O N
S AT E L L I T E I M A G E D ATA
C L O U D C O M P U T I N G
![Page 3: EVALUATION OF DIFFERENT CLOUD COMPUTING RESOURCES … · cloud-computing platform, provided access via web portal, downloadable current trend software companies provided processing](https://reader034.vdocuments.mx/reader034/viewer/2022042913/5f4b2b7309a9b07336662b03/html5/thumbnails/3.jpg)
B I G D ATA
Volume Variety Velocity
analysis, capture, data curation, search, sharing, storage, transfer, visualisation, querying,
updating and information privacy.
CHALLENGE
![Page 4: EVALUATION OF DIFFERENT CLOUD COMPUTING RESOURCES … · cloud-computing platform, provided access via web portal, downloadable current trend software companies provided processing](https://reader034.vdocuments.mx/reader034/viewer/2022042913/5f4b2b7309a9b07336662b03/html5/thumbnails/4.jpg)
E A R T H O B S E R VAT I O N S AT E L L I T E D ATA
L A N D S AT 5
L A N D S AT 7
L A N D S AT 8
160 MB (compressed) 366 MB (uncompressed)
900 MB (compressed) 1.6 GB (uncompressed)
1 GB (compressed) 2 GB (uncompressed)
1 scene
![Page 5: EVALUATION OF DIFFERENT CLOUD COMPUTING RESOURCES … · cloud-computing platform, provided access via web portal, downloadable current trend software companies provided processing](https://reader034.vdocuments.mx/reader034/viewer/2022042913/5f4b2b7309a9b07336662b03/html5/thumbnails/5.jpg)
E A R T H O B S E R VAT I O N S AT E L L I T E D ATA
RapidEye
PlanetLab
Landsat
Sentinel-2
Multi-sensors
GeoEye
![Page 6: EVALUATION OF DIFFERENT CLOUD COMPUTING RESOURCES … · cloud-computing platform, provided access via web portal, downloadable current trend software companies provided processing](https://reader034.vdocuments.mx/reader034/viewer/2022042913/5f4b2b7309a9b07336662b03/html5/thumbnails/6.jpg)
E A R T H O B S E R VAT I O N S AT E L L I T E D ATA
RapidEye
PlanetLab
Landsat
Sentinel-2
Multi-sensors
Ground-truth
GeoEye
![Page 7: EVALUATION OF DIFFERENT CLOUD COMPUTING RESOURCES … · cloud-computing platform, provided access via web portal, downloadable current trend software companies provided processing](https://reader034.vdocuments.mx/reader034/viewer/2022042913/5f4b2b7309a9b07336662b03/html5/thumbnails/7.jpg)
E A R T H O B S E R VAT I O N S AT E L L I T E D ATA
RapidEye
PlanetLab
Landsat
Sentinel-2
Multi-sensors
Ground-truth
Modelling
GeoEye
![Page 8: EVALUATION OF DIFFERENT CLOUD COMPUTING RESOURCES … · cloud-computing platform, provided access via web portal, downloadable current trend software companies provided processing](https://reader034.vdocuments.mx/reader034/viewer/2022042913/5f4b2b7309a9b07336662b03/html5/thumbnails/8.jpg)
E A R T H O B S E R VAT I O N S AT E L L I T E D ATA
RapidEye
PlanetLab
Landsat
Sentinel-2
Multi-sensors
Ground-truth
Modelling
Spatial analysis
….
GeoEye
![Page 9: EVALUATION OF DIFFERENT CLOUD COMPUTING RESOURCES … · cloud-computing platform, provided access via web portal, downloadable current trend software companies provided processing](https://reader034.vdocuments.mx/reader034/viewer/2022042913/5f4b2b7309a9b07336662b03/html5/thumbnails/9.jpg)
Building Petabyte Data Production Systems with the NASA Earth Exchange
(http://www.nas.nasa.gov/SC15/demos/demo22.html)
• Production pipeline for Landsat and NASA Global Imagery and Browse Service – 37 steps, 7 petabytes of data, and 670,000 processor hours.
• NEX Downscaled Climate Projections (NEX-DCP30) and NEX Global Daily Downscaled Projections (NEX-GDDP) pipeline – 40 terabytes (TB) of data and 320,000 processor-hours.
• Estimating biomass by counting trees across the United States by using machine learning and computer vision – 40 TB of data/100,000 compute hours.
![Page 10: EVALUATION OF DIFFERENT CLOUD COMPUTING RESOURCES … · cloud-computing platform, provided access via web portal, downloadable current trend software companies provided processing](https://reader034.vdocuments.mx/reader034/viewer/2022042913/5f4b2b7309a9b07336662b03/html5/thumbnails/10.jpg)
C L O U D C O M P U T I N G
… a service over the Internet
I N F R A S T R U C T U R E ( I A A S )
P L AT F O R M ( PA A S )
S O F T WA R E ( S A A S )
![Page 11: EVALUATION OF DIFFERENT CLOUD COMPUTING RESOURCES … · cloud-computing platform, provided access via web portal, downloadable current trend software companies provided processing](https://reader034.vdocuments.mx/reader034/viewer/2022042913/5f4b2b7309a9b07336662b03/html5/thumbnails/11.jpg)
![Page 12: EVALUATION OF DIFFERENT CLOUD COMPUTING RESOURCES … · cloud-computing platform, provided access via web portal, downloadable current trend software companies provided processing](https://reader034.vdocuments.mx/reader034/viewer/2022042913/5f4b2b7309a9b07336662b03/html5/thumbnails/12.jpg)
![Page 13: EVALUATION OF DIFFERENT CLOUD COMPUTING RESOURCES … · cloud-computing platform, provided access via web portal, downloadable current trend software companies provided processing](https://reader034.vdocuments.mx/reader034/viewer/2022042913/5f4b2b7309a9b07336662b03/html5/thumbnails/13.jpg)
R E M O T E S E N S I N G D A T A S T O R E D O N T H E C L O U D - C O M P U T I N G P L A T F O R M , P R O V I D E D A C C E S S V I A W E B P O R TA L , D O W N L O A D A B L E
C U R R E N T T R E N D
S O F T W A R E C O M PA N I E S P R O V I D E D P R O C E S S I N G S E R V I C E S O N T H E C L O U D
A G E N C I E S P R O V I D E C L O U D - B A S E D P R O C E S S I N G T O O L S
L A N D S AT O N AW S
D I G I TA L G L O B E
E S R I C L O U D
E N V I C L O U D
E S A C L O U D T O O L B O X
N A S A E A R T H E X C H A N G E
![Page 14: EVALUATION OF DIFFERENT CLOUD COMPUTING RESOURCES … · cloud-computing platform, provided access via web portal, downloadable current trend software companies provided processing](https://reader034.vdocuments.mx/reader034/viewer/2022042913/5f4b2b7309a9b07336662b03/html5/thumbnails/14.jpg)
R E M O T E S E N S I N G D A T A S T O R E D O N T H E C L O U D - C O M P U T I N G P L A T F O R M , P R O V I D E D A C C E S S V I A W E B P O R TA L , D O W N L O A D A B L E
C U R R E N T T R E N D
S O F T W A R E C O M PA N I E S P R O V I D E D P R O C E S S I N G S E R V I C E S O N T H E C L O U D
A G E N C I E S P R O V I D E C L O U D - B A S E D P R O C E S S I N G T O O L S
I N C R E A S I N G R E S E A R C H O N S PAT I A L D ATA P R O C E S S I N G S E R V I C E S O N T H E C L O U D
L A N D S AT O N AW S
D I G I TA L G L O B E
E S R I C L O U D
E N V I C L O U D
E S A C L O U D T O O L B O X
N A S A E A R T H E X C H A N G E
![Page 15: EVALUATION OF DIFFERENT CLOUD COMPUTING RESOURCES … · cloud-computing platform, provided access via web portal, downloadable current trend software companies provided processing](https://reader034.vdocuments.mx/reader034/viewer/2022042913/5f4b2b7309a9b07336662b03/html5/thumbnails/15.jpg)
The most open platform so far …
![Page 16: EVALUATION OF DIFFERENT CLOUD COMPUTING RESOURCES … · cloud-computing platform, provided access via web portal, downloadable current trend software companies provided processing](https://reader034.vdocuments.mx/reader034/viewer/2022042913/5f4b2b7309a9b07336662b03/html5/thumbnails/16.jpg)
N A S A E A R T H E X C H A N G E
• A collaborative compute platform • Data • Tools & Libraries • Models • Disk storage
• Pleiades 1.3 PB total • Sandboxes: lnxsrv105: 163 TB
total and externally accessible sandbox 305TB
![Page 17: EVALUATION OF DIFFERENT CLOUD COMPUTING RESOURCES … · cloud-computing platform, provided access via web portal, downloadable current trend software companies provided processing](https://reader034.vdocuments.mx/reader034/viewer/2022042913/5f4b2b7309a9b07336662b03/html5/thumbnails/17.jpg)
D ATA S E T
Landcover Climate Datasets
MODIS Land MODIS Atmosphere
Landsat 5 Landsat 7 Landsat 8
Landsat GLS Web-Enabled Landsat
ASTER TRMM AVHRR
Ecosystem Modeling Datasets DEM Soils Aerial
![Page 18: EVALUATION OF DIFFERENT CLOUD COMPUTING RESOURCES … · cloud-computing platform, provided access via web portal, downloadable current trend software companies provided processing](https://reader034.vdocuments.mx/reader034/viewer/2022042913/5f4b2b7309a9b07336662b03/html5/thumbnails/18.jpg)
M O D E L S
Ecosystem Models • Terrestrial Observation and Prediction System (TOPS) North America • Biome-BGC USA • Lund-Potsdam-Jena Dynamic Global Model (LPJ) Global
Climate Models • The Weather Research and Forecasting (WRF) Model Regional
![Page 19: EVALUATION OF DIFFERENT CLOUD COMPUTING RESOURCES … · cloud-computing platform, provided access via web portal, downloadable current trend software companies provided processing](https://reader034.vdocuments.mx/reader034/viewer/2022042913/5f4b2b7309a9b07336662b03/html5/thumbnails/19.jpg)
T O O L S
Analytical Tools MATLAB, IDL/ENVI,
R for statistical computing and graphics Numpy and SciPy for Python
Matplotlib for plotting and graphics GNU Octave
Database Management Systems MySQL, SQLite, MongoDB
Data Manipulation Tools and Libraries MODIS Reprojection Tool (MRT)
MODIS Reprojection Tool for Swath (MRTSwath) HDF-EOS To GeoTIFF Conversion Tool (HEG)
The Land Data Operational Products Evaluation (LDOPE) Tools PROJ.4 Cartographic Projections Library
GDAL - Geospatial Data Abstraction Library NetCDF (Network Common Data Form) tools and libraries
….
![Page 20: EVALUATION OF DIFFERENT CLOUD COMPUTING RESOURCES … · cloud-computing platform, provided access via web portal, downloadable current trend software companies provided processing](https://reader034.vdocuments.mx/reader034/viewer/2022042913/5f4b2b7309a9b07336662b03/html5/thumbnails/20.jpg)
• a computing platform
• for petabyte-scale scientific analysis and visualisation of geospatial datasets
C O D E E D I T O R E X P L O R E R C L I E N T L I B R A R I E S
web-based IDE for writing and running scripts
a lightweight web app for
exploring our data catalog and running simple
analyses
Python and JavaScript
wrappers around our web API
![Page 21: EVALUATION OF DIFFERENT CLOUD COMPUTING RESOURCES … · cloud-computing platform, provided access via web portal, downloadable current trend software companies provided processing](https://reader034.vdocuments.mx/reader034/viewer/2022042913/5f4b2b7309a9b07336662b03/html5/thumbnails/21.jpg)
D ATA S E T
Landsat
Sentinel
MODIS
1m resolution aerial image data US National Agriculture Imagery Program
(NAIP)
Other sub-meter resolution sample images
SRTM 30-m
Landcover MODIS, MERIS
Surface temperature
Climate & Weather
World population
![Page 22: EVALUATION OF DIFFERENT CLOUD COMPUTING RESOURCES … · cloud-computing platform, provided access via web portal, downloadable current trend software companies provided processing](https://reader034.vdocuments.mx/reader034/viewer/2022042913/5f4b2b7309a9b07336662b03/html5/thumbnails/22.jpg)
T O O L S
Most of tools you need are here
Image Math operation, convolutions, edge detection, spectral transformation, object-based methods, compositing & mosaicking, etc.
Reducer Image reductions, statistics, raster-to-vector and vector-to-raster conversion, linear regression
Supervised classification Landsat algorithms Sentinel-1 algorithms
Chart Array …..
![Page 23: EVALUATION OF DIFFERENT CLOUD COMPUTING RESOURCES … · cloud-computing platform, provided access via web portal, downloadable current trend software companies provided processing](https://reader034.vdocuments.mx/reader034/viewer/2022042913/5f4b2b7309a9b07336662b03/html5/thumbnails/23.jpg)
A C C E S S I B I L I T Y & L I C E N S E
NASA/NDC or OpenID account, need to signup and be approved
NEX sandbox AWS, GEE, Azure, etc.
![Page 24: EVALUATION OF DIFFERENT CLOUD COMPUTING RESOURCES … · cloud-computing platform, provided access via web portal, downloadable current trend software companies provided processing](https://reader034.vdocuments.mx/reader034/viewer/2022042913/5f4b2b7309a9b07336662b03/html5/thumbnails/24.jpg)
A C C E S S I B I L I T Y & L I C E N S E
https://earthengine.google.com/signup. Google evaluates the request
Free for research, education, and nonprofit use.
• Can import your own data, download data & images to work with your own tools
• You own your written algorithms and results on Google EE
NASA/NDC or OpenID account, need to signup and be approved
NEX sandbox AWS, GEE, Azure, etc.
![Page 25: EVALUATION OF DIFFERENT CLOUD COMPUTING RESOURCES … · cloud-computing platform, provided access via web portal, downloadable current trend software companies provided processing](https://reader034.vdocuments.mx/reader034/viewer/2022042913/5f4b2b7309a9b07336662b03/html5/thumbnails/25.jpg)
• Cloud-computing platforms/resources are currently available
• Google Earth Engine is growing very fast, both basic and advanced remote sensing processing tools are available, free access
• NEX is more than just a computing platform, is a community and knowledge based system
![Page 26: EVALUATION OF DIFFERENT CLOUD COMPUTING RESOURCES … · cloud-computing platform, provided access via web portal, downloadable current trend software companies provided processing](https://reader034.vdocuments.mx/reader034/viewer/2022042913/5f4b2b7309a9b07336662b03/html5/thumbnails/26.jpg)
• Cloud-computing platforms/resources are currently available
• Google Earth Engine is growing very fast, both basic and advanced remote sensing processing tools are available, free access
• NEX is more than just a computing platform, is a community and knowledge based system
Ground truth?
Regional, local context?
Users (ready?)
![Page 27: EVALUATION OF DIFFERENT CLOUD COMPUTING RESOURCES … · cloud-computing platform, provided access via web portal, downloadable current trend software companies provided processing](https://reader034.vdocuments.mx/reader034/viewer/2022042913/5f4b2b7309a9b07336662b03/html5/thumbnails/27.jpg)
Crowd-2-Cloud
CROWD-SOURCINGREMOTE SENSING
PROCESSING SERVICES ON CLOUD COMPUTING
Training
Validation
Applications
![Page 28: EVALUATION OF DIFFERENT CLOUD COMPUTING RESOURCES … · cloud-computing platform, provided access via web portal, downloadable current trend software companies provided processing](https://reader034.vdocuments.mx/reader034/viewer/2022042913/5f4b2b7309a9b07336662b03/html5/thumbnails/28.jpg)
URBAN
AGR ICULTURE
E C O L O G Y
PUBL IC HEALTH
D ISASTER
• Remote sensing processing services on the cloud
• Disaster recovery monitoring • Damage mapping
• Urban green space
• Urban construction monitoring
• Megafauna index
BIG DATA ANALYTICS
• Crowd-sourcing data quality assessment
• Oil palm plantation management
• Health care delivery • Disaster Health
![Page 29: EVALUATION OF DIFFERENT CLOUD COMPUTING RESOURCES … · cloud-computing platform, provided access via web portal, downloadable current trend software companies provided processing](https://reader034.vdocuments.mx/reader034/viewer/2022042913/5f4b2b7309a9b07336662b03/html5/thumbnails/29.jpg)
on-demand, scalable processing capabilities efficiently co-located with large image databases rapid deployment of new algorithms sharing, collaborative platform
CLOUD COMPUTING PLATFORM
![Page 30: EVALUATION OF DIFFERENT CLOUD COMPUTING RESOURCES … · cloud-computing platform, provided access via web portal, downloadable current trend software companies provided processing](https://reader034.vdocuments.mx/reader034/viewer/2022042913/5f4b2b7309a9b07336662b03/html5/thumbnails/30.jpg)
on-demand, scalable processing capabilities efficiently co-located with large image databases rapid deployment of new algorithms sharing, collaborative platform
CLOUD COMPUTING PLATFORM
Promote open resources (data, tools, etc.) NEX and GEE
![Page 31: EVALUATION OF DIFFERENT CLOUD COMPUTING RESOURCES … · cloud-computing platform, provided access via web portal, downloadable current trend software companies provided processing](https://reader034.vdocuments.mx/reader034/viewer/2022042913/5f4b2b7309a9b07336662b03/html5/thumbnails/31.jpg)
on-demand, scalable processing capabilities efficiently co-located with large image databases rapid deployment of new algorithms sharing, collaborative platform
CLOUD COMPUTING PLATFORM
Potential or limitation depends on user’sOBJECTIVES
TECHNICAL LEVELS
Promote open resources (data, tools, etc.) NEX and GEE
![Page 32: EVALUATION OF DIFFERENT CLOUD COMPUTING RESOURCES … · cloud-computing platform, provided access via web portal, downloadable current trend software companies provided processing](https://reader034.vdocuments.mx/reader034/viewer/2022042913/5f4b2b7309a9b07336662b03/html5/thumbnails/32.jpg)
on-demand, scalable processing capabilities efficiently co-located with large image databases rapid deployment of new algorithms sharing, collaborative platform
CLOUD COMPUTING PLATFORM
Potential or limitation depends on user’sOBJECTIVES
TECHNICAL LEVELS
Promote open resources (data, tools, etc.) NEX and GEE
PaaS and SaaS
potential to deploy to various SaaS on demand
![Page 33: EVALUATION OF DIFFERENT CLOUD COMPUTING RESOURCES … · cloud-computing platform, provided access via web portal, downloadable current trend software companies provided processing](https://reader034.vdocuments.mx/reader034/viewer/2022042913/5f4b2b7309a9b07336662b03/html5/thumbnails/33.jpg)
on-demand, scalable processing capabilities efficiently co-located with large image databases rapid deployment of new algorithms sharing, collaborative platform
CLOUD COMPUTING PLATFORM
Potential or limitation depends on user’sOBJECTIVES
TECHNICAL LEVELS
Promote open resources (data, tools, etc.) NEX and GEE
PaaS and SaaS
potential to deploy to various SaaS on demand
Many other infrastructures, platforms were not discussed here including ESA, AIST (Japan), other national programmes
![Page 34: EVALUATION OF DIFFERENT CLOUD COMPUTING RESOURCES … · cloud-computing platform, provided access via web portal, downloadable current trend software companies provided processing](https://reader034.vdocuments.mx/reader034/viewer/2022042913/5f4b2b7309a9b07336662b03/html5/thumbnails/34.jpg)
T H A N K Y O U