This material is based upon work supported by the National Science Foundation under Grant No. ANT-0424589. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author (s) and do not necessarily reflect the views of the National

Science Foundation.

Center for Remote Sensing of Ice SheetsCenter for Remote Sensing of Ice SheetsHeadquarters, University of KansasHeadquarters, University of Kansas

Workshop

DetailsWho: Association of Computer/Information Sciences and Engineering Departments at Minority Institutions (ADMI) faculty/students

Where: Elizabeth City State University (ECSU)

When: June 7 - July 5 2011

What: A Teach-One-Teach-Many approach to cloud computing

Purpose•Introduce ADMI to the basics of the emerging Cloud Computing paradigm

•Understand the computer systems constraints, tradeoffs, and techniques of setting up and using cloud

•Understand how different algorithms can be implemented and executed on cloud frameworks

•Evaluating the performance and identifying bottlenecks when mapping applications to the clouds

A Cloudy View on Computing workshop and CReSIS Field Data Accessibility Jerome Mitchell1, Jun Wang1, Geoffrey Fox1, Linda Hayden2

Indiana University1, Elizabeth City State University2

Schedule

Now I understand Cloud Computing

Now I appreciate why Cloud Computing is important

Now I really understand Cloud Computing!

ParallelProcessing

Map /Reduce

Algorithm Hadoop Twister

Programming Model

Used by

Parallelized by

Apache’s implementation

CGL’s implementation

End of 1st Week

End of 3rd Week

End of 5th Week

Time

Iine

FunctionalProgramming

Compute Resources•FutureGrid

•Virtual machines + virtual networking to create sandboxed moduleso Virtual “Grid” appliances: self-contained, pre-packaged

execution environmentso Group VPNs: simple management of virtual clusters by students

and educators

CReSIS Field Data Accessibility

Current CReSIS Data Organization •CReSIS’s data products website lists

o direct download links for individual files•The data are organized by season

o Seasons are broken into data segments•Data segments are arranged into frames

o Associated data for each frame are stored in different file formats CSV (flight path) MAT (depth sounder data) PDFs (image products)

•File-based data system has no spatial data access support

Spatial Data Accessibility Project•Two main components: Cloud distribution service and special service for PolarGrid field crew.

•Data is supported among multiple spatial databases.

Google EarthGoogle Earth

Matlab/GISMatlab/GIS

GeoServerSpatial Database

GeoServerSpatial Database

GIS Cloud Service

WMS

KML

Online Data Distribution Field Data Access

SpatiaLiteSQLite Database

SpatiaLiteSQLite Database

Field Data Service

Spatial DatabaseVirtual Appliance Spatial DatabaseVirtual Appliance

Data PortalData Portal

Single UserSingle User

Multiple Users(local network)Multiple Users(local network)

Virtual StorageService

Virtual StorageService

Cloud GIS Distribution Service

Google Earth Example

2009 Antarctica Season

Overview of 2009 Flight Paths Data Access for Single Frame

SpatiaLite Databaseo Spatial extension to manages both vector and raster data and supports a rich

set of GIS analysis functions through SQL.

•The data can be directly accessed through GIS software and MATLAB

SpatiaLite Database Example•2009 Antarctic flight path data

o ~ 4 million entries - originally stored as 828 separate files and imported into one SpatiaLite database file

2009 Antarctica Season Vector Data Visual Crossover Analysis for Quality Control (development project)

Flight path data stored as YYYYMMDD_segID_frameID.txtSQLite command to create the segs table:

CREATE TABLE segs ( UTCTime Number, Thickness Number, Elevation Number, FrameID VARCHAR(12), Surface Number, Bottom Number, QualityLevel Integer)

SELECT AddGeometryColumn ('segs','geometry',4326,'POINT',2)*note: geometry: 2 -> xy, (longitude, latitude), 4326 -> WGS84 coordinate system

SpatiaLite: MATLAB Direct AccessMksqlite package: a MEX-DLL to access SQLite databases from MATLAB http://mksqlite.berlios.de/Add this flag to build.m to enable SQLite to load SpatiaLite as an extension: -DSQLITE_ENABLE_LOAD_EXTENSION=1Testing in MATLAB:dbid = mksqlite(0,'open', ‘test.sqlite' )sql = ['SELECT load_extension(''', path_to_spatialite, ''')'];mksqlite(dbid, sql) % load extensionmksqlite(dbid, 'SELECT sqlite_version()')mksqlite(dbid, 'SELECT spatialite_version()')mksqlite(dbid, 'SELECT X(geometry) as lon, Y(geometry) as lat from segs where FrameID=2009101601001'); mksqlite(dbid, 'close')

ReferencesPolarGrid Data Products: https://www.cresis.ku.edu/dataSpatiaLite: http://www.gaia-gis.it/spatialite/Quantum GIS: http://www.qgis.org/