marine geospatial ecology tools open source geoprocessing for marine ecology

Click here to load reader

Post on 11-Jan-2016

58 views

Category:

Documents

1 download

Embed Size (px)

DESCRIPTION

Marine Geospatial Ecology Tools Open Source Geoprocessing for Marine Ecology. Jason Roberts, Ben Best, Dan Dunn, Eric Treml, Pat Halpin Duke University Marine Geospatial Ecology Lab 4-Mar-2009. Talk outline. Overview of MGET Quick tour of the MGET tool collection - PowerPoint PPT Presentation

TRANSCRIPT

  • Jason Roberts, Ben Best, Dan Dunn, Eric Treml, Pat HalpinDuke University Marine Geospatial Ecology Lab4-Mar-2009

  • Talk outlineOverview of MGETQuick tour of the MGET tool collectionExample application: habitat modeling

  • What is MGET?A collection of geoprocessing tools for marine ecologyOceanographic data management and analysisHabitat modeling, connectivity modeling, statisticsHighly modular; designed to be used in many scenariosEmphasis on batch processing and interoperabilityFree, open source softwareWritten in Python, R, MATLAB, C#, and C++Minimum requirements: Win XP, Python 2.4 ArcGIS 9.1 or later currently needed for many toolsArcGIS and Windows are only non-free requirementsAlso useful for terrestrial problems!

  • Set goals & prioritiesMake & implement EBM decisionsMonitor &assessDevelop conceptual modelsAnalyze data & develop models and scenarios Collect physical, biological, and socioeconomic data

  • Set goals & prioritiesMake & implement EBM decisionsMonitor &assessDevelop conceptual modelsAnalyze data & develop models and scenarios Collect physical, biological, and socioeconomic dataMonitoring & assessment toolsProject management toolsData collection and management toolsConceptual modeling toolsData processing toolsModeling toolsModel development toolsWatershed modelsDispersal and habitat modelsMarine ecosystem modelsSocial science modelsSector-specific decision support toolsConservation and restoration site selectionCoastal zone management toolsFisheries management toolsHazard assessment and resiliency planning toolsLand use planning toolsStakeholder communication & engagement toolsScenario visualization tools

  • Set goals & prioritiesMake & implement EBM decisionsMonitor &assessDevelop conceptual modelsAnalyze data & develop models and scenarios Collect physical, biological, and socioeconomic dataMonitoring & assessment toolsProject management toolsConceptual modeling toolsSector-specific decision support toolsConservation and restoration site selectionCoastal zone management toolsFisheries management toolsHazard assessment and resiliency planning toolsLand use planning toolsStakeholder communication & engagement toolsScenario visualization toolsData collection and management toolsData processing toolsModeling toolsModel development toolsWatershed modelsDispersal and habitat modelsMarine ecosystem modelsSocial science modelsData collection and management toolsData processing toolsModeling toolsModel development toolsWatershed modelsDispersal and habitat modelsMarine ecosystem modelsSocial science modelsMGET

  • MGETs software architectureMGET tools are really just Python functions, e.g.:MGET exposes them to several types of external callers:def MyTool(input1, input2, input3, output1)

    IMyInterface

    IMyInterface

    IMyInterface

    IMyInterface

    IMyInterface

    IMyInterface

    IMyInterface

    IMyInterface

    IMyInterface

    IMyInterface

    IMyInterface

    IUnknown

    ObjectName

    pythoncom2x.dll

    IDispatch

    IMyTool

    MyToolCOM class

    MyTool.py

  • MGET interface in ArcGISThe MGET toolbox appears in the ArcToolbox window

  • MGET interface in ArcGISDrill into the toolbox to find the toolsDouble-click tools to execute directly, or drag to geoprocessing models to create a workflow

  • IntegrationThe Python functions can invoke C++, MATLAB, R, ArcGIS, and COM classes.

    IMyInterface

    IMyInterface

    IMyInterface

    IMyInterface

    IMyInterface

    IMyInterface

    IMyInterface

    IMyInterface

    IMyInterface

    IMyInterface

    IMyInterface

    IUnknown

    ObjectName

    R interpreter

    MyTool.m

    MyTool.r

    MyTool.py

  • MGET utilizes a lot of other softwareInterpreters / RuntimesPythonMATLAB Component RuntimeRR PackagesgamMASSmgcvrgdalROCRApplicationsArcGISNOAA CoastWatch UtilitiesPython Packagesdocutilshttplib2lxmlnetcdf4numpyosgeopydappyparsingpyprojpywin32rpysetuptoolsC LibrariesGDAL/OGRgziphdflibxmllibxsltnetcdfproj4zlibAll but one of these (pywin32) are installed automatically

  • Quick tour of the tools

  • Analyzing larval connectivityCoral reef ID and % cover mapsOcean currents dataTool downloads data for the region and dates you specifyLarval density time series rastersEdge list feature class representing dispersal networkOriginal research by Eric A. Treml

  • Converting data

  • Batch processingCopy one raster at a time

  • Batch processingCopy rasters that you list in a table

  • Batch processingCopy rasters from a directory tree

  • Tools for specific productsDownloads sea surface height data from http://opendap.aviso.oceanobs.com/thredds

  • Identifying SST frontsCayula and Cornillion (1992) edge detection algorithmFrequencyTemperatureOptimal break 27.0 CStrong cohesion front presentStep 1: Histogram analysisStep 2: Spatial cohesion testWeak cohesion no frontBimodalExample outputMexicoArcGIS model

  • Identifying geostrophic eddiesNegative W at eddy coreSSH anomalyAvailable in MGET 0.8Example output

  • Mapping species biodiversity

  • Invoking R from ArcGIS

  • Example application: habitat modelingPresence/absence observations Sampled environmental dataMultivariate statistical modelProbability of occurrence predicted from environmental covariatesBinary classificationWarning: Habitat modeling is complicated! This simplified example is meant to briefly illustrate tools. Consult the literature for best practices!

  • Focal species: Stenella frontalisPhoto: Garth MixMap: OBIS-SEAMAPCommon name:Atlantic Spotted DolphinDistribution:Tropical and warm temperate AtlanticStudy area: Eastern U.S.

  • Species observation dataThe Ocean Biogeographic Information System (OBIS) is a global database of marine species observations.The OBIS-SEAMAP system at Duke University holds the records for seabirds, marine mammals, and sea turtles, including records gathered during NOAA cruises.

  • Environmental predictor variablesBathymetry:ETOPO2V2 from NOAA NGDCSST:Monthly climatological 4km AVHRR Pathfinder from NOAA NODCChlorophyll:Monthly climatological SeaWiFS chlorophyll-a from NASA GSFCImages shown above are for month of March

  • Step 1: Download species pointsDownload points using MGET tool:Presence: Records of Stenella frontalisAbsence: Records of other cetaceansThe tool uses the DiGIR protocol to retrieve data from OBIS servers

  • Red: PresenceGreen: Absence

  • Step 2: Convert oceanography to Arc rastersDownload with FTP from NOAA and NASA:ETOPO2 bathymetry 1 binary fileAVHRR Pathfinder monthly climatological SST 12 HDF filesSeaWiFS monthly climatological chlorophyll 12 HDF filesConvert to ArcGIS rasters using MGET tools:

  • Step 3: Sample oceanography at points Need to sample rasters and populate fields Must sample SST and chlorophyll by date

  • Step 3: Sample oceanography at pointsSampling bathymetry is easy because it is staticTo sample dynamic data such as SST and chlorophyll, you must first calculate the paths to rasters to sample from the points datesThen use an MGET batch sampling tool

  • Step 4: Create exploratory plotsBest predictors: SST and Chl

  • Step 5: Fit, evaluate, and predict modelPresence ~ s(SST) + s(log10(Chlorophyll))

  • Partial plots produced by the Fit GAM toolSSTSSTlog10(Chlorophyll)s(SST,8.97)s(log10(Chlorophyll),5.6)Presence more likely at higher SSTPresence more likely at lower Chl

  • Plotting a receiver operating characteristic curve

  • The ROC plotFalse positive rateTrue positive rateCutoff = 0.020By default, tool selects the cutoff closest to the point of perfect classification (0, 1)Model summary statistics:

    Area under the ROC curve (auc) = 0.960779Mean cross-entropy (mxe) = 0.030566Precision-recall break-even point (prbe) = 0.001866Root-mean square error (rmse) = 0.087781

    Contingency table for cutoff = 0.019638:

    Actual P Actual N TotalPredicted P 287 3541 3828Predicted N 26 32408 32434 Total 313 35949 36262

    Accuracy (acc) = 0.901633Error rate (err) = 0.098367Rate of positive predictions (rpp) = 0.105565Rate of negative predictions (rnp) = 0.894435

    True positive rate (tpr, or sensitivity) = 0.916933False positive rate (fpr, or fallout) = 0.098501True negative rate (tnr, or specificity) = 0.901499False negative rate (fnr, or miss) = 0.083067

    Positive prediction value (ppv, or precision) = 0.074974Negative prediction value (npv) = 0.999198Prediction-conditioned fallout (pcfall) = 0.925026Prediction-conditioned miss (pcmiss) = 0.000802

    Matthews correlation coefficient (mcc) = 0.246384Odds ratio (odds) = 101.026394SAR = 0.650065ROC summary stats for cutoff:

  • Predicting presence for oceanographic rasters

  • Rasters output by the Predict GAM toolPredicted presence:Range: 0 - 0.25Standard errors:Range: 0 - 0.11Binary classification:Species range map produced by classifying presence into 0 or 1 according to ROC cutoffSimilar to OBIS-SEAMAP range map?Predictions for October

  • AcknowledgementsA special thanks to the many developers of the open source software that MGET is built upon, including:Guido van Rossum and his many collaborators; Mark Hammond; Travis Oliphant and his collaborators; Walt

View more