week 10 lecture: data and batch geoprocessing...
TRANSCRIPT
![Page 1: Week 10 Lecture: Data and Batch Geoprocessing …web.clas.ufl.edu/users/forrest/gis/lectures/2013-10-22...Week 10 Lecture: Data and Batch Geoprocessing With ArcGIS Introduction to](https://reader031.vdocuments.mx/reader031/viewer/2022030702/5aeda03f7f8b9ab24d91c352/html5/thumbnails/1.jpg)
Week 10 Lecture:
Data and Batch Geoprocessing With ArcGIS
Introduction to Programming for GIS & Remote Sensing GEO6938-1469 GEO4938-147A
![Page 2: Week 10 Lecture: Data and Batch Geoprocessing …web.clas.ufl.edu/users/forrest/gis/lectures/2013-10-22...Week 10 Lecture: Data and Batch Geoprocessing With ArcGIS Introduction to](https://reader031.vdocuments.mx/reader031/viewer/2022030702/5aeda03f7f8b9ab24d91c352/html5/thumbnails/2.jpg)
Re-cap From Past Lectures
• We’ve discussed modularization
– Subroutines
– Functions
– Object-orientation
• All this leads to one thing:
– Code Re-use
• Functions
• Imports
• Packages
![Page 3: Week 10 Lecture: Data and Batch Geoprocessing …web.clas.ufl.edu/users/forrest/gis/lectures/2013-10-22...Week 10 Lecture: Data and Batch Geoprocessing With ArcGIS Introduction to](https://reader031.vdocuments.mx/reader031/viewer/2022030702/5aeda03f7f8b9ab24d91c352/html5/thumbnails/3.jpg)
Code Re-Use
• The goal is to make life easier for us!
• Complex operations and procedures can be stored
– Including documentation Straight into articles…
– Write once… run many times
– Entire processing frameworks can be incorporated:
• ArcGIS’ Geoprocessing!
![Page 4: Week 10 Lecture: Data and Batch Geoprocessing …web.clas.ufl.edu/users/forrest/gis/lectures/2013-10-22...Week 10 Lecture: Data and Batch Geoprocessing With ArcGIS Introduction to](https://reader031.vdocuments.mx/reader031/viewer/2022030702/5aeda03f7f8b9ab24d91c352/html5/thumbnails/4.jpg)
Objectives This Week
• Learn about ArcGIS’ Geoprocessing objects
• Python facilities within ArcGIS
• Building Models and integrating with scripts
• Interface ArcGIS and other tools using Python as “glue”
![Page 5: Week 10 Lecture: Data and Batch Geoprocessing …web.clas.ufl.edu/users/forrest/gis/lectures/2013-10-22...Week 10 Lecture: Data and Batch Geoprocessing With ArcGIS Introduction to](https://reader031.vdocuments.mx/reader031/viewer/2022030702/5aeda03f7f8b9ab24d91c352/html5/thumbnails/5.jpg)
What is the ArcGIS Geoprocessing Framework?
• An “ArcObjects” collection
– Really just a single object with many, many sub-components
• Allows accessing geoproecessing tools as native methods
![Page 6: Week 10 Lecture: Data and Batch Geoprocessing …web.clas.ufl.edu/users/forrest/gis/lectures/2013-10-22...Week 10 Lecture: Data and Batch Geoprocessing With ArcGIS Introduction to](https://reader031.vdocuments.mx/reader031/viewer/2022030702/5aeda03f7f8b9ab24d91c352/html5/thumbnails/6.jpg)
From Last Week: Models/Tools to Python
![Page 7: Week 10 Lecture: Data and Batch Geoprocessing …web.clas.ufl.edu/users/forrest/gis/lectures/2013-10-22...Week 10 Lecture: Data and Batch Geoprocessing With ArcGIS Introduction to](https://reader031.vdocuments.mx/reader031/viewer/2022030702/5aeda03f7f8b9ab24d91c352/html5/thumbnails/7.jpg)
ArcGIS: Models/Tools to Python
![Page 8: Week 10 Lecture: Data and Batch Geoprocessing …web.clas.ufl.edu/users/forrest/gis/lectures/2013-10-22...Week 10 Lecture: Data and Batch Geoprocessing With ArcGIS Introduction to](https://reader031.vdocuments.mx/reader031/viewer/2022030702/5aeda03f7f8b9ab24d91c352/html5/thumbnails/8.jpg)
Pre-ArcGIS 10.0: The Geoprocessor Object
• How do we instantiate the Geoprocessor? import arcgisscripting
gp = arcgisscripting.create()
• How do we interact with the Geoprocessor? gp.clip_analysis(In,Clip,Out) # alias for:
gp.Toolbox = “Analysis”
gp.clip(In,Clip,Out)
Pyt
ho
n
![Page 9: Week 10 Lecture: Data and Batch Geoprocessing …web.clas.ufl.edu/users/forrest/gis/lectures/2013-10-22...Week 10 Lecture: Data and Batch Geoprocessing With ArcGIS Introduction to](https://reader031.vdocuments.mx/reader031/viewer/2022030702/5aeda03f7f8b9ab24d91c352/html5/thumbnails/9.jpg)
Post-ArcGIS 10.0: The arcpy Object
• How do we instantiate the Geoprocessor? import arcpy
• How do we interact with the Geoprocessor? arcpy.Clip_management( … )
arcpy.CheckOutExtension("Spatial")
from arcpy.sa import *
arcpy.env.overwriteOutput = overwrite
…
Pyt
ho
n
![Page 10: Week 10 Lecture: Data and Batch Geoprocessing …web.clas.ufl.edu/users/forrest/gis/lectures/2013-10-22...Week 10 Lecture: Data and Batch Geoprocessing With ArcGIS Introduction to](https://reader031.vdocuments.mx/reader031/viewer/2022030702/5aeda03f7f8b9ab24d91c352/html5/thumbnails/10.jpg)
The arcpy Geoprocessor Object
• Uses environment settings to define the operating conditions, input/output, defaults
![Page 11: Week 10 Lecture: Data and Batch Geoprocessing …web.clas.ufl.edu/users/forrest/gis/lectures/2013-10-22...Week 10 Lecture: Data and Batch Geoprocessing With ArcGIS Introduction to](https://reader031.vdocuments.mx/reader031/viewer/2022030702/5aeda03f7f8b9ab24d91c352/html5/thumbnails/11.jpg)
So Geoprocessing Opens Doors
• Besides access to ArcToolbox tools it gives properties and methods to create and manipulate datasets directly!
• Explore the object diagram to find functionality…
– Cataloguing
– Describing
– Listing
– Editing
![Page 12: Week 10 Lecture: Data and Batch Geoprocessing …web.clas.ufl.edu/users/forrest/gis/lectures/2013-10-22...Week 10 Lecture: Data and Batch Geoprocessing With ArcGIS Introduction to](https://reader031.vdocuments.mx/reader031/viewer/2022030702/5aeda03f7f8b9ab24d91c352/html5/thumbnails/12.jpg)
Class diagram on steroids…
![Page 13: Week 10 Lecture: Data and Batch Geoprocessing …web.clas.ufl.edu/users/forrest/gis/lectures/2013-10-22...Week 10 Lecture: Data and Batch Geoprocessing With ArcGIS Introduction to](https://reader031.vdocuments.mx/reader031/viewer/2022030702/5aeda03f7f8b9ab24d91c352/html5/thumbnails/13.jpg)
The Geoprocessor Acts as Gateway…
• To all of the power of ArcGIS and its tools:
Geoprocessor Programming Model
![Page 14: Week 10 Lecture: Data and Batch Geoprocessing …web.clas.ufl.edu/users/forrest/gis/lectures/2013-10-22...Week 10 Lecture: Data and Batch Geoprocessing With ArcGIS Introduction to](https://reader031.vdocuments.mx/reader031/viewer/2022030702/5aeda03f7f8b9ab24d91c352/html5/thumbnails/14.jpg)
Reading the Programming Model
![Page 15: Week 10 Lecture: Data and Batch Geoprocessing …web.clas.ufl.edu/users/forrest/gis/lectures/2013-10-22...Week 10 Lecture: Data and Batch Geoprocessing With ArcGIS Introduction to](https://reader031.vdocuments.mx/reader031/viewer/2022030702/5aeda03f7f8b9ab24d91c352/html5/thumbnails/15.jpg)
Putting it all together…
• Python gives you full functionality to programming environment
– Operating/file system integration
– Fully extensible
– All the logic you can stomach
• Geoprocessing framework gives you all the tools in ArcToolbox plus ability to create and edit data directly
… == “Amazing Possibilities”
![Page 16: Week 10 Lecture: Data and Batch Geoprocessing …web.clas.ufl.edu/users/forrest/gis/lectures/2013-10-22...Week 10 Lecture: Data and Batch Geoprocessing With ArcGIS Introduction to](https://reader031.vdocuments.mx/reader031/viewer/2022030702/5aeda03f7f8b9ab24d91c352/html5/thumbnails/16.jpg)
Cataloguing and Listing Spatial Data
• One common and critical task:
– Batch Processing
• We do this with arcpy’s List methods
– Use for different data types
– List objects for iteration
![Page 17: Week 10 Lecture: Data and Batch Geoprocessing …web.clas.ufl.edu/users/forrest/gis/lectures/2013-10-22...Week 10 Lecture: Data and Batch Geoprocessing With ArcGIS Introduction to](https://reader031.vdocuments.mx/reader031/viewer/2022030702/5aeda03f7f8b9ab24d91c352/html5/thumbnails/17.jpg)
Batch Processing
Initialize Batch
Read Data
Process Data
Output Results
• Such a common programming structure
– Sequence of operations
– Iterated over list of items
– Geoprocessing requires the inputs and outputs be organized
![Page 18: Week 10 Lecture: Data and Batch Geoprocessing …web.clas.ufl.edu/users/forrest/gis/lectures/2013-10-22...Week 10 Lecture: Data and Batch Geoprocessing With ArcGIS Introduction to](https://reader031.vdocuments.mx/reader031/viewer/2022030702/5aeda03f7f8b9ab24d91c352/html5/thumbnails/18.jpg)
Enumerations
• Some methods return enumerations:
– List (object) of values without a known count and of any type
• Objects to be listed can be restricted by data types and name
![Page 19: Week 10 Lecture: Data and Batch Geoprocessing …web.clas.ufl.edu/users/forrest/gis/lectures/2013-10-22...Week 10 Lecture: Data and Batch Geoprocessing With ArcGIS Introduction to](https://reader031.vdocuments.mx/reader031/viewer/2022030702/5aeda03f7f8b9ab24d91c352/html5/thumbnails/19.jpg)
Parameters
• Wild cards: We can restrict the objects and datasets to be inserted into the list by name using an asterisk:
myPolygonList = arcpy.ListFeatureClasses(“C*”)
myPolygonList = arcpy.ListFeatureClasses(“*community*”)
• Type filters: Restrict returns by certain type keywords:
myPolygonList = arcpy.ListFeatureClasses(“C*”,”polygon”)
• Input dataset values to restrict by items that are a part of a certain object:
myPolygonList = arcpy.ListFields(table,“C*”,”Integer”)
![Page 20: Week 10 Lecture: Data and Batch Geoprocessing …web.clas.ufl.edu/users/forrest/gis/lectures/2013-10-22...Week 10 Lecture: Data and Batch Geoprocessing With ArcGIS Introduction to](https://reader031.vdocuments.mx/reader031/viewer/2022030702/5aeda03f7f8b9ab24d91c352/html5/thumbnails/20.jpg)
Type Filters
• List methods by default list objects of all types
• If you specify a keyword, you restrict the list returned to those types
![Page 21: Week 10 Lecture: Data and Batch Geoprocessing …web.clas.ufl.edu/users/forrest/gis/lectures/2013-10-22...Week 10 Lecture: Data and Batch Geoprocessing With ArcGIS Introduction to](https://reader031.vdocuments.mx/reader031/viewer/2022030702/5aeda03f7f8b9ab24d91c352/html5/thumbnails/21.jpg)
Iterating Through Enumerated Features
• In pre-10.0, the “gp.” Geoprocessor Enumerated features are not Python lists
• So we iterate through them using while loops, not as for item in list: loops.
• You test the condition before the loop, then the loop iterates until the test results in null/false
![Page 22: Week 10 Lecture: Data and Batch Geoprocessing …web.clas.ufl.edu/users/forrest/gis/lectures/2013-10-22...Week 10 Lecture: Data and Batch Geoprocessing With ArcGIS Introduction to](https://reader031.vdocuments.mx/reader031/viewer/2022030702/5aeda03f7f8b9ab24d91c352/html5/thumbnails/22.jpg)
Iterating Through Enumerations (cont’d.)
• Two methods and no properties for pre-10.0 enumerations:
– list.reset() : Points to the top of the stack of objects, makes sure the first element is on top
– list.next() : returns the currently selected value and increments the list index
![Page 23: Week 10 Lecture: Data and Batch Geoprocessing …web.clas.ufl.edu/users/forrest/gis/lectures/2013-10-22...Week 10 Lecture: Data and Batch Geoprocessing With ArcGIS Introduction to](https://reader031.vdocuments.mx/reader031/viewer/2022030702/5aeda03f7f8b9ab24d91c352/html5/thumbnails/23.jpg)
Pre-10.0 Example - Iterating Through Enumerations (cont’d.)
![Page 24: Week 10 Lecture: Data and Batch Geoprocessing …web.clas.ufl.edu/users/forrest/gis/lectures/2013-10-22...Week 10 Lecture: Data and Batch Geoprocessing With ArcGIS Introduction to](https://reader031.vdocuments.mx/reader031/viewer/2022030702/5aeda03f7f8b9ab24d91c352/html5/thumbnails/24.jpg)
Post 10.0 Example - Iterating Through Enumerations (cont’d.)
![Page 25: Week 10 Lecture: Data and Batch Geoprocessing …web.clas.ufl.edu/users/forrest/gis/lectures/2013-10-22...Week 10 Lecture: Data and Batch Geoprocessing With ArcGIS Introduction to](https://reader031.vdocuments.mx/reader031/viewer/2022030702/5aeda03f7f8b9ab24d91c352/html5/thumbnails/25.jpg)
Listing Tools and Their Usage
Pyt
ho
n
![Page 26: Week 10 Lecture: Data and Batch Geoprocessing …web.clas.ufl.edu/users/forrest/gis/lectures/2013-10-22...Week 10 Lecture: Data and Batch Geoprocessing With ArcGIS Introduction to](https://reader031.vdocuments.mx/reader031/viewer/2022030702/5aeda03f7f8b9ab24d91c352/html5/thumbnails/26.jpg)
Your first batch “Union”
![Page 27: Week 10 Lecture: Data and Batch Geoprocessing …web.clas.ufl.edu/users/forrest/gis/lectures/2013-10-22...Week 10 Lecture: Data and Batch Geoprocessing With ArcGIS Introduction to](https://reader031.vdocuments.mx/reader031/viewer/2022030702/5aeda03f7f8b9ab24d91c352/html5/thumbnails/27.jpg)
Interactive Python in ArcGIS
![Page 28: Week 10 Lecture: Data and Batch Geoprocessing …web.clas.ufl.edu/users/forrest/gis/lectures/2013-10-22...Week 10 Lecture: Data and Batch Geoprocessing With ArcGIS Introduction to](https://reader031.vdocuments.mx/reader031/viewer/2022030702/5aeda03f7f8b9ab24d91c352/html5/thumbnails/28.jpg)
![Page 29: Week 10 Lecture: Data and Batch Geoprocessing …web.clas.ufl.edu/users/forrest/gis/lectures/2013-10-22...Week 10 Lecture: Data and Batch Geoprocessing With ArcGIS Introduction to](https://reader031.vdocuments.mx/reader031/viewer/2022030702/5aeda03f7f8b9ab24d91c352/html5/thumbnails/29.jpg)
Wrap-up
• The arcpy geoprocessing framework offers the gateway to ArcGIS’ tools
• It also provides functionality for sophisticated interactions with data
– Batching is the most simple way to interact
• Python’s native utility (e.g. string manipulation) and extensible modules (charts, statistics) can be married with ArcGIS
![Page 30: Week 10 Lecture: Data and Batch Geoprocessing …web.clas.ufl.edu/users/forrest/gis/lectures/2013-10-22...Week 10 Lecture: Data and Batch Geoprocessing With ArcGIS Introduction to](https://reader031.vdocuments.mx/reader031/viewer/2022030702/5aeda03f7f8b9ab24d91c352/html5/thumbnails/30.jpg)
In Lab This Week
• Explore Geoprocessing framework
– Build simple models/tools
– Export to scripts
– Create batch processes
import arcpy
help(arcpy)
• Supplemental reading – http://resources.arcgis.com/en/help/main/10.1/index.html#/What_is_geoprocessing/002s0000000100000
0/
– http://resources.arcgis.com/en/help/main/10.1/index.html#/Geoprocessing_tools/002s00000004000000/
– http://resources.arcgis.com/en/help/main/10.1/index.html#/What_is_ArcPy/000v000000v7000000/