Using Python with ArcGIS Jason Pardy, Ghislain Prince, Nawajish Noman

Agenda

• Essentials - Why use Python scripting? - What is ArcPy?

• Automation

- Executing tools - Messages & Error handling - Batch processing - ArcPy Classes - ArcPy Functions - Accessing Data

• Creating Script tools • Spatial Analyst Module

Learning Python Scripting with ArcGIS

• Python Resource Center - http://resourcesbeta.esri.com/en/communities/python/

• Desktop Help • Have a good Python Reference

- "Learning Python" by Mark Lutz

- published by O’Reilly & Associates

- "Core Python" by Wesley J. Chun - published by Prentice-Hall

Esri Training for Python http://www.esri.com/training

• Instructor-Led Course • Web Course

• http://training.esri.com/gateway/index.cfm?fa=search

.results&searchterm=Python

Python IDEs

- Review of IDEs: - http://blogs.esri.com/Dev/blogs/geoprocessing/archive/2010

/09/14/Review-of-IDEs-for-Python.aspx

Why Python?

• Fulfills the needs of our user community

- Simple and easy to learn - Scalable - Modular - Object oriented - Easy to maintain - Cross platform - Established and active user community

A brief history of Python in ArcGIS

• dispatch–based Geoprocessor • Python 2.1

• arcgisscripting module • Cross-platform • Python 2.4

• arcgisscripting module, 9.3 version • "Pythonic" • Python 2.5

• ArcPy site-package • Mapping support • Map Algebra support • Python window • Python 2.6

• data access module • Python toolboxes • Python add-ins • improved na module • Python 2.7 • more mapping functionality

What is ArcPy?

• ArcPy is a native Python site-package • Increases productivity with a richer and more native

Python Experience • Includes code completion and intellisense • Includes modules covering other areas of ArcGIS:

- Data access - Mapping - Extensions – Spatial Analyst (map algebra) - Network analyst - Time

• Includes classes and functions making it easier to create objects such as spatial references, geometries, etc.

Geoprocessing Tools

• Tools are the fundamental unit of geoprocessing • There are hundreds of tools at your disposal

- You can create your own tools (ModelBuilder, Python, etc.)

• Any tool, once created, can be called in Python by using the arcpy.ImportToolbox function

- Creates tool wrappers for your toolbox

Tool Messages

• Executing a tool will produce 3 types of messages. - Informative messages (severity = 0) - Warning messages (severity = 1) - Error messages (severity = 2)

# start try block try: arcpy.analysis.Buffer("c:/ws/roads.shp", "c:/outws/roads10.shp", 100) # If an error occurs when running a tool, print the tool messages except arcpy.ExecuteError: print arcpy.GetMessages(2) # Any other error except Exception as e: print e.message

* A note on tool organization

• Tools can be accessed directly from arcpy

• Or from arcpy ‘toolbox’ modules

• Matter of preference – functionally no difference

import arcpy arcpy.GetCount_management(fc)

import arcpy arcpy.management.GetCount(fc)

Environments

• Script writers set the environment and tools use them - General settings

- Current Workspace, Output Spatial Reference, Extent - Raster analysis settings

- Cell Size, Mask - Many more

arcpy.env.workspace arcpy.env.outputCoordinateSystem arcpy.env.extent arcpy.env.cellSize

Python window

• An embedded Interactive Python window within ArcGIS

- Can access ArcPy, including tools and environments - Can access any other Python functionality

Setting Environments Tool messages

Exception handling

Demo

Functions

• The ArcPy module contains functions necessary to perform many scripting tasks

- Listing data - Describing data - GDB management & administration - Validating table and field names - Analyzing data - Updating data - Map production - etc.

• Allows automation of manual tasks

Batch processing

• Geoprocessing tasks/jobs are often repeated on a set of data

- Converting from one format to another (CAD to GDB) - Clipping a set of feature classes with a study area - Spill Modeling/Land use studies, etc.

• Several list methods exist

to support these cases:

Describing Data

• Allows script to determine properties of data - Data type (shapefile, coverage, network dataset, etc) - Shape type (point, polygon, line, etc) - Spatial reference - Extent of features - List of fields

• Returns an object with dynamic properties • Logic can be added to a script to branch based on data

properties

• Batch processing Demo

Classes

• Classes can be used to define parameters - easily defines more involved parameters (i.e. spatial

reference or field mapping)

# Create a spatial reference using a projection file sr = arcpy.SpatialReference("Hawaii Albers Equal Area Conic") # Run CreateFeatureclass using the spatial reference arcpy.management.CreateFeatureClass(workspace, fc_name, "POLYLINE", "", "", "", sr) arcpy.AlterAliasName(os.path.join(workspace, fc_name), alias_name)

sr = arcpy.SpatialReference("Hawaii Albers Equal Area Conic") ptgeom = arcpy.PointGeometry(arcpy.Point(-4323932.927, 8399579.114), sr)

Accessing Data

New Data Access module at 10.1

• Improved cursor support (faster performance)

• Control of the edit session, edit operation

• Functions for converting tables and feature classes to and from NumPy arrays

• Support for versioning, replicas, domains, and subtypes workflows

Cursors

• Cursors provide record-by-record access - Are a workhorse for many workflows

• The ‘classic’ cursor model works, but… - Not fast enough - Bottleneck for some Python workflows

SearchCursor Read-only access UpdateCursor Update or delete rows InsertCursor Insert rows

Data Access cursors (New @ 10.1)

• Much faster • Supports with statements (no del needed) • No need to access the full geometry

At 10.1

import arcpy # Print the WELL_ID, WELL_TYPE, and the feature's X,Y. fields = ["WELL_ID", "WELL_TYPE", "SHAPE@XY"] with arcpy.da.SearchCursor("c:/data/base.gdb/well" , fields) as cursor: for row in cursor: print("{0}, {1}, {2}".format(row[0], row[1], row[2]))

Accessing geometry with Cursors

• Full geometry objects can be returned using "shape@" token

• Or, specific geometry properties can be retrieved without accessing entire geometry (faster)

- shape@area, shape@length, shape@xymz, shape@trueCentroid

# Find the total length of all line features import arcpy with arcpy.da.SearchCursor("C:/data/base.gdb/roads", "shape@length") as rows: print sum([row[0] for row in rows])

Reading Feature Geometry

with arcpy.da.SearchCursor(polygonFC, ("oid@", "shape@") )as rows:

for row in rows: print("Feature {0}:".format(row[0])) for partnum, part in enumerate(row[1], 0): print("Part {0}:".format(partnum)) for pt in part: if pt: print("{0} {1}".format(pt.X, pt.Y)) else: print("Interior Ring")

Loop through each row

Loop through each part in a feature Loop through each point in a part

For polygons, watch for interior rings

Writing Feature Geometry

• Insert cursors must be used to create new features

rows = arcpy.da.InsertCursor("D:/data.gdb/wells", ["shape@XY"])) rows.insertRow([(1229133.111, 5440104.685)])

Writing Feature Geometry

• Use Point and Array objects to create feature parts • An Update cursor can be used to replace a row’s

existing geometry • See help for more examples (creating multipart

features, etc.) # Open an insert cursor for the feature class with arcpy.da.InsertCursor("roads", ["shape@"]) as cur: # Create array and point objects ptList = [arcpy.Point(358331, 5273193), arcpy.Point(358337, 5272830)] lineArray = arcpy.Array(ptList) # Create a new Polyline from the array polyline = arcpy.Polyline(lineArray) # Insert the feature cur.insertRow([polyline])

Geometry as tool output

• Can direct output to geometry by setting a Geometry object as the output parameter

arcpy.env.workspace = “c:/data/base.gdb” # Returns a list of geometry objects gList = arcpy.CopyFeatures_management(“rivers”, arcpy.Geometry()) # Print the extent of the first geometry in the list print gList[0].extent

Geometry as input

• Don’t necessarily have to go through the steps of creating a feature class, opening a cursor and adding features

• Can create geometry objects on the fly and use those directly as input to geoprocessing tools

# Create a list of point objects ptList = [arcpy.Point(358331, 5273193), arcpy.Point(358337, 5272830)] # Create an array from the list of points lineArray = arcpy.Array(ptList) # Create a polyline feature from the array polyline = arcpy.Polyline(array) # Copy out the geometry by adding as input to CopyFeatures arcpy.CopyFeatures_management(polyline, "c:/base.gdb/newline")

Cursors

• Need coordinate information in a different coordinate system?

• Features may be projected on-the-fly using the Spatial Reference parameter

# Create a SR object from a projection file SR = arcpy.SpatialReference("Hawaii Albers Equal Area Conic")

# Create search cursor, using spatial reference rows = arcpy.da.SearchCursor("d:/data.gdb/roads","", SR)

• Allows use of edit sessions and operations to manage database transactions.

• “with” statements supported

Editor class

At 10.1 import arcpy with arcpy.da.Editor("c:/data/boston.sde") as cursor: arcpy.management.CalculateField("c:/data/boston.sde/roads", "DISTRICTID", "!ZONEID! ")

Geometry operators

• Geometries support relational operators at 10.0 - contains, crosses, disjoint, equals, overlaps, touches, within

• 10.1 adds topological operators - intersect, convexHull, buffer, boundary, clip, difference,

symetricalDifference, union

• 10.1 adds other operators

- distanceTo, projectAs, positionAlongLine

Cursors Geometry operators

Demo

Creating your own tools

Why we create tools

• Easy to share

• Generic - Can be used with different data and varied scenarios

• Communicates with the application - Layers from the map - Messages - Geoprocessing settings/environments

Why we create tools cont…

• Becomes part of the geoprocessing framework - Run from a tool dialog, ModelBuilder, Python - Can be shared as GP service or package

Geoprocessing tools

• A geoprocessing tool does three types of work:

- Defines its parameters - Validates its parameters - Executes some code that performs the actual work

Creating tools

• At 10.1 we can create tools from Python in 2 ways

1. Python toolboxes

2. Script tools (since 9.0)

Creating script tools

• Parameters defined through a wizard • Validation code that lives in the toolbox • Separate source code

•Script tools Demo

Python Toolboxes (new @ 10.1)

• Geoprocessing toolboxes created entirely in Python

Customizing Desktop for ArGIS with Python Add-Ins

• An add-in is a customization, such as a collection of tools on a toolbar

ØNo dll’s!

ØNo compiling!

ØNo ArcObjects!

ØLess code!

mapping module

• A mapping module is part of the ArcPy site-package

• Manage map documents, layer files, and the data within

- Find a layer with data source X and replace with Y - Update a layer’s symbology across many MXDs - Generate reports that lists document information - Data sources, broken layers, spatial reference, info, etc. - Automate the exporting and printing of map documents - Automate map production/map series

Spatial Analyst module

Spatial Analyst module

• Integrates Map Algebra into Python - Defines geographic analysis as algebraic expressions

- Includes all Spatial Analyst tools - Supports operators in Map Algebra expressions - Helper classes that can be used to support complex

parameter

- Output on the left-side

from arcpy.sa import * demm = Raster("DEM") / 3.28 slpdeg = Slope(demm, "DEGREE") demfs = FocalStatistics("DEM", NbrRectangle(3,3), "MEAN")

Raster class

• Returned output from Spatial Analyst tools - Temporary

- Can be used as inputs to tools and Spatial Analyst Map Algebra expressions

• Can be used to cast a raster dataset or layer inRas = Raster(“c:/data/mydataset1”)

• Supports operators in Map Algebra expressions

• Has properties and methods for analysis - raster.min

- raster.save()

Classes

• Used to specify parameters to tools - Varying number of arguments depending on the selected

parameter choice (neighborhood type) - The number of entries into the parameters can vary

depending on the specific situation (a remap table)

• More flexible to work with

• Specify or query the individual arguments

NumPy Integration

• NumPy is a Python library for array processing - A powerful array object - Sophisticated analysis capabilities

• Raster can be converted to NumPy arrays for analysis

• NumPy array can be converted to a raster - RasterToNumPyArray(), NumPyArrayToRaster()

inras = "ras100" # convert raster to Numnpy array rasArray = arcpy.RasterToNumPyArray(inras) # ARRAY SLICING: get the total sum of every third value # from every third row of the raster sampArray = rasArray[::3,::3] sum = numpy.sum(sampArray) print sum

SA Module Summary

• Import Spatial Analyst module for raster analysis

• Map Algebra is powerful, flexible, easy to use

• User Raster and helper classes in expressions

• Can write complex expressions

• Extend analytical capabilities using NumPy arrays