copyright © 2002, 2003, 2004 esri. all rights reserved. what's new in arcgis 9 introduction to...

Copyright © 2002, 2003, 2004 ESRI. All rights reserved. What's New in ArcGIS 9

Introduction to Geoprocessing in

ArcGIS 9

Jason GrootensESRI-Minneapolis

1-2What's New in ArcGIS 9Copyright © 2002, 2003, 2004 ESRI. All rights reserved.

Workshop objectives

Introduce Geoprocessing at ArcGIS 9: New Framework Dialogs (Tools)

Modelbuilder

Command Line

Scripting

Ask questions and participate in discussions


What is ArcGIS?

ArcGIS clients

Application/Data servers

ArcGIS DesktopEXTENSIONS

ArcGIS Engine

Components ArcObjectsArcObjects

ArcReaderArcReader ArcViewArcView

ArcInfoArcInfo

ArcEditorArcEditor

Customapplication

Customapplication

RDBMS

ArcSDEArcSDE

ArcGIS ServerArcGIS Server ArcIMSArcIMS

EXTENSIONS

ArcPadArcPad

Network

Webbrowser

Webbrowser


What is geoprocessing?

Perform a variety of geographic based tasks

Spatial Analysis

Spatial Analysis

ConversionConversionProjectionsProjections

Data Management

Data Management

Database

CAD GDB

Proximity?Proximity?


Geoprocessing answers spatial questions

What is the

fastest route over

rugged terrain?

How many people are within a contamination zone?

What is the closest firehydrant to each building?

Where is the best location for a store?

Census Block GroupsDistance to RoadsLand CostDistance to competitors


Geoprocessing in ArcGIS

ArcGIS 9 enhances geoprocessing

Hundreds of tools available

Wide variety of methods for accessing geoprocessing tools

Geoprocessing is what makes a GIS


Geoprocessing at ArcGIS 9

Geoprocessing available with ArcGIS 9

Command LineCommand Line

ModelsModels

ScriptsScripts


Accessing the ArcToolbox window

A dockable window with toolboxes Obtained in ArcMap, ArcCatalog, ArcGlobe, or ArcScene

Tree view in ArcCatalog

No longer a separate application


Toolsets and tools

Toolbox: Container for tools and toolsets

Toolset: Logical container of tools and other toolsets

Tool: Single geoprocessing operation System tool, model, script, custom built tool

All tools look and act the sameToolbox

Toolset

Tool


Executing tools

Double-click any tool tospecify parameters

Messages appear in progress dialogs andCommand Line window


Overview of toolboxesCore toolbox Analysis Geoprocessing for feature classes

Conversion CAD tools, conversion tools

Data Management Field and workspace tools

Coverage – *Workstation install Geoprocessing for coverages

Linear Referencing Creating routes, locations

Geocoding Building locators, table matching

Cartography Masking annotation, features

Spatial Statistics Analyzing patterns, distributions

Core toolbox Analysis Geoprocessing for feature classes

Conversion CAD tools, conversion tools

Data Management Field and workspace tools

Coverage – *Workstation install Geoprocessing for coverages

Linear Referencing Creating routes, locations

Geocoding Building locators, table matching

Cartography Masking annotation, features

Spatial Statistics Analyzing patterns, distributions

Extensions

3D Analyst

TIN tools

Geostatistical Analyst

Convert to GRID tool

Spatial Analyst All tools currently in Spatial Analyst

Extensions

3D Analyst

TIN tools

Geostatistical Analyst

Convert to GRID tool

Spatial Analyst All tools currently in Spatial Analyst

*Ability to create yourown toolboxes


Tools and licensing

Ability to use ArcToolbox, command line, models, scripts included at all license levels

The amount of tools vary depending on license/extensions ArcView = 102 tools

ArcEditor = 104 tools

ArcInfo = 251 tools

Spatial Analyst extension = 158 tools

3D Analyst extension = 45 tools

Geostatistical Analyst = 1 tool

Total of 455 tools with ArcInfo and all extensions


Toolbox tabs

Favorites Most frequently used

tools

Index Alphabetical listing of

all tools

Search Search tools by name

or keyword


Supported data types

At ArcGIS 8.x, ArcToolbox worked primarily with coverages

At ArcGIS 9, ArcToolbox works with all data types used in ArcGIS

CAD

Coverage GeodatabaseShapefile

Raster

Tables

Geoprocessing

Layer files

Behavior


Accessing geoprocessing functionality

Command Line

Simple executionShortcut to tools

Command Line

Simple executionShortcut to tools

Scripts

Batch processingBranching

Scripts

Batch processingBranching

ArcToolbox

Execute as dialogsSimple operations

ArcToolbox

Execute as dialogsSimple operations

ArcObjects

Use tools with ArcObjects

ArcObjects

Use tools with ArcObjects

Model Builder

Graphical DocumentationChain tools into one process

Model Builder

Graphical DocumentationChain tools into one process


Creating your own toolbox

Used to store models, scripts, custom tools, or a shortcut to frequently used tools

My Toolboxes

Created in ArcToolbox or Catalog Tree

Stored as .tbx in user profile

Folder

Created in Catalog Tree

Stored as .tbx in folder

Geodatabase (personal orenterprise)

Created in Catalog Tree

Stored as table in GDBStored as .TBX file

Stored in geodatabase


Creating your own tools

Three ways of creating a tool ModelBuilder

Script

Custom-built tool with ArcObjects

All tools act and behave the same Execute as a dialog

Accessing documentation


Using and creating models

Drag and drop environment

Allows processing of multiple tools at once

Documentation of process

Easy-to-change parameters and re-execute model

Project data Tool Derived data


Using and creating tools with scripts

Use any language with a COM interpreter Python, VBScript, JScript, Perl

Geoprocessor also accessible in programming languages

VBA, VB, C# , etc

Batch processing

Branching basedupon conditions

Delayed processing

Can run outsideof ArcGIS

#For each shapefile in the list

fc = fcs.next()

while fc:

outname = outworkspace + "\\" + fc

#Clip each feature class in the list

#Give output fc the name of the input gp.Clip_analysis(fc, clipfc, outname)

fc = fcs.next()

#For each shapefile in the list

fc = fcs.next()

while fc:

outname = outworkspace + "\\" + fc

#Clip each feature class in the list

#Give output fc the name of the input gp.Clip_analysis(fc, clipfc, outname)

fc = fcs.next()


Using tools at the command line

Run a tool from the interface without a dialog

Shortcut for advanced users

Similar to ArcInfo Workstation

Clip_analysis <in_features> <clip_features> <out_featureclass>

Clip_analysis county_censusblocks citylimit city_censusblocks


Using and creating tools with ArcObjects

All geoprocessing tools can be called as COM components

Create custom tools (.dll)

Geoprocessing Object Model Diagram


Environment Settings

Common parameters that can be applied to many tools within a geoprocessing session

Settings persist through all tools (models, scripts, custom tools)

Geodatabase settings

X/Y Domain

Precision

Geodatabase settings

X/Y Domain

Precision

Geodatabase raster

Statistics

Compression

Geodatabase raster

Statistics

Compression

General settings

Current workspace

Output spatial reference

Extent

General settings

Current workspace

Output spatial reference

Extent

Raster settings

Cell size

Mask

Raster settings

Cell size

Mask


Saving environment settings

Where environment settings get saved Normal.gxt for ArcCatalog

A map document (.mxd) or template (.mxt)

Save settings into an XML file to load into other sessions Favorites tab

Environment settings

Command line variables

Save settings to default


Using ArcGIS 9 ModelBuilder


Modeling spatial problems

Models help understand and solve complex problems Models simplify reality

Models combine geographic layers to answer questions

Where should we build our next store?

Reality GIS layers Suitability for store

Model criteria:- Zoned commercial- Near target population- Away from competition

Model criteria:- Zoned commercial- Near target population- Away from competition


Types of models

Repetitive tasks Minimize grunt work

Efficiently execute a series of frequently used tools

Suitability models Use to find best location

(businesses, vineyards, evacuation centers)

Process models Show the landscape as conditions change

(fire spreads, rivers flood, oil slicks move)

Best store location

Filling a reservoir

GIS data layers


Why use ModelBuilder?

Fast analysis; efficient processing

Re-execute the same model, slightly changing parameters to see how end results differ

Complex models

Graphical documentation of work

Project data Tool Derived data


Using ModelBuilder

Obtained from a toolbox

Create new models in your own toolboxes

All tools within ArcToolbox can be used in ModelBuilder

Existing model


Model elements

Project data: Data that exists before model is run

Blue oval

Tool: Operation performed on input data

Yellow-orange rectangle

Derived Data: Output data created by a function

Green oval

Process: Set of elements

Run one process at a time

or all at once

Process

Process

Process

Model

Project

data

Project

dataTool

Tool

Tool

Derived data

Derived data

Derived data


Adding model elements

Drag and drop from ArcCatalog or ArcMap Tools from ArcToolbox

Data

Add empty variables Supply data source at a later time


Tools within a model

Right-click or double-click to obtain parameters Same dialog as tools from a toolbox


Three states of model elements

Not ready to run: Parameters not defined

Ready to run: All elements colored

Already run: All elements colored and shadowed

Not Ready to Run

Ready to Run

Already Run


How to execute models

Entire model from ModelBuilder Can execute one process at a time from ModelBuilder

As a tool dialog


Parameters

Input/Output data and values for a tool

Used for running model as dialog

Right-click model element and choose to create parameter

Data and location

Buffer distance Cell size

Output name and locationInput data

Project Data Function Derived DataProject data Tool Derived data


Running models

All parameters created in model appear in model dialog


Validating a model

Validating a model Returns model elements to

ready-to-run or not ready-to-run states

Validate to determine if model is invalid

Invalid models Parameter values no longer valid (referencing non-existent data)

Tools referenced no longer exist

COM tool inside the model is unregistered


Intermediate data

You determine which derived data elements are intermediate

Can delete after model has been executed Save disk space

Roads

Land use

Buffer

Select

Roads buffer

Suitablesites

EraseBest sites

Intermediate data


Navigating the model

Auto Layout Navigate tool Add Connection


Model diagram properties

Use grids for better control in placing elements

Change flow orientation

Change the color, shape, and font


Adding labels

Give the model canvas more detail

Diagram labels Free floating labels

Element labels Maintain position relative to model element

Connector labels (tool or data) Maintain position relative

to connector


Exporting models

Export to a graphic Export as .bmp, .jpg, .emf

Useful for placing in map layouts

Printing models Add borders, captions, page

numbers


Tool documentation

Ability to edit documentation for tools and toolboxes Documentation built into

ArcGIS online help system

Ability to create metadata Useful for Metadata server


Model Report

XML document available for viewing or creation

Useful for historical viewof processes executed

Holds information on Variables

Processes

Messages

Time and date of execution


Saving and sharing models

Model is saved to .tbx file or geodatabase Give others .tbx or geodatabase to share model

Set parameters if used with different data

Export model as a script

WindowsExplorer


Example Model

Find suitable area for airport based upon available land and distance from existing airports


ModelBuilder

Demo


Using the Command Line window


Command Line window

Available in all ArcGIS applications

Holds messaging for all tools executed

Similar interface to ArcInfo Workstation Commands and arguments may be different

ArcInfo Workstation 7.x

ArcGIS 9


Why use command line?

Saves time if familiar with geoprocessing tools Allows you to obtain quick view of all tools available

Execute scripts and models

Shortcut to tools in ArcToolbox


What can be accomplished at command line?

Executing tools

select soils48 C:\undrained.shp "OTHERPH" = 'UNDRAINED'

Environment settings

CellSize 50

Workspace C:\Arc9output\OrangeCounty.mdb

Create variables


Syntax notation

Similar to ArcInfo Workstation and Spatial Analyst

ToolName <Argument> {Argument} {KEYWORD|KEYWORD}

Required Optional Optional Keywords

First optional keyword will be taken as default

Use # to skip arguments

Allows multi-line commands Shift + Enter


Passing in parameters

Usage appears showing all parameters

Dropdown list shows available layers in ArcMap TOC

Type in name of command and press Enter Usage appears in Command Line window


Messages

Successful completion of a tool Includes Start and End Time

Errors (red font) Indicates a tool did not execute

Warnings (green font) Indicates that a step may have been skipped; gives usage


Editing and re-executing existing processes

Allows to edit parameters or re-execute without changing If source of layers changes

Change a parameter (buffer distance)

Edits are made (new features added)

Selection changes

*Can also use up and down arrows on keyboard for previous commands


Adding a toolbox to ArcToolbox

Only tools in ArcToolbox recognized in command line

In ArcToolbox Browse for toolboxes

In ArcCatalog Add any toolbox by right-clicking

Copy and paste into ArcToolbox


Variable manager

Make variables for parameters Data and workspaces

Expressions

Reclassification

Spatial reference


Accessing data and tools

Some tools can have the same name ToolName_ToolboxAlias

Buffer_analysis infeatures outfeatures

or

Buffer_arc infeatures outfeatures

Core Toolbox

Analysis = Analysis

Conversion = Conversion

Data Management = Management

Coverage = Arc

Geocoding = Geocoding

Core Toolbox

Analysis = Analysis

Conversion = Conversion

Data Management = Management

Coverage = Arc

Geocoding = Geocoding

Extensions

3D Analyst = 3D

Spatial Analyst = SA

Extensions

3D Analyst = 3D

Spatial Analyst = SA


Adding a toolbox alias

Add aliases to your toolboxes


Executing a model from command line

Toolbox containing model must be in ArcToolbox

Execute model by calling model name, not the label


Command Line

Demo


Scripting and the Geoprocessor


Why write scripts for geoprocessing?

Similar advantages that models have

Efficiently execute series of different tasks

Easy to read and document

Easy to share

Perform batch operations

Use logic to control tool execution

Interface with other systems

Self contained (single file)

Run any time

Familiar environment for AML and Avenue users

Users don’t have to learn a proprietary language


GpDispatch

An ArcObject accessible in all COM languages/scripts

Also referred to as the Geoprocessor

Exposes all geoprocessing functionality Object is late-bound (no code completion)

VBScriptVBScript

JScriptJScript

PythonPythonCommunication

through IGPDispatch

ProgrammingLanguages

ProgrammingLanguages

Geoprocessor

Access to all geoprocessing tools

Buffer

Clip

AddField


Where to write scripts?

Choice of scripting languages: Any language with a COM interpreter

VBScript

Based on VB: Simpler

Popular Web language

VBScript

Based on VB: Simpler

Popular Web language

JScript

Syntax similar to C/Java

Works in many Web browsers

JScript

Syntax similar to C/Java


Python

Flexible, powerful

Easy to learn

Python

Flexible, powerful

Easy to learn

Perl and other scripting languagesPerl and other scripting languages


Why use Python?

Flexible, easy scripting language

Object-oriented

Ability to compile scripts

Offers a debugging environment

Modular: Can be broken apart

Cross platform


It is FREE

ESRI samples provided

Pythonscript

Pythonscript


Installing Python for geoprocessing (Windows)

Included with ArcGIS 9

Also a free download

Python Foundation (www.python.org)

Includes command line, code editor, and documentation


Writing scripts

Import COM client support

import win32com.client

Instantiate the Geoprocessor objectgp = win32com.client.Dispatch("esriGeoprocessing.GPDispatch.1")

Set properties (e.g., where processing will occur?)

gp.workspace = "c:\\Florida.mdb"

Comment code

# Buffer roads by 100 meters

Run tools

gp.Buffer_analysis("roads", "rdbuf100", "100")


Constructing Python statements

Use the Geoprocessor with object-oriented programming

Assign a value to an environment setting with Geoprocessor.EnvironmentSetting = Value

gp.Workspace = "c:\\temp"

Call tools with Geoprocessor.ToolName (parameter1, .., parameterN)

gp.CreateFolder("c:\\Florida", "Scripts")

gp.Clean("Europe", "#", "#", "#", "POLY")


Referencing data in Python

Use \\ or / for path names Python is based upon the C programming language

Defined by double (") or single (') quotes

gp.workspace = "c:\\Florida"

gp.CreateFolder('c:\\Florida', 'Scripts')

Concatenate strings

outputfc = gp.workspace + "\\" + inputfc


Referencing tools and toolboxes

Only system toolboxes available by default from Geoprocessor Analysis, coverage, data management, etc.

Use toolbox alias if tools have the same name

gp.Buffer_analysis(roads, Out_Buff, "10 Meters", "FULL")

Add your own toolbox

gp.AddToolbox("c:\\MyProject\\CrossCountryMobility.tbx")

gp.BestPath("start.shp", "destination.shp", "results.shp")


Example script

# Import COM support


# Create the Geoprocessor object

gp = win32com.client.Dispatch("esriGeoprocessing.GpDispatch.1")

roads = "C:/Data/Florida/ROADS.shp"

Out_Buff = "C:/Data/Florida.mdb/ROADS_Buffer"

citylimit = "C:/Data/Florida/citylimit.shp"

Out_Clip = "C:/Data/Florida.mdb/BUFF_Clip"

# Process: Buffer...

gp.Buffer_analysis(roads, Out_Buff, "10", "FULL", "ROUND")

# Process: Clip...

gp.Clip_analysis(Out_Buff, citylimit, Out_Clip)


Running scripts in different environments

Check syntax and run in PythonWin ArcGIS applications can be closed

Run at DOS prompt

Python C:\PythonScripts\RoadBuffer.py

Create a tool Add a script to a toolbox

Open command runs script tool

Edit command opens scriptsource in text editor


Distributing scripts

As Python source file .py

As script tools in toolboxes User creates script tool for parameters

Share the .py file and toolbox


Batch processing and advanced scripting


Batch processing

Scripts are ideally suited for batch processing

Example: Clip all shapefiles in a folder to a boundary

Rerun script when new data is added to folder

Batch processing script

Batch processing script


Listing data Enumeration: Lists of objects without a known count

Use a looping structure to process one object at a time ListFeatureClasses (Wildcard (optional), Type (optional))

#return a list of shapefiles in a workspace

gp.workspace = "C:\\Yellowstone"

fcs = gp.ListFeatureClasses("*", "all")

Examples of Enumerations ListFeatureClasses

ListFields

ListTables

List Indexes

ListRasters

ListWorkspaces

ListDatasets

fcs = all shapefilesin C:\Yellowstone


The Geoprocessor Programming Model

Geoprocessor diagram lists full functionality


Looping While a condition is true

gp.workspace = "C:\\Yellowstone" fcs = gp.ListFeatureClasses ("*", "all")

fc = fcs.next()

clipfc = "C:\\Yellowstone\\studyarea.shp"

outputws = "C:\\Yellowstone\\model"

while fc != "":

gp.clip_analysis(fc, clipfc, outputws + "\\" + fc)

fc = fcs.next()

Loop is defined by indentation in Python

Indentation is a language construct in Python Needs to be consistent

Use <Tab> or spaces

Checked by TabNanny utility in PythonWin


Batch processing example


gp = win32com.client.Dispatch("esriGeoprocessing.gpDispatch.1")

gp.workspace = "C:\\Database\\Wyoming\\AlbanyCounty"

#Get a list of rasters in the workspace

rasters = gp.listrasters("*", "all")

#Store path to the output folder

outputPath = "C:\\Database\\Wyoming\\AlbanyCounty\\PilotPeakStudyArea"

#For each raster in the list

raster = rasters.next()

while raster != "":

#Set the outputname to be the same as the input

outName = outputPath + "\\" + raster

#For each raster, clip it to a specific extent

gp.clip_management(raster, "387211 231491 410734 250303", outName)

raster = rasters.next()


Built-in geoprocessor methods

Check whether an object (feature class, GDB) exists

gp.Exists(myGDB)

Refresh Catalog to notify script of new data created in script

gp.RefreshCatalog("c:\wksp\geodatabase.mdb")

File Management tools existing in toolboxes

gp.Delete(myGDB)

gp.Copy(myGDB, "C:\temp\newGDB")


Describe

Method that returns an object with a set of properties

Properties vary depending on what is being described

Describe (InputValue):Object


Making decisions

If statement

dsc = gp.describe(fc)

inputtype = dsc.ShapeType

if inputtype == "Point":

gp.Buffer_analysis(fc, "hospital_buff", "500.00 Meters")

else: #inputtype is polygon or line

gp.Clip_analysis(fc, "cntybnd", "hospital_buff_clp")

Break statement to jump out of closest enclosing loop

while field:

if field.Name == "Flag":

gp.CalculateField(fc, "Flag", "1")

break

field = fields.next()


Why use parameters with script tools?

Makes scripts more flexible

Dynamically assigns script arguments

Workspace

Feature classes

String, numbers

Write scripts to capture input arguments

import win32com.client, sys

gp = win32com.client.Dispatch("esriGeoprocessing.gpDispatch.1")

gp.workspace = sys.argv[1] # first argument

buffc = sys.argv[2] # second argument

buffoutput = sys.argv[3]

buffdistance = sys.argv[4]


Parameter properties

Display nameDisplay name

Data type (workspace)Data type (workspace)

Required/OptionalRequired/Optional Input/

OutputInput/Output


Running scripts that have parameters

Use form to browse for data types you specify Form automatically built based

upon parameters you specify

Name of parameter you set whencreating the script

Supply arguments at command line or DOS prompt


Advanced modeling


Using advanced functionality

Models can carry out actions beyond what is within the toolbox Run a previous model as a tool

Run a script as a tool

Use an .exe or external function as part of a model

Model

Script

Executable


Running a previous model in another model

Drag and drop model from toolbox like any other tool

If output from a previous model is a parameter, it will be exposed in a larger model

P

P

P


Relative paths

Property of a model or a script

Keeps track of all tools and data elements


History

C:\Documents and Settings\<USER>\Application Data\ESRI\ArcToolbox\My Toolboxes

Each history model contains tools for each process executed in each ArcMap/ArcCatalog session

Honor generating reports


Using a script

Drag and drop a script from a toolbox Create parameters in script to add connections between tool

Attach an .exe or AML to a script


Exporting to a script

Easy way to start writing scripts


Run a script at a specific time

Windows AT command

AT 9:00 C:\yellowstone\yellowstone.py

Windows scheduler Control Panel > Scheduled Tasks


Precondition property

Used to control flow of model execution

Used to branch within a model

Precondition

Preconditions


Branching in a model

Precondition property from a scriptIf the input

is a line or point

If the inputis a polygon

Script Output willbe polygons

Output willbe points


Steps for branching within a model

Create a script that will set parameters as output-based upon if/then conditions

Add the script to a toolbox, specifying output parameters that are in the script

Apply precondition properties to tools to determine model flow


Writing a script to branch in a model

Specify parameters to control which tools are executed in a model

inputfc = gp.GetParameterAsText(0) #SAME AS sys.argv[1]

dsc = gp.describe(inputfc)

if dsc.shapetype == "polygon":

gp.SetParameterAsText(1, "True")

gp.SetParameterAsText(2, "False")

else:

gp.SetParameterAsText(1, "False")

gp.SetParameterAsText(2, "True")

A


Adding output text parameters

Direction = OutputDirection = Output

Data type = BooleanData type = Boolean

Type = DerivedType = Derived

A


Applying a precondition property

Set for tools accepting parameters from a script

1. Apply Precondition to next tool

2. Select condition from script output


Scripting with Python

Demo


Resources for learning Python

Books

Learn to Program Using Python

Learning Python

The Quick Python Book

Python, Essential Reference

Web sites

The Python Foundation (www.python.org): Tutorials, documentation

ESRI Instructor-led course

Introduction to Geoprocessing Scripts Using Python

Writing Geoprocessing Scripts with ArcGIS .pdf

Online help


Learning paths

Learning Guide Learning paths organized

by software and topic

Learning options Instructor-led courses

Virtual Campus courses

Training seminars

Web workshops

http://www.esri.com/training_events.html


ESRI Support Center a gateway to resources

http://support.esri.com

Knowledge Base Technical articles

White papers

System requirements

Downloads Patches and service packs

Data models

ArcScripts

User Forums Discussion groups

E-mail lists

Software support resources

For Developers Developer resources

Request ESRI Technical Support Available to support subscribers